DeeAnna, you have been as fair as possible to him in attempting to create some balance for him and the gay side of his sexuality.

He deserves no further consideration, as you have found.

I notice he used a typical M.O. to blame it all on you because "you said he could contact gays on the internet."

That's another part I hate about these situations. They REFUSE to admit any accountability on their part when they're the ones sneaking around in the first place.

I wish you peace and continued bravery!

I agree 100% with Kel and Still Wondering. My situation is similar to yours on some levels - not the swinging, but the attempting to please our partners to levels that we are uncomfortable with. THIS is NOT okay. We cannot keep bending just to appease them out of fear of being alone. It took a long time to get it through my thick head that being alone (or at least without them) is way better than to keep giving and giving until I'm depleted. They are masters of manipulation and know exactly what they are doing. He just keeps pushing the envelope hoping you'll fold and go along with all of his a desires, yet he clearly does not respect your boundaries.  Like the others have said, you're raising a baby and you both should be concerned about that, yet he's only concerned with whether or not you'll "let" him have side relationships with men.  That's not healthy because he is disrepecting you and your family by not listening/obeying your boundaries.

Just like you, I've bargained/struggled with whether or not this is fair to them, am I being "mean" by not letting him be who he is, etc. Guess what? That is NOT our burden to bear. We are only responsible for ourselves and the choices we make - therefore, if you're not comfortable with something and you tell him, he can either agree to that or not. Your choice is the boundary you make, and his choice is whether he will agree to that and vice-versa. There comes a point when it's not compromising, but giving everything to someone who isn't respecting you because of pure selfishness. It is also extremely likely that he's going to meet up with men behind your back, especially considering he's already started conversations, etc. w others after you told him that it's not okay. At least in my situation, an inch given was always a mile taken by him. At one point I said he could continue to live in the house as long as he didn't do xxx anymore. The next weekend he did xxx and told me "I thought you just meant I couldn't do it IN the house." Such a load of bs just so he could have his cake and eat it to. Again, they are masters of manipulation, excuses, and also making you feel sorry for them.

As to whether you should stay or leave, I think you should think about whether you're okay living a life never truly knowing the truth or constantly searching/digging for the truth. After 4 years I couldn't handle that anymore. On top of the health risks he exposed me to and had already actually given to me.  It becomes a sick, toxic relationship to not trust your partner to the point where you drive yourself crazy trying to know the truth. Our partners don't change, and wil do what they please with or without your permission, and you may or may not know. That's not love, respect, healthy, or anything else a marriage is supposed to be. If you do choose to leave, it will be a very tough road, but I guarantee it's not as tough or as damaging as sticking out another few years.

As a side note, I fully agree with still wondering on the "who cares" comment in regards to supporting the LGBT community. That community and the healthy people in that community who do not lie/manipulate their spouses have nothing to do with your marriage. I may get a lot of flack for this, but prior to being with my Ex I didn't have a problem with LGBT, live and let live, but afterwards I wrongly misdirected my anger and began hating/fearing "those people," even though I didn't hate my gay friends or the gay people I knew, but I hated the idea of the community. Because in my warped little brain I equated LGBT with my lying, manipualative, cowardly partner, but nothing could be further from the truth. As Still Wondering explained, the political correctness and our partners' actions cause us to think in terms of that community when in reality they're completely separate. Thankfully I don't harbor thpse feelings anymore, and am ashamed I did, but that's what this emotional chaos has caused. Although you went the other way and just questioned whether you truly supported them, it's the same thing - we're relating to incomparable things that have nothing to do with one another.

I hope this helps in some way, even if you can tell that I still have a lot of anger about my situation! Just know that things can get better.