I don't have a particularly strong opinion either way about whether you should confront him.  But, if that's what you decide to do -- or if you decide not to hide it from his family -- I would strongly suggest you avoid being drawn into explaining HOW you know what you know.  It only invites gaslighting.  He'll be challenging you on every word you respond.  A simple "It doesn't matter how I know, the fact is I know."  Otherwise you end up litigating what the meaning of "is" is.

maybe you can get word through to a friend of the woman?  

word of caution - you will come to see that there are a lot of MOMs, a lot of closet lesbians, at least as many as closeted gay men.