I have a similar feeling about my trans (closeted) ex.  I am also no contact with him, and have been since 2019.  I don't want to run into him.  I doubt he would go out of his way to approach me, but just seeing him at a distance is enough to upset me, even though it's been seven years since I left him.  We don't live in the same neighborhood (I moved out), but we do live only a couple of miles apart.  I try to stay out of the area of town where he lives, but I do occasionally shop at the grocery store that is about a half mile from our, now his, house, and where I shopped for over twenty-five years when I lived there.   I always scan the parking lot for his car, and, on the occasions when I have seen it, I turn right around and leave.   A few times I have seen him on the road, driving, including once when I realized his car was just in front of me, and once when he was just behind me.   (He works out in a gym near where I live.)  It's always destabilizing for me.  

It must be far worse for you, knowing your ex would like to force himself on you.