When I first got married, twenty years ago, there was a girl at work who wasn't subtle about her feelings for me. She told me my wife was lucky, that she'd give anything to find a man like me. Sometimes, just like TMS said in the OP, she even moved uncomfortably close. She was undeniably attractive—absolutely! But what made it easy for me to resist was imagining the aftermath. I have a vivid imagination, and I could see every detail of how my life would turn into a living nightmare: the guilt, the tension between my wife and me, the uncomfortable conversations until I revealed what happened, and then my wife crying. That's where my imagination always stopped—my wife crying about something like that. An absolute, unbearable nightmare.
And so, the more I saw that office girl as a threat, the more unpleasant I became with her, until she finally "snapped out of it" and got a boyfriend.