it does mess you up. I married too young, I didnt love my husband - I was allowed to court him as he was a MS. i cheated on him, I wanted out but the elders encouraged we work on our marriage. - when that didnt work I cheated again. - when i eventually decided to leave I left everything - my husband and the cong - looking back on life had I not all that fear of disfellowshipping and the elders and I truly believed there was NO WAY out of my marriage unless I fornicated, - i cant believe it didnt actually occur to me to just leave him and keep my dignity like you would in the world - its not working, Im gone.
when I had a few single years not a witness before I met my now husband, lets just say I got that out of my system and Im glad I did. I now know Im not missing out on anything and sometimes worry I would think I was -
teh psychologist said to me the problem I was having was that I had not been allowed to have a youth where you make mistakes and learn from them so I was having my teenage years in my thirties!
also if you are having those thoughts constantly, it may not be a good indication of the state of your relationship in general, not just that you are 'wondering'