First, Welcome Aphrodite (another Aussie)
Well, I ' ve been in both sides of this picture in my life.
How I see your situation is kinda common if people will be honest. When in a relationship, there can be an attraction to the sad/bad boy (or girl).
Life can be just wonderful with much love but we all have a dark side. The attraction can be so unreasonable but the more we deny, the stronger it becomes. There was something in the bible, proverbs I think, which talks about a stallion on heat. There's no stopping that horse.
So, if your serious about sticking it out with hubby, let this one go. There's no way you can go back to being friends now.
Don't be too hard on yourself as it's natural to be sexually attracted to others, it's just what you do with it that counts.