One of the greatest blessings that came from breaking free from the 'tower was letting go of guilt. I'd hate to see you making decisions motivated by guilt even after getting out.
Perhaps you could go as a gift to your wife? Because that's what it would be. You shouldn't feel guilty for not going, any more than she should feel guilty for staying. She's making your life more difficult by remaining a JW -- but does she feel guilty about it? No, nor should she.
If you go, you may well find yourself DF'd at some point. I was DF'd for "apostasy" because I celebrated Christmas. The more you're in their face, the more of a target you become. If your wife doesn't like having a "weak" husband, how much less would she like a disfellowshipped, apostate one?
Good luck with whatever you decide. :-)
Dave