I think part of the reason dubs assume we are negative/unhappy/bitter is because that's how (many of us?) are when we see them.
I have non-jw enemies too (people who have done me harm, who are evil or have acted wrongly and not attempted to make amends). And when I run into them, they can thank their lucky stars that I'm only negative/unhappy/bitter and not worse toward them.
That does not mean I am negative/unhappy/bitter in the rest of my life. Quite the contrary.
I just hold people responsible for their own acts. If they want to blame me for holding them accountable, well then that's their problem, isn't it?
I will not change my actions to impress or affect them in any way. It does not matter what they think about me. If my words or facial expressions make them feel guilty about their own actions, and they twist that around to a reason to stay in the cult longer, then they're getting what they deserve in life.