being told I don't have to believe or come back but can simply come to show my support for him.
No, you don't have to believe or 'come back', but simply going to 'show support' for him as he gives a talk marketing the JW/WT religion is showing support for his efforts. You do not support those efforts.
If it were me, I would not go, even in my inactive, but not DFd state. But as someone never baptized and not held to any kind of expectation, you can make it clear why you're not going, why you do not support him in this area.
If you cave on this, they'll smell blood and hound you unrelentlessly. If they can get you in the hall this one time, they'll have the hope, false as it may be, that they can get you to come back. Do you really want to give them any excuse to pursue this any further?