Why are you even considering these people as your friends?
Before the dinner one of them probably mentioned to a jw they were going for dinner with you. The jw probably said you are bad association because you did this or that.
Enjoy the company of your wife and forget about these people.