Geez Rebel, I am sorry to read about this, and of course, I know how disabilitating this can be.
I'm going post part of the end of your post, then go with that:
"..I know it's really stupid - it is just a building with lots of people, but it really makes me ill."
Rebel, it seems like you are under a great deal of pressure for starters to attend meetings: damned if you do/damned if you don't. You mentioned something about your husband, perhaps, if you care to, you can elaborate on that just in case this is a key factor to the anxiety.
The thing is NOT the physical building with people inside, but WHAT the building and the people inside REPRESENT . It's what it represents that causes you this anxiety.
Because of what it represents, that's why there are so many of us HERE and no longer JWs. We no longer feel the grief, anxiety, depression and misery of struggling with our conscience now that we are free of the WTBTS.
Of course Rebel, I don't know you personally, but the elders will no doubt encourage you to ATTEND MORE MEETINGS so that you will get over this and to PRAY about it and god knows what else. I can tell you right now, the anxiety won't go away.
I hope there is some means for you to discuss this with someone, someone who is not a Jehovah's Witness. Or if you have a contact that's an ExJW, that may help. If professional assistance is within your reach, I would strongly urge anyone to utilize it. It helps tremendously, and the opinion will be objective.
I hope you will be able to break free.