Ugh, I'm sorry wannabefree! I know what it's like to be hounded and chased by elders...but also know that, while not easy, it's possible to just avoid them altogether.
00DAD had some really great input, as well as flipper and others. Don't give them anything...because, in short, you OWE them NOTHING. Your life is YOUR business. Your beliefs (or lack of) are YOUR business.
If they speak to your wife, they still only have ONE witness to your so-called "apostasy". That isn't enough. They need 2. If you haven't spoken to anyone else about your doubts and feelings, then I wouldn't worry about it too much.
Your family needs to grasp and really understand what the result will be if you are disfellowshipped. They themselves might be shunned. They might have to limit their conversations with you, even within the same household. Family outings may come to a halt if your wife decides to be Watchtower Loyal. Sad to say, but some spouses decide that even sexual relations with a DF'ed mate are a no-no (that's not "necessary family business"), and may even file for separation on the grounds of so-called "spiritual endangerment". Tread lightly.
If they are wanting to meet at the hall, you can bet they've filled their tackle boxes and bought extra worms because they're going fishing for something. Don't give them that opportunity. Refuse to meet. You're busy. You have a lot going on...they do NOT have to know specifics!! Don't say "Well, I have this huge project at work, but when that's done, maybe...". Just be vague and say you're busy. Period. If they press, tell them it's personal and you will not be discussing it with them.
If they mention that someone has brought an accusation against you, THIS is where you use the Bible to your advantage! This is something you could say:
"Brother XXXX, according to the Bible, I do not have to speak to you about this matter. Jesus himself said at Matthew 5: 23, 24 “Therefore, if you are offering your gift at the altar and there remember that your brother or sister has something against you, 24 leave your gift there in front of the altar. First go and be reconciled to that person; then come and offer your gift." Apparently, Brother XXXX, someone has breached the protocol that Jesus set out for us when dealing with disputes in the congregation. Therefore, I need to go to this brother or sister FIRST and see what the problem is. Then, if there is no response, I would follow the Bible's counsel to bring another brother or sister with me as a witness...and then if there is still no solving the matter, THEN we would come to you... Or is that not how we are doing things anymore? You see, Brother, several steps have been skipped here, so asking me to meet with you is kind of going against Jesus' words, is it not? I'm sorry this person breached the order of things, but if you'll tell me who it was, we can get it straightened out between the two of us instead of involving the elders...I'm sure you and the Body have much more important matters to tend to."
I actually know someone who got out of meeting with the elders because they used this line of reasoning. They COULD NOT refute it and agreed that the person was correct.