Sorry this is happening to you. The only advice I have is to remind yourself over and over that you have no obligation to discuss anything with them. If they knock on your door, you have no obligation to answer it, even if they can see you inside. You don't have to talk on the phone with them, or respond to any other method of contact.
They make you feel like you have to meet them on their terms, but you don't. I know it seems an obvious thing to say, but sometimes we have to say it out loud.
When my husband and I were leaving (well over 10 years ago,) they tried really hard to maintain contact, one time even opening my door and walking into my house uninvited when my husband wasn't home. Unfortunately for them, I'm not easily intimidated, and stood my ground (in the entry way,) while this brain-dead drone elder cried his crocodile tears and declared his intention to help me in spite of whatever ideas my husband was filling my head with. AS IF!!!
But we just refused to give them any response at all, unless it was on our terms.
Early on, somebody on this forum told me not to be above using their own stick to beat them with. That was the best advice I got, and made me unafraid to point out that it was unscriptural of them to corner me without the presence and support of my "spiritual head." I also realized that they had to get the words to come out of my own mouth, OR they had to have two verifiable accusers/witnesses to any offenses I'd commit, AND those offenses had to be major sins, or else there was absolutely nothing they could do, judicially. Questioning is not a major sin, it is just a concern.
We were also warned not to answer their loyalty type questions if we did decide to converse with them, and so never made that mistake.
Just keep reminding yourselves that you have the right to control the circumstances of them interacting with you.