To a greater or lesser degree, what you've experienced is what many of us have experienced. When my wife and I were fading, our book study conductor phoned to say he'd like to come and have a chat with us.
I said, "You mean a shepherding call?" He said, "Well, yes," I said, "No thanks, I'm actually doing fine. If I want to talk, I'll let you know."
I understand that because of the power elders have over publishers it takes a bit of guts to stand up to them and assert yourself. But they always count on the expectation that publishers will wilt and accept their "theocratic" superiority. In reality they have no more power than the person behind a stall at your local market, or a door-to-door seller who pushes you to buy something you don't want. I bet in that case you say, "No thanks" and keep walking (or shut the door). The fact is elders are humans like you and the spruiker at the market, and you can just say, "No thanks". If they don't accept that, treat them exactly as you would any other persistent tout.
When we finally did quit going, the same elder came and visited my wife at her workplace to see how she was going. He happened to mention the CO was in town that week, which was of course the only reason he was calling. She told him, "No, we're fine, thanks" and wouldn't say any more. He never came back, probably because he could then go and tell the CO that he'd done his best. Your elders there are certainly persistent, but maybe it's because they sense you're vacillating and maybe they think you can still be won over.
Sure, it takes guts first time, but it gets a whole lot easier after that. Just try it.