I have been out of the organization - oops I meant cult - for at least 3 years now. No one comes to my door when they're working my street. They know I have a goatee and that I have joined another church. But a couple of weeks ago I was about a block away from my house, walking home, when an elder and his wife were driving by. They slowed down, honked the horn, waved and smiled. (Maybe they never got the memo.) They seemed to be very happy to see me. I smiled and waved back. (Maybe they count that as a shepherding call now?)
Maybe it can help if you think of the witnesses and elders as coworkers and bosses from a job you quit long ago. Back when you worked for that company you obeyed that boss, and you supported the philosophies/goals of that company. You took pride in your job and recommended that company to other people. But then you quit that job and have been working in an entirely different career for many years. If you now see the old boss walking down the street, you know you have no connection to him or to that company; you don't care if that company is doing well or not because it has no effect on your life now.