When we first got married someone told our families that we were not attending but we were.
Move and don't have your publisher cards sent over to the new congregation. Don't share your address with anyone and live your life and be happy.
Don't give the busy body spies another thought. If you don't start going and they don't have your address it won't take long for them to forget all about keeping any eye out for you.
Then the only thing you will have to deal with are your parents. That is the price of fading.