It's worked well for me... my elder husband has told the other elders that I have issues and doubts but I don't want to talk about it. They've not tried anything. Maybe my husband is my 'firewall'!
I went to the meeting the other day, first time in a couple of months and an old friend came and talked to me. She was my pioneer partner decades earlier who had moved elsewhere but was now back in the congregation for a while and she said she'd not seen me on her visits. I told her I don't go much anymore and she asked me why. I told her I had my reasons but didn't want to talk about it. She was fine with it, she thought it was sad but never said anything more about it. It really doesn't matter who wants to know the 'whys', if you don't want them to know, then don't tell them! Be firm that it is your business and not theirs.
I've been fading for five years now and not had any problems at all. It takes patience and self-control (not spilling the beans!) but it's worth it when you can keep your family and close friends and even be on speaking terms with everyone else.