There have been loads of thoughtful comments in this thread, I've found them very helpful personally, thank you.
This is the issue, in a nutshell. It's like the Mafia: they won't let you just leave!
I agree with willyloman. At our last CO visit, he made a comment during the elders meeting that we should never destroy the Record Cards of inactive publishrs, as they're always part of the congregation. My heart sank, as I was already planning how to to get out.
And I sympathise with you, minimus: I appear to be in an identical situation to you, now fading. Best wishes, and I hope it isn't the prelude to something worse...
Regards,
Ken