I said I wanted to step down for health reasons (that part was true at the time). Two elders didn't visit me to talk it over which the instructions say should happen. I later found out that a member of the service committee who didn't like me drafted up a letter, copied and pasted my signature from a piece of correspondence on file, got the other 2 stooges on the service committee to sign it and it was sent to the Bethel. When I found out about this I asked to see "my letter" but was told that I couldn't.
I was peeved about it for a while but then thought "not my circus not my monkeys" and left them to their in-fighting. I do feel that you become invisible once you are no longer serving because you are out of the insider gossip circle. But hey! You got your own life and can get on with it.
It's easier to step aside and walk away, than to say the wrong thing and get kicked out.