I think many of us overestimate just how much of an effect our disappearing off into the sunset will have on the congregation and also how we will be missed. Sure there will be an initial flurry of gossip and questions as to your spiritual fitness, but if you just keep acting like "I'm ok, what about you how are you doing" whenever you see someone they soon stop asking, lose interest and look elsewhere for more juicy gossip.
They actualy give up on people very quickly these days, so hang in there and think of it this way.
Your association with the congregation is a like rock being thrown into a pond - it may make a splash, even create a small ripple, but then it disappears without a trace of ever being there at all.