Just start working on a reason to quit instead of looking bad.
OTWO you know, I'm sure, that no matter what if an elder resigns there's going to be rumors and innuendo. There's no way to leave gracefully. If you don't do anything wrong the perception that you did will still be there and people who didn't like you will make sure something negative is passed around. That's the way it works in the KH. That's the Truth.