Well obviously they'll notice, so you're not being clear in your thinking in this regard.
As counter-intuitive as it may seem, the absolute best strategy is to simply inform the elders you will no longer be able to carry out your responsibilities and other assignments as a ministerial servant effective immediately. (If you want to be nice, you might give them a couple of weeks, but that's just like ripping the band-aid off slowly).
When they ask why, respond politely, "I have personal reasons. I trust you'll respect my privacy in this regard. Thanks!"
If, or more likely when, they press you on your reasons, say, "Look, I already explained that to you. I'm kind of frustrated you're not respecting me and the explanation I've given you." It's ok to show a little irritation at this point.
If they try and play games and fish with a question like, "Have you committed a sin, Brother?" respond with a terse, "Now you're really being offensive," and walk away shaking your head. Refuse to engage.
That's it. You're done.
No other strategy works as well.
