Many I have known over the years have stepped aside for various legit reasons and have been re-appointed straight back as an Elder in 1 to 3 years (no wrongdoing). The rule of thumb I always heard was within a year but what is said and what is done are sometimes different. Of course the original issue would have been needed to be addressed, sickness, work issues etc. and would be asked of you. If you took a break longer than 1 year you could or would be used as a MS first before becoming an Elder again. I don't think there is a hidden agenda to humiliate one.
The announcement to the cong. does not give away any reason for your removal and my experience was there was much speculation in the hall as to why I was no longer serving. One of my children was targeted as disqualifying me by several families and was being partially shunned by some but the wife and I straightened that out, without any help from the BOE I might ad.
I don't comment, quit the school, no FS report and barely make meetings any longer so no "privileges" are asked of me but when the trash cans out back are full I still get a call. lol guess some "privileges" you just can't get rid of!