As a Witness, I was trained not to feel bad for "those that left us". If they didn't want to stay, that was THEIR decision. If they decided to stop the "sweet fellowship" of the brotherhood, we needed to avoid them so that they could miss us, see the error of their way and "come back to Jehovah".
When you were a Jehovah's Witness, did you avoid disfellowshipped ones and mentally stop caring about them?