I think that those who were "exemplary" and then left the congregation, totally befuddle the rest. They just can't grasp what happened. It also makes them wonder and "think".
I wish it did make them think. In fact it is more likely that they only start to blacken the reputation and memory of that one " I always thought there was something not quite right about him" they say afterwards - " But they went out that it might be shown up that not all are of our sort" 1 John 2.19 I have used that verse myself in the past