It might be also important to realize that many ideological movements don't believe they are "keeping to themselves." They tend to see it as avoiding those who reject them.
Jehovah's Witnesses, for instance, insist that they are not required to avoid or shun anyone who is disfellowshipped, that they do so because the disfellowshipped person took a path that led to a place where they have brought being shunned, rejected, and separated upon themselves. This is why Witnesses insist that they don't shun others or separate themselves from society, when in all reality we know they do.