Longhairgal, I don't think they hate you for leaving. It's more of a learned response out of fear that someone else may see them smile or speak. They have a HUGE fear of man and of being seen by sister self righteous snitch.
It's kinda like living in a theocratic Nazi Germany ... fear rules.