It sounds a little irrational, but then again you were traumatized being in this religion. It's an obvious unresolved issue with you, one that you haven't met head on.
A therapist may help you put things in their right prospective, so that you can move on from this.