you can have your religion and even your apathy... and not be harming anyone. but when you take it to the level where you become the judge of others, based on your beliefs, then you begin to hurt other people
i would argue that apathy definitely harms.
but as far as i'm concerned, evil can't be boiled down to a product of one specific thing. i don't even think you could come up with a specific and universal definition of evil; what's considered evil varies from person to person.