Yes..... because it violates trust and loyalty.
I think the better observation is that an affair is an act of selfishess and often times cowardice.
Sex is an extention of emotions often times. So the problem lies within the emotions of couple in the relationship.
The better question is why the person having an affair isn't fixing the problems in their current relationship (and in effect adding more problems via the affair) or simply acknowledging the end and moving on...... ultimately the person is either selfish, indecisive, and on whole completely irresponsible....... which begs the question why would anyone want to get involved with such a person?
~ but I also live in a 'crime of passion' state so an affair could prove deadly~