Some people just have weaker sex drives than others.
I am an atheist and have no religious reasons not to have sex. But I'm celibate simply because I don't have much of a sex drive at ALL.
So it's not unimaginable to me for people to be celibate when under the threat of divine wrath, shunning and death. Since I'm celibate without those worries.
Maybe you are just at the other end of the bell curve and it's a lot harder for you to abstain than for most other people. And it's only hard for you to imagine because you think everyone else has urges as powerful as your own.