Unfortunately, I've been on your 'side' of this a few times. In my experience, once a cheater, always a cheater. I truly believe that some people have it in them to cheat, and some don't. I've never cheated, and not because the temptation hasn't been there, I just wouldn't let myself do that. People make all kinds of excuses for cheating, but the fact is that you can control who you have sex with. My first wife said it was an 'accident'. I was like, ACCIDENT? Did you trip and fall onto his dick a few hundred times?
Whether you leave or not is up to you. Just because you caught him doesn't mean he has done it before. But once he has, and you've taken him back, the chances that it will happen again are very good. Again, I say this from experience. In both cases that it's happened to me, I took the person back, and it happened again at a later point in time. I've made a firm committment to myself that if I'm ever in that situation again, it's ADIOS mf'er, regardless of whether or not I love that person. Cheating is a selfish, bullshit act - nothing more, nothing less. There is no 'rationale', no reasonable excuse. It happens because people let it happen - they pursue someone else and follow through with it. It shows a lack of love and respect for their mate. All that "I cheated, but I still love you" is nothing but pure BS. You don't do that to someone you truly love. My advice is to begin breaking things off now and save yourself the heartache in the long run. Remember too, that even if you 'get over this', you'll never feel the same, you'll never trust that person, and when you get mad, you'll want to bring it up as ammo.
Just my $0.02