There's a third option: admitting that you don't know.
I think that many of the discussions about the existence of god assume a specific god, and not just the possibility of a being that is responsible for the universe but does not care to inform us of its work. And why would it? Do you ever stop to explain yourself to a microscopic dust mite when you are admiring a personal accomplishment? Maybe our universe is just another in a long line of experiments, or maybe the creator does not want to interact with us. Or maybe it doesn't even know we're here. And so on.
But we get a sort of bait-and-switch, where the atheist is asked to account for the probability of the existence of any god, and then confronted with belief in a specific god. And that's not the same thing. And if the specific god being put forth doesn't make sense, it's even less convincing. I don't have to explain how or why a godless universe exists, because I am not even sure that is the case. But the gods that humanity has put forth over our existence? None of them make sense, and I can dismiss those as an option.