All babies are born into an atheistic state and are then subsequently taught beliefs.
There has never been a historical society observed that did not have some kind of religion. It is natural for man to seek guidance from a higher power.
A much more accurate statement might be something like:
All babies are born with an empty mental data base and then are subsequently taught beliefs and conclusions from a variety of sources.