The problem could lie more in the fact that people on both sides of the issue, religious people included, are highly ignorant of Scripture and the words often rendered as "faith."
"Enumah" is the Hebrew word, and "pistis" is the Greek word. While most often rendered as "faith" or "belief," neither word actually means that.
Many religious people equate "belief" or "faith" to be little more than a mental acknowledgment that something is so, such as the existence of God. But the words used in Scripture never refer to a simple credulous mental exercise of that type.
The words in Hebrew and Greek mean "to be/act faithfully" or "to employ trust." They never mean to "just believe" as if mentally accepting the existence of God is all that is implied.
It is even unscriptural to have such a "faith" in God. James 2:18 states: "You believe in the one God, do you? Well and good. But so do the demons, and for them it is a cause to tremble in fear." Mere mental acknowledgment that God is real or exists is never meritorious on its own, the text explains, as demons have met and seen God yet their knowledge of his existence doesn't bring them salvation or change their fate.
Such credulity is never advertised in Scripture as a substitute for learning or study, but you will often find religious people claiming it is.
And faith in Scripture is not a requisite for salvation either. Abraham did not have the Scriptures to follow, nor Isaac or Jacob. Yet God claims to be their God, speaking as if they are alive or "saved." (Matthew 22:31, 32) They never had the Bible for a single day to put faith in or follow.
Unbelievers are also spoken of in Scripture as being excused from their sins on Judgment Day merely because they follow the law of their own conscience at Romans 2:14-16. Others are condemned for worshipping false gods due to ignoring not Scripture but failing to study the truths found in nature that argue against the existence of elemental deities. (Romans 1:19-23) Faith is not the option offered or requisite for these persons. Instead learning about the world around us and being faithful to one's own conscience is the alternative to falling to "the wrath of God" in these texts.
The over simplification of "faith" is often a reason many atheists reject the arguments of many religious people. It isn't a bad idea either to do so becuase the idea that faith somehow replaces knowledge is nowhere found in Scripture and absent from Judaism, Catholicism and Orthodoxy which reject such views.