Why are college freshman even taking algebra? For students attending a four-year college, that is remedial work. They should have two years of algebra, one of geometry, and one of trigonometery under their belts before graduating high school. Yes, that means four years of college-prep math. It is shameful that colleges and college students must routinely waste time (and tuition money) doing what should have been done in high school.
Mathematics is part of the curriculum for the same reason that literature, history, economics, laboratory science and social science is required for college-bound students. You need a broad, comprehensive foundation to understand the connections between the branches of knowledge.
The solution to large numbers of students failing algebra is not abandoning rigor in the curriculum. If anything, a general strengthening of academic standards is warranted. Dedicated teachers who refuse to accept the "I can't do it!" alibi prove the point. The late Jaime Escalante (of Stand and Deliver fame) is the best known example, but there others like him. Unfortunately, there are not enough others, nor enough school administrators with the courage to allow those extraordinary teachers to do their best.