minimus:
There is no chosen people.
This is basically the view of all Jews, religious and secular, outside of the Orthodox religion.
The Conservative, Reform and Reconstructive Jewish denominations all acknowledge that the idea that the Jews are "chosen" is merely an ancient view that most ancient cultures held regarding their place with their own deities. There were no flags, no national anthems, just deities and religion to show national patriotism in those days in the various cultures regardless if you were Jewish, Greek, Roman, whatever.
Modern Jews embrace critical scholarship regarding their religion (Conservation, Reform, Reconstructionist, Humanist), but the Orthodox generally reject it with great distrust, even hatred. This is why the critical view that Jews are actually chosen or special among nations is no longer taught in non-Orthodox Jewish religions. This may also explain why the Orthodox don't allow other Jews into Israel. ("If you don't believe we are special, you don't belong on our land!")
Critical-thinking religious Jews tend to view God less as a Person and more as ineffable. It is not uncommon for rabbis from all four of the more liberal Jewish religions to have this transcendent view, with God being less the "being that hears and answers prayers, rewarding good and punishing evil," and more the concept of Good and Power among the Jews.
Fanatics are the problem, unfortunately. Critical thinking tends to allow for the welcoming of women as rabbis, LGBT Jews, atheists, and agnostics to create and find their own ways among the Jewish people that are just as valid--and these things "just aren't allowed" with the Orthodox.
Unfortunately, as you've mentioned, too many people see only the fanatical behavior, blame it on all the Jews (Orthodox are the minority) and the anti-Semitic behavior gets directed at a Jewish majority that doesn't fit the "excuses" for the hatred.