I guess it depends on where you live. Here, the Y has signs all over that say that the YMCA is a completely equal opportunity organization and that membership is open to anyone regardless of religious faith. Signing up is a matter of paying the bill. They take a picure for an ID so that they can enforce the "pay the bill" thing.
This link is what I am familiar with- a community support organization.
http://www.ymca.net/about_the_ymca/
PS- Do you suppose its also "religious harlotry" to join the JCC (Jewish Community Center)- also a non-religious, great place to swim and work out.