The 'curt' answer I always used to get was that it's out of repect for God...but shouldn't our respect for God be a personal thing, doesn't mean too much when it's forced and resented. In your heart, you could be being respectful to God in whatever you wear, whatever feels appropriate to you, and with God being able to read your heart condition and all....there shouldn't be limits.
I can understand where it's come from in a way but I still don't agree, if the 'truth' is meant to be so personal a thing, why won't the society let us make up our own minds?