Um... in America everyone (by everyone I mean American citizens 21 and over who isn't a convicted felon) is allowed to carry a gun on their hip. You can carry a holstered weapon and you don't even need a permit. That's the law pretty much everywhere except for those few nutty cities where handgun ownership is outlawed. And those cities that have outlawed gun ownership are in trouble now because those laws have been declared unconstitutional. By the way, if you happen to live in one of those cities where handgun ownership is illegal, you know just how bad the crime rates are.
If you do chose to carry a gun around in a holster, be prepared to be asked questions constantly by people who are ignorant of the law, and that includes law enforcement officials who are ignorant of the law.
What you can not do is carry a concealed weapon without a permit. The reason many people choose to get a consealed weapons permit isn't because they want to conceal their weapon, it's to avoid the unwanted attention that carrying a holstered weapon causes.
I carry a concealed weapon on occasion and nobody ever knows I have it. If you know I have it, then that means I'm about to use it on you. After 11 years of carrying a concealed weapon, I've never had to fire it outside of a gun range. And no, I have never contemplated shooting someone because they flipped me the bird in traffic or called me a name. It's to defend my life, not my pride or feelings. I don't want to use it, but if I have to I will.