Your husband sounds just like my dad!
You don't want to live your life the way my mother has had to and though it will be very very difficult for you for the first few months and maybe years if he really cannot change then you have to think of yourself. No one should have to live like you are doing, or how my mother has lived.
Leave whilst you still can but cover yourself for every eventuality that you can think of. Don't be afraid to call the police if he is ever even once violent.
Go for it. In time you'll wonder why you didn't leave years ago.
He will never change, if he is anything like my father at all, they are unable to see that what they do is wrong. They always excuse themselves in their own minds.
Go girl!