Nita, I don't want to give advice but to offer my perspective:
Your children did not marry this guy. If you want to live with him, that's fine, but have you ever asked your children if they want to live this way? Is there another relative or other relatives your children can stay with if they want out?
The marriage does not seem to be working as you see it, but working just fine as he sees it. It does not sound like an equal partnership (from your description, which is all I have to go on), nor does it sound like a good model for your children to learn from.
On another thread, I saw a quotation from Maya Angelou that went something like "When someone shows himself to you, believe him." From yet another thread, I remember the advice to ask yourself what you would say to one of your best friends or one of your children in the same situation.
This is difficult for you. I can tell you are torn. I hope you make the choice that is best for you and all other parties involved. BTW, I do not think it best for anyone who is an adult to be tolerated or enabled when engaging in behavior that is harmful to self or others.
Keep us updated. Best wishes and Happy New Year!
SandraC