I know its hard but try to keep things from snowballing if you can, at least in your mind and heart.
Dont worry for now about the kids being brainwashed. You mentioned school and kindergarten so they are getting good socialization there.
JWs have a terrible retention rate even when both parents try hard. Your kids will (statistically speaking) move away from this religion all on their own shortly.
I know opinions are a dime a dozen but I SO like the idea of locking your man cave. If a key lock seems to obvious, get a combination lock....either way!