Hi Katie,
I had somewhat the same issue...my ex was, probably still is...extremely controlling (I haven't seen him in a couple of years now) When our son started refusing to go to the meetings (about age 11) he actually called the police to enforce the visitation schedule. My son was scared out of his mind.... which actually worked against my ex... because he won the battle that day, but lost the war. My son never became a jw.
I used to shake in my boots every time there was an issue with him... I figured out, finally that it went back to the jw submission thing..and here I was finally standing up to him....scary at the time, but eventually I grew out of that mindset. I'm not a confrontational person by nature, but standing up for my children was something that had to be done...and they relied on me to do it.
Your ex sounds like the kind of person who will keep pushing and is used to getting his way. Stand your ground. You have the stronger position.
Coffee