Well done Joel (and Andrea!)
"We would love to have dinner with you as a family. Myself, Andrea, the kids, and you."
Brilliant.
My dad told me (after years of my battling to keep in touch with them) that it was a mistake to ever have a relationship with me. Obviously that includes my children then. If they cannot respect me, my husband and my marriage, then they sure aren't having time alone with my children. I have said I will take the children to their house, and they are welcome in my house, but they will not be allowed the children on their own.
That was years ago, and was 'pretty much it'. The only time I have contact with me parents now is if one of my brothers is at their house (eg one of my borthers was sick, and I went to visit him while he stayed with my parents).
You have done the right thing. But you knew that already!