This is not an easy situation for sure, but I will share how one of my kids handles this with her elder father. I faded first, then one by one the three girls did the same. (We have been divorced for many years) Once in a while he will invite her for a talk he is giving at the KH or a convention. She tells him that honestly, she loves to hear him speak, but she would only go to please him. She tells him she has no interest in JW anymore and she thinks it is cruel to give him false hope that she will ever return to the org. He still asks, but she is firm in telling him that she will not attend.
As others have already said, no one knows your family like you do, but it does seem more honest to simply say no and stay away. You have your own family now, with a child on the way. My advice is to focus on them and let the JW relatives figure out how to deal with their own feelings about the whole thing. Don't let them use emotional blackmail to push you into something you don't want to do.