I'd ignore the calls. If it was important, he'd leave a message telling why it's important. That's pretty much my rule of thumb for all phone calls. I never answer my home phone. If the person calling, even if I recognize the name/number, doesn't leave a message, then I assume it's not important enough for me to call back. If they leave a message then I know the deal and can act accordingly.
Even though you can ignore the phone calls, it's good to be fully prepared for them to finally show up one day when you're at home.
But you can still stand your ground. If you don't wanna talk, don't talk. You can be pleasant enough but just explain that you don't need to discuss anything more with them. If you feel the need for their help, you'll call them.
I remember telling an elder, "I'm in a good place right now. If and when I feel I need your help, I'll call you." Nothing left to discuss after a comment like that.
If it's as your husband says, they have some info on a doctrine or something, just explain that you've done some research since then and you've come to understand it quite well now. Thanks anyway. End of discussion.