best bet............... meet him somewhere. say you have an errand early that morning and find a place not to far from his place... that way if he starts some s*** you can hightail it outta there.
otherwise id thank him and pass on that invitation, and express joy that he is doing better.