A few comments... not having the full details of what occurred, I may be off on my observations.
First - what others have already said - and I agree - don't trust him - or anyone - who 'happens to stop by'. Never - ever.
I am not aware if you or he brought up the 'apostate' issue. If he did - then it IS an issue, even though he said it wasn't. It was/is in his mind, and is a problem (to them).
He may have revealed to you what you may think is damaging remarks for an elder to make, but since he was alone - you can't point the finger (ala Ivasion of the Body Snatchers) and scream that he made them. He will deny it, and claim you're a witch - after which, you'll be burned at the stake. (Your word against his.)
You have the correct approach - do NOT trust any of them - do NOT tell them anything.
At the same time, you are a wonderful caring person that could see that he was stressed out, and you were able to help calm him down, possibly averting a potential heart attack (on his part).
While he may be in the beginning stages of 'seeing' again, and having problems dealing with it, he will have to find his own way out. You cannot trust him, or anyone else, right now - lest they 'turn you in', so to speak.
Others in this thread have suggested that he may have been 'hitting on you'. Do not dismiss that too lightly as being ridiculous. It is good that your husband was home.
A long time ago, when I was still married to the jw-wifey, we had a plumbing issue that she called a JW 'friend' to come fix. I was at work. He was the father of a woman that was the same age as my wife, and they went to school together, so she trusted him.
Anyway, he came over, fixed the problem, and then tried to push himself on my wife, groping her, and wanting to have sex with her. She somehow managed to push him off (more than once), and get him to leave. He was not an elder, but was married.
She told me about it, and also reported it to the elders - who did nothing but tell her that they knew about him. (Oh, and were a bit worried, and asked what I was going to do about it.)
So - don't trust him - in more ways than one.
Regards,
Jim TX