The reason this man thought you were attacking him is that Witnesses have poor boundaries and a permeable sense of self.
In their eyes, God and their Organization are the same. They also sometimes mix themselves in there as "servants of God's Organization."
They also have poor boundaries with other JWs since they are all servants of Jehovah. Then this extends to those that the preach to.
Watch. You may have to defend your personal boundaries. Personally, I think that asking a perfect stranger to give you their phone number is a boundary violation.