A lot of information is also noted on the house-to-house record. For example, if you spoke to someone or left a tract, whatever, you would note down as much info as possible. Some you kept for yourself as RVs and others you would pass on the "brother" taking out the group or aux pioneer to help them with their time. Publishers would often pass their house-to-house records with notes about the person to aux and reg pioneers to followup on any interest found.
Also, too folks like to show off how familiar they are with the territory, so its common for them to share details about the folks in the area as they work from house to house. So its very possible that they know all about you before they even meet you.