Are you Hispanic? If you had answered in Spanish, they may have engaged you in a conversation. A congregation is told to focus only on the language of the congregation. This is to prevent harassing the same territory/people repeatedly and often. The English/Spanish/ASL/French congregations do not coordinate which territories they will be working. If they happen to see another congregation in that territory, then they will move to another, but otherwise the foreign language congregations will not normally engage English speakers. That is the English congregation's responsibility and territory.
Foreign language groups/congregations have a list of names/addresses that they target and don't usually go from door-to-door. Sometimes they will knock on a few doors to ask if anyone in that area speaks Spanish, French, or whatever language they are involved in, otherwise they focus on the addresses they already have on file.
Source: I'm still stuck in an ASL congregation and have had many friends in Spanish congregations.