Daniel 9:1,7 shows that the land including Jerusalem was inhabited by Jews before the exiled Jews returned from exile and set foot on the land.
Daniel 9:1 In the first year of Da·riʹus the son of A·has·u·eʹrus—a
descendant of the Medes who had been made king over the kingdom of the
Chal·deʹans
Daniel 9:7
7 To you, O Jehovah, belongs righteousness, but to us belongs shame as
is the case today, to the men of Judah, the inhabitants of Jerusalem,
and all Israel, those nearby and far away, in all the lands to which you
dispersed them because they acted unfaithfully toward you.
The land was not without inhabitants for 70 years. Now what does desolation exactly mean? That's a different topic.