Babylon was still one of the most populous cities in the world in the first century and wasn't depopulated until several hundred years after the Roman Empire fell. It was finally destroyed when they had several times rebelled against and Iraqi leader and he forcibly removed the population to other areas in the area. So really Babylon was never destroyed after Assyria but just abandoned.
Thanks Kepler for your post about the curse of the Assyrian over Babylon, I read about his a little but never connected the dots. It's clear to me this is what Isaiah was talking about and Jeremiah possibly and that to use it in connection with Jerusalem was per propaganda.