Imagine if you will a large room with those wooden block you played with as a child. A whole heap in the middle of the floor.
The room is empty otherwise.
To build anything you must resort to some number of those blocks. As you build you make the heap smaller and the object larger; but, the total of the blocks remains a constant.
This is an analogy.
All things are made of a "smallest part". We used to call them atoms. Suffice it to say these building blocks are what "things" are made of.
Now, to disprove time travel!
If you build a little house out of blocks the heap grows smaller by the same amount as what appears in the building.
TODAY'S HOUSE is from the heap. (It diminishes the heap).
For a number of houses to be built they must always be built from actually existing blocks in the heap. OTHERWISE, the TODAY'S house must be scavanged.
This is the key to understanding.
Things fall apart, die, disintegrate and dissolve freeing up their blocks (atoms) to be used to build something else.
There are probably atoms from Julius Caesar in your body right now. He won't mind; he's not using them.
BUT!
You (being made from part of Caesar) cannot go back in time and visit with Caesar since his blocks have been used for TODAY'S constructions.
Yesterday has been demolished and the parts recycled!
You'd have to double/triple/etc. all the atoms existing in the universe many times over to have enough blocks for a continuum to allow the past to "exist" and still build a future.
So--this is a physical limitation.
That is why there can be no time travel. Not enough parts and reused parts.
(p.s. "TIME" doesn't exist the way atoms exist. Time is a bookkeeping method of marking the distance between actual events.)