The idea that everything that exists stands in an infinite chain without anything outside as the ultimate cause is an interesting idea but I think it calls for some evidence - SBF
It requires no evidence to say that we can currently trace history back to the Big Bang but no further. Beyond that the only sensible answer is, 'I don't know but I'm open to new evidence'.
The material is currently all we have evidence for. It is the person who wants to introduce an ultimate cause who has all of the burden.
'There is an ultimate cause that stands outside of all things', and, 'I have no idea why there is something rather than nothing, but I am unconvinced by your theological hand-waving', are not epistemologically equal positions. The first is a faith position, the second is rational.