Everything is your fear, your anxiety, your judgement. This isn't about you this is about him. And no doubt he is a bright capable man. From what you describe, everything he's said is true.
There is good debt and bad debt - bad debt is generally for things, TV, eating out, etc good debt is called an investment and it can lead to better things education generally improves a person as a person and gives them skills to provide for themselves through meaningful work
I'd suggest that you get counseling to deal with your anxiety because that is all about you not trusting yourself and has nothing to do with the reality of what you describe