Sounds to me like he was a control freak. He wanted you down and depressed, that way he could treat you whatever way he wanted and you would have to put up with it. He didn't feel the need to make any effort to please you and you were supposed to be happy with it, because he was the 'head', and you were to be the submissive wife.
Now, he has started doing all the things you wanted to do, and he is still controlling you, you feel you can't move on due to his behavior. Of course he won't answer, he might not even realize he is doing it consciously. But you are still being submissive by letting him toy with your mind. Refuse to listen to what he is doing, it doesn't matter what he is doing. It only matters what you are doing, you are wasting you life being concerned about him. Get out there and hike-there are hiking clubs and all different kinds of clubs listed on Meetup and they do lots of things that don't require a ton of money. Here is wishing happiness to you!
.