Sure, you can break it off. Then you will miss this opportunity to develop relationship skills, and you will probably be more lonely and depressed.
Yes, I lacked relationship skills. I am not great at it even now. I broke off a lot of relationships when I became uncomfortable with my lack of skills or when I was asked a lot of questions about my past. I always hid my jw past. Nobody could really relate to me when they tried to talk about their childhoods.
People learn how to be in a relationship slowly IRL. They interact with kids their own age. They date. They go to parties and proms. Their little hearts get broken. All of this is how a person learns how to do relationships. We skipped all of that and must learn it in adulthood.
So here you are. It's up to you.
If you decide to stick with it--
Have you told this person about your past & feeling that you don't know what to do?
Ask specific questions from people you trust, for example, how often should I call him?