To go out with a non believer is treated as disloyalty to Jehovah and will result in marking which is one step up from disfellowshipping. It is unchristian and extremely insulting to the other person who is punished as well. Jws do not see that this behaviour makes hem appear arrogant and extreme they are so busy worrying about rules and punishment. The message of Christian love has been buried under a pile of regulations no wonder people from mainstream churches which teach about the grace of god, love and forgiveness are confused and insulted by the witness approach.
If you love one another then that is what counts. If he has a family that doesn't follow bat shit crazy regulations then that is a blessing. It may be that your family will soften their approach later but you can't count on it. However if this man cares for you and is willing to support you through this then be there for each other. Your long term happiness and your life is your own, don't allow people trapped in a cult to determine who you can or cannot go out with.