I understand it is not fair for him not to pay his share or some share. Your situation sounds not all that bad. He is easy to deal with and makes few demands.
I wouldn't take or deliver the child to him, that makes everything easy for him. At least swap weekends when his turn is due. Make detailed notes and records of his true interest in the child. Without dates and documentation the courts and lawyers are less likely to be successful, your relation ship could turn to the bad side anytime. You can get custody if he continues to neglect the payments. In the US if you have a lawyer they will pick up and jail him for pastdue child support, that will change his attitude also.
I have a daughter that had an ex that was a total ass, and wouldn't pay support, everytime the law would allow we had him jailed, that became to embarrasing for him so he allowed me to adopt the child. we promised that he could see the child anytime after the adoption, but he was too difficult to deal with so we just ignored him and now he is completely out of our life. If he ever shows up, we will call the police immediately.
As long as you have an amiable relationship maybe it is best left alone. You do have legal options, just get a lawyer and they will jerk him into line.
BREEZE