It was then that my ex told the elders of our premaital relationship.
Not having read comments yet I will put in my "two cents" worth. Elders, like police, are NOT your friends when it comes to confessing sins. It is too late now for you buy the wisest course is to keep silent. The best you could have done was to tell the elders that HE was only accusing you as a means of retaliation, then quit talking. You would not have been lying. That is the only real reason he said anything about it. He was only interested in taking you down with him. They would have had no second witness. It was your confession that counted as the second witness to his accusation.
Now though you have the chance to get away from those oh so loving wolves. RUN!