You obviously sound like you and you're wife want to be reinstated, I think that is commendable if that is what you want.
My sister went through a similar problem, her husband went of with another sister and they were both disfellowshiped, after nearly two years they were both reinstated.
What they mean by the adulterous relationship has me concerned, once adultry has been commited and you don't sleep with youre' wife again after it, that means she has not taken you back so you are free to divorce and remarry. If you and youre x wife slept together after the adultry her being fully aware of the adultry and she willfully slept with you, she then forgives and takes you back as her husband, scriptually if you married the other women you are in an adulterous marraige.
You may have to talk to youre x wife and find out what she has said to the elders, even though she herself committed adultry or fornication that in itself would end the marraige, but the elders may not know about what she did, only what you did, and her story may be different.
It amazes me how damn ignorant witnesses can be, imagine if King David walked in and wanted reinstatment after all he did, I'm sure he would be treated differnently.
I would persist with it, and hand deliver a letter to each elder in the congregation.
Good luck.