I can't quote any WTs, but I would venture that transexualism is prohibited because men are expected to dress as men, and women as women.
There is a scripture in Deuteronomy they like to bring up, conveniently ignoring that it is an Israelite law, which JWs are no longer supposed to be bound by. That is one way to refute it, but maybe the best way to refute it is to tell them that there is a verse nearby prohibiting the wearing of polyester suits!
Then counter with the argument that their own Bible says that when God created humans, it says "male AND female he created them." Not male OR female! And in fact, some babies are born with both male and female genitalia.
They also believe that life as an individual in God's eyes begins at conception. Hence, abortion is wrong. If that is true, then how can all the men in the congregation, including the elders and Governing Body, be in good standing when they all started life as girls! Hormones to change some girls into boys are naturally introduced to the embryo weeks later. Do they all need to start living as women to be acceptable JWs in God's eyes?
There are so many ways you can twist around their arguments, but the bottom line is, like homosexuality, they will never accept it. Hopefully with these arguments your brother can at least shut them up.
Good luck to him!
Tammy