Can you take a day off work next trash day and videotape him bringing your bin out of his house? Then when you see your new bin, empty it into the ratty old bin (or onto his front lawn if you're feeling pissy ) and take yours inside. Then report him to NIS... maybe they'll deport him if he's in the country illegally. If he's here legally, he shouldn't be doing things that are socially unacceptable like stealing other peoples' property.
I used to live in an apartment complex with a common laundry area. I bought a couple of new laundry baskets and put my name and apartment # on the bottom of the baskets. Since it seems stupid to truck an empty laundry basket up and down between loads, it appeared that everyone just left their baskets on top of the machines with their laundry inside. Makes sense, right? Well, some idiot decided to steal both my new baskets and left their crappy broken ones in their place. I reported it to the building manager and told him that mine were marked with my name and apartment #. He suggested staking out the laundry room for a few days and if someone showed up with my baskets, to just take them back. That's what I did, and from then on, I just carried the empty laundry basket back up to my apartment until it was time to bring my laundry upstairs.
Maybe when you get your new bin back you can start using two bins to prevent him from stealing it again. Keep the clean new bin inside your premises, to collect and sort your recycling. On trash day, transfer the contents of the new bin into the old bin and bring it to the curb. The new bin never leaves your garage, and he won't be tempted to steal the old one.