I still do volunteer work (I haven't been able to though the last 5 weeks.....not till I'm recovered) and have asked this question many times to our elders. Their response is: while there's nothing WRONG with doing volunteer work, our time would be much better spent in the ministry. Charity work will not serve mankind's long term problems, only the Kingdom will. Therefore, don't bother wasting your time helping some poor soul out now, as hours in Service are far more important!
Whenever we have a WT study on the Good Samaritan, there's always comments made saying that that's what WE'RE doing: we are preaching to our "enemies". I get really irritated and the last time it happened, I put up my hand and explained that this illustration that Jesus gave is not talking about preaching, it's talking about performing acts of kindness and helping people out with MATERIAL needs, end of story.
Needless to say, the Hall was dead quiet after I said that. After all, everyone knows its true. But seeing as no one in the Hall actually DOES any charity work, then there's that uneasy feeling that they're not doing what Jesus told them to do.