We need to be careful about making a bit too much of this.
I was an elder for a decade. Yes, there was sacrifice involved. Many times, I "lost sleep" due to elders' meetings that were held after the Service Meeting. I'd never want to do that again. But it was part of the business of the congregation. The alternative was no better - holding elders' meetings on a non-meeting night - it would take you away from your family for yet another night.
That's basically the nature of the gig. The only thing that would make it better would be for the religion to stop demanding so much of these guys.
But it's not all negative. Some people need their clery to be available, sometimes at the drop of a hat. I would think church members lean on their clergy just as much. It's part of his job, too. And in most cases, there's only one pastor or priest to serve an entire church.
The difference is that the clergy get paid. Ideally, elders should too, but then the flock would be expected to donate more and perhaps even a collection plate would need to be passed.