Regarding your time to fart around house ... I'd be careful that you're not breaking up your work and play well enough. Although it might be nice that during, say, an 8 hour work day, most of it is farting around, wouldn't it be better to work hard for 4 hours, concentrating the work load, and then go fishing for 4 hours? Or whatever else you like to do - riding your bike, swimming in your pool, going to the beach?
Even if the nature of your business requires you to be home, manning the phones during business hours fielding calls from employees and what have you, it still seems like you need to break up your work and play a bit more, simply to enjoy the play more.
I fell into a trap in my business, which was not based at home, where I was at the office all day every day. It was pretty cushy most of the time, but I thought I had to be there all the time just in case. And then someone suggested to me, "why don't you go home for a few hours?" I realized I could! I could cue up what happened while I was gone for when I got back. Then I got smarter and smarter and consolidated my work and delegated more. Now when I am at work I work my ass off, but when I don't have to be at work I am enjoying myself and there's a real separation.