Ah beer! My favorite subject besides guns.
I did the homebrew thing a few years ago and it was fun. The beer was so much better than storebought, in my opinion.
Most important part of homebrewing is CLEAN CLEAN CLEAN! Don't use tap water, use distilled water. Sterilize everything. Twice. That's the most time-consuming part.
Like others said, go ahead and splurge for a starter kit from a home brewer supply store. I got mine for $100 and it included everything, even 2 cases of bottles, a big bag of caps, and even the press to cap the bottles. Well worth it as an initial investment, because after that all you really need to buy are the actual ingredients and some more sanitizer and bottle caps. Everything else you just clean and reuse, even the bottles.