It depends on how they're modified. If they modify corn to store up a little more water in order to be more drought tolerant, I've seen no reason to believe that this would result in the corn being bad for our health (well no more than regular corn). If they modify it to produce toxic insecticides, then maybe we shouldn't be eating it.
This is the sort of problem that happens when you lump a broad group of things into one catchy label. We've been producing hybrid plants (for example, the tangelo) for a very long time, and in most cases GMOs are just more advanced versions of that. Most of the fear over GMOs, in my opinion, is based solely around it's novelty.
Since there's not much market for corn that kills people, I'm not too worried...