like you I grew up in one of thee worst cities in America , Newark NJ. I made it out and made a success of my life as well. However I would never be so arrogant as to insinuate that because I did everyone else could. With anyone all it takes is one bad mistake to keep someone from moving ahead.
I also grew up in one of the worst cities in America (and have seen cops detail and assault people for no reason, personally), and also made it out, and would take what you said a step further - any part of me that is a success, including my personal effort, is still based on luck of the draw (my effort being rewarded where other peoples' weren't, me happening to meet the right people at the right time, etc.).
I just shake my head at the bootstraps/personal responsibility crowd. There have been many cases where I know I started lower than some of the same people claiming that, worked harder, and achieved more, and yet I'm still not ignorant enough to deny that all that is still just a coinflip and with one minor changed circumstance, I could have ended up on the streets or something. I think a big part of it is that, starting so "low" on the totem pole, you tend to meet more peers who have as much or more potential than you growing up, who never realized it just due to the wrong thing at the wrong time. People that start of relatively middle, or who completely leave their pasts behind on leaving to not SEE things like that as much, tend to just not make that connection, I guess.