Books have been written about this.
Off the top of my head, the whole of prayer has to do with the will of God--Thy Will Be Done.
So, in a sense, each prayer is answered positively when, like Christ, we can say, "Nevertheless, not my will but Thine be done," even if the answer is not the precise answer we are asking. When you ask with faith, you know that the bottom line is that God sees the whole picture--for everyone at everytime at every place, not simply our point of view. I saw that recently, when a friend died of cancer. And the year before, when another friend died of cancer. The bereaved families and church members have grown thru the sorrow--we love each other more kindly, and we can truly comfort each other. It is hard to explain; you would have to experience this.
That sounds like a copout to a lot of folks. So let me add some more to this.
Consider the story of the widow who kept bothering the unjust judge. Or the Gentile woman who asked for her daughter to be cured of demons, and met with an answer that sounded like a put-off: It is not right to give the children's bread to the dogs. Sometimes God doesn't answer yes immediately--and it looks like a 'no'. The purpose is often to strengthen in us our resolve, or in the latter's case, to show the resolve that was actually there. We define ourselves in a sense.
Another reason, among many others, is that prayer is like a training method of sorts. Thru prayer we begin to think God's thoughts after Him. Our hearts get tenderized. Face it, there is a dang lot of evil out there, and it is easy to shield yourself from the knowledge of all this endless pain going on by just asking "why??" and walking away. You can't see an answer. But when you pray for someone, you put yourself into a dangerous kind of position: you are identifying yourself with that person's painful situation. It can increase your own pain! So, it has a sharp edge to it; it isn't all just nice fluffy good thoughts. When you really care about someone or something, you suffer with him/her/it. Prayer is gambling a bit: you could get a wonderful answer and be joyful, or you could get no answer and still deal with pain--perhaps worse pain. But we see that even in an answer with pain, there is yet a blessing.
If every answer to prayer was just as every Christian prayed, then the entire world would become Christians thru the magic of push-button prayers. Everyone would be Rice ball Christians, or Cargo Cult Christians--people who are in it for the payoff. --This is not really any kind of argument of proof, by the way, but just an observation, since many here think that it's a pretty low policy for God to dangle carrots of any sort.
Prayer is a discipline, like going on a diet--for life. What I mean by that is, it does discipline your heart. You learn humility. You learn how to accept grace. You learn how to be bold. You learn how to love better. One's character changes, imperceptably, but it changes over time. There are many kinds of people, and many kinds of praying. know a S. Korean gal who spent hours praying at a time... some are quiet, some noisy. I'm not judging anyone--it's God who knows whether their prayers are real, or "for show" like the pharisees.
Why does God need our prayers if He already knows what He is going to do? Or what we will be praying? Well, He may know, but I don't always know yet. I guess it's all for my sake. He knows what He is going to do, and puts it in my heart to pray for it so He can answer it. Not just my heart alone, but many people's hearts. You learn that it isn't all about asking for stuff, but spending time WITH God. There is relationship development going on.
I guess, in a nutshell, prayer has a lot more to it than a pushbutton framework. It's not just utilitarian. God kills lots of birds with one little stone. He is mostly concerned with our character, which grows thru time and thru situations. And our character is not formed by our saying simply, "Gee, I want to be a nice person", but thru the blood, sweat, and tears of faith. Faith and prayer grow each other.
(No wonder trials are the crucible for us--they really do force us to define ourselves more clearly by our choices. And prayer is also a great method thru which God molds us. We are more important to God than we believe. We just want an easy life, and to just get along till we die. God wants us to find our center in Him, a much greater blessing.)
BTW, my friends who died are not lost to God or to me; it is, to me, like their leaving the shuttered home with us and going into the sunlight, simply out of our view. We grieve, but only for a time.
I hope some of these things might make a little sense to you.
bebu