I have heard many testimonies of children who have died being in heaven.
If you want to see it in scripture read verse 23.
2 Samuel 12:19-23 (New International Version)
19 David noticed that his servants were whispering among themselves and he realized the child was dead. "Is the child dead?" he asked.
"Yes," they replied, "he is dead."
20 Then David got up from the ground. After he had washed, put on lotions and changed his clothes, he went into the house of the LORD and worshiped. Then he went to his own house, and at his request they served him food, and he ate.
21 His servants asked him, "Why are you acting this way? While the child was alive, you fasted and wept, but now that the child is dead, you get up and eat!"
22 He answered, "While the child was still alive, I fasted and wept. I thought, 'Who knows? The LORD may be gracious to me and let the child live.' 23 But now that he is dead, why should I fast? Can I bring him back again? I will go to him, but he will not return to me."
For those who don't know the story, that child was conceived from an adulterous affair.
Blessings,
Stephen