Thanks Phizzy. That was the direction I was heading. I suspect it was part of the process of cementing control though claims of apostolic succession. That wasn't an idea immediately seized upon.
I also wanted to mention Irenaeus's intro to his third book Against Heresies, wherein he quotes Luke 10 (the sending of the 70/72) as being the words the Jesus said to the 'apostles'. He continues to use the term when recounting the Pentecost scene of 120 'apostles' getting 'perfect knowledge' . What's especially surprising is it was part of his argument against 'heresies' in which he enumerates a succession of church leadership in Rome. In short, he seems to be using the term 'apostles' as meaning first generation Christian. Peter and Paul are described as 'the two most glorious apostles'.
Call to mind then, the things which I have stated in the two preceding books, and, taking these in connection with them, thou shalt have from me a very copious refutation of all the heretics; and faithfully and strenuously shalt thou resist them in defense of the only true and life-giving faith, which the Church has received from the apostles and imparted to her sons. For the Lord of all gave to His apostles the power of the Gospel, through whom also we have known the truth, that is, the doctrine of the Son of God; to whom also did the Lord declare: "He that heareth you, heareth Me; and he that despiseth you, despiseth Me, and Him that sent Me." (Luke 10:16 story of the 70).... But Polycarp also was not only instructed by apostles, and conversed with many who had seen Christ, but was also, by apostles in Asia, appointed bishop of the Church in Smyrna, whom I also saw in my early youth, for he tarried [on earth] a very long time, and, when a very old man, gloriously and most nobly suffering martyrdom, departed this life, having always taught the things which he had learned from the apostles, and which the Church has handed down, and which alone are true.
Even in this context of attempting to argue for the superiority of his Church's doctrine due to succession legends, he seems to be using the term 'apostle' to describe more than 12 guys in the Gospels.
The growth of this concept obviously really gained a footing after the Gospels were written. While the names of the 12 (to match 12 tribes) obviously were not terribly important, the inclusion of a few historical persons into the list, (Peter, James and a John) was necessary.
Once again, we are stymied by uncertainty of dating and the originality of all of our material, but it seems clear that the final solidification of the concept of 12 Apostles as the foundation of the Church took some time.