The Acts of the Apostles recounts the story of the original Pentecost (Acts 2). Jews from all over were gathered in Jerusalem to celebrate the Jewish feast. On that Sunday, ten days after the Ascension of Our Lord, the Apostles and the Blessed Virgin Mary were gathered in the Upper Room, where they had seen Christ after His Resurrection:
And suddenly there came a sound from heaven, as of a mighty wind coming, and it filled the whole house where they were sitting. And there appeared to them parted tongues as it were of fire, and it sat upon every one of them: And they were all filled with the Holy Ghost, and they began to speak with diverse tongues, according as the Holy Ghost gave them to speak. [Acts 2:2-4]
Christ had promised His Apostles that He would sent His Holy Spirit, and, on Pentecost, they were granted the gifts of the Spirit. The Apostles began to preach the Gospel in all of the languages that the Jews who were gathered there spoke, and about 3,000 people were converted and baptized that day.
The Birthday of the Church
That is why Pentecost is often called "the birthday of the Church." On this day, with the descent of the Holy Spirit, Christ's mission is completed, and the New Covenant is inaugurated. It's interesting to note that St. Peter, the first pope, was already the leader and spokesman for the Apostles on Pentecost Sunday (see Acts 2:14).