Only a portion of the Israelites took advantage of the permission of Cyrus. Many remained in Babylon, since they were disinclined to relinquish their property. They who returned were persons whom God had especially stirred up to make sacrifices for His glory.
Related Commentaries of Ezra 1:5
Ezra 1:5
5 Then rose up the chief of the fathers of Judah and Benjamin, and the priests, and the Levites, with all them whose spirit God had raised, to go up to build the house of the LORD which is in Jerusalem.