Pirkei D’Rabbi Eliezer, Horev 24

Rabbi Pinchas said that they did not have enough bricks to build the city and the tower [of Babel]. What did they do? They would make bricks and count them, like an artisan, until they had built the tower to a height of seven levels. This height was accomplished on its east and its west. The builders brought the bricks up on the east side and brought them down on the west. If a man fell down and died, no heed was given to him. But when a brick fell down, they stopped work and wept, saying, “Woe unto us! When will another be brought up in its stead?” When Abraham son of Terach passed by and saw them building the city and the tower, he cursed them in the name of his God: “Oh God, confound their speech!” [AJWS translation]