Maharil Hilchos Rosh HaShanah
There was a minhag to go on Rosh HaShanah, after the meal, and throw our sins into the sea.
2. What is the reason for Tashlich?
The Maharil writes, There was minhag to go to the river and say [the pasuk of] Tashuv Terachmeinu (Michah 7:19). The reason for this is found in a midrash, (Yalkut 99) [Tashlich] is in remembrance of the time when Avrham passed through the water and it went up to his neck...But [the Sefer Minhagim] adds that we should see live fish. And it is possible that this is for an omen that the evil eye not rule over us, and that we should be fruitful and multiply as fish.
Rabbi Jonathan Saks: Commentary to Tashlikh from pp. 936-947 of Koren-Sacks Machzor for Rosh HaShana
It is a custom, on the afternoon of the first day of Rosh HaShana (or second, if the first is Shabbat) to go to the shore of the sea, the bank of a river, or other running stream of water, as a symbolic enactment of the words of the prophet Micah: "He God] will cast (tashlikh) into the depths of the sea all their sins" (Micah 7:19)...The first mention of the custom is in Sefer Maharil of Rabbi Jacob Moellin (d. 1425)...Many folk customs have become associated with Tashlikh, among them the custom of throwing crumbs into water as a symbolic gesture to accompany the process of repentance, begun on Rosh HaShana, as if we were "casting away" our sins. This practice was dismissed by some halakhic authorities and ridiculed by gentiles. However it is less ridiculous than it seems. Maimonides writers about the scapegoat on Yom Kippur over which the High priest confessed the sins of the people and which was then sent out into the wilderness:
"There is no doubt that sins cannot be carried like a burden, and taken off the shoulder of one being and laid on that of another. But these ceremonies are of a symbolic character, and serve to impress people with a certain idea, and to induce them to repent, as if to say: we have freed ourselves of our previous deeds, have cast them behind our backs and removed them frum us as far as possible (Guide III:46)."