1. What is the source for Tashlich?
רש׳י
Rashi, Shabbat 81.
The Geonim used to plant a seed fourteen to twenty days before Rosh HaShanah and then on Erev Rosh HaShanah, after it sprouted, they would wave it over their head and recite זה תחת זה and then throw it into the river.
מהרי׳ל
דרכי משה
Maharil Hilchos Rosh HaShanah
There was a minhag to go on Rosh HaShanah, after the meal, and throw our sins into the sea.
2. What is the reason for Tashlich?
Darchei Moshe
The Maharil writes, There was minhag to go to the river and say [the pasuk of] Tashuv Terachmeinu (Michah 7:19). The reason for this is found in a midrash, (Yalkut 99) [Tashlich] is in remembrance of the time when Avrham passed through the water and it went up to his neck...But [the Sefer Minhagim] adds that we should see live fish. And it is possible that this is for an omen that the evil eye not rule over us, and that we should be fruitful and multiply as fish.
לבוש
Levush Orach Chaim, 596
It was normal to go to place that had live fish. To remember that we are like these live fish, who are ensnared suddenly in a net. So are we ensnared in the abyss of death and judgement. And in doing so, we will contemplate repentance more.
תורת העולה
Toras HaOlah 3:56
The natural order demands that water should cover the entire planet and it is only through the chesed of God that He allowed us to have portions of land on which to live. Therefore, we go to the water to see that idea first hand and through that we coronate God as King of the World and it causes us to repent. We don't throw our iniquities into the water- they are figuratively thrown in the river when we are inspired to repent.
3. Is this Minhag accepted by everyone?
מעשה רב
Maaseh Rav
The [Vilna Gaon] did not go to a river or a well to say Tashlich.
ערוך השולחן
Aruch HaShulchan Orach Chaim 583:4
...you should be careful that women don't come [to Tashlich], and in a place where women go [to Tashlich], it is better that men not go at all. There are a lot of people who hold that you shouldn't go to tashlich, and their reasons are known to them.
Iggeres HaGra
Concerning solitude, the main thing is to remain at home. Even your visit to the synagogue should be very short. In fact, it is better to pray at home, for it is impossible to be spared from jealousy or from hearing idle talk or lashon hara in the synagogue. And one receives punishment for this, as we find (Shabbos 33a), "Also one who hears and is silent...." This is even the more so on Shabbos and Yom Tov when they gather to talk - It is then better that you don't pray at all. Refrain also from going to the cemetery (especially women), as it leads to all kinds of sorrow and sin. It is also advisable that your daughter not go to the synagogue, because she'll see beautiful clothes there, become jealous and talk about it at home. This will lead to lashon hara, etc.