It is taught: A bunch of people in caravans were on a journey when they ran into a gang of gentiles who said "Give us one of your group so that we may kill him otherwise we will kill all of you!" Even if the entire group will be killed it is forbidden to give up a single Jew to be killed.
But if they demand a specific person by name, such as Sheva Ben Bichri, then they give that person up and are not killed. Says Rabbi Shimeon Ben Lakish: this only applies if the person specified had a death penalty upon him, like Sheva Ben Bichri.
And Rabbi Yochanan [disagrees and] says even if he does not have the death penalty upon him [they give the specified individual up].
Ullah bar Koshev was wanted by the [Roman] Kingdom. He fled and went to Lod, to Rabbi Yehoshua Ben Levi's. They came and surrounded the city, and they said: if you don't give him to us, we will destroy the city. Rabbi Yehoshua Ben Levi went to Ullah and persuaded him, and then gave him [to the Roman authorities].
But Elijah - may he be remembered for good - who would appear to Rabbi Yeshoshua ben Levi and teach him things, suddenly stopped appearing to him. He fasted several fasts, and Elijah appeared, an said, "So now I appear before informers?!" Rabbi Yehoshua ben Levi responded: "Didn't I do according to rabbinic teaching?" And Elijah answered: "And was that teaching for the pious?!"