But if the manslayer shall at any time come outside the border of the city of his refuge where he has fled,