As far as that goes, Angel left Sunnydale because he believed that his presence was potentially harmful to both Buffy and himself because their relationship, as conditions existed at that time (and have never changed, except for one day that was erased) could never be allowed to develop into what they both wanted. Just the act of being close together presented a temptation that, if succumbed to, could lead, quite literally, to massive death, destruction, and and even the possibility of apocalypse. With that in mind, why did either Buffy or Angel EVER tempt fate by being even within shouting distance of each other? About the only kind of contact that they should have been risking should have by phone, mail, or the internet. Yes, Buffy had no business going to Angel's town, and, likewise, he had no business ever coming to hers. We have seen that any time these two get too close, lives are forfeit or hearts are broken, whether Angel's soul is lost or not.