1. Glory
2. The Mayor
3. The Master
4. Angelus
They stand far above the rest.
Glory requires the most effort on the part of the gang to defeat. They had a Slayer at her peak, a sophisticated robot, a magic macguffin that weakened and disoriented Glory, a literal wrecking ball, a troll god's hammer, two powerful witches, probably the most capable Watcher in the world, an ex-vengeance demon, and a powerful vampire and ultimately the battle still claimed Buffy's life. And, in general, Glory does damage to the gang or puts them on the run every time they encounter her. She brainsucks Tara, she takes Dawn, she beats Spike to a bloody pulp, she beats the hell out of Buffy more than once, Willow nearly dies trying to take her out, she chases the whole gang out of town.
The Mayor became completely invulnerable for a time, had a rogue Slayer at his side, and in the final battle against him several significant characters were killed (Larry, Harmony, Snyder, etc) and the entire school was destroyed.
The Master killed Buffy himself. Glory created circumstances that led to Buffy's death, but the Master straight up drained her blood and drowned her in a puddle. And you can just see how much fear he inspires in characters like Angel and Darla. Season 1 didn't have the budget or episode count to give him as much space to grow and menace the cast as the other big bads, but the performance and the writing for the character hold up.
Angelus kills Jenny and puts a lot of scares into the gang, but his intimidation factor suffers significantly because Buffy pretty much whoops his ass over and over again. The only time he really beats her is when she's super sick in Killed by Death. Every other time Buffy fights Angelus (or Angel, for that matter) she wins, usually rather easily. And the fact that Angelus breaks in to watch characters sleep, draw pictures of them, and kill their fish instead of actually hurting them sort of wears thin. What saves the character is that you can feel how much it hurts Buffy to have to fight him.
Angel as a show has a problem with a lack of strong individual villains. Instead it's mostly Wolfram and Hart as a whole and nothing ever gets resolved either way. Season 4 has Jasmine, but I hate everything about that character, her storyline, and what they did to Cordelia.
Buffy season six has a similar group villain problem with Willow, Warren, Andrew, and Jonathan all sharing villain duties and consequently making it all feel less satisfying.
The First doesn't have a physical ass to kick and the final battle is just a bunch of faceless grunts. Unsatisfying.
And Adam? Adam just sucks.