I'm one to go for the highschool years... S2+3 were, to me, the high point of the series, it didn't reach that intensity again for me up until S6.
While I definitely wouldn't say in hindsight they should have done it any other way (anyway, it wouldn't have been realistic at all, duh!), at first viewing I really disliked S4 for breaking up (literally 🙂 ) the entire setting and part of the cast I'd grown to love.
And yes, it's personal reasons again, too - I watched this around 98-00, and at that time, I regarded my own last two-and-a-half school years (91-93) as the high point of my life, which pretty much fell apart when I went to university... so I reacted badly to the same transition on screen (Buffy's disorientation at UC Sunnydale hit home too much for me, I guess).