Because I'm such a huge fan of (...wait for it ...) Giles, I have to say the college years, because his character was more interesting, more involved, and more evolving during those years. And I loved the season 4 set of his pad, and the season 5 set of the Magic Shop.
That said, the high school setting of the show was the very most important thing that kept me watching after I saw the premiere for free on iTunes last September, so I did love the high school years. And I did love the library. It's always a bit wrenching to see it destroyed in Graduation Day Pt. 2, and very moving to see the destroyed and abandoned library in Doomed. (Though every time I watch that ep I wonder why in heck they hadn't had it cleaned up and demolished by then.)