I don't think it was specifically about Olivia though. We only saw her in one episode, and he never really spoke about her again, I don't think she personally was all that important to him. But I do think it was symbolic of everything he has given up to be a Watcher, and Buffy's Watcher specifically. We can see that this life is not exactly conducive to healthy romantic relationships (for everyone, but for Giles specifically- death of Jenny, Oliva leaving, etc). Hmm, I wonder if it also foreshadowed him losing his "daughter", Buffy, in The Gift. Remember in Nightmares that his nightmare is Buffy dying. It's the ultimate failure of a Watcher, and more poignant to him because of his relationship with Buffy.
And I'm sorry about your difficulties too. I have had troubles that make it really emotional to watch Darla's pregnancy arc in ATS season 3 for example.