I wasn't overly sad when he left, but I think you have to look at Riley in terms of what Joss and company were trying to accomplish with his character.
He was never intended to be a true love for Buffy like Angel was. He was supposed to be (and they harped on this forever) nice, stable...in short, he was safe and boring. Because that's what Buffy needed at that point in her life. Sure, he turned out to be a pawn working for a morally ambiguous black-ops military operation, but he was still safe and boring. And later, he turned to pimping out his arteries to vampire crack-whores because he knew Buffy didn't love him but he couldn't bring himself to leave her in order to find emotionally fulfilling love with someone else...but he was still safe and boring.