To me, Spike is the biggest loser on the show. The entire point of Spike for me is that that "coolness" you're talking about? Is all an act, and that he's painfully, painfully, painfully aware at all times that if you strip away Nikki's jacket and the peroxide and the scar, he's still William, sitting in that Victorian drawing room, getting rejected. Spike is the ultimate outsider on a show about outsiders. He's not cool, he knows it, and one of the things I love about the Scoobies is that they know it, too (they see right through him, and it's fantastic). Yes, he's a giant loser who somehow, sometimes manages to convince others (and himself) that he is, in fact, a badass. Even rarer (but more awesome) are the times when he really is a badass and exudes it for all to see.
This is the point of "Fool for Love," but if you go back to the earlier seasons, you'll see that those clues were there all along. He's a whipped boyfriend (I hate that phrase, but that's how our society would phrase it) who gets beat up by the Slayer's mom, makes a deal with his worst enemy to save his own skin, takes off like a coward, comes back drunk and sniveling, and then gets emasculated by the Initiative. Loooo-ser.
I will agree with you that the writers did force him where he didn't necessarily fit in S4: he's got some fun stuff going on, but it doesn't quite work because they don't take they chip seriously; they treat him as comedic relief. It doesn't entirely work, and I don't blame you at all for feeling that it was forced. (On the DVDs, James Marsters straight-up says, "They tried to make me Cordelia, and it didn't work.")
But I've always been confused by the idea that character development is a bad thing, that a character should stay static and not develop. I mean, Lilah Morgan (my personal fav on Angel) is a villain but becomes incredibly complex. If they'd tried to redeem her, I wouldn't have complained; I would have thought it was fantastic (they sort of did the same thing with Darla, who I also love).
I'm also really confused by the idea that they had to come up with stuff for him to do. That's what they have to do with every character. Again, I agree that the chip didn't work well in S4, and I'm not so crazy about the pawn of the First in S7, but other than that, I think it all works really, really well.
When I was watching OOMM for the first time and Spike has the whole "I love you" dream, I went: "DUH!" And smacked my forehead. Because of course Spike's in love with Buffy. Of course he is. It made perfect sense to me. So I didn't see that as grasping for straws at all. It fit perfectly into my view of who he is, that he would fall in love with this strong, smart, fierce woman who is so far "above him." That's what he does. He needs someone to latch onto in that way.
I also disagree about S7, a season that has so many ridiculous weaknesses (but that I love since I love the characterizations of Buffy and Spike and Dawn and Anya). Why is it Spike's fault if the writers had run out of things to do with Xander by that time (I can think of a number of things they could have done with him, but they never asked me) but still had things to say about Spike? I think if you go back and watch the season, you'll see that Spike actually doesn't take over the show: he doesn't have more scenes than anyone else; his writing is just better.
his final sacrifice to stop the uber-vamps was his final "look how cool I am" speech. And again, I disagree: to me it's: "Look how far I've come." With Spike, the important word is "Real." Very little that he has or is is actually real. So that's why "Intervention" is so important; that's why the soul is so important; that's why "Chosen" is so important: because finally, finally he's found something that's real and his. After searching for it since he was alive. He thought it was what Drusilla offered him in the stable in 1880, but that wasn't it. "Chosen" was it. There's a reason that he's (literally) "effulgent" in that scene.
