People have already touched upon how the show started to play a little fast and loose with magic but I wanted to just add that, in my mind, a big problem with magic in the show comes from the Season 6 magic-as-drugs metaphor, which just did not work at all.
Until Season 6, magic had been shown to be a good thing, a weapon to be utilized when the Slayer needed. While Willow's thirst for power and recklessness with magic was already firmly established, the show never portrayed magic itself as a bad thing, as something destructive and "evil" ... until Season 6, with the introduction of Rack and the completely unsubtle magic-is-a-drug metaphor. We literally see Amy and Willow use magic to get high, the way one would do with illicit drugs, and it didn't work because we'd never seen magic used this way before and never saw it function that way again after Season 6.
Additionally, the magic-as-drugs metaphor didn't work because magic had proved to be helpful in many cases before. Buffy could not have defeated Adam or Glory without magic and there were many smaller battles in which Willow's magic proved useful and good. But illicit drugs don't work that way. There is never a point where illicit drugs are "good" or "helpful". You can't use x as a metaphor for drugs if x has been shown to be "good" at some point, because then your metaphor collapses, as it does in BtVS, when Willow is shown in Season 7 to have to learn to control her magic, rather than not use it. In what world would we ever tell a recovering addict to "control" their addiction, rather than give it up completely.
To see how Willow's magic-as-drugs metaphor really falls short, compare it to Supernatural and the storyline involving Sam drinking demon's blood. Supernatural does the metaphor right, showing how, even though the blood makes Sam more powerful and makes him feel better and gives him a "high", it is dangerous and ultimately leads Sam down a very dark path, which culminates in him accidentally freeing Lucifer from his cage. The demon blood is never portrayed in a positive light and after Season 4 Sam is forced to quit cold-turkey, and the series shows him suffering the withdrawal and shows how the addiction never leaves.
Had BtVS gone with power as the addiction, rather than magic itself, it would have worked better and, in my mind, would have allowed for magic within the 'Verse to still function as a force of good and a weapon to be used.