Yeah I've struggled a lot with this scene, trying to figure out what was really going on-- as it's said over and over in the series by Buffy and others that she would never kill a human. She refrains repeatedly from-- or tries to delay and any cost, killing humans even when they put the lives of her loved ones in danger. The party line is always made very clear, very black and white: humans, can't touch. In fact, beings with a soul are always off limits. (Furthermore she won't harm those who could be dangerous but are incapable of doing harm at present). And though she tells everyone at the end of "The Gift" that she'll kill anyone who comes near Dawn, we have no way of knowing whether she actually would. In fact, there is evidence that she wouldn't: she doesn't even kill Ben.
So here's my take on the situation: Buffy doesn't consider the Knights to be human. It's the only way in which the series doesn't completely trip all over itself in this regard, and it can work. When she first encounters them in the alley, she lets them go. At that point, there's no reason for her to think they're anything other than human beings. But by "Spiral," she has more information. They're looking for the key, which is older than time, and they're after Glory. She might have assumed, (as I did), that the knights actually either a)came from another dimension, through a portal, to stop Glory or b)traveled through time from Byzantium itself to stop Glory, or c)have existed since 600BC -ish when Byzantium was around, and are trying to stop Glory. I think C is most likely the case.
Considering all of the above scenarios, it could be argued that the Knights are not human. If Buffy assumed that they came from another dimension or traveled through time she could have also supposed, especially in the heat of the moment, that they were some other type of being that could do so easily--the fact that they'd have to have had the power to see into the future or across dimensions would support the theory that they aren't humans. Most of all, if they are truly Knights of Byzantium--ie the same knights of Byzantium that existed when Byzantium did, (as opposed to some descendants taking up the cause) then they are clearly not humans .
Of course this is all my take on the situation and it allows me to watch season 5 without getting frustrated at the complete shift in principle sans explanation. To clarify: if Buffy decides she wants to start killing humans who put the lives of her loved ones in danger, that's ok by me. But why now? That has to be addressed. It's something she'd never done before, (if you don't count the circumstances surrounding Faith which I don't because I don't believe Buffy's intent at any point was to kill her), and since she had gone out of her way to make this distinction numerous times, there should be some sort of discussion or follow through if she'd decided to handle things differently. Moreover Giles would have to be a complete idiot for announcing a few episodes later that "Buffy could never kill a human because she's a hero" since he was fully aware that she killed some Knights. And I don't think that's the case.