Remember that she'd known Xander since they were toddlers, and, while she said once that they hadn't always been close, both of them state that they've been best friends their whole lives.
Now, it may have been that Xander was her first genuine crush, but then consider this:
Who was there to support her when she broke the yellow crayon in kindergarten?
Who used to call her every night to talk about what went on today in school?
It's entirely possible that her crush on Xander was just the same response to being treated as if she mattered (one might even suggest that that's a large part of her friendship with Buffy, as well,) and that, if Oz or someone else had come along sooner, Willow might have given up her unrequited crush on Xander years before.
Edited to add: While Xander is studiously indifferent to her romantically since we first see them, we know from things Willow said that's not always been entirely the case, nor does he ever really ignore her in the same way that her parents and the other students do (with the exception of "The Pack.")
