Good points.I agree with @Priceless that Xander already found a family in Willow. I don't think there was a Scoobies to join in Welcome to the Hellmouth and the Harvest, Xander was fighting with Buffy and Giles to save Jesse, but after that, seems that he and Willow joined because it was the right thing to do and because they both have found a purpose, and Buffy was their friend.
I would say that as years go on, Willow and Buffy became Xander's true family. He sums it up when he tells Buffy in Seeing Red, "I don't know what I'd do without you [Buffy] and Will." Girlfriends come and go, Giles rarely showed interest in Xander, and Dawn was closer to Tara at that point. Buffy and Willow were always there for him and always showed interest in him and his life. They're his real family.