I'd say you've got it right Five, but there are others who would disagree. Not so much about the books not being canon. There's never been any kind of credible reason for considering them such. Joss doesn't read them. He doesn't contribute to them. There's never been any kind of respected voice referring to them as canon, I've heard.
There is good reason to think of ToTS, and TotV as canon though. Joss may have had some sort of supervisoral role, and pieces of them have carried on into the canon Buffy 8 implying Joss's authorization. The wider spectrum of fandom appears to have accepted them as canon. Nevertheless I've heard at least one person here who doesn't accept them as canon, and he argues a decent point. If Joss does not give them specific Canon status, this person does not consider them canon.
When Buffy 8 started up, the issue of what is, or isn't canon appeared more clear than it is today. If Joss said it was, then it was. Since that time however there's some question as to what exactly it is he puts the canonical stamp of approval onto, so now it's a little trickier.