Lol yes I'm the same, I can listen for so long and then the cumulative annoyances start to stack up and eventually I'm not agreeing with anything the reviewers say. 🙂 But if I leave the podcast for a while, then go back after a month or two I can enjoy it again.
My favourite Buffy podcasts are Once More With Podcast (of curse) Buffering, Return to the Hellmouth, Rewatchable, Hot Chicks With Superpowers and I used to like Babes Watch Buffy but they've not done a Buffy episode since July when they finished Season 3.
Hope that list doesn't make me sound too obsessed 😀 What do you listen to?