Maybe the problem that Joss had with Angel is that he is just too typical, for want of a better word. Okay, granted, he's a vampire, and, for the most of the time, the only ensouled vampire in Whedon's universe, but beyond that, his story, specially on his own show, is pure cliche'. He's the tall, dark,handsome hero with the shady past and the tortured conscience, trying to atone for his past sins by becoming the self appointed champion of the little people. Other than the fact that he's a vampire who mostly fights demons, and he could be any P I or cop who's ever been in movies, on TV, or the subject of Harlequin Romance Novels. There's nothing there that hasn't been done a few dozen times with "only the names changed to protect the innocent." Even his crew fits the standard mold.
There's the airhead secretary who has far more depth to her character than is at first revealed, and who, as time goes by, develops an ambiguous relationship with the hero. (Do they, or don't they? The viewer has to guess, because it's never actually revealed)
Then we have the stick-in-the-mud assistant who's usually a former college professor or scientist and a treasure trove of pertinent information necessary to the hero's investigations. Only later do we discover that Ol Prof is also a kick ass fighter who begins to dress shabbily and doesn't shave as often as he should.
The ex street punk who's working with the hero, not so much out of friendship (at least at first), but from a burning desire to get back at the criminals who killed his baby sister/parents/wife. He generally starts out as more or less just additional muscle, but somewhere along the way becomes a more important member of the gang.
At some point in the story, the hero's ex shows up to announce that they actually have a son...
That, I think, is a large part of Joss' difficulty in "Angel." Joss likes to work on things that take the cliche' and rip it to shreds, like Buffy, the cute little blond cheerleader who actually goes out and pulverizes the serial killers and other bad guys, or Mal Reynolds, who lives in a space ship but livs by the code of the Old West.
With Angel, take away his fangs, and he's Mike Hammer, Thomas Magnum, T J Hooker, or any other character. Boring. Joss just doesn't do boring very well.