I think anyone raised by Holtz, in some god forsaken dimension, is going to grow up with one hell of a grudge against demons, including his 'blood' father. So it was no surprise to me that Connor behaved how he did. Why would he disregard everything that Holtz had told him over the years, and suddenly believe what these strangers where telling him.
I think I only disliked him, because as a viewer, I had the advantage of knowing what really happened. At the end of the day Connor had been raised by Holtz, and he was the only father he'd ever known, so he was bound to favour him over Angel.
That's what I found so annoying, I wanted to shout at Connor, 'can't you see you've been lied to, can't you see Angel is not the bad guy here'
I guess that means that Vincent played the part well though, because I did really dislike him, but then in Season 5, when his past had been erased and replaced with a different one, a 'normal' one, he was really likeable. If he had been like that in Season 4, then I feel it would of been out of place. Season 4 Connor needed to be this angry, confused, and lonely kid.