Is that wrong?

There's this guy at work who's a really nice man. He's also really annoying. I mean really annoying. Every morning, he insists on not just doing the "good mornings," but also going through this whole rigmarole about how we're doing, and the state of the weather. Then, after lunch, the same thing! I mean, how much does he think changes during the hour I'm at lunch? He also says "libary" instead of library, which I cannot stand, and he talks really loudly on the phone at least once a day to either his wife or his mother, sometimes both.
Now I know that this man is a good person, so I should like him. But I can't. He's just too annoying. Does this make me a bad person?