I think you're going to have to try and adopt a philosophical attitude and don't let his bad behaviour get under your skin. You're never going to change him, so it's a matter of making the best of things I think, you can allow it to get to you, or you can accept that it's the way it is and let it go.
All the best
I agree with mojo, you will have to accept that his behaviour won't change, but I'd also be inclined to sometimes simply be selfish for your own sake. If you pick him up, and he starts complaining, turn rounnd and drive him back to his own house if he won't stop. Yes, he'll complain, but from the sound of it, he would anyway, and if you do that and leave him there a couple of times, he might get used to the idea that you coming to get him is something he, and you should be enjoying.
Of course, you're both right......
I just find it so tough that the fella, that for all your life, you've looked up etc can be like this and be so [censored] difficult to get along with.