..and i do think that is the entire point of his character.
Its important to remember that characters in a story aren't just supposed to mimic real people, they are supposed to be reflections on ideals and narratives.
Garp is from the old generation. A generation that has seen the change the new pirate brought about. He has also bared witness to the atrocities that higher ups in the World Government have done (and in turn, the crimes the marines let then get away with)
Despite this he has stayed in the marines, stating the reason for this choice was that someone has to protect the marines. Which is unfortunately a choice that will have consequences.
He places the fate of the marines on the shoulders of the younger generation, because he knows he won't be the one to lead them. S.W.O.R.D is a good example of people who he personally believes can change the marines for the better, and its a part of why he won't let go of his ideals.
Garp does believe in change, I don't doubt that. I don't think he wants things to stay the same. But follows the mindset that the change must be slow and steady, or else it will turn violent and end in bloodshed. He thinks it's better to follow the rules and make change from inside the system rather than break it down and rebuild.
Ironically, it reflects the boomers mindset that a lot of the older generations have in real life. There may be that one old grandparent who doesn't really agree with the state of the world and is even disgusted by it. But they still go along with it and don’t think you should go too far against it.
I know it well enough because I have people in my life like that. And it's frustrating because you can't change their mind. They love you and you know it, but you won't ever be on the same page about how to go about things. All you can do is just leave them behind, and move forward.
However, Garp can recognise the flaw in his mindset. He is too stubborn to change it, and still stick by it, but he knows if it keeps going on like this, he will lose people.
That's why he gives Dragon the keys and sets him free. He doesn't argue with him over Dragon quiting the Marines nor saying that he hates him. Garp knows he has every right to, and is okay with it. Its not him being uncaring, its him understanding exactly where Dragon is coming from.
In the end, there is a valid reason to hate Garp and there is a valid reason to love him. He is neither a wholly good nor wholly bad person. There is a reason people in story trust him as much as they do though.