Garp has never faced Luffy with the intention of harming him, and he still doesn't when he faces Luffy at Marineford. His grandson by blood is in front of him, his grandson by promise behind him, and they are equally precious to him. Garp does the only thing he can do—close his eyes mid-punch and believe in Luffy's will, yet despite it all, he still loses one grandson, almost loses the other as well.
At Foosha Village, Garp lets Dadan pummel him into the ground. He can still see Ace's smile right before he died, Luffy's utterly stricken expression afterward, and he remembers his own inability to help create the outcome he wanted more than almost anyone else.
Makino stands up for him, but Garp knows he deserves being the target of Dadan's helpless anger. What's the use of being a hero of the marines when he can't even protect his own family?