Lucifer doesn't kill Gabriel because he's a threat—he kills him because of what he did to Sam.
After months of sifting through Sam's troubled dreams, he knows exactly why Sam hates falling asleep on Monday night. It's not even because of Lucifer. It's because he's afraid of the day that follows. He's always on edge, always waiting for his brother to be gone—but if Dean notices Sam's weekly clinginess, he doesn't say anything. Sam always stays close on Tuesdays. Always.
Even still, Sam is haunted by those months—months he lived, months he endured and aged and somehow survived. But he didn't come out of it unscathed; to this day, Sam isn't sure what's real.
Some days he wakes up, wondering if any of this is real. Maybe the Apocalypse is fake. Maybe the reason Gabriel so easily popped them back into TV Land is because Sam never left the nightmare, and maybe Cas was able to save them because Cas could do whatever Gabriel wanted him to. Maybe Cas isn't real at all.
Maybe neither is Lucifer.
Maybe Lucifer escaped because Gabriel wanted him to escape, or wanted Sam to think he did.
Maybe angels aren't real, and maybe Heaven isn't either. Maybe Dean never went to Hell, and this is all some fucked up nightmare.
Sam wants to be sure, but he never can be. It's been almost two years, and some mornings he still wakes up and isn't sure if he himself is real at all.
Lucifer saw this torture and he couldn't help but hate—hate what Gabriel had done to Sam, hate what Gabriel had become in the wake of Lucifer's Fall. He was no longer the laughing little brother, eyes bright with mischief. Now he was only a fallen, false god, his eyes dark with gleeful malice.
He had taken that malice out on Sam, and that was unforgivable.
Lucifer doesn't want to kill Gabriel, but when the time comes and all the other Pagan gods lay slaughtered, he looks at his brother and no longer sees an Archangel of the Lord. He sees one more Pagan whose longevity had turned him corrupt and foul.
He regrets, but only in witnessing Gabriel's broken Grace escape his empty shell. He does not regret what he's done or why, but he mourns for the beloved brother lost long ago, smothered to death under the weight of a Trickster's cruelty. He mourns for the child-angel that once splashed in the bubbling brooks of Earth, long before humanity was even a thought.
Lucifer hates, because his brother would never have Fallen so far if it weren't for the humans. Gabriel had taken the punishment of the wicked upon himself, and in doing so, he himself had turned wicked.
Lucifer hates, because if it weren't for one small, weak, perfect human, his brother may never have died in the first place.