He's forgotten what Oz looks like.
Gilbert's only distantly aware that he's dropped his mug, of the hot coffee splashing onto his desk and the map he was trying to memorise. He can't have forgotten, everything he's done in the three years since Oz was dropped into the Abyss was for Oz, there's no way he could have forgotten —
Except when he tries to remember Oz's face, all he gets is a smudge of a face beneath a mop of blond hair, like an overexposed photograph. He clutches his head, trying to force himself to remember, to fix Oz into his memory where he'd been so sure he was.
There's a photograph of Oz with Oscar and Ada that Ada has taken to school with her, Ada hoisted against Oscar's chest, Oscar's other hand resting on Oz's shoulder. Gilbert remembers it being taken, but he can't remember their faces in the result. They're gone, as thoroughly as if he'd burned them out with a cigarette.
Trying to remember feels like he's holding the cigarette to it longer, burning it away faster, and his breathing stutters with panic.
Gilbert curls his hand against the scar carved down his chest, and there — there it is, a flash of memory, Oz's face as the sword came down on Gilbert instead, face slack with horror at what he'd done. It's a frozen moment, but it's enough that Gilbert can draw breath again. He's not forgotten. It was a momentary lapse, one that can be fixed by talking to Ada again, or chasing Break about Pandora's progress.
It won't happen again. He's doing all of this for Oz, after all.
(A few years later Break asks him if he remembers Oz, popping a boiled sweet into his mouth. Gilbert stares at him, too tired for panic, and asks why the hell Break cares. Break just smiles, looking almost resigned, and shatters the candy between his teeth.)