The Dream of the Serpent
AI solved an open math problem and outperformed a world-class engineer. The process looked a lot like genius.
In 1865, the German chemist August Kekulé was stuck. He’d spent years trying to work out the structure of benzene — a molecule that chemists knew the ingredients of but couldn’t figure out the shape. Carbon and hydrogen atoms in the right quantities, but no arrangement anyone proposed could explain how they held together. Kekulé had been turning the problem over for so long it followed him into his sleep.
As the story goes, one evening he dozed off by the fire and dreamed of atoms dancing and twisting into chains. Then one of the chains curled around and seized its own tail — a serpent forming a ring. He woke up and spent the rest of the night working out the implications. Benzene wasn’t a chain. It was a ring. The ouroboros — the ancient image of a serpent eating itself — had revealed one of the foundational structures of organic chemistry.
It’s a great story, and it’s the kind of story we love to tell about problem-solving — the answer arrives in a flash, delivered by the subconscious as a gift, genius as something that happens to you rather than something you grind out. We conveniently forget that Kekulé had been grinding for years before the dream arrived.
The common criticism of AI follows a similar logic. AI is just autocomplete — it can remix what it’s seen, predict the next likely word, maybe even sound convincing, but it can’t solve anything new. It doesn’t have insight. It can’t have that Kekulé moment where the pieces suddenly fall into place. It’s a pattern-matching engine, not a mind that can discover.
It’s a reasonable position. But last week, two things happened that made me wonder whether we’ve been too sure about what “discovery” requires.
The Mathematician
Donald Knuth is not the kind of person who gets excited about AI. A Turing Award winner and the author of The Art of Computer Programming — the textbook I used in my own computer science program in college, and still the definitive reference in the field — Knuth is often called the father of algorithm analysis. He has been openly skeptical about the productivity of large language models.
So when Knuth published a note on Stanford’s website last week, what he described sounded almost like a Kekulé moment — for a machine.
Knuth had been stuck for weeks on an open problem from his textbook: finding paths through a three-dimensional grid of points where each path visits every point exactly once, returns to its start, and shares no connections with the other paths. He’d solved it for small grids but couldn’t crack the general case. His friend fed the problem to Claude, Anthropic’s AI model, and what came back wasn’t brute force. Claude recognized that the grid had a deeper mathematical structure — a kind of symmetry that Knuth’s original framing hadn’t exploited. It invented a way to decompose the grid into layers, then built serpentine patterns that wove through each layer in a way that satisfied all the constraints simultaneously. It reframed the problem, then solved the reframed version.
Knuth verified the mathematics himself and called the approach “remarkably creative.” Not just correct — creative. “It seems that I’ll have to revise my opinions about ‘generative AI’ one of these days,” he wrote.
Now here’s the thing: Claude didn’t arrive at that solution in a flash. It worked through the problem over about an hour, trying different approaches, hitting dead ends, adjusting its framing each time. The structural insight that cracked the problem — the decomposition into layers, the serpentine patterns — emerged from that process of trying and revising. Not randomly, but not in a single leap either. More like a mind circling a problem, getting closer with each pass, until the right angle of approach comes into focus.
Which sounds a lot like what Kekulé’s subconscious was probably doing while he slept — running variations on carbon structures, testing arrangements against what he knew about molecular bonding, until one arrangement clicked and surfaced as a dream about a serpent.
The Engineer
The same week, Andrej Karpathy — a founding member of OpenAI and former head of AI at Tesla — ran a different experiment.
Karpathy maintains nanochat, an open-source project for training small language models from scratch. Think of it as a workbench for building AI — minimal, transparent, designed for tinkering. He’d spent months hand-tuning the system’s internal settings to squeeze out better performance, and he thought the project was well-optimized.
Then he gave an AI agent his code with a simple instruction — make it better — and left it running for two days.
What came back wasn’t just a list of minor tweaks. The agent noticed that a key part of the model’s attention mechanism was missing a component, making it too diffuse — a subtle architectural oversight, not an obvious bug. It found that certain parts of the model needed regularization Karpathy hadn’t applied, that settings he’d forgotten to tune were working against each other, and that the initialization values were off. Twenty real improvements in all, each one something Karpathy — after months of manual work — had missed.
Stacked together, they produced an 11% gain on the project’s key benchmark. The commit message reads: “All of these improvements were developed by Claude running autonomously. I didn’t touch anything — incredible.”
“I am mildly surprised,” Karpathy wrote, “that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project.” From Karpathy, “mildly surprised” is the equivalent of shock.
But there’s something in the Karpathy case that goes beyond the Knuth story. Knuth’s problem was mathematics — AI solving a puzzle for humans. Karpathy’s problem was AI itself. An AI agent improving the process by which AI gets built. The improvements it found will feed into the next generation of models, which will be better at finding the next round of improvements.
“All LLM frontier labs will do this,” Karpathy wrote. “It’s the final boss battle.”
The Serpent Eats Its Tail
I argued a while back that what we call creativity is really search through a space of possibilities — and that the line between searching and creating is blurrier than we’d like to admit. Knuth’s case and Karpathy’s case didn’t settle that question, but they sharpened it. When an AI reframes a problem that stumped a Turing Award winner, or spots architectural flaws that one of the world’s best engineers missed, something is happening that doesn’t fit neatly into the “just autocomplete” box.
The Karpathy case is the one I keep thinking about, because of where it points. When AI solves a math problem, it’s doing something for us. When AI improves the system that builds AI — and those improvements feed into the next round — the loop starts to close. I wrote a few months ago about early signs of this loop forming. Karpathy’s result suggests it’s forming faster than expected. And once an AI that improves AI training produces a better AI that’s even better at improving AI training — well, that’s the serpent eating its own tail. Kekulé would recognize the shape.
Related
