Distilling the Distillation Debate

What Jensen Huang calls learning, the White House calls theft. Both miss the point.

photo: Igor Omilaev

On July 22, Treasury Secretary Scott Bessent told Fox Business the government was considering sanctions on Chinese AI labs for stealing from American companies. He pointed to what he called “watermarks” of U.S. models turning up inside Chinese LLMs. The training of AI models on the outputs from other models, a practice known as distillation, has emerged as one of the most pressing public conversations, balancing the demands of AI proliferation with the need for public policy.

The following day, Axios published its interview with Nvidia CEO Jensen Huang, whose perspective sharply contrasts the administration’s position. “Distillation - learning from AI, learning from other sources of knowledge, is fundamental to intelligence,” he asserted. According to Huang, we are constantly learning from one another. AI also has to learn from something.

These two worldviews map the newest battle lines being drawn between tech leaders and politicians. It’s a polemical that has taken center stage in the global town square, colloquially now known as the the conflict between “doomers” - those who believe we have sealed our extinction with the advent of AI, and “zoomers” - AI evangelists who believe we are on a path to superhuman capabilities.

Neither is arguing from neutral ground. Most media coverage characterizes one side as principled and the other as self-interested. Both are both. Nvidia sells more chips whichever way this goes. Anthropic and OpenAI lose ground, if not their entire revenue models, if their LLMs can be mined for free.

Strip away the war over narrative and framing and the essence of the argument becomes clear. In February, Anthropic published its own account of three labs: DeepSeek, Moonshot, and MiniMax running fraudulent accounts against Claude at industrial scale: over 16 million exchanges, roughly 24,000 accounts, architecture built so that banning one changes nothing. Neither side openly disputed what happened. The broader public policy dispute is over the implications.

Huang’s position on distillation may be valid, but it’s beside the point. Humans learn from teachers, from books, from each other, and every one of those sources has a name on it, a lineage one could trace. What Anthropic described isn’t learning from a source. It’s learning from a source with the source itself erased: rotating accounts, proxy networks, built specifically so the exchange can’t be traced back to anyone. I made a version of this argument in July, writing about AI hallucinations in the judicial system, in “What AI Still Can’t Tell Us.” The fix isn’t policing what a model says after the fact; it’s governing what it learns from and trains on in the first place. The real issue inside the “open versus closed” battle is whether you can reliably track where knowledge or information came from.

An unverified citation and an untraceable capability are the same system failure wrapped in different packages. One shows up as a fabricated case citation in a legal brief. The other shows up as a model that performs like Claude with no reasoning to show why it can, via its own work or weights. Provenance isn’t a tax on learning. It’s the difference between learning and laundering.

I don’t think the solution is banning distillation. Nor is it useful to simply shrug off the issue as inevitable. The fix is building for traceability at the point where capability is sourced, so that when something trains on something else, you can trust what it learned and know where the knowledge came from. That’s a higher standard than whether a model is open or closed. It’s also the only one that ultimately survives the argument.

- S