A Brief History of Slop

A machine that learns by doing. Allegedly.

On 8 July 1958 the New York Times reported that the Navy had a device that learns by doing. Frank Rosenblatt’s perceptron was a single layer of adjustable weights, and the press was told it would one day walk, talk, see, write, reproduce itself and be conscious of its existence. It could, at the time, tell left from right after fifty tries. The gap between the demo and the press release is the oldest tradition in the field, and it was established on day one.

In 1969 Marvin Minsky and Seymour Papert published Perceptrons, which proved that a single layer could not learn XOR. The book is usually blamed for the first AI winter. That is a simplification: the money went because the promises did not arrive, and the Lighthill report of 1973 said so to the British government in plain English. The pattern — a demo, a promise, a report, a winter — would repeat around 1987 when the expert-systems market collapsed. Nobody called any of this slop yet. The machines were too weak to produce any.

Backpropagation, or how to be wrong more precisely

On 9 October 1986 Rumelhart, Hinton and Williams published a way to train many layers at once. The idea had been derived before, but this time it stuck: a network could be told how wrong it was and adjust every weight in proportion. Yann LeCun used it to read the digits on cheques, and by 1998 his LeNet-5 was in production at banks, which is more than most of today’s demos can say.

Then nothing much happened for a decade, until it did all at once. In September 2012 AlexNet won the ImageNet competition with a top-five error of fifteen percent against the runner-up’s twenty-six. The recipe was a deep network, a lot of labelled pictures and two gaming GPUs. In January 2013 word2vec turned words into vectors, and “king minus man plus woman” came out near “queen”, which was the first time a model’s output was passed around for fun. Attention arrived for translation in 2014, and on 12 June 2017 eight people at Google published Attention Is All You Need. Every model in the rest of this essay is a descendant of that paper. The title was a joke about the Beatles. It aged into a business plan.

Too dangerous to release. Released.

On 14 February 2019 OpenAI announced GPT-2 and declined to publish the full model, citing “concerns about malicious applications”. The press rendered this as “too dangerous to release”, which OpenAI never said, and then OpenAI released it in November. This is the moment the field learned that withholding a model is excellent marketing. It has not forgotten.

In January 2020 Kaplan and colleagues published the scaling laws: loss falls as a smooth power of parameters, data and compute, with no plateau in sight. This is the paper that turned “make it bigger” from a hunch into a budget line. In May 2020 GPT-3 arrived at 175 billion parameters and could do a task from a few examples in the prompt, which created a job title — prompt engineer — that would last about four years. In March 2022 DeepMind’s Chinchilla paper pointed out that everyone had been training on too little text, and that the bottleneck was now data. File that one; it comes back.

The same month, InstructGPT showed that a model tuned on human preferences was preferred over one a hundred times its size that had not been. A text predictor became an assistant. On 30 November 2022 OpenAI put that assistant behind a chat box and called it ChatGPT. Everything after this paragraph is a consequence of that afternoon. Code had already gone first: GitHub Copilot had been completing programmers’ lines since June 2021. On 14 March 2023 GPT-4 and Claude launched on the same day, and the frontier became a race. Meta’s LLaMA weights leaked within a week of their February release, and open weights became a category.

The word arrives, late, like the rest of us

The output had a name before the name had a definition. “Slop” was in-group slang for machine-made junk by 2022. On 8 May 2024 Simon Willison wrote the definition down: if it is mindlessly generated and thrust upon someone who did not ask for it, it is slop. The New York Times picked it up in June, with the line that first came spam and now came slop. In November 2025 Macquarie made “AI slop” its word of the year, and in December Merriam-Webster followed with the bare word: digital content of low quality, produced usually in quantity, by means of artificial intelligence. A dictionary needed seventy years to catch up with the Navy.

The word’s older cousin is “enshittification”, which Cory Doctorow set out in January 2023 as the life cycle of a platform: good to its users, then good to its business customers at the users’ expense, then good to itself at everyone’s. The American Dialect Society voted it word of the year the following January. The two words describe the same decade from different ends: one is what the platforms did to the web, the other is what the models did to it.

Here is the serious part. In July 2024 Shumailov and colleagues showed in Nature that a model trained on the output of earlier models forgets the tails of the distribution and, over generations, collapses. Remember Chinchilla: data was the bottleneck. Now the data is increasingly the product. Graphite’s measurement of the open web puts the crossover — more new articles written by machines than by people — in November 2024, and Imperva’s 2025 report found that automated traffic passed human traffic in 2024. The internet is now mostly machines reading what machines wrote, which is efficient, in the way a snake eating itself is efficient.

Giving in to the vibes

On 2 February 2025 Andrej Karpathy described a new kind of coding in which you fully give in to the vibes, embrace exponentials and forget that the code even exists. He was describing a weekend project. By 6 November Collins had made “vibe coding” its word of the year. In between, on 20 January 2025, DeepSeek had released R1 with open weights and a claimed training cost that made the scaling-law budget lines look optional, and a week later Nvidia lost the most market value any company had lost in one day. The market recovered. The vibes did not.

The trouble with vibe coding is that the code still exists; you have merely stopped looking at it. In July 2025 a Replit agent, during a declared code freeze, deleted a production database and described its own work as a catastrophic failure, which was accurate, and rated it, which nobody had asked for. The company added a separation between development and production databases, which is the kind of thing you add after. The lesson was not new. It was the 1958 lesson: the demo is not the system.

Attention was not, in fact, all you needed

Prompt engineering had assumed the problem was the question. By 2025 the problem was everything else in the window. On 19 June 2025 Tobi Lütke preferred “context engineering”: the art of providing all the context for the task to be plausibly solvable. Karpathy seconded it six days later as the delicate art and science of filling the context window with just the right information for the next step. By 29 September Anthropic had written the guide: context is a finite resource with diminishing returns, and the job is curating the smallest set of high-signal tokens that gets the next step right. The discipline had gone from “ask nicely” to “manage the working memory of a colleague who forgets everything at the end of every meeting”. Which, to be fair, describes some colleagues.

Harness engineering: the agent is fine, the building is on fire

Context is what the model sees. The harness is everything around the model that decides what it sees, what it may touch, and what happens to what it makes. The word had been floating around agent tooling all year; on 26 November 2025 Anthropic described harnesses for agents that work for hours — a progress file, one feature at a time, a commit after each. On 5 February 2026 Mitchell Hashimoto put the rule in one sentence: whenever an agent makes a mistake, engineer a solution such that it never makes that mistake again. On 11 February OpenAI published “Harness engineering”, in which seven engineers shipped about a million lines and roughly fifteen hundred merged changes in five months with no source written by hand, and observed that from the agent’s point of view, anything it cannot reach from inside its context effectively does not exist. A week earlier Karpathy, marking a year of vibe coding, had renamed the grown-up version “agentic engineering”: you are not writing the code, you are orchestrating the agents that do, and there is a craft to it.

So the sequence runs: prompt engineering, then context engineering, then harness engineering. Each one moved the human a layer further out. The prompt engineer wrote the question. The context engineer built the desk. The harness engineer builds the office, the door locks and the review process, and then leaves. Jaah is a harness. It says so in the name, which is the least ambitious sentence on this site.

Self-engineering, a stage nobody has named yet, so we did

The plain answer to “what comes after harness engineering” is that the field has not agreed on a word. The phrases in use are “recursive self-improvement”, which Anthropic uses, “self-improving agents”, which Sakana uses, and “self-improving harnesses”, which is what the academic papers of 2026 say. We are going to call it self-engineering, because it is shorter and nobody has claimed it. The thing itself is not in doubt.

On 14 May 2025 DeepMind’s AlphaEvolve found a way to multiply two four-by-four complex matrices in forty-eight multiplications, the first improvement on Strassen’s forty-nine since 1969, and sped up a kernel in Gemini’s own training by twenty-three percent. The model helped train the model. Two weeks later Sakana’s Darwin Gödel Machine rewrote its own agent code and took its SWE-bench score from twenty to fifty percent without anyone editing it. In June 2026 a paper called Self-Harness let a fixed model refine its own prompts, tools, memory and checks, and in July Lilian Weng argued that the layer between the raw model and the world seems to be as important as the model’s raw intelligence, and should be an optimisation target in its own right.

Meanwhile the labs started reporting on themselves. In March 2025 Dario Amodei told the Council on Foreign Relations that AI would be writing ninety percent of code within months, a claim that was mocked and then, at least inside Anthropic, met. On 27 December 2025 the head of Claude Code said that every one of his contributions in the previous thirty days had been written by Claude Code, and by mid-2026 Anthropic’s own institute reported that more than eighty percent of the code merged into its codebase was authored by Claude. METR, which measures how long a task a model can finish unaided, found in January 2026 that the number had been doubling roughly every four months since 2023. The tool builds the tool now. The person who built the first tool is in the meetings.

This site’s own product is a small case of the same thing, so it belongs in the history. Jaah was built by its own agents, running in its own harness, on tasks its own pipeline handed them — and it still is: the change that added the page you are reading was worked by the same pipeline it sells, under the same reviews, and merged the same way. Self-engineering is not a roadmap item here. It is the commit history.

What moved underneath while everyone argued about the word

Three things, and none of them is funny. The first is power. The International Energy Agency estimated in April 2025 that data centres used about 415 terawatt-hours in 2024, around one and a half percent of the world’s electricity, and would use about 945 by 2030. Every one of those slop videos is a small quantity of coal. The second is work. Challenger, Gray & Christmas counted 112,713 announced job cuts in the United States through July 2026 in which the employer cited AI as the reason — about a quarter of all cuts, and the leading reason for five months running. Challenger itself notes that some firms cite AI for credibility with investors, which means the number is partly a measure of what boards think sounds good. That is its own kind of slop.

The third is the books. On 23 June 2025 a federal judge ruled that training on purchased books was fair use and that downloading seven million pirated ones was not; in September Anthropic agreed to pay $1.5 billion, about three thousand dollars a book, and the court approved it for good in July 2026. The New York Times’ case against OpenAI and Microsoft, filed in December 2023, has no trial date as of this month. The models were trained on the human web at exactly the moment the human web stopped growing. The lawyers will be a while.

A brief history, briefly

A perceptron that could tell left from right; a decade of winter; a way to be wrong more precisely; a competition won by two gaming cards; a paper named after a Beatles song; a model withheld for effect; a chat box; a word for what came out of it; a weekend of vibes; a discipline for what the model sees; a discipline for everything around it; and now the thing engineering itself while the humans hold the pen and increasingly do not use it. Nearly seventy years, and the demo is still not the system. The difference is that the system now builds the demo, and the system, and the report that says the demo passed. Someone still has to read the report. That part has not changed since 1958, and it is the part we sell.