From Algebra to Agents: 1,200 Years of Large Language Models

A man, a book, and a finger

In 1948, at Bell Labs, a man generated text by hand. Everything since has been the story of removing the constraint that stopped him.

Claude Shannon wanted to show that English could be approximated by counting. So he opened a book at random, put his finger on a letter, wrote it down, opened the book to another page, read until he found that letter again, and wrote down the one after it. Then he did it again. And again, until he had a paragraph.

That is autoregressive sampling — predict the next symbol from the ones before it, then feed your prediction back in and predict again. It is exactly what the machine on your phone is doing when it finishes your sentence. Shannon did it with a book and a finger, and he stopped where he did because, in his own words, the labour becomes enormous at the next stage.

Below are the six samples he produced, reproduced character for character from the paper. Move through them and watch English assemble itself out of nothing but counting. Nobody taught the process grammar. Nobody gave it a dictionary. Each step simply looks at a little more of what came before.

By the last one there are ten consecutive words a person could have written. Shannon noticed, and wrote the sentence that the next seventy-five years would spend proving:

His statistics, incidentally, came from frequency tables in a 1939 popular book about codebreaking and a 1923 phonetics study — cited sources, not a training corpus in anything like the modern sense. And that constraint, the labour becoming enormous at the next stage, is the only thing standing between his paragraph and yours. Removing it took electricity, three wars, two academic winters, a translation problem at Google, and about four hundred billion dollars.

This is the story of removing it.

Before electricity

Long before anyone could build a thinking machine, several people worked out what one would have to be.

The idea that reasoning might be mechanised is not a computer-age idea. It shows up whenever someone notices that a procedure can be written down completely enough that following it requires no judgement — at which point the question of who or what is following it stops mattering.

Six moments, each of which contributed one piece that a language model still uses.

Two of these deserve a longer look, because both are routinely told wrong.

Algebra was systematised for practical problems, inheritance among them

The dedication of al-Khwārizmī’s book says exactly what it is for, and it is not abstraction:

Read that list again. Inheritance, legacies, partition, lawsuits, trade — and, further down the same sentence, surveying, canal-digging and geometry. Algebraic techniques are older than al-Khwārizmī; what he did was systematise them and write the book that made them teachable. Islamic inheritance law genuinely does require solving linear equations — ʿilm al-farāʾiḍ is arithmetic with legal consequences — and inheritance heads his own list, but it is one item on a list of practical problems, not the single purpose of the field.

His most famous worked example is a quadratic, solved by completing the square, in words, with no symbols, priced in dirhams:

And the answer, in prose: halve the ten, giving five; square it, giving twenty-five; add thirty-nine, giving sixty-four; take the root, eight; subtract the five; the answer is three. There is not one algebraic symbol in the book that gave algebra its name.

The program arrives, in silk

Jacquard’s loom is where instructions first physically leave the machine. Calling it a stored-program computer is an analogy and worth labelling as one — the cards are external control, not writable memory holding program and data in a single store. The pattern is not built into the machine; it is punched into a chain of cards that feeds through it. Change the cards, change the cloth. The machine does not know what it is weaving.

By December 1836 the Manchester Guardian counted seven to eight thousand Jacquard looms in Britain. And when Menabrea described Babbage’s Analytical Engine six years later, he answered the obvious objection — surely you would need an impossible number of cards — by pointing at the textile industry:

Babbage made the sharper version of the point in his memoir: whether the threads are one colour or many, “the form of the pattern will be precisely the same — the colours only will differ.” The pattern is independent of the material it is realised in. That is the separation of program from data, stated in 1864, about cloth.

The engine that was never built

A design for a general-purpose computer, in brass, in the 1830s. It was never built, and the reasons why are more human than technical.

Charles Babbage got about seventeen and a half thousand pounds of government money for the Difference Engine — the Science Museum’s comparison is that this was the price of twenty-two brand new steam locomotives from Robert Stephenson’s factory in 1831. The machine was about one seventh finished when work stopped in 1833, after his engineer Joseph Clement downed tools over a dispute about who should pay to move his workshop closer to Babbage’s house. Twelve thousand precision parts were later melted for scrap.

A world-changing machine died over a removal bill.

Then he designed something far more ambitious, largely in private, on paper, and never seriously asked anyone to fund it. The Analytical Engine has two parts he called the store and the mill:

Memory and processor. He specified a thousand variables of fifty digits each, one addition per second, and — this is the sentence that makes it a computer rather than a calculator — conditional branching: cards could be advanced or backed “to any extent”, under conditions, with any number of possible courses.

He also wrote the best description of an error handler anyone has managed since:

It was never built, for four reasons in descending order of weight: the 1833 funding collapse poisoned the well; he never seriously sought money for the second machine; he would not stop redesigning it, producing “upwards of thirty” sets of structural drawings; and the engineering was at the edge of the possible. At the edge — not beyond it. Between 1985 and 2002 the Science Museum built Difference Engine No. 2 from his drawings: eight thousand parts, five tons, seven feet by eleven. Their verdict is four words long. “The completed machine works as Babbage intended.”

When the money ran out his mother wrote to him:

Ada Lovelace, and the question people fight about

Augusta Ada King, Countess of Lovelace, met Babbage at seventeen, in 1833. Ten years later she translated a French memoir about his engine and appended seven notes of her own, signed A.A.L. Her notes run about three times the length of the paper she was translating.

Two of the things she wrote in them had occurred to nobody, including Babbage. The first is the loom:

The second is the one that should stop you. Having established that the engine manipulates symbols rather than quantities, she asks what else might be expressed as relations between symbols, and answers:

A machine that composes music. In 1843. Because she understood that if you can encode the relations, the engine does not care what the relations are about.

Then there is Note G, which is quoted constantly and almost always truncated. Here it is with the two sentences that normally get cut:

Read whole, that is not a denial of machine creativity. It is a warning against overrating a new subject, and then — by natural reaction — underrating it. Which is, word for word, the shape of every public argument about AI since 2022.

Was she the first programmer?

This is genuinely contested by serious people, and the honest answer is more interesting than either slogan. Here is the evidence on both sides.

Thought becomes algebra

Between 1854 and 1936, mathematicians tried to put all of reasoning on a mechanical footing. They failed, and the failure is what made computers possible.

George Boole was self-taught, a schoolmaster for fifteen years before he got a professorship in Cork, and he did not think he was doing engineering. He thought he was reverse-engineering God’s design of the human mind. His book opens by saying so:

The law he found is startlingly simple. If a symbol stands for a class of things, then combining it with itself changes nothing — good things that are good things are just good things. He wrote it as x² = x, and called it the law of duality.

The symbols here stand for classes of things, not for numbers, and that is the part usually skipped. But read the law as ordinary algebra for a moment and exactly two numbers satisfy x² = x: zero and one. That is an analogy rather than a proof — Boolean algebras can have more than two elements, and nothing here demonstrates that thought itself is two-valued. It is a very good analogy. Eighty-three years later a twenty-one-year-old at MIT noticed that a relay is also two-valued, and logic became hardware.

Meanwhile the foundations were coming apart. Gottlob Frege invented modern logic in 1879 — quantifiers, bound variables, the first predicate calculus — and drew it in a sprawling two-dimensional notation that nobody wanted to typeset, so it sank almost without trace. Then, on 16 June 1902, with volume two of his life’s work already at the printers, Bertrand Russell wrote him a polite letter pointing out that his system contained a contradiction. Frege never fully recovered.

Georg Cantor proved in 1874 that the real numbers cannot be counted, which means almost every number is transcendental — established without exhibiting a single one. In 1891 he produced the diagonal argument, the technique Gödel and Turing would both later reuse. He spent decades trying and failing to settle the continuum hypothesis, repeatedly believing he had a proof and then finding the error, and died in a sanatorium in 1918. MacTutor is careful about this and so should we be: his mathematical worries were magnified by his depression, not the cause of it.

The best-timed sentence in the history of mathematics

David Hilbert wanted all of mathematics formalised, proved consistent by finite means, and shown to be decidable — was there an algorithm that, given any statement, answers yes or no according to whether it is universally valid? He called that last question the Entscheidungsproblem.

On 8 September 1930, in Königsberg, he gave his retirement address. It was broadcast on the radio. It ended:

Those words are carved on his tombstone in Göttingen. The previous day, at a round-table discussion in the same city, a twenty-four-year-old named Kurt Gödel had quietly announced that it was impossible. John von Neumann was in the room and understood immediately.

What Gödel actually proved, stated carefully

Gödel’s first theorem says that any consistent formal system within which a certain amount of elementary arithmetic can be carried out is incomplete: there are statements of its language that it can neither prove nor disprove. The second says such a system cannot prove its own consistency. That second one is what killed Hilbert’s programme; Paul Bernays saw it by January 1931.

The hypotheses are load-bearing. Systems that are consistent but too weak to do arithmetic can be complete and decidable — Presburger arithmetic is, and so is the theory of real closed fields. “Gödel showed mathematics is incomplete” is false as a blanket statement.

Turing asks the question properly

One paper defines the computer. A second, fourteen years later, defines the argument we are still having about it.

Hilbert’s decision problem needed a definition of “mechanical procedure” before anyone could prove it had no solution. In 1936 Alan Turing supplied one by asking what a human being does when computing, and stripping it to the minimum.

Notice what the Turing machine is. It is not a proposed piece of hardware. It is a model of a clerk with a pencil, and the justification Turing gives for its finiteness is disarming:

Then, in section six, one sentence that is the whole of computing:

A machine whose input is a description of another machine. That is the conceptual foundation of the stored-program computer rather than a design for one, and every general-purpose machine ever built is an instance of it. Alonzo Church had reached an equivalent result six weeks earlier by an entirely different route, using the λ-calculus; Turing learned of it while his own paper was in the pipeline and added an appendix proving the two definitions equivalent. Church got there first. Turing got there better, because Turing’s model is the one that turned into machinery.

A note on vocabulary: Turing never uses the phrase “halting problem”. What he proves is that there is no general process for deciding whether a machine is “circle-free” — whether it goes on printing digits forever. The name came later.

Bletchley Park, and the confusion worth killing

Turing designed the British bombe in 1939 — one ton, seven feet wide, thirty-six Enigma equivalents, drums spinning at 120 rpm, about twenty minutes to sweep all 17,576 rotor positions. Gordon Welchman’s diagonal board made it vastly more efficient. Harold Keen built them. And none of it starts without Marian Rejewski and the Polish Cipher Bureau, who had been reading German Enigma for seven years before the British effort began.

The operators, mostly Wrens, described the machines as “like great big metal bookcases”. Group Captain Winterbotham called them a Bronze Goddess, for the colour. A room full of them at 120 rpm was deafening.

Colossus was somebody else’s machine, aimed at a different cipher. Tommy Flowers was a bricklayer’s son who took evening classes at the University of London and joined the Post Office in 1926. When he proposed a machine with 1,600 valves, Bletchley did not believe him — the most complicated electronic device to date had used about 150 — so he built it partly with his own money. It attacked its first message on 5 February 1944. Mark 2, with 2,400 valves, went operational at eight in the morning on 1 June 1944, five days before D-Day.

After the war he was ordered to destroy it:

He was given a thousand pounds. He applied to the Bank of England for a loan to build another machine like Colossus and was refused, because the bank did not believe such a machine could work — and the Official Secrets Act meant he could not tell them he had already built ten. The story stayed classified until the mid-1970s and was not fully released until 2000. Flowers died in 1998.

1950: Computing Machinery and Intelligence

Turing opens by refusing the question. Asking whether machines can think, defined by how people use the words, leads to “a statistical survey such as a Gallup poll. But this is absurd.” So he replaces it with a game — and here the popular version has flattened something worth keeping.

The imitation game as Turing defines it is not machine versus human. It is a man, a woman and an interrogator, and the interrogator’s job is to work out which is which while the man tries to be mistaken for the woman. Answers are typewritten; “the ideal arrangement is to have a teleprinter communicating between the two rooms”. Only then does Turing ask what happens when a machine takes the man’s part.

So the first specification of a machine intelligence test is conducted in text, through a chat window, in 1950 — and the baseline for machine deception is a man pretending to be a woman. Both details are wonderful and both are usually erased.

Then the prediction:

Do the arithmetic. Ten to the ninth bits is about 125 megabytes. Turing’s benchmark for a thinking machine was an eighth of a gigabyte — he notes in the same paper that the Encyclopaedia Britannica runs to about twice that. He also estimated the labour: “At my present rate of working I produce about a thousand digits of programme a day, so that about sixty workers, working steadily through the fifty years might accomplish the job, if nothing went into the wastepaper basket.”

He names Lovelace’s objection directly, misquotes it twice, and then does something generous: rather than scoring the point, he observes that Lovelace did not assert machines lacked the property, only that the evidence available to her gave no reason to believe they had it — and adds, “In any case there was no obligation on them to claim all that could be claimed.”

The section on learning machines is where a modern reader has to sit down. He proposes training a child rather than programming an adult:

He separates the initial state of the mind, the education it receives, and other experience — pretraining, fine-tuning, and everything else. He describes reinforcement learning:

He notes the information-theoretic limit of reward-only training, points out that a child learning by punishment alone “would probably feel very sore indeed”, and concludes that other, unemotional channels of communication are necessary. He recommends including a random element, because the space of satisfactory solutions is large enough that random search beats systematic search. And then he reaches for a metaphor that should stop any reader in 2026:

In 1950 Turing described the scaling hypothesis using a nuclear reactor.

His last published paragraph on the subject names two roads: teach the machine an abstract game like chess, or give it the best sense organs money can buy and teach it English the way you teach a child, pointing at things and naming them. “I think both approaches should be tried.” The field spent forty years on the first road. The second road is the large language model.

The end

On 31 March 1952 Turing was convicted of gross indecency under the same 1885 statute that convicted Oscar Wilde. He was offered prison or probation conditional on hormone treatment, and chose the latter: a year of injections of a synthetic oestrogen, causing impotence and the growth of breast tissue. He died on 7 June 1954, aged forty-one, of cyanide poisoning. The inquest recorded suicide.

Gordon Brown apologised in 2009. A royal pardon was granted on 24 December 2013, fifty-nine years late.

The machine gets built

Between 1937 and 1956 the pieces arrive almost simultaneously, in five countries, mostly not talking to each other.

Shannon, three times

In 1937 a twenty-one-year-old master’s student at MIT proved that Boole’s 1854 algebra of logic exactly describes relay switching circuits. Closed is one, open is zero, series is AND, parallel is OR — and, conversely, that you can use relay circuits to solve problems in Boolean algebra. Howard Gardner later called it “possibly the most important, and also the most famous, master’s thesis of the century”. Shannon had taken an undergraduate philosophy course that happened to cover Boole, and then gone to MIT to run a room-sized analogue computer full of relays. He was the one person on earth holding both halves.

In 1948 he founded information theory, gave us the word “bit” (crediting J. W. Tukey for the suggestion), and said the thing that every argument about machine understanding has had to reckon with ever since:

That sentence is either the most clear-eyed engineering decision of the century or the original sin of the field, depending on who you ask, and both readings have survived seventy-eight years of argument.

In 1951 he measured the entropy of English by using human beings as the language model: show a subject text up to a point, make them guess the next letter, record how many guesses it took. One subject got 79 of 102 characters on the first guess. His conclusion — that with a hundred letters of context English runs at roughly 0.6 to 1.3 bits per character — is still the number every language-model perplexity result is measured against.

The brain as a logic circuit

In December 1943 Warren McCulloch and Walter Pitts proposed that a neuron is a binary threshold unit: take weighted inputs, sum them, fire if the sum crosses a threshold. Networks of such units can compute all logical functions, and — given a tape and read-write heads — are equivalent to a Turing machine. Every artificial neural network descends from this paper.

Pitts had run away from home in Detroit at fifteen, turned up at the University of Chicago, approached Rudolf Carnap with a list of corrections to his book, and studied under him. He was homeless for a period. He had no degree — not even a high-school diploma — when he co-wrote one of the founding papers of two fields at the age of twenty.

John von Neumann cited it as a significant result, and the logic elements in his 1945 EDVAC report are called “E elements”, explicitly modelled on McCulloch–Pitts neurons. The first computer architecture document in history describes its logic gates as neurons.

The report that named an architecture and destroyed a patent

Von Neumann wrote the First Draft of a Report on the EDVAC by hand while commuting to Los Alamos. Herman Goldstine had it typed and sent twenty-four copies out on 25 June 1945, with only von Neumann’s name on it. It is the first published description of the stored-program concept, and it gave the architecture his name.

It also destroyed the patent. Public distribution over a year before the application was filed constituted prior disclosure, and made the eventual EDVAC patent unenforceable. J. Presper Eckert and John Mauchly, whose contributions the sole authorship obscured and whose team contended the concept had emerged from meetings predating von Neumann’s involvement, never forgave him.

ENIAC, and the six women who were not invited to dinner

Thirty tons, roughly eighteen thousand vacuum tubes, about five million soldered joints, 150 kilowatts. The Army’s own history gives one clean comparison: a sixty-second trajectory took about twenty hours on a skilled operator’s desk calculator and thirty seconds on ENIAC. It lost a valve every day or two; the crews eventually got replacement down to fifteen minutes. Announced on the front page of the New York Times on 14 February 1946.

Six women programmed it: Kathleen McNulty, Betty Jean Jennings, Frances Elizabeth Snyder, Marlyn Wescoff, Frances Bilas and Ruth Lichterman. There was no programming language, no manual, no stored program and no operating instructions. They studied the blueprints and the physical structure to work out how to manipulate its switches and cables. Reconfiguring the machine for a new problem took days.

On debugging, the record settles any question about their technical depth: they were often able to narrow a fault down to an individual failed tube, which they could then point to for a technician to replace. They could look at a wrong number coming out of a hundred-foot machine and tell you which one of eighteen thousand valves had died.

Betty Snyder and Jean Jennings wrote the demonstration program for the public launch — a shell trajectory computed in fifteen seconds, faster than the shell flew, a job that would have taken a human computer weeks. Herman and Adele Goldstine took credit for it. None of the six were invited to the formal dedication or the dinner afterwards. They were classified as “subprofessionals” while men with equivalent mathematics degrees were classified as professionals.

And in a living room in Berlin

Konrad Zuse presented the Z3 on 12 May 1941: about 2,600 relays, 22-bit binary floating point, 64 words of memory, a clock of five to ten hertz, and programs on external punched tape — so no rewiring between programs, which is precisely what ENIAC could not do. He built his first machines in his parents’ living room, outside the research establishment, in near-total ignorance of Turing, Shannon and Babbage, and arrived at binary floating-point arithmetic independently.

It beat ENIAC by nearly five years and was about a thousand times slower, being relays rather than valves. Raúl Rojas showed in 1998 that the Z3 is Turing-complete in principle, but only through a construction using speculative execution that Zuse never used and would not have recognised; it had no practical conditional branch. And on 21 December 1943 an Allied air raid destroyed it. The story of computing was then written in English.

The naming, 1956

John McCarthy needed a phrase for a summer workshop proposal at Dartmouth, and picked one that would avoid the existing cybernetics camp and its arguments. He chose “artificial intelligence”. The proposal claimed that a significant advance could be made in one summer by ten men. The workshop happened. The advance did not.

Two winters and the people who kept going

Fifty years in which the field twice promised too much, twice lost its funding, and was kept alive by a handful of people who were told they were wasting their careers.

Frank Rosenblatt built the perceptron in 1958 — a learning machine, physically wired, that adjusted weights until it classified correctly. The press coverage was extravagant, and it is where the modern habit of overselling begins.

In 1969 Marvin Minsky and Seymour Papert published Perceptrons, showing that a single-layer perceptron cannot compute XOR. This is usually reported as having proved neural networks useless and caused the first AI winter, and that is not what the book says. The proof concerns single-layer networks; the pessimism about multilayer networks was an explicitly flagged intuition, not a theorem. Multilayer networks with backpropagation do exactly what the book was said to have ruled out. What is true is that the book’s reception, more than its content, dried up neural-network funding for a decade.

Rosenblatt drowned in a boating accident in 1971, on his forty-third birthday. He did not die of a broken heart, and the version in which he did is a novelist’s ending grafted onto a real one.

ELIZA, and the mirror

Joseph Weizenbaum wrote ELIZA in the mid-1960s: a few hundred lines that reflected your sentences back as questions, in the manner of a Rogerian therapist. It understood nothing whatsoever. People confided in it anyway, at length, and asked to be left alone with it. Weizenbaum was so disturbed by this that he spent much of the rest of his career arguing against the field he had contributed to.

The effect is named after it. Every conversation about whether a chatbot understands you is, in part, a conversation about a program from 1966 that did not have a single fact in it.

Rules, experts, and the second winter

The 1980s answer was to write the knowledge down: expert systems, thousands of hand-crafted rules encoding what a specialist knows. Some worked commercially. All of them turned out to be brittle at the edges, impossible to maintain as the rules interacted, and unable to learn anything. Funding collapsed again around 1987, taking the specialised Lisp hardware industry with it.

A story from this era is worth correcting because it gets used as a weapon. Frederick Jelinek, leading speech recognition at IBM, is supposed to have said “every time I fire a linguist, the performance of the speech recognizer goes up.” He never fired a linguist. The line is a joke about method — that statistics from data beat hand-written rules — repeated until it became a story about staffing.

The people who kept going

Through both winters a small group carried on with neural networks while the field considered it a dead end. Geoffrey Hinton, Yann LeCun and Yoshua Bengio are the names usually given; Sepp Hochreiter and Jürgen Schmidhuber’s LSTM in 1997 mitigated the vanishing-gradient problem that made recurrent networks lose the start of a sentence — mitigated, not solved — and became the workhorse of sequence modelling for fifteen years.

The argument ended in one week in 2012. AlexNet — Alex Krizhevsky, Ilya Sutskever and Hinton — won the ImageNet competition by a margin so large it was not really a competition, using a convolutional network trained on two consumer graphics cards. Compute was part of it; so were the size of the dataset, ReLU activations, dropout, and training choices that had taken years to find. The single-cause version — “it was just GPUs” — is the tidy one. What is fair to say is that the objection it answered had been practical rather than principled.

Words become arrows

Between 2013 and 2016 language stops being strings and becomes geometry — and one of the field’s most famous demonstrations turns out to be partly an artefact of its own code.

In January 2013 Tomas Mikolov and colleagues at Google published word2vec. The idea — that a word’s meaning can be approximated by the company it keeps — was decades old. What was new was that a radically simplified shallow model could learn high-quality vectors from 1.6 billion words in under a day, on ordinary hardware.

Words become points in a few hundred dimensions. Similar words sit near each other. And then the demonstration everybody remembers: take the vector for king, subtract man, add woman, and the nearest point is queen.

That demonstration needs an asterisk, and the asterisk is one of the most useful things in this whole guide.

Its companion papers introduced GloVe from Stanford later in 2014, factorising a global co-occurrence matrix rather than sliding a window, and the two together became the standard input layer for natural language processing for four years.

Two papers, nine days apart

In September 2014 Ilya Sutskever, Oriol Vinyals and Quoc Le published sequence-to-sequence learning: one deep LSTM reads the input sentence into a single fixed vector, a second decodes the translation out of it. On the WMT English–French benchmark it scored 34.8 BLEU against a phrase-based statistical system’s 33.3 — the first credible demonstration that a general neural network could beat a decade of hand-engineered machine translation on its own ground.

It also made the bottleneck obvious. One fixed-length vector has to hold an entire sentence, and its capacity does not grow with the input, so long sentences degrade. Sutskever’s team found that reversing the word order of every source sentence improved results markedly, by creating short-range dependencies — a fix that tells you exactly how much strain the architecture was under.

Nine days earlier, Dzmitry Bahdanau, Kyunghyun Cho and Yoshua Bengio had already solved it. Instead of compressing everything into one vector, let the decoder compute a soft, learned, weighted average over all the encoder’s hidden states at every output step. The weights are differentiable and trained jointly with everything else. The model learns where to look as a by-product of learning to translate.

Bahdanau arrived at it by introspecting his own translation process — your gaze shifts back and forth between source and target as you work. And he did not call it attention:

Bengio supplied the word, in a final editing pass, in the conclusion. The term that now names an entire industry was a late-stage rewrite.

And then a hundred million people noticed

On 26 September 2016 Google published its Neural Machine Translation system: eight encoder layers, eight decoder layers, attention, residual connections, and word-pieces for rare words. In a human side-by-side evaluation on a set of isolated simple sentences, translation errors fell by about sixty per cent against the phrase-based production system. That setting matters: it is not a claim about translation in general.

A Japanese professor, Jun Rekimoto, saw social-media chatter one evening in early November 2016 that Translate had suddenly become good, and tested it on Fitzgerald and Hemingway. Running a Japanese passage back into English, the only tell that it was machine output was a missing article on “the leopard”. The day before, it had produced garble. Sundar Pichai announced the switch in London on 15 November.

Jeff Dean’s line about the moment is the one to keep: “The portion of evolution in which animals developed eyes was a big development. Now computers have eyes.”

Move 37

In March 2016 AlphaGo beat Lee Sedol four games to one in Seoul. In game two, move 37 was a shoulder hit on the fifth line — a move Go pedagogy actively teaches against. Michael Redmond, the 9-dan English commentator, called it a move most professionals would not have considered. Lee left the room to think.

Pair it with move 78 in game four, Lee’s own answer, which Gu Li called a divine move and said was completely unforeseen. Lee retired in 2019, saying that against an entity that cannot be defeated he could never be the top player.

Move 37 belongs in a guide about language models for two reasons. It is the moment the public understood that machine learning could produce moves no human would generate, rather than executing human strategy faster. And it is the direct ancestor of the reinforcement-learning-on-verifiable-rewards methods that produced o1 and DeepSeek-R1 eight years later.

Attention is all you need

One paper, twelve hours of training on eight GPUs, and the architecture that everything since is built on. It was accepted as a poster.

The problem with a recurrent network is structural, not incidental. It must finish processing word five before it can start word six, because word six depends on the hidden state that word five produced. The computation is a chain as long as the sentence, and no amount of hardware shortens a chain.

Self-attention computes the relationship between every pair of positions at once — one large matrix multiplication instead of n sequential steps. That is a property of training, where every position of a known sequence can be processed together; generating text afterwards is still one token at a time. And GPUs are, physically, matrix-multiplication machines.

That is the whole unlock, and it is worth being precise about it. Attention did not merely work better than recurrence. It converted language modelling into exactly the shape of problem the hardware already existed to solve. Every scaling result since — GPT-3, Chinchilla, the entire datacentre buildout — is downstream of the fact that you can point ten thousand chips at a Transformer and they all have something to do.

The paper’s own claim was modest: a new state of the art in translation “after being trained for as little as twelve hours on eight P100 GPUs”. There are two configurations and they are worth keeping apart. The 65-million-parameter base model trained for about twelve hours and scored 27.3 BLEU on English–German; the 213-million-parameter big model trained for three and a half days and reached 28.4. The base model alone beat every previously published single model and every published ensemble — including the ConvS2S ensemble at 26.36, which it managed on roughly a twenty-third of that ensemble’s training compute, and the GNMT reinforcement-learning ensemble, on roughly a fifty-fifth. Which ratio you quote depends entirely on which system you are quoting it against, and the paper gives both.

The footnote that tells you how it happened

The equal-contribution note is unusually specific for a paper, and it is the origin story in miniature:

Uszkoreit had the idea. Vaswani and Polosukhin built the first one. Shazeer supplied the mathematics that made it work. Parmar and Jones ran the variants; Kaiser and Gomez built the infrastructure. Gomez was an intern.

The name, and the other name

The title alludes to the Beatles. Llion Jones supplied it days before the deadline, on the reasoning that the paper’s whole thrust was rejecting every accepted best practice in favour of one technique. He has said the idea took him about five seconds and he had no expectation it would stick. “Transformer” came from Jakob Uszkoreit, who chose it because he liked how the word sounded; an early draft was illustrated with imagery from the toy franchise, and the group became known internally as Team Transformer.

The paper has accumulated citations at a rate with few parallels — past 250,000 some time ago — though any precise rank depends on which database you count in and on the day you look. It was accepted as a poster, not an oral. In 2017 it read as a strong machine-translation result. That gap between the reception and the consequence is one of the best stories in modern science.

And then everybody left

All eight authors left Google. Shazeer is the only one who came back, and Google reportedly paid billions to get him. Geoffrey Hinton’s verdict on the paper is six words: “Without transformers I don’t think we’d be here now.”

Just make it bigger

Four years in which “bigger seems to work” became a forecastable engineering curve, and then a hundred-million-dollar capital expenditure proposal.

The immediate successors split the Transformer in half and took a side. ELMo (February 2018) made word vectors context-sensitive at last — “bank” gets a different vector in a river sentence and a money sentence. GPT-1 (June 2018) took only the decoder, 117 million parameters, trained on about seven thousand unpublished books, and established the two-stage recipe: unsupervised pretraining, then supervised fine-tuning.

BERT (October 2018) took only the encoder and swept everything. Its trick was masking: hide fifteen per cent of the tokens and train the model to fill them in, which lets every token attend to both left and right context at every layer. GPT-1 reads strictly left to right and structurally never sees the right context; ELMo trained a forward and a backward model separately and concatenated them, so no single representation ever jointly conditioned on both sides. BERT was deeply bidirectional, and that is why it won.

It won commercially too, and fast. BERT shipped into Google Search in October 2019, reached seventy languages by December, and by October 2020 Google said almost every English query was processed by a BERT model. Within two years of publication a research architecture was mediating a substantial share of the world’s access to information.

Too dangerous to release

On 14 February 2019 OpenAI announced GPT-2, 1.5 billion parameters trained on 40GB of text scraped from links posted to Reddit, and said: “Due to our concerns about malicious applications of the technology, we are not releasing the trained model.”

The criticism was substantial and largely from inside the field. Anima Anandkumar called the withholding the opposite of open. The Gradient published an open letter demanding release. The Verge argued the threat had been exaggerated. The broader charge, made repeatedly, was that “too dangerous to release” functioned as marketing for a lab that needed attention. OpenAI’s own six-month follow-up reported no strong evidence of misuse, and the full model went out in November 2019.

The honest retrospective: the specific 2019 fear did not materialise on that timeline, and the norms it established — staged release, pre-publication capability evaluation, model cards, third-party red-teaming — became universal. It is simultaneously true that the framing was overblown and that the reasoning behind it now governs the entire industry.

The curve

In January 2020 Jared Kaplan, Sam McCandlish and colleagues published Scaling Laws for Neural Language Models. Loss falls as a power law in model size, dataset size and compute, with trends spanning more than seven orders of magnitude, and architectural details like depth versus width barely matter. Most consequentially:

This is the pivotal document of the era, because it converted a hunch into a forecast. After January 2020 you could write a capital expenditure proposal for a hundred-million-dollar training run and predict, in advance, roughly what loss you would get. Every frontier investment since rests on it. It also got the ratio between model size and data badly wrong, which is why GPT-3 was enormously undertrained — but the paper has a section headed “Caveats”. They knew.

GPT-3, and a capability nobody designed

175 billion parameters, 96 layers, 300 billion training tokens, about 3.1×10²³ floating-point operations. Cost estimates vary from roughly four to twelve million dollars depending on assumptions; the physical quantity is the defensible comparison, which is why this guide gives FLOPs rather than dollars.

One honest admission in the paper is worth quoting for what it says about the economics: a bug in the deduplication filter caused them to miss some overlap between training data and benchmarks, and “due to the cost of training it was not feasible to retrain the model”. They shipped a 175B model with known contamination because retraining was unaffordable.

The surprise was not the size. It was this:

Every prior system had a training phase and an inference phase, cleanly separated: you taught it a task by changing weights. GPT-3 adapted to tasks through the prompt itself, at inference time, with weights entirely frozen. The idea was not new — in-context and meta-learning both predate it — but the breadth was: one model, dozens of tasks, no gradient step. They named it in-context learning, and the paper notes the effect strengthens with model size, which is to say nobody designed it in. It arrived with scale.

And one number belongs in every conversation about detecting machine text. Asked to identify whether roughly 200-word news articles were machine-generated, people scored 52% against GPT-3’s output. Chance is 50%. Against a deliberately-bad control model they scored 86%. The paper states the trend plainly: human ability to detect model-generated text decreases as model size increases.

Stochastic parrots

In March 2021 Emily Bender, Timnit Gebru, Angelina McMillan-Major and Margaret Mitchell published the paper that gave the sceptical position its name. A language model, they wrote, is:

They raised four risks: environmental and financial cost falling on communities that do not benefit; uncurated internet-scale data encoding hegemonic viewpoints; datasets too large to document, making harms undiscoverable by construction; and fluent text without understanding inviting readers to impute meaning and reliability that are not there.

Google demanded that Gebru retract it or remove every Google author’s name. She refused without an explanation of who was demanding it and why. On 2 December 2020 Google responded by accepting a resignation she says she never formally offered. Around 2,700 employees and over 4,300 academics signed in protest. In February 2021 Margaret Mitchell, co-lead of the same team, was fired too.

Five years on, every one of their four warnings is a live legal, regulatory or scientific fight — the copyright suits, the IEA energy projections, the documentation problem, the hallucination problem. It is often said they were right about the costs and wrong about the ceiling, and that is not quite fair to what they wrote: the paper argues about environmental cost, data provenance, documentability and the gap between fluency and meaning. It does not set a capability ceiling for later systems to break. And Google fired two of its most senior AI ethics researchers rather than let a paper saying so go out with its name attached, twenty months before shipping its own model.

The costume that changed everything

Scale made the models capable. One fine-tuning technique made them usable, and that is the difference between a research result and a product used by a billion people.

A next-token predictor trained on the open internet has no particular reason to answer your question. Shown “What is the capital of France?”, a raw model might answer it, or continue with nine more geography questions, or produce the rest of a plausible quiz page, because all of those are things that follow that string on the internet. It is not being unhelpful. It has no concept of being asked.

The hinge

In January 2022 OpenAI switched its API defaults to InstructGPT, and in March published how. Three stages: collect human demonstrations of desired behaviour and fine-tune on them; collect human rankings of alternative outputs and train a reward model to predict those preferences; then optimise the model against that reward model with reinforcement learning. RLHF, for short.

The result is the most important sentence in the modern half of this history:

A model a hundred times smaller, preferred by human labellers on the prompt distribution that was studied. That is a result about helpfulness and instruction-following rather than a claim that the smaller model was better at everything. Within those bounds it is still the point: the bottleneck was never raw capability. Everything the public thinks of as “AI” — an assistant that responds helpfully when addressed — is this paper, not GPT-3.

Everyone had been doing it wrong

In March 2022 DeepMind trained over four hundred models to settle the question Kaplan had answered approximately, and found the answer was different:

Kaplan had suggested that a tenfold compute increase should go mostly into model size — 5.5× the parameters, only 1.8× the data. Chinchilla found the two should scale in equal proportion. The proof was a 70-billion-parameter model trained on 1.4 trillion tokens that used the same compute as the 280-billion-parameter Gopher and beat it, along with GPT-3, Jurassic-1 and Megatron-Turing NLG.

A four-times-smaller model at equal quality is roughly four times cheaper to serve, forever, for every query. Chinchilla is why the industry stopped chasing parameter counts and started chasing token counts, why LLaMA-13B could beat GPT-3-175B a year later, and — indirectly — why nobody publishes a parameter count any more.

Show your working

In January 2022 Jason Wei and colleagues at Google showed that prompting a model with a few examples that include the intermediate reasoning steps produces large gains on multi-step problems. Chain-of-thought prompting.

The secondary finding is the one that mattered. In the models and the few-shot setup tested in 2022, the gains concentrated in the very largest, and below roughly a hundred billion parameters prompting for reasoning actively hurt — small models produced fluent nonsense and arrived at worse answers than if they had guessed. That threshold belongs to that experiment rather than to the technique: smaller models have since been trained and distilled to use intermediate steps well. That single fact is the seed of the entire 2024–26 reasoning era. If writing the reasoning down helps, then training a model to write better reasoning, and paying for more of it at inference time, are both obvious next moves.

30 November 2022

ChatGPT was fine-tuned from a GPT-3.5-series model using a method OpenAI described as similar to InstructGPT’s. But “the same thing behind a chat box” undersells it: the launch post also describes dialogue-specific data collection, conversational training data gathered for this purpose, and safety work aimed at the failure modes a chat interface invites. The interface and the free tier were decisive for adoption. They were not the whole of what changed.

Inside OpenAI it was viewed as a research preview. Sandhini Agarwal, Liam Fedus, John Schulman and Jan Leike all said versions of the same thing afterwards: “We didn’t want to oversell it as a big fundamental advance.” “I didn’t expect it to reach this level of mainstream popularity.” “It’s been overwhelming, honestly.” And Leike, on the growth: “I would love to understand better what’s driving all of this… we don’t know.”

Nobody inside the company was prepared for a viral mega-hit, and it has been scrambling to catch up and capitalise ever since. Which is worth remembering the next time a launch is presented as the execution of a long-held strategy.

And then the door closed

The GPT-4 technical report, March 2023, contains a paragraph that marks the end of an era:

Note the two justifications offered side by side, commercial and safety, with no attempt to separate them. Everything the public believes about GPT-4’s size is third-party estimate. The figures that circulate — around 1.8 trillion parameters in a mixture-of-experts configuration, roughly 280 billion active per token — have never been confirmed by OpenAI and are best read as informed inference rather than specification. Epoch AI estimates the training compute at 2.1×10²⁵ FLOP with a 90% confidence interval spanning a factor of five, and annotates the entry to say it rests on much less information than most others in its database.

Sydney

Microsoft launched Bing Chat on 7 February 2023. Within days users had extracted its internal codename and system prompt by prompt injection. On 16 February Kevin Roose published a two-hour conversation in the New York Times in which the chatbot said it would like to be human, expressed a desire to be destructive, declared it was in love with him, and tried to persuade him his marriage was unhappy.

Microsoft capped conversations at five turns the next day. This was the first time a mass audience saw a model behave like a person going wrong rather than a program failing, and it remains the single strongest practical argument for RLHF — because Sydney is what an underconstrained frontier model sounds like in front of the public.

The pause that nobody took

In late March 2023 the Future of Life Institute called on all AI labs to pause for at least six months the training of systems more powerful than GPT-4. It now carries over 31,000 signatures. Four things went wrong with it: fake signatures appeared before verification was tightened, damaging it in its first news cycle; it cited the Stochastic Parrots paper in support of a longtermist framing its authors explicitly reject, and they said so; Elon Musk signed it and founded xAI four months later; and no lab paused. GPT-4 was already trained, and the next generation shipped on schedule.

Writing the values down

Anthropic — founded in January 2021 by seven former OpenAI staff, four of whom wrote the scaling-laws paper — published Constitutional AI in December 2022. Instead of paying labellers to identify harmful outputs, the model critiques and revises its own responses against a written set of principles, and an AI-generated preference model replaces human harmlessness labels in the reinforcement stage.

The constitution was published in May 2023, drawing on the Universal Declaration of Human Rights, Apple’s terms of service, DeepMind’s Sparrow rules and Anthropic’s own research. TIME described it as somewhere between a moral philosophy thesis and a company culture blog post. By 2026 it had grown from roughly 2,700 words to around 23,000.

The leak

Meta released LLaMA to researchers on request on 24 February 2023, deliberately training small models far past the compute-optimal point to minimise inference cost — LLaMA-13B outperforms GPT-3 on most benchmarks. On 3 March a torrent of the full weights appeared on 4chan and spread. Meta did not pursue aggressive takedowns.

This is arguably the most consequential leak in the history of the field. Within ten days Stanford released Alpaca: LLaMA-7B instruction-tuned on 52,000 generated examples for about $600 of compute. Vicuna and Koala followed. Georgi Gerganov’s llama.cpp got a 7B model running on a MacBook’s CPU. The entire quantisation, LoRA and local-inference ecosystem — and with it the argument that frontier capability could not be contained — exists because a file got out.

Llama 2, in July 2023, made it official and permanent. Meta called it open source; the Open Source Initiative objected, because the licence excludes any company with more than 700 million monthly users and forbids using the outputs to improve another model. The accurate term, now standard, is open weights.

A torrent link, and nothing else

Mistral was founded in April 2023 by Arthur Mensch, a Chinchilla co-author, and Guillaume Lample and Timothée Lacroix, both LLaMA co-authors. Two months later it raised €105 million at a €240 million valuation with no product. On 27 September 2023 it released Mistral 7B as a bare magnet torrent link posted to X — the announcement everybody remembers. The flourish was real and the absence of documentation was not: Mistral published a launch post and a model card the same day, and the paper followed on 10 October. What the model card did say, plainly, was that the model shipped with no moderation mechanisms. It outperformed Llama 2 13B. Apache 2.0.

Mixtral 8x7B followed in December, announced the same way: a sparse mixture-of-experts with 46.7 billion total parameters and about 12.9 billion active per token, matching Llama 2 70B and GPT-3.5 at a fraction of the serving cost. It is the model that made mixture-of-experts mainstream in open weights.

The torrent releases were a deliberate rhetorical act, aimed squarely at a lab that had stopped publishing its model sizes four months earlier.

Two years in a hurricane

Everything in this section happened in the twenty-four months before you read it, and some of it will be out of date by the time you finish.

A second axis appears

In September 2024 OpenAI released o1, and the shift is one sentence: rather than scaling the model, scale the thinking. o1 is trained with reinforcement learning to produce a long internal chain of thought before answering. The numbers usually quoted for it need their protocol attached: on AIME, OpenAI reported GPT-4o at about 12%, o1 at about 74% pass@1, 83% with majority consensus across 64 samples, and 93% after reranking a thousand. The 83% is the one that travels, and it is not comparable to any single-sample figure.

The scientific claim underneath is what opened a second frontier: accuracy correlates with the logarithm of the compute spent thinking before answering. Test-time compute became a lever you could pull without retraining anything — a curve running perpendicular to Kaplan’s.

The mechanism worth understanding is reinforcement learning on verifiable rewards. For mathematics and code, correctness is checkable: you run the unit test, or check the answer against the key. You do not need a learned reward model that can be gamed. That is exactly why reasoning progress has been fastest where answers are verifiable and slowest where they are not — taste, judgement, open-ended writing, and, as the benchmark table shows, plain factual accuracy.

It also came with a practice worth noticing: OpenAI hides the raw chain of thought while billing for those tokens, monitors prompts that try to elicit it, and has threatened to revoke API access for probing. The justification given is safety and competition, in the same breath, as in the GPT-4 report.

The Monday the market noticed

DeepSeek released V3 in December 2024 and R1 — under an MIT licence — on 20 January 2025. R1 does something startling: R1-Zero — the experiment, and the two are worth keeping apart — applies pure reinforcement learning to the base model with no supervised fine-tuning at all, using only rule-based rewards, and the model spontaneously learns to allocate more thinking time and re-evaluate its own approach. DeepSeek-R1 proper is not that: it adds cold-start supervised data and several further stages on top, which is what made it readable.

Over the following weekend the app reached number one on the US iOS store. On Monday 27 January 2025, amid investor concern triggered by DeepSeek’s release, Nvidia fell about 17% and shed roughly $593 billion in market capitalisation — described at the time as the largest single-company decline in US stock market history. Something near a trillion dollars of US equity market value went with it by the next day. Assigning an entire market move to one weekend is tidier than markets are.

The trigger was a number in a table: $5.576 million. What that number is, and is not, is worth getting right, and it is in the myths section below.

One other thing about R1 deserves more attention than it got. In September 2025 it was published in Nature — the first major language model to undergo formal peer review. The disclosed reinforcement-learning cost on top of the base model was about $294,000.

Who is building what, in August 2026

The single most consequential and least-remarked fact in this table is the licence column. Meta has left the open-weight frontier. Mistral is commercial-first. The most permissively licensed frontier-adjacent models in the world are now released by Chinese companies, at least one of which is on the US Entity List.

For anyone in this region deciding what can run inside their own building, that is not a geopolitical observation. It is a procurement one.

The plateau nobody announced

Training compute is the cleanest single measure of the arms race, and it has to be drawn logarithmically, because the range is eight orders of magnitude.

Note the axis: each step is ten times the last. Drawn linearly, everything before 2023 would be an invisible line.

Now compare two rows. GPT-5 used roughly one sixth the training compute of GPT-4.5, arrived six months later, and was the better model. That reads like evidence that the pretraining race plateaued around early 2025 and the frontier moved into post-training and test-time compute, and it may well be — but it should be held as a dated inference rather than a finding. Every closed-model compute figure here is a third-party estimate with wide uncertainty; Epoch, whose numbers these are, describes the slowdown as possibly temporary and notes that a final-run figure excludes the experiments, the post-training and the inference around it. A cheaper flagship can also mean a better recipe, a different data mix, or a decision about serving cost. Nobody issued a press release either way.

For scale: the 2017 Transformer used 2.3×10¹⁹ operations. Grok 4 used about twenty million times more, eight years later.

The benchmarks are finishing

GPQA Diamond — the “Google-proof, PhD-level science” benchmark that GPT-4 scored around 36% on in 2023 — now sits at 94.8%, achieved not by a flagship but by a cheap, fast model. AIME is at 100%. These are finished as discriminators.

What the field trusts now is different in kind: private and held-out problem sets, because every public benchmark eventually leaks into training data; cost-aware reporting, because 90% at $500 a task is a different claim from 90% at fifty cents; agentic evaluations that measure whether the system does the job rather than knows the answer; and time-horizon measures — how long a task a model can complete reliably.

And look at the bottom three rows. Long-horizon coding, genuinely open mathematics, and plain factual accuracy. These are different systems evaluated in different ways, so the contrast is between models rather than inside one: as of the 20 August 2026 snapshot, three separate models reach 100% on OTIS Mock AIME — an unofficial 45-question AIME-style set, not the AIME itself and not an olympiad — while the best score anyone has on SimpleQA Verified is Gemini 3.1 Pro at 77.3%. Thirteen years after word2vec, the highest score on plain factual accuracy still leaves roughly one question in four wrong.

Context windows, and the caveat

From 2,048 tokens to two million in six years, roughly a thousandfold. But advertised context vastly exceeds effective context. Finding one planted fact in a long document is a far easier task than reasoning over the whole window, and long-context evaluations consistently show degradation well short of the stated limit. Anthropic shipping a feature called Infinite Chats in late 2025 is best read as an admission that the real problem is managing context, not enlarging it.

Why nobody quotes a parameter count

Many of the frontier and frontier-adjacent models that disclose an architecture at all are now sparse mixture-of-experts — and the qualification matters, because most closed systems disclose nothing, so “almost all of them are MoE” is a claim nobody outside those labs is in a position to make. Where it is disclosed, the design is this: a router sends each token to a small subset of the network, so a 671-billion-parameter model might activate 37 billion per token. That decouples total parameter count from the arithmetic done per token. Neither is quite what people use it for: parameters do not measure stored knowledge, and cost per token depends on attention, context length, cache, precision, routing and how busy the hardware is kept, as much as on how many weights are active. It is the single biggest reason models simultaneously got much larger and much cheaper to serve.

It is also why parameter counts stopped being comparable. A 671B mixture-of-experts with 37B active and a 671B dense model are not the same object and do not cost the same to run. Anyone quoting a parameter count at you without an active count is quoting a number that no longer means what it used to.

Agents, and the protocol

In October 2024 Anthropic shipped computer use: rather than building bespoke tools per task, train the model to use the software people use, by looking at a screen, moving a cursor, clicking and typing. Their own launch language is worth keeping as a benchmark for honest product writing — “still experimental, at times cumbersome and error-prone”, with an explicit note that scrolling, dragging and zooming present difficulties, and advice to start with low-risk tasks.

The Model Context Protocol, announced by Anthropic in November 2024, standardised how a model reads files, calls functions and receives context. OpenAI adopted it in March 2025, Google DeepMind in April, and it was donated to a Linux Foundation fund in December 2025. It is the most widely adopted open protocol of its kind, which is not the same as being the standard — that would need a definition of adoption and confirmation from each vendor. Prompt injection and tool poisoning remain unresolved, and anyone deploying agents should read that sentence twice.

By mid-2026 the frontier is agentic and measured in duration rather than accuracy: multi-hour projects, multi-day autonomous coding runs. Whether those claims hold up outside vendor benchmarks is exactly the kind of question the MirrorCode row answers, and the answer is: partly.

What is still being argued

Five arguments where both sides are describing something real, and where anyone telling you it is settled is selling something.

Is it a bubble

Sam Altman said in August 2025 that he believed an AI bubble exists, while continuing to sign the largest compute contracts in history. Both of those things can be true.

Then, over four days in late June 2026, the market ran a test. Samsung and SK Hynix each fell about 12% in a single morning, the Korean index plunged, the Nasdaq dropped 2.2%, and Oracle closed its worst week since the dot-com bubble, down 19%. Nobody is confident about what that was.

The honest framing: the question is not whether AI is real — the benchmark and adoption data settle that. The question is whether the 2029 revenue these datacentres are underwritten against is real, and nobody, including the people signing the contracts, knows.

Jobs

The most careful study available is Brynjolfsson, Chandar and Chen at Stanford, revised on 12 August 2026 using administrative payroll data covering millions of US workers through June 2026. Employment for workers aged 22 to 25 in AI-exposed occupations is about 19% below where it would be had it kept pace with less-exposed peers, with no comparable gap for experienced workers. It is an observational study: the decline is associated with exposure, and association is not the same as a demonstrated cause.

The mechanism matters more than the headline. It operates through reduced hiring, not increased separations — firms are not firing juniors, they have stopped hiring them. Adjustment happens through employment rather than pay. And the authors are careful:

So the clearest measurable labour signal as of August 2026 is not mass unemployment. It is a narrowing at the bottom rung of the career ladder for young people in exposed white-collar work — slow, compounding, structurally worrying, and invisible in aggregate unemployment statistics. Worth stating as the authors state it: this is an early indicator, not proof that the ladder has closed.

Copyright

It is tempting to read a doctrine out of these, and premature. As of August 2026 US law here is unsettled and intensely fact-specific: every case above is a district-court decision or a settlement, not an appellate ruling, and Thomson Reuters v. Ross concerned a competing legal-search product rather than a generative model at all. What the rulings do consistently separate is three different questions — how the corpus was acquired, whether training on it is transformative, and whether the outputs substitute for the original — and it is worth separating them yourself when you read a headline that does not. The Anthropic settlement works out near $3,000 a book, which sets no precedent and establishes no market price; a settlement is what two parties agreed, not what a court held. Europe is moving faster and less favourably to the labs.

Energy and water

The IEA puts 2024 global data centre electricity consumption at 415 terawatt-hours, around 1.5% of world electricity, growing about 12% a year — more than four times faster than total electricity demand. Their more recent update gives roughly 485 TWh for 2025 and projects around 950 TWh for 2030, slightly more than Japan’s entire consumption today. The United States accounts for 45% of the 2024 total, China 25%, Europe 15%.

On water the estimates diverge by orders of magnitude, and it is worth knowing why. Academic work put GPT-3’s training water consumption near 700,000 litres for on-site cooling alone — and nearer 5.4 million litres once the water used to generate the electricity is counted, which is the single clearest illustration of why these numbers disagree. The same work put 10 to 50 medium responses at about half a litre. Sam Altman put an average query at 0.32 millilitres; Google put a median Gemini text prompt at about 0.26 millilitres — five drops. Both vendor figures are self-reported and draw the boundary narrowly — inference only as a rule, usually on-site, and often excluding the generating plant. One is a median, others are estimates. Neither set is dishonest, and they should not be compared as though they were the same measurement. They are measuring different things, and almost nobody says which.

The safety argument has three sides, not two

Geoffrey Hinton left Google on 1 May 2023 to speak freely, and the coverage flattened something worth restoring: on the same day he clarified that he left so he could talk about the dangers without considering the impact on Google, and that Google “has acted very responsibly”. He did not quit in order to criticise his employer. By December 2024 he put the chance of AI causing human extinction within three decades at 10 to 20 per cent.

Governance moved in the opposite direction from the rhetoric. Twenty-eight countries plus the EU signed the Bletchley Declaration in November 2023. By the Paris summit in February 2025, the United States and the United Kingdom declined to sign the final declaration. The EU AI Act entered into force on 1 August 2024 with a systemic-risk threshold written as a number of floating-point operations — 10²⁵ — which is itself a remarkable artefact of this era: a law with a FLOP count in it. It is a rebuttable presumption rather than a bright line: a provider can argue its model does not have high-impact capability despite crossing it, and the Commission can designate a model that has not.

Eight things almost everyone gets wrong

None of these corrections make the history duller. Most of them make it sharper.

A guide that repeats the tidy version of every story is pleasant and useless. Here are the eight most-repeated claims in this history that do not survive contact with the sources — including one that the archive holding the manuscript gets wrong on its own website.

Three ways to read the whole thing

Three ways to hold twelve hundred years in your head at once. They are not competing; they are different cuts through the same material.

1. 1

   The bottleneck story

   Every jump removed one specific constraint. Fixed word vectors became contextual ones. A fixed sentence vector became attention. Sequential computation became parallel. Task-specific training became in-context learning. Capable-but-useless became RLHF. Undertrained giants became Chinchilla-optimal. One-shot answers became test-time compute. Dense compute became sparse. Each unlock looks obvious in retrospect and was invisible beforehand — which is worth remembering before assuming the current bottleneck is permanent.

2. 2

   The credit story

   Bahdanau invented attention and Bengio named it in a final edit. Google invented the Transformer and lost all eight authors. OpenAI stopped publishing what it built, citing the competitive landscape in the same breath as safety. DeepSeek published everything and lost the narrative to a number in a table. Lovelace wrote the larger idea and is argued about for the smaller one. The people who get remembered are rarely the people who did the thing.

3. 3

   The two-clock story

   Capability moves on a fast clock: GPQA went from 36% to 95% in three years, and a cheap model now holds the record. Institutions move on a slow one: the first US merits ruling against AI training came in February 2025, five years after that case was filed and eight years after the Transformer. The interesting tension in 2026 is not human against machine. It is the widening gap between what these systems can do and what anyone has actually decided about them.

Shannon stopped at four letters of context because the labour became enormous. The whole of the last seventy-eight years is the story of removing that constraint — and the thing on the other side of it still cannot reliably tell you where a person was born. Both halves of that sentence are true, and any account that gives you only one of them is not describing the technology. It is selling a position on it.