What Are Transformers, and Why Did They Make AI So Much More Capable?

The autocomplete you already trust

You type "see you" on your phone and it suggests "later." You start an email with "thanks for" and it offers "your time yesterday." Nobody finds this surprising any more. Predictive text has been guessing your next word for over a decade, and Gmail's Smart Reply was doing the same thing to whole sentences as far back as 2015, suggesting three short replies to an email before you'd read past the subject line.

That's the same underlying idea that now sits behind ChatGPT, Claude, and every large language model your business is evaluating: predict the next likely word, given what came before. Between 2015 and today, the idea stayed exactly the same. What grew was how much of "what came before" a model could actually use when making that prediction. That difference is the whole story, and it has a name: the transformer.

The problem with reading one word at a time

Before 2017, the leading approach to language models read text the way you'd read through a keyhole: one word at a time, left to right, carrying forward a compressed memory of everything seen so far. Google's 2014 sequence-to-sequence research, the technique behind the first neural version of Google Translate, worked exactly this way. It read a French sentence word by word, squeezed the whole thing into a fixed-size summary, then generated the English translation word by word from that summary.

Two problems came with it. The first was forgetting. By the time the model reached the end of a long sentence, its memory of the beginning had been diluted or overwritten, in the same way it's hard to hold an entire paragraph in your head when you've only ever seen it one word at a time. The second was speed. Word ten couldn't be processed until word nine had been, which couldn't be processed until word eight had been, and so on. That strict ordering meant training was slow and difficult to scale onto hardware built to do vast numbers of calculations at once.

What attention actually does

In June 2017, Google researchers published a paper called "Attention Is All You Need", proposing an architecture built entirely around a different idea: instead of reading a sentence in order and trying to remember it, read the whole thing at once and work out which words matter most to which other words, regardless of how far apart they sit. Nothing has to be carried through a bottleneck, because everything is visible in a single pass. They called it the transformer.

Take the sentence "I took the fish out of the river and put it on the bank." Then take "I took my card to the bank." Same word, two completely different meanings. A model reading word by word has to guess at "bank" before it's seen enough of the sentence to know which one is meant. A model using attention weighs "river" and "fish" heavily when it reaches "bank" in the first sentence, and weighs "card" heavily in the second, because it can see the whole sentence at once and score every word against every other word.

Or take the classic example researchers use to stress-test this: "The trophy didn't fit in the suitcase because it was too big." Swap "big" for "small" and the answer to "what does 'it' refer to?" flips entirely, from the trophy to the suitcase. Resolving that correctly means weighing a word against another word several places away, not just the word sitting next to it. That's what attention is built to do, and it's a large part of why models built on it made far fewer of the basic comprehension errors their predecessors were prone to.

A useful way to picture it: word-by-word reading is like a note passed around a circle, each person adding a little and passing it on, so that by the time it reaches the far side a lot has been lost or garbled. Attention is a round-table meeting where anyone can address anyone else directly, so whoever is summarising at the end has heard everything first-hand.

Why reading in parallel mattered more than reading well

The comprehension gains from attention would have mattered on their own. What made them compound was a second consequence of the same design: because attention doesn't need word nine processed before word ten, the calculations for every word in a sentence can run at the same time rather than in strict sequence. That's what let these models be trained in parallel, on hardware built for exactly that kind of simultaneous computation, at a scale the old word-by-word approach never could have reached in the same timeframe.

That's the direct, unglamorous cause of the jump from GPT-1 in June 2018, a modest 117 million parameters, to GPT-3 in May 2020, at 175 billion. Parallel training is why that jump happened inside two years rather than a decade.

The ten-year run-up, dated

The transformer didn't appear from nowhere in 2017. It's the product of roughly a decade of research and product work, most of it out of Google, with two other moments that shifted how the industry and the public thought about what deep learning could do.

Two points on that table need stating precisely. AlphaGo is reinforcement learning and tree search, a different lineage from word2vec through to ChatGPT, and its contribution here is the belief shift it caused in boardrooms and research funding, not a technical link to the transformer itself. Attention was already running in production inside Google Translate by September 2016, nine months before the 2017 paper. What that paper showed was that attention on its own, with no word-by-word reading at all, outperformed the older approach.

What this means if you're running an AI transformation

The practical consequence of all this for a business leader sits in one phrase you'll hear constantly from vendors: context window. That's the amount of text a model can hold in view and attend across in a single pass, and it's the direct descendant of the mechanism above. A small context window means the model is, in effect, back to reading through a keyhole once your document, conversation history, or product catalogue runs past a certain length. A large one means it can genuinely weigh your entire contract, your entire support ticket history, or your entire policy manual against each other at once.

The quality of what you feed a model carries the same weight as which model you pick. Attention lets a model weigh everything it's given, but it can only weigh what's actually in front of it. A model given your real returns policy will attend to your real returns policy. A model given nothing will generate a plausible-sounding one instead. Feeding a model the specific, relevant material it needs before it answers, rather than relying on what it happened to absorb during training, is the problem Retrieval-Augmented Generation exists to solve.

Which model you use for which job is a separate decision, covered in detail in our guide to the current LLM landscape, but the underlying mechanism is the same across all of them. Every model on that list, GPT, Claude, Gemini, Llama, is a transformer, and the differences between them sit in scale, training data, and fine-tuning rather than in the basic trick of reading everything at once.

Where to go from here

The architecture is now eight years old and every frontier model, including GPT-4 and Anthropic's Claude, both launched the same day in March 2023, is built on some variant of it. None of that requires understanding the mathematics. When someone in your organisation asks for a bigger context window, argues for a RAG system over fine-tuning, or asks which model to standardise on, that history is what they're actually drawing on. The two linked guides above take the retrieval and model-selection decisions further.