How transformers actually work — the architecture behind every modern AI

You’ve probably used a transformer today. ChatGPT, Claude, Gemini, GitHub Copilot, Google Translate — every one of them is built on the same foundational architecture, first described in a 2017 paper titled “Attention Is All You Need.” It’s one of the most influential pieces of computer science research in decades. But what does a transformer actually do?

The problem transformers solved

Before transformers, the dominant approach to processing language was recurrent neural networks (RNNs). An RNN reads a sentence word by word, left to right, maintaining a kind of rolling memory of what it’s seen so far. The problem is that this memory fades. By the time an RNN reaches the end of a long paragraph, the information from the beginning has been diluted through dozens of processing steps. Long-range dependencies — like connecting a pronoun at the end of a sentence to the noun it refers to at the beginning — were genuinely difficult.

Transformers abandoned the sequential, word-by-word approach entirely.

Attention: looking at everything at once

The core mechanism of a transformer is called self-attention. Instead of processing words in order, the model looks at every word in the input simultaneously and figures out which words should pay attention to which other words.

Take the sentence: “The animal didn’t cross the street because it was too tired.”

What does “it” refer to — the animal or the street? For a human, this is obvious. For a machine, it requires connecting the pronoun to a noun that might be many positions away. Self-attention solves this by computing, for every word, a score of how relevant every other word in the sentence is to understanding it. “It” ends up attending strongly to “animal” and weakly to “street.”

These attention scores are calculated using three learned vectors for each word: a Query (what am I looking for?), a Key (what do I offer?), and a Value (what information do I actually contain?). The similarity between a Query and all the Keys in the sequence determines how much attention is paid to each word’s Value.

Multiple heads, multiple perspectives

A transformer doesn’t run attention just once — it runs it several times in parallel, each with different learned weights. These are called attention heads. One head might learn to track grammatical agreement between subjects and verbs. Another might learn to connect pronouns to their referents. Another might focus on sentiment. Each head sees the same input but learns to look for something different. Their outputs are combined.

This is called multi-head attention, and it’s what gives transformers such rich, nuanced representations of language.

Layers, feedforward networks, and depth

After the attention mechanism, each position in the sequence passes through a simple feedforward neural network — the same network applied to every position independently. Then the output feeds into the next layer, which runs attention again, and then another feedforward network, and so on.

Modern large language models stack dozens or hundreds of these layers. GPT-4 is estimated to have around 120. Each layer refines the representation — early layers tend to capture surface-level patterns, while deeper layers capture abstract reasoning and semantic relationships.

Positional encoding: putting words back in order

Since transformers process all positions simultaneously, they don’t inherently know that word 3 comes before word 7. To preserve order information, a positional encoding is added to each word’s representation before processing begins — a mathematical signal that encodes position. This is how the model knows “the dog bit the man” and “the man bit the dog” are different, even though they contain the same words.

Why transformers scale so well

One of the most important properties of transformers — and the reason they’ve come to dominate AI — is that they scale predictably. The more parameters and compute you throw at them, the better they get, in a remarkably consistent way. This observation, sometimes called scaling laws, was documented by OpenAI researchers in 2020 and triggered an arms race in model size that’s still ongoing.

RNNs never showed this property clearly. Transformers did, and that made them the architecture of choice for any serious language model — and increasingly for vision, audio, and multimodal systems as well.

The limits

Transformers aren’t without problems. The attention mechanism’s computational cost scales quadratically with sequence length — doubling the input roughly quadruples the compute required. This makes very long context windows expensive, which is why enormous effort has gone into alternatives like sparse attention, sliding window attention, and state-space models (like Mamba) that try to preserve transformer-level quality at lower cost for long sequences.

There’s also the question of whether attention is really “understanding” in any meaningful sense, or a sophisticated pattern-matching system that has learned to fake it at scale. That debate continues — and it’s one of the most interesting open questions in AI research today.