Don't Panic — Transformer Models

A transformer is a neural-network architecture for finding useful relationships inside a sequence. It became famous through language models, but text is only one guest at this particular party. Image patches and audio frames can also arrive as a sequence, provided someone has given them labels and seats.

Earlier recurrent models read tokens in a line, carrying a summary forward like a relay runner with an increasingly alarming amount of luggage. Transformers use self-attention instead. Each token can compare itself with every other token directly, so a later word can connect with an earlier one without waiting for every intermediate step to preserve the message.

The comparison has three names: query, what a token seeks; key, what a token offers; and value, the information that can be carried forward. The resulting relevance scores decide how values are blended into a new token representation. Several such comparisons run at once in multi-head attention, because a single view of a sentence would be a remarkably bold career choice for a neural network.

There is a catch. Attention alone knows relationships, but not order. A token at the beginning and the same token later in a sequence look alike until positional encoding supplies location information. Modern models often use rotary positional encoding, which changes query and key vectors according to position. This is why word order did not vanish into a very expensive bag of tokens.

A full transformer stacks blocks. Each block combines multi-head attention, a feed-forward network, residual connections, and layer normalization. The familiar family split follows from how those blocks are arranged: encoder-only models build representations for understanding, decoder-only models generate one token at a time, and encoder-decoder models turn a full input into another sequence. The names are less mystical than they first appear, which is fortunate because there are already enough mysterious names in machine learning.

Start with the Intro for the architecture and its limits. Use Slides when the query-key-value flow needs a compact map, and keep the Cheatsheet nearby when comparing the three architecture families. The practice reference then turns one attention row into numbers you can inspect, while the exercise asks you to show exactly what changes when a key or position changes.

Don't Panic — Transformer Models

Relevant careers