Unchanged for Nine Years: Dissecting the Eight Core Components of the Transformer Engine

Introduction: An Engine Unchanged for Nine Years

In 2017, a seminal paper titled Attention Is All You Need introduced an entirely new neural architecture. It abandoned the prevailing recurrent neural network (RNN) paradigm of reading tokens sequentially, relying instead solely on an “attention mechanism” to process an entire sequence of text in parallel. In the nine years since, artificial intelligence has witnessed multiple waves of breakthroughs: parameter counts have surged from tens of millions to hundreds of billions, training corpora have expanded from a few gigabytes to nearly the entire public web, and deployments have transitioned from laboratory demonstrations into mission-critical production systems.

Yet beneath the hood, the core engine driving almost every state-of-the-art language model has barely swapped a single fundamental part.

If we compare a large language model (LLM) to an engine, taking off the casing reveals eight foundational components assembled along a fixed pipeline. Understanding these eight parts provides the key to deciphering model behavior: why it retains context, why training demands massive compute, and why generation unfolds token by token. Below is a step-by-step breakdown tracing the exact flow of data through the model.

---

The Processing Pipeline: What Happens to a Sequence of Text

From the moment text enters the model until a response token is produced, data moves through a well-defined sequence:

Raw Text → Tokenization → Vector Embedding → (Positional Encoding Addition) → Multi-Layer Stack of “Attention + Feed-Forward” Blocks → Logit Distribution → Next-Token Sampling

Every layer in the multi-layer stack consists of two primary operational components—attention and a feed-forward network—stabilized by residual connections and layer normalization. Let us break down each part in order.

---

Component 1: Tokenization and Embedding — The Digital Ticket for Text

Neural networks cannot process raw human language; they only understand numbers. Translating human text into numeric form occurs in two stages.

Stage 1: Tokenization

A tokenizer slices input text into discrete chunks called tokens. For example, the English sentence “The cat sat on the mat” is split into roughly six tokens. Longer or compound words are broken down into subword units—such as splitting “unbelievable” into “un”, “believ”, and “able”. This subword strategy controls the vocabulary size while retaining the ability to represent unseen or rare words. Tokenization determines the model’s perceptual resolution: splitting too coarsely makes rare words intractable, whereas splitting too finely creates overly long sequences and fragments semantic coherence.

Stage 2: Embedding

Each discrete token ID is mapped to a dense, continuous vector of real numbers via a lookup table. The embedding matrix has a row count equal to the vocabulary size and a column count equal to the model’s hidden dimension ($d{model}$). The token ID simply serves as the row index. Through training, these vectors adjust continuously to become semantic coordinates: words with similar meanings cluster together in geometric space, and vector arithmetic can capture relationships such as $\vec{v}{\text{King}} - \vec{v}{\text{Queen}} \approx \vec{v}{\text{Man}} - \vec{v}_{\text{Woman}}$.

A single metric illustrates the scale change over nine years: the original 2017 base Transformer used an embedding dimension of 512. Modern production models frequently scale to 4,096 dimensions or higher. The vectors have broadened eightfold, and vocabularies have expanded from tens of thousands to hundreds of thousands of tokens. However, the core operation—retrieving a vector from an index table—remains identical.

---

Component 2: Positional Encoding — Assigning Order to Unordered Vectors

The embedding layer suffers from an inherent limitation: it records which words appear, but not where they appear. The phrases “the cat chased the mouse” and “the mouse chased the cat” produce the exact same set of token embeddings. Without additional intervention, the model cannot distinguish between them.

Because self-attention operates as a set-based operation over vectors, it is naturally permutation-invariant. Positional information must be explicitly injected. The original paper introduced an elegant, non-learned method: adding fixed vectors generated by sinusoidal functions of varying frequencies directly to the token embeddings. High-frequency dimensions capture relationships between adjacent tokens, while low-frequency dimensions distinguish distant tokens. This mathematical formulation also enabled extrapolation to sequence lengths unseen during training.

Over the past nine years, positional schemes have evolved: from learned absolute positional embeddings to relative positional encodings, and ultimately to rotary position embeddings (RoPE), which inject position dynamically via complex rotations in the attention stage. While the mathematical formulations have been refined, the objective has remained unchanged: assigning sequence order to unordered vectors.

---

Component 3: Q, K, and V Projections — Preparing Three Representations per Token

Once positional signals are incorporated, the model prepares each token for contextual interaction by projecting it into three distinct representations: Query ($Q$), Key ($K$), and Value ($V$).

All three vectors originate from the same token embedding via independent linear transformations, each multiplied by a learnable weight matrix ($W_Q, W_K, W_V$):

• Query ($Q$): Acts as a search query, representing “what information I need from the context.”
• Key ($K$): Acts as an index tag, representing “what type of information I hold.”
• Value ($V$): Acts as the underlying content, representing “the actual semantic payload to extract if you attend to me.”

These projection matrices are optimized entirely through backpropagation without manual rules. As the model trains on extensive corpora, it autonomously learns how to encode requirements into $Q$ and capabilities into $K$. Empirical studies have shown that individual attention heads naturally specialize—some handle syntax parsing, others track coreference, and some monitor positional boundaries.

---

Component 4: Scaled Dot-Product Scoring and Causal Masking — Determining Who Attends to Whom

With the materials prepared, the model calculates interaction scores. The mechanism is straightforward: take the Query vector of a given token and compute the dot product with the Key vectors of all tokens in the sequence. A higher dot product signifies greater relevance.

A critical detail is the scaling factor: the raw dot product is divided by $\sqrt{d_k}$ (the square root of the key dimension). As vector dimensionality increases, the variance of the dot product grows, pushing values into extreme ranges. When passed into a softmax function, these large values cause gradients to saturate near zero, stalling optimization. Scaling bounds the scores within a stable numerical window, ensuring smooth gradient flow during early training stages.

After scaling, a softmax operation normalizes each row into a probability distribution: positive values that sum to one, representing how much attention the current token allocates to every other token.

In generative models, an additional mechanism is essential: causal masking. When predicting the $N$-th token, the model must not access tokens at position $N+1$ and beyond. This constraint is enforced by populating the upper triangle of the score matrix with $-\infty$. After the softmax function is applied, these masked positions drop to zero attention weight, restricting information flow strictly to past tokens.

def scaled_dot_product_attention(Q, K, V, mask=None):
    d_k = K.shape[-1]
    scores = matmul(Q, K.T) / sqrt(d_k)   # Compute attention scores and apply scaling
    if mask is not None:
        scores = scores + mask            # Causal mask: set future positions to -inf
    weights = softmax(scores, axis=-1)    # Normalize into attention distribution
    return matmul(weights, V)             # Weighted aggregation (see Component 5)

---

Component 5: Weighted Summation — Re-mixing Contextual Information

The attention weights derived from scoring are applied directly to the Value ($V$) vectors. By computing the weighted sum of all Value vectors across the sequence according to these weights, the model constructs a newly contextualized representation for each token.

This step is sometimes simplified as a “weighted average,” but it represents a fundamental semantic transformation. The resulting vector is no longer an isolated representation of a single word; it is an aggregated synthesis of the entire context based on relevance. For example, the token “Apple” will have its representation shifted toward either agricultural concepts or consumer technology, determined entirely by contextual attention weights.

With every attention layer traversed, sequence representations are re-mixed and abstracted further: lower layers focus on surface-level syntax and grammar, middle layers handle sentence structure and coreference, and deeper layers resolve high-level semantics, reasoning, and factual relationships.

---

Component 6: Multi-Head Attention — Parallel Perspectives Aggregated into One

Using a single set of $Q, K, V$ projections limits the model to observing the sequence from a single perspective. However, understanding language requires monitoring multiple linguistic facets simultaneously—such as grammatical dependencies, semantic references, and chronological order.

To resolve this, the hidden dimension is divided across multiple “heads,” each executing an independent attention calculation in parallel. This can be compared to a panel of distinct reviewers analyzing a draft: one verifies factual consistency, another checks grammatical structure, and a third evaluates narrative flow. Each head maintains independent $W_Q, W_K, W_V$ matrices and develops distinct attention patterns during training.

To merge these perspectives, the outputs of all attention heads are concatenated and passed through a final linear projection matrix ($W_O$) to map them back to the model’s base dimension, synthesizing multiple distinct viewpoints into a unified vector.

---

Component 7: Residual Connections and Layer Normalization — Supporting Deep Stacks

As Transformer stacks grow deeper, training stability becomes a primary engineering hurdle: gradients can degrade or explode across dozens of layers. Two complementary mechanisms maintain signal stability across the network.

• Residual Connections: These create shortcut pathways around each sublayer (attention and feed-forward):
  $$\text{Output} = \text{Input} + \text{Sublayer}(\text{Input})$$
  Even if a specific sublayer learns an ineffective transformation, gradients can still flow unobstructed through the identity path, allowing dozens or hundreds of layers to be stacked without architectural degradation.
• Layer Normalization: This stabilizes the distribution of activations across feature dimensions for each individual token, maintaining mean and variance within controlled bounds to prevent numerical instability. While modern architectures often adjust the placement (such as shifting from Post-LN to Pre-LN or using RMSNorm), the underlying principle—preserving gradient paths and bounding activation distributions—remains foundational.

---

Component 8: Feed-Forward Networks and the Output Head — From Understanding to Next-Token Prediction

While attention mixes information across sequence tokens, the model requires a stage to process and store factual knowledge per token position. This is the role of the Feed-Forward Network (FFN).

Applied independently to each token, the FFN consists of a two-layer multi-layer perceptron (MLP): it first expands the hidden dimension (typically by a factor of four), applies a non-linear activation function, and then projects the vector back to the original dimension. This expansion-and-compression bottleneck is widely regarded as the primary repository for parametric memory in large language models. The original 2017 Transformer expanded from 512 dimensions to 2,048; modern models expand from 4,096 dimensions to tens of thousands, preserving the original geometric ratio.

The final component is the Output Head. After passing through all stacked Transformer blocks, the final representation reaches an unembedding linear layer. This layer projects the vector back into a space the size of the vocabulary, producing a raw score (logit) for every candidate token. This projection matrix often shares weights directly with the input embedding table (weight tying).

Finally, logits are converted into a probability distribution via softmax. Rather than deterministically choosing the highest-probability token every time, inference engines apply sampling strategies (such as temperature scaling, top-$k$, or top-$p$ nucleus sampling) to balance determinism and diversity. The selected token is appended to the input sequence, and the entire sequence is fed back into the pipeline to predict the subsequent token—repeating autoregressively until an end-of-sequence token is generated.

---

Conclusion: The Scale Has Changed, but the Skeleton Remains

Tracing the pipeline reveals a clean sequence: vector lookup, positional injection, triple linear projections, scaled dot-product scoring, weighted summation, multi-head aggregation, residual normalization, feed-forward expansion, and logit projection.

Over the past nine years, parameter scales have grown by orders of magnitude, context windows have expanded from hundreds of tokens to hundreds of thousands, and training infrastructure has grown from single workstations to clusters of thousands of accelerators running for months. Yet the underlying assembly of these eight components has remained virtually untouched.

For practitioners—whether designing training pipelines from scratch or integrating enterprise endpoints on platforms such as Baidu AI Cloud—large language models are not inscrutable black boxes. Strip away the scale, and what remains is an engine of eight interconnected parts, operating precisely on the mathematical foundations laid out nine years ago.