Autoregressive Transformer Language Models

A GPT-style language model is best understood as a causal Transformer over one growing token sequence. The prompt is not normally encoded by one network and decoded by another: prompt tokens and generated tokens pass through the same stack, while a causal mask prevents each position from seeing future tokens. ChatGPT adds product behavior—conversation roles, post-training, safety policy, tools, and an interface—around a model, so “how ChatGPT works” has more than one layer.

The probability model is a chain of next-token decisions #

For tokens $x_1,\ldots,x_T$, the chain rule writes the sequence probability as

$$p(x_{1:T})=\prod_{t=1}^{T}p(x_t\mid x_{<t}).$$

The model learns one conditional distribution: given every token before position $t$, assign probabilities to the token at $t$. Training minimizes negative log-likelihood, usually called next-token cross-entropy:

$$\mathcal L(\theta)=-\sum_{t=1}^{T}\log p_\theta(x_t\mid x_{<t}).$$

This simple objective is broad because text can serialize many tasks. A question followed by an answer, source code followed by a function, or an article followed by a summary are all prefixes and continuations. GPT-2 demonstrated that a sufficiently large model trained this way could exhibit some task behavior without a new task-specific output layer.

Tokens become position-aware vectors #

Raw text is first split into tokens from a fixed vocabulary. Tokens may be whole common words, word pieces, punctuation, or byte-level fragments. Tokenization is reversible, but token boundaries need not match linguistic words.

Each token id selects an embedding vector. A position representation is added or otherwise incorporated so that the same token at positions 5 and 500 is distinguishable:

$$h_t^{(0)}=E_{\text{token}}[x_t]+E_{\text{position}}[t].$$

The exact tokenizer and positional mechanism are model-specific. GPT-2 used byte-level BPE and learned position embeddings; later systems may use different tokenizers and relative or rotary position mechanisms. Architecture names alone do not reveal those choices.

Conversation roles are also model inputs. Current OpenAI text-generation APIs expose ordered developer, user, and assistant messages. The service converts that structured request into model context, but the public API contract does not promise a particular hidden token serialization. Treat role formatting as a product/model interface, not a second semantic encoder.

A causal Transformer block mixes only the available prefix #

At each layer, a token representation is projected into a query, key, and value. For one attention head,

$$Q=HW_Q,\qquad K=HW_K,\qquad V=HW_V,$$

$$\mathop{\mathrm{Attention}}(Q,K,V)= \mathop{\mathrm{softmax}}\left(\frac{QK^T}{\sqrt{d_k}}+M\right)V.$$

For causal language modeling, mask $M$ contains $-\infty$ wherever key position $j$ is later than query position $i$. After softmax, those forbidden weights are zero:

$$M_{ij}=\begin{cases} 0 & j\le i,\\ -\infty & j>i. \end{cases}$$

For a four-token training sequence, the visibility pattern is lower-triangular:

Multiple heads perform learned routing in different subspaces. A position-wise MLP then transforms each position independently. Residual connections preserve an additive information path, while normalization stabilizes deep composition. A schematic block is therefore

$$h' = h + \mathop{\mathrm{MaskedAttention}}(\mathop{\mathrm{Norm}}(h)),$$

$$h^{\text{next}} = h' + \mathop{\mathrm{MLP}}(\mathop{\mathrm{Norm}}(h')).$$

Normalization placement varies by model family. The durable mechanism is the combination of causal content routing, per-position nonlinear transformation, residual paths, and repeated depth.

Parallel training, sequential generation #

The causal mask prevents leakage but does not force training to loop over tokens. Given a complete training sequence, the model can compute hidden states and losses for all positions in parallel because the correct preceding tokens are already present. This is teacher forcing at the token level.

Generation is different. The next token does not exist yet, so inference must repeat:

tokens = tokenize(prompt)
while not stop(tokens):
    logits = model(tokens)[-1]
    next_token = select(softmax(logits))
    tokens.append(next_token)

Real serving systems cache the keys and values computed for earlier tokens, avoiding a full recomputation of the prefix at every step. The new token still depends on the existing cache, so generation remains sequential across output positions.

Hidden states become a vocabulary distribution #

After the final block, the hidden state at each position is projected to one logit per vocabulary token. GPT-2's reference implementation ties this projection to the input token-embedding matrix:

$$z_t=h_tE_{\text{token}}^T, \qquad p(x_{t+1}=v\mid x_{\le t})=\mathop{\mathrm{softmax}}(z_t)_v.$$

Selection turns the distribution into a token. Greedy decoding chooses the maximum; sampling draws according to probabilities; temperature rescales logits; top-$k$ or top-$p$ truncates the candidate set. These procedures change diversity and error shape, not the knowledge encoded in the weights.

The chosen token is appended and becomes ordinary context for the next step. A response that reads like a paragraph is therefore the result of many conditional decisions, not one hidden sentence emitted whole.

Encoder-only, decoder-only, and encoder–decoder are different masks and paths #

The 2017 Transformer paper describes the third row. GPT-2 is evidence for the second. Calling both “Transformers” identifies their building block, not their entire information flow.

An encoder–decoder model keeps source and target paths structurally separate. A decoder-only model instead serializes conditioning material and desired output into one sequence. The boundary between prompt and response can still be marked by special tokens or role metadata, but both sides use the same causal stack.

Pretraining does not by itself create a good assistant #

Next-token pretraining rewards likely continuation, not obedience to a user, calibrated uncertainty, refusal of harmful requests, or concise explanation. Web text contains all of those behaviors and their opposites. A base model may therefore continue the apparent genre instead of following the instruction.

The published 2022 InstructGPT pipeline shows one way to reshape behavior while retaining the underlying GPT architecture:

1. labelers write good responses and the model receives supervised fine-tuning;
2. labelers rank alternative responses and a reward model learns those preferences;
3. PPO fine-tunes the policy against the learned reward, with constraints that limit drift from the pretrained model.

This distinction explains why two models with the same architecture can behave very differently. Architecture defines the computation graph; pretraining builds broad predictive capability; post-training changes which capabilities and response styles are selected in interaction.

Human feedback is not a neutral oracle. The InstructGPT authors explicitly note that the result reflects a particular set of labelers, researcher instructions, customer prompts, and aggregation choices. Better preference scores also did not eliminate fabrication, bias, harmful compliance, or capability regressions.

ChatGPT is a system boundary, not one published neural-network diagram #

The public facts are narrower than many architecture diagrams imply. OpenAI's current documentation describes language models as generating text from prompts; exposes developer, user, and assistant messages; and lists models with text/image inputs, text outputs, and optional tools. The chat-latest reference says that alias points to the current ChatGPT Instant model and that its underlying snapshot is updated regularly.

From those facts, it is reasonable to infer a layered system:

1. a model consumes the prepared context and generates model output;
2. product logic manages conversation state, roles, limits, safety behavior, and model selection;
3. tool orchestration may execute web search, file search, code, image generation, or other external operations and return results to the model;
4. the interface renders the resulting conversation.

That is a system-level inference, not proof of an undisclosed internal architecture. Official product documentation does not specify every current ChatGPT model's layer count, attention variant, routing structure, tokenizer, training corpus, or post-training algorithm. The exact model may also change behind a product alias.

Tools are especially important to separate from model weights. A web-search result can supply current evidence in the context; it does not mean the neural network updated its parameters during the conversation. Likewise, conversation history is context supplied to inference, not permanent learning by default.

Why fluent models hallucinate #

The loss asks, “Which continuation fits this prefix?” It does not directly ask, “Which claim has been verified against reality?” If training data contains a pattern such as a confident biography, the model can reproduce the form with an invented name or date when its conditional distribution supports it.

Three boundaries follow:

• parametric memory is lossy: weights compress statistical regularities rather than store a queryable copy of every source;
• probability is not provenance: a high-probability token need not have a traceable source;
• post-training changes tendencies, not guarantees: preference optimization can reduce some errors while introducing hedging, sycophancy, reward gaming, or capability trade-offs.

Retrieval and tools can ground an answer in external evidence, but only if the system invokes them, receives relevant material, and faithfully uses it. For important claims, inspect sources rather than treating fluency or confidence as verification.

A practical mental checklist #

When someone says a system is “a Transformer,” ask four separate questions:

1. Information path: bidirectional encoder, causal decoder, encoder–decoder, or a mixture/routed variant?
2. Training objective: next-token likelihood, masked reconstruction, preference optimization, or several stages?
3. Inference system: plain generation, retrieval, tools, memory, routing, or safety layers?
4. Disclosure boundary: which details are documented for this exact model, and which are analogies from older public models?

For GPT-style generation, the safest default mental model is: tokenize one context, run a causal Transformer, obtain next-token logits, select one token, append it, and repeat. Everything beyond that should be attached only when a model-specific source supports it.