Modes of Operation & AEAD (GCM)

AES encrypts exactly one 128-bit block. A mode of operation is the construction that turns that primitive into something that encrypts a message — and essentially every famous "encryption bug" of the last twenty years was a mode bug, not a cipher bug. The modern answer is AEAD: one construction that provides confidentiality and integrity together, because the decades-long experiment of providing them separately failed repeatedly and in the same ways.

The unauthenticated modes, and why they lost #

ECB — not a mode, a mistake #

Encrypt each block independently. Identical plaintext blocks produce identical ciphertext blocks, so structure survives encryption intact — the "ECB penguin" image is legible after encryption. It fails IND-CPA by inspection: it is deterministic. There is no correct use of ECB for messages.

CBC — chaining, and the padding problem #

$$C_i = E_K(P_i \oplus C_{i-1}), \qquad C_0 = IV$$

CBC is IND-CPA secure if the IV is unpredictable and random per message. Two structural problems ended it:

CBC is also strictly sequential when encrypting (each block needs the previous ciphertext), so it cannot use instruction-level parallelism on the encrypt path.

CTR — a stream cipher made from a block cipher #

$$C_i = P_i \oplus E_K(\text{nonce} \,\|\, \text{counter}_i)$$

The block cipher generates a keystream; the message is XORed with it. This is a good construction: no padding (so no padding oracle), fully parallelizable and seekable in both directions, and only the encrypt function of the cipher is needed.

It inherits the stream-cipher hazard exactly: reusing a (key, nonce) pair reproduces the keystream, and $C_1 \oplus C_2 = P_1 \oplus P_2$ — the two-time pad. And, being XOR-based, it is perfectly malleable: flipping a ciphertext bit flips that plaintext bit. Unauthenticated CTR lets an attacker make targeted, predictable edits to the plaintext.

Encryption is not integrity #

That last point generalizes. Confidentiality does not imply integrity, and shipping unauthenticated ciphertext to a system that acts on the plaintext is a vulnerability. The historical orderings:

AEAD #

An AEAD scheme takes a key, a nonce, a plaintext, and associated data, and returns ciphertext plus an authentication tag:

$$\text{Seal}(K, N, P, A) \rightarrow (C, T), \qquad \text{Open}(K, N, C, T, A) \rightarrow P \ \text{or}\ \bot$$

Associated data is authenticated but not encrypted — for headers that must be readable in transit (routing information, packet sequence numbers, record type) but must not be tamperable. Getting this into the primitive is what stops the recurring bug of encrypting the body while leaving the header forgeable.

GCM in detail #

GCM = CTR mode for confidentiality + GHASH for authentication. GHASH is a polynomial evaluation MAC in $GF(2^{128})$ (see finite fields) under the hash subkey $H = E_K(0^{128})$:

$$\text{GHASH}(H, X_1..X_n) = \sum_{i=1}^{n} X_i \cdot H^{\,n-i+1} \quad \text{in } GF(2^{128})$$

$$T = \text{GHASH}(H, A \,\|\, C \,\|\, len) \oplus E_K(N \,\|\, 0^{31}1)$$

flowchart LR
  K["Key"] --> H["H = E_K(0^128)"]
  N["Nonce ‖ ctr"] --> CTR["CTR keystream"] --> X((XOR))
  P["Plaintext"] --> X --> C["Ciphertext"]
  A["Associated data"] --> G["GHASH_H"]
  C --> G
  G --> T2((XOR)) --> T["Tag"]
  N --> J0["E_K(N ‖ 0..01)"] --> T2

GCM won on performance: GHASH is one field multiplication per block, and since 2010 x86 has had PCLMULQDQ (carry-less multiply) alongside AES-NI, so AES-GCM runs at several GB/s per core and is fully parallelizable and constant-time in hardware.

Nonce discipline for GCM, concretely:

• 96-bit nonces only. Other lengths get run through GHASH first, which is slower and removes the clean structure.
• Use a counter, not the RNG, whenever a sequence number exists (TLS derives the nonce from the record sequence number — no randomness, no collisions).
• If you must randomize, 96 random bits gives a birthday collision around $2^{32}$ messages per key — rekey well before that.
• Never let the nonce come from the message or from user input, and never persist a counter without a durable write, or a crash-restart replays nonces.
• Rekey on any state restore — VM snapshot rollback replays counters.

Choosing a mode today #

AES-GCM-SIV (RFC 8452) deserves the callout: it derives the keystream from a synthetic IV computed over the plaintext, so repeating a nonce leaks only that two identical messages were sent — no key recovery, no keystream reuse. It costs a second pass over the data. If nonce uniqueness cannot be guaranteed by construction — distributed encryptors, no shared counter, restore-from-backup in the threat model — SIV is the correct answer rather than a hardened GCM deployment.