The Transmission Chain, Source to Sink

Every link — a phone call, a satellite feed, a disc read head — is built out of the same six transformations in the same order, and the ordering is not a convention. The source coder removes the redundancy the source happens to have; the channel coder then adds back redundancy shaped to the noise it expects; the modulator maps bits onto whatever the physical medium can actually carry. Redundancy is destroyed and re-created on purpose, because the redundancy a source comes with (a vowel is predictable from the consonants around it) is not the redundancy that survives a fade.

The reason this looks like a pipeline rather than one big joint optimizer is a theorem. Shannon proved that designing the two coders independently loses nothing — in the limit of unbounded blocklength. Every place a real system deviates from the clean chain (unequal error protection, bad-frame concealment, analog video over a fading link) is a place where that limit does not apply.

The chain #

flowchart LR
  A["Source"] --> B["Source encoder"]
  B --> C["Channel encoder"]
  C --> D["Modulator"]
  D --> E["Channel"]
  N["Noise"] --> E
  E --> F["Demodulator"]
  F --> G["Channel decoder"]
  G --> H["Source decoder"]
  H --> I["Sink"]

This is Shannon's Fig. 1 with the transmitter and receiver opened up. His own diagram had exactly five parts — information source, transmitter, channel, receiver, destination — plus a noise source drawn feeding the channel and pointedly not counted among the five. In 1948 the transmitter was a single box that turned a message into a signal the channel could carry, and the receiver was defined as its inverse; his own examples of it ranged from a telephone's pressure-to-current conversion to a multiplex PCM system that samples, compresses, quantizes, encodes and interleaves. Splitting that box into source encoder + channel encoder + modulator is what the following decades were about.

Note what the channel is not: the channel is the only block in the diagram that nobody designs. The course that shapes this territory organises the physical layer around that asymmetry, and splits what the channel does to a waveform into three families — amplitude-frequency distortion (the magnitude response is not flat), phase-frequency distortion (group delay varies with frequency, so a pulse's harmonics arrive misaligned and the pulse smears even though no energy was lost), and nonlinear distortion (the transfer characteristic bends, creating intermodulation products at frequencies the input never contained). These are deterministic channel misbehaviours the demodulator's equalizer undoes. Additive noise is the separate, non-invertible part.

Redundancy out, redundancy in #

The single most counterintuitive property of the chain is that stage 2 spends effort destroying exactly the thing stage 3 spends effort creating. Shannon addressed this directly, and his answer is not "the two cancel":

The redundancy must be introduced in the proper way to combat the particular noise structure involved.

Source redundancy does protect against noise — English survives a badly typed telegram — it is just a wildly inefficient code for the job. Shannon's own example is telegraphy: proper encoding of English would save roughly 50% of the transmission time, nobody does it, and the leftover redundancy is precisely what lets a sizeable fraction of letters arrive wrong and still be reconstructed from context. The chain's bargain is to give up that accidental protection and buy back a designed one that costs less redundancy for far more error resilience.

So the rate profile through a real link is a V:

$$\text{source rate} \;\xrightarrow{\;\div\,8\;}\; \text{compressed} \;\xrightarrow{\;\times\,1.75\;}\; \text{channel-coded} \;\xrightarrow{\;\times\,1.5\;}\; \text{on the air}$$

with the exact factors below.

Why the split is legal #

Shannon's Theorem 11 (1948) states the lossless point-to-point version directly: for a discrete channel of capacity $C$ and a source of entropy per second $H$, if $H \le C$ there exists a coding system that transmits the source over the channel with an arbitrarily small frequency of errors; if $H > C$ the equivocation can be brought down to $H - C + \varepsilon$ and no lower. Theorem 9 is the noiseless-channel companion: a source of entropy $H$ bits/symbol over a channel of capacity $C$ bits/second can be sent at $C/H - \varepsilon$ symbols per second and no faster.

The lossy version, as the modern survey states it, is: a rate–distortion pair $(r, D)$ is achievable if $r \cdot R(D) < C$, and conversely achievability implies $r \cdot R(D) \le C$, where $r$ is the source-to-channel symbol ratio.

The engineering consequence is enormous and easy to under-appreciate: one channel code serves every source. The LDPC code in a satellite modem does not know or care whether it is carrying H.265, a filesystem image, or telemetry. The interface between the halves of the chain is a bare bit rate. Every codec ever written and every FEC ever designed compose combinatorially because of this theorem.

Where separation breaks #

The theorem's conditions are load-bearing: memoryless (or stationary ergodic) source and channel, point-to-point, single-letter additive distortion, and the limit of infinite blocklength. The survey is blunt about the scope — separation is optimal only asymptotically, and only with respect to capacity; error exponents and second-order rates are both improvable by a joint code.

Finite blocklength #

At blocklength $n$ the maximal rate is not $C$ but, to second order,

$$R(n,\epsilon) \;\approx\; C - \sqrt{\frac{V}{n}}\,Q^{-1}(\epsilon) + \frac{\log_2 n}{2n}$$

where $V$ is the channel dispersion. Take a BSC with crossover $\delta = 0.11$ — the running example in Polyanskiy, Poor and Verdú, whose capacity works out to $C = 1 - h(0.11) = 0.500$ bit/use — with dispersion $V = \delta(1-\delta)\left(\log_2\frac{1-\delta}{\delta}\right)^2 = 0.891$ and block error rate $\epsilon = 10^{-3}$:

Rates in bits per channel use, computed from the normal approximation above; the rows are illustrative of blocklength, not of each system's actual channel.

Two things follow. First, short blocks lose a lot — on this channel a 378-bit codeword forfeits nearly a third of capacity to dispersion alone, before any implementation loss, and the shape of the curve (a $1/\sqrt{n}$ penalty) is universal even though the constant is not. Second, when both halves of the chain are short, they each pay their own second-order penalty, and a joint code pays only once: Kostina and Verdú show the optimal joint code satisfies $nC - kR(d) \approx \sqrt{nV + k\mathcal{V}(d)}\,Q(\epsilon)$, i.e. the dispersions add inside one square root rather than as two separate margins. That is the entire finite-blocklength argument for joint source–channel coding.

More than two parties #

Separation fails outright in networks. The canonical counterexample is correlated sources over a multiple-access channel: for a binary adder MAC, independent channel inputs achieve a sum rate of 1.5 bits per channel use, while the joint entropy of the two correlated sources is $\log_2 3 = 1.58$ bits per sample — so the best separation-based source–channel rate is $1.5/1.58 \approx 0.949$, whereas simply transmitting the sources uncoded achieves $r = 1$. Separation is strictly suboptimal, and no cleverness inside either block recovers the gap. The same holds for broadcast channels with unequal receiver side information.

Non-ergodic and unknown channels #

A separated design must pick a code rate, which means picking a design SNR. Below it, the decoder fails abruptly and completely; above it, extra SNR buys nothing. These are the cliff effect and the levelling-off effect, and they are what people mean when they say analog TV "degraded gracefully" while digital TV either works or blocks up. A joint scheme whose output degrades continuously with SNR has no cliff — the reason analog and hybrid schemes keep being reinvented for broadcast and for wireless video.

Delay and complexity #

Separation is an asymptotic statement, so the price of approaching it is blocklength, and blocklength is latency. This is not an implementation detail you can engineer away; it is the same trade Shannon flagged. Delay was already required to approach ideal encoding in the noiseless case; on a noisy channel it acquires a second job, letting a large enough sample of the noise act on the signal before the receiver commits to a decision, because bigger samples sharpen the statistics. A conversational voice link has a few tens of milliseconds of budget, which caps the blocklength, which caps how close to capacity it can get.

A real chain: one GSM voice call #

GSM is worth walking end to end because every stage is a published number and the arithmetic closes exactly.

flowchart LR
  A["Mic + ADC<br/>104 kbit/s"] --> B["RPE-LTP<br/>13 kbit/s"]
  B --> C["CRC + conv 1/2<br/>22.8 kbit/s"]
  C --> D["Interleaver<br/>8 bursts"]
  D --> E["A5 cipher"]
  E --> F["Burst format<br/>148 bits"]
  F --> G["GMSK<br/>270.833 kbit/s"]

Source coder. GSM 06.10 RPE-LTP takes blocks of 160 speech samples in 13-bit uniform PCM at 8000 sample/s and produces 260 bits, i.e. 13 kbit/s. Eight-to-one, and irreversibly: what comes back is a resynthesis from an LPC model, not the waveform. See speech coding.

Channel coder (GSM 05.03 §3.1). The 260 bits are not equal, and the standard says so explicitly: 182 bits of class 1 (protected) and 78 bits of class 2 (no protection); class 1 splits further into 50 class 1a bits and 132 class 1b.

1. 50 class 1a bits get 3 parity bits from a shortened cyclic code (53,50,2), $g(D) = D^3 + D + 1$ — detection only.
2. 189 = 182 class 1 + 3 parity + 4 tail bits, reordered.
3. 378 after the rate-½ convolutional code, $G_0 = 1 + D^3 + D^4$, $G_1 = 1 + D + D^3 + D^4$ (constraint length 5).
4. 456 after appending the 78 class 2 bits verbatim: $c(378+k) = d(182+k)$, $k = 0\ldots77$.

456 bits per 20 ms is 22.8 kbit/s, and the overall code rate is $260/456 = 0.570$.

Interleaver. The 456 bits are spread "block diagonal" over 8 bursts, 57 bits each, with a new block starting every 4th burst — so each burst carries 57 bits from one speech frame and 57 from the next. A fade that destroys one whole burst therefore costs each of two speech frames 57 of 456 bits, spread across the convolutional trellis, instead of destroying 456 consecutive bits of one frame. Interleaving buys nothing on an AWGN channel; it exists solely to convert the burst errors of a fading channel into the independent errors the code was designed for (see fading and diversity).

Modulator. 148 useful bits per burst — 3 tail, 57 data, 1 stealing flag, 26 training (midamble), 1 stealing flag, 57 data, 3 tail — plus 8.25 bit periods of guard, filling a 156.25-bit-period timeslot of $3/5200$ s ≈ 577 µs. Eight timeslots make a TDMA frame of ≈ 4.615 ms. Modulation is GMSK with $BT = 0.3$ at $1625/6$ kbit/s ≈ 270.833 kbit/s, on carriers spaced 200 kHz apart: 1.354 bit/s/Hz gross, before the eight-way split.

Latency. 20 ms to fill a speech frame, then the wait for the eighth burst. The TCH/FS mapping repeats every 13 TDMA frames with blocks at frames $(0\ldots7)$, $(4\ldots11)$, $(8\ldots11, 0\ldots3)$ — so a block spans 7 TDMA frames ≈ 32 ms, or 8 ≈ 37 ms when it wraps past the frame reserved for signalling. Call it ~55 ms of structural one-way delay before any propagation, processing, or network queueing — bought, precisely, to get a longer effective blocklength against fading.

GSM does not actually obey separation #

Two of its stages are joint source–channel designs wearing a separation costume.

Unequal error protection. The channel coder knows which source bits matter: class 1a bits get a CRC and the convolutional code, class 1b get the convolutional code only, class 2 get nothing at all. That is the channel encoder consuming the source encoder's semantics — precisely the coupling separation says is unnecessary. It is there because a 378-bit codeword sits deep in the finite-blocklength regime — whatever the exact channel, the dispersion penalty at that length is tens of percent of capacity — and once the total redundancy budget is that tight, spending it uniformly is measurably worse than spending it where a bit error is audible.

Error concealment. The 3 class-1a parity bits detect errors on 50 bits; they cannot correct anything. Their output is a bad-frame flag routed to the source decoder, which (per 3GPP TS 46.011) substitutes the first lost frame with the previous good one, then for subsequent losses decrements the block amplitude parameter by 4 per frame across its 0–63 range, silencing the output after at most 320 ms. Channel state information flowing into the source decoder is the other half of a joint design, and it is why a dropping GSM call fades to silence rather than screeching.

The other end: a long-blocklength link #

DVB-S2 is the same chain with the finite-blocklength constraint relaxed, and it shows what separation buys when you let it. Its FEC is LDPC concatenated with BCH over a normal FECFRAME of 64 800 bits (16 200 for the short frame), with code rates from 1/4 to 9/10 and four constellations spanning 2 to 5 bit/s/Hz, achieving quasi-error-free operation at 0.7 to 1 dB from the modulation-constrained Shannon limit. Roll-off is selectable at 0.35, 0.25 or 0.20.

The pattern generalises: the tighter the latency budget, the more the chain leaks. Interactive links violate separation; broadcast and storage links honour it.

This chain is not the OSI stack #

The two decompositions get conflated constantly, because both draw boxes stacked in an order. They cut along different axes.

The whole chain lives inside layer 1, with the source coder straddling into what OSI would call presentation and applications actually call "the codec." Two consequences worth holding on to:

• A protocol stack has stages the Shannon chain has no slot for. In GSM's own reference configuration the order is speech coder → channel coder → interleaving → encryption → burst formatting/multiple access → modulator. Ciphering sits between the channel coder and the modulator and is invisible to information theory as Shannon set it up in 1948 — he treated secrecy in a separate paper, with a separate model.
• Multiple access is orthogonal to the chain. TDMA/FDMA/CDMA decide whose bits go where; they do not change what any block in the chain does to a given user's bits. See multiplexing.

Conversely, the transmission chain has a stage the protocol stack keeps forgetting it has: the source decoder is allowed to know how bad the channel was. Every codec with a concealment path — GSM's muting, a jitter buffer's packet-loss concealment, a video decoder's error resilience — is exploiting that, and it works only if the channel decoder passes a flag rather than silently handing up wrong bits.

Sharp edges #

What to take away #