Protocol Layering & Encapsulation

Layering manages complexity by giving each level a narrow contract. A layer uses a service below, offers a service above, and exchanges protocol messages with its peer. Encapsulation is the concrete mechanism that preserves those peer-to-peer meanings while data crosses lower layers.

The model is a reasoning tool, not a claim that every implementation is seven independent boxes. RFC 1122 defines the Internet host around application, transport, Internet, and link layers and explicitly notes that strict layering is imperfect. The OSI vocabulary remains valuable because it distinguishes services, interfaces, protocol entities, and data units precisely.

Protocol, service, and interface #

These three are easy to collapse but answer different questions:

• A service says what an upper layer can request: deliver a datagram, create a reliable byte stream, resolve a name.
• An interface says how a local consumer requests that service: function call, socket API, queue, or driver boundary.
• A protocol says what peer entities exchange across the network to provide it: headers, messages, state transitions, timers, and error rules.

Changing an API does not necessarily change the wire protocol. Changing a wire protocol does not necessarily change the application-facing service. Good abstractions let either evolve within the contract.

OSI and Internet models #

This is a mapping, not an equivalence table. TLS, for example, performs representation and security functions but is deployed above transport APIs; QUIC combines transport, security, and stream multiplexing. The correct question is “which service and state does this component own?” rather than “which single OSI number is it?”

Encapsulation and decapsulation #

Let an application supply bytes $D$. Each layer prepends control information needed by its peer:

$$T = H_T \parallel D, \qquad N = H_N \parallel T, \qquad L = H_L \parallel N \parallel F_L.$$

$H_T$ may identify transport endpoints and sequence state; $H_N$ supplies network-layer addressing and forwarding information; $H_L$ and $F_L$ delimit and protect transfer on one link. The receiver validates and removes these fields in reverse order before delivering $D$.

OSI terminology calls the upper layer's data an SDU (service data unit) at the boundary and the result after adding the current layer's protocol control information a PDU (protocol data unit). The bytes are the same; the name records which contract is being discussed.

Multiplexing and demultiplexing #

One lower-layer service commonly carries many upper protocols or conversations. A receiver needs an explicit discriminator at every fan-out:

• link-layer type or service-access information selects the network protocol;
• the IP protocol/next-header field selects the transport or extension processing;
• transport port numbers and connection identifiers select an endpoint or conversation;
• an application protocol may multiplex streams or objects again.

The discriminator is part of the protocol contract. Guessing a payload type from its contents makes ambiguity and security failures likely.

Size transformations #

Every layer adds overhead and may impose a maximum data-unit size. If an upper SDU does not fit, the system has several choices:

• segmentation: divide it before transmission and reassemble at the intended peer;
• fragmentation: divide a network-layer unit to fit a lower-layer constraint;
• blocking/aggregation: combine several small SDUs into one PDU to reduce per-unit overhead;
• concatenation: place multiple complete units in one lower-layer transfer while retaining boundaries.

The placement of reassembly determines failure cost. Reassembling only at the destination avoids maintaining per-unit state in every router, but a lost fragment can invalidate the whole original unit. Transport segmentation usually gives recovery finer-grained sequence and acknowledgement semantics than network fragmentation.

Which devices terminate which layers? #

A repeater regenerates physical symbols without interpreting frames. A bridge receives and validates a frame, makes a link-layer forwarding decision, and transmits another frame. A router terminates the incoming link PDU and makes a network-layer decision. A transport proxy terminates one transport connection and creates another. The application exchange remains end-to-end only up to the highest device that does not terminate it.

This yields a reliable troubleshooting rule: inspect the highest shared layer first. If link reception fails, transport debugging is premature. If an HTTP proxy returns an error, an end-to-end TCP trace alone cannot reveal the upstream application exchange it created.

Why layers leak #

Perfect opacity is neither achieved nor always desirable:

• Path MTU discovery lets transport size segments using a property of the network path.
• Explicit congestion signals carry lower-layer or network observations toward endpoint control loops.
• TLS exposes enough metadata for transport and middlebox interaction while hiding application content.
• NIC offloads move segmentation, checksumming, and coalescing across the apparent kernel/device boundary.
• Wireless schedulers use channel state while serving higher-layer traffic classes.

These are controlled cross-layer contracts, not evidence that layering has no value. The danger is an undocumented dependency: a higher layer silently assuming a frame size, timing behavior, or ordering property the lower service never promised.

Trace one real packet correctly #

For one representative exchange, record at each boundary:

1. the SDU and PDU names;
2. the endpoint identifiers used by that layer;
3. the discriminator selecting the next protocol;
4. the maximum size and any segmentation/reassembly;
5. which device terminates or merely forwards the unit;
6. integrity checks and what they cover.

If those six facts are explicit, the protocol stack stops being a memorized seven-row diagram and becomes a causal model for packet capture, performance, and failure diagnosis.