Serialization Formats: Self-Describing vs Schema-Driven

Serialization is the point where a program's types stop being enforced and become bytes that anyone can write. Three questions separate the formats and predict almost everything else about them: does the encoding carry its own schema, does it encode data or an object graph, and is the decoder allowed to decide what types to instantiate from the input. The last question is not a design nicety — it is the difference between a parser and a remote code execution vector.

The landscape #

Language-native serialization is an RCE surface #

The common defect: the byte stream names the classes to instantiate, and decoding runs code — constructors, readObject, __reduce__, property setters. An attacker who controls the bytes controls which classes get built and in what order. A gadget chain is a sequence of classes already on the classpath whose deserialization side effects compose into arbitrary command execution. Nothing in the application has to be wrong; the vulnerability comes from the classpath plus the decision to decode untrusted input at all.

What JSON silently loses #

JSON is the right default for most APIs and it has four sharp edges worth knowing rather than rediscovering:

• Numbers are doubles in most parsers. Integers above $2^{53}$ lose low bits. 64-bit IDs go on the wire as strings (conversion).
• No binary type. Base64 costs +33% size and a copy on both ends.
• No date type. Everything is a string convention — use RFC 3339 with an explicit offset, and say so in the schema.
• No canonical form. Key order, whitespace, number formatting (1.0 vs 1), and Unicode escaping all vary between encoders.

Zero-copy formats #

FlatBuffers and Cap'n Proto arrange the buffer so that reading a field is a pointer offset, with no allocation and no decode pass. That turns deserialization cost from $O(\text{message})$ into $O(\text{fields you actually read})$ — the reason they appear in games, IPC, and mmap'd on-disk data. The costs are real too: the encoded form is larger than protobuf's varint packing, writing is more awkward, and the buffer must be treated as untrusted memory (both projects ship verifiers for exactly this).

Use them when a message is large, read partially, and hot. For a request body that is fully consumed once, they buy nothing.

Choosing #