Characterization Testing & Seams for Legacy Code

When code has weak tests and unclear intent, the immediate problem is not designing the ideal behavior; it is knowing what a change disturbed. A characterization test records an observed behavior of the current system and fails when that observation changes. It gives a refactor a tripwire, but the captured output may include defects, accidents and irrelevant noise.

Specification and characterization have different oracles #

Calling current output “expected” can hide this distinction. Label characterization tests and preserve how the baseline was obtained. Passing means parity with the observation, not correctness.

Find a stable observation boundary #

Good characterization points are public functions, commands, API responses, generated files, messages and database effects that callers already observe. Very low-level snapshots of private object graphs change during harmless refactoring; very high-level end-to-end snapshots include noise from unrelated systems.

Choose the narrowest boundary that still contains the behavior being changed. Then define an input corpus that covers known uses, boundary values, production incidents and important control-flow partitions.

flowchart LR
  I["Representative inputs"] --> L["Existing implementation"] --> N["Normalize declared noise"] --> B["Reviewed baseline"]
  I --> C["Changed implementation"] --> N2["Same normalization"] --> D["Diff"]
  B --> D

The baseline must be reviewed. Automatically accepting whatever the old system emitted creates a precise archive of unknown quality. For large output, use approval/golden-master tooling that makes the diff inspectable and requires an explicit update.

A seam creates control without rewriting the behavior #

Michael Feathers defines a seam as a place where behavior can be altered without editing at that place. The enabling point is where the alternate behavior is selected. Examples include a function parameter, constructor argument, imported implementation, link target, environment/configuration value, HTTP proxy or command boundary.

Seams serve two testing needs:

• sensing: make an otherwise hidden result observable;
• separation: replace a dependency that prevents the code from running in a harness.

Prefer an existing explicit seam. If none exists, introduce the smallest behavior-preserving change that creates one, then verify that change at a broader existing boundary before relying on it. Dependency injection can create an enabling point, but a global service locator trades one hidden dependency for another.

Build a characterization net incrementally #

Feathers's change-oriented workflow can be made concrete:

1. Identify the exact behavior or region that must change.
2. Find public observations and dependencies around it.
3. Run representative inputs and inspect the outputs.
4. Add characterization tests for the relevant current behavior.
5. Create a seam only where control or sensing is blocked.
6. Make one structural change and rerun the net.
7. Add specification/regression tests for the intended new behavior.
8. Decide which old baselines remain contracts and which intentionally change.

This is not a campaign to snapshot the entire codebase. Tests far from the change point consume maintenance while providing little local information. Expand coverage along actual change paths and operational risk.

Normalize only declared nondeterminism #

Legacy output often contains timestamps, generated ids, map order, host paths or floating noise. Normalize these only when they are explicitly outside the behavior contract. Store the raw output on failure so an over-broad normalizer cannot hide a real change.

Dangerous normalizations include sorting records when order may matter, deleting all error text when error class matters, rounding away a boundary defect, or replacing every identifier when referential relationships matter. The normalizer is part of the oracle and needs tests of its own.

For nondeterministic concurrency, a golden master from one schedule is weak. Record invariant outcomes, event partial orders or aggregate properties, and add race/schedule exploration separately. See async and concurrent testing.

Decide what to do with suspected bugs #

If the current system returns a suspicious value, first capture it as characterization when the goal is to make an unrelated safe change. Mark the case with context rather than silently “correcting” the baseline. When fixing the behavior deliberately:

1. confirm the intended rule with an authoritative source or owner;
2. add a specification/regression test with the corrected oracle;
3. change the implementation;
4. update or remove only the characterization assertions that conflict;
5. verify downstream compatibility because callers may depend on the old behavior.

Current behavior can be an accidental contract. Hyrum's Law still applies: observability creates dependencies whether the behavior was documented or not.

Worked example: opaque pricing export #

Suppose a nightly program reads orders and emits a large settlement CSV. The code mixes pricing, date handling and formatting and has no tests.

• Collect a small corpus: ordinary order, refund, month boundary, empty batch and a past incident.
• Run the current executable in a pinned locale/time zone and retain raw CSV plus exit status.
• Normalize only generated run id and absolute output path; keep row order, amounts and dates.
• Review and approve the baseline.
• Introduce a seam around the rate source so the corpus uses a fixed dated rate table.
• Extract the formatter in small steps, comparing the complete output after each step.
• For a known rounding defect, add an independently calculated specification test, change the rule and approve the one intentional CSV diff.

The golden master protects the wide behavior during extraction; the focused specification test explains the corrected rule. Keeping both roles explicit prevents “parity” from becoming “correctness.”

When to retire characterization tests #

A broad snapshot can be replaced when the behavior is understood and covered by smaller, intention-revealing tests plus representative integration evidence. Compare the coverage of claims before deletion, not line counts. Retain a high-level parity test when its boundary is itself a stable product contract; remove baselines that only fossilize obsolete internal structure.