Evaluating Generative Models

A generative model is not a classifier with labels hidden. It defines or induces a distribution and may be used for compression, simulation, synthesis, augmentation, editing, retrieval, or decision support. Evaluation must therefore begin with the intended use and decompose the claim into observable properties instead of searching for one universal score.

Write the claim before choosing the metric #

For each use, define the target distribution, conditioning information, valid output, unacceptable failure, and decision the evaluation must support. Examples:

• a simulator must reproduce downstream statistics and rare events relevant to policy;
• a music assistant must produce editable continuations preferred under blinded comparison;
• a data augmenter must improve a fixed downstream model on untouched real data;
• a density model for compression must assign probabilities that translate into code length;
• a creative generator may value novelty and controllability over matching every training mode.

The same model can rank differently under these contracts. That is not metric noise; it reflects different objectives.

Evaluation needs three data samples, not two #

Keep distinct:

1. training data used to fit the generator;
2. held-out real data sampled from the target population;
3. generated data produced by a frozen, declared sampling policy.

The held-out set estimates generalization. The training set is still needed to detect copying. Generated sample count, random seeds, prompts/conditions, temperature, guidance, truncation, rejection, and manual selection must be recorded. Reporting only selected examples estimates a curator, not the generator.

Split at the unit that can leak identity. For music, split by composition/source before extracting bars; for images, group near-duplicates and bursts; for text, group documents or templates; for subjects, keep one subject out of both train and test when deployment concerns new subjects.

Likelihood and sample quality are different axes #

Average held-out negative log-likelihood estimates how much probability the model assigns to real observations when the likelihood is tractable or accurately estimated. It is directly relevant to density estimation and compression. It does not tell whether typical generated samples look useful to a person or cover the semantics the application values.

In high dimensions, probability density can be greatest away from the typical set containing most mass. Small amounts of broad noise can change likelihood substantially without producing visually or musically better samples. Conversely, a sharp generator can make attractive samples while omitting much of the target distribution.

When likelihood is approximated with importance sampling, annealed importance sampling, variational bounds, or finite MCMC, report estimator assumptions, sample count, uncertainty, and comparability across models. A biased estimator can reverse rankings.

Fidelity and coverage must be separated #

Fidelity

Generated samples lie in regions judged plausible under the target domain.

Coverage

The generator represents the important modes and variation of the target distribution.

A generator that emits one perfect prototype has high local fidelity and terrible coverage. A noisy generator can cover every region while producing implausible samples. Precision/recall-style distribution metrics try to expose this trade-off rather than compressing it into one number.

Embedding-based distances such as FID or FAD compare features from real and generated samples, commonly through low-order distribution summaries. Their meaning depends on the feature extractor, reference set, preprocessing, sample size, and domain match. An image classifier embedding is not a universal perceptual metric; an audio embedding sensitive to recording quality can rank synthesis artifacts more strongly than composition.

Report at least the embedding model/version, layer, sample rate or resolution, sample count, reference corpus, confidence interval or repeated-sample variability, and known domain mismatch. Never compare scores computed with different pipelines as though they share a scale.

Domain metrics are diagnostics, not substitutes for utility #

Rule- or feature-based metrics can reveal specific failures:

• syntax validity and constraint violations;
• marginal pitch, duration, color, length, or category distributions;
• repetition, novelty, sparsity, polyphony, or transition statistics;
• conditional consistency with prompt, label, melody, or physical boundary conditions;
• downstream performance when synthetic data augment a fixed learner.

These metrics are useful because they are interpretable, not because matching them proves realism. A model can optimize a few musical statistics while producing incoherent phrases; a simulator can match marginals while breaking causal relationships. Use a vector of diagnostics tied to failure hypotheses.

Test memorization directly #

Novel-looking output can still copy training data with small edits. Compare generated samples against both training and held-out sets under representations that capture meaningful similarity. Exact hashes catch byte copies; nearest-neighbour searches catch local variants; domain fingerprints catch transposition, crop, tempo, paraphrase, or other invariances.

A three-sample test can ask whether generated items are systematically closer to training data than held-out real items are. Interpretation still depends on the distance function and density: rare legitimate modes may be close to their few training examples, while a poor embedding can miss semantic copying.

Report nearest examples rather than only aggregate distances. For each suspicious sample show the generated item, closest training item, closest held-out item, transformation-invariant distance, and review decision. Privacy and copyright risk require stricter thresholds than a creativity study.

Human studies must match the decision #

Human evaluation is indispensable when the target includes perception, preference, usefulness, or creativity, but it is easy to bias. Use randomized, blinded presentation; matched rendering and loudness; balanced order; clear criteria; practice examples; multiple raters per item; and a predeclared analysis.

Pairwise comparison often asks less of raters than absolute scoring. Separate questions such as technical validity, fidelity, preference, long-range coherence, novelty, and usefulness rather than combining them into “quality.” Report participant population, exclusions, agreement, uncertainty, and the exact prompt/context shown.

For interactive tools, measure task outcomes: time to an accepted edit, fraction retained, error rate, or improvement over unaided work. Listening to isolated clips does not establish that a model helps a composer.

A reproducible evaluation protocol #

1. Freeze model checkpoint, preprocessing, decoder, and sampling configuration.
2. Declare target population, conditions, and split unit; deduplicate before splitting.
3. Generate a predetermined number of samples for predetermined seeds/conditions without manual replacement.
4. Measure held-out density or predictive loss only when estimator comparability is defensible.
5. Measure fidelity and coverage separately using a domain-validated representation.
6. Test copying against training and held-out sets.
7. Run domain-specific diagnostics linked to concrete failure hypotheses.
8. Run blinded human or downstream evaluation for claims that require it.
9. Report uncertainty, failed samples, filtering, compute, and all selection rules.
10. Publish enough outputs or hashes for another evaluator to reproduce the sample set.

Music illustrates why one score fails #

For symbolic music, token NLL depends on tokenization and teacher forcing. Syntax checks catch invalid note events. Pitch-class, rhythm, duration, polyphony, groove, repetition, and inter-track statistics reveal local distribution shifts. Long-range form and aesthetic preference often require listening or structured musical analysis.

For rendered audio, FAD can be sensitive to sample-size bias, embedding choice, and reference quality. Keep the renderer identical across symbolic generators, otherwise the score mixes composition with timbre and recording quality. A defensible music report therefore combines protocol validity, distributional diagnostics, copying tests, long-range structure analysis, and blinded listening—without pretending that agreement is guaranteed.

Good evaluation does not find the number that makes the model look best. It constructs enough independent evidence that the intended claim would be difficult to satisfy through mode collapse, leakage, memorization, cherry-picking, or a metric-specific shortcut.