Synthetic Data Validation Checklist

Synthetic data has become the default answer to a familiar problem. The fraud team has 300 confirmed cases of a scheme and needs 3,000. The claims model has never seen water-damage notes written in Spanish. The contracts team can't send real redlines to an outside annotator. Ask a strong model to write more examples and you have them by the afternoon.

The risk has a name. In a 2024 Nature paper, Shumailov and colleagues showed that models trained recursively on the output of earlier models lose the tails of the original distribution first, then drift toward a narrow, low-variance version of it. They called it model collapse. The photocopy of a photocopy is the right picture: every copy is legible, and every copy loses a little of what made the original specific.

Few enterprise teams will run fifty rounds of recursive pretraining. Many run smaller loops with the same shape, and the checks below are how we keep those loops from eating their own tails.

Where collapse shows up in a business dataset

Three loops account for most of the exposure:

Self-feeding fine-tunes. A model writes examples, gets fine-tuned on them, and the fine-tuned model writes the following batch.
Synthetic records drifting into evaluation. Generated examples land in the shared pool that someone later samples for a test set, and the model is graded on its own handwriting.
AI-assisted work flowing back as "real" data. Drafted denial letters, summarized call notes and auto-filled work orders return to the warehouse without a flag. Next year's training data is partly model-written, and nobody can say which part.

The symptoms are consistent. Rare categories thin out first: the denial code that appears in half a percent of claims, the handwritten addendum, the caller who switches languages mid-sentence. Lengths converge, phrasing repeats, and the confident middle of the distribution grows.

The research is specific about what helps and what doesn't. In the OPT-125m experiments, preserving 10% of the original data in each generation kept degradation minor. A decoding fix did not: pushing generators to avoid repeats doubled perplexity, and the models stayed just as susceptible to collapse. Guo and colleagues measured a steady decline in lexical, syntactic and semantic diversity across recursive fine-tuning, steepest on creative tasks.

Three rules that hold up

Accumulate; don't replace. Gerstgrasser and colleagues found that replacing real data with each generation's synthetic data tends toward collapse, while accumulating synthetic data alongside the original real data avoided it across model sizes and architectures. Bertrand and colleagues proved that iterative retraining is stable when the starting model is good and the proportion of real data is large enough. Real data is never retired. Synthetic data is an addition with a budget.

Anchor every round in fresh real data. Alemohammad and colleagues concluded that without enough fresh real data in each generation, quality or diversity declines. In practice, seed every generation run from real records sampled that round, stratified toward the tails you're trying to fill, never from last round's synthetic output.

Verify before you keep. Feng and colleagues showed that filtering synthesized data with a verifier prevented collapse, even when the verifier was imperfect. Business data is full of cheap verifiers: a schema validator, a rule that a paid amount can't exceed the billed amount, a classifier trained only on real data, a claims examiner reviewing a sample.

The decision rule we apply: a generator never trains on its own unverified output, and no record is more than one generation away from a real seed.

Measure the tails before anything else

Fluency is the wrong first test. Synthetic text almost always reads well. Compare its shape with the real data it's supposed to extend. The table below is an illustrative composite from a claims-denial classifier with 40 codes.

Check	Real sample (12,000)	Synthetic batch (12,000)	Gate
Codes with at least 20 examples	31 of 40	22 of 40	No fewer than real
Share of the five most common codes	61%	78%	Within 5 points of real
Distinct 4-grams per 1,000 tokens	412	268	At least 85% of real
Median length in tokens	186	141	Within 15% of real
Near-copies of a seed record	Source of the seeds	3.1%	Under 0.5%

That batch fails four gates, and in the familiar direction: it over-produces the head, flattens the language, shortens the notes and copies its seeds. Regenerating with prompts stratified by code and a seed set rotated each run usually fixes the first three. The fourth is a privacy problem, covered below.

Then read records. Self-Instruct, one of the best-known synthetic instruction sets, reported that only 54% of 200 sampled records had a valid instruction, an appropriate input and a correct output, and 58% of outputs were correct. At web scale the volume of good examples outweighed the noise. In a narrow domain set of a few thousand examples, a 40% defect rate teaches the defects. Have a domain expert review a random sample from every batch against the labeling rubric, and set a defect-rate gate before generation starts, not after.

Filter for diversity at generation time too. Self-Instruct admitted a new instruction only if its ROUGE-L similarity to every existing one was below 0.7. Apply the same idea with MinHash or embeddings, against both earlier synthetic records and the real seeds.

Utility: train on synthetic, test on real

The test that settles whether synthetic data helps is old and simple: train on synthetic, test on real, a protocol Esteban and colleagues used for generated medical time series. The test set must be real records your experts trust, with no generated examples anywhere in it. (Our notes on building a golden set cover how.)

Run a mixing sweep rather than a single comparison: real data only, then real plus 25%, 50% and 100% synthetic. Read per-class results, not just the average. If the tail classes the synthetic data was meant to fill improve and the head holds, keep it. If the average rises while the tail stays flat, the generator added more of what you already had.

Synthetic data can carry a model a long way when the generation is curated and the evaluation is real. phi-1, trained largely on filtered web text plus synthetic textbooks and exercises, reached 50.6% on HumanEval. The lesson is the filtering and the real benchmark, not the volume.

Privacy: synthetic does not mean anonymous

It's tempting to treat generated records as automatically safe to share. Stadler and colleagues found that synthetic data either failed to prevent inference attacks or failed to retain utility, offered no better privacy-utility trade-off than traditional anonymization, and made the privacy gain hard to predict.

Generators also memorize. Carlini and colleagues extracted hundreds of verbatim training sequences from GPT-2, including names, phone numbers and email addresses, some of which appeared in only one training document. If a model wrote your synthetic claims notes from real seed notes, check whether it copied them:

Near-copy rate against the seed set, using MinHash at a similarity of 0.8 or tighter.
Distance to closest record. If synthetic records sit closer to their seeds than real held-out records sit to the training set, the generator is copying.
Entity scan. Run your PII detector over the synthetic output and match hits against seed identifiers: names, member IDs, policy numbers.
Canaries. Plant a few unique fictitious records in the seed set and search the output for them.

When the seeds are patient or customer records, the synthetic set should go through the same review as any other de-identified release. Our synthetic data validation work runs these tests alongside utility, so a dataset that passes one and fails the other doesn't ship.

Lineage: tag every record

Yang and colleagues found test-set contamination in synthetic datasets generated by GPT-3.5 and GPT-4, so decontaminate generated data against every evaluation set you use. California's AB 2013, in effect for public generative AI systems, lists whether synthetic data generation was used among the 12 items a developer's training-data disclosure must cover. Both are easy if every record carries its lineage from the moment it's written:

{
  "record_id": "syn-claims-2026-05-000184",
  "origin": "synthetic",
  "generation": 1,
  "generator": { "model": "gen-model@rev-2026-04-30", "prompt_template": "denial-note-v7", "temperature": 0.8 },
  "seed_record_ids": ["clm-88213-n4", "clm-90551-n2"],
  "target_stratum": "denial_code=CO-197",
  "verifiers": [
    { "check": "schema_v3", "result": "pass" },
    { "check": "business_rules_v12", "result": "pass" },
    { "check": "seed_near_copy_minhash_0.8", "result": "pass" }
  ],
  "expert_review": { "sampled": true, "reviewer_role": "claims_examiner", "verdict": "accept" },
  "decontaminated_against": ["golden-denials-v4"],
  "batch_id": "syn-batch-0112"
}

The generation field enforces the one-step rule. The seed_record_ids field lets you delete synthetic descendants when a seed record has to be removed. The origin field keeps generated records out of test sets automatically.

The validation checklist

Real data stays in every training mix; synthetic data is added with a stated ratio, never substituted.
Every generation run is seeded from real records sampled that round, stratified toward the tails.
No record is more than one generation from a real seed, and no generator trains on its own unverified output.
Each batch passes tail-coverage, head-share, diversity and length gates against the real distribution.
A domain expert reviews a random sample per batch against the rubric, with a defect-rate gate set in advance.
Generated records are deduplicated against each other and against seeds at generation time.
Train-on-synthetic, test-on-real results are reported per class, from a mixing sweep.
Near-copy, closest-record, entity-scan and canary tests pass before anything leaves the environment.
Synthetic records are decontaminated against every evaluation set and never placed in one.
Every record carries origin, generation, generator version, seeds and verifier results.

Synthetic data that doesn't collapse: a validation checklist