Formats
- The same schema and format as your real data: Parquet, CSV, JSONL, or rendered documents
- Validation report as PDF plus the notebook behind it
Synthetic data is worth exactly what it scores against the real thing.
We generate synthetic records when real ones are scarce, too sensitive to move, or missing the cases that matter. Every batch is tested three ways: does it look like your data, does a model trained on it work on real data, and does it leak anyone's real record?
Good reasons to generate data, and one bad one
The good reasons are specific. You have 40 examples of a fraud pattern and need 4,000 to train a detector. Your claims data can't leave a secure enclave, but a vendor needs realistic records to build and test against. A new product line has no history yet. An agent sandbox needs a database full of plausible customers with known problems.
The bad reason is to avoid working with the real data. A generator can only reproduce what it learned, and it learns the common cases best. A 2024 paper in Nature showed that models trained recursively on generated data develop irreversible defects in which the tails of the original distribution disappear first. Those tails, the rare and unusual cases, are usually why you wanted synthetic data in the first place.
So we treat synthetic data as a measured supplement to real data, never a replacement, and we test it against held-out real records the generator never saw.
The generator depends on the data
There is no single synthetic data tool. We choose the generator per data type and per use, and document the choice in the manifest.
| Data | Typical generator | What we watch for |
|---|---|---|
| Tabular records: claims, transactions, sensor readings | Constraint-aware statistical models such as copulas, deep tabular generators, or LLM row generation validated against the schema | Broken business rules, impossible combinations, rare categories smoothed away |
| Free text: notes, emails, tickets | LLM generation seeded from de-identified real examples, expert-written scenarios, and personas, then filtered against a rubric | Repetitive phrasing, language cleaner than any real note, memorized seed text |
| Instructions and dialogue | Expert-authored seed tasks expanded by a model, then reviewed by experts | Shallow variety; answers that sound right and aren't |
| Documents and forms | Templates filled with synthetic entities, rendered with realistic scan noise, skew, and stamps | Layouts too uniform to test a real parser |
| Event sequences and sandbox data | Simulators and seeded databases that follow your workflow's rules | States the real system could never reach |
Three tests every batch has to pass
Fidelity: does it look like your data?
We compare marginal and joint distributions, correlations, rare-category frequencies, and constraint violations against real data. For text, we compare topic, length, and vocabulary distributions and n-gram diversity, and run a blind review in which domain reviewers try to pick the synthetic records out of a mixed sample.
Utility: train on synthetic, test on real
We train the same model on real data and on synthetic data and score both on a held-out real set. The gap between them is the number that matters. For augmentation, we compare real data alone with real plus synthetic, sliced by the rare classes the synthetic data was meant to help.
Privacy: does it leak real records?
We measure each synthetic record's distance to its closest real training record and compare that distribution against a real holdout set's; synthetic records sitting much closer than real ones do are a warning. We run membership-inference tests and search for verbatim and near-verbatim copies of seed text. For regulated data, the results feed your de-identification analysis.
The sprint at a glance
You provide
- A real dataset to model, or expert-written seed examples where none exists
- The downstream task the synthetic data must support
- Privacy constraints: what may not be reproduced, and who may see the output
We deliver
- Up to 100,000 synthetic records
- The generator, its configuration, and its seed data, versioned in your environment
- A validation report on fidelity, downstream utility, and privacy leakage
- A manifest that marks every record as synthetic and links it to its generator version
Quality gates
- Utility measured on held-out real records the generator never saw
- Distance-to-closest-record and membership-inference results reported against a real holdout baseline
- Blind expert review of at least 200 records
- Business-rule constraints pass on every record
Turnaround
Four to six weeks
Team
An ML engineer, a privacy engineer, and domain reviewers for the blind review
Starts at
$12,800
Generator design, up to 100,000 synthetic records, and a validation report on fidelity, downstream utility, and privacy leakage.
Guardrails against model collapse
These rules are part of every synthetic data engagement, not options.
- Real data stays in every training mix; synthetic records fill named gaps
- Generators are never retrained on their own output
- Every synthetic record is tagged, so it can be counted, filtered, or removed later
- Rare-class frequencies are checked against real data every batch, not just averages
- Diversity is tracked across batches, and a drop stops the run
- Evaluation sets contain real records only
Sprint pricing
Synthetic data sprint
Starts at
$12,800
Generator design, up to 100,000 synthetic records, and a validation report on fidelity, downstream utility, and privacy leakage.
Request a scoped quoteLarger volumes, image or document synthesis, and ongoing generation for agent sandboxes are quoted once the sprint has shown what works for your data.
Questions about synthetic data
Is synthetic data automatically de-identified?
No. Generators can memorize and reproduce real records, especially rare ones. That's why every sprint includes privacy tests. If the source data is regulated, your privacy team still needs to decide whether the output meets your de-identification standard, and our measurements are the evidence they work from.
How much synthetic data can we safely mix in?
It depends on the task, so we test it. We train at two or three mixing ratios and watch performance on real holdout data, especially on the rare slices. When adding more synthetic data stops helping or starts hurting, that's the ceiling.
Which model generates the data?
We have no model vendor to favor. When seeds are sensitive, we run open-weight models inside your tenant. We also check the license terms of any model used, since some providers restrict using outputs to train other models.
Do we have to disclose that we used synthetic data?
That's for your counsel. For context, whether synthetic data generation was used is one of the 12 items in California AB 2013's dataset summary. Our manifest records it either way, and Benner House is not a law firm and does not provide legal advice.
Can synthetic data replace our evaluation set?
No. Evaluation should be real cases, verified by your experts. Synthetic cases are useful as clearly labeled stress tests for situations you haven't seen yet, alongside a real evaluation set, never instead of one.
Keep reading
Bring us one messy data source and one question.
Tell us what the data is and what you want a model to do with it. A data engineer, not a salesperson, will reply within one business day with questions and a proposed first step.