Methods, numbers, and checklists from preparing, protecting, labeling, and evaluating enterprise data for AI. Each article is dated and checked by people who do the work, and every statistic links to its primary source.
When sensitive training data needs a US-only, employed team instead of a crowd: the rules that decide who may see a record, the quality evidence, and a decision table.
The Digital Omnibus delayed the AI Act's high-risk rules, including Article 10 on data. What is settled, what is pending, and what US data and model vendors owe now.
A worked estimate for embedding 50 million documents: tokens, FLOPs, GPU-hours, padding, storage and index memory, with every assumption stated so you can swap in your own.
Exact hashes, MinHash and embeddings each catch a different kind of duplicate. The arithmetic, the thresholds, the failure modes, and why decontamination is a separate job.
What California's AB 2013 asks generative AI developers to publish about training data, who counts as a developer, and a manifest that produces the page on every release.
What Safe Harbor and Expert Determination each require when clinical notes become LLM training data, why free text and memorization change the math, and how teams choose.
Model collapse starts in the tails. How to generate synthetic training data that holds up: real-data anchors, verification, tail and privacy tests, and lineage tags.
What to capture from production agents now, from tool calls and versions to human edits and outcomes, so the logs can become eval sets, fine-tuning data and RL tasks.
How to turn your own workflows into RL environments for agents: resettable state, production-shaped tools, task families, verifiers that resist reward hacking, and difficulty calibration.
What moves accuracy when you fine-tune a 1B–8B model for a domain task: narrow scope, curated examples, abstention cases, realistic inputs and learning-curve checks.
Why experts disagree on preference pairs, and how gated rubrics, calibration rounds and per-criterion alpha turn that disagreement into usable training signal.
Treat an LLM judge like an instrument: measure it against your experts, correct for its known error rates, and recalibrate whenever the judge or the task changes.
How to choose, specify, grade and maintain the first fifty tasks of an agent evaluation set, and what fifty tasks can and cannot tell you.
Most RAG failures on scanned files start in parsing. A field guide to tables, forms, handwriting, layout and chunking, with the metric that catches each.
Twelve tests, each run on a few hundred sampled records, that tell you whether a use case is ready for fine-tuning, for retrieval, or for neither yet.
Tell us what the data is and what you want a model to do with it. A data engineer, not a salesperson, will reply within one business day with questions and a proposed first step.
