Benner House Insightshttps://bennerhouse.com/insights/Field notes on AI-ready data, evaluation, agents, privacy, and data infrastructure.en-usWed, 07 Oct 2026 14:00:00 GMT Onshore vs. crowd: when sensitive data needs a US-only data teamhttps://bennerhouse.com/insights/onshore-vs-crowd-sensitive-data/https://bennerhouse.com/insights/onshore-vs-crowd-sensitive-data/Wed, 07 Oct 2026 14:00:00 GMTTraining DataWhen sensitive training data needs a US-only team instead of a crowd platform: ITAR, EAR, HIPAA, the DOJ bulk data rule, tax data, and the quality case. The EU AI Act's data rules after the delay: what US vendors still owehttps://bennerhouse.com/insights/eu-ai-act-data-rules-after-the-delay/https://bennerhouse.com/insights/eu-ai-act-data-rules-after-the-delay/Wed, 09 Sep 2026 14:00:00 GMTPrivacy & ComplianceThe Digital Omnibus moved the AI Act's high-risk deadlines to 2027 and 2028. What changed for training data, what didn't, and what US vendors owe now. Embedding 50 million documents on a budget: batching, GPUs, and cost mathhttps://bennerhouse.com/insights/embedding-50-million-documents/https://bennerhouse.com/insights/embedding-50-million-documents/Wed, 05 Aug 2026 14:00:00 GMTInfrastructureWorked cost and throughput math for embedding 50 million documents: tokens, FLOPs, GPU-hours, padding waste, vector storage and the re-embedding trap. Near-duplicate detection at scale: MinHash, embeddings, and decontaminationhttps://bennerhouse.com/insights/near-duplicate-detection-at-scale/https://bennerhouse.com/insights/near-duplicate-detection-at-scale/Wed, 08 Jul 2026 14:00:00 GMTInfrastructureFinding near-duplicates in tens of millions of records: MinHash and LSH arithmetic, when embeddings help, why global dedup can hurt, and decontamination. California AB 2013: a training-data provenance manifest you can publishhttps://bennerhouse.com/insights/california-ab-2013-provenance-manifest/https://bennerhouse.com/insights/california-ab-2013-provenance-manifest/Wed, 17 Jun 2026 14:00:00 GMTPrivacy & ComplianceCalifornia AB 2013 requires a public summary of generative AI training data. A field-by-field manifest that answers its 12 items and stays current. HIPAA de-identification for LLM training: Safe Harbor vs. Expert Determinationhttps://bennerhouse.com/insights/hipaa-deidentification-llm-training/https://bennerhouse.com/insights/hipaa-deidentification-llm-training/Wed, 27 May 2026 14:00:00 GMTPrivacy & ComplianceSafe Harbor or Expert Determination for clinical notes that train an LLM: what each requires, why free text strains it, and how to measure what's left. Synthetic data that doesn't collapse: a validation checklisthttps://bennerhouse.com/insights/synthetic-data-without-collapse/https://bennerhouse.com/insights/synthetic-data-without-collapse/Wed, 13 May 2026 14:00:00 GMTTraining DataKeep synthetic training data from collapsing your model: anchor it to real data, verify before you keep, and test tails, privacy and contamination. Log agent trajectories now, so you can train agents laterhttps://bennerhouse.com/insights/log-agent-trajectories-now/https://bennerhouse.com/insights/log-agent-trajectories-now/Wed, 22 Apr 2026 14:00:00 GMTAgentsThe agent traces you discard today are tomorrow's training and evaluation data. What to capture per step, which schema to follow, and how to keep it governable. Building an enterprise RL gym from your own workflowshttps://bennerhouse.com/insights/enterprise-rl-gym-from-your-workflows/https://bennerhouse.com/insights/enterprise-rl-gym-from-your-workflows/Wed, 08 Apr 2026 14:00:00 GMTAgentsHow to turn claims, work orders and service requests into RL environments for agents: resettable state, real tool schemas, and verifiers that resist gaming. Dataset design for 1B–8B domain models: what actually moves accuracyhttps://bennerhouse.com/insights/dataset-design-small-domain-models/https://bennerhouse.com/insights/dataset-design-small-domain-models/Wed, 25 Mar 2026 14:00:00 GMTTraining DataFor 1B–8B models the dataset is the main lever. What moves accuracy in small domain fine-tunes: scope, selection, abstention, input realism and learning curves. Rubrics that reduce expert disagreement in preference datahttps://bennerhouse.com/insights/rubrics-reduce-expert-disagreement/https://bennerhouse.com/insights/rubrics-reduce-expert-disagreement/Wed, 11 Mar 2026 14:00:00 GMTTraining DataTrained labelers disagree on about a quarter of preference pairs. How gated rubrics and calibration rounds raise agreement without hiding real disagreement. Calibrating LLM-as-a-judge against your own expertshttps://bennerhouse.com/insights/calibrating-llm-as-a-judge/https://bennerhouse.com/insights/calibrating-llm-as-a-judge/Wed, 25 Feb 2026 14:00:00 GMTEvaluationAn LLM judge is a measuring instrument. How to calibrate one against your domain experts: bias checks, kappa, false-pass rates, score correction and drift. Fifty right examples: how to build a golden set for an AI agenthttps://bennerhouse.com/insights/golden-set-fifty-examples/https://bennerhouse.com/insights/golden-set-fifty-examples/Wed, 11 Feb 2026 14:00:00 GMTEvaluationHow to source, specify, grade and maintain the first fifty tasks of a golden evaluation set for a tool-using AI agent, and what fifty tasks can tell you. Why RAG fails on scanned PDFs: tables, forms, handwriting, and chunkinghttps://bennerhouse.com/insights/rag-scanned-pdfs-tables-forms/https://bennerhouse.com/insights/rag-scanned-pdfs-tables-forms/Wed, 28 Jan 2026 14:00:00 GMTDocument AIOn scanned documents, RAG usually breaks in parsing, not retrieval. How tables, forms, handwriting and naive chunking fail, and how to measure each one. AI-ready data: 12 tests to run before you fine-tune or build RAGhttps://bennerhouse.com/insights/ai-ready-data-12-tests/https://bennerhouse.com/insights/ai-ready-data-12-tests/Wed, 14 Jan 2026 14:00:00 GMTData ReadinessTwelve sample-based tests that show whether your documents, tickets and records can support fine-tuning or RAG, with pass, fix and stop rules for each.