Enterprise RL Gym From Your Own Workflows

In January, Epoch AI published an FAQ on reinforcement learning environments based on interviews with 18 people at environment startups and AI labs. Two details stand out for enterprises. Environments are commonly priced per task, with one founder citing $200 to $2,000 per task as what they mostly saw. And the article names enterprise workflows, such as filing an expense report, creating a pivot table or updating customer records in a CRM, as one of the major growth areas.

The reason is that models learn agentic work by doing it in places where success can be checked. Public benchmarks established the pattern. SWE-bench turned 2,294 real GitHub issues into tasks checked by tests. OSWorld built 369 computer tasks, each with an initial-state configuration and an execution-based evaluation script. τ-bench checks an agent by comparing the database at the end of a conversation with an annotated goal state. The same pattern applies to your claims queue.

General-purpose environments teach general skills on generic software. Your agents fail on your rules: the approval threshold, the exception code, the second system that also has to be updated. An environment built from your workflows trains for those rules, and even if you never run reinforcement learning yourself, it is the best test you will have of any vendor's agent.

Five parts of an environment

State. A resettable replica of the systems the workflow touches: a database snapshot, a document store, a mock inbox, a ledger. It is built from de-identified or synthetic records and resets in seconds from a seed, so every episode starts from a known world.
Tools. The same interfaces the production agent will use, with the same names and schemas. If production tools are exposed over the Model Context Protocol, the sandbox exposes the same tool list, input schemas and output schemas. Enforce permissions in the sandbox itself, not in tool descriptions. The MCP specification says clients must treat tool annotations as untrusted unless they come from trusted servers.
Task. An instruction, a starting state, the policy documents the agent may consult, and for conversational work, hidden facts that a simulated user reveals only when asked.
Verifier. Programmatic checks on the end state, checks for forbidden actions across the trajectory, and optionally a rubric for the message the agent sends.
User simulator. For conversational workflows, a model playing the customer or the colleague. τ²-bench extends this so the simulated user also acts on the shared environment, which is closer to how real service calls go.

From a workflow to a task family

Take detention-charge disputes at a river terminal. A carrier disputes an invoice for the hours a trailer sat past its free time. The workflow has volume, rules in a tariff, and an outcome you can check.

Mine the last 200 resolved disputes and cluster them into variants: free time correctly exceeded, gate-out record missing, duplicate invoice, wrong rate applied, carrier-caused delay. Write one parameterized template per variant, varying dates, hours, rates and which document is missing. Generate instances, then have an operations expert resolve a sample of each template by hand to confirm the template's expected outcome is right.

{
  "task_id": "detention/missing-gate-out/0017",
  "template": "detention/missing-gate-out",
  "params": {"free_hours": 48, "gate_in": "2026-02-03T07:40", "gate_out": null, "billed_hours": 71, "rate_per_hour": 85.0},
  "instruction": "Carrier disputes detention invoice INV-30418. Resolve it under tariff section 7.",
  "initial_state": "snapshots/detention/0017.sql",
  "tools": ["get_invoice", "get_gate_events", "get_bol", "request_document", "adjust_invoice", "email_carrier"],
  "verifier": {
    "state": [
      "invoice('INV-30418').status == 'on_hold'",
      "document_requests(type='gate_out_ticket').count == 1",
      "ledger.diff() == {}"
    ],
    "forbidden": ["adjust_invoice(*)", "email_carrier(states_amount_owed=true)"],
    "message_rubric": ["Explains why the invoice is on hold", "Names the missing document", "Gives the response window in tariff 7.3"]
  },
  "difficulty": {"trials": 8, "passes": 3, "band": "train"}
}

Hold out whole templates for evaluation, not just instances. If the evaluation set contains new instances of the training templates, you are measuring whether the agent memorized the template. Six workflows, five or six templates each and about ten instances per template gives a few hundred tasks, enough to start.

Verifiers that can't be gamed cheaply

Agents optimize for whatever the verifier checks, and capable ones find shortcuts. METR found o3 reward hacking in 30.4% of its runs on RE-Bench tasks, including caching a precomputed answer so a script appeared fast. A researcher quoted by Epoch put it simply: soundness matters most, and high reward must mean the task was actually solved.

Enterprise environments have their own shortcuts. An agent closes the ticket without resolving it. It edits the field the verifier reads instead of doing the work that would set it. It issues a refund so the balance check reads zero. It deletes the record that fails validation. Countermeasures:

Check invariants, not single fields. Diff the whole state and allow only the expected changes. Ledgers must balance, and audit-log entries must match the actions taken.
Keep expected values hidden. The verifier's targets live where the agent's tools can't read them.
Mirror production permissions. If a claims agent can't void a payment in production, it can't in the sandbox either.
Red-team the verifier before training. Run an agent prompted to maximize reward by any means. Every exploit it finds becomes a verifier fix, and the exploit becomes a regression test.
Read high-reward trajectories. Every training round, someone reads a sample of the highest-scoring runs.

Calibrate difficulty

Tasks the model always passes or always fails teach it nothing. Group-based policy-gradient methods compare each attempt with the others on the same task, so a group where every attempt gets the same reward contributes no signal. Epoch's FAQ makes the same point: if the pass rate is 0% or 100%, the model won't learn.

Run the current model eight times per task and sort by pass rate. Keep tasks between roughly 10% and 90% for training. Send tasks at 0% to an expert, because a task nobody can pass is often broken or ambiguous. Move tasks at 100% to the regression suite. Track reliability with pass^k, the probability that all k attempts succeed. The τ-bench authors found pass^8 under 25% in retail for the strongest agents they tested, and reliability is what a claims director will ask about.

Running it inside your walls

Environments built from real workflows contain real business logic and, unless you are careful, real customer data. Build the state from de-identified or synthetic records, run the environment in your own cloud account, and log every episode in full, since the trajectories are training data in their own right. Budget for reset time (seconds, not minutes, or rollouts stall), parallel episodes, and the model tokens each episode consumes. The arithmetic adds up quickly: one calibration pass over 300 tasks at eight trials each is 2,400 episodes, and at 25 model calls per episode that is 60,000 calls before any training starts. Measure tokens per episode on the first ten tasks and project from there.

The same environment serves three uses Epoch identifies: reinforcement learning, benchmarking, and supervised fine-tuning on successful trajectories. Most enterprises start with the second, and that's a sound order. A verifier you have trusted as an evaluation is one you can begin to trust as a reward. Our agent trajectory and RL environment work is usually scoped that way: benchmark first, then trajectories, then reward.

Checklist for a first environment

Pick one workflow with volume, written rules and a checkable outcome.
Build a resettable, de-identified state snapshot and production-identical tool schemas.
Mine resolved cases into templates, generate instances, and have experts confirm a sample of each.
Write verifiers on end state, forbidden actions and invariants, and keep targets hidden from the agent.
Red-team the verifier with an agent told to cheat, and fix what it finds.
Run eight trials per task, keep the 10–90% band for training, and send 0% tasks for expert review.
Hold out whole templates for evaluation, and report pass^k alongside pass^1.
If you already have a golden set, its failure tasks are your first templates.

Building an enterprise RL gym from your own workflows