Onshore Data Annotation vs. the Crowd

Crowd platforms made labeling cheap. Post a task, thousands of anonymous workers pick it up, and results arrive by morning. For a lot of work that's the right tool: sentiment on public product reviews, bounding boxes on street scenes, relevance judgments on public web pages.

For sensitive data the deciding question isn't price per label. It's who is legally allowed to see the record, and whether you can show afterwards who did. What follows describes the rules that answer that question for the data regional enterprises most often want to train on, the quality evidence that matters regardless of the rules, and a decision table. It describes; it doesn't advise.

Three questions decide it

For every dataset, ask:

Does a law or contract restrict who may see these records by nationality, location or relationship?
Does it require an agreement that runs all the way down to the person reading the record?
Can you show, after the fact, exactly who viewed which record, from where?

An anonymous crowd fails the third question by design. The platform knows its workers. You don't, and you usually can't find out.

Export-controlled technical data

Under the ITAR, a U.S. person is defined in 22 CFR 120.62: U.S. citizens and nationals, lawful permanent residents and certain other protected individuals, or an entity incorporated to do business in the United States. Releasing technical data to a foreign person inside the United States is a deemed export (22 CFR 120.50), and a release includes visual inspection and giving someone access credentials that let them view unencrypted technical data. The EAR works the same way for controlled technology and source code: a release in the United States to a foreign person is a deemed export to that person's most recent country of citizenship or permanent residency (15 CFR 734.13(b)).

This reaches training data more often than people expect. Manufacturers, energy companies and aerospace suppliers want models that read work orders, maintenance manuals, inspection reports and drawings. If any of that is controlled, a labeling task that asks someone to box the components on a drawing is a release. With an anonymous crowd, you can't show who saw it.

Two cautions cut the other way. Onshore isn't the same as U.S. person: a foreign national on a work visa in Jackson is a foreign person for these rules, so an onshore vendor still needs person-level screening. And classification is your export control officer's call. The data team's job is to flag suspect documents early and route them to people who are cleared to see them.

Health data: the business associate chain

A vendor that creates, receives, maintains or transmits protected health information on behalf of a covered entity is a business associate, and so is a subcontractor doing the same for a business associate. Each link needs written assurances that the information will be safeguarded.

HIPAA does not prohibit offshore work. HHS guidance says a cloud provider may store ePHI outside the United States under a business associate agreement, while noting that the risks can vary greatly by location, including the enforceability of protections, and that entities should weigh this in the risk analysis the Security Rule requires.

So the test for a crowd is concrete. The chain of agreements has to reach the person reading the discharge summary. Ask the platform what binds the individual worker, how each record view is logged, and where the worker sits. If the answers are vague, so is the chain.

De-identifying first reduces what anyone sees, and it's worth doing whoever does the labeling. It doesn't remove the question, because redaction never reaches perfect recall; our piece on de-identification for LLM training does the arithmetic.

Bulk personal data and countries of concern

The Justice Department's Data Security Program rule, effective April 8, 2025, restricts vendor, employment and investment agreements that give a country of concern or a covered person access to bulk U.S. sensitive personal data, unless specified security requirements are met. The countries of concern are China, Cuba, Iran, North Korea, Russia and Venezuela.

The bulk thresholds are low for training-data projects. Personal health data or personal financial data on more than 10,000 U.S. persons qualifies, as do covered personal identifiers on more than 100,000, measured over any 12 months; thresholds for biometric, geolocation and genomic data are lower still. A year of claims notes from a regional insurer clears 10,000 people easily. The rule defines a vendor agreement broadly, as any agreement to provide goods or services for payment, so an annotation contract is one. Knowing who is in a vendor's workforce, and where, is now part of buying labeling for consumer data.

Other location rules

Some regimes are blunter. The IRS states that federal tax information is not allowed offshore and requires agencies to certify that contractor personnel and systems with access to it are in the United States or its territories. Beyond statute, client contracts in banking, insurance and the public sector frequently carry their own location and personnel clauses. Read the master services agreement before choosing a labeling model, not after.

The quality case, apart from the law

Even where the rules permit a crowd, sensitive domain data tends to need people you know.

The labels may not be human. Veselovsky and colleagues estimated that 33–46% of crowd workers on Mechanical Turk used LLMs on a summarization task. For preference data and evaluation sets, whose entire value is human judgment, that is contamination of the signal you're paying for.

Price the usable answer, not the task. In a 2023 comparison of research platforms, Douglas and colleagues paid $4.36 per high-quality respondent on MTurk against $1.90 on Prolific: MTurk respondents were less likely to pass attention checks and give meaningful answers, so each usable answer cost more. Cheap per task can be expensive per usable label.

Domain tasks need domain people. Assigning denial codes, reading maintenance fault histories or judging whether a contract clause shifts liability takes training a marketplace doesn't screen for.

Agreements alone don't control people. MIT Technology Review obtained 15 screenshots from development robot vacuums, one showing a woman on a toilet, that contracted labelers had posted in private online groups. iRobot said it had shared over 2 million images with its annotation vendor, and both companies said the posting breached their agreements. The agreements existed. Nothing in the working environment stopped a screenshot.

The controls that do work are environmental: employed staff with background checks, access only through managed virtual desktops in the client's own tenant, copy, paste and screen capture disabled, every session logged, nothing on personal devices. Knowing who labeled what also makes quality measurable. We hold reviewers to at least 95% on hidden gold questions and put a second reviewer on at least 10% of every batch, which is only possible when every label has a named author.

A decision table

This is practitioner judgment, not a legal determination; the right answer for a given dataset comes from your counsel and, where relevant, your export control officer.

Data	What usually governs it	Anonymous crowd	What the team needs
Public web text, product reviews, street scenes without faces	Platform terms, copyright	Often workable	Gold questions, agreement checks
De-identified text with measured low residual risk	Contracts, privacy commitments	Sometimes workable	Residual-identifier measurement, logged access
Protected health information	HIPAA business associate chain	Only if agreements and logging reach each worker	Named staff, logged access, often a contractual US-location requirement
Health, financial or identifier data above DOJ bulk thresholds	DOJ Data Security Program	Only with workforce diligence	A known workforce with no covered-person access
ITAR or EAR controlled technical data	Export controls	Not without authorization	Screened U.S. persons or a license, controlled environment
Federal tax information	IRS requirements	No	Onshore personnel and systems

Our own model sits at the strict end of this table: an employed, onshore, U.S.-person team working inside the client's environment, described on our security page. For specialist judgment we add domain experts from our expert network, who join as part-time employees under the same checks and agreements.

What to ask any vendor, crowd or not

Who, by name and role, will see records, and are they employees or contractors?
Where do they sit, and what is their export-control status if the data needs it?
What agreement binds each individual, not just the company?
Do records leave your tenant, and if so where do they go?
Is every record view logged, and will you get the logs?
Can workers copy, download or screenshot records?
How does the vendor detect labels written by an LLM?
What are the quality gates: gold questions, agreement thresholds, second review?
When is data deleted after the project, and do you get a written deletion record?
How quickly will you be told about an incident?

Onshore vs. crowd: when sensitive data needs a US-only data team