Research is only as global as its evidence.

Zoom automation, web panels, transcription pipelines — the modern research stack quietly assumes a digitally fluent participant. In Vietnam, Indonesia, Brazil, Mexico, people often won’t, or can’t, complete digital onboarding or sit for a remote study.

We learned this operating a marketplace across 130+ countries: the method breaks long before the insight does.

GPT–human response similarity falls with cultural distance from the United States at r = −.70.Atari et al. 2023, World Values Survey

The pattern replicated across five GPT generations and 107 countries.Tao et al. 2024, PNAS Nexus

The model is most confident where it is least valid. WEIRD in, WEIRD out.

Synthetic panels compress variance to about half of real humans — SD 16.1 against 31.4. Roughly 48% of regression coefficients differ significantly from human data, and the sign flips in about 32% of those.

Identical prompts on different model versions return different distributions. RLHF caricatures minority groups worst of all. The mean survives; the structure does not.

For cheap exploratory piloting, ideation, and questionnaire debugging, synthetic panels are extraordinary: about 5,441 responses for roughly $0.70, inside an hour.

The discipline is knowing the ceiling. Replicability breaks below ~200 samples, and none of it is verified outside WEIRD populations. Use it where being wrong is cheap.

Kantar, NIQ, Qualtrics — the incumbents ran the experiments and retreated to the same line.

Even the best statistical hybrid — calibrating 100,000 model responses against 10,000human ones — adds only about 13% effective sample size.

The bottleneck is real people, in hard places, answering honestly. Whoever can collect real human data cheaply in difficult markets owns the one thing a model upgrade can’t erode.

The MIT and Nature Human Behaviour meta-analysis of 106experiments is blunt: human–AI teams win only when the human is the stronger performer, and lose when the AI is.