AI cannot replace human participants in behavioral research

AI tools offer researchers real benefits, including literature reviews, data analysis, and even drafting articles. A more troubling use of this technology, however, is becoming widespread. Researchers of human behavior are substituting AI-generated data for human responses in experimental studies.

Until researchers are able to resolve some fundamental concerns, behavioral research should not accept AI output as a substitute for human data. Image credit: iStock/bonezboyz.

AI responses—often called “silicon samples”—aim to mimic the answers that real people give to surveys and polls. A robust technical literature has emerged on how to build them, from one-shot prompting to full sets of “digital twins” constructed from thousands of survey responses (1–4).

The appeal is obvious. Silicon samples are cheaper and faster than recruiting human participants, and they let researchers study sensitive topics such as attitudes to drug use or bigotry without the ethical constraints. But are silicon samples broadly representative? We argue here that they are not—and that until three specific, fundamental concerns are resolved, the field should not accept AI output as a substitute for human data.

LLMs Can’t Know What They Don’t Know

Our first critique is that behavioral research exists to uncover previously undocumented patterns of human behavior. An AI solely trained on documented patterns cannot do that.

AI participants do replicate some well-established psychological findings. Large language models (LLMs), for instance, can exhibit overconfidence and show the same framing, certainty effects, and overweighting biases that researchers documented in humans decades ago.

But this is data leakage, not genuine behavioral simulation. The “replicated” phenomena arise from the AI training on published papers, rather than from true experiences (5). The limitations of AI are obvious the moment work with them moves beyond documented results into anything novel. LLMs generalize on the basis of tokens, not semantics (6–8). They predict likely sequences of words, not meaning.

It’s not sufficient for an AI to mimic a human. It must represent humanity.

That distinction sounds abstract until you see it in practice. In one study, researchers took a moral-judgment survey item—a campaign to release “wrongfully convicted prisoners”—and revised it to “rightfully convicted prisoners.” Human participants sharply reduced their moral approval of the revised campaign, as any person would. Yet, an LLM rated both versions as similarly ethical. It had learned the surface pattern “campaign to release prisoners = sympathetic” and never registered that reversing a single word also reversed the entire moral premise (6). The same study found that LLMs judged setting traps to kill stray cats and traps to kill rats as equally unethical, whereas humans reliably distinguished between the two. Again, the models tracked the surface frame (“killing an animal”) rather than the morally relevant distinction that humans recognize.

An AI participant may occasionally capture the direction of a comparable human sample’s behaviors, but it will always carry some irreducible error relative to a real human sample in unfamiliar territory—what we call representational error, distinct from ordinary statistical noise. Until researchers can specify how much of that error is tolerable and then demonstrate that AI samples fall under that threshold, every synthetic-sample finding will need independent validation against real human data. That validation requirement erases the biggest claimed advantage of synthetic samples: efficiency. It is the equivalent of testing food quality at every single meal rather than trusting the kitchen once.

AI Has No Mortal Coil

Short of being able to reliably reproduce every possible human response to any situation, a useful AI participant would need to reason the way people do. But AI lacks the very qualities that researchers are trying to study, including personality, cognition, emotion, and sensation.

Individual differences research continually develops new scales to quantify aspects of behavior and maps the relationships between them. There is no reason to expect an AI to demonstrate a trait through a scale it has never encountered, much less to map how it relates to underlying thought processes. Other methodological traditions, such as thought-listing protocols, response-time measures, eye-tracking, and physiological measurement, have no analog in an AI participant at all. That’s because these models have no actual cognitive processes.

Theory-driven experimental manipulation, arguably the behavioral researcher’s single most powerful tool, depends on human cognitive mechanisms that current LLMs simply do not have (9, 10). Consider emotions, physical and mental states produced by neural and physiological changes. Whenever an AI attempts to represent an emotional state, it selects the most probable next word given the context presented. AIs, lacking consciousness, do not bring their own experiences to answer prompts as an individual would, however fluent any description may be.

Researchers have built various frameworks for “artificial emotion” in AI systems, and these are useful for recognition and expression tasks. But even proponents of that work concede that meeting some technical requirements of an emotion model “does not suffice for emotion legitimacy in AI” (11). The same is true of embodied, sensory experience. An AI can fluently describe the sensation of a motion aftereffect illusion, for example, or the irritation of being cut off in traffic. But that description is drawn from other people’s prior descriptions in its training data, not its own “experience” in any true sense of the word.

This drawback drastically affects a major area of behavioral inquiry. Any subconscious process, or one connected to causes through multiple steps not shown by simple text associations, will be invisible to a model. LLMs approach the world by predicting likely words and can only convey what has already been written about an experience, decision, or emotion. Only human participants can convey reliable human nature to researchers.

We Are Large, We Contain Multitudes

Behavioral research samples need to reproduce a distribution of responses across a population. It’s not sufficient for an AI to mimic a human. It must represent humanity.

Evidence suggests that it cannot. AI participants can land close to the human average on some tasks, but this is because the probabilities regress to the mean response. Studies consistently show that LLMs compress or distort the shape of the distribution around that average because they do not accurately represent the probability distribution of these individual responses. Put differently, an AI output is likely to overrepresent the modal human’s responses, rather than those of the broader population. This results in AI replications routinely producing larger effect sizes than the human studies they’re meant to replicate (1).

Successfully reproducing a single variable’s distribution is only the beginning. To be useful, a synthetic sample must also reproduce the covariance structure that links a variable to every other variable a researcher might eventually want to test. Curiously, few silicon-sample research studies even check this, let alone report it. One study that does compare human and AI covariance structures in moral decision-making found that LLMs do not simply misrepresent human distributions; they actively amplify existing biases, compounding rather than approximating human variation (12).

Finally, if synthetic samples are to have predictive value for novel questions, they need to capture human causal models, the actual structure of what leads to what (13, 14). Take a simple case: does pleasant background music make consumers more likely to purchase, and, if so, why? Presumably, an AI can accurately represent the directional relationship between this music and purchase behaviors. But even then, to identify the causal process, it would need to also demonstrate corresponding relationships among variables. For example, if nostalgia causes this effect, then the AI would need to judge pleasant music as more nostalgic and also understand the positive relationship between nostalgia and purchase intentions.

In sum, we raise these three concerns not as Luddites keen to undermine cutting-edge technology, but because they represent fundamental limitations. The best available tool for understanding human behavior remains human participants. We are not against all uses of AI in behavioral science. Synthetic samples can help at the planning stage—testing item wording and experimental design, building analysis pipelines before real data collection begins, or identifying flaws in a study’s inferential logic. Still, planning is not testing, and synthetic samples should not be involved in the testing of hypotheses about human behavior.

The decision on what to publish belongs to journal editors. But we call on the behavioral sciences to pause the use of LLMs as subjects and address these three critiques before treating AI output as equivalent to human data. The use of synthetic samples misleads researchers, distorts the published record, and will impede the very science that these studies claim to advance.