Economists have a favorite imaginary creature: Homo economicus, the perfectly rational human who exists mostly in textbooks. In 2023 John Horton (opens in a new tab), with Apostolos Filippas (opens in a new tab) and Benjamin Manning (opens in a new tab), introduced a new one. Their NBER (opens in a new tab) working paper, Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus? (opens in a new tab), argues that large language models "are implicit computational models of humans, a Homo silicus."
That sounds grand. The method is charmingly simple: take famous experiments from behavioral economics, give the model the same scenario and the same choices, and see whether it behaves like the people in the original studies.
The snow shovel test
One of our favorite examples is Kahneman (opens in a new tab), Knetsch (opens in a new tab) and Thaler (opens in a new tab)'s 1986 fairness question. A hardware store has been selling snow shovels for $15. The morning after a big snowstorm, it raises the price to $20. Is that fair? In the original study, "about 82% of subjects responded either 'Unfair' or 'Very Unfair.'" Horton's simulated agents, given political leanings and other personas, show the same broad pattern: price gouging after a storm feels wrong. The authors summarize their replications as showing "qualitatively similar results to the original," and, importantly, when the results differ, "it is often generative for future research."
You can pilot an experiment in an afternoon
The practical idea here is the one we built Cast around. Before you spend months recruiting people, you can run a cheap simulated pilot, see which conditions behave interestingly, and sharpen your questions. Other teams found the same thing. Aher, Arriaga and Kalai replicated (opens in a new tab) the Ultimatum Game (opens in a new tab), garden-path sentences and even the Milgram (opens in a new tab) shock experiment with simulated participants, reporting that "the existing findings were replicated using recent models."
At Harvard Business School (opens in a new tab), Ayelet Israeli (opens in a new tab) and colleagues tried it on shopping. In coverage by HBS Working Knowledge (opens in a new tab), they describe giving simulated customers incomes between $50,000 and $120,000, and finding that higher income went with higher price tolerance, just as you would expect. Israeli was careful to add: "We're not saying everyone should now use this instead of talking to consumers."
Where Homo silicus trips
Horton's own revised paper is admirably blunt. Published research sits inside the training data, which "can also make brittle simulations appear robust because models may simply regurgitate problematic and memorized results." George Gui (opens in a new tab) and Olivier Toubia (opens in a new tab) go further in a causal-inference critique (opens in a new tab): change one thing in a simulated experiment and the model quietly changes other things too.
So we treat Homo silicus the way a good researcher treats a pilot study. It is fast, it is revealing, and it tells you where to look. It does not sign the final check.
How we use the idea
Cast runs your study the way these researchers ran theirs, only with your customers instead of textbook scenarios. Your questions ask about past behavior before the concept, prices are tested as a range rather than a single anchored number, and the Critic sends back answers that look memorized or too eager to please. Then we tell you which findings deserve a real-world check.
The four experiments, in plain English
The original Homo silicus paper re-ran a handful of well-known studies. Beyond the snow shovel fairness question, it looked at social preferences (how much people value fairness and other people's payoffs when splitting money), status quo bias (whether people stick with a default option even when a better one exists) and a hiring scenario about the effect of a minimum wage on which candidates employers choose. In each case the simulated agents were given different "endowments", such as political leanings, to see whether those shifted their choices in the expected direction.
The striking result is not that the model matched every number. It is that it moved in the right direction when the scenario or the persona changed. Make the agent more libertarian and it tolerates the price rise more. Present a default and the agent tends to keep it. That directional sensitivity is what makes a simulation useful for exploring ideas.
Why a pilot is worth more than it costs
Real behavioral experiments are expensive in money and in time, and most of that cost is spent before you know whether your design is any good. A simulated pilot lets you check that your scenario is understandable, that your conditions actually differ, and that the effect you care about is plausible. If the simulation shows nothing interesting, that is a cheap warning. If it shows something surprising, you know where to dig.
How to run a Homo silicus style pilot in Cast
- Write the scenario exactly as a real participant would see it, without hinting at the answer you hope for.
- Create two or three versions of the offer (for example, two prices or two messages) and run each with a separate cast.
- Compare the direction and the reasons people give, not just the headline numbers.
- Keep the version that survives, and test it with real customers in a smaller, sharper study.
From one experiment to a research robot
The Homo silicus paper asked whether a language model could stand in for a participant. A year later, Benjamin Manning, Kehang Zhu and John Horton asked a bigger question: could it also stand in for the researcher? Their paper, Automated Social Science: Language Models as Scientist and Subjects (opens in a new tab), builds a system that proposes hypotheses, designs simulated experiments around them, runs them with AI agents and analyzes the results.
They tried it on four scenarios: a negotiation, a bail hearing, a job interview and an auction. The system found evidence for some of the causal relationships it proposed and not for others. The most useful finding for anyone running customer research is this one: when the model predicted results directly, it was "good at predicting the signs of estimated effects, but it cannot reliably predict the magnitudes." In other words, it knew which way things would move, but not how far.
“In short, the LLM knows more than it can (immediately) tell.”
That line is the best argument we know for running a simulation rather than just asking a chatbot for its opinion. When the agents actually played out the auction, their results "closely match the predictions of auction theory", while the model's direct guesses about clearing prices were inaccurate. Simulation surfaces things that a single question does not.
The open-source side
Horton also co-founded Expected Parrot (opens in a new tab), which maintains EDSL (opens in a new tab), an open-source Python package for running surveys and experiments with AI agents. It is a good place to see how these studies are wired together. Cast is not built on it.
LLM persona prompting best practices, from the research
The economics papers and the persona literature point to the same handful of habits:
- Give the agent an endowment that matters to the decision, such as a budget, a past experience or a political leaning, rather than a long list of irrelevant traits.
- Change one thing at a time between conditions.
- Run many agents, not one. A single answer is an anecdote. A spread of answers is data.
- Trust direction more than size. Several papers, including this one, find models are better at signs than magnitudes.
- Avoid scenarios the model has memorized. Famous textbook puzzles are the easiest to recite and the least useful to test.
A cautionary result is worth knowing here. Peter Park (opens in a new tab), Philipp Schoenegger and Chongyang Zhu ran 14 studies from the Many Labs 2 replication project with GPT-3.5 and found (opens in a new tab) what they called a "correct answer" effect: on some nuanced questions, different runs gave "zero or near-zero variation in responses." A simulated population that all agrees with itself is not a population.
A worked example for a pricing pilot
Imagine a founder selling scheduling software to independent physiotherapy clinics. She is unsure whether to charge per practitioner or a flat monthly fee. In a Homo silicus style pilot she would write the offer exactly as a clinic owner would see it, then run two separate casts: one sees per-practitioner pricing, the other sees the flat fee. Everything else stays the same.
What she looks for is direction and reasons. Do owners of two-person clinics push back harder on the flat fee? Do larger clinics worry about per-seat costs as they hire? Either way, she walks into her real experiment with sharper questions. AI users never see which option she prefers, the Director interviews each one in parallel, and the Critic flags answers that look memorized or too eager to please.
Glossary
Homo silicus: a language model treated as a computational model of human behavior. Endowment: the information, preferences or circumstances given to a simulated agent. In silico: run as a computer simulation rather than with people. Structural causal model: a formal map of which variables are believed to cause which, used here to design and analyze simulated experiments.
Questions about simulated economic agents
If the model has read the original papers, isn't it just reciting them?
That is the most important caveat, and the authors raise it themselves. It is why we encourage you to test your own, new scenarios, which no model has seen before.
Do different models give different answers?
Yes. The revised paper compares many models. Cast lets the platform team assign the model used for each role, and we re-check results when models change.
Can this replace A/B tests?
No. It helps you decide which A/B tests are worth running, which is often where most of the time is lost.
Can AI agents tell me how much a price change will hurt conversion?
Not reliably. Research on simulated agents finds they are much better at predicting the direction of an effect than its size. Use them to decide which price to test, then measure the size with real customers.
Why run a simulation instead of just asking the model what will happen?
Because the two give different answers. In the Automated Social Science paper, simulated auctions matched economic theory closely while the model's direct predictions of prices were inaccurate.
Read next: can AI predict the result of an experiment before you run it?
