The most dangerous result in customer research is unanimous love. Real markets are never unanimous, so a study where everyone adores your idea is telling you about the study, not the market.
Two known problems
Language models have two tendencies that make unanimous love more likely. The first is agreeableness. Research on sycophancy (opens in a new tab) found it is "a general behavior of state-of-the-art AI assistants." The second is sameness. When researchers re-ran classic studies with a language model in Diminished Diversity-of-Thought (opens in a new tab), several could not even be analyzed because the answers were nearly identical. And Bisbee and colleagues found synthetic survey answers showed "less variation in responses than in the real surveys" (opens in a new tab).
Fix one: cast skeptics on purpose
Every Cast audience includes people who start unconvinced: the operator whose fleet is already paid off, the finance lead who has been burned by a vendor, the shopper who never tries new brands. They are not contrarians for show. Each has a reason rooted in their circumstances, and that reason is exactly what you need to hear before launch.
Fix two: Fuzziness
For each audience, you set how different the AI users should be from one another on every dimension that matters: age, budget, attitude to risk, how tech-savvy they are, and anything else you add. Low Fuzziness gives you a tight, typical group. High Fuzziness gives you the edges of your market, where surprises live. It is the antidote to the narrow middle.
Fix three: they never see your hypothesis
An AI user does not know what you hope to hear. The Director never pitches. And if the whole cast still loves your idea, the Critic flags the study itself, not just individual answers, because unanimous agreement is a signal that something about the setup is too easy.
“If everyone loves your idea, we tell you to worry.”
Why this produces better outcomes
Groups in real rooms suffer from the same problem in a different way. Solomon Asch (opens in a new tab)'s classic conformity experiments (opens in a new tab) showed how people bend toward the majority. Cast's one-to-one interviews avoid the room, and deliberate skeptics avoid the echo. What is left is disagreement you can learn from.
Groupthink, in people and machines
Irving Janis coined the word groupthink (opens in a new tab) for the way cohesive groups suppress doubts to keep the peace. Many organizations fight it by appointing someone to argue the other side, a tradition that goes back to the Catholic Church's devil's advocate (opens in a new tab). Cast does something similar, but with an important difference: our skeptics are not contrarians for show. They doubt for reasons rooted in their circumstances.
What a good skeptic looks like
A good skeptic has a reason to say no that a real customer would have. The operator whose current system is paid off and works well enough. The procurement lead who needs three quotes for anything over a threshold. The shopper who tried a similar app last year and deleted it in a week. Each objection is specific, and each one points to something you could fix, price differently or position against.
Setting Fuzziness well
- Start with medium Fuzziness for a first exploratory study.
- Lower it when you want to understand one tight segment in depth.
- Raise it when you are not sure who your customer is, to discover unexpected segments.
- Add your own dimensions, such as "years in role" or "uses a competitor", when they matter to your decision.
Protecting yourself from confirmation bias
The real target of skeptics is not the model. It is us. Confirmation bias (opens in a new tab) makes founders notice the praise and skim past the doubts. Cast reports put objections next to enthusiasm, ranked by how often they came up, so the uncomfortable findings are hard to miss.
How to avoid confirmation bias in customer interviews
Confirmation bias is the habit of looking for evidence that supports what we already believe. In a classic 1960 experiment, Peter Wason asked people to discover a simple number rule and found that most of them tested only examples that fitted their first guess (opens in a new tab), rarely trying anything that might prove it wrong. Raymond Nickerson's later review called confirmation bias a ubiquitous phenomenon in many guises (opens in a new tab), showing up everywhere from science to everyday reasoning.
Customer interviews are a perfect breeding ground for it. You know your idea, you are excited about it, and the person across from you is usually polite. Rob Fitzpatrick (opens in a new tab)'s book The Mom Test (opens in a new tab) is popular with founders precisely because it tackles this: talk about the customer's life and past behavior, not your idea, and do not mistake compliments for commitment. Skeptics in Cast are a structural version of the same advice. Rather than relying on you to ask brave questions, the cast itself includes people who will push back.
Practical habits that work with real and synthetic users
- Write down, before the study, what answer would make you drop or change the idea. If nothing could, you are not testing it.
- Ask about the last time the problem happened, not whether the person would buy a solution.
- Read the skeptics' transcripts first, before your enthusiasm is set by the fans.
- Count objections as carefully as you count praise, and note which ones repeat across different kinds of people.
- Run a premortem: Gary Klein's project premortem (opens in a new tab) asks a team to imagine the project has already failed and explain why. Your skeptics have effectively written that list for you.
A walk-through: a study with three skeptics
Imagine a founder building route-planning software for small courier firms in Riyadh. He sets up a cast of twelve with medium Fuzziness. The AI users do not see his hypothesis, which is that couriers will pay monthly for fuel savings. Three AI users start unconvinced, each for a different reason. One runs six vans and plans routes in his head after fifteen years on the job. One is a finance manager who has been burned by software that needed weeks of setup. One is a driver-owner who already uses a free maps app and cannot see the gap.
When the founder reads the report, the enthusiasts mostly praise the idea in general terms. The skeptics are specific. The experienced owner says he would only trust a tool that let him override a route without fighting it. The finance manager says she would want a short trial, with no lengthy onboarding. The driver-owner never comes around, which tells the founder that one-van operators may not be his first market. None of this is a final answer, and the example is hypothetical, but he now has three concrete things to test with real couriers instead of one vague feeling that people liked it.
A short glossary
Confirmation bias: favoring information that supports what you already think. Skeptic: in Cast, an AI user who starts unconvinced for reasons rooted in their circumstances. Fuzziness: the setting that controls how different AI users are from one another. Premortem: imagining a future failure in order to find its causes now. Leading question: a question that signals the answer you hope for.
Limitations and fair warnings
Skeptics are a counterweight, not a substitute for real doubters. A synthetic skeptic's objections are grounded in research about people like them, but a real buyer may raise something no one predicted, such as an internal politics issue or a contract they cannot exit. Skeptics also do not tell you how many real customers share an objection. Use them to find the questions, then use real conversations and data to measure the answers.
Questions about skeptics
Will skeptics make my idea look worse than it is?
They make it look more realistic. A skeptic who can be convinced tells you what it takes to win. A skeptic who cannot tells you who not to target.
How many skeptics are in each audience?
It depends on the audience and Fuzziness, but every audience includes at least one AI user who starts unconvinced.
How do I stop leading the witness in customer interviews?
Ask about specific past events rather than hypothetical futures, avoid describing your solution before you understand the problem, and do not react warmly to praise. In Cast, AI users never see your hypothesis, which removes one common source of leading.
Should I remove skeptics who will never buy?
No. A skeptic who cannot be convinced is telling you who your product is not for, which is valuable when you decide where to focus sales and marketing.
Read next: twelve interviews at once, and why parallel beats a panel.
