Most of what we write is about what a synthetic panel is good for. This one is the other list, and it exists because a tool that never tells you when not to use it isn't actually being honest about its limits. It's just marketing with extra steps.
“Don't use one to validate a decision you've already made emotionally.
If you've decided to ship the pricing page and you're running a panel hoping it agrees with you, you'll unconsciously read the results that way regardless of what they say. A panel is most useful before you're attached to an answer, not after, when it's being asked to rubber-stamp a decision rather than test one.
Don't use one where the stakes are high enough that being wrong is expensive and irreversible. A pricing experiment you can walk back in a week is a reasonable place for a five-minute directional check. A decision to lay off staff, sign a multi-year contract, or make a safety-relevant product change is not: those deserve real research, real numbers, and ideally a professional who does this for a living, not five generated personas.
Don't use one to represent a population it has no real basis for representing. A panel built from a two-sentence brief about "small business owners in India" is working from broad, generic training-data associations about that group, not lived specifics. The more niche, local, or culturally specific your actual audience is, the less a generic language model has genuinely absorbed about them, and the more a panel's confident-sounding answer can be confidently wrong.
Don't quote its numbers as if they were survey data. "Three out of five personas said yes" is not a 60% adoption rate. It's three qualitative opinions from generated characters, and treating it as statistically meaningful borrows the language of real research without the sample size, sampling method, or real respondents that make that language meaningful in the first place.
“And don't use one as the last step before shipping something that matters.
The honest place for a synthetic panel is early and often: before you've written the real survey, before you've booked the real interviews, before you've committed to the direction. If it's the only check something gets before it goes live, it's being asked to do a job it was never built to do.