Here’s a scenario every insights leader has lived through at least once. You need a fast read. You brief a low-cost panel provider. The data comes back in 48 hours, looks clean on the surface, ticks the boxes you specified. You present it to the leadership team. Someone — usually the person in the room who spends the most time with customers — looks at a number and says: “That doesn’t feel right.”
And they’re correct. It isn’t right. Because the panel you bought from was cheap, and cheap panels are cheap for a reason.
The hidden cost of low research panel quality is not the invoice. It’s the decision that goes wrong when the data is wrong, and the months it takes to unwind.
What you actually pay for when you buy cheap sample
A low-cost panel bid is not cheaper research. It is the same research with one crucial thing removed: the operational cost of keeping a panel honest. What that missing cost buys, when you skip it, is a set of problems that show up months later.
Re-fielding. When your stakeholders don’t believe the data, you run it again. That second wave erases your budget saving and doubles your timeline.
Strategic mistakes. When the data is directionally wrong but you believe it, you launch the wrong product, target the wrong segment, price at the wrong point. These errors do not show up on a research invoice. They show up on a P&L eight months later.
Reputational damage. An insights team that delivers findings the business cannot act on stops getting commissioned. Budget moves elsewhere. Seats get cut.
The invoice saves you a few thousand pounds. The consequences cost six figures.
Where cheap panels cut the corners
Every panel provider performs the same four operations: recruit respondents, verify they are who they say they are, screen them during studies, and look after them over time. Cheap providers cut corners on all four.
1. Single-channel recruitment
Premium panels recruit through multiple channels — targeted digital, professional association partnerships, offline intercepts, client-branded communities. Cheap panels rely on one channel, usually digital. That makes the panel easy to game: the same respondents, bounced between providers, learning the screening questions, learning which demographics pay best.
2. Weak or missing verification
A healthcare research panel that does not check NPI numbers in the US, GMC numbers in the UK, or equivalent registries elsewhere is a panel of self-declared doctors. A trade panel that doesn’t verify licences is a panel of people who say they are plumbers. A B2B panel that accepts free webmail for C-suite signups is a panel of whoever decides to tick the C-suite box.
3. No in-study quality layer
Good panels run every survey response through a quality stack: digital fingerprinting to detect duplicate respondents, IP and geolocation checks, attention traps, straight-lining detection, open-end gibberish detection, speeder traps. Cheap panels skip this. You get data that looks complete but is infested with respondents who answered randomly to collect the incentive.
4. Over-surveyed respondents
Premium panels throttle invitations. The average member receives a limited number of survey invites per month to prevent fatigue and preserve considered answers. Cheap panels push their members constantly. The result is professional respondents — people who treat panel completion as a side income, race through surveys, and tell you whatever they think will get them to the incentive fastest.
Ten questions to ask before you sign a panel contract
Regardless of who you buy from, these questions separate serious providers from everyone else. Ask them. Watch how specific the answers are.
- How do you recruit respondents? You want to hear about at least three channels, not one.
- How do you verify credentials for specialist audiences? Specific registry names. If they can’t name them, walk.
- What percentage of completes do you typically reject post-field, and why? A real answer is somewhere between 8% and 20%. “We don’t reject any” means their quality stack is broken.
- How often do you refresh respondent profiles? Anything less than annually is a red flag.
- What’s your average invitations-per-member-per-month? Less than eight is healthy. More than twenty means the panel is being farmed.
- Can you provide an anonymised sample of your attention-trap failure rate? Real data means real quality control. Vague answers mean they don’t measure it.
- How do you prevent duplicate respondents across studies? Digital fingerprinting, device IDs, email hashing — there should be specifics.
- What certifications do you hold? ISO 20252 is the baseline for market research. ISO 27001 for information security. ESOMAR, MRS or Insights Association corporate membership.
- For healthcare work, do you comply with BHBIA, HIPAA and applicable pharmacovigilance protocols? If they ask what those acronyms mean, the call is over.
- Can you share a case study where you detected and removed fraudulent respondents mid-study? Every serious panel has done this. If they can’t give an example, they don’t have the tooling.
Red flags in the pitch deck
Beyond the questions, there are patterns in how cheap panels present themselves that tell you everything you need to know.
They quote large panel sizes without breaking down active respondents. A “5 million member panel” where only 8% respond is a 400,000-respondent panel with 92% dead weight.
They promise 24-hour turnaround on any study, any country. Fast is good. Fast-regardless-of-complexity means they’re not screening.
They describe verification in generic language — “respondents are verified” — without specifics about how, against what database, at what cadence.
They can’t show you the senior researcher who would actually run your project. Because there isn’t one. Your brief will be handed to whoever has capacity.
What panel quality actually looks like
At MSG Opinion we organise our work around four pillars: recruit, verify, screen, protect. Multi-source recruitment across targeted digital, professional associations and offline outreach. Credential verification against NPI, GMC, NMC and equivalent national registries on every healthcare professional; licence verification on every tradesperson; LinkedIn and corporate email validation on every B2B respondent. An in-study quality stack running on every survey we field. Invitation throttling, fair incentives, and opt-out-anytime respondent welfare policies that keep engagement rates roughly double the industry average.
These are not differentiators we invented for marketing. They are the baseline operational costs of running a panel that deserves to be trusted with decisions that matter.
The economics are simple
Good research panels cost more because the operational work behind them is real. What you are paying for is not sample; it is the absence of problems. No re-fielding. No embarrassing stakeholder meetings. No strategic mistakes that trace back to bad data.
If you’ve been burned by a cheap panel recently, you’re not alone, and you now know the questions to ask next time. If you’d like to see how we evaluate research panel quality for our own audiences across consumer, B2B, healthcare, patient and specialist panels, we’d be happy to walk you through it.
Request a proposal or explore our research panels to see what rigorously-recruited sample actually looks like.