Maximizing Quality in a Modern Online Sampling World: Overcoming Today’s Challenges
By Brian Lamar, Chief Insights Officer, EMI – Research Solutions and MRR
It’s no secret that online sampling today is going through quality challenges, much like market research methodologies have gone through in the past—and continue to go through. These challenges are complex and interconnected, and this makes finding solutions for them difficult.
Challenges in Modern Online Sampling
The narrative about online sampling providing poor quality often arises from a variety of challenges that are all interwoven with one another. They include:
Perception of Commoditization: The challenges really start with the perception by many that online sample is a commodity. This belief couldn’t be farther from the truth. Much like snowflakes, panels are unique and exhibit distinct differences from one another, whether because of their recruitment practices, management, incentives, and validation methods. These variations, often overlooked, play a crucial role in the quality of data received from different panels.
Low Incentives: The diminishing incentives in surveys, a byproduct of the perceived commoditization, raise concerns about respondent engagement and data quality. Understanding and ensuring reasonable incentives is pivotal to mitigate non-response bias.
New Measures of Success: The adoption of measures like Cost Per Complete (CPC) or Earnings Per Complete (EPC) from the advertising industry into online sampling introduces a new dynamic. While they make sense from a business profitability standpoint, these measures significantly impact the traffic, feasibility, timing, and cost of surveys—presenting challenges in predicting study timing, managing costs, and understanding study representativeness.
Lack of Engagement, Representivity, and Distrust: These intertwining challenges contribute to the industry’s struggle to maintain data quality and reliability. Distrust in data validity, lack of engagement, and representivity issues hinder the accuracy of research outcomes.
Poor Quality and Fraud: There have been some alarming statistics released that highlight the prevalence of fraudulent respondents within the sample industry. This is casting a shadow over the integrity of data.
Imperium, a Dynata-owned company, released data that suggested that, on average, 38% of respondents taking surveys are dupes or fraudulent/bots.
PureSpectrum, an online sampling marketplace, recently wrote about survey quality and looked at block rates over the past 3 years. They found that between 10-20% of traffic (outside of the Middle East and Africa) is blocked because of poor quality.
EMI, using our internal data and our QOR metric, found that we were blocking between 14-15% of overall traffic due to fraud and duplication.
Professional Survey Takers: According to a study by CASE (Coalition for Advancement of Sampling Excellence), and supported by EMI, 50% of respondents admitted to taking surveys daily while 66% took multiple daily. While the industry is in the learning phase on how many surveys a respondent should access each day, there is no doubt that these respondents can impact data.
There Is Hope
While these challenges seem daunting, it’s crucial to contextualize them. The online research industry isn’t alone in grappling with fraudulent activities. Cybercrime is estimated to cost the world $10.5 trillion annually by 2025. From ticket scalping to the stealing of personal data through data breaches, the challenges posed by fraudulent activities extend beyond survey research, creating a global challenge.
In the face of these challenges, industry collaborations and technological advancements offer hope. There are various digital fingerprinting fraud products now available, and each is pushing the others to get better. Collaboration by industry associations like ESOMAR, Insights Association, Market Research Society, and others are spearheading initiatives to enhance information sharing and trust-building within the industry.
Navigating these challenges necessitates a strategic approach. Leveraging the expertise of sample experts and embracing a holistic understanding of the evolving online sampling landscape becomes imperative for building ideal sample plans. This approach minimizes risks, improves data quality, and enhances decision-making capabilities.
In this evolving space of online sampling, these challenges, rather than being deterrents, serve as catalysts for innovation, collaboration, and a commitment to elevating the standards of data quality in market research.
