Numerous industry groups have reported that the levels of respondent cooperation and response rates have been dropping over the past 20 years. Phone surveys have an average answer rate of less than 8% and of those, less than 4% agree to participate. In the early 2000s Web-based panels produced average response rates of around 48%, whereas today those rates have plummeted to less than a third of their original cooperation rates—and that is among opt-in panelists.
Over the years, we have tried to determine why people are not participating in surveys as often as in the past. A model specifically related to Market Research that I have found useful is “The Leverage-Salience Theory of Survey Cooperation” (Groves, Singer & Corning.) It is an off-shoot of Social Exchange Theory: You do something for me; and I’ll do something for you.
In general, the Leverage-Salience Theory says that there are two opposing forces driving survey participation on one hand, or survey avoidance on the other. Stated succinctly:
The Value Perception of Survey Participation = Personal Return – Burden
This first is the concept of Personal Return: Am I getting something worthwhile or of value to me out of this activity? Personal Return Reducers depress the respondent’s satisfaction and focus within the survey environment and include things like “poorly or obviously leading written questions,” “logic problems,” “incomplete answer options,” “solicitation of personal ‘targeting’ data,” etc.
The second force is Burden: How much of a pain in the neck is this activity going to be? Burden Intensifiers also depress satisfaction and focus and include many common survey design flaws such as “time needed to complete is too long,” “repetitive,” “boring,” “poor online usability,’’ or in phone applications “too much human interaction.” Understating the length of the survey in the invitation (aka lying), is a prime source of respondent complaints and mid-terminates.
In both of these cases, there are also positive applications that can be used to boost respondent satisfaction and increase attention. Burden Reducers include “keeping the survey short and to the point,” “allowing for completion at the respondent’s convenience,” “assuring anonymity,” “limiting the number of open-ends or other difficult question types (e.g. ranking large number of items, forcing things to add to 100%, etc.)”
But the area that is perhaps the most effective in boosting satisfaction and data quality is that of Personal Return Intensifiers. These include “offering appropriate levels of honorarium,” “enhancing the feeling that the activity is contributing value,” “expressing thanks for the effort and communicating appreciation,” and last, and perhaps most important “designing surveys that are entertaining and engaging.”
Successfully engaging respondents in the survey process may include techniques such as gamification of the questionnaire design or transforming the exercise into a partnering experience (“you are helping us build this new product.”) When respondents are engaged, they provide better, more thoughtful answers—resulting in higher data quality as well as much higher levels of survey experience satisfaction.
Another interesting phenomenon that occurs when elements of engagement are employed (particularly animations and gamified question structure) is that the respondent’s sense of “time spent” in the activity is reduced. In a side-by-side experiment, one group was asked a standard matrix question design, the second matched cell saw a drag-and-drop gamified version of the same question. Both cells were then asked to estimate the amount of time the exercise had taken them to complete. The first, standard question cell, was fairly accurate (on average within 6% of the actual time spent). But the second, gamified cell, underestimated the time spent by almost 20%.
Using this sense-of-time-reduction phenomenon, we see that engagement also acts as a Burden Reducer. We estimate that you can keep people “playing” a gamified survey between five and eight more minutes longer than a survey with only straight-forward, standard questions. Time does apparently “fly by” when you are having fun.
Over the years, we have tracked the relationship between satisfaction with one survey and the likelihood to respond to the next survey invitation within online panels and client communities. The link between the satisfaction and future participation is significant and produces a high correlation (r2 =.72). Since engagement is highly related to satisfaction, we conclude that an engaging survey today is also preserving respondent cooperation tomorrow.
Fostering engagement requires both intrinsic elements—producing a satisfactory and rewarding survey experience and extrinsic elements—building affinity and a sense of relationship with the survey sponsor.
Bill MacElroy is Chairman of Socratic Technologies, Inc.