DIY survey research

I’m old enough to remember typing pools – though not old enough to remember when the secretaries in them were using typewriters. They were using word processors – not software like Microsoft Word but hardware like the Wang OIS, a dedicated system for timeshared word processing. Researchers would handwrite their reports and give them to the typing pool to type up.

With the IBM PC, the researchers themselves became typists, though there were many jokes about having “ten thumbs” and “hunt and peck” typing. The researchers would type the report and then give it to the desktop publishing pool to fix the typos, produce the charts and graphs, and format the reports that would go to the clients.

I’m also old enough to remember desktop-publishing departments – and I remember when the LaserWriter printer packed the most powerful computer of any Apple system sold.

With the Macintosh, the researchers became the desktop publishers too and were now doing tasks that dedicated professionals had done before. Even though this led to “ransom note” formatting and non-esthetic results, the typing pool and desktop publishing pool staff were let go.

Has the time now come for survey researchers themselves to follow typists and desktop publishers out the door and into obscurity? Will the quality of survey research decline as a result?

End users create 500,000 surveys a month in SurveyMonkey and 200,000 a month in SurveyGizmo.

Survey software, like desktop-publishing software and word-processing software before it, has transformed a generation of business people into Do-It-Yourself researchers. More people are conducting surveys than ever before. Many of these are people who would not be able to afford to hire a researcher: doing it themselves is the only option.

Yes, the quality is often atrocious. What typos were to WordPerfect documents and what awful font choices were to Harvard Graphics presentations, leading questions and bad scales and missing skip patterns are to survey software.

Yet the worst surveys I have ever fielded were written by market researchers. The 200 questions all using an agreement scale (known to be obsolete since the 1960s). The 10 x 10 matrix of 1-to-10 dropdown boxes – 100 questions on a page! Questionnaire after questionnaire failing to reflect research-on-research best practices or even a basic awareness that consumers are predictably irrational — not number-crunching automatons.

The amateur starting with a professionally written questionnaire template has a better foundation to start with.

So I celebrate the Do-It-Yourselfer!

Many will even realize that it’s harder than it looks to conduct good survey research. And those will be our customers of the future: they will have some hands-on practical experience with having done a survey, and a respect for the effort involved. They will appreciate expertise, but they will pay lower rates for it than ever before.

No, the survey researcher will not go the way of the typist or the desktop publisher.

We will be landscapers in a country where most people own lawnmowers. Suitably humbled, following behind our self-propelled survey engines. And rededicating ourselves to better knowing our craft than we did before.

Jeffrey Henning (@jhenning) is president of Researchscape International and a Board member of the MRII. He doesn’t mow his own lawn.

3 thoughts on “DIY survey research

  1. I believe the #MRX professional starting with a professionally written questionnaire template has a better foundation to start with than an amateur! Sure some of the worst surveys you ever fielded were written by market researchers. However, those market researchers could have been forced into those lousy survey designs by any number of forces. If you could take a close look at the 500K surveys per month in SurveyMonkey written by amateurs, the sheer volume of lousy surveys would make you cringe.

    There is a big disconnect in the typing analogy. When a hunt and peck typist finishes typing, they can check their work and assess how they did. Most of the time when an amateur designs a survey, they have no idea how truly bad the survey the created is.

  2. Yes, absolutely! I was a very nervous amateur before getting kickstarted with my first survey. But you can say that I was fortunate enough to pick up sogosurvey, a DIY survey tool. Initially I was contemplating survey monkey, which costs a bomb. But then reading a few blogs here and there, I realized that sogosurvey is quite affordable and known for its quality+ performance + ease of use. So I thought to myself, WHY NOT! I think this software is a savior, especially when you are not an expert and have time constraints, and cannot settle down for anything but an effective survey.

  3. Tread lightly with DIY. Sometimes the cost-savings outweigh professional options but not as commonly as you’d think. Question writing is a science and having a professional market research firm manage the process ensures your results will be unbiased and reliable. The firm may also provide new and insightful perspectives that may not be uncovered in-house with the same data. Two things to think about. Thanks for sharing!

Leave a Reply

Your email address will not be published. Required fields are marked *