The following is an edited excerpt from the book, Applied MaxDiff, by Keith Chrzan and Bryan Orme.
MaxDiff (short for maximum difference scaling and the name marketers have given to a method more commonly known in academia as Best-Worst Scaling, or BWS) has become the measurement equivalent of the Swiss army knife. A tremendously useful method, it has an expanding base of users who continue to find new and innovative applications for it.
Conceived as a multiple-choice version of Thurstone’s (1927) method of paired comparisons, the basic case of MaxDiff quantifies the relative value of each of the several items on a list. For a study of 20 ice cream flavors, for example, we might ask 12 questions that look like this:
MaxDiff surveys wring a great deal of information out of just two mouse clicks per question. For example, in the above question the respondent indicates that they like Vanilla the most and Raspberry the least. From these two mouse clicks we learned that they like:
- Vanilla more than Raspberry
- Vanilla more than Cookie dough
- Vanilla more than Coconut
- Vanilla more than Peach
- Cookie dough more than Raspberry
- Coconut more than Raspberry
- Peach more than Raspberry
Thus, the MaxDiff question identifies the respondent’s preference in seven of the 10 possible pairs one can form from the five flavors. With a similar amount of information coming from 11 more such questions, we have a wealth of information about their relative preferences among ice cream flavors. We learn their preference with less effort than having them answer a large number of paired comparison questions or than forcing them to rank order the full list of 20 flavors.
MaxDiff has a number of other advantages over ordinary rating scales.
Because it constrains the way respondents can answer the question, MaxDiff removes the scale-use biases commonly seen with rating scales, particularly for cross-cultural research studies (i.e., respondents use rating scales in different ways—some use the top end of the scale and others the lower end, while some respondents use all the scale points and others pile up their answers in a narrow portion of the scale).
Also, MaxDiff forces respondents to make tradeoffs among the items, so it tends to discriminate among items more powerfully than ordinary rating scales do and it more readily identifies between-group differences on the items, as Cohen (2003) found in a much regarded paper comparing MaxDiff to rating scales. Comparing MaxDiff to rating scales and four other methods for measuring attribute importance, Chrzan and Golovashkina (2006) confirmed these findings and added that MaxDiff had greater predictive validity than any of the other methods they tested.
Researchers can use MaxDiff to generate utility information for pretty much any list of items; this gives it great flexibility in the hands of survey researchers. Often, commercial researchers use MaxDiff to measure the relative appeal of a list of items like new product concepts, flavors, or menu items so that marketers can prioritize their development efforts. Applied to advertising, MaxDiff can help researchers evaluate a list of advertisements or advertising claims, or even to prioritize a list of message elements that can then be combined into advertising executions. Healthcare researchers can employ MaxDiff to understand how patients value different health states or different aspects of treatment regimens. In quality improvement research, MaxDiff can inform decisions about which aspects of a product or service customers most want to see improved.
Beyond appeal, researchers frequently use MaxDiff to measure attribute importance, a common need in marketing research staples like brand image and customer satisfaction research. Moreover, MaxDiff serves as a general measurement technique, a method to consider using any time one has a list of items that need to be compared or evaluated. In their initial publication on MaxDiff, Finn and Louviere (1992) used MaxDiff in a public policy study of food safety concerns. Lee, Soutar and Louviere (2007) found MaxDiff to provide a more valid measure of the List of Values (LOV) scale than the original rating scale formulation while Chrzan (2014) used MaxDiff to replace rating scales in measuring the Five Factor Model of personality. In another example, which brings MaxDiff back to one of Thurstone’s original case studies for the method of paired comparisons, Sá Lucas (2004) used MaxDiff to measure the relative severity of each of 25 crimes.
In time Louviere and coauthors distinguished three variants of best worst scaling:
- The first of these, now known as “BWS case 1” or “the object case,” is the basic MaxDiff shown above: we ask subsets of a longer list of items in a series of questions to put the relative appeal of each item on a numerical scale.
- BWS Case 2 scaling, or “the profile case,” shows respondents a question that looks just like a MaxDiff question, except that each item is the level of a different attribute, as in a conjoint analysis profile, like this… Brand: PaperMate, Price: $0.99, Color: Black, Medium: Gel.
- The third variety of BWS, called Case 3 or “the multi-profile case,” resembles a choice-based conjoint (CBC) question formatted to allow respondents to choose both their most and least favorite profiles from a choice set of three or more profiles. This fits more naturally with conjoint analysis topics than with MaxDiff topics.
Those familiar with multi-attribute models (like conjoint experiments) know that they are the equivalent of grumpy old dogs: analysts can get bitten if they aren’t careful enough in the design, analysis and modeling stages of their choice-based conjoint experiments. MaxDiff is more like a friendly puppy: with a very modest amount of effort and very little chance of getting bitten, you can get your MaxDiff model to roll over and let you tickle its belly!
Keith Chrzan has over 35 years of marketing science experience in a variety of client, supplier and consulting roles. Keith is a regular presenter at technical conferences like the AMA’s Advanced Research Techniques Forum, the INFORMS Marketing Science Conference and the Sawtooth Software Conference, where he has presented at every event since 1990.
Bryan Orme is the president of Sawtooth Software, Inc. Sawtooth Software is a leading provider of advanced tools for interviewing, conjoint analysis, MaxDiff scaling, cluster/ensemble analysis, perceptual mapping, and hierarchical Bayes (HB) estimation.
Sawtooth Software is a sponsor of the Principles Express course, Advanced Analytic Techniques.