How to ask Why: Stated versus Revealed Preference Research

“Revealed preference techniques provided 1.5 times better predictions of real-word outcomes compared to the best-performing stated preference technique”

Traditional market research aims to understand consumer preferences by asking individuals to simply state them, e.g. “Which phone features are most important to you?”. However, evidence from psychology suggests people struggle with such direct questions in two ways:

1. People aren’t always aware of their preferences or how these drive their decisions
2. People’s preferences aren’t always consistent and can be influenced by context

To address these problems with stated preference techniques, we employ a technique called revealed preference that aims to uncover people’s preferences through observing their actual choices and behaviour. Consumers are shown a hypothetical store and asked which product they would buy. Different groups of consumers are shown stores that differ in terms of price, product range or any other feature. Analysis of the choices made by each group reveals consumers’ underlying preferences. No introspection is required, and contextual effects can be captured if the presentation reflects the real-world environment.

However, this approach involves more effort to design, build and analyse compared to a traditional survey, so an important question is whether it performs better at predicting behaviour. To answer this, we worked with a PhD student from the Behavioural Science Group at Warwick University to compare the two techniques. Key to this study was obtaining actual sales data from a major UK supermarket to validate the predictions made by each method. This data covered 52 weeks of sales in 600 stores.

Figure 1. Revealed preference example. Respondents were presented with a realistic range of products and asked to select the item they would be most likely to purchase. Details were varied between respondents.

First, an online survey of 2,700 UK consumers asked questions on three commonly purchased supermarket products: milk, eggs, and canned tuna. The survey included one revealed preference technique (see Figure 1) and three stated preference techniques. The stated preference techniques involved ranking product attributes (see Table 1), and then combining these into statistical models designed to predict real-world weekly sales.

Table 1. List of preference elicitation methods used

Second, the survey results were built into models used to forecast the real-world sales data in each week and each store. Model performance was assessed by measuring the amount of variability in real-world sales that each model explains, where a perfect model would predict 100% of the variance and a random guess would explain close to 0%.

The revealed preference model outperformed all stated preference models across all the tested products (see Figure 2). On average, the revealed preference model explained 49% of the observed variance in the actual sales data, while the best stated preference model explained 32% of the variance.

Figure 2. Average variance explained across each store-week in the real-world sales data. The models were also compared using SSE and log-likelihood, and again the revealed preference model outperformed the other methods.

The supermarket sales will be driven by many external factors that haven’t been accounted for by any methodology (e.g. how much shelf space was assigned to each item). However, if this information is known, it could be incorporated into a revealed preference design, resulting in an even better model. In contrast, stated preference methods cannot incorporate the impact of this kind of contextual information.

In conclusion, revealed preference offers a versatile method of measuring consumer preferences, and result in much better predictions of real-world outcomes. Revealed preference designs allow us to trial new scenarios such as price cuts or new product launches using the same method and be confident that predicted behaviour will be replicated in the real world.