Can’t Tell, Won’t Tell: The Pitfalls of Traditional Consumer Research

Established methods for forecasting customer behaviour rely on the truthfulness of people’s self-report. Yet decades of research have demonstrated how people are extraordinarily poor at such introspection. In this Buzz we explore two main causes and how market researchers can circumvent them.

Conventional research methods, like focus groups and quantitative surveys, depend entirely on people accurately introspecting their motives and beliefs. “X% of people said Y was extremely important in deciding Z” is their stock-in-trade, where Y is often “price” or “convenience” and Z is “where they shop”, “which product they buy” and so on.

Setting aside the difficulty of translating such statements into a commercial strategy, there remains the problem that they are also inaccurate and misleading. First, people tend to post-rationalise their behaviour and generally don’t know why they do what they do. Second, even if they have an inkling of what’s causing their behaviour, they are inclined to improve their responses. They either can’t tell or won’t tell.

Can’t Tell

People’s inability to comprehend their true motives whilst being able to fluently fabricate a narrative to explain that behaviour is well documented. Figure 1 details an excellent example. People are shown two faces and indicate which they find more attractive. They are then asked to explain their choice but, by sleight of hand, are given the wrong picture. Only 13% of people detect the switch and the other 87% go on to confidently elaborate on why they picked the person that they didn’t pick.

Figure 1: People’s Capacity to Post-Rationalise Choice¹

Won’t Tell

Likewise, people often revise their responses to appear self-consistent, socially acceptable and so on. For example, we recently asked over 2,500 people to state their preference for ethically sourced goods. Their actual purchase behaviour was then tracked. Figure 2 shows how their self-report bears absolutely no relationship to their actual shopping. Those who self-report the lowest preference for ethical goods were just as likely to shop ethically as those with the highest.

Figure 2: The Attraction of Socially Desirable Responses

In summary, people have little insight into their own decision-making. Our preferences are unstable and influenced by a blizzard of, often extraneous, factors. It’s not surprising that we can’t identify what causes our behaviour and, given the desire for agency over our own decisions, resort to post-rationalisation and revisionism. Particularly on socially loaded topics where there’s pressure to self-censor, provide strategic responses, polish our credentials and so forth.

It’s not that focus groups or quantitative surveys are without merit. The former are great for understanding the conditions under which people make decisions and the latter for sizing addressable markets, tracking advertising awareness and so on. But for understanding and forecasting people’s behaviour they are fatally flawed.

The only way to circumvent these problems is to use randomised controlled trials, like A/B tests or our Behaviourlab approach, to measure how different conditions change people’s actual behaviours. Whilst these methods are more expensive, they are higher accuracy and more actionable. Paying less to get the wrong advice isn’t saving money.

1. Johansson, P., Hall, L., Sikström, S., & Olsson, A. (2005). Failure to Detect Mismatches Between Intention and Outcome in a Simple Decision Task. Science, 310, 116-119.