Here’s one that Dr Ali wrote for our friends at Sumo Insight
“Any research that is asking people to consciously reconstruct their lives mentally, will always result in an inaccurate answer”
The influence of behavioural economics has come on at a pace in the last few years. It has taught us that our behaviour is part of two mental processes. One (System 2) that as slow and considered, where we consciously decide on a course of action. The other (System 1) is fast and intuitive where we act through what ‘feels’ right, be it intuitively or emotionally. This observation has brought into question many of the research techniques we have known and loved, specifically because although we may be aware of the outcome of System 1 we are not always aware of why we feel the way we do. As such people are not very good at reporting what underlies their feelings.
However a bigger problem for research lies in another aspect of the psychology of the two-process mind. For many years psychologists have defined various mental processes as being constructed of two mental processes. The largest literature within this is the field of memory. As far back as the beginning of the last century people have noted that memory comes to us in two distinct ways. We have one memory where we literally bring to mind and mentally re-live a past event, and another where knowledge just comes to us automatically without any effort. These two kinds of memory have been described in various ways however the most useful distinction for researchers is to know them as Episodic and Semantic memory.
Episodic memory literally means ‘re-living an episode of our lives’ where we think about an event, make an effort to bring it to mind and then mentally re-live the experiences. Semantic memory is simply ‘knowing’ something with no effort. For example the cup of tea on my desk. I know what a cup is without having to use effort to search through my mind to recall what a cup is and what to do with it.
We have to think about how human memory works because all traditional market research is mediated by the memory of the respondent, whether they are sitting in a group or at a computer. So which of these two memory processes are going to the most influential in traditional research? The answer is all too often Episodic memory, as the act of doing research is getting respondents to introspect about their lives and inevitably think about events in their lives when they have encountered brands or adverts etc., and then report back on what that have been able to consciously reconstruct in their minds about it.
This is a big problem for conventional research, as we know much about episodic memory, and far from it being a perfect record of the events in our lives it is often full of gaps, particularly about how we felt about things. Here I am not talking about major emotional experiences but the subtle ones that drive our behaviour every day. It also records our lives in what are known as Schemas, rough aggregations of information across many experiences rather than single discrete events. For example a number of studies have shown that taking people into an office and asking them to describe it afterwards often leads to people describing a generic office (with photocopiers computers etc), rather than the specifics of that particular office. This is in fact a very clever evolutionary strategy that allows us to cope with the vast amounts of information that we come into contact with every day and store just enough to allow us to interact with our world efficiently.
However this very clever way in which our mind has evolved is a problem for research as any research that is asking people to consciously reconstruct their lives mentally, will always result in an inaccurate answer, not because the person is trying to deliberately mislead but simply because our brains are not designed to record what happened in our lives with absolute accuracy.
So how does research respond to this observation? Well obviously in some contexts research can work so this cannot always be the case. However there is an argument that we should start to think about when and where these problems may occur. Certainly in terms of voting, people’s reports on the way they are going to vote are often pretty accurate, though the polls can get it wrong such as with the vote for Scottish independence last year. This is most likely caused by the cognitive miser telling people to stick with what they knew (but that’s another blog story). However other research, particularly around media communications that is designed to work by influencing our ‘feellings’ is likely to be inaccurate simply because Episodic memory is not very good at recording how and why we feel.
Another obvious solution is the use of new mobile technology. Gathering data real-time is an advantage as not only does it cut out issues with memory, many studies have shown that people are far better at reporting how they feel ‘in the moment’ than trying to recall what they felt in the past. Using mobile tech is far more likely to get an accurate response. Take for example some research that I was involved with a few years ago. My PhD supervisor was tasked with finding out why students chose my university (Sussex) as a place to study. This is an important life decision so people are going to seriously think about this right? Asking students who came to study proved fruitless so we conducted a study of all of the grades, places they came from A levels studied and any information gained in the application process yet nothing gave a clear answer, all we knew was that some open days were more successful than others. Again we correlated all that happened on those days, where they went, who talked to them etc. and still nothing came up. After a few years of this we had a brain wave as to the possible factor and the following year we tracked the mystery factor and were proved right. And what was it? Simply if the student came on a day when it was sunny, they were far more likely to choose Sussex. Had we had some mobile tech that could have got feedback about how they were enjoying their open-day (and the other open-days at other university) I’m sure our accuracy at predicting who would come would have increase dramatically.