The market research industry has always been a “cuckoo business”, tending to pilfer new ways of working from other sectors and make them its own, says Jane Frost CBE, chief executive of UK trade group the Market Research Society (MRS).
It is not surprising to learn then that the industry has changed radically over the last ten years as the rise of big data analytics, new technology solutions and an increasing tendency to in-source research have reshaped the way operators do business.
It’s mostly been good for clients, who have more choice when it comes to seeking help with their business, says Ms Frost. But it has been uncomfortable for some market research providers, who face more pressure to deliver quickly and cheaply in an increasingly competitive market.
Despite this, the market research industry continues to grow strongly, with research from PwC showing the UK sector alone was worth £4.8 billion in 2016. That was up 62 per cent in four years and larger than the country’s public relations and communications sector and its music industry.
The data challenge
The rise of big data analytics has been the main driver of the industry’s expansion, according to PwC, with this sector expanding 350 per cent over the period.
From crunching sales and performance statistics to scraping social media sites, companies are increasingly using available data sets to get a more granular picture of their customers, while making use of low-cost analytics services from the likes of Google.
For an industry whose foundations are based largely on qualitative, as opposed to quantitative, research it has been a huge shift, but Ms Frost says it’s created opportunities too. People are waking up to the limitations of big data and realising they need help in understanding it, which is where traditional market research companies play a role.
In fact PwC found that as the data analytics side of the UK market research has expanded, there has been a simultaneous increase in qualitative research being used to help interrogate findings.
“Increasingly firms are generating data for their own purposes, but a lot of it is not very usable, so they are coming to the market research industry for help,” says Ms Frost. “People have started to talk not about big data, but smart data, which is data collected ethically that will also help you answer a question.”
Cutting out the middle man
There has also been a growing tendency for consumer-focused companies to take their market research in house as they seek to take control of the growing number of sources of information available to them. Take UK media company Sky, which five years ago established “a more robust” internal team, according to Sarah Jousiffe, the firm’s head of qual, closeness and insight engagement.
Sky now employs an array of quantitative and qualitative researchers in house while outsourcing about 40 per cent of research work to a roster of third-party agencies. “Before the majority would have been done externally,” Ms Jousiffe says.
Consumer-facing companies also have a plethora of new technology tools available to them to help, from programmatic sampling services that let you find samples of consumers online, to survey scripting programs such as Survey Monkey.
Jenn Vogel, vice president of marketing at Voxpopme, a video insights platform, says the company does work with some market research agencies, but very often companies like hers cut out the middleman by letting firms go direct to consumers.
Through Voxpopme’s platform, consumers can upload videos of themselves giving feedback on a product or service via their smartphones. The audio is automatically transcribed and assessed for sentiment. “We are replacing the old focus groups which can be time consuming to arrange,” she says.
Just six years old, Voxpopme already has 60 staff and boasts clients such as food giant Mondelez and detergent maker Clorox. Ms Vogel says the firm grew by 150 per cent in 2018.
In response, traditional market research companies have been racing to integrate new technologies and data analytics into their offerings. As happened in the advertising industry a few years ago, such operators also face more pressure to explain their pricing.
“Customers want faster, cheaper, more innovative research,” she says, but warns many do not realise what actually goes into achieving this. “If you want to hear from ten white-van drivers with one leg who vote Conservative, you don’t just go out and talk to ten people, you need to speak to 100 people and whittle it down. We need to do a better job of explaining those costs.”
Technology is creating new research opportunities
In the next five years, Ms Vogel expects to see the continued transfer of new technology into market research, which at the bleeding edge will mean greater use of developments in neuroscience and behavioral economics to help companies gain an advantage. She also expects social media analytics to become more sophisticated, noting that at present social insight is fairly limited in what it can tell you.
“People using social media are self-selecting. So the government would never use social media insight to get a picture of what the public are thinking,” she says.
Ms Jousiffe expects data to play an ever bigger role in the industry, pointing out that her firm is currently “drowning in it”. She also thinks there is a job to do in creating a bridge between those managing qualitative and quantitative research within companies as they are such different disciplines.
“The people who work in research and in data analytics are very different,” she says, “and joining this up has been our biggest challenge over the last five years.”
She also recommends that more traditional market research companies should work in partnership more often with the new tech firms nibbling at their market share as it could create new opportunities.
Ms Frost at MRS agrees, urging market incumbents to be “flexible, and to identify and adapt to new ways of doing things”.
“Look at your clients in the same customer-centric way that they do with their customers,” she concludes.
The data challenge
Cutting out the middle man
