How did data get so big? Less than a decade ago, analysts at IT market research firm IDC estimated the size of the “digital universe” at around 130 exabytes. By 2010, it had grown to 1,227 exabytes. By next year, they forecast, it will reach 7,910 exabytes. To put that into some perspective, every exabyte equals one billion gigabytes.
“Every day, we create 2.5 quintillion bytes of data – by some estimates that’s one new Google every four days and the rate is only increasing,” according to venture capitalist Peter Levine of Silicon Valley firm Andreessen Horowitz. More than 90 per cent of the data in the world today, he notes, was created in the last two years alone.
The numbers are dazzling, confusing, alarming. And with every year that passes, it becomes harder for business leaders to separate the “signals” from the “noise”. They’re collecting as much data as they can, from customer transactions, web-browsing data trails, social network posts and, increasingly, machine-embedded sensors. But many are struggling to make sense of the deluge, let alone derive real business value from it.
“For all the big data talk, most companies are still at an early stage of maturity. They simply haven’t progressed all that far yet,” says David Meer, a partner at Strategy&, the consultancy group formerly known as Booz & Co. “There’s a very real sense that companies are still trying to figure out what the data economy means for them. They’ve made big investments; they’ve built out infrastructures capable of processing big data – but, in many cases, they haven’t yet outlined solid use-cases for getting the most value out of them.”
If they are to compete effectively in the data economy, they simply have to grow up, says colleague James Walker, a partner in Strategy&’s European operations. “It’s the new business imperative,” he says, because big data has the potential to improve or transform existing business operations. It will reshape entire economic sectors. It will pave the way for disruptive, entrepreneurial newcomers to emerge. “But it could also be the death of incumbents who fail to adapt and evolve,” Mr Walker warns.
The trouble is that, when it comes to making business decisions, too many bosses continue to rely on intuition and experience alone, says Steve Gold, vice president in IBM’s big data Watson Business Group. “They need to learn to be guided more by the empirical evidence hidden in data. They need to embrace it. If I’m a patient, that’s the kind of doctor I want to see. If I’m an investor, that’s the kind of financial planning advice I want,” he says.
This, he says, is the thinking behind IBM’s decision to open up its Watson cognitive computing capabilities to customers as a cloud-based, big data service and to invest $1 billion in the Watson Business Group.
In 2011, Watson hit the headlines when it beat champions of the TV game show Jeopardy at answering general knowledge questions. Now IBM’s betting that Watson’s cognitive capabilities, which enable the system to “learn” about data and understand its context, will improve decision-making in a wide range of industry sectors, from healthcare to financial services.
DATA INSIGHTS
But cultural changes will be needed, too, as a report from Strategy& makes clear. In
Big Data Maturity: An Action Plan for Policymakers and Executives, Mr Meer and three co-authors implore business leaders to “remould their decision-making culture, so that senior executives make more judgments based on clear data insights, rather than on intuition”.
The report says: “They [business leaders] must build the necessary internal capabilities, deploying the technical and human resources to interpret data in an astute manner. Moreover, because they rely on governments to provide the requisite environment, they must ask policymakers to create the regulatory framework and information and communications technology infrastructure to remove external obstacles.”
In fact, technology should perhaps be a secondary issue for most companies. First and foremost must be a focus on “carefully formulating the business questions that enable the swift and accurate identification of those nuggets of data that [executives] believe can improve their organisation’s performance or allow them to gain access to new revenue pools,” says Mr Meer.
David Keene, Google Enterprise regional marketing manager, north and central Europe, adds: “Big data is meaningless without adding the people dimension. The most powerful insights grow quickly when small, agile groups connect unstructured data analytics with organisational memory. They ask the right questions, correlate with the best operational data and collaborate in real-time. People are the real secret source for big data.”
Bosses need to learn to be guided more by the empirical evidence hidden in data
Early examples of success are already emerging. Take, for example, UK broadcaster Channel 4 where they are using a Hadoop infrastructure, based in Amazon Web Service’s cloud, to store and analyse more than 170 terabytes of data relating to viewer interactions with its on-demand and catch-up services through a wide range of computing devices.
“In part, this is about enabling us to better understand our viewers, their preferences and their behaviours. That, in turn, allows us connect more closely with them, to schedule more relevant content and to commission new content,” says Sanjeevan Bala, head of data planning and analytics at Channel 4.
“At the same time, by knowing more about the socio-economic class, age and gender of viewers, we’re able to help our advertisers buy up media spots, so that their messages reach the right viewers at the right time of day or night,” he says.
Another example is travel company National Express. It is using a Microsoft-based data warehouse and business intelligence tools from Qlikview to understand more about the 18 million journeys its passengers make to over 1,000 locations nationwide each year.
CUSTOMER FOCUS
“As a business, we have a very strong focus on customers and their travel experience,” says Frank Kozurek, the company’s business intelligence manager. This new analytics environment, he explains, is helping the company to build a better picture of what that experience looks like. “We can see more clearly what are customers say about it, what we do well and what we need to improve,” he says, adding that, over time, this information will help National Express make important, revenue-generating decisions about ticket prices, frequency of services and journey routes.
At waste management services company Biffa, management information manager Laura Lewis and her team have used tools from Microstrategy to build a “depot dashboard” for bosses at each of the company’s 65 units nationwide. This shows them how their own depot is performing and compares it to the performance of others, says Ms Lewis.
“If they’re not hitting their sales targets, for example, they can see where the problems lie in terms of which customers, which routes, which areas of town and so on,” she says. “They can see how many wasted journeys their drivers clock up or their individual pick-up rates.” This has fostered a healthy sense of competition at Biffa between drivers and depots, “driving behaviour that, for our customers, improves their experience”, says Ms Lewis. The end-goal here, then, is greater customer loyalty and higher revenues.
As these projects at Channel 4, National Express and Biffa show, in a data economy, good business decisions are now data driven.
And so are good companies, according to a Harvard Business Review report. Researchers Andrew McAfee and Erik Brynjolfsson explored the impact of big data on the performance of 330 publicly traded companies in the United States. The more companies characterised themselves as “data driven”, they found the better these companies performed on objective measures of financial and operational results.
“In particular, companies in the top third of their industry in the use of data-driven decision-making were, on average, 5 per cent more productive and 6 per cent more profitable than their competitors,” the researchers conclude.
The connection between data-driven decision-making and success in the data economy could hardly be clearer.
