UK authorities are forging ahead with an attempt to make the most of data; a quest that includes fighting crime, tackling illness, bringing in tax revenue and improving the nation’s standing on the world stage. But the sheer volume of information they hold, combined with some worrying lapses in privacy, leave a long road to success.
There is immense positive potential from using public, non-sensitive big data, according to Professor Sir Nigel Shadbolt, chairman of non-profit organisation the Open Data Institute. “When you think about collecting and analysing data, such as that on geography, transport, spending and house prices, there can be great economic and social benefits,” he says.
This data can support well-informed research and policy-making. Professor Matthew Woollard, director of the UK Data Archive, an organisation that collates and manages survey and government data, explains: “Recently, extensive data sets have been used by the government to inform policy on alcohol pricing, tax, business practices and property.”
A more thorny area is how the government itself stores and controls data, particularly concerning individuals. This information can include health records, address details or personal spending preferences, where there is great potential for improving services, but also a risk of misuse.
When handled correctly, one of the fields of potential benefit is health. “By making good use of data to monitor diseases and treatment, the government can really improve lives,” says Mark Little, principal analyst at Ovum.
Crime fighting can also be more effective with the right data. During the widespread riots of 2011, police forces examined traffic on social networks to improve incident prediction and response.
Trend analysis of social network traffic also helps predict potential terrorist actions. Counter-terrorism, as well the development of weapons systems, is the main big data priority of Western European governments, with an International Data Corporation (IDC) study showing 61 per cent of senior IT personnel investing in this area.
This is followed by improving tax collection and cutting fraud. In the UK, extensive efforts by HM Revenue & Customs (HMRC) and the Department for Work and Pensions include using big data to predict and track problem individuals.
It’s important that people have the ability to control their data and who accesses it
By monitoring the daily movements of people within towns and even in buildings, data is also informing better design of public services and facilities, and helping manage energy.
Government departments draw on big data to assess how services are consumed. A positive example is the Passport Office, where problematic application times have been vastly improved. The Passport Office also requires less information from citizens renewing their documents, because it draws on data held by other departments.
Similarly, local councils assess usage data to predict new services that will be needed. Importantly, predicting and responding to natural disasters, following recent floods, have become their top data priority.
The UK is ahead of the curve in Europe when it comes to its use of good business intelligence to interpret data, says Massimiliano Claps, research director at IDC, but it could do more to harness its true value.
There remain serious issues over the complexity of data. Many departments are drowning in information, struggling to structure it properly and derive useful analysis.
The much-touted idea, that organisations “can feed data into a system and suddenly find a useful answer to a problem”, is unrealistic, says Sir Nigel Shadbolt, who points to a need for extensive work in data structure and business intelligence.
Good data formatting is an important starting point with reaching useful results and organisations, such as the UK Data Archive and the Open Data Institute, set out clear formats they require for information. After this, the application of strong data analysis and consistent processes can provide results.
Continued budget pressures also present difficulties. While it may be acceptable for HMRC to increase expenditure on analytics, when there is a demonstrable return to the public purse from improved tax collection, it is harder to convince the public of the immediate financial value of schemes with other major benefits, such as better health or fewer road potholes.
Eduardo Ustaran, a privacy lawyer and data expert, says that while the government provides staff with data training that is as effective as in any company, the cost pressures mean the public sector “lacks the basic human resources” to make the most of its efforts.
Budget limitations can lead to departments cutting corners, which in turn creates other serious problems. The most sensitive issues are around the privacy of people’s data and limiting which organisations can access it, as well as the security that keeps criminals at bay.
One of the biggest privacy controversies has been the disastrous launch of the care.data programme, intended to make patients’ health records available outside the NHS. While the official intention is to monitor and improve healthcare and research, real concerns have been raised whether patients understand the changes.
The scheme has been temporarily paused amid heightened concern that the data will become available to health insurance companies, damaging privacy and potentially pushing up premiums. A leaked NHS risk assessment document described an obvious danger that people could be identified by matching their records against other databases.
Another high-profile failure was the identity cards scheme, which was scrapped in 2010 after large numbers of citizens objected to “creeping” surveillance.
The public sector tends to “run before it can walk” with data schemes, Mr Little says, pushing ahead without properly assessing the information risks or discussing with citizens.
“I wouldn’t say the government is cavalier with data, but it is often incompetent,” he adds, noting that people expect clear information on what data is being taken, how it will be used, and what the pros and cons are.
The government needs to foster a much more informed discussion on the topic, says Sir Nigel Shadbolt. “It’s important that people have the ability to control their data and who accesses it. It must be clear exactly what they are agreeing to and what their options are,” he says.
Research shows that when the right safeguards are in place and the benefits are communicated properly, citizens will support positive uses of data, says Professor Woollard. “But it has to be done properly and trust must be established,” he says.
The government has often seen data protection as “an afterthought, applying much less sophistication than it does to strategy planning”, Mr Ustaran notes. Data privacy must be factored in at the start of any project, focusing on using specific information for clear purposes, and providing strong anonymisation and encryption.
Regulation around data is behind the times, according to Mr Claps, who warns that it is tailored to paper and e-mails, when “we are in the era of social media, big data, the internet of things and large surveillance”.
Information collection can do great good. But the public sector must remember that these efforts are not just about big data, but also big trust with citizens, Mr Little concludes. If the government does not establish understanding and engagement with the public, there is a risk of damaging well-intended projects in the future.
