Anthony Hilton: Statistics are lifeblood of democracy, and need to be protected

But the job is bigger than this because ultimately the quality, trustworthiness and value of our national statistics is the responsibility of the regulatory body.

Statistics are an essential public asset, the lifeblood of democratic debate, and a driver of decisions — not just by government but by businesses, the media, individuals and community groups. So their preparation matters and this is why, last week, the department announced a public consultation on the relaunch and upgrading of the code of practice for official statistics.

The document covers how statistical data should be collected, how its objectivity and reliability is maintained and does not fall victim to vested interests, how the data can be used and how those doing the work should be protected from external influences. It is a guide to how people should work and a shield to protect them while they do.

At its core it is about defending the integrity of those who do the work, and maintaining the integrity of what they do, being honest about the limitations of statistics and not pretending they are what they are not.

The fundamental purpose is to produce robust and reliable data which is also consistent and relevant to society. It is about creating an organisation which acts transparently and in the public interest while respecting the privacy of all the individuals and businesses. As such its guidance has applications well beyond Whitehall and could usefully be applied to private-sector data collection too.

There is also a wider issue which again affects us all whether we understand it or not. The old way was where statisticians sourced and collected the data on which they worked — they sent out surveys and analysed the responses.

What has changed in recent times, however, is that the quality and speed of data collection by the private sector has grown out of all recognition.

Facebook, Google and Amazon obviously have huge amounts of information on their users and customers, but so too does Tesco via its store cards and logistics firms which can track parcels and taxi companies which know who is going where. What is called big data can deliver information about patterns which have never been obvious before.

The mining of this information to give insights into behaviours which can be turned to commercial advantage is now a major pre-occupation of business and one of the things which is going to define success and failure in future.

But this also significantly widens the pool of data potentially open to use by the Office for National Statistics so it is vital that equivalent standards of quality and integrity apply if trust is to be maintained.

So, statistics are as vital to society as all those other bits of infrastructure — water, electricity, broadband, sewers — that we take for granted and never think about until they don’t work. What goes wrong with statistics is that people lose trust in them.

What the regulator’s office is trying to do is show that truth does still matter. We have had the fake news and the rubbishing of experts and can already see that society does not function well in such circumstances. Humpherson with his code is part of the fightback.