LIES, DAMNED LIES AND STATISTICS

People often wonder how the government come up with the scary statistics needed to get behind their policy of reducing gases from car exhausts. Below is an exchange of letters between the head of the UK Statistics authority, Ed Humpherson and his counterpart at DEFRA (UK department for the Environment, Food and Rural Affairs) Ken Roy.

Ed Humpherson to Ken Roy 
06 February 2017

Dear Ken,
The Office for Statistics Regulation was recently contacted about Defra statistics on the number of deaths associated with air pollution in the UK.    

The latest estimates were published in the September 2015 report ‘Valuing impacts on air quality: Updates in valuing changes in emissions of Oxides of Nitrogen (NOx) and concentrations of Nitrogen Dioxide (NO2)’. 

Given the complexities of estimating mortality due to NO2 and PM2.5, it is important that the report presents full details of the calculation methodology, uncertainty around the estimates, and strengths and limitations of the figures. Currently, it is unclear how you arrived at the upper and lower ranges for the combined estimate of mortality (44,750-52,500). Adding this information would aid understanding and interpretation of the figures.  

Whilst we recognise that this report is not a formal statistical output, given the importance of the figures, we consider that it would be helpful to enhance compliance with Principle 4 of the Code of Practice for Official Statistics (Sound Methods and Assured Quality).

Ken Roy's reply to Ed Humpherson

Date:   15 March 2017

Dear Ed,

In response to your letter of 6th February re Defra statistics on the implications, in terms of human health, of air pollution.

We recognise and accept the criticism re the completeness of the commentary provided in the Defra report ‘Valuing impacts on air quality: Updates in valuing changes in emissions of Oxides of Nitrogen (NOx) and concentrations of Nitrogen Dioxide (NO2)’.

As you know this is an area of public policy that Defra, along with other partners across government, continue to focus on – and hence we welcome the feedback.  The Committee on Medical Effects of Air Pollution (COMEAP) continue to consider how estimates of health impacts of NO2 can be refined on the basis of a more detailed analysis of the medical literature, and intend to publish updated estimates with a more complete commentary that reflects the uncertainties inherent in quantifying health impacts. 

 I understand that COMEAP is currently planning to publish its advice this summer.  This will enable Defra to update its analysis – and we intend do so as soon as is possible thereafter hence replacing the original report.  

Kind Regards

Ken
Head of Profession for Statistics (Defra group)  Natural England

So what do we learn from this friendly exchange of letters? Answer - nothing at all about how they came up with these statistics. Simply an apology for not providing sufficient details and a vague promise to do better in future. And this is what the government rely on to make very costly policy decisions. No wonder we are all becoming cynics!