Margins of Error: Increase understanding but also challenge the public’s perception of statistics


  • Author: Statistics Views
  • Date: 13 Jun 2013
  • Copyright: Image appears courtesy of iStock Photo

On the evening of Tuesday 14th May, the Royal Statistical Society and Ipsos Mori hosted a packed debate at Kings College London to explore the public understanding of statistics in this era of big data we live in entitled ‘Margins of Error’. This was the first in a series of events organised by the Society, Ipsos Mori and Kings College during the International Year of Statistics.

thumbnail image: Margins of Error: Increase understanding but also challenge the public’s perception of statistics

This debate formed a panel chaired by the Royal Statistical Society’s President John Pullinger, which included Bobby Duffy, Managing Director of Ipsos Mori - Social Research Institute, Andrew Dilnot CBE, Chair of the UK Statistics Authority and Denise Lievesley, Head of School of Social Science and Public Policy at King’s College, London.

The debate raised issues on how society will deal with and what it can gain from big data; whom do we trust, how the media currently portrays statistics and the challenges facing statisticians in dealing with such vast quantities of information. John Pullinger opened by describing that the event had been “triggered by a sense that in this era of Big Data, statistics matters more than ever. Decisions based on data are being made well or badly and it matters. It matters that we stand up and resist when they are being made badly. A significant problem in understanding why there is this risk in decisions being made is people’s capacity in understanding numbers.” He described the getstats campaign and its aim for improving the statistical literacy of the nation, especially amongst politicians, journalists and employers.

Each panellist gave a presentation on their thoughts and ideas on Big Data. More information about the debate and a link to the slides presented is available here.

Bobby Duffy – Focus on understanding and value, but firstly on trust…

Mr Duffy ran through a survey that was recently sent out to the public which focussed on the trust in statistics. According to the results, statisticians came fourth in a list of most trusted after scientists, academics and accountants but more trusted than economists, actuaries, pollsters and politicians, who came last.

In the last few years, there has been a significant increase in the public trusting scientists more than the clergy and another graph demonstrated the trust in civil servants vs. politicians with results dating back to 1983. This is one of the most sustained and remarkable increases, with the trust in civil servants doubling whilst government ministers, politicians and journalists generally had to ‘fight it out amongst themselves’ as to whom were the next trusted with very similar results between them.

Andrew Dilnot had to explain to UK Prime Minister David Cameron the difference between ‘debt’ and ‘deficit’. 52% of the public correctly understood the difference.

Overall, government were less trusted than supermarkets and online retailers with our data, scoring only 2% with the answer ‘A great deal’ under ‘To what extent, if at all do you trust the government to use the data they collect about you appropriately?’ 41% ticked ‘Not very much’ compared to online retailers who scored marginally better at 5% and 40% respectively. In an era where data could be shared in order to improve services, these are not encouraging results.

He touched upon the recent news where Andrew Dilnot had to explain to UK Prime Minister David Cameron the difference between ‘debt’ and ‘deficit’. 52% of the public correctly understood the difference. 40% of the public believe that planned cuts have already been made.

Duffy further emphasised John Pullinger’s statement that a basic understanding of numbers is key to statistical literacy. 89% correctly answered ‘What is 50 expressed as a percentage of 200’ (answer -25%) but 30% correctly answered, ‘If you spin a coin twice, what is the probability of getting two heads?’ with 58% incorrectly answering ‘50%’ compared to the 26% who correctly answered ‘25%’.

There are also known biases in how we consider statistics. We tend to focus on negative information from a personal perspective which is evident when the public were split into two groups and asked the questions with one of the different answers below ‘Imagine you have a life-threatening illness and your doctor has told you that you need an operation to treat it. How likely, if at all, are you to have this operation if your doctor tells you that...

- 90% of people who have had the operation are alive for at least five years following the operation
- 10% of people given the operation die within 5 years of the operation’

Presented with the same chance, the results were different. 56% given the first answer clicked on ‘Very likely’ that they would have the operation, compared to 39% given the second answer.

As to whether people consider evidence, or think their leaders do, when asked whether politicians will take decisions partly based on what do they think is right and partly on evidence of what works, an overwhelming 52% favoured the former option.
We do not appear to be embarrassed by our lack of understanding of numbers with the question asked which skill would make one most proud as a parent, an overwhelming 55% voted for reading and writing compared to 13% with numbers.

Duffy’s conclusion was that there are many challenges instead and we still have a long way to go.

Andrew Dilnot – Understanding and Trust in Statistics

Dilnot emphasised how the whole statistical community, all from academics to journalists, from the ONS to the private sector, can help people trust and understand statistics.

Firstly, telling the big story is very important and it is essential for the community not to get tied down by details. He started with a graph showing GDP from 1948-2012, which has increased by a multiple of five over this time. This is underestimated, even by economists in the private sector.

GDP from 1948-2012...has increased by a multiple of five over this time. This is underestimated, even by economists in the private sector.

There has been much talk about double dip recession lately when in actual fact, GDP grew for around 20 years, then it fell, then it rose slightly and for the last two years, it has been basically flat. The big story is that it increased for an unprecedented long time and this is an example of a big story. It is also very important for statisticians to be clear about uncertainty. When a preliminary estimate is offered, it is done within 25 days after the end of the quarter and when some people say that is shocking that estimates change, it is an “astonishingly difficult task and remarkable that it is done” in that length of time. Dilnot stressed that statisticians should not be embarrassed that sometimes estimates will change.

Secondly, trust can be boosted by surprising people and challenging their preconceptions. For instance, in this country there is the preconception that teenage pregnancy is on the rise and we are constantly confronted with newspaper headlines of this rise, teenage mothers relying on benefits, etc. We do not however hear of the actual numbers. Dilnot presented to the audience a graph showing the conceptions per 1000 women aged 15-17 from a recent ONS survey. The graph in fact showed a dramatic decline in teenage pregnancy since 2006.

Thirdly, it is essential not to exaggerate. Most winters in the UK, the headlines of some newspapers predict a dreadful virus that will affect millions. Dilnot referred to a 2008 Telegraph article where norovirus cases were claimed to have reached 3 million. What is the ONS actually doing to check how many do have norovirus? Dilnot joked that staff are peeping through bathroom windows to see who is vomiting, or checking the concentration of vomit at local sewers. Joking aside, the public should be taught to look at the number of 3 million and question how this number was reached. According to actual data from Figure Laboratory reports, the number of cases peaked at 250. How did the number of 3 million come about? The numbers were added up by 10 weeks but even if this is added, you only come up with around 1500. Counting numbers is hard and the more open we are, the more likely people will trust what we do.

Lastly, Dilnot explained how trust can be built by explaining. He used an example of crime statistics which confuses people with both police recorded reports and CSEW available. The ONS has recently explained the output for both which is helping to make a difference, alluding to a recent article published in The Guardian on crime statistics, which was an intelligent and calm response to the coherence provided by the ONS.

Dilnot concluded, “We want people to think of statistics as lights, not crutches. The more we can allow people to see that statistics can enlighten their world, the more chance I think there is that people will trust us. So, be open, tell a story, be surprising, don’t exaggerate and try always to help understand that there is uncertainty in the marvellous beautiful statistics that we love so much.”

Denise Lievesley – Public understanding of statistics in an era of big data

Professor Lievesley wanted to address the subject from the perspective of a statistician. She raised five tensions in terms of the challenges facing statisticians – humility vs. confidence, relevance vs. autonomy, trust vs. scepticism, measurement vs. quality and pragmatism vs. purism.

For example, with humility, it is about statisticians being aware of their limitations. She touched upon the ISI declaration on professional ethics in 1985 and emphasised that statisticians should provide high quality information, be confident in their research and be prepared to account for them. Achieving this balance is difficult.

Risks need to be taken and sometimes statisticians won’t always get it right.

Professor Lievesley referred to Dilnot’s talk that communication skills needed improvement. Communication is not what is delivered but what is received, using an example from when she worked at UNESCO, the cost of putting children into school seemed an insurmountable number until you translated into this - ‘The cost of putting all children into school is less than is spent on ice cream in Europe each year’. Trying to tell a story with data is very important. 

She referred to former UK Cabinet Secretary, Gus O’Donnell maybe inadvertently said he wanted the ONS to be boring and produce just the facts, and leave the interpretation up to politicians and government press officers. The RSS had an emergency society meeting and responded the same today, saying that it is clearly the role of statisticians to interpret the data and the UK Statistics Authority’s Code of Practice was quoted back.

Professor Lievesley also considered whether statisticians are too cautious due to an underlying concern that they may be accused of having political motives and hence they dare not make reports useful for public debate. This will be a recurring issue and needs to be addressed inside and outside of the statistical community. Risks need to be taken and sometimes statisticians won’t always get it right. She concluded that in order to develop statisticians for the future, we need to foster adaptability, build research and innovation skills, not to mould statisticians in our own image nor to create homogeneous communities, and create a cadre of people who challenge preconceptions. ‘Education is about opening minds, not closing them.’

John Pullinger summed up the debate saying that Duffy had shown the evidence that there is clearly a job for statisticians to do, Dilnot has shown that beautiful data has the answers within it and it is up to statisticians to bring it out, and Lievesley had expressed that statisticians need to get over themselves – the messy reality is that there is no purity here or pragmatism there but instead a balance and it is up to statisticians to use that balance and not be frightened of it.

Related Topics

Related Publications

Related Content

Site Footer


This website is provided by John Wiley & Sons Limited, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ (Company No: 00641132, VAT No: 376766987)

Published features on are checked for statistical accuracy by a panel from the European Network for Business and Industrial Statistics (ENBIS)   to whom Wiley and express their gratitude. This panel are: Ron Kenett, David Steinberg, Shirley Coleman, Irena Ograjenšek, Fabrizio Ruggeri, Rainer Göb, Philippe Castagliola, Xavier Tort-Martorell, Bart De Ketelaere, Antonio Pievatolo, Martina Vandebroek, Lance Mitchell, Gilbert Saporta, Helmut Waldl and Stelios Psarakis.