Data visualization, social stats and how stats students love to party

Professor Danny Dorling is currently based at the Department of Geography at the University of Sheffield. He has recently written a book for Wiley, The Visualization of Spatial Social Structure, based on his research for his PhD. Statistics Views chats to Professor Dorling about writing the book and the influence of statistics on his career.

 

1. Your current research is about how far understanding the patterns to people’s lives can be enhanced using statistics about the population. Part of this research involves developing new techniques to analyse and popularise quantitative information about Human Geography. Your background is a BA and PhD in Geography, Mathematics and Statistics at Newcastle-Upon-Tyne. When did you first become aware of statistics as a discipline and what led you to choose pursuing the area as a career?

At school I found mathematics and statistics easy but not that enjoyable, however I found that I could do them. There were only two universities in the country at the time (Queen Mary and Newcastle) that ran a combined course of mathematics, statistics and geography and I wished to do something as well as mathematics and statistics, in case I found the maths and stats boring. But in fact, it was the other way round which was a bit of a shock! When I got to university, I found the maths and stats students the most fun to be with. Part of the reason was to get into university at that time to do stats you needed a grade C at A level, whereas to do geography, you needed an A. The students who did maths and stats tended to party a bit more, so perhaps as they had partied a lot earlier, they did not achieve so many A or B’s! But that’s how I ended up doing all three combined subjects.

2. Congratulations on the publication of The Visualization of Spatial Social Structure, which was also the subject of your PhD in 1991. What is it in particular about data visualization that appeals to you?

I’ve always liked graphs and maps. They help you think in different ways when you can see data and back then in the late 80s and early 90s, computer graphics was only just beginning so suddenly you could create your data on the computer rather than drawing by hand, so it was a very exciting time to be involved in that.

3. What were your main objectives during the writing process? Were there areas that you found more challenging and if so, why?

The main thing was almost entirely rewriting the text from my PhD for the book. Although I was good at computer programming in my early 20s, I was absolutely useless at writing. It’s been interesting to look back at my PhD thesis 20 years later and realising what a different person I was back then and finding that the text needed to be rewritten to make sense to people nowadays. In hindsight it looks as if I was very good at maths and stats when I was younger but less so at grammar and spelling, but all these years later, I’ve had a lot more time to practice! I am amongst a small group of people who find numbers easy and spelling hard. But I made the career decision to write books!

4. As a university professor, what do you think the future of teaching statistics will be? What do you think will be the upcoming challenges in engaging students?

I think it will be necessary to make it more interesting than it has been in the past. People’s attention spans have reduced in general. If I look at the books my children read now, they are different from what I read when I was their age, and what I read is different from what my parents read. It is impossible for me to take a book my parents read and read it to my children. Our attention spans have become much shorter because of the environment we now live in. People are much more visual now because they have been exposed to much more graphics, television, films and so on. So they need more images and this is not the fault of students, which is important to note as it is best that your students these days have shorter attention spans. You then have to be a better lecturer and provide them with visuals as that is what they are used to.

5. You also write regularly for The Guardian. How do you feel now about the pressure to publish? Has your view changed over the years?

I was very excited when I was young about publishing in academic journals as that was a great challenge. I think publishing is what academics need to do otherwise academia can be more of a hobby but you can change what you publish in over time. It may sound odd but it’s harder to get published in The Guardian than it is in a lot of academic journals. There are only so many pages in an edition of The Guardian and of course, far more pages in all the academic journals that might be published in your field.

I like changing what I try to publish in. I still publish in both journals and newspapers but I like writing books more as, in a way, you have more freedom to say what you think, whereas in journals, for good reasons, you have to write in a strict way. However, if I make a wonderful discovery, I will try to put it into a journal paper first!

There are lots of different routes now – more opportunities with more journals and books being published, so you need to be selective about where you publish. It was so hard to publish in the past, especially, for example, when you wanted to publish a colour diagram. Twenty years ago, printing one colour diagram could cost thousands of pounds, whereas now it certainly costs less now, a journal can have a colour budget, and of course, it can be in colour online for free. One was more controlled by the environment then, so I don’t think it is so much about the pressure to publish now. It is more about the danger of over publishing.

All the graphics in my book were first produced in 1989 but I could never see them then at the quality they are printed in now because the resolution of computer monitors was smaller and the printers not as advanced as they are now. I had an idea of what they could be like but I could never see the actual detail that I had produced. Of course, advances are still being made and in many ways the printed page is still the highest resolution you can get, and there are problems with colour on some devices. Kindle, for example, has not yet gone to colour but it will soon – it’s a consistently changing environment.

6. In relation to the above question, do you think that statistics undergraduates and postgraduates starting out today are under more pressure to publish and to obtain grants than when you were a student yourself?

I think the perceived pressure is greater. Graduates receive lots of advice about what to do and not to do. When I was a PhD student, there were only four of us in our department. I was even given my own office! But the nice thing about there being four of us was that there was very little competition. It did get a bit lonely but it was a completely different environment. Young academics can also be advised that they need to do more and more and it can become overwhelming and professors need to advise up-and-coming statisticians to simply hang in there and not to be too down when their papers are turned down by a journal. In fact, it is a good survival strategy to try not to be too introspective.

There are far more jobs in this industry now than there ever were before. Also the number of graduates is going down for the first time this year, the birth rate 18 years ago was going down so in a sense, I would say to young academics “Relax. There are fewer competitors behind you and the pressure you are being told about does not really exist.” The post-war baby boom generation are now retiring and the 1960s generation are beginning the end of their careers. You do not have to compete as hard as possible whereas doing good work slowly in the long run is far better, work that you are able to look back on and be happy with ten or twenty years later.

7. Do you have any advice for students considering a university degree in statistics?

It’s well worth it! It’s a fun degree and there is government money as it’s a stem subject, so you get a wider range of choice. The stereotype that it’s a nerdy subject is very wrong. As I’ve said, the statistics students I knew had far more parties and many were aiming to become teachers. I would advise that, if you can, do a combined mathematics and statistics degree but also get out a bit and see what else is going on at your university otherwise you get trapped into one way of thinking. Outside your department, people’s confidence with mathematics and statistics is incredibly low and advice is always gratefully received and you don’t have to become the best mathematician in the world to work outside the department.

8. Over the years, how has your teaching, consulting, and research motivated and influenced each other? Do you get research ideas from statistics and incorporate your ideas into your teaching on geography?

Almost all my research comes from statistics. I just spot things in numbers that look odd and think someone needs to say something about this. For example, last year I spotted worldwide the greatest fall in infant mortality the world has ever had. I see these numbers and think “This matters!” Almost every piece of work has started from a number or I’ve drawn a graph and seen something that I have not seen before. It has changed what I do. My teaching has shown me that you can very quickly get disconnected from people. Explanations that you find are easy actually do not make sense to other able-minded people, including university students, so teaching helps keep your feet on the ground otherwise you can get stuck in your ivory tower dealing with more complicated issues, then no one understands you when you come out and want to talk about what you have found!

I’ve done some consultancy and it’s interesting to see what people want to be done. Curiously, the most interesting work I’ve done has not involved money. When I’ve received funding, which has been fantastic, there has however been less output because of certain constraints. So whilst I think to have your research funded is valuable, it’s not that important if you don’t succeed as the funding does not necessarily improve your research.

9. What do you think the most important recent developments in the field have been? What do you think will be the most exciting and productive areas of research in statistics/social statistics during the next few years?

For me in social statistics, it is the incredible increase in data visualization. The Guardian now has a Datablog and produces award winning graphics. The whole nature of graphics has changed because people are programming graphics again, partly due to small hand held devices and the Web. There was a period from 1989 to mid-2000s when software took over and if the software did not produce a graph in a particular way, there was no way round it and this did halt people’s imaginations developing on how to do graphics. As there is not a single software on how to produce graphics on the web, people have essentially started to write computer programs again at a very high level of sophistication.

The Web allows people to experiment with creating images in different ways and now there is no problem publishing your research as you can show it to millions of people online and get instant feedback, which is very exciting. For example, the work of Mark Newman, a physicist in the US, produces these stretched maps – his most recent is one on Obama’s election victory and, with his PhD student Michael Gastner, he solved a basic equation on how to draw maps almost perfectly in the 1970s which was an exciting breakthrough. The work of Ben Hennig is very intriguing, who is a student of mine admittedly (!) but he has just won one of the top prizes in Germany of €30,000 for his PhD thesis. He has a website (www.viewsoftheworld.com) which is about very different ways of looking at the world and its statistics which just weren’t around four years ago.

I do think people who are producing new graphics could benefit if they looked at what was learnt in the 80s and 90s and their research would progress far more quickly instead of trying to come up with new graphical inventions off the top of their heads.

10. What do you see as the greatest challenges facing the profession of statistics in the coming years?

There is an interesting point of view from social statistics in that when statistics began in the 1880s, there was great interest in social statistics in Britain and Germany. Darwin’s work sparked off an interest in numbers and the early history of statistics – Francis Galton and others is an area of history that is quite hazy as most were eugenists and trying to prove that some people were better than others. At some point, the profession of statistics needs to look at its origins. Similarly in Geography whose modern disciplinary origins lie in how to take over a country, statistics also link to social science and statisticians need to become more involved in social statistics for the planet as a whole as people can get frightened by great quantities of numbers and don’t understand them. Part of the challenge is to explain these numbers but also to confront the past and the origins of statistics and not to be scared about getting involved in social statistics and all the different numbers about people. Some statisticians want to be pure, neutral, apolitical and stand back from such issues. However, I think it is best to be better informed.

The RSS used to have documents concerning their own origins which spoke about promoting society in general and the clause was taken out of the Charter of the Society. If you look at the RSS now, they are trying to get back to their origins as to when it was set up under Queen Victoria in how can we make the world a better place by understanding statistics better. Lots of statisticians would prefer not to work on this side but this is how it all started in terms of statistics being connected with the word ‘state’ and populations. I did the Beveridge lecture this year for the RSS and at the end of the lecture, I went through all the past Presidents like Harold Wilson and Beveridge himself who were interested in statistics and society. The profession of statistics needs to become more involved with every day society rather than deciding if a particular drug is efficient or not.

The getstats campaign is helping to start this and I think there was an uncomfortable awkwardness in the 1980s when it was realised what the origin of statistics was. The 2021 census has currently been cancelled along with a large amount of government survey work. Pharmaceutical companies are hiring more statisticians now which may mean that there are more options for statisticians but the pendulum may swing back again to more statisticians wishing to work in government and social science. There was a similar swing in the years after 1929 in what young graduates chose to do after the Wall Street Crash and they became more interested in public service than before (using statistics to increase profits in banking went down in popularity after 1929), so you may see the same thing happen again due to the economic crisis. I could be completely wrong and things may not change but it’s possible for a rallying call for social statistics to be taken more seriously than in other areas.

11. Are there people or events that have been influential in your career?

The most influential was Stan Openshaw, my supervisor for the PhD simply because he really let me do whatever I liked and provided the equipment that I needed (please see video link below). Richard Boys, my tutor in Statistics at Newcastle was a very young lecturer at the time and he is still there as Professor now. He was great as he let me as a young undergraduate write a giant dissertation as part of the statistics element of my third year work. In hindsight he was particularly encouraging, although at the time you just think that this is normal as, when you are a student, you don’t have many tutors to compare between.

Professor David Rhind who was Chair of the former Statistics Commission and is now Chair of the Nuffield Foundation was a great inspiration, the atlas he and colleagues produced for the 1971 census inspired me, and also Waldo R. Tobler from University of California, Santa Barbara who wrote his first paper on computer cartography in 1959 and he has been writing longer than most people. Lastly Dame Karen Dunnell, a past National Statistician and Chief Executive of ONS gave a talk about the rise of suicides in young people during the 1980s and I asked her after the talk whether I could have her data to draw some graphs and maps. A week later, a whole set of magnetic tapes arrived in the post for me. We produced dozens and dozens of papers on health, equalities and mortality from that data which would not have happened if she had not simply sent the data thinking that someone should look at it.

I’m 44 now and the people that helped me when I was young are the ones who have later gone on themselves to do impressive work so it’s clearly advisable to help people when you’re young and not to worry too much about yourself!