Taking an Interest: What Makes Someone Respond to a Survey

Features

  • Author: Catherine Saunders
  • Date: 05 Jan 2016
  • Copyright: Image appears courtesy of Getty Images

Survey research is fascinating. It’s amazing what people will reveal, just because they were asked. As a researcher and statistician, I’m also taken with the methods behind the survey and with thinking about how the research can be carried out in the most appropriate and rigorous way. This also means having a keen interest in survey non-response.

Researchers have selfish, non-academic reasons to care about survey response rates. For example, top medical journals have minimum survey response rate requirements for accepting research for publication. The United States Supreme Court is interested in non-response too, ruling in 1999 on the statistical methods that could be used to account for non-response in the decennial U. S. Census.

Researchers have two main reasons for caring about non-response. Firstly, it is important to ensure that survey findings are unbiased and representative, and secondly, concern about survey non-response is a key reason given by those who ignore or chose not to act on the findings of research.

thumbnail image: Taking an Interest: What Makes Someone Respond to a Survey

In 2001, a systematic review of strategies that attempted to improve survey response rates identified several concrete steps that can make a difference. They include paying people to take the survey, making the survey shorter, trying to make it more interesting, getting it sponsored by a university and using coloured ink. Online surveys emailed on a Monday also tend to get the highest response rate, according to Survey Monkey, an online survey development company.

At RAND Europe, we conducted a survey earlier this year for a cohort of healthcare quality improvement experts recruited to a new initiative—and got a response rate of 90 percent (compared with typical response rates of about 15% for online surveys). Although it would be nice to say the response was due to our short interesting survey, designed with coloured ink, which was sent out early in the week, something else was probably at work as well. The high levels of enthusiasm displayed by the founding cohort of the new initiative, and interest in the subject of the survey likely factor into the explanation.

Another example of this phenomenon can be found in comparing the response rates of the three major patient experience surveys in England. The General Practice Patient Survey, sent to people registered with a general practitioner, has a response rate of around 37 percent; the Adult Inpatient Survey, sent to those who have stayed overnight in hospital, has a response rate of about 50 percent; and the Cancer Patient Experience Survey, sent to people treated in hospital for cancer, garners a response rate of about 67 percent.

The three surveys are similar in design, subject and content, so one explanation for the variation in response rates could be how interested people are in the survey topic. For instance, those who have recently undergone cancer treatment—a major life event—are likely keener to provide a response to a survey than healthier people, who probably see their doctor far less.

Another recent survey sheds further light on the issue of the relationship between respondent interest and survey response rates. In 2014, the Department of Health commissioned RAND Europe to conduct a survey of senior National Health Service (NHS) innovation stakeholders. After briefly and unsuccessfully wrestling with the issue of how to define the population of “senior NHS innovation stakeholders,” we identified a purposive sample of subjects based on NHS seniority, innovation engagement or both. Potential survey respondents were identified mainly by relying on various lists that identified NHS hospital chief executives, senior managers from Clinical Commissioning Groups (CCGs), previous respondents to a consultation about NHS innovation, people involved with Innovation Health and Wealth (IHW) initiatives; and individuals from Academic Health Science Networks (AHSN).

The overall survey response rate from email contacts was 15.6 percent, or about what was expected. However, the response rates varied substantially depending on which way the potential respondents were identified for inclusion in the survey invitation mailout. For example, response rates ranged from 2.8 percent from CCG senior managers to 51.9 percent from invitations sent to people working in AHSNs. This is presented in Table 1 below.

Table 1. Response rates among different groups of survey invitees

Email invites sent to valid email addresses Number of responses Response rate
All
744
116 15.6
Academic Health Science Network representatives (AHSN) 27 14 51.9
NHS Hospital Chief Executives 100 8 8.0
CCGs 284 8 2.8
People involved in IHW initiatives 54 10 18.5
People who responded to a previous innovation review 156 49 31.4

What do these findings mean? People who responded to previous surveys, or in this case previous consultations on innovation, are more likely to respond again. This phenomenon isn’t limited to surveys— fundraisers well know the value of follow-up requests for further donations.

Responses to the survey on innovation came overwhelmingly from people who were already interested in the topic. Those who were not necessarily interested in the NHS innovation landscape, or did not consider it the main part of their job, responded at a much lower rate.

For surveys in general, one point is abundantly clear. A respondee’s interest in the topic, or interest in responding, is likely to be more important than the colour of the ink or the time of day that a survey is sent.

Catherine Saunders is an analyst and statistician at RAND Europe.

Related Topics

Related Publications

Related Content

Site Footer

Address:

This website is provided by John Wiley & Sons Limited, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ (Company No: 00641132, VAT No: 376766987)

Published features on StatisticsViews.com are checked for statistical accuracy by a panel from the European Network for Business and Industrial Statistics (ENBIS)   to whom Wiley and StatisticsViews.com express their gratitude. This panel are: Ron Kenett, David Steinberg, Shirley Coleman, Irena Ograjenšek, Fabrizio Ruggeri, Rainer Göb, Philippe Castagliola, Xavier Tort-Martorell, Bart De Ketelaere, Antonio Pievatolo, Martina Vandebroek, Lance Mitchell, Gilbert Saporta, Helmut Waldl and Stelios Psarakis.