Designing and Conducting Business Surveys

Books

thumbnail image: Designing and Conducting Business Surveys
  • Published: 01 March 2013
  • ISBN: 9780470903049
  • Author(s): Ger Snijkers, Gustav Haraldsen, Jacqui Jones, Diane Willimack
  • View full details
  • Buy the book
Designing and Conducting Business Surveys provides a coherent overview of the business survey process, from start to finish. It uniquely integrates an understanding of how businesses operate, a total survey error approach to data quality that focuses specifically on business surveys, and sound project management principles. The book brings together what is currently known about planning, designing, and conducting business surveys, with producing and disseminating statistics or other research results from the collected data. This knowledge draws upon a variety of disciplines such as survey methodology, organizational sciences, sociology, psychology, and statistical methods. The contents of the book formulate a comprehensive guide to scholarly material previously dispersed among books, journal articles, and conference papers.

This book provides guidelines that will help the reader make educated trade-off decisions that minimize survey errors, costs, and response burden, while being attentive to survey data quality. Major topics include:

• Determining the survey content, considering user needs, the business context, and total survey quality

• Planning the survey as a project

• Sampling frames, procedures, and methods

• Questionnaire design and testing for self-administered paper, web, and mixed-mode surveys

• Survey communication design to obtain responses and facilitate the business response process

• Conducting and managing the survey using paradata and project management tools

• Data processing, including capture, editing, and imputation, and dissemination of statistical outputs

Designing and Conducting Business Surveys is an indispensable resource for anyone involved in designing and/or conducting business or organizational surveys at statistical institutes, central banks, survey organizations, etc.; producing statistics or other research results from business surveys at universities, research organizations, etc.; or using data produced from business surveys. The book also lays a foundation for new areas of research in business surveys.

Preface xix

Contributors xxiii

Abbreviations xxv

1. Surveys and Business Surveys 1
Jacqui Jones, Ger Snijkers, and Gustav Haraldsen

1.1. The Objective of this Book, 1

1.2. The Statistical Production Process, 2

1.3. Surveys, 4

1.3.1. Definition of Surveys, 4

1.3.2. Survey Constraints, Considerations, and Tradeoffs, 5

1.3.2.1. Survey Errors and the Process Quality Approach, 5

1.3.2.2. Response Burden, 7

1.3.2.3. Trust, 7

1.3.3. A Brief Historical Look at Surveys, 8

1.4. Types of Business Data Outputs, 10

1.4.1. Official Statistics, 10

1.4.1.1. National Accounts, 11

1.4.1.2. Structural Business Statistics, 12

1.4.1.3. Short-Term Statistics, 13

1.4.1.4. International and European Comparisons, 13

1.4.2. Other Types of Business Data Outputs, 14

1.5. Use of Business Data Outputs, 16

1.5.1. National, Regional, and Local Economic Performance, 17

1.5.2. Individual Business Monitoring and Policymaking, 18

1.5.3. Everyday Decisions, 18

1.6. Obtaining Business Data, 19

1.6.1. Business Data Collection Methods, 19

1.7. Business Surveys, 22

1.7.1. Types of Business Surveys, 22

1.7.2. The Business Survey–Output Production Process, 23

1.7.2.1. Business Survey Constraints, Considerations, and Tradeoffs, 23

1.7.2.2. Unique Features of Business Surveys, 25

1.7.3. Perspectives of the Business Survey Process, 26

1.7.3.1. Survey Organization Perspective, 26

1.7.3.2. Respondent Perspective, 27

1.7.3.3. User Perspective, 28

1.8. Overview of the Book, 28

1.8.1. The Audience, 28

1.8.2. Organization of the Book, 29

1.8.3. Chapter Summaries, 29

Acknowledgments, 33

Appendix, 33

2. The Business Context and its Implications for the Survey Response Process 39
Diane K. Willimack and Ger Snijkers

2.1. The Business Context from the Perspective of the Organizational Sciences, 40

2.1.1. Business Goals and Behaviors and Implications for Survey Response, 40

2.1.2. Dimensions of Social Behavior in Organizations Pertinent to the Survey Response Task, 45

2.1.3. The Response Process Viewed as Work, 49

2.1.3.1. How Work is Accomplished, 49

2.1.3.2. Social Behavioral Dimensions of Work, 49

2.1.3.3. Accomplishing the Work of Survey Response, 50

2.2. A Comprehensive Approach Integrating the Business Context and the Survey Response Process, 51

2.2.1. The Decision to Participate, 54

2.2.1.1. External Environment Factors, 54

2.2.1.2. Business/Management Factors, 55

2.2.2. Performing Response Tasks, 60

2.2.2.1. Organizational Context for Performing Response Tasks, 61

2.2.2.2. Organizational Context and Social Behavior, 66

2.2.2.3. Organizational Context and the Cognitive Response Process, 69

2.2.3. Release of the Questionnaire, 75

2.2.4. Role of the Survey Organization in the Business Survey Response Process: Implications for Survey Design, 75

2.3. Summary, 79

Acknowledgments, 82

3. Quality Issues in Business Surveys 83
Gustav Haraldsen

3.1. Survey Quality from a User/producer Perspective, 84

3.1.1. Users and User Evaluations of Business Surveys, 86

3.1.2. The Total Survey Error Approach, 88

3.1.2.1. Quality Constraints, 90

3.1.2.2. Survey-Related Effects, 93

3.2. Sample-Related Quality Issues, 96

3.2.1. A Glimpse into Transaction Studies, 96

3.2.2. Specifying Observational Units, 97

3.2.3. The Convenience Sample Element, 99

3.2.4. Sampling Error Issues, 101

3.2.5. Reporting Units and Informants, 102

3.2.6. Response Rate Issues, 103

3.2.6.1. The Significance of Legal Obligations, 104

3.2.7. Quality Effects of Postsurvey Adjustments, 106

3.3. Measurement-Related Quality Issues, 106

3.3.1. Validity Challenges during Survey Specification, 107

3.3.2. Sources of Measurement Error in Business Surveys, 108

3.3.2.1. The Business Survey Context, 110

3.3.2.2. Question and Questionnaire Elements, 112

3.3.2.3. The Content of Business Questionnaires, 116

3.3.3. Measurement Issues Summarized, 118

3.4. Sample and Measurement Issues Combined, 120

Acknowledgment, 125

4. Planning the Survey 127
Ger Snijkers, Gustav Haraldsen, and Jacqui Jones

4.1. Introduction, 127

4.2. Business Surveys and Project Management Principles, 128

4.3. Specifying and Planning the Survey, 131

4.3.1. Specifying the Survey, 132

4.3.2. Planning the Survey in Detail: The Planning Process, 135

4.3.2.1. The Project Plan, 135

4.3.2.2. The Business Survey Production Environment, 138

4.3.3. Planning the Fieldwork, 142

4.3.4. Planning the Designing, Building, and Testing of the Survey Components, 143

4.3.4.1. Planning Mode(s) of Data Collection and Questionnaire Design, 143

4.3.4.2. Planning the Sample Design and Estimation, 146

4.3.4.3. Planning the Survey Communication Strategy Design, 147

4.3.4.4. Planning the Data Collection Process, 148

4.3.4.5. Planning the Data Processing Procedures, 150

4.3.5. Planning the Data Analysis, Dissemination, and Archiving, 151

4.3.6. Planning the Survey Evaluation, 152

4.4. Required Resources, 153

4.5. The Timetable, 153

4.6. Planning Managing, Controlling, Monitoring, and Evaluation of the Survey Project, 154

4.7. Risk Management, 156

4.8. Finalizing the Project Plan and Getting Started, 162

4.9. Summary and Progress Status Overview, 162

Acknowledgment, 163

5. Sampling and Estimation for Business Surveys 165
Paul Smith

5.1. Basic Principles, 165

5.2. Registers and Frames, 167

5.2.1. Business Registers, 167

5.2.1.1. Sources for Constructing Registers, 170

5.2.1.2. Classification, 170

5.2.1.3. Frames from Registers, 171

5.2.1.4. Updating Registers, 171

5.2.2. Constructing a Frame (without a Business Register), 174

5.2.3. Register and Frame Quality, 175

5.2.4. Dual-Frame Surveys, 176

5.3. Sample Design, 177

5.3.1. Some Basic Ideas, 177

5.3.2. Stratified Sampling, 178

5.3.2.1. Stratification Variables, 179

5.3.2.2. Defining the Strata, 180

5.3.2.3. Sample Allocation, 188

5.3.2.4. Extra Details of Allocation, 191

5.3.2.5. Multivariate and Multipurpose Sampling, 193

5.3.2.6. Numerical Methods and Software for Stratification, 194

5.3.2.7. Sample Selection, 194

5.3.3. Probability Proportional to Size (PPS) Sampling, 195

5.3.4. Cutoff Sampling, 196

5.3.5. Sample Rotation and Overlap, 197

5.3.6. Special Sample Designs, 199

5.3.6.1. Product Surveys and Other Detailed Breakdowns of Activity, 199

5.3.6.2. Price Index Surveys, 200

5.3.6.3. Surveys for Rare Characteristics (“Needle in a Haystack” Surveys), 201

5.4. Estimation, 202

5.4.1. Horvitz–Thompson Estimation, 202

5.4.2. Some Basic Ideas, 203

5.4.3. Ratio Estimation, 205

5.4.4. Combined Ratio Estimation, 207

5.4.5. More Complex Estimators, 208

5.4.6. Accuracy of Auxiliary Information, 208

5.4.7. Optimal Uses of Auxiliary Information, 209

5.4.8. Estimating Discontinuities, 209

5.5. Outliers, 210

5.5.1. Inspection and “Surprise Strata”, 211

5.5.2. Winsorization, 212

5.5.3. Other Approaches, 213

5.6. Variance Estimation, 214

5.7. Small Area Estimation, 216

5.8. Summary, 217

Acknowledgments, 218

6. Understanding and Coping with Response Burden 219
Gustav Haraldsen, Jacqui Jones, Deirdre Giesen, and Li-Chun Zhang

6.1. A Cost versus a Cost Benefit Approach, 220

6.1.1. Politicians’ Bird’s-Eye View, 221

6.1.2. Commercial Worries, 222

6.1.3. Perceived Response Burden, 223

6.2. Measuring and Monitoring Business Response Burden, 226

6.2.1. Measuring and Calculating Actual Response Burden, 229

6.2.2. Measuring Perceived Response Burden, 232

6.2.3. Calculations, 234

6.3. An Analytical Approach to Response Burden, 235

6.3.1. Disentangling Some Sources of Response Burden, 238

6.3.2. Suggesting a Curvilinear Correlation between Response Burden and Response Quality, 239

6.4. Measures to Reduce Response Burden, 241

6.4.1. Survey Reduction Measures, 241

6.4.2. Sample Coordination, 243

6.4.2.1. Common Frame and Regime, 244

6.4.2.2. Effects on Perceived Response Burden, 245

6.4.3. Survey Communication Measures, 248

6.4.4. Questionnaire Improvement Measures, 249

6.4.5. Tailored Evaluations of Response Burden Reduction Measures, 251

Acknowledgments, 252

7. Methods for the Development, Testing, and Evaluation of Data Collection Instruments 253
Diane K. Willimack

7.1. A Framework for Questionnaire Development, Testing, and Evaluation, 254

7.1.1. Business Survey Response Model and Measurement Error, 256

7.1.2. Constraints and Limitations to Questionnaire Development, Testing, and Evaluation in the Business Survey Context, 258

7.2. Developing Content, Measurements, and Survey Questions, 260

7.2.1. The Roles of Subject Area Specialists, Stakeholders, and Data Users, 262

7.2.2. Methods for Content Development and Specifying Measurements, 262

7.2.2.1. Top–Down Theory-Driven Approaches, 263

7.2.2.2. Bottom–Up Data-Driven Approaches, 264

7.3. Testing Questions and Questionnaires, 267

7.3.1. The Response Model as a Framework for Testing, 267

7.3.2. Methods, 269

7.4. Evaluating Survey Questions and Questionnaires, 276

7.4.1. Methods Used during Data Collection, 276

7.4.2. Methods Used after Data Collection, 278

7.4.3. Methods for Identifying Questionnaire Problems in Recurring Surveys, 280

7.5. Development, Testing, and Evaluation of Electronic Data Collection Instruments, 282

7.5.1. Development Strategies for Electronic Instruments, 282

7.5.2. Usability Testing, 285

7.5.3. Evaluating Web Survey Data Collection Instruments, 286

7.6. Analyzing Qualitative Data, 287

7.6.1. Data Capture, 288

7.6.2. Types of Analysis, 288

7.6.3. Data Reduction, 289

7.6.4. Data Display, 290

7.6.5. Drawing Conclusions and Verification, 290

7.7. Multimethod Approaches to Development, Testing, and Evaluation, 290

7.8. Organization and Logistics, 293

7.8.1. Survey Organizational Structures for Instrument Development, Testing, and Evaluation, 293

7.8.2. Pretesting Logistics, 294

7.9. Summary, 295

Acknowledgments, 298

Appendix 7.A.1 Example of Conclusions from Qualitative Data Analysis: Public Trust in Statistics: A Conceptual

Model of Trust Determinants, 298

Appendix 7.A.2 Case Study: Logistics of Questionnaire Pretesting for Economic Survey Programs at the US Census Bureau, 299

8. Questionnaire Communication in Business Surveys 303
Gustav Haraldsen

8.1. Communication Modes, 303

8.2. Elements in Concert, 306

8.3. Tailoring for Businesses, 311

8.3.1. Setting the Scene, 313

8.3.1.1. Creating Trust, 314

8.3.1.2. Assigning Tasks, 315

8.3.2. Common Access to Several Web Surveys, 317

8.3.2.1. Simple and Advanced Business Web Portals, 317

8.3.2.2. Visual Design of the Web Portal, 320

8.3.2.3. Data Collection Instruments for Small Companies and Small Questionnaires, 323

8.3.2.4. Mobile Data Collection Instruments in a Mixed-Mode Design, 324

8.3.3. Layout and Functional Tailoring for Internal Data Collection, 325

8.3.3.1. Guiding the Questionnaire to the Right Respondent(s), 325

8.3.3.2. Guidelines and Tailoring Tools in Web Questionnaires, 325

8.3.3.3. Intercomputer Communication, 331

8.4. Phrasing Questions in Business Surveys, 332

8.4.1. From Requirements to Inquiries, 332

8.4.2. Clarifying Terms and Tasks, 335

8.4.3. Listening to Questionnaires, 340

8.4.3.1. How Question Order Affects Comprehension, 340

8.4.3.2. Implicit Expectations, 341

8.4.3.3. Looking for Hints, 341

8.4.3.4. Talking with Web Questionnaires, 345

8.5. Minimizing Response Burden, 346

8.5.1. Time Saving Measures, 347

8.5.1.1. IT Tools, 348

8.5.1.2. Presentations, 350

8.5.2. Reducing the Cognitive Burden, 351

8.5.2.1. From Preprinting to Dependent Dialogs, 351

8.5.2.2. Manageable Matrices, 353

8.5.2.3. Computerized Matrices, 355

8.6. Concluding Reflections, 357

Acknowledgments, 357

9. Business Survey Communication 359
Ger Snijkers and Jacqui Jones

9.1. Introduction, 359

9.2. The Role, Objectives, and Process of Business Survey Communication, 360

9.2.1. Role, 360

9.2.2. Objectives, 361

9.2.3. Process, 362

9.2.3.1. Prefield Survey Communication Stage, 365

9.2.3.2. Field Survey Communication Stage, 368

9.2.3.3. Postfield Survey Communication Stage, 371

9.3. Tailoring to the Business Context, 373

9.3.1. Actors and Their Roles in the Response Process, 373

9.3.2. Business Size and Economic Sector, 374

9.3.3. Factors Affecting the Response Process, 375

9.3.4. Questions and Complaints, 377

9.4. The Survey Organization Context, 377

9.4.1. Staff in the Communication Process and Their Roles, 377

9.4.2. Survey Organization Policies and Procedures, 383

9.4.2.1. Informed Consent and Confidentiality, 383

9.4.2.2. Mandatory Surveys, 387

9.4.3. Survey Communication in Perspective, 388

9.5. Designing an Effective Business Survey Communication Strategy, 389

9.5.1. Planning and Designing the Communication Strategy, 390

9.5.2. Effectiveness of Communication Modes, 398

9.5.3. Effectively Motivating and Facilitating, 400

9.5.3.1. Motivation Strategies, 401

9.5.3.2. Facilitation Strategies, 407

9.6. Business Survey Communication Measures, Practices, and Guidelines, 407

9.6.1. Prefield Communication Measures, 408

9.6.1.1. Measures for Increasing the Chances of Establishing Contact, 408

9.6.1.2. Measures for Introducing the Survey and Seeking Cooperation, 410

9.6.1.3. Reducing Response Burden, 412

9.6.2. Field Communication Measures, 412

9.6.2.1. Making Contact, Dispatching the Questionnaire, and Informing the Business about the Survey, 412

9.6.2.2. Measures for Increasing Survey Participation, 414

9.6.2.3. Providing Assistance with Questions and Complaints: Help Desk, Website, Web Care, and Ombudsman, 416

9.6.2.4. Nonresponse Follow-ups and Refusal Conversion, 418

9.6.2.5. Measures for Maintaining a Relationship, 420

9.6.3. Special Cases: Large Businesses and Mixed-Mode Designs, 421

9.6.3.1. Dealing with Large and Multisurveyed Businesses, 421

9.6.3.2. Introducing Web Mixed-Mode Designs, 422

9.7. Summary and Example of a Communication Strategy, 426

Acknowledgment, 430

10. Managing the Data Collection 431
Ger Snijkers and Gustav Haraldsen

10.1. Introduction, 431

10.2. Implementing the Survey, 432

10.3. Active Fieldwork Management, 434

10.4. Paradata, 436

10.5. Monitoring the Quality of the Response, 440

10.5.1. A Classification of Data Collection Outcome Codes, 441

10.5.2. Response and Return Rates at Unit Level, 445

10.5.3. Weights of Businesses, 447

10.5.4. Distribution of the Response: Representativeness, 448

10.5.5. Quality of the Measurements, 449

10.6. Monitoring the Survey Production Process, 451

10.7. Survey Evaluation and Quality Report, 454

10.8. Summary, 456

Acknowledgment, 457

11. Capturing, Coding, and Cleaning Survey Data 459
Jacqui Jones and Mike Hidiroglou

11.1. Introduction, 459

11.2. Data Capture, 461

11.2.1. Step One: Initial Data Capture, 461

11.2.2. Minimizing, Measuring, and Monitoring Errors in Initial Data Capture, 462

11.2.3. Step Two: Data Transfer, 464

11.2.4. Step Three: Data Capture Processing, 465

11.2.5. Capturing Data from Paper Questionnaires, 466

11.2.6. Electronic Scanning, 468

11.2.7. Design of Paper Self-Completion Questionnaires for Electronic Scanning, 468

11.2.8. Instructions for Completing a Paper Self-Completion Questionnaire to be Scanned, 471

11.2.9. Minimizing, Measuring, and Monitoring Errors in Data Capture Processing, 473

11.3. Data Coding, 475

11.3.1. Coding Descriptions to Classifications, 478

11.3.2. Questionnaire Design Implications, 478

11.3.3. Two Examples of Data Coding Tools, 480

11.3.4. Minimizing, Measuring, and Monitoring Errors in Data Coding, 481

11.4. Editing and Imputation, 483

11.4.1. An Overview of Editing and Imputation in Business Surveys, 483

11.4.2. Microediting, 485

11.4.3. Types of Microdata Error, 485

11.4.4. Methods for Identifying Microerrors, 489

11.4.4.1. Data Validation Edits, 490

11.4.4.2. Statistical Edits, 491

11.4.5. Treatment of Microdata Errors, 492

11.4.6. Identifying Errors at the Macroediting Stage, 496

11.4.6.1. Macroediting Methods, 496

11.4.7. Imputation, 498

11.4.7.1. Deterministic Imputation Methods, 499

11.4.7.2. Stochastic Imputation Methods, 500

11.4.8. Minimizing, Measuring, and Monitoring Errors in Editing and Imputation, 500

11.4.8.1. Editing Quality Process Measures, 501

11.4.8.2. Imputation Quality Process Measures, 502

11.4.9. Editing Strategies, 502

11.5. Concluding Comments, 503

Acknowledgments, 504

12. From Survey Data to Statistics 505
Jacqui Jones

12.1. Introduction, 505

12.2. Analysis, 505

12.2.1. Analysis and Different Types of Statistical Outputs, 507

12.2.2. Analytical Technique Considerations, 510

12.2.2.1. Analyzing Different Types of Data, 510

12.2.2.2. Aggregate and Microlevel Data Analysis, 510

12.2.2.3. Analysis of Time Series Data, 511

12.2.2.4. Analyzing Single- or Multicomponent Series, 511

12.2.2.5. Analyzing Data with Different Sample Designs, 512

12.2.2.6. Pitfalls to Avoid in Statistical Analysis, 512

12.2.3. Analytical Techniques for Improving Statistical Interpretation, 513

12.2.3.1. Seasonal Adjustment, 513

12.2.3.2. Index Numbers, 518

12.2.3.3. Values and Volumes, 521

12.2.3.4. Consumer Price Index (CPI), 524

12.2.3.5. Retail Prices Index (RPI), 525

12.2.3.6. Producer Price Index (PPI), 525

12.2.3.7. Services Producer Price Index (SPPI), 526

12.2.3.8. Import and Export Indices, 526

12.2.3.9. Implied Deflators, 527

12.3. Preparing for Dissemination, 527

12.3.1. Preparing Statistical Tables and Datasets, 528

12.3.2. The Importance of Maintaining Confidentiality, 530

12.3.3. Disclosure Control Methods, 532

12.3.3.1. Microdata Disclosure Control Methods, 532

12.3.3.2. Tabular Disclosure Control Methods, 533

12.3.4. Presenting the Statistics, 536

12.3.4.1. Writing Statistical Commentary, 536

12.3.4.2. Identifying the Headline Message, 537

12.3.4.3. Information to Support Disseminated Statistics, 537

12.4. Dissemination, 539

12.4.1. Evaluating Statistical Dissemination, 539

12.5. Archiving, 540

12.6. Concluding Comments, 542

Acknowledgments, 543

References 545

Index 597

Related Topics

Related Publications

Related Content

Address:

This website is provided by John Wiley & Sons Limited, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ (Company No: 00641132, VAT No: 376766987)

Published features on StatisticsViews.com are checked for statistical accuracy by a panel from the European Network for Business and Industrial Statistics (ENBIS)   to whom Wiley and StatisticsViews.com express their gratitude. This panel are: Ron Kenett, David Steinberg, Shirley Coleman, Irena Ograjenšek, Fabrizio Ruggeri, Rainer Göb, Philippe Castagliola, Xavier Tort-Martorell, Bart De Ketelaere, Antonio Pievatolo, Martina Vandebroek and Lance Mitchell.