Research Synthesis Methods

The value of a second reviewer for study selection in systematic reviews

Early View

  • Author(s): Carolyn R.T. Stoll, Sonya Izadi, Susan Fowler, Paige Green, Jerry Suls, Graham A. Colditz
  • Article first published online: 18 Jul 2019
  • DOI: 10.1002/jrsm.1369
  • Read on Online Library
  • Subscribe to Journal

Background Although dual independent review of search results by two reviewers is generally recommended for systematic reviews, there are not consistent recommendations regarding the timing of the use of the second reviewer. This study compared the use of a complete dual review approach, with two reviewers in both the title/abstract screening stage and the full‐text screening stage, as compared with a limited dual review approach, with two reviewers only in the full‐text stage. Methods This study was performed within the context of a large systematic review. Two reviewers performed a complete dual review of 15 000 search results and a limited dual review of 15 000 search results. The number of relevant studies mistakenly excluded by highly experienced reviewers in the complete dual review was compared with the number mistakenly excluded during the full‐text stage of the limited dual review. Results In the complete dual review approach, an additional 6.6% to 9.1% of eligible studies were identified during the title/abstract stage by using two reviewers, and an additional 6.6% to 11.9% of eligible studies were identified during the full‐text stage by using two reviewers. In the limited dual review approach, an additional 4.4% to 5.3% of eligible studies were identified with the use of two reviewers. Conclusions Using a second reviewer throughout the entire study screening process can increase the number of relevant studies identified for use in a systematic review. Systematic review performers should consider using a complete dual review process to ensure all relevant studies are included in their review.

Related Topics

Related Publications

Related Content

Site Footer


This website is provided by John Wiley & Sons Limited, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ (Company No: 00641132, VAT No: 376766987)

Published features on are checked for statistical accuracy by a panel from the European Network for Business and Industrial Statistics (ENBIS)   to whom Wiley and express their gratitude. This panel are: Ron Kenett, David Steinberg, Shirley Coleman, Irena Ograjenšek, Fabrizio Ruggeri, Rainer Göb, Philippe Castagliola, Xavier Tort-Martorell, Bart De Ketelaere, Antonio Pievatolo, Martina Vandebroek, Lance Mitchell, Gilbert Saporta, Helmut Waldl and Stelios Psarakis.