Research Synthesis Methods

A comparison of methods for meta‐analysis of a small number of studies with binary outcomes

Journal Article

Meta‐analyses often include only a small number of studies (≤5). Estimating between‐study heterogeneity is difficult in this situation. An inaccurate estimation of heterogeneity can result in biased effect estimates and too narrow confidence intervals. The beta‐binominal model has shown good statistical properties for meta‐analysis of sparse data. We compare the beta‐binominal model with different inverse variance random (eg, DerSimonian‐Laird, modified Hartung‐Knapp, and Paule‐Mandel) and fixed effects methods (Mantel‐Haenszel and Peto) in a simulation study. The underlying true parameters were obtained from empirical data of actually performed meta‐analyses to best mirror real‐life situations. We show that valid methods for meta‐analysis of a small number of studies are available. In fixed effects situations, the Mantel‐Haenszel and Peto methods performed best. In random effects situations, the beta‐binominal model performed best for meta‐analysis of few studies considering the balance between coverage probability and power. We recommended the beta‐binominal model for practical application. If very strong evidence is needed, using the Paule‐Mandel heterogeneity variance estimator combined with modified Hartung‐Knapp confidence intervals might be useful to confirm the results. Notable most inverse variance random effects models showed unsatisfactory statistical properties also if more studies (10‐50) were included in the meta‐analysis.

Related Topics

Related Publications

Related Content

Site Footer


This website is provided by John Wiley & Sons Limited, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ (Company No: 00641132, VAT No: 376766987)

Published features on are checked for statistical accuracy by a panel from the European Network for Business and Industrial Statistics (ENBIS)   to whom Wiley and express their gratitude. This panel are: Ron Kenett, David Steinberg, Shirley Coleman, Irena Ograjenšek, Fabrizio Ruggeri, Rainer Göb, Philippe Castagliola, Xavier Tort-Martorell, Bart De Ketelaere, Antonio Pievatolo, Martina Vandebroek, Lance Mitchell, Gilbert Saporta, Helmut Waldl and Stelios Psarakis.