Research Synthesis Methods

Estimation of an overall standardized mean difference in random‐effects meta‐analysis if the distribution of random effects departs from normal

Journal Article

  • Author(s): María Rubio‐Aparicio, José Antonio López‐López, Julio Sánchez‐Meca, Fulgencio Marín‐Martínez, Wolfgang Viechtbauer, Wim Van den Noortgate
  • Article first published online: 30 Jul 2018
  • DOI: 10.1002/jrsm.1312
  • Read on Online Library
  • Subscribe to Journal

The random‐effects model, applied in most meta‐analyses nowadays, typically assumes normality of the distribution of the effect parameters. The purpose of this study was to examine the performance of various random‐effects methods (standard method, Hartung's method, profile likelihood method, and bootstrapping) for computing an average effect size estimate and a confidence interval (CI) around it, when the normality assumption is not met. For comparison purposes, we also included the fixed‐effect model. We manipulated a wide range of conditions, including conditions with some degree of departure from the normality assumption, using Monte Carlo simulation. To simulate realistic scenarios, we chose the manipulated conditions from a systematic review of meta‐analyses on the effectiveness of psychological treatments. We compared the performance of the different methods in terms of bias and mean squared error of the average effect estimators, empirical coverage probability and width of the CIs, and variability of the standard errors. Our results suggest that random‐effects methods are largely robust to departures from normality, with Hartung's profile likelihood methods yielding the best performance under suboptimal conditions.

Related Topics

Related Publications

Related Content

Site Footer


This website is provided by John Wiley & Sons Limited, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ (Company No: 00641132, VAT No: 376766987)

Published features on are checked for statistical accuracy by a panel from the European Network for Business and Industrial Statistics (ENBIS)   to whom Wiley and express their gratitude. This panel are: Ron Kenett, David Steinberg, Shirley Coleman, Irena Ograjenšek, Fabrizio Ruggeri, Rainer Göb, Philippe Castagliola, Xavier Tort-Martorell, Bart De Ketelaere, Antonio Pievatolo, Martina Vandebroek, Lance Mitchell, Gilbert Saporta, Helmut Waldl and Stelios Psarakis.