Research Synthesis Methods

Critical interpretation of Cochran's Q test depends on power and prior assumptions about heterogeneity

Journal Article

Abstract

We describe how an appropriate interpretation of the Q‐test depends on its power to detect a given typical amount of between‐study variance (τ2) as well as prior beliefs on heterogeneity. We illustrate these concepts in an evaluation of 1011 meta‐analyses of clinical trials with ⩾4 studies and binary outcomes. These concepts can be seen as an application of the Bayes theorem. Across the 1011 meta‐analyses, power to detect typical heterogeneity was low in most situations. Thus, usually a non‐significant Q test did not change perceptibly prior convictions on heterogeneity. Conversely, significant results for the Q test typically augmented considerably the probability of heterogeneity. The posterior probability of heterogeneity depends on what τ2 we want to detect. With the same approach, one may also estimate the posterior probability for the presence of heterogeneity that is large enough to annul statistically significant summary effects; that is half the average within‐study variance of the combined studies; and that is able to change the summary effect estimate of the meta‐analysis by 20%. The discussed analyses are exploratory, and may depend heavily on prior assumptions when power for the Q‐test is low. Statistical heterogeneity in meta‐analyses should be cautiously interpreted considering the power to detect a specific τ2 and prior assumptions about the presence of heterogeneity. Copyright © 2010 John Wiley & Sons, Ltd.

Related Topics

Related Publications

Related Content

Site Footer

Address:

This website is provided by John Wiley & Sons Limited, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ (Company No: 00641132, VAT No: 376766987)

Published features on StatisticsViews.com are checked for statistical accuracy by a panel from the European Network for Business and Industrial Statistics (ENBIS)   to whom Wiley and StatisticsViews.com express their gratitude. This panel are: Ron Kenett, David Steinberg, Shirley Coleman, Irena Ograjenšek, Fabrizio Ruggeri, Rainer Göb, Philippe Castagliola, Xavier Tort-Martorell, Bart De Ketelaere, Antonio Pievatolo, Martina Vandebroek, Lance Mitchell, Gilbert Saporta, Helmut Waldl and Stelios Psarakis.