Reproducibility: Principles, Problems, Practices, and Prospects


thumbnail image: Reproducibility: Principles, Problems, Practices, and Prospects

2017 PROSE Award Honorable Mention
The PROSE Awards draw attention to pioneering works of research and for contributions to the conception, production, and design of landmark works in their fields.

Featuring peer-reviewed contributions from noted experts in their fields of research, Reproducibility: Principles, Problems, Practices, and Prospects presents state-of-the-art approaches to reproducibility, the gold standard of sound science, from multi- and interdisciplinary perspectives. Including comprehensive coverage for implementing and reflecting the norm of reproducibility in various pertinent fields of research, the book focuses on how the reproducibility of results is applied, how it may be limited, and how such limitations can be understood or even controlled in the natural sciences, computational sciences, life sciences, social sciences, and studies of science and technology.

The book presents many chapters devoted to a variety of methods and techniques, as well as their epistemic and ontological underpinnings, which have been developed to safeguard reproducible research and curtail deficits and failures. The book also investigates the political, historical, and social practices that underlie reproducible research in contemporary science studies, including the difficulties of good scientific practice and the ethos of reproducibility in modern innovation societies.

Reproducibility: Principles, Problems, Practices, and Prospects is a guide for researchers who are interested in the general and overarching questions behind the concept of reproducibility; for active scientists who are confronted with practical reproducibility problems in their everyday work; and for economic stakeholders and political decision makers who need to better understand the challenges of reproducibility. In addition, the book is a useful in-depth primer for undergraduate and graduate-level courses in scientific methodology and basic issues in the philosophy and sociology of science from a modern perspective.

“A comprehensive, insightful treatment of the reproducibility challenges facing science today and of ways in which the scientific community can address them.” Kathleen Hall Jamieson, Elizabeth Ware Packard Professor of Communication, University of Pennsylvania

“How can we make sure that reproducible research remains a key imperative of scientific communication under increasing commercialization, media attention, and publication pressure? This handbook offers the first interdisciplinary and fundamental treatment of this important question.”Torsten Hothorn, Professor of Biostatistics, University of Zurich

Harald Atmanspacher, PhD, is Associate Fellow and staff member at Collegium Helveticum, ETH and University Zurich and is also President of the Society for Mind-Matter Research.  He has pioneered advances in complex dynamical systems research and in a number of topics concerned with the relation between the mental and physical.

Sabine Maasen, PhD, is Professor for Sociology of Science and Director of the Munich Center for Technology in Society (TU Munich) and Associate Fellow at Collegium Helveticum (ETH and University Zurich).  Her research focuses on the interface of science, technology, and society, notably with respect to neuroscience and its applications. 

Contributors ix

Introduction 1
Harald Atmanspacher and Sabine Maasen


Introductory Remarks 9
Harald Atmanspacher

Reproducibility, Objectivity, Invariance 13
Holm Tetens

Reproducibility between Production and Prognosis 21
Walther ChZimmerli

Stability and Replication of Experimental Results: A Historical Perspective 39
Friedrich Steinle

Reproducibility of Experiments: Experimenters’ Regress, Statistical Uncertainty Principle, and the Replication Imperative 65
Harry Collins


Introductory Remarks 83
Harald Atmanspacher

Statistical Issues in Reproducibility 87
Werner AStahel

Model Selection, Data Distributions and Reproducibility 115
Richard Shiffrin and Suyog Chandramouli

Reproducibility from the Perspective of Meta-Analysis 141
Werner Ehm

Why Are There so Many Clustering Algorithms, and How Valid Are Their Results? 169
Vladimir Estivill-Castro


Introductory Remarks 201
Harald Atmanspacher

Facilitating Reproducibility in Scientific Computing: Principles and Practice 205
David H Bailey, Jonathan M Borwein, and Victoria Stodden

Methodological Issues in the Study of Complex Systems 233
Harald Atmanspacher and Gerda Demmel

Rare and Extreme Events 251
Holger Kantz

Science under Societal Scrutiny: Reproducibility in Climate Science 269
Georg Feulner


Introductory Remarks 287
Harald Atmanspacher

From Mice to Men: Translation from Bench to Bedside 291
Marianne Martic-Kehl and P August Schubiger

A Continuum of Reproducible Research in Drug Development 315
Gerd Folkers and Sabine Baier

Randomness as a Building Block for Reproducibility in Local Cortical Networks 325
Johannes Lengler and Angelika Steger

Neural Reuse and in-Principle Limitations on Reproducibility in Cognitive Neuroscience 341
Michael L Anderson

On the Difference between Persons and Things–Reproducibility in Social Contexts 363
Kai Vogeley


Introductory Remarks 385
Sabine Maasen and Harald Atmanspacher

Order Effects in Sequential Judgments and Decisions 391
Zheng Wang and Jerome Busemeyer

Reproducibility in the Social Sciences 407
Martin Reinhart

Accurate But Not Reproducible? The Possible Worlds of Public Opinion Research 425
Felix Keller

Depending on Numbers 447
Theodore M Porter

Science between Trust and Control: Non-Reproducibility in Scholarly Publishing 467
Martina Franzen


Introductory Remarks 487
Sabine Maasen and Harald Atmanspacher

Repetition with a Difference: Reproducibility in Literature Studies 491
Ladina Bezzola Lambert

Repetition Impossible: Co-Affection by Mimesis and Self-Mimesis 511
Hinderk Emrich

Relevance Criteria for Reproducibility: The Contextual Emergence of Granularity 527
Harald Atmanspacher

The Quest for Reproducibility Viewed in the Context of Innovation Societies 541
Sabine Maasen

Index 563

Related Topics

Related Publications

Related Content

Site Footer


This website is provided by John Wiley & Sons Limited, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ (Company No: 00641132, VAT No: 376766987)

Published features on are checked for statistical accuracy by a panel from the European Network for Business and Industrial Statistics (ENBIS)   to whom Wiley and express their gratitude. This panel are: Ron Kenett, David Steinberg, Shirley Coleman, Irena Ograjenšek, Fabrizio Ruggeri, Rainer Göb, Philippe Castagliola, Xavier Tort-Martorell, Bart De Ketelaere, Antonio Pievatolo, Martina Vandebroek, Lance Mitchell, Gilbert Saporta, Helmut Waldl and Stelios Psarakis.