Over the summer, Wiley was proud to publish Reproducibility: Principles, Problems, Practices, and Prospects which presents state-of-the-art approaches to reproducibility, the gold standard of sound science, from multi- and interdisciplinary perspectives. Including comprehensive coverage for implementing and reflecting the norm of reproducibility in various pertinent fields of research, the book focuses on how the reproducibility of results is applied, how it may be limited, and how such limitations can be understood or even controlled in the natural sciences, computational sciences, life sciences, social sciences, and studies of science and technology.
The book presents many chapters devoted to a variety of methods and techniques, as well as their epistemic and ontological underpinnings, which have been developed to safeguard reproducible research and curtail deficits and failures. The book also investigates the political, historical, and social practices that underlie reproducible research in contemporary science studies, including the difficulties of good scientific practice and the ethos of reproducibility in modern innovation societies.
Reproducibility: Principles, Problems, Practices, and Prospects is a guide for researchers who are interested in the general and overarching questions behind the concept of reproducibility; for active scientists who are confronted with practical reproducibility problems in their everyday work; and for economic stakeholders and political decision makers who need to better understand the challenges of reproducibility. In addition, the book is a useful in-depth primer for undergraduate and graduate-level courses in scientific methodology and basic issues in the philosophy and sociology of science from a modern perspective.
Alison Oliver talks to co-author Harald Atmanspacher about the process of writing the book.
1. Congratulations on the upcoming publication of Reproducibility: Principles, Problems, Practices, and Prospects. How did the book come about in the first place?
It’s been realized for more than a decade that in almost all domains of science the reproducibility of research results has become problematic. This is not only due to faked data, unsuitable experimental design, flawed data analysis, or inappropriate modeling — there are also cases with in-principle limitations with respect to reproducibility. More information about this state of affairs is included in the introduction to the handbook.
In 2009, an interdisciplinary initiative at Collegium Helveticum (jointly operated by University and ETH Zurich) was established to study this situation in more detail and develop ways to meet the resulting challenges. The handbook is the outcome of six years of work of this activity, with leading scientists and philosophers of science discussing and assessing the topic. It is the first comprehensive account on reproducibility and its ramifications on the book market.
2. What were the primary objectives that you had in mind when writing the book?
As the subtitle of the book says, reproducibility is based on principles (which are basically founded on philosophical assumptions), it entails a number of problems like those mentioned under 1), it is implemented by certain practices (e.g., statistical methods), and there are interesting prospects to overcome existing difficulties (by changing or extending established ways of thinking). All these aspects are addressed and illustrated in the book by examples, in order to give the reader a sense of the full width of the issues at stake, and how they can be dealt with. We consider the book as both an intellectual invitation to a deep current problem in the sciences and its cultural and ethical implications.
3. The book presents many chapters devoted to a variety of methods and techniques, as well as their epistemic and ontological underpinnings, which have been developed to safeguard reproducible research and curtail deficits and failures. Please could you give us a taster of such a method in which reproducibility can be a valuable tool?
I guess a good taster should concern a development which is intensely in discussion right now: data mining and knowledge discovery in big databases. It is known that the number of spurious correlations explodes with the size of databases from which they are extracted. This is to say that one will identify many patterns that have high statistical significance yet are meaningless or irrelevant.
Several chapters in the book refer to such cases. To check relevant and meaningful patterns, it is necessary to look for substantive rather than statistical significance. In other words, it is important to find ways to move from purely syntactic to semantic kinds of analysis. Needless to say, this is also a key demand in modern information retrieval techniques, such as semantic search algorithms or the semantic web.
4. If there is one piece of information or advice that you would want your reader to take away and remember after reading your book, what would that be?
Science is not a monolithic block of established knowledge and methodological rules. Problems with reproducibility emphasize that we have to continuously reflect our scientific practices when the frontiers of science are extended. And scientific results are in general not context-free and value-free, as the discussion about meaning in 3 may indicate.This entails important messages for good scientific practice and the ethos of reproducibility in modern innovation societies.
5. Who should read the book and why?
The book is a guide for researchers who are interested in the general and overarching questions behind the concept of reproducibility, for active scientists who are confronted with practical reproducibility problems in their everyday work, and for economic stakeholders and political decision makers who need to better understand the challenges of reproducibility. In addition, the book is suitable as an in-depth primer for upper-undergraduate and graduate-level courses in scientific methodology and basic issues in the philosophy and sociology of science from a modern perspective.
6. Why is this book of particular interest now?
Almost all sciences have now recognized the pressing nature of the issue, and science and technology studies have begun to reflect about it intensely.
7. Were there areas of the book that you found more challenging to write, and if so, why?
Each of the six parts of the book is introduced by background information for the chapters that it contains. These background introductions required additional research beyond individual topics, and they indicate where readers can find additional material not covered in the book itself.
8. What is it about reproducibility that fascinates you?
The massively inter- or transdisciplinary scope of the problem is very special, and the way in which issues raised in different scientific disciplines can cross-fertilize progress. The related difficulties can be seen in different terminologies and different ways to assess problems, which was particularly evident at various workshops that we organized to select and prepare the contents of the book. The whole complex of issues with reproducibility, within and beyond their scientific focus, can hardly be understood without involving philosophy and sociology of science.
9. What will be your next book-length undertaking?
Two other books of mine are in production right now. One of them is a collection of essays resulting from a symposium about the legacy of Hans Primas, a scientist with very broad interests who died recently. The other one is an edited version of a manuscript that he left, the title will be “Knowledge and Time”.
10. Please could you tell us about your educational background and what inspired you to pursue your career in your discipline?
My background is in physics, more precisely the theory of complex dynamical systems. I have never found delight in designing a cliché carrier for myself, with its unfavourable side effects (in my opinion), but rather worked on problems that caught my interest. It’s as simple as this. I always tried to stay away (as much as possible) from the seductions and temptations of the celebrity shows and business manoeuvres influence much of contemporary academic life.
11. Please could you tell us more about your current research interest? What are you working on currently? What are your main objectives and what do you hope to achieve through the results?
In addition to dynamical systems theory, I have been working on conceptual and theoretical aspects of algebraic quantum theory, and I am doing research on selected topics within the wide scope of mind-matter relations. Together with colleagues worldwide, I belong to a group of scientists who started applying mathematical structures from quantum theory outside physics, mainly in cognitive science.
This has turned out to be a pretty successful new direction of research, with much recent progress and corresponding momentum. Another focus of my research is on relations between levels of description in science, where we established and applied a concept that is now known as “contextual emergence”. My primary driving force in science has been insight and understanding rather than media attention, ratings, and rankings, though this may appear somewhat old-fashioned in times where the latter pervade science in general and science funding in particular.
12. Are there people that have been influential in your career?
Yes, and most influential have been those who had the format to give their students and collaborators the freedom to think themselves and encourage them to find and follow their own path. I have been lucky to work with two or three of those, and am grateful for this.