Responsible Conduct of Research & Scholarly Activity

One of the pathways by which the scientific community confirms the validity of a new scientific discovery is by repeating the research that produced it. When a scientific effort fails to independently confirm the computations or results of a previous study, some fear that it may be a symptom of a lack of rigor in science, while others argue that such an observed inconsistency can be an important precursor to new discovery. Concerns about reproducibility and replicability have been expressed in both scientific and popular media. As these concerns came to light, Congress requested that the National Academies of Sciences, Engineering, and Medicine conduct a study to assess the extent of issues related to reproducibility and replicability and to offer recommendations for improving rigor and transparency in scientific research. Reproducibility and Replicability in Science defines reproducibility and replicability and examines the factors that may lead to non-reproducibility and non-replicability in research. Unlike the typical expectation of reproducibility between two computations, expectations about replicability are more nuanced, and in some cases a lack of replicability can aid the process of scientific discovery. This report provides recommendations to researchers, academic institutions, journals, and funders on steps they can take to improve reproducibility and replicability in science.

Why psychology is in peril as a scientific discipline--and how to save it Psychological science has made extraordinary discoveries about the human mind, but can we trust everything its practitioners are telling us? In recent years, it has become increasingly apparent that a lot of research in psychology is based on weak evidence, questionable practices, and sometimes even fraud. The Seven Deadly Sins of Psychology diagnoses the ills besetting the discipline today and proposes sensible, practical solutions to ensure that it remains a legitimate and reliable science in the years ahead. In this unflinchingly candid manifesto, Chris Chambers draws on his own experiences as a working scientist to reveal a dark side to psychology that few of us ever see. Using the seven deadly sins as a metaphor, he shows how practitioners are vulnerable to powerful biases that undercut the scientific method, how they routinely torture data until it produces outcomes that can be published in prestigious journals, and how studies are much less reliable than advertised. He reveals how a culture of secrecy denies the public and other researchers access to the results of psychology experiments, how fraudulent academics can operate with impunity, and how an obsession with bean counting creates perverse incentives for academics. Left unchecked, these problems threaten the very future of psychology as a science--but help is here. Outlining a core set of best practices that can be applied across the sciences, Chambers demonstrates how all these sins can be corrected by embracing open science, an emerging philosophy that seeks to make research and its outcomes as transparent as possible.

Named by Amazon as one of the "Best Nonfiction Books of the Month" An award-winning science journalist pulls the alarm on the dysfunction plaguing scientific research--with lethal consequences for us all American taxpayers spend $30 billion annually funding biomedical research. By some estimates, half of the results from these studies can't be replicated elsewhere-the science is simply wrong. Often, research institutes and academia emphasize publishing results over getting the right answers, incentivizing poor experimental design, improper methods, and sloppy statistics. Bad science doesn't just hold back medical progress, it can sign the equivalent of a death sentence. How are those with breast cancer helped when the cell on which 900 papers are based turns out not to be a breast cancer cell at all? How effective could a new treatment for ALS be when it failed to cure even the mice it was initially tested on? In Rigor Mortis, award-winning science journalist Richard F. Harris reveals these urgent issues with vivid anecdotes, personal stories, and interviews with the nation's top biomedical researchers. We need to fix our dysfunctional biomedical system-now.

2017 PROSE Award Honorable Mention The PROSE Awards draw attention to pioneering works of research and for contributions to the conception, production, and design of landmark works in their fields. Featuring peer-reviewed contributions from noted experts in their fields of research, Reproducibility: Principles, Problems, Practices, and Prospects presents state-of-the-art approaches to reproducibility, the gold standard of sound science, from multi- and interdisciplinary perspectives. Including comprehensive coverage for implementing and reflecting the norm of reproducibility in various pertinent fields of research, the book focuses on how the reproducibility of results is applied, how it may be limited, and how such limitations can be understood or even controlled in the natural sciences, computational sciences, life sciences, social sciences, and studies of science and technology. The book presents many chapters devoted to a variety of methods and techniques, as well as their epistemic and ontological underpinnings, which have been developed to safeguard reproducible research and curtail deficits and failures. The book also investigates the political, historical, and social practices that underlie reproducible research in contemporary science studies, including the difficulties of good scientific practice and the ethos of reproducibility in modern innovation societies. Reproducibility: Principles, Problems, Practices, and Prospects is a guide for researchers who are interested in the general and overarching questions behind the concept of reproducibility; for active scientists who are confronted with practical reproducibility problems in their everyday work; and for economic stakeholders and political decision makers who need to better understand the challenges of reproducibility. In addition, the book is a useful in-depth primer for undergraduate and graduate-level courses in scientific methodology and basic issues in the philosophy and sociology of science from a modern perspective. "A comprehensive, insightful treatment of the reproducibility challenges facing science today and of ways in which the scientific community can address them." Kathleen Hall Jamieson, Elizabeth Ware Packard Professor of Communication, University of Pennsylvania "How can we make sure that reproducible research remains a key imperative of scientific communication under increasing commercialization, media attention, and publication pressure? This handbook offers the first interdisciplinary and fundamental treatment of this important question."Torsten Hothorn, Professor of Biostatistics, University of Zurich Harald Atmanspacher, PhD, is Associate Fellow and staff member at Collegium Helveticum, ETH and University Zurich and is also President of the Society for Mind-Matter Research.  He has pioneered advances in complex dynamical systems research and in a number of topics concerned with the relation between the mental and physical. Sabine Maasen, PhD, is Professor for Sociology of Science and Director of the Munich Center for Technology in Society (TU Munich) and Associate Fellow at Collegium Helveticum (ETH and University Zurich).  Her research focuses on the interface of science, technology, and society, notably with respect to neuroscience and its applications. 

In computational science, reproducibility requires that researchers make code and data available to others so that the data can be analyzed in a similar manner as in the original publication. Code must be available to be distributed, data must be accessible in a readable format, and a platform must be available for widely distributing the data and code. In addition, both data and code need to be licensed permissively enough so that others can reproduce the work without a substantial legal burden. Implementing Reproducible Researchcovers many of the elements necessary for conducting and distributing reproducible research. It explains how to accurately reproduce a scientific result. Divided into three parts, the book discusses the tools, practices, and dissemination platforms for ensuring reproducibility in computational science. It describes: Computational tools, such as Sweave, knitr, VisTrails, Sumatra, CDE, and the Declaratron system Open source practices, good programming practices, trends in open science, and the role of cloud computing in reproducible research Software and methodological platforms, including open source software packages, RunMyCode platform, and open access journals Each part presents contributions from leaders who have developed software and other products that have advanced the field. Supplementary material is available at www.ImplementingRR.org.