Reproducibility and Replicability have meant different things to different disciplines. While most individuals are aware that disciplines have identified issues in the areas of reproducibility and replicability, the disciplines themselves have not been in agreement regarding the common description and definition of these two terms. If you want to start a heated argument, add repeatability to the list!
For a brief discussion of this topic, please see: Plesser, Hans E. "Reproducibility vs. Replicability: A Brief History of a Confused Terminology," in Frontiers of Neuroinformatics, 18 January 2018. https://doi.org/10.3389/fninf.2017.00076
Working definitions are as follows though we recognize that the terms have been used interchangeably among disciplines.
Of course, "sameness" of procedures and results can be further defined in different disciplines. For example...
And reproducibility can come in a number of flavors -- Computational reproducibility and transparency, Scientific reproducibility and transparency, Computational correctness and evidence, and Statistical reproducibility
Reproducibility and replicability do not refer to one standard set of guidelines but rather are set through individual research communities and organizational standards.
Methods that support these types of activities are considered part of Open Science.
Kitzes, Justin, Daniel Turek, and Fatma Deniz. 2018. The practice of reproducible research: case studies and lessons from the data-intensive sciences. Oakland, CA: University of California suggests a link between early and current research practice.
Since the 17th century, budding scientists have been trained in the scientific method consisting in systematic observation, measurement, and experiment, and the formulation, testing, and modification of hypotheses. This helps to standardize methodologies and communication between researchers and research groups.
Journal articles have long been an expression of the results of research. However with the rise of the Internet, sharing of data and code have assumed greater potential for supporting the goals of scientific method. In an extension of the Kitzes' introduction, the goals of reproducibility and replicability take “ the basic principles of the scientific method that you learned at the lab bench and translate them to your laptop.” ( p. xxii)
For a brief article on scientific method, please see.BROAD, C. Francis Bacon and Scientific Method. Nature 118, 523–524 (1926). https://doi.org/10.1038/118523a0
Open science typically refers to the process of conducting science with recognition that this process is often collaborative in nature. In addition, there is also a focus on the need for research communication. Open scholarship is a broader term encompassing Open Science.
What we know today as open science comprises both principles (transparency, reuse, participation, accountability, etc.) and practices (open publications, data-sharing, citizen science, etc.)
Open science is an ambitious goal that aims to ensure the availability and usability of scholarly publications, the data that result from scholarly research, and the methodology, including code or algorithms, that was used to generate those data.
While complete openness is ambitious, there are many methods and tools that can be employed that can make your research more open.
For more details on Open Science Practices please see the Open Science Training Handbook
In recent years there have been many strides made to promote reproducibility and replicability practices. New tools to assist in the process are developed practically every day.
Why might you want to adapt some of these practices?
It is important to note that there are barriers to full reproducibility and replicability. By acknowledging these barriers, we can better make informed decisions that move our work toward openness.