One reason for the dissonance among disciplines when discussing reproducibility and replicability is that there are a number of ways to discuss and assess the depth of reproducibility that is attained.
Three measures for level of reproducibility are the following:
The level of R and R that you offer is dependent on a number of factors such as the instrumentation that is needed, the availability of software for the types of analyses you need and commonality of tools within your discourse community.
Workflow based systems provide explicit representation of the structure of the experiments, automation of repetitive tasks and computations, and transparently capture provenance information.
Provenance in workflow systems include:
Prospective provenance - description of the experiment workflow structure such as modules, connections, and inputs.
Retrospective provenance – information on the execution of workflow and what happened when it was run.
Workflow evolution –history and versions of workflow (especially when data is iteratively refined)
To capture research provenance, 3 classes of tools need to be install: a shell or terminal program for access to command line, a plain text editor or a development environment and software allowing the user to write and execute code in a chosen programming language
Examples of Workflow Management tools include:
When you preregister your research, you're simply specifying your research plan in advance of your study and submitting it to a registry. In clinical research in particular, the preregistration of a study is mandatory.
Preregistration separates hypothesis-generating (exploratory) from hypothesis-testing (confirmatory) research. Both are important. But the same data cannot be used to generate and test a hypothesis, which can happen unintentionally and reduce the credibility of your results. Addressing this problem through planning improves the quality and transparency of your research. This helps you clearly report your study and helps others who may wish to build on it.
For additional insight and context, you may wish to read The Preregistration Revolution.
Tools for Registration appear below. Please note that OSF provides registration and registed report services. For more information see our guide on the Open Science Framework
Data Cleanup and Processing are key components of replicability and reproducibility.
Here are some best practices:
Best practices for Acquisition of data include:
Electronic Lab Notebooks are frequently used to encourage reproducible data acquisition practices.
File versioning is a great way to maintain order in your research computations as well as to allow for better collaboration. There are many collaborative tools that support file versioning.
Check our guide for information on principles of good file naming, versioning and maintenance.
Code documentation provides human readable elements embedded in your code to allow other users to follow your code.
Many of these documentation tools will also allow you to create runnable code within a text. You can also make complete documents and even books!
Packaging tools allow you to collect multiple files (data, text, etc.) into...well...packages for easier portability. They are very helpful in moving files to and from a computer to a cloud environment.
Some packaging systems are being developed to also provide the appropriate software that is needed to run analyses within the code packages. CodeOcean is one of these systems.