Systematic Reviews

Time

Systematic reviews generally take between 6 to 18 months to complete.

The Cochrane Handbook for Systematic Reviews of Interventions notes that

"The amount of time required will vary, depending on the topic of the review, the number of included studies, the methods used (e.g. the extent of efforts to obtain unpublished information), the experience of the authors, and the types of support provided by the editorial team of the CRG. The workload associated with undertaking a review is thus very variable. However, consideration of the tasks involved and the time required for each of these might help authors to estimate the amount of time that will be required. These tasks include training, meetings, protocol development, searching for studies, assessing citations and full-text reports of studies for eligibility, assessing the risk of bias of included studies, collecting data, pursuing missing data and unpublished studies, analysing the data, interpreting the results, writing the review, and keeping the review up to date."

"II.3.1 Identifying resources and support." Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). Cochrane, 2020. Available from www.training.cochrane.org/handbook

Team

Systematic review teams should include at least two people to screen articles, a researcher familiar with the type of data synthesis to be used, and a researcher with strong database searching skills (like a librarian)

The Cochrane Handbook for Systematic Reviews of Interventions notes that

"Review teams must include expertise in the topic area under review. Topic expertise should not be overly narrow, to ensure that all relevant perspectives are considered. Perspectives from different disciplines can help to avoid assumptions or terminology stemming from an over-reliance on a single discipline. Review teams should also include expertise in systematic review methodology, including statistical expertise."

"1.3 Who should do a systematic review?" Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). Cochrane, 2020. Available from www.training.cochrane.org/handbook

Research Question

Systematic reviews require specific and well-defined questions

Broad or unfocused questions will lead to large and potentially unmanageable reviews. A systematic review question should be an answerable question that contributes new knowledge to a field by synthesizing existing literature.

Cummings, et al, propose that strong research questions will have following characteristics:

"Chapter 2: Conceiving the Research Question and Developing the Study Plan," in Hulley, S. B., Cummings, S. R., Browner, W. S., Grady, D. G., & Newman, T. B. (2013). Designing clinical research.

Written Protocol

A systematic review protocol describes the review question and methodology in detail, providing a road map for your research

Some researchers may choose to submit their protocol to a registry, which serves as a notification to other researchers that your work is in progress and dissuades other researchers from taking on a similar project.

In Systematic Reviews in the Social Sciences: A Practical Guide (2006), authors Mark Petticrew and Helen Roberts state that:

"...a systematic review needs a detailed protocol that describes in advance the process and methods that will be applied. This protocol includes a description of and rationale for the review question, and the proposed methods, and includes details of how different types of study will be located, appraised, and synthesized."

Petticrew, M. & Roberts, H. (2006) Systematic reviews in the social sciences: a practical guide (p.44). Blackwell Pub.

Search Expertise

While searching for studies might seem like an easy task, it requires a certain expertise and experience to do well

Systematic reviews require a clear and replicable search strategy. Multiple relevant databases should be searched. The goal of a systematic review search should be to return all relevant results and as few irrelevant results as possible.

The Cochrane Handbook for Systematic Reviews of Interventions states

"Systematic reviews require a thorough, objective and reproducible search of a range of sources to identify as many eligible studies as possible (within resource limits). This is a major factor distinguishing systematic reviews from traditional narrative reviews, which helps to minimize bias and achieve more reliable estimates of effects and uncertainties. ... Time and budget restraints require the review team to balance the thoroughness of the search with efficiency in the use of time and funds. The best way of achieving this balance is to be aware of, and try to minimize, the biases such as publication bias and language bias that can result from restricting searches in different ways... It is strongly recommended, however, that all search strategies should be peer reviewed by a suitably qualified and experienced medical/healthcare librarian or information specialist."

"4.2.2 Minimizing bias," Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). Cochrane, 2020. Available from www.training.cochrane.org/handbook

Citation Management & Search Documentation

Systematic review searches often retrieve hundreds if not thousands of studies; researchers should have a plan for managing this data and documenting their search process

Several citation management software tools are available to researchers than can help make the process of collecting, organizing, and deduplicating citations easier. Some registries and publishers will require the use of a specific documentation format or form.

Study Selection and Appraisal Guidelines

Retrieved studies must be screened set inclusion and exclusion criteria and appraised based on pre-determined characteristics

Many appraisal checklists and tools are available for you to use or reference. Studies should be appraised and screened by at least two independent reviewers.

Reporting Guidelines

Publishing venues often have specific guidelines for systematic review manuscripts

Researchers should be aware of any guidelines or checklists before beginning the review process.