Ten essential steps for performing a systematic review: A quick tutorial

Liang-Tseng Kuo; Shih-Chieh Shao; Ching-Chi Chi

doi:10.4103/1027-8117.362992

REVIEW ARTICLE

Year : 2022 | Volume : 40 | Issue : 4 | Page : 204-206

Ten essential steps for performing a systematic review: A quick tutorial

Liang-Tseng Kuo¹, Shih-Chieh Shao², Ching-Chi Chi³
¹ Division of Sports Medicine, Department of Orthopedic Surgery, Chang Gung Memorial Hospital, Chiayi; School of Medicine, College of Medicine, Chang Gung University, Taoyuan, Taiwan
² Department of Pharmacy, Chang Gung Memorial Hospital, Keelung, Taiwan
³ School of Medicine, College of Medicine, Chang Gung University; Department of Dermatology, Chang Gung Memorial Hospital, Linkou, Taoyuan, Taiwan

Date of Submission	06-Sep-2022
Date of Decision	06-Nov-2022
Date of Acceptance	15-Nov-2022
Date of Web Publication	08-Dec-2022

Correspondence Address:
Prof. Ching-Chi Chi
Department of Dermatology, Chang Gung Memorial Hospital, Linkou, No. 5, Fuxing St., Guishan Dist., Taoyuan
Taiwan

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/1027-8117.362992

Abstract

In recent decades, the volume of health publications has increased. A well-conducted systematic review (SR), with or without a meta-analysis, is considered the best reference for health care professionals to catch up with the best evidence. With the rapid increase in the number of published SRs, their study quality varies despite well established reporting guidelines and references. Understanding the process of producing SRs is critical to the execution and is not trivial due to the obstacles researchers may encounter. The aim of this article was to provide a step-by-step approach to help researchers (mostly novices) keep their SR up to standard. The steps presented here are a quick tutorial with reference to known and accepted international guidances and our expertise.

Keywords: Evidence-based medicine, meta-analysis, systematic review

How to cite this article:
Kuo LT, Shao SC, Chi CC. Ten essential steps for performing a systematic review: A quick tutorial. Dermatol Sin 2022;40:204-6

How to cite this URL:
Kuo LT, Shao SC, Chi CC. Ten essential steps for performing a systematic review: A quick tutorial. Dermatol Sin [serial online] 2022 [cited 2023 Mar 21];40:204-6. Available from: https://www.dermsinica.org/text.asp?2022/40/4/204/362992

Introduction

With the increased attention to evidence-based health care,^[1] the number of published systematic reviews (SRs) has been rapidly growing in recent years.^[2] The quality of SRs varies despite the existence of reporting guidelines and references, including the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guideline^[3] and the Cochrane Handbook for SRs of interventions.^[4]

Herein we summarize the essential steps in performing an up-to-standard SR. Relevant examples of SRs are cited.

Identify and Formulate your Review Question (PICOS)

The review question should be formulated in a clear and structured format at the beginning stage of a SR. The review question should include patients or problem (P), intervention (I), comparison intervention (C), and outcomes (O). In addition, the type of study design (S) for the review question should also be indicated. For example, the review question on treatment effects usually relies on the summarized evidence of randomized controlled trials (RCTs). Furthermore, this step should also prespecify the inclusion and exclusion criteria for selecting studies.^[5],[6],[7]

To increase research transparency, registering the study protocol is highly suggested before performing a SR. The PROSPERO is the most widely used platform for registering SRs.^[8]

Evidence Searches

After devising a review question, the authors should develop a search strategy along with experienced information specialists, such as medical librarians, to identify potential studies from the following resources:

Electronic databases: Searching of at least three databases (e.g., PubMed or MEDLINE, Embase, and Cochrane Central Register of Controlled Trials [CENTRAL]) are suggested.^[9] Furthermore, some speciality databases should be searched for specific topics (e.g., PsycINFO for psychiatry-related review questions). Restrictions on language or geographic regions should be avoided.^[10]
Snowballing: Scanning the references lists of included studies for potentially eligible studies is suggested.^[11],[12]
Handsearching: Scanning the conference abstracts of top specialty meetings may help to find the latest relevant studies.^[13]
Unpublished or ongoing studies: Consulting the clinical experts and searching trial registers (e.g., clinicaltrial.gov) are suggested.^[14],[15]

Study Selection

After merging the search results from various data sources, review authors identify eligible studies based on the inclusion and exclusion criteria step by step:^[16]

Removal of duplicate records.
Scanning the titles and abstracts of the records and identifying potentially eligible studies.
Obtaining and reading the full texts of potential studies, and confirming the included studies.

Two review authors should independently perform study selection, and a third review author should be consulted if the disagreement cannot be resolved by discussion.^[17]

Data extraction from Included Studies

In this step, the review authors can use a prespecified sheet to extract the data from the included studies.^[18] Usually, the essential items include details of the study (e.g., first author name and publication year), patient characteristics (e.g., mean age, proportion of sex, and important comorbidity), interventions (e.g., dose and frequency), comparisons, and outcomes of the included studies.^[19]

Critical Appraisal of the Included Studies (Risk-of-Bias Assessment)

Two review authors should evaluate the validity of included studies using appropriate tools. For example, the Cochrane Risk of Bias 2 tool^[20] is the preferred tool for RCTs, and risk of bias in non-randomised studies - of interventions (ROBINS-I)^[21] is the preferred tool to be used in Cochrane reviews for nonrandomized studies of the intervention. Any disagreements should be resolved by discussion,^[22] or a third review author should be consulted.^[23]

Data Synthesis (Meta-Analysis)

Before performing this step, review authors should judge if synthesizing the outcomes of included studies is appropriate. For example, if the clinical diversity (heterogeneity) of included studies is too large to be accepted, a meta-analysis (MA) should not be performed.^[13]

If the review authors decide not to do a MA, they should summarize the outcomes of included studies. If the review authors choose to synthesize the results of the included studies, two statistical models are available (fixed-effect and random-effects models). The choice of statistical model usually depends on the clinical heterogeneity (e.g., the similarity of PICOS details in the included studies) rather than statistical heterogeneity (e.g., the similarity of point estimates with 95% confidence intervals [CIs] in the included studies).^[4]

Present and Interpret the Data

The forest plot is usually used for presenting the synthesized outcomes.^[24] The following items should be reported:

Point estimate and 95% CI for the pooled effects.
Degree of statistical heterogeneity (e.g. Cochran's Q test, I² statistics) among the included studies.^[25]

Investigate the Heterogeneity

For considerable heterogeneity, additional analyses may be needed. For example, review authors could conduct subgroup or sensitivity analyses to investigate the potential sources of heterogeneity.^{[26],[27],[28]} Meta-regression is another statistical approach to explore potential heterogeneity.^[29] However, the heterogeneity is seldom confirmed by meta-regression unless a sufficient number of studies are available.

Judge the Certainty of Evidence

Two review authors are suggested to follow the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) methodology to independently assess the certainty of evidence for each pooled outcome as “very low,” “low,” “moderate,” or “high.”^[30],[31] This method judges the overall certainty of the evidence for each outcome by considering the risk in overall risk of bias,^[32] imprecision,^[33] inconsistency,^[34] indirectness,^[35] and publication bias.^[36] Disagreements will be solved by discussion. If arguments still exist after discussion, a third author should arbitrate.^[17]

In summary, following the above 10 essential steps would ensure an SR meets the general requirements. Further tailoring based on the journal's instructions for authors is still needed before submitting a manuscript.

Financial support and sponsorship

Nil.

Conflicts of interest

Prof. Ching-Chi Chi, the Editor-in-Chief of Dermatologica Sinica, had no role in the peer review process of or decision to publish this article. The other authors declared no conflicts of interest in writing this paper.

References

1.	Chi CC. Evidence-based dermatology. Dermatol Sin 2013;31:2-6.
2.	Yiu ZZ, Chi CC, Ingram JR, Flohr C. Checking for update living systematic reviews and clinical practice guidelines in the BJD. Br J Dermatol 2022;186:761-2.
3.	Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021;372:n71.
4.	Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane Handbook for Systematic Reviews of Interventions. Chichester: John Wiley & Sons; 2019.
5.	Fang TP, Lin YC, Chi CC. Association of psoriasis with asthma: A systematic review and meta-analysis of observational studies. Dermatol Sin 2020;38:22-7. [Full text]
6.	Yang SF, Chen TH, Tsai SH, Chen PE, Chi CC, Tung TH. Risk of chronic kidney disease and end-stage renal disease in patients with psoriasis: A systematic review and meta-analysis of cohort studies. Dermatol Sin 2021;39:19-26. [Full text]
7.	Shao SC, Lai CC, Chen YH, Lai EC, Hung MJ, Chi CC. Associations of thiazide use with skin cancers: A systematic review and meta-analysis. BMC Med 2022;20:228.
8.	Wang FY, Chi CC. Association of rosacea with inflammatory bowel disease: A MOOSE-compliant meta-analysis. Medicine (Baltimore) 2019;98:e16448.
9.	Wang CH, Fu Y, Chi CC. Association of atopic dermatitis with inflammatory bowel disease: A systematic review and meta-analysis. Dermatol Sin 2020;38:159-65. [Full text]
10.	Hung WK, Tung TH, Wang TY, Liao SC, Chi CC. Risk for incident suicidality among psoriasis patients: A systematic review and meta-analysis. Arch Dermatol Res 2022;10.1007/s00403-022-02377-5. doi:10.1007/s00403-022-02377-5.
11.	Tan CW, Hsu WH, Yu PA, Chen CL, Kuo LT, Chi CC, et al. Anterior cruciate ligament reconstrucetion in patients older than 50 years: A systematic review and meta-analysis. Orthop J Sports Med 2020;8:2325967120915698.
12.	Chi CC, Wang SH, Kirtschig G. Safety of topical corticosteroids in pregnancy. JAMA Dermatol 2016;152:934-5.
13.	Chi CC, Lee CW, Wojnarowska F, Kirtschig G. Safety of topical corticosteroids in pregnancy. Cochrane Database Syst Rev 2009;(3):CD007346. Published 2009 Jul 8. doi:10.1002/14651858.CD007346.pub2.
14.	Kuo LT, Chen CL, Yu PA, Tsai YS, Hsu WH, Chi CC, et al. Bone marrow-stimulating techniques in arthroscopic rotator cuff repair: A systematic review protocol. BMJ Open 2018;8:e022086.
15.	Kuo LT, Chen CL, Yu PA, Hsu WH, Chi CC, Yoo JC. Epinephrine in irrigation fluid for visual clarity in arthroscopic shoulder surgery: A systematic review and meta-analysis. Int Orthop 2018;42:2881-9.
16.	Lin PT, Wang SH, Chi CC. Low molecular weight heparin for prevention of microvascular occlusion in digital replantation. Cochrane Database Syst Rev 2020;4:CD009894.
17.	Lin HS, Lin PT, Tsai YS, Wang SH, Chi CC. Interventions for bacterial folliculitis and boils (furuncles and carbuncles). Cochrane Database Syst Rev 2021;2:CD013099.
18.	Kuo LT, Hsu WH, Chi CC, Yoo JC. Tranexamic acid in total shoulder arthroplasty and reverse shoulder arthroplasty: A systematic review and meta-analysis. BMC Musculoskelet Disord 2018;19:60.
19.	Lee CH, Fu Y, Yang SJ, Chi CC. Effects of omega-3 polyunsaturated fatty acid supplementation on non-alcoholic fatty liver: A systematic review and meta-analysis. Nutrients 2020;12:2769.
20.	Sterne JA, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: A revised tool for assessing risk of bias in randomised trials. BMJ 2019;366:l4898.
21.	Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: A tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016;355:i4919.
22.	Chen WT, Chi CC. Association of hidradenitis suppurativa with inflammatory bowel disease: A systematic review and meta-analysis. JAMA Dermatol 2019;155:1022-7.
23.	Ko SH, Chi CC, Yeh ML, Wang SH, Tsai YS, Hsu MY. Lifestyle changes for treating psoriasis. Cochrane Database Syst Rev 2019;7:CD011972.
24.	Lien KH, Ger TY, Chi CC. Association of vitiligo with high-frequency sensorineural hearing loss: A systematic review and meta-analysis. J Eur Acad Dermatol Venereol 2022;36:373-9.
25.	Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med 2002;21:1539-58.
26.	Fu Y, Lee CH, Chi CC. Association of psoriasis with colorectal cancer. J Am Acad Dermatol 2021;85:1429-36.
27.	Chen TL, Lee LL, Huang HK, Wang JH, Chen LY, Tsai HR, et al. Association of psoriasis with incident venous thromboembolism and peripheral vascular disease: A systematic review and meta-analysis. JAMA Dermatol 2022;158:59-67.
28.	Kuo LT, Chi CC, Chuang CH. Surgical interventions for treating distal tibial metaphyseal fractures in adults. Cochrane Database Syst Rev 2015;2015:CD010261.
29.	Yu CL, Chou PY, Liang CS, Tu YK, Chi CC. Interventions for molluscum contagiosum: A systematic review and network meta-analysis with normalized entropy assessment. J Am Acad Dermatol 2022;S0190-9622(22)02349-0. doi:10.1016/j.jaad.2022.07.022.
30.	Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008;336:924-6.
31.	Chi CC, Chen TH, Wang SH, Tung TH. Risk of suicidality in people with psoriasis: A systematic review and meta-analysis of cohort studies. Am J Clin Dermatol 2017;18:621-7.
32.	Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso-Coello P, et al. GRADE guidelines: 4. Rating the quality of evidence – Study limitations (risk of bias). J Clin Epidemiol 2011;64:407-15.
33.	Guyatt GH, Oxman AD, Kunz R, Brozek J, Alonso-Coello P, Rind D, et al. GRADE guidelines 6. Rating the quality of evidence – Imprecision. J Clin Epidemiol 2011;64:1283-93.
34.	Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 7. Rating the quality of evidence – Inconsistency. J Clin Epidemiol 2011;64:1294-302.
35.	Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 8. Rating the quality of evidence – Indirectness. J Clin Epidemiol 2011;64:1303-10.
36.	Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, et al. GRADE guidelines: 5. Rating the quality of evidence – Publication bias. J Clin Epidemiol 2011;64:1277-82.