The rationale behind systematic reviews in clinical medicine: a conceptual framework

Abstract

A systematic review (SR) is a type of review that uses a systematic method to provide a valid summary of existing literature addressing a clear and specific question. In clinical medicine (CM), the concept of SR is well recognized, especially after the introduction of evidence-based medicine; The SR of randomized clinical trials (RCTs) is considered the highest level of evidence on therapeutic effectiveness. Despite the popularity of the SRs and the increasing publication rate of SRs in CM and other healthcare literature, the concept has raised criticisms. Many of proper criticisms can be due to the deviation of some existing SRs from the original philosophy and well-established rationale behind the concept of SR. On the other hand, many criticisms are misconceptions about SRs which still exist even several decades after introducing the concept. This article presents a conceptual framework for clarifying the rationale behind SR in CM by providing the relevant concepts and their inter-relations, explaining how methodological standards of an SR and its rationale are connected, and discussing the rationale under the three-section: SR as a type of synthetic research, SR as a more informed and less biased review, and SR as an efficient scientific tool.

Background

A systematic review (SR) provides an accurate summary of studies addressing a clear and specific question through an objective and explicit method employed in the whole research process. A meta-analysis (MA) refers to both the statistical procedure that combines multiple studies’ data and an SR that includes this procedure [1].

A typical SR follows these steps: (i) determining a specific and often narrowed-down review question, a search strategy, inclusion and exclusion criteria, a synthesis approach, and preparing a structured protocol; (ii) conducting a systematic search to retrieve original studies; (iii) screening the retrieved studies by checking their titles, abstracts and then full texts, as well as using predefined inclusion and exclusion criteria; (iv) critically appraising the included studies and characterizing them by certain features (e.g., research design); (v) extracting the relevant quantitative and/or qualitative data; (vi) synthesizing the data. Subsequently, depending on the question and the available literature, and also the nature of the data, MA can be employed [1].

In clinical medicine (CM), the concept of SR is well recognized, especially after introducing evidence-based medicine (EBM); The SR of randomized clinical trials (RCTs) is considered the highest level of evidence on the effectiveness of treatment options. Moreover, the recommendations provided by SRs for clinical practice as well as the degree of factual uncertainty (about those recommendations) determined by SRs are valuable for both implementation in clinical practice as well as for future research to address those factual uncertainties [2].

The terms “systematic review” and “meta-analysis” are extensively used in the current healthcare literature; and, the publication rate of SRs has increasingly grown in recent decades. According to Ioannidis et al., among PubMed-indexed articles between 1986 and 2015, 266,782 records were tagged as “systematic review” and 58,611 records as “systematic review and meta-analysis” [3]. Moreover, a comparison between the articles published in 1991 and those published in 2014 demonstrates an increase in the publication rate of SRs and MAs by 2728% and 2635%, respectively; while this rate is only 153% for all other publication types [3].

On the other hand, some criticisms have been made about the concept of SR [3–7]. These criticisms fall into two main categories. The first category includes the proper criticisms. For instance, Ioannidis et al. discussed that despite the increasing number of SRs in clinical biomedicine, many SRs have limitations such as little value, wide redundancy, misleading claims, and/or vested interests. They have argued that many authors consider SR as a straightforward way to publish highly cited articles, rather than to clarify the evidence to promote evidence-based practice [3]. In this category, the criticisms are centered on how existing SRs have deviated from the original philosophy and rationale behind the concept of SR. Another category includes fallacious and ill-founded criticisms about SRs which are in fact misconceptions about SR, as Petticrew discussed [7]. For example, SRs have been criticized that they cannot provide easily digestible summaries of the evidence for decision-makers (while it is a common misconception as discussed later) [8].

Moreover, other areas of healthcare research (e.g., public health, medical education, health policy, and even biomedical ethics as a discipline mainly based on philosophical theories) have gradually initiated to develop their customized conceptions of SR. Although they have adjusted the original concept for their fields, these new conceptualizations still originate from the SR concept in CM [9–13]. Consequently, misconceptions about SR in CM could be transferred to these areas of research.

Therefore, critical reflection on the rationale behind the concept of SR in CM and its misconceptions can be valuable, yet again. Furthermore, despite the long establishment of the rationale behind SR in the literature such as Petticrew’s or Mulrow’s work [5, 14], an overall picture of ideas that are behind developing SR is needed to clarify the ideas and facilitate the understanding of the rationale behind SR especially in CM. Providing a conceptual framework, the value added by this review article to the current literature on the rationale behind SR, is useful to organize ideas, make conceptual distinctions, and map out concepts and the presumed relations among them.

The article can serve as a useful reference for learning and reflecting on the concept of SR in CM as defining the concept of SR, reviewing the history of the concept, clarifying the rationale behind SR through a conceptual framework, and reviewing the common misconceptions about SR in CM, all together. As conceptual maps are employed in medical education to organize and present the knowledge as well as promote meaningful learning [15], this article provides a conceptual map that is useful to teach and learn the above-mentioned concepts.

The history of systematic reviews

In the 1930s, the term “systematic review” was originally introduced in social sciences to summarize the previous studies conducted for testing a hypothesis. Gradually, new elements are added to the concept of SR; the reviewers started to appraise the results obtained through the literature search to provide an answer to a specific question; Statistical pooling of the individual studies’ results to synthesize a pooled result (i.e., MA) was initially introduced in the late 1970s. Then, healthcare systems started to employ this method. Cochrane Collaboration and the Centre for Reviews and Dissemination (CRD) were established in the UK in 1993 and 1994, respectively [5]. These pioneers aimed to promote evidence-based practice via preparing, maintaining, and disseminating SRs addressing the efficacy of various healthcare interventions (i.e., SRs of RCTs) [1].

However, the concept of SR was not exclusive to RCTs. SRs gradually appeared in other areas of science (e.g., public health) as well as for various clinical questions. For example, six different types of Cochrane reviews have been introduced so far: (i) Intervention reviews, evaluating the harms and benefits of the interventions used in healthcare and health policy; (ii) Diagnostic test accuracy reviews, evaluating the diagnostic value of diagnostic tests in the detection of a particular disease; (iii) Methodology reviews, addressing how SRs and RCTs are performed and how their results are reported; (iv) Qualitative reviews, addressing questions on the aspects other than the effectiveness by reviewing and synthesizing both qualitative and quantitative studies; (v) Prognosis reviews, addressing the probable natural course of patients with a particular disease; and (vi) Overviews of the Reviews (OoRs), compiling from the results of several SRs into a single article to be accessible, easily understandable, and useful for facilitating healthcare decision-makings. The Cochrane Library includes both Cochrane SRs and non-Cochrane SRs in The Cochrane Database of Systematic Reviews (CDSR) and CRD databases, respectively [1].

Over time, other institutes have gradually joined the research synthesis movement in healthcare including SRs and MAs, promoted the relevant methods, published and collected SRs on various subjects (e.g., Joanna Briggs Institute (JBI) and the National Institute for Health and Care Excellence (NICE)) [16, 17].

The importance of SRs in healthcare and the rapid expansion of biomedical literature, as well as the demanding and time-consuming processes of generating the SRs, require the assistance of information technology advances. Currently, various platforms are available to help researchers conduct an SR (e.g., Review Manager (Rev Man)) [18]. Since 2015, the International Collaboration for the Automation of Systematic Reviews (ICASR) has initiated activities to employ the recent advances in natural language processing, data mining (especially text mining), and machine learning to produce algorithms that are capable of a more accurate, rapid, and low-cost simulation of human work in SRs. The vision that the collaboration attempts to turn into reality is to develop such technology enabling the researchers to edit, test, and then implement an SR protocol “at the push of a button” [19].

The rationale behind SR concept in CM

Valuable works such as Petticrew’s or Mulrow’s [5, 14] have discussed the rationale behind SR in CM including the underlying reasons for conducting SRs. We reviewed and classified these underlying reasons under three categories for more clarification (Table 1): SR as a type of synthetic research, SR as a more informed and less biased review, and SR as an efficient scientific tool. These categories are considered as the core concepts in our conceptual map that illustrating these categories, the underlying reasons for conducting SRs, the relevant concepts, and their inter-relations (Fig. 1). In the following sections, we have discussed these categories, respectively.

Table 1.

A Classification of Underlying Reasons Clarifying the Rationale behind Systematic Review (SR) in Clinical Medicine

Category	Underlying Reasons
A. SR as a type of synthetic research	1) to summarize and integrate original knowledge
	2) to address a clearly formulated question
B. SR as the more informed and less biased review	3) to increase the external validity of findings
	4) to increase the explanatory power of findings
	5) to increase the precision of estimated values
	6) to increase the statistical power to detect true associations
	7) to increase the internal validity or accuracy of findings
	8) to increase the reliability of findings
C. SR as a scientific efficient method	9) to implement the current best evidence less costly
	10) to implement the current best evidence more rapidly

Fig. 1.

The Rationale behind Systematic Review in Clinical Medicine: a Conceptual Map. Yellow and dark blue boxes demonstrate the underlying reasons behind systematic review (SR) in clinical medicine and three arbitrary categories to clarify them, respectively. Grey boxes show strict high standards for SR. Light blue boxes refer to other relevant concepts for clarifying the rationale behind SR in clinical medicine

A. SR as a type of synthetic research

The first underlying reason for conducting SRs in CM (Table 1), which is common to other types of synthetic research, is:

• The first reason: to summarize and integrate original knowledge

To explain this underlying reason, we will discuss three topics: the necessity of synthetic research, SR versus other review types, and SR versus scientometric studies (Fig. 1).

I . The Necessity of Synthetic Research

In recent decades, rapid advances in technologies have led to an increased rate of publications called “information explosion”. Despite the positive sides of information explosion, managing, searching, and employing this produced information have become more challenging, referred to as “challenges of information overload”. For instance, the doubling time of medical knowledge was 50, 7, and 3.5 years in 1950, 1980, and 2010, respectively; and it is estimated to be 0.2 years or about 2.5 months in 2020 [20]. Also, decision-makers, including physicians, are expected to be evidence-based which means to employ the current best knowledge in their practice. Therefore, physicians have two responsibilities: (i) searching the literature to find the best answers to the questions arising from their daily practice, and (ii) keeping themselves updated with the latest literature by continuously acquiring the newly-produced knowledge in their field. However, both responsibilities need more time and expertise in searching the information. For example, a study on family physicians reported that for every 10 visited patients, around 3.2 new questions arose that, 64% of them remained unanswered [21].

Moreover, the problem is not only the information overload but is also the “pointless publications” and “inconsistencies” among studies (e.g., one study emphasizes the effectiveness of a medication, another on the ineffectiveness of the same medication) [5]. As more information is produced and published, decision-makers confront more difficulties in practicing based on the current best knowledge [5].

To resolve these issues, the concept of synthetic research has been introduced. In synthetic studies, researchers identify, evaluate, and summarize the existing literature, mainly original studies focused on a specific subject. This type of research is also known as “desk research” or “secondary research” because it involves gathering existing data previously published by other researchers instead of conducting original research on the study subjects in the real-world (e.g., patients) [5]. SR is a type of synthetic research that summarizes and integrates important and critical pieces of existing information which saves a large amount of time for the decision makers and decreases the confusion among them [22].

II. SR versus Other Review Types

The second underlying reason for conducting SRs in CM (Table 1), that differentiates SR from other types of synthetic research, is:

• The second reason: to address a clearly formulated question

To discuss the rationale behind SR as a type of synthetic research, the distinction of SR from other types of synthetic research (or reviews) is important. SRs have traditionally been compared with a traditional type of synthetic research referred to as narrative review (NR). NRs are also known as authoritative reviews because they are narratively written by experts in specific subjects after reviewing the literature unsystematically, while SRs combine comprehensive search with critical appraisal to address a clearly formulated question in a systematic way. As a result, NRs may draw different conclusions from the same studies. For example, Cipriani and Geddes compared seven NRs and two SRs on the effectiveness of atypical antipsychotics. The NRs that reviewed the same studies reached different conclusions both among themselves and compared to the SRs [23, 24].

Besides, several other review types are also increasingly produced in healthcare literature which their methods and reports are similar to those of SRs but, these reviews are not considered SR as they do not include all components of SR, and also, their rationales are not the same as the rationale behind SR. However, this should not imply that these reviews are not valuable as SRs are because depending on the purpose of the studies and the nature of the questions, these various review types are the choice [25]. To provide a comparative insight, Table 2 are listed twelve main types of reviews (other than SR), as well as the most important rationale behind them.

Table 2.

Main Types of Review Articles (Other Than Systematic Review(SR)) and Rationale behind Them

Literature Review / Narrative Review	Literature review is a generic name for all review activities; however, it is used interchangeably with Narrative Review. Its rationale is to examine the existing literature including a wide range of information in a special area to answer wide questions. It’s usually useful for providing a summary of current knowledge in that area, avoiding duplications, and identifying omissions or gaps, subsequently developing hypotheses for new research studies.
Qualitative Systematic Review/ Qualitative Evidence Synthesis	Its rationale is to seek, summate, compare, and integrate qualitative findings such as themes, concepts or constructs (in contrast with aggregate quantitative findings in meta-analysis), and subsequently to develop a new theory, an overarching narrative, a wider generalization, or an interpretative translation. Qualitative meta-synthesis and meta-ethnography are other names of this review type.
Systematized Review	The rationale behind this type is to provide a foundation for a more extensive piece of work such as a dissertation. Its rationale is in contrast with that of SR providing a more comprehensive and less biased answer to a narrowed-down, specific question. Such a review does not meet the demanding standards of SR because of insufficient resources; however, similar to SRs, it is systematic in the searching process and the reporting form.
Rapid Review	The rationale behind rapid review or rapid evidence assessment is to assess what is already known about a policy or practice issue using several techniques for rapid searching and appraising the existing literature while making a balance between limiting timescale and militating against biases.
Overview	It is a generic term for such review surveying the literature, describing its features although not necessarily systematically. The rationale is to provide a broad and often comprehensive summation of the literature on a topic, valuing especially for those who face that topic for the first time.
Mixed Studies Review/ Mixed Methods Review	The rationale behind this type, occasionally known as realistic review, is usually to address a complex issue of what works under which circumstances. By reviewing quantitative research and qualitative ones, it combines several aspects: (i) explaining what work (e.g., a therapeutic intervention) and under which circumstances (e.g., in a sub-group of patients), and (ii) how and why it works (e.g., attitudes toward that intervention.
State-of-the-art Review	The rationale behind this type is to offer new perspectives on an issue and highlight hot topics. Its audience is usually researchers seeking potential opportunities for future research. Its cross-sectional method is based on the surveying of a field comprehensively within current research.
Critical Review	As its name shows, it goes beyond mere review and integration of existing literature; extensive critical analysis and conceptual innovation are the most important foundations of this type. The emphasis is on the conceptual contribution of the included studies, not on the formal quality assessment. It includes subjective interpretation and production of a hypothesis or a model as a start point for further evaluation instead of objective review and providing an answer to a specific question as an endpoint.
Systematic Search and Review	The rationale behind this type is to provide a big picture of a broad topic. Therefore, this type combines the strengths of a critical review with a comprehensive search process often incorporating multiple study types. Unlike SR, the synthesis process, as well as inclusion and exclusion criteria are not explicit and predefined.
Mapping Review/ Systematic Map	As its name shows, mapping out and descriptively categorizing existing literature on a particular topic are key features of this review type. Its rationale is to identify literature gaps and degree of sufficient coherency, providing a base for formulating informed-based research questions for further primary or secondary research on that topic.
Scoping Review	The rationale behind this type is to categorize a large, complex, or heterogeneous literature of a broad topic, in terms of its nature, features, volume, and scope. The results of this type are usually descriptive and provide a preliminary categorization, not a final answer to recommend policy or practice.
Umbrella Review	The rationale behind this type is to provide an accessible and usable document discussing competing interventions for a broad condition or problem as well as reviews addressing these interventions and their results. To this end, this type collects evidence from multiple existing (systematic) reviews.

III. SR versus Scientometric Studies

Moreover, it is necessary to distinguish SR, as a type of synthetic research, from scientometric research, as a type of meta-science, to better understand the rationale behind SR. Scientometrics along with history, philosophy, and sociology of science consider science as an object of study (i.e., scientific research is not only the tool for study but also the object of study, in these fields). So, scientometrics is called research on research [26]. The most important rationale behind this type of research is to help research policymakers or research managers evaluate and subsequently enhance scientific research impacts. This field uses scientific methods to evaluate research and provides a powerful lens for examining the scientific system itself [27, 28].

All topics in CM can be the subject of three types of research that are original, synthetic, and scientometric; but, the choice of research type depends on the research goal as the rationale behind each type is different.

First, original research is the choice to find out what the outside world or our mind tells us about a topic (using one of the philosophical, empirical, historical, or interpretive approaches). An example for original research can be a work by Moattar et al. The study was a historical cohort from a pilot randomized controlled trial that evaluated the effect of chloroquine on pro-inflammatory cytokines levels in Graves’ disease [29].

Second, synthetic research (including SR) is appropriate to find out what the current literature tells us about a topic. A meta-analysis and systematic review by Chin et al. that examined the prevalence of Grave’s ophthalmopathy can be a good example [30].

Third, scientometric research is proper to find out how the condition of the existing scientific research on a given topic and its impacts are. An example for scientometric research can be the work by Muñoz-Velandiaae et al. which assessed the correlation between metrics of activity in social networks and traditional metrics of scientific impact in endocrinology journals [31].

B. SR as a more informed and less biased review

Providing a more informed and less biased review is the most important and distinguishing feature of the SR method (Table 2). To this end, SRs require to meet strict high standards in all steps of the review process; we have categorized these standards into four groups (Fig. 1):

• (i)Sensitive search
• (ii)Critical appraisal of included studies
• (iii)Inter-rater reliability among reviewers
• (iv)A detailed, predefined protocol

I. Sensitive Search

Sensitive search, as a foundation of SR, provides comprehensive data. The more sensitive the search is, the more the reviewers include relevant existing knowledge. Comprehensive data enables reviewers to pursue five aims of conducting SRs (i.e., the third to the seventh reasons in Table 1). These underlying reasons, related concepts, and causal relationships among them (Fig. 1) are discussed in the following subsections.

• The third reason: to increase the external validity of findings

The terms of external validity, generalizability, applicability, and transferability are used interchangeably. External validity refers to the extent to which the study results can be generalized and applied to patients who are external to the study and may be different from the study cases (e.g., in demographic characteristics, clinical setting, or disease stages). Comprehensive data provided by an SR through sensitive search increases external validity. SRs use different strategies to assess and enhance the generalizability of clinical findings. Key components affecting generalizability (e.g., demographic characteristics, event rates, interventions, underlying risks, etc.) are routinely reported in SRs [14, 32, 33].

• The fourth reason: to increase the explanatory power of findings

Also, the comprehensive data provided by an SR increases the explanatory power of findings. An SR can find consistencies among the results of different studies and helps explain the source of inconsistencies. For example, an SR can determine whether the existing knowledge consistently shows an association between a drug and disease treatment or not. In the case of inconsistency, it also helps explain conditions under which the drug is effective using different ways such as a subgroup analysis (i.e., all participants are split into subgroups and then compared) [14]. For instance, an MA of RCTs by Johnston et al. examined the potential protective effects of probiotics to prevent antibiotic-associated diarrhea in children. This study showed an inconsistency between the results of included trials. However, the subgroup analysis showed that some of the probiotics (e.g., Lactobacillus GG) had stronger preventative effects against antibiotic-associated diarrhea in children [34].

• The fifth reason: to increase the precision of estimated values

A quantitative SR (i.e., MA) is more precise than each included individual study in the estimation of the magnitude of the association between two variables. Comprehensive data provided by an SR provides a higher sample size for quantitative analyses when compared with individual studies (Fig. 1). The increased sample or pooled data along with systematic quantitative analyses lead to narrower confidence intervals and more precise estimated values [14]. For example, a cumulative MA of therapeutic trials for myocardial infarction by Lau et al. shows how conventional and cumulative meta-analysis of 33 trials of intravenous streptokinase for acute myocardial infarction decreased 95% confidence intervals or increased the precision of Odds ratios for the effect of treatment on mortality [35].

• The sixth reason: to increase the statistical power to detect true associations

A quantitative SR (i.e., MA) can increase the statistical power for detecting true associations that are not detected by an individual study. It occurs via comprehensive data provided by an SR along with quantitative analyses (Fig. 1). Especially, this issue is important when the association magnitude (e.g., the effect of a medication) is too small [14]. The above-mentioned study by Lau et al. also shows how individual trials with a low sample size alone did not establish a statistically significant relationship between intravenous streptokinase and acute myocardial infarction, but increased power through statistical analyses aggregating all the available data could establish such a relationship [14, 35].

• The seventh reason: to increase the internal validity or accuracy of findings

SR and MA can improve the internal validity or accuracy of findings through four pathways (Fig. 1):

• (i)Increased values of explanatory power, statistical power, and precision
• (ii)Increased reliability of review findings (because reliability generally affects the accuracy or validity)
• (iii)High quality included data resulted from the critical appraisal of included studies
• (iv)A predefined and precise protocol that is proper to answer a given question by reviewing the studies

The last three pathways are described in more detail in the following subsections.

II. Critical Appraisal of Included Studies

Critical appraisal of Included Studies, as the second strict high standard of SR, allows the inclusion of only high-quality studies in the review. High-quality studies, including high-quality data, increase the accuracy or internal validity of the review study (Fig. 1) [1].

III. Inter-rater Reliability among Reviewers

Every SR requires more than one reviewer for screening and data extraction and reporting the details of inter-rater reliability among Reviewers is a feature of a methodologically sound SR. To meet this standard, an SR reports how reviewers agreed on the screening and data extraction stages of a study, as well as how they resolved disagreements. Acceptable inter-rater reliability increases the reliability of the review findings (i.e., the review process is reliable and has the least random errors) (Fig. 1) [36].

• The eighth reason: to increase the reliability of review findings

As shown in Fig. 1, the reliability of review findings improves through several pathways:

• (i)Acceptable inter-rater reliability between reviewers (as mentioned above)
• (ii)Increased systematicity of review process
• (iii)Increased objectivity of review process

The last two pathways are pertinent to the next standard for SR, a detailed predefined protocol (discussed in the next part).

IV. A Detailed, Predefined Protocol

A protocol is required to be developed before starting the review process to promote transparency in the review process. A detailed and well-defined protocol includes the review question(s), search strategies, eligibility criteria (e.g., study design, characteristics of the participants or intervention), the definition or status of outcome measurements, and methods relating to effect measures, data analysis, and exploration of heterogeneity. SRs should be registered at inception (i.e., at the protocol stage) in certain networks, such as PROSPERO or PRISMA, and/or published as an article before conducting the SR [1, 37–39].

SR protocol has three major functions (Fig. 1). First, a detailed predefined protocol serves as a road map for authors and reduces their reliance on post hoc decisions. Registration of SR protocols (like clinical trial registration) helps minimize bias in conducting and reporting the review (i.e., it enables comparison of the reported review methods with what was planned in the protocol). Therefore, preparation and registration of the protocol provide more systematicity and objectivity and reduce the subjective role of the reviewers in the review process (in other words, increase the reliability of review findings). Additionally, registration might also encourage full publication of the review’s findings. Second, protocols could be peer-reviewed (and revised if necessary) via publishing as a separate article before conducting the review. Peer-reviewing SR protocol helps researchers to frame a proper protocol (e.g., including proper search strings providing a sensitive search) to answer best a given question. Third, registration of SR protocols helps avoid unplanned duplications which is discussed in more detail in the next section [14, 40–42].

C. SR as an efficient scientific tool

SR is considered an efficient scientific tool from at least two perspectives which are the two last underlying reasons for SR (Table 1) as discussed in the following.

• The ninth reason: to implement the current best evidence less costly

The ninth underlying reason for SR is to implement the current best evidence less costly (Table 1) through three ways (Fig. 1). First, SRs avoid unnecessary re-examination of evidence by new original research. Although conducting a high-quality SR is challenging and cannot ignore the fact that there should be enough high-quality evidence to conduct a reasonable SR, it is usually quicker and cheaper to conduct an SR than conducting new original research. If a well-conducted and homogeneous SR shows a relation between two variables (e.g., a drug and therapeutic effects) with an acceptable confidence interval, it can eliminate further needs for testing the same hypothesis by additional studies. Second, SRs avoid unnecessary duplication of reviews. As mentioned above, it results from registering and /or publishing SRs’ protocols separately before conducting the review. Third, SRs help decreases the clinical practice costs by the implementation of evidence more rapidly [14].

• The tenth reason: to implement the current best evidence more rapidly

The tenth and last underlying reason for SR is to implement the current best evidence more rapidly (Table 1). Well-conducted SRs are placed at the highest level in the hierarchy of evidence (e.g., SRs of RCTs are the highest level of evidence for therapeutic questions). While none of the individual studies investigating a clinical question can detect a clinically and statistically significant relationship between two variables (e.g., a drug and treatment), a well-conducted SR addressing the same question can have enough power to detect and influence to establish it; consequently, the SR can reduce the time spent between the detection of that relationship and its application in clinical practice (Fig. 1) [14].

A landmark example is the life-saving effect of intravenous streptokinase in hospitalized patients for acute myocardial infarction; while available data had the potential to establish the efficacy of this medication in 1971, it took twenty years to detect the beneficial effects and its general implementation in clinical practice. Because in the absence of an SR (and MA) aggregating all the available data, individual trials alone did not establish a statistically significant efficacy of the medication [14, 35].

Common misconceptions about SRs in CM

Although SRs have become popular, there are still several common misconceptions about them. These misconceptions about SRs are mostly in the following topics: the purpose (e.g. SRs can only address the question about the efficacy of clinical interventions), the methodology (e.g. an SR must include only one particular type of study), importance (e.g. SRs are the same as narrative reviews, only bigger) and application of SRs (e.g. SRs can only be used in medicine and health). Table 3 shows 20 common misconceptions followed by specific clarifications for each of them [5, 7, 8, 25, 26, 43].

Table 3.

Several Common Misconceptions about Systematic Review (SR) in Clinical Medicine

	Misconception	Clarification
1	SRs include only randomized controlled trials (RCTs) [5, 7, 8, 43].	SRs can be run on non-randomized or even observational studies; It is also not unusual to find an SR on qualitative or argumentative studies [5, 7, 8, 43].
2	SRs can only address the question about the efficacy of clinical interventions [8, 43].	Many of the early SRs in the healthcare field initially addressed the efficacy of interventions; however, theoretically and empirically, the method is not limited to this question type [8, 43].
3	An SR must include only one particular type of study [8].	Studies that are included in an SR can be diverse in participants, disease definitions, measuring methods, exposure or treatment types, study designs, etc. Although diversity might increase heterogeneity, the findings of diverse studies can increase explanatory power. On the other hand, including all the published studies, even with different methodologies, will decrease the risk of publication bias [8].
4	SRs can only be used to answer narrow questions [8].	Some have claimed that SRs focus on narrow questions that have limited practical utility and that SRs are capable of investigating only single populations, interventions, and outcomes. While many of the early healthcare SRs had a relatively narrow scope, the range of populations, interventions, and outcomes now included in SRs, in healthcare and other fields are not necessarily limited [8].
5	SRs are the same as narrative reviews, only bigger [5, 7].	SRs are qualitatively not quantitatively different from narrative reviews. SRs answer a very specific question through a systematic search and appraising the included studies which sometimes make them smaller than narrative and traditional reviews [5, 7].
6	SRs require the adoption of a biomedical model of health; SRs cannot capture socio-political-, economic-, or health- context [5, 7, 8].	The underlying assumption is that as health is more than the absence of disease and humanistic aspects of health need to be considered, important outcomes of interventions other than biomedical ones (e.g., as social impacts) are somehow inappropriate for inclusion in SRs. However, all the outcomes are regularly defined, measured, and evaluated in both qualitative and quantitative primary studies, and these studies can be included in SRs [5, 7, 8].
7	SRs can only be used in medicine and health [5, 7, 8].	As mentioned previously, SRs originally have been conducted in the social science field and later extended to medicine and other fields. Today, SRs are gradually used in many fields “from astronomy to zoology” [5, 7, 8].
8	SRs necessarily involve statistical synthesis (i.e., meta-analysis) [5, 7, 8].	SRs can include qualitative research and different forms of analysis. Quantitative synthesis can be run or omitted in an SR based on the quality and the included studies and their data. When SR accompanies by a statistical synthesis, it is called a meta-analysis [5, 7, 8].
9	SRs can be done without the support of an experienced librarian [5, 7].	To produce good SRs, thorough knowledge, and experience in designing search strategies and searching the databases are needed; researchers are not generally qualified for these tasks as experienced librarians are [5, 7].
10	SRs have to be done by experts [5, 7].	Expert clinical practitioners or even other potential users of SRs (e.g., consumers or policymakers) may be involved in the process of review to produce well-focused and relevant questions or to disseminate results to appropriate audiences effectively. However, the rigor of SRs is in the strict methodological requirements and the degree of adherence to the requirements, not in the experience of authors in the field, unlike narrative reviews [5, 7].
11	SRs are substitutes for doing high-quality individual studies to take money away from primary research [5, 7].	Conducting SR needs including and appraising high-quality studies. On the other hand, some SRs are concluded with the lack of enough evidence or high-quality studies on some subjects that will result in more (and not less) primary research [5, 7].
12	SRs are not necessarily considered added-value. To add value to existing knowledge, it is necessary to do an original research [25].	SRs (or even NRs) as synthetic research add value to the existing knowledge through identifying, evaluating, and integrating information about a subject. SRs provide stronger evidence by pooling the data of several studies, with higher power and accuracy [25].
13	SR is the best review method to address every medical and biomedical research question [25].	The development of evidence-based practice mandates more variety of reviews. Thus each review has its key characteristics with special strengths and weaknesses. They could be chosen and used according to the aim of the study and the context. The aim of the study is the key guide to select the review type [25].
14	SR is a type of scientometric analysis [26].	SR, as a type of synthetic research, is different from scientometric research, as a type of meta-science, in terms of their rationales. Whilst synthetic research is appropriate to find out what the current best knowledge tells us about a topic, scientometric research is proper to find out how the condition of existing scientific research on a given topic and its impacts are [26].
15	SRs are undertaken by outside researchers detached from the realities of practitioners; they are of no relevance to the real world [5, 7, 8].	This misconception is based on the belief that SRs are limited to the efficacy of clinical interventions investigated by RCTs which are carried out in controlled conditions that are of no relevance to the complex real world. However, the SR method has not this limitation; as a variety of primary research studies is needed to capture the real world, a variety of SRs to systematically review those studies is needed for this goal. Moreover, the SR method has the potential to capture the real world much better than individual primary studies by increasing the reliability, accuracy, precision, generalizability, and explanatory power of research findings as well as increasing the statistical power to detect true associations in the real world [5, 7, 8].
16	A review is systematic if articles are identified through a systematic search or a stepwise approach to inclusion [8].	Performing search and inclusion of articles in a systematic way is a necessity but not an enough condition for a review to be considered SR [8].
17	SRs must be registered with a collaboration [8].	According to Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA), pre-registration of an SR protocol in a relevant registration body is not a Must standard but also it is a should standard which has more benefits such as avoiding unplanned duplications and enabling the comparison between what carried out finally in a review and what planned in its protocol in advance [8].
18	SRs provide definitive answers to questions [8].	The rationale behind SR is principally to provide an informed and less biased summary and synthesis of existing literature for addressing a question. The summary does not necessarily include a definitive answer to that question because the strength of the conclusions of an SR is dependent on the availability, heterogeneity, and quality of primary studies addressing the given question. However, even inconclusive SRs are invaluable because they make definitive beliefs not supported by evidence under question and also show the gaps of literature for developing future research [8].
19	SRs fail to provide decision-makers with easily digestible summaries of the evidence and its synthesis [8].	Since SRs contain explicit details about the methodology and included studies, they can be difficult to read and interpret by not-trained readers. However, it is not a fault of SRs but it is the strength of SR methodology. Therefore, solving the difficulty necessitates training in evidence-based reasoning especially critical appraisal of SRs. Besides, for having a glance at new evidence other products (e.g., fact sheets or policy briefs) are suggested [8].
20	SRs take long periods of time to complete and require considerable funding [8].	The time and financial cost required to produce an SR is various depending on key factors such as the complexity of the review question, the type, and volume of evidence available, or the size, availability, and experience of the review team [8].

Conclusion

This article presented a conceptual framework clarifying the rationale behind SR in CM by providing the relevant concepts and their inter-relations, explaining how methodological standards of an SR and its rationale are connected, and discussing the rationale under the three-section. SR, as a type of synthetic research, is to address a clearly formulated question by summarizing and integrating the existing literature. On the other hand, SR, as a more informed and less biased type of review, adheres to high strict standards (i.e., sensitive search, critical appraisal of included studies, acceptable inter-rater reliability among reviewers, and a detailed, predefined protocol). These standards increase the external validity and the explanatory power of the findings, the precision of estimated values, the statistical power to detect true associations, and the internal validity or accuracy as well as the reliability of the findings. Finally, SR as a highly efficient scientific tool helps implement the current best evidence less costly and more rapidly. SR can play an important role to implement evidence-based practice as far as the rationale behind it is well understood.

Authors’ contributions

HM had substantial contributions to concept and design, drafting, writing, and revising the manuscript critically for important intellectual content and approving the final version to be published. FS had substantial contributions to drafting, writing, and revising the manuscript and approving the final version to be published. MA had contributions to drafting the manuscript and approving the final version to be published. AS had contributions to concept and design, revising the manuscript critically for important intellectual content, and approving the final version to be published. BL had contributions to concept and design, revising the manuscript critically for important intellectual content, and approving the final version to be published.

Declarations

Conflict of interest

The authors declared that they had no conflict of interest.