Why Systematic Review rather than Narrative Review?

Sir: Recently review articles including systematic and narrative reviews have been significantly increasing in most psychiatric journals in the world alongside "Psychiatry Investigation (PI)". Since the launch of the "PI" at March 2004, there have been a number of review articles; indeed 54 papers were published as format of regular review papers or special articles in the "PI" from 2004 to 2014. However, of the 54 papers, only one review paper partially met the contemporary criteria of systematic review, otherwise were written as a format of narrative review for diverse topics such as epidemiological findings, concept and hypothesis of certain psychiatric disease, current understandings on certain disease, psychopharmacology, and treatment guidelines. This is unsatisfactory when reflecting the fact that systematic reviews have been rapidly and increasingly replacing traditional narrative (explicit) reviews as a standard platform of providing and updating currently available research findings as confident evidence. Most journals have started to change their policy in acceptance of review papers, they have been giving a priority to systematic review only as a regular review article and excluding narrative reviews, to provide the best evidence for all basic and clinical questions and further hypotheses. Of course, there should be Pros and Cons between systematic and narrative reviews; for instance, the major advantage of systematic reviews is that they are based on the findings of comprehensive and systematic literature searches in all available resources, with minimization of selection bias avoiding subjective selection bias, while narrative reviews, if they can be written experts in certain research area, can provide experts' intuitive, experiential and explicit perspectives in focused topics.¹

The absence of objective and systematic selection criteria in review method substantially results in a number of methodological shortcomings leading to clear bias of the author's interpretation and conclusions. Such differences are quite clear when referring to the review paper of Drs. Cipriani and Geddess,² where 7 narrative and 2 systematic reviews were compared and found that narrative reviews including same studies reached different conclusions against each other, indicating the difficulties of appraising and using narrative reviews to have conclusion on specific topic. Hence, narrative reviews may be evidence-based, but they are not truly useful as scientific evidence.

Even in reported as systematic review, it is also frequent that those papers are not true systematic review or they have certain bias in data search method and conclusions. For instance, due to lack of satisfactory pharmacotherapy for post-traumatic stress disorder (PTSD) and its frequent comorbid psychotic symptoms, a possible role of atypical antipsychotics (AAs) for PTSD has been consistently proposed.³ In fact various AAs have demonstrated positive antidepressant and ant-anxiety effects in a number of small-scale, open-label studies (OLSs) or randomised, controlled clinical trials (RCTs).⁴ In this context, a recent systematic review (4 olanzapine, 7 risperidone and 1 ziprasidone trials) by Wang et al.⁵ has also suggested the positive prospect on the role of AAs for the treatment of PTSD; however, the review has a number of faulty and wrong selection of clinical trials data and interpretation of studies included in their review. The authors neglected wide range of clinical information such as patient characteristics (particularly, initial severity of disease), comorbidity issues, trial duration issues, trial design characteristics, primary endpoint difference, study sponsoring; that is, heterogeneity of clinical trials would substantially influence the quality and clinical implications of the study results. The basic problem of non-systematic search of data is that beneath the shining surface, it seems that the authors utilizing it often misunderstand the true value, underpinning meanings and correct nature of the data, or their true limitations and strengths, and they often go too far or short with the interpretation.⁶ Indeed, the main conclusion of a narrative review may often be based on evidence, but such reviews themselves are not rigorous evidence since such reviews are too selective and thus little good quality information could be included.² In addition, I found one olanzapine trial was OLS but they included the study in the result (this is a mixture of data yielding a huge heterogeneity).⁷ This clearly indicates they were not consistent in collection of the study for their review. Olanzapine has a lot of OLSs beyond the study, likewise other AAs also have a plenty of OLSs. Regarding an inclusion of OLSs for systematic reviews, an interesting metaanalyses are available on the role of olanzapine for adolescent bipolar disorder⁸ and aripiprazole augmentation therapy⁹ for depression. According to Pae et al.⁹ the treatment effects were not significantly different between OLSs and RCTs in efficacy of aripiprazole augmentation for treating depression; the pooled effect size was statistically significant in both study design and also in a meta-analysis regression, study design was not a significant predictor of mean change in the primary endpoint, clearly indicating that OLSs are useful predictors of the potential safety and efficacy of a given compound. This finding was also supported by another meta-analysis.⁸ Hence, the value of OLSs should be carefully re-evaluated for practical information source, development of new drugs or acquisition for new indications, and should not be neglected for data research, especially for narrative reviews. Furthermore, Dr. Wang et al.⁵ did not include one important RCT; quetiapine has a RCT for PTSD,¹⁰ which was presented in the thematic meeting of the CINP 2009. A 12-week RCT was conducted for 80 PTSD patients. Finally, Wang et al.⁵ surprisingly did not present any effect size (ES) for studies, although such calculations are conventionally included in the review papers. Another critical example is Hickie and Rogers's review,¹¹ according to their article, agomelatine was efficacious antidepressant; however, subsequent researchers who avoided selection bias have clearly demonstrated its weak efficacy as an antidepressant.¹² Therefore, reflecting two review papers,⁵,¹¹ we can realize that inappropriate aggregation of studies may definitely bias conclusion. Hence, entire published and unpublished dataset should be considered in systematic review, especially, when clinical data is not sufficient and the medication has no officially approved indication by the regulatory agency.

To summarize, systematic review should include followings respecting recommendation from currently available systematic review guidelines (e.g., The Cochrane Library www.cochrane.org); clear basic and clinical hypothesis, predefined protocol, designation of search resources, through data search (regardless of publication), transparent selection criteria, qualification of studies selected, synthesis of study data and information, relevant summary and conclusion. Table 1 compares systematic and narrative reviews (Table 1). Since the evidence-based medicine is the current trend and also mandatory for establishment of heath policy, the PI should also turn to encourage submission of systematic reviews rather than narrative reviews.

Table 1. Comparison between narrative vs systematic review.