Abstract
Objective
To examine the quality of reports of complementary and alternative medicine (CAM) systematic reviews in the pediatric population. We also examined whether there were differences in the quality of reports of a subset of CAM reviews compared to reviews using conventional interventions.
Methods
We assessed the quality of reports of 47 CAM systematic reviews and 19 reviews evaluating a conventional intervention. The quality of each report was assessed using a validated 10-point scale.
Results
Authors were particularly good at reporting: eligibility criteria for including primary studies, combining the primary studies for quantitative analysis appropriately, and basing their conclusions on the data included in the review. Reviewers were weak in reporting: how they avoided bias in the selection of primary studies, and how they evaluated the validity of the primary studies. Overall the reports achieved 43% (median = 3) of their maximum possible total score. The overall quality of reporting was similar for CAM reviews and conventional therapy ones.
Conclusions
Evidence based health care continues to make important contributions to the well being of children. To ensure the pediatric community can maximize the potential use of these interventions, it is important to ensure that systematic reviews are conducted and reported at the highest possible quality. Such reviews will be of benefit to a broad spectrum of interested stakeholders.
Introduction
Healthcare providers, consumers, and others cannot keep up-to-date with the healthcare literature. For example, healthcare professionals attempting to keep abreast of their field would need to read, on average, 19 original articles each day [1]. Systematic reviews offer the potential to reach that elusive goal of keeping up-to-date without sacrificing quality and thoroughness. There has been a striking increase in the number of published systematic reviews, particularly of RCTs. One of the first 'medical' systematic reviews was published in 1955 [2]. Currently, in the Cochrane library alone, there are more than 1000 published systematic reviews and several hundred protocols even though it has existed for only seven years.
Systematic reviewers have little control over random errors but can exert some influence over systematic errors (bias). Therefore, evaluating the quality of reports of systematic reviews is likely to provide important information to gauge the extent of these problems in any given report. There have been a several publications examining the quality of reports of systematic reviews of conventional interventions, such as pharmaceuticals. Overall, the data indicate that systematic reviews are not reported optimally [3-5].
More recently attention has been focused on a similar examination of systematic reviews in complementary and alternative medicine (CAM) [6-8]. The result of these investigations is similar to those found in the conventional medicine literature. Much of this evidence pertains to the adult population with far less attention devoted to evaluating the pediatric literature.
We set out to examine the quality of reports of CAM systematic reviews in the pediatric population (PedCAM). We examined whether there were differences in the quality of reports of PedCAM reviews compared to reviews in conventional medicine. We also evaluated how specific issues in the systematic review process, such as searching to identify potentially relevant primary studies, are reported by reviewers.
Methods
Two main sources were used for identifying systematic reviews. The first was a registry of 437 CAM systematic reviews provided by University of Maryland Complementary Medicine Program http://www.compmed.umm.edu/Compmed/Cochrane/Cochrane.htm This registry was constructed and maintained through searches of Medline, Embase, Psyclnfo, AMED (Allied and Complementary Medicine Database, British Library), and IDAG (Informati en Documentatiecentrum Alternateve Geneeswigzen, Be Amersfoort, The Netherlands).
Secondly, bibliographic databases were searched to identify additional studies. Several incidental sources were discovered and reviewed, for instance, a list of systematic reviews of herbal medicinal products published as a British Medical Journal 'web extra' [9]. The databases searched are reported in Table 1 and the search strategies are presented in see Additional file 1. appendix.rtf.
Table 1.
Databases searched and years of search
| Database | Host | Date Range Searched | Strategy |
| Mediine | Ovid | 1966-March 2001 | 1* |
| Alt-HealthWatch | EBSCO | 1990-March 2001 | 2 |
| AM ED | EBSCO | 1985-March 2001 | 2 |
| Cochrane Database of Systematic Reviews | Ovid | Issue 1, 2001 | 3 |
| Cochrane DARE | Ovid | Issue 1. 2001 | 3 |
All bibliographic records were imported into a reference database where duplicate items were resolved. Bibliographic records or, in many cases, full copies of the documents were screened for inclusion according to the following criteria: 1) one or more studies in the review included children, 2) a CAM therapy was investigated and 3) the article was a systematic review. The screening was completed in an unblinded fashion. Eligibility was determined by a single reviewer (MS) because the criteria were relatively objective, and many reviews had been pre-qualified through the University of Maryland Complementary Medicine Program decision process and were only reviewed to determine whether the review included studies involving children. When there was any doubt about eligibility, a second reviewer (DM) examined record and a final decision was arrived at by consensus. We attempted to match each PedCAM report with conventional pediatric comparator reviews of the same disease drawn from our existing collection [10-13]. We were able to match 17 PedCAM systematic reviews with 19 conventional therapy reviews.
One member of the group (KS) extracted descriptive information and aspects related to the conduct of each systematic review report using a 37-item structured data collection form. The questions pertained to the type of CAM used, the International Classification of Disease (ICD-9) under investigation, how the studies in the review were identified, the number, gender, and ages of the included children, the number and type of outcomes used, the data synthesis conducted, information about handling of heterogeneity and publication bias, the reporting of adverse events and cost information (the complete questionnaire can be obtained from the authors).
Two members of the research team (DM, MS) completed a comprehensive quality assessment of each report using the Oxman and Guyatt validated scale [14]; [15]. This instrument includes nine items pertaining to individual aspects in the reporting of a systematic review (e.g., were the search methods used to find evidence on the primary question stated?). Each item is assessed using a three-point scale (i.e., no, partially/can't tell or yes). A final question elicits an overall scientific quality of the systematic review based on the previous items. The scoring ranges from one to seven, with higher scores indicating superior quality. We did not complete any formal training prior to evaluating the systematic reviews. We have extensive experience using this assessment tool and have previously conducted training with results indicating substantial agreement between raters [13,16]. Discrepancies were resolved by consensus between both raters.
Fisher's exact test was used to compare the two types of systematic reviews with respect to their quality assessment on each of the first nine items of the Oxman and Guyatt scale. The Kruskal Wallis test was used to assess the difference in the overall scientific quality of the both types of systematic reviews.
Results
We screened 479 potentially relevant articles. Of these 432 failed to meet our inclusion criteria, or were otherwise unusable. The vast majority of reports were excluded (84%) because they did not include any randomized trial involving children (Figure 1).
Figure 1.
Flow of citations and articles through the phases of screening and eligibility evaluation.
The reviews were recent with 1998 being the median year of publication. Diseases of the nervous system and sense organs, mental disorders, and respiratory system were the most common ICD categories investigated (Table 2). Psychotherapy and vitamins were the most common interventions examined (Table 3).
Table 2.
Freqency of international classification of disease (ICD) categories investigated in systematic reviews of pediatric complementary and alternative medicine
| ICD category | N | % |
| Diseases of the nervous system and sense organs [6] | 13 | 27.7% |
| Mental disorders [5] | 12 | 25.5% |
| Diseases of the respiratory system [8] | 10 | 21.3% |
| Diseases of the musculoskeletal system and connective tissue [13] | 6 | 12.8% |
| Certain conditions originating in the perinatal period [15] | 6 | 12.8% |
| Diseases of the digestive system [9] | 5 | 10.6% |
| Intestinal infectious diseases [1] | 4 | 8.5% |
| Endocrine, nutritional and metabolic diseases and immunity disorders [3] | 4 | 8.5% |
| Diseases of the skin and subcutaneous tissue [12] | 4 | 8.5% |
| Neoplasm's (2) | 3 | 6.4% |
| Diseases of the circulatory system [7] | 2 | 4.3% |
| Injury and poisoning [17] | 2 | 4.3% |
| Complications of pregnancy, childbirth and the puerperium [11] | 1 | 2.1% |
| Symptoms, signs and ill-defined conditions [16] | 1 | 2.1% |
| Diseases of the blood and blood-forming organs [4] | 0 | 0.0% |
| Diseases of the genitourinary system [10] | 0 | 0.0% |
| Congenital anomalies [14] | 0 | 0.0% |
| Supplemental classification of external causes of injury and poisoning [18] | 0 | 0.0% |
| Supplementary classification of factors influencing health status and contact with health services [19] | 0 | 0.0% |
Table 3.
Freqency of interventions investigated in systematic reviews of pediatric complementary and alternative medicine
| Intervention used | ||
| Psychotherapy | 8 | 17.0% |
| Vitamins | 8 | 17.0% |
| Behavioral interventions | 4 | 8.5% |
| Energy healing | 4 | 8.5% |
| Massage | 4 | 8.5% |
| Herbs | 3 | 6.4% |
| Homeopathy | 3 | 6.4% |
| Hypnosis | 3 | 6.4% |
| Acupuncture | 2 | 4.3% |
| Biofeedback | 2 | 4.3% |
| Drugs (as well as some CAM therapy) | 2 | 4.3% |
| Exercise | 2 | 4.3% |
| Lifestyle diet | 2 | 4.3% |
| Megavitamins | 2 | 4.3% |
| Non-nutritive sucking | 2 | 4.3% |
| Relaxation | 2 | 4.3% |
| Spiritual healing by others | 2 | 4.3% |
| Chiropractic | 1 | 2.1% |
| Art/music therapy | 0 | 0.0% |
| Folk remedies | 0 | 0.0% |
| Self-Help group | 0 | 0.0% |
| Self-Prayer | 0 | 0.0% |
| Other | 7 | 14.9% |
The median number of primary studies included in the reviews was 12, of which 9 (median) included children. The median number of participants was 604, of which 362 (median) were children. Although 40% of the reviews were limited to children only, another 32% of the reviews did not provide separate results for children included. Only 2 (of 47) reviews reported sex distribution of the included children and none reported the age distribution of included children.
Medline was the most commonly searched database to help identify primary studies for inclusion in the reviews. The Cochrane library was the second most commonly searched database (Table 4). However, only about half of the reviews (51.1%) reported the years of coverage for the search. The same percentage of reviews reported the search terms used while only a minority (8.5%) of reports reproduced the entire search. Reviewing reference lists was the most commonly reported other search method used to identify potentially relevant primary studies (Table 4). About two thirds of the reviews (68.1%) reported including unpublished material when it existed and most reviews (78.7%) did not report having any primary study language restrictions.
Table 4.
Freqency of databases searched and other search methods used to identify primary studies included in systematic reviews of pediatric complementary and alternative medicine
| Databases Searched | N | % |
| Medline (or Index Medicus) | 32 | 68.1% |
| Cochrane Library | 17 | 36.2% |
| Embase (or Excerpta Medica) | 13 | 27.7% |
| PsycLit (or Psych. Abstracts) | 13 | 27.7% |
| CINAHL | 10 | 21.3% |
| Dissertation Abstracts | 7 | 14.9% |
| Biosis (or Biological Abstracts) | 6 | 12.8% |
| Science Citation Index (ISI) | 5 | 10.6% |
| Current Contents | 3 | 6.4% |
| Pysclnfo | 3 | 6.4% |
| HealthStar | 2 | 4.3% |
| SocioFile or Sociological Abstracts | 2 | 4.3% |
| Other Bibliographic Databases1 | 12 | 25.5% |
| Other Trials Registries2 | 2 | 4.3% |
| Other search methods | ||
| Reference Lists Reviewed | 34 | 72.3% |
| Experts / Corresponding Authors | 16 | 34.0% |
| Hand Searching | 11 | 23.4% |
| Conference Proceedings / Abstracts | 8 | 17.0% |
| Pharmaceutical Companies / Manufacturers | 6 | 12.8% |
| No search reported | 4 | 8.5% |
| Personal Files | 3 | 6.4% |
| Search mentioned but details not reported | 3 | 6.4% |
| Subject Bibliography | 1 | 2.1% |
1Other databases searched included ALTA religions database, Cancelit, CISCOM, ERIC, Index to the Chiropractic Literature, MANTIS, Mental Health Abstracts, Phytodok and SIGLE 2Other trial registries searched included National Library of Medicine database of published trials and Oxford database of perinatal trials.
The quality of reporting of all 47 PedCAM reviews is presented in Table 5. Authors were particularly good at reporting: eligibility criteria for including primary studies, combining the primary studies for quantitative analysis appropriately, and basing their conclusions on the data included in the review. Reviewers were weaker in reporting: how they avoided bias in the selection of primary studies, and how they evaluated the validity of the primary studies. Overall the scientific quality of the reports achieved 43% (median = 3) of their maximum possible total score (Table 5).
Table 5.
Quality of reports of complementary and alternative medicine systematic reviews and comparative conventional medicine systematic reviews
| Matched reviews | ||||
| Question | Complementary and Alternative Medicine Reviews (n = 47) n (%) | Complementary and Alternative Medicine Reviews (n = 17) n (%) | Comparative Conventional Medicine Reviews (n = 19) n (%) | p value |
| 1. Were the search method used to find evidence reported? | 24 (51) | 12 (71) | 10 (53) | 0.270 |
| 2. Was the search for evidence reasonably comprehensive? | 19 (40) | 7(41) | 6(32) | 0.549 |
| 3. Were the criteria for deciding which studies to include in the overview reported? | 34 (72) | 12 (71) | 8(42) | 0.086 |
| 4. Was bias in the selection of studies avoided? | 10 (21) | 5(29) | 5(26) | 0.836 |
| 5. Were the criteria used for assessing the validity of the included studies reported? | 24 (51) | 9(53) | 5(26) | 0.101 |
| 6. Was the validity of all the studies referred to in the text assessed using appropriate criteria? | 17 (36) | 7(41) | 6(32) | 0.549 |
| 7. Were the methods to combine the findings of the relevant studies reported? | 21 (64)1 | 7 (70)4 | 6 (55)3 | 0.466 |
| 8. Were the findings of the relevant studies combined appropriately relative to the primary question the overview addresses? | 25 (81)2 | 8 (80)3 | 7 (64)3 | 0.407 |
| 9. Were the conclusions made by the author(s) supported by the data and/or analysis reported in the overview? | 36 (77) | 12 (71) | 6(33) | 0.021 |
| 10. How would you rate the scientific quality of this overview?4 | 3 (2, 4) | 3 (2,4) | 3 (2, 3.5) | 0.752 |
1 Percentage calculated after removing those systematic reviews that were not applicable (n=33).
2 Percentage calculated after removing those systematic reviews that were not applicable (n=31).
3 Percentage calculated after removing those systematic reviews that were not applicable (n=11).
4 Median and 95% confidence intervals
We were able to use 17 PedCAM systematic reviews along with 19 conventional therapy reviews (available from the authors upon request) and compare their quality (Table 5). The PedCAM reports were always assessed as higher quality for all nine items on the Oxman and Guyatt scale and this reached statistical significance for one item (Table 5). There was no difference in the overall scientific quality of either type of review with both types achieving 43% of their maximum possible score.
Approximately a third of the reviews (38%) reported evaluating statistical heterogeneity. Less than a quarter of them (17%) reported assessing for the presence of publication bias. Information regarding adverse events was reported in less than a quarter of the reviews (14.9%). Similarly, information regarding costs (e.g., cost effectiveness) was only reported in one review. About half (55.3%) of the reviews reported the funding source for the systematic review.
Discussion
We found relatively few systematic reviews that focused on evaluating CAM interventions in a pediatric population. This contrasts sharply with the 1468 PedCAM randomized trials known to exist [17,18]. This suggests that the pediatric CAM community have not synthesized all of the available evidence. We found information to support this contention. In our efforts to identify PedCAM randomized trials we identified 36 studies that evaluated the efficacy and/or effectiveness of CAM interventions, such as hypnosis, for the management of pain in a variety of settings. However, we were unable to identify any systematic reviews that had pooled this evidence.
In 1989 Crowley and colleagues [19] published a systematic review evaluating the benefits of corticosteroids (versus placebo) in reducing mortality and morbidity (e.g., respiratory distress syndrome) in expectant mothers of premature infants. Their review showed that corticosteroids were significantly more effective in reducing mortality and morbidity. These researchers also noted that this evidence was available 10 years earlier (i.e., 1979) had it been synthesized. Because of the delay in pooling this evidence it is likely that some children suffered unnecessarily. The results of this review were subsequently used as the logo of the Cochrane Collaboration.
Our results indicate that the quality of reporting of the PedCAM systematic reviews, and their conventional medicine comparators, is similar and less than optimal, with considerable room for improvement. The quality of reports of systematic reviews can influence reviewers' conclusions concerning the effectiveness of an intervention. After examining 51 systematic reviews on the effectiveness of spinal manipulation reviewers' were more likely to judge the intervention as positive if the report was of high quality [20].
Resources exist to help ensure that appropriate methods are used to conduct and report systematic reviews [21-24]. Another way to improve the quality of reporting of PedCAM systematic reviews is for more pediatric journals to endorse the QUOROM statement [25]. A multi journal evaluation of QUOROM has recently been completed and the results are being written up currently. Examining the quality of reporting is 'after the fact' when the review is already completed. QUOROM can also be used by granting agencies to encourage prospective systematic reviewers to improve the conduct of their reviews. This is already starting to happen in the conduct and reporting of randomized controlled trials [26].
Conducting and reporting systematic reviews with the highest possible quality is likely to minimize the possibility that their results are influenced by bias, enabling clinicians to be more confident of using them in their practice. Reports of PedCAM systematic reviews seem particularly weak in terms of the comprehensiveness in their search to identify primary studies. For example, excluding a report identified uniquely in Embase, compared to its inclusion, can statistically exaggerate the estimates of an intervention's effectiveness by 6%, on average [13].
Systematic reviewers of the CAM literature appear to be more conscious of the consequences excluding primary studies published in languages other than English. Excluding CAM trial reports in languages other than English, compared to their inclusion, is likely to exaggerate the estimates of an intervention's effectiveness by 37%, on average [27]. This result is interesting from two perspectives: it is contrasts that found when examining the conventional medicine interventions (i.e., no effect when excluding reports of trials in languages other than English), and most of the methodological research to date has focused on the impact of bias within conventional interventions. There is an important need to develop a research agenda that focuses specifically on the impact of bias when pooling CAM interventions.
This study had several limitations. Our focus was on the quality of reporting of PedCAM systematic reviews. It is possible that the reviews were appropriately conducted but had deficiencies in their reporting. Despite the paucity of data addressing this important question, the evidence that is available points in the direction of a reasonably good correlation between how investigators conduct their research and how it is subsequently reported [28], [29]. We only selected a sample of CAM studies to compare to reviews of conventional interventions. It is possible that our sample is not representative and that our results cannot be generalized to all PedCAM systematic reviews. We selected the reports to enable comparators investigating the same disease. Given that the quality of reporting of the sample is very similar to the quality of all 47 reviews we believe that our sampling approach is representative and enables us to generalize the observed results.
Evidence based health care continues to make important contributions to the well being of children. To ensure the pediatric community can maximize the potential use of interventions it is important to ensure that systematic reviews are conducted and reported at the highest possible quality. Such reviews will be of benefit to a broad spectrum of interested stakeholders.
Competing intersts
None declared.
Pre-publication history
The pre-publication history for this paper can be accessed here:
Supplementary Material
Electronic searches used to identify aritcles included in this review
Acknowledgments
Acknowledgements
We thank Isabelle Gaboury for additional help with the data analysis.
Contributor Information
David Moher, Email: dmoher@uottawa.ca.
Karen Soeken, Email: KSoeken@compmed.umm.edu.
Margaret Sampson, Email: msampson@uottawa.ca.
Leah Ben-Porat, Email: poratl@mskcc.org.
Brian Berman, Email: BBerman@compmed.umm.edu.
References
- Davidoff F, Haynes RB, Sackett D, Smith R. Evidence-based medicine: a new journal to help doctors identify the information they need. BMJ. 1995;310:1085–6. doi: 10.1136/bmj.310.6987.1085. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Beecher HK. The powerful placebo. JAMA. 1955;159:1602–6. doi: 10.1001/jama.1955.02960340022006. [DOI] [PubMed] [Google Scholar]
- Sacks HS, Reitman D, Pagano D, Kupelnick B. Meta-analysis: an update. Mt Sinai J. 1996;63:216–24. [PubMed] [Google Scholar]
- Shea B, Moher D, Pham B, Graham I, Tugwell P. The quality of reporting of meta-analyses. Evaluation & The Health Professions. 2002;25:116–129. doi: 10.1177/0163278702025001008. [DOI] [PubMed] [Google Scholar]
- Jadad AR, Moher M, Browman GP, Booker L, Sigouin C, Fuentes M, et al. Systematic reviews and meta-analyses on treatment of asthma: critical evaluation. BMJ. 2000;320:537–40. doi: 10.1136/bmj.320.7234.537. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Linde K, Vickers A, Hondras M, ter Riet G, Thormahlen J, Berman B, et al. Systematic reviews of complementary therapies – an annotated bibliography. Part 1: Acupuncture. BMC Complement Altern Med. 2001;1:3. doi: 10.1186/1472-6882-1-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Linde K, Hondras M, Vickers A, Riet GG, Melchart D. Systematic reviews of complementary therapies – an annotated bibliography. Part 3: Homeopathy. BMC Complement Altern Med. 2001;1:4. doi: 10.1186/1472-6882-1-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Linde K, ter Riet G, Hondras M, Vickers A, Saller R, Melchart D. Systematic reviews of complementary therapies – an annotated bibliography. Part 2: Herbal medicine. BMC Complement Altern Med. 2001;1:5. doi: 10.1186/1472-6882-1-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ernst E. Herbal medicines: where is the evidence? BMJ. 2000;321:395–6. doi: 10.1136/bmj.321.7258.395. [DOI] [PMC free article] [PubMed] [Google Scholar]
- McAuley L, Pham B, Tugwell P, Moher D. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet. 2000;356:1228–31. doi: 10.1016/S0140-6736(00)02786-0. [DOI] [PubMed] [Google Scholar]
- Moher D, Pham , Klassen TP, Schulz KF, Berlin JA, Jadad AR, et al. What contributions do languages other than English make on the results of meta-analyses? J clin Epidemiol. 2000;53:964–72. doi: 10.1016/S0895-4356(00)00188-8. [DOI] [PubMed] [Google Scholar]
- Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet. 1998;352:609–13. doi: 10.1016/S0140-6736(98)01085-X. [DOI] [PubMed] [Google Scholar]
- Sampson M, Barrowman NJ, Moher D, Klassen TP, Pham B, Platt R, et al. Should meta-analysts search Embase in addition to Medline? Submitted. 2001. [DOI] [PubMed]
- Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol. 1991;44:1271–8. doi: 10.1016/0895-4356(91)90160-b. [DOI] [PubMed] [Google Scholar]
- Oxman AD, Guyatt GH, Singer J, Goldsmith CH, Hutchison BG, Milner RA, et al. Agreement among reviewers of review articles. J Clin Epidemiol. 1991;44:91–8. doi: 10.1016/0895-4356(91)90205-n. [DOI] [PubMed] [Google Scholar]
- Moher D, Klassen TP, Pham B, Lawson MW. Evidence of journal bias against Complementary and Alternative Medicine. Presented at the 4th International Peer Review Congress in Biomedical Publication, Barcelona, 2001.
- Campbell K, Sampson M, Ajiferuke I, Moher D. Publication characteristics of randomized controlled trials in pediatric complementary and alternative medicine. Submitted for publication. [DOI] [PMC free article] [PubMed]
- Moher D, Sampson M, Campbell K, Beckner W, Lepage L, Gaboury I, Berman B. Assessing the quality of reports of randomized controlled trials in pediatric complementary and alternative medicine. Submitted for publication. BMC Pediatrics. 2002;2:2. doi: 10.1186/1471-2431-2-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Crowley P, Chalmers I, Keirse MJNC. The effects of corticosteroid administration before preterm delivery: an overview of the evidence from controlled trials. British Journal of Obsterics and Gynaecology. 1990;97:11–25. doi: 10.1111/j.1471-0528.1990.tb01711.x. [DOI] [PubMed] [Google Scholar]
- Assendelft WJJ, Koes BW, Knipschild PG, Bouter LM. The relationship between methodological quality and conclusions in reviews of spinal manipulation. JAMA. 1995;274:1942–8. doi: 10.1001/jama.274.24.1942. [DOI] [PubMed] [Google Scholar]
- Klassen TP, Jadad AR, Moher D. Guides for reading and interpreting systematic reviews: I. Getting started. Arch Pediatr Adolesc Med. 1998;152:700–4. [PubMed] [Google Scholar]
- Jadad AR, Moher D, Klassen TP. Guides for reading and interpreting systematic reviews. II. How did the authors find the studies and assess their quality? Arch Pediatr Adolesc Med. 1998;152:812–7. doi: 10.1001/archpedi.152.8.812. [DOI] [PubMed] [Google Scholar]
- Moher D, Jadad AR, Klassen TP. Guides for reading and interpreting systematic reviews. III. How did the authors synthesize the data and make their conclusions? Arch Pediatr Adolesc Med. 1998;152:915–20. doi: 10.1001/archpedi.152.9.915. [DOI] [PubMed] [Google Scholar]
- Egger M, Davey Smith G, Altman DG. Systematic reviews in health care: meta-analysis in context. BMJ Books, 2001. [DOI] [PMC free article] [PubMed]
- Moher D, Cook DJ, Eastwood S, OIkin I, Rennie D, Stroup DF, et al. Improving the quality of reports of meta-analyses of randomized controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet. 1999;354:1896–900. doi: 10.1016/S0140-6736(99)04149-5. [DOI] [PubMed] [Google Scholar]
- O'Toole LB. MRC uses checklist similar to consorts. BMJ. 1997;314:1127. [PMC free article] [PubMed] [Google Scholar]
- Moher D, Pham B, Lawson M, Klassen TP. The inclusion of reports of randomized trials published in languages other than English in systematic reviews. Submitted for publication. [DOI] [PubMed]
- Liberati A, Himel HN, Chalmers TC. A quality assessment of randomized control trials of primary treatment of breast cancer. J Clin. 1986;4:942–51. doi: 10.1200/JCO.1986.4.6.942. [DOI] [PubMed] [Google Scholar]
- Hadhazy V, Ezzo J, Berman B. How valuable is effort to contact authors to obtain missing data in systematic reviews Presented at the VII Cochrane Colloquium, Rome, Italy. 1999.
- The Cochrane Library. Oxford, Update Software. 2001.
- Nahim AM. Complementary medicine – new PubMed subset. NLM Technical Bulletin. 2001;318:e7. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Electronic searches used to identify aritcles included in this review