Abstract
Emerging health problems require rapid advice. We describe the development and pilot testing of a systematic, transparent approach used by the World Health Organization (WHO) to develop rapid advice guidelines in response to requests from member states confronted with uncertainty about the pharmacological management of avian influenza A (H5N1) virus infection. We first searched for systematic reviews of randomized trials of treatment and prevention of seasonal influenza and for non-trial evidence on H5N1 infection, including case reports and animal and in vitro studies. A panel of clinical experts, clinicians with experience in treating patients with H5N1, influenza researchers, and methodologists was convened for a two-day meeting. Panel members reviewed the evidence prior to the meeting and agreed on the process. It took one month to put together a team to prepare the evidence profiles (i.e., summaries of the evidence on important clinical and policy questions), and it took the team only five weeks to prepare and revise the evidence profiles and to prepare draft guidelines prior to the panel meeting. A draft manuscript for publication was prepared within 10 days following the panel meeting. Strengths of the process include its transparency and the short amount of time used to prepare these WHO guidelines. The process could be improved by shortening the time required to commission evidence profiles. Further development is needed to facilitate stakeholder involvement, and evaluate and ensure the guideline's usefulness.
The authors describe the development and pilot testing of a systematic, transparent approach to develop rapid advice guidelines on pharmacological management of avian influenza.
Clinical practice guidelines generally, and some WHO guidelines specifically, have been criticized for not being based on the best available evidence, for being exposed to undue influence by industry and experts who participate in guideline panels, and for not adhering to guidelines for preparing guidelines [1–7]. Guidance that is not informed by the best available evidence or by statements that the available evidence is of low quality can harm patients, waste limited resources, and hinder research to address important uncertainties [8].
While there is broad agreement that rigorous and transparent methods should be used [9–12], rigorous development of guidelines can take two years or more [13,14]. This timeframe is not practical for providing rapid advice, for example for emerging infectious diseases such as avian influenza (H5N1 infection) or severe acute respiratory syndrome (SARS). Indeed, one of the most frequently cited weaknesses in guideline development is the length of time that it takes to develop a guideline [15]. Organizations including the National Centre for Health and Clinical Excellence in the United Kingdom and the National Institutes of Health in the United States are investigating ways of streamlining guidelines development processes [16,17].
The WHO Advisory Committee for Human Research, an independent committee appointed by the Director-General of the WHO, evaluated existing WHO and other guidance processes and suggested ways to improve WHO's methods[18,19]. Consistent with these recommendations, a new model for developing WHO rapid advice guidelines was designed and tested through the development of guidelines for the pharmacological management of avian influenza A (H5N1) virus infection [20,21]. Because the approach to developing these rapid advice guidelines was novel for WHO, we describe the methods, the strengths of the approach, and ways in which this approach should be further developed.
The Process
In January 2006, the WHO decided to convene a rapid advice guidelines panel for the pharmacological management of H5N1 patients in response to requests for advice from frontline clinicians and public health professionals managing H5N1 infections. The key steps and timeline for developing the guidelines are summarized in Box 1.
Box 1. Key Steps in the Development of WHO Rapid Advice Guidelines.
Group selection and composition
In selecting members for the panel, we wanted to include several important stakeholders: clinical, methodological, and basic science experts and member country representatives, including low- and middle-income countries. We used the WHO's international network of response teams for viral pandemics and searched the medical literature for experts on H5N1 infection to identify panel members. We asked methodologists with experience in applying the Grading Recommendations Assessment, Development and Evaluation (GRADE) approach that was officially adopted by WHO to participate in the creation of evidence profiles and the guideline development process. Thirteen voting panel members supplemented by WHO experts participated in the panel meeting. We followed a thorough process to declare conflicts of interest (see Text S1 under Supporting Information).
Formulating questions and rating the importance of outcomes
The original questions were identified by clinicians managing patients with H5N1 infections and refined by panel members. For each question an evidence profile was prepared using the GRADE approach (Figure 1) [22]. The GRADE approach required the identification of relevant outcomes to be included in each evidence profile before developing the evidence profiles. Two reviewers identified potentially important outcomes and this list was circulated to the panel chair, WHO staff, and the scientific reviewers by email for independent scoring of the relative importance of each outcome and identification of additional potentially important outcomes. Outcomes were rated on a scale from 1–9; a rating of 7–9 indicated that the outcome was critical for a decision or recommendation, 4–6 indicated it was important, and 1–3 indicated it was not important. The evidence profiles included only critical or important outcomes based on the mean value of the ratings by the panel members.
Figure 1. Example of a GRADE Evidence Profile.
Footnotes 1–15 in [21] provided detailed information about the rationale underlying the decisions.
To obtain consumer input, the Cochrane Consumer Network was also invited to provide feedback through their electronic discussion list. We received only four responses, but despite reported difficulties with the rating, the relative importance of the outcomes did not differ importantly from those of the panel and no additional outcomes were identified.
Preparation of evidence profiles
An independent review team searched for systematic reviews and recent randomized trials (published in 2005 or 2006) for the treatment and chemoprophylaxis of any influenza virus infection; and case series, animal studies, and in vitro studies for the treatment or chemoprophylaxis of H5N1 infection (from 1966). The team prepared evidence profiles using the GRADE profiler software (v1.12, http://www.gradeworkinggroup.org) and proposed quality ratings according to the criteria in Box 2. The quality of outcomes measured in each animal study was judged based on whether (1) the pathogenicity of H5N1 virus was tested in the model (e.g., mortality), (2) statistical methods were adequate, and (3) a significant effect was demonstrated.
Box 2. Key Quality Criteria.
The quality of evidence for each important or critical outcome is based on:
the study design
limitations of the studies (execution)
consistency of the evidence across studies
the directness (generalizability) of the evidence to the population, intervention, comparison, and outcomes
the precision of the estimate or sparseness of data
Evidence was classified as “high”, “moderate”, “low”, or “very low” based on these criteria and the following definitions:
High: Further research is very unlikely to change confidence in the estimate of effect.
Moderate: Further research is likely to have an important impact on confidence in the estimate of effect and may change the estimate.
Low: Further research is very likely to have an important impact on confidence in the estimate of effect and is likely to change the estimate.
Very low: Any estimate of effect is very uncertain.
A summary of the findings for each question, including both trial evidence for non-H5N1 influenza and the available evidence for H5N1 infection, was prepared for each question (Figure 2) [21].
Figure 2. Summary of Findings for the Following Scenario: Should Oseltamivir Be Used for Treatment of Patients Hospitalized with Avian Influenza (H5N1)?
Panel meeting
Guideline group members and an independent expert received evidence profiles about two weeks prior to the meeting, including information about the applied methods. They were asked to identify any important missing evidence.
A draft of the guidelines was prepared by the WHO secretariat and the panel chair prior to the meeting. For the actual meeting, the agreed process rules were that: (1) Additional evidence would only be allowed at the meeting if it had been omitted from the evidence summaries, or was new and critical for decision making; (2) The GRADE approach would be used to grade the quality of evidence and the strength of recommendations [10,21]; (3) Recommendations would be based on a consensus of the panel and voting would be used if agreement could not be reached; (4) All panel members would be asked to consider their own and other conflicts during the discussion and decision making and to abstain from discussion and voting if necessary (see Supporting Information); (5) Subsequent interaction and discussion would take place through email but recommendations would not be changed after the meeting, except for minor wording changes or correction of factual errors.
Deliberation regarding the balance of benefits, harms, and costs
For each intervention considered, the panel formulated a recommendation based on the panel members' judgments regarding the balance between the benefits, harms (adverse effects), burdens (e.g., taking medication daily), costs, and values and preferences (the desirability or preference that individuals exhibit for a particular outcome) of the intervention (see Supporting Information Text S1 for information on cost).
Recommendations were classified as “strong” or “weak.” The panel was informed that strong recommendations should be interpreted as: (1) Most individuals should receive the intervention; (2) Most well-informed individuals would want the recommended course of action and only a small proportion would not; (3) The intervention could unequivocally be used in policy making.
Weak recommendations were to be interpreted as: (1) The majority of well-informed individuals would want the suggested course of action, but an appreciable proportion would not; (2) Values and preferences related to this intervention are likely to vary widely; (3) Policy making will require extensive debates and involvement of many stakeholders.
The panel used the factors listed in Figure 3 as a basis for going from a strong to a weak recommendation. This information was recorded for recommendations where a formal vote was required, as illustrated in Figure 3.
Figure 3. Decisions about the Strength of a Recommendation.
Outcomes
It took approximately one month to put together a team to prepare the evidence profiles, but once the team was assembled it took only five weeks to prepare and revise the evidence profiles and prepare draft guidelines prior to the panel meeting (Box 1). A draft manuscript for publication was prepared within 10 days following the panel meeting. It took only two additional weeks to complete a final draft of the document (on April 21, 2006) that included all of the considered evidence and recommendations.
Overall the quality of the underlying evidence for all recommendations was very low because the evidence was based upon observational data from case series describing small numbers of patients infected with avian influenza A (H5N1) virus, laboratory research, or on extrapolation from randomized trials of treatment and prophylaxis for seasonal influenza. The panel evaluated the rationale for the quality rating during the meeting and subsequently via electronic correspondence. The reasons for considering the evidence to be of low or very low quality (i.e., for having little confidence in the available estimates of effect) were: (1) the lack of direct evidence from trials among patients with H5N1 infection or exposure, (2) lack of evidence for important outcomes for H5N1 infection that were not common or not measured in seasonal influenza trials, and (3) sparse data for other important outcomes. Additional factors and specific judgments regarding the quality of the evidence are included in detailed footnotes in the evidence profiles. The panel expressed concern about the categories used to grade the quality of evidence. For example, although the quality of the evidence for treatment and chemoprophylaxis with oseltamivir and zanamivir was very low for both, there was nonetheless a difference in the quality of the evidence within the categories for these two drugs.
The panel considered several different specific patient and exposure groups when considering the balance between benefits, harms, and costs for chemoprophylaxis. This led them to develop a risk categorization for exposure to assist decision makers in prioritizing use of antivirals [20,21].
Of the 27 recommendations, 15 were strong recommendations, but most (11) were strong recommendations against a specific intervention. The arguments for making strong recommendations for specific interventions, despite the low quality of the available evidence, were the high risk of serious outcomes including death, the lack of alternative treatments, a low risk of serious adverse effects based on the available evidence, relatively low costs, and a possibility of beneficial effects, although there is much uncertainty about these. The strong recommendation regarding treatment of H5N1 patients with oseltamivir required a vote (one panel member abstained and one voted for a weak recommendation). The main argument that led to voting about this recommendation was that it is uncertain whether the intervention does more good than harm in the face of very low-quality evidence. Only one other recommendation required voting. Other factors that influenced the recommendations, value judgments, and other directly relevant information are described in the remark sections following each recommendation [21].
Discussion
These guidelines were prepared in less than two months through an intense international collaboration, the use of electronic communication, and one panel meeting. However, it took one additional month initially to put together the team that prepared the evidence profiles.
There are at least two ways in which the time needed to prepare rapid advice could be shortened: first, by identifying or establishing collaborating centres with the competency needed to prepare evidence profiles and second, by building up in-house capacity to reduce the time needed to organize a review team.
Strengths of the process
Strengths of the applied process include its transparency and the short amount of time used to prepare the guidelines. Guideline developers are increasingly using the GRADE approach because it includes transparent judgments about each of the key factors that determine the quality of evidence for each important outcome, and overall across outcomes for each recommendation [10,22,23]. In addition, the approach used in developing these guidelines included transparent consideration of the key factors that determine the strength of a recommendation. The rapid preparation of evidence profiles was possible because of the availability of high-quality systematic reviews of the evidence from seasonal influenza and the involvement of a review team with experience in preparing evidence profiles and relevant clinical expertise. In the absence of higher-quality direct evidence, the panel considered case reports, animal studies, and in vitro studies for H5N1, as well as the available indirect evidence from systematic reviews of clinical trials for seasonal influenza, systematically and transparently.
The broad representation of stakeholders in the guideline group allowed the inclusion of different perspectives for making informed judgments about the importance of outcomes, the quality of evidence, and the strength of recommendations. The publicly available evidence profiles facilitate adaptation of the guidelines to specific settings and updating of the guidelines, as well as contributing to transparency [13,21,24].
Limitations of the process
Limitations of this process relate to its very purpose: providing rapid advice. Thus, the time available for developing these guidelines did not permit detailed consideration of all clinical questions that clinicians may face. For example, the discussions about whether to use prophylactic antibiotics and which recommendations apply to situations of human-to-human transmission were short and, therefore, did not result in specific recommendations. It is unlikely that important evidence that would have led to different recommendations was missed given the nature of the problem; i.e., an emerging disease. In areas with an ample evidentiary base, evidence could be missed by relying on systematic reviews of the indirect evidence. However, this generally should not be the case when rapid advice is needed. It is not clear whether the differences the panel identified within the category of very low-quality evidence are truly important for decision making, but panel members requested that this information should not be lost in translating the quality of evidence into the four categories based on the GRADE approach.
The total budget required for this guideline development project amounted to approximately US$150,000, for a focused guideline. This was made possible in part by limiting the number of meetings through the use of electronic communication tools. While this cost compares favourably with the cost of other guideline development processes, it remains out of reach for many countries trying to develop national rapid advice guidelines.
Involvement of stakeholders through consultation is limited by the rapid process. There are at least three ways in which stakeholder involvement could be improved. First, rapid consultations could be facilitated by establishing stakeholder groups and mechanisms for involvement such as those used by the National Centre for Health and Clinical Excellence. Second, evaluation and updating of the guidelines offer opportunities for more stakeholder involvement in revisions of the guidelines. Third, because the guidelines need to be adapted to specific settings, stakeholders could be involved in local adaptation processes.
Although research needs were identified by the guideline panel, these do not provide clear guidance for what research should be prioritized to address the most important uncertainties about pharmacological management of H5N1 infection. While this is partly due to the mandate that was given to the panel, recommendations for research should be viewed as an integral part of making recommendations.
Conclusion
In summary, we found that it is feasible to develop evidence-based guidelines systematically and transparently in as little as two months. The cost of doing this is prohibitively high for low- and middle-income countries and it would be wasteful for high-income countries to duplicate this process unnecessarily. WHO, or others developing rapid advice, can therefore provide an important service by using a robust and transparent process that simplifies adaptation to specific settings. Further work is required to develop systematic processes for WHO to give even faster or immediate guidance for emerging infectious diseases.
Supporting Information
(25 KB DOC).
(27 KB DOC).
(27 KB DOC).
(26 KB DOC).
(27 KB DOC).
(27 KB DOC).
(26 KB DOC).
(28 KB DOC).
(26 KB DOC).
(27 KB DOC).
(30 KB DOC).
(60 KB DOC).
(34 KB DOC).
(26 KB DOC).
(76 KB PDF).
(112 KB PDF).
(117 KB PDF).
(42 KB PDF).
Acknowledgments
The authors acknowledge the expert administrative support of Julia Brass and Madelyn Clemente. The authors thank Dr. Irene Terrenato and Prof. Paola Muti for help with the translations of the abstract into Italian.
Author contributions. All authors declare that they participated in the development of the methods for rapid advice guidelines and that they have seen and approved the final version of this manuscript. HJS chaired the guideline panel as a voting member, drafted recommendations, contributed to the design and development of the process for this guideline, reviewed the evidence summaries, provided additional evidence, and wrote the first draft of this manuscript. HJS is the guarantor for this article. SRH led the guideline development process, contributed to the design and development of the process for this guideline, reviewed the evidence summaries, provided additional evidence, participated in the guideline panel meeting, critically reviewed early drafts of the manuscript, and approved the final draft of the manuscript. MK prepared and reviewed the evidence summaries, provided additional evidence, participated in the guideline panel meeting, compiled literature, tracked versions of the full guidelines, and critically reviewed and approved the final draft of the manuscript. GEV, LS, and TFW prepared the evidence summaries and critically reviewed and approved the manuscript. RB prepared and reviewed the evidence summaries, provided additional evidence, participated as voting member in the guideline panel meeting, and critically reviewed and approved the final draft of the manuscript. CDM, FH, TMU, JF, YY, JB, TC, TTH, BÖ, and NS reviewed the evidence summaries, provided additional evidence, participated as voting members in the guideline panel meeting, and critically reviewed and approved the final draft of the manuscript. HZ participated in the guideline panel meeting and critically reviewed and approved the final draft of the manuscript. ADO contributed to the design and development of the process of this guideline development, organized and led the evidence review and evidence summary preparation, and critically reviewed and approved the final draft of the manuscript.
Abbreviations
- GRADE
Grading Recommendations Assessment, Development and Evaluation
- WHO
World Health Organization
Footnotes
Holger J. Schünemann is with the Italian National Cancer Institute Regina Elena, Department of Epidemiology, Rome, Italy. Suzanne R. Hill and Howard Zucker are with Health Technology and Pharmaceuticals, World Health Organization, Geneva, Switzerland. Meetali Kakad, Gunn E. Vist, Torbjørn Fosen Wisløff, and Andrew D. Oxman are with the Norwegian Knowledge Centre for the Health Services, Oslo, Norway. Richard Bellamy is with the Department of Infection and Travel Medicine, James Cook University Hospital, Middlesbrough, United Kingdom. Chris Del Mar is on the Faculty of Health Sciences and Medicine, Bond University, Gold Coast, Queensland, Australia. Frederick Hayden is with the Departments of Internal Medicine and Pathology at the University of Virginia Health Sciences Center, Charlottesville, Virginia, United States of America. Lauren Stockman and Timothy M. Uyeki are with the Influenza Division, National Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention, Atlanta, Georgia, United States of America. Jeremy Farrar is with the Oxford University Clinical Research Unit, Hospital for Tropical Diseases, Ho Chi Minh City, Viet Nam. Yazdan Yazdanpanah is with the Service Universitaire des Maladies Infectieuses, Centre Hospitalier de Tourcoing, Faculté de Médecine de Lille, Tourcoing, France. John Beigel is with the National Institutes of Health, Bethesda, Maryland, United States of America. Tawee Chotpitayasunondh is with the Queen Sirikit National Institute of Child Health, Ministry of Public Health, Bangkok, Thailand. Tran Tinh Hien is with the Hospital for Tropical Diseases, Ho Chi Minh City, Viet Nam. Bülent Özbay is head of the Pulmonary Department, Yüzüncü Yil University Van, Van, Turkey. Norio Sugaya is with the Department of Pediatrics, Keio University Faculty of Medicine, Keiyu Hospital, Yokohama-shi, Kanagawa, Japan.
Funding: This guideline project was funded by the World Health Organization.
Competing Interests: Panel members made the following declarations regarding possible conflicts of interest according to WHO rules for the period 2002–2006. HJS received no personal funding, but has received research grants and honoraria deposited into research accounts or given to a research group that he belongs to from Pfizer, Amgen, Roche, and AstraZeneca for development or consulting regarding quality-of-life instruments for chronic respiratory diseases. CDM has received funding for trials in unrelated therapeutic areas, but not in the last four years. FH has received research grants for trials of antivirals and vaccines from Roche, Abbott, BioCrest, and Merck. YY has received no personal funding, but is an investigator on trials with Tibotec Pharmaceutical and has received travel grants from GlaxoSmithKline, Roche, Boehringer Ingelheim, Bristol-Myers Squibb, Pfizer, Abbott, and Gilead to attend scientific meetings. JB has received no personal funding, but is an investigator (through the National Institutes of Health) on trials with Roche and BioCrest. NS has received travel grants from Roche to attend meetings on avian influenza. The remaining members of the panel had no relevant conflicts of interest to declare.
References
- Bradbury J. Storm over WHO-ISH hypertension guidelines. Lancet. 1999;353:563. [Google Scholar]
- Horton R. WHO: The casualties and compromises of renewal. Lancet. 2002;359:1605–1611. doi: 10.1016/S0140-6736(02)08523-9. [DOI] [PubMed] [Google Scholar]
- Laing R, Waning B, Gray A, Ford N, 't Hoen E. 25 years of the WHO essential medicines lists: Progress and challenges. Lancet. 2003;361:1723–1729. doi: 10.1016/S0140-6736(03)13375-2. [DOI] [PubMed] [Google Scholar]
- McCarthy M. Critics slam draft WHO report on homoeopathy. Lancet. 2005;366:705. doi: 10.1016/S0140-6736(05)67159-0. [DOI] [PubMed] [Google Scholar]
- Shaneyfelt TM, Mayo-Smith MF, Rothwangl J. Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA. 1999;281:1900–1905. doi: 10.1001/jama.281.20.1900. [DOI] [PubMed] [Google Scholar]
- Grilli R, Magrini N, Penna A, Mura G, Liberati A. Practice guidelines developed by specialty societies: The need for a critical appraisal. Lancet. 2000;355:103–106. doi: 10.1016/S0140-6736(99)02171-6. [DOI] [PubMed] [Google Scholar]
- Baverstock K. A time to ask what you want of WHO. BMJ. 2003;327:111. [Google Scholar]
- Grol R, Dalhuijsen J, Thomas S, Veld C, Rutten G, et al. Attributes of clinical guidelines that influence use of guidelines in general practice: Observational study. BMJ. 1998;317:858–861. doi: 10.1136/bmj.317.7162.858. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Woolf SH. Practice guidelines: A new reality in medicine. I. Recent developments. Arch Intern Med. 1990;150:1811–1818. [PubMed] [Google Scholar]
- Global Programme on Evidence for Health Policy. Guidelines for WHO guidelines. 2003. World Health Organization. Available: http://whqlibdoc.who.int/hq/2003/EIP_GPE_EQC_2003_1.pdf. Accessed 5 April 2007.
- Guyatt G, Hayward R, Richardson W, Green L, Wilson M, et al. Moving from evidence to action. In: Guyatt G, Rennie D, editors. Users' guide to the medical literature: A manual for evidence-based clinical practice. Chicago: AMA Press; 2002. [Google Scholar]
- Schünemann HJ, Munger H, Brower S, O'Donnell M, Crowther M, et al. Methodology for guideline development for the Seventh American College of Chest Physicians Conference on Antithrombotic and Thrombolytic Therapy: The Seventh ACCP Conference on Antithrombotic and Thrombolytic Therapy. Chest. 2004;126:174S–178S. doi: 10.1378/chest.126.3_suppl.174S. [DOI] [PubMed] [Google Scholar]
- Raine R, Sanderson C, Black N. Developing clinical guidelines: A challenge to current methods. BMJ. 2005;331:631–633. doi: 10.1136/bmj.331.7517.631. [DOI] [PMC free article] [PubMed] [Google Scholar]
- National Institutes for Health and Clinical Excellence. [ www.nice.org.uk]
- Moynihan R, Oxman A, Lavis J, Paulsen E. Evidence-informed health policy: Using research to make health systems healthier. Oslo: Norwegian Knowledge Centre for the Health Services; 2006. [PubMed] [Google Scholar]
- National Heart, Lung, and Blood Institute. Research translation, dissemination, and application—Moving toward a new vision and strategic framework. 2005. Discussion and consensus findings. NHLBI Cardivascular Thought Leaders Meeting; 17 June 2005; Bethesda, Maryland, United States of America. Available: http://www.nhlbi.nih.gov/health/prof/heart/other/cvd_tlead/cvd_tlead.pdf. Accessed 5 April 2007.
- National Institute for Health and Clinical Excellence. Independent experts from World Health Organisation to review NICE clinical guidelines programme. 2006. Available: http://www.nice.org.uk/page.aspx?o=313823. Accessed 5 April 2007.
- Oxman AD, Fretheim A, Schünemann HJ. Improving the use of research evidence in guideline development: Introduction. Health Res Policy Syst. 2006;4:12. doi: 10.1186/1478-4505-4-12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schünemann HJ, Fretheim A, Oxman AD. Improving the use of research evidence in guideline development: 1. Review of guidelines for guidelines. Health Res Policy Syst. 2006;4:13. doi: 10.1186/1478-4505-4-13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schünemann HJ, Hill S, Kakad M, Bellamy R, Uyeki T, et al. WHO Rapid Advice Guidelines for the pharmacological management of sporadic human infection with avian influenza A (H5N1) virus. Lancet Infect Dis. 2007;7:21–31. doi: 10.1016/S1473-3099(06)70684-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- World Health Organization. WHO Rapid Advice Guidelines on pharmacological management of humans infected with avian influenza A (H5N1) virus. 2006. Available: http://www.who.int/medicines/publications/WHO_PSM_PAR_2006.6.pdf. Accessed 5 April 2007.
- Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, et al. Grading quality of evidence and strength of recommendations. BMJ. 2004;328:1490. doi: 10.1136/bmj.328.7454.1490. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schünemann HJ, Jaeschke J, Cook D, Bria WF, Fahy BF, et al. An official ATS statement: Grading the quality of evidence and strength of recommendations for guidelines and recommendations. Am J Respir Crit Care Med. 2006;174:605–614. doi: 10.1164/rccm.200602-197ST. [DOI] [PubMed] [Google Scholar]
- Briss PA. Evidence-based: US road and public-health side of the street. Lancet. 2005;365:828–830. doi: 10.1016/S0140-6736(05)71019-9. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
(25 KB DOC).
(27 KB DOC).
(27 KB DOC).
(26 KB DOC).
(27 KB DOC).
(27 KB DOC).
(26 KB DOC).
(28 KB DOC).
(26 KB DOC).
(27 KB DOC).
(30 KB DOC).
(60 KB DOC).
(34 KB DOC).
(26 KB DOC).
(76 KB PDF).
(112 KB PDF).
(117 KB PDF).
(42 KB PDF).