Abstract
Background: International interest in clinical practice guidelines has never been greater but many published guidelines do not meet the basic quality requirements. There have been renewed calls for validated criteria to assess the quality of guidelines.
Objective: To develop and validate an international instrument for assessing the quality of the process and reporting of clinical practice guideline development.
Methods: The instrument was developed through a multi-staged process of item generation, selection and scaling, field testing, and refinement procedures. 100 guidelines selected from 11 participating countries were evaluated independently by 194 appraisers with the instrument. Following refinement the instrument was further field tested on three guidelines per country by a new set of 70 appraisers.
Results: The final version of the instrument contained 23 items grouped into six quality domains with a 4 point Likert scale to score each item (scope and purpose, stakeholder involvement, rigour of development, clarity and presentation, applicability, editorial independence). 95% of appraisers found the instrument useful for assessing guidelines. Reliability was acceptable for most domains (Cronbach's alpha 0.64–0.88). Guidelines produced as part of an established guideline programme had significantly higher scores on editorial independence and, after the publication of a national policy, had significantly higher quality scores on rigour of development (p<0.005). Guidelines with technical documentation had higher scores on that domain (p<0.0001).
Conclusions: This is the first time an appraisal instrument for clinical practice guidelines has been developed and tested internationally. The instrument is sensitive to differences in important aspects of guidelines and can be used consistently and easily by a wide range of professionals from different backgrounds. The adoption of common standards should improve the consistency and quality of the reporting of guideline development worldwide and provide a framework to encourage international comparison of clinical practice guidelines.
Full Text
The Full Text of this article is available as a PDF (107.1 KB).
Selected References
These references are in PubMed. This may not be the complete list of references from this article.
- Appraisal of Guidelines, Research, and Evaluation in Europe (AGREE) Collaborative Group Guideline development in Europe. An international comparison. Int J Technol Assess Health Care. 2000 Autumn;16(4):1039–1049. doi: 10.1017/s0266462300103101. [DOI] [PubMed] [Google Scholar]
- Bland J. M., Altman D. G. Cronbach's alpha. BMJ. 1997 Feb 22;314(7080):572–572. doi: 10.1136/bmj.314.7080.572. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cluzeau F. A., Littlejohns P. Appraising clinical practice guidelines in England and Wales: the development of a methodologic framework and its application to policy. Jt Comm J Qual Improv. 1999 Oct;25(10):514–521. doi: 10.1016/s1070-3241(16)30465-5. [DOI] [PubMed] [Google Scholar]
- Grilli R., Magrini N., Penna A., Mura G., Liberati A. Practice guidelines developed by specialty societies: the need for a critical appraisal. Lancet. 2000 Jan 8;355(9198):103–106. doi: 10.1016/S0140-6736(99)02171-6. [DOI] [PubMed] [Google Scholar]
- Grimshaw J., Russell I. Achieving health gain through clinical guidelines. I: Developing scientifically valid guidelines. Qual Health Care. 1993 Dec;2(4):243–248. doi: 10.1136/qshc.2.4.243. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Grol R., Dalhuijsen J., Thomas S., Veld C., Rutten G., Mokkink H. Attributes of clinical guidelines that influence use of guidelines in general practice: observational study. BMJ. 1998 Sep 26;317(7162):858–861. doi: 10.1136/bmj.317.7162.858. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hayward R. S., Wilson M. C., Tunis S. R., Bass E. B., Guyatt G. Users' guides to the medical literature. VIII. How to use clinical practice guidelines. A. Are the recommendations valid? The Evidence-Based Medicine Working Group. JAMA. 1995 Aug 16;274(7):570–574. doi: 10.1001/jama.274.7.570. [DOI] [PubMed] [Google Scholar]
- Moher D., Jones A., Lepage L., CONSORT Group (Consolidated Standards for Reporting of Trials) Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. JAMA. 2001 Apr 18;285(15):1992–1995. doi: 10.1001/jama.285.15.1992. [DOI] [PubMed] [Google Scholar]
- Moher D., Pham B., Jones A., Cook D. J., Jadad A. R., Moher M., Tugwell P., Klassen T. P. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet. 1998 Aug 22;352(9128):609–613. doi: 10.1016/S0140-6736(98)01085-X. [DOI] [PubMed] [Google Scholar]
- Moher D., Schulz K. F., Altman D., CONSORT Group (Consolidated Standards of Reporting Trials) The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA. 2001 Apr 18;285(15):1987–1991. doi: 10.1001/jama.285.15.1987. [DOI] [PubMed] [Google Scholar]
- Shaneyfelt T. M., Mayo-Smith M. F., Rothwangl J. Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA. 1999 May 26;281(20):1900–1905. doi: 10.1001/jama.281.20.1900. [DOI] [PubMed] [Google Scholar]
- Shekelle P. G., Woolf S. H., Eccles M., Grimshaw J. Clinical guidelines: developing guidelines. BMJ. 1999 Feb 27;318(7183):593–596. doi: 10.1136/bmj.318.7183.593. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Thomson R., Lavender M., Madhok R. How to ensure that guidelines are effective. BMJ. 1995 Jul 22;311(6999):237–242. doi: 10.1136/bmj.311.6999.237. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Woolf S. H., Grol R., Hutchinson A., Eccles M., Grimshaw J. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ. 1999 Feb 20;318(7182):527–530. doi: 10.1136/bmj.318.7182.527. [DOI] [PMC free article] [PubMed] [Google Scholar]