Abstract
Objectives—To identify the desirable characteristics of review criteria for quality improvement and to determine how they should be selected.
Background—Review criteria are the elements against which quality of care is assessed in quality improvement. Use of inappropriate criteria may impair the effectiveness of quality improvement activities and resources may be wasted in activities that fail to facilitate improved care.
Methods—A two round modified Delphi process was used to generate consensus amongst an international panel of 38 experts. A list of 40 characteristics of review criteria, identified from literature searches, was distributed to the experts who were asked to rate the importance and feasibility of each characteristic. Comments and suggestions for characteristics not included in the list were also invited.
Results—The Delphi process refined a comprehensive literature based list of 40 desirable characteristics of review criteria into a more precise list of 26 items. The expert consensus view is that review criteria should be developed through a well documented process involving consideration of valid research evidence, possibly combined with expert opinion, prioritisation according to health outcomes and strength of evidence, and pilot testing. Review criteria should also be accompanied by full clear information on how they might be used and how data might be collected and interpreted.
Conclusion—The desirable characteristics for review criteria have been identified and will be of use in the development, evaluation, and selection of review criteria, thus improving the cost effectiveness of quality improvement activities in healthcare settings.
Key Words: review criteria; Delphi process; audit criteria; quality improvement
Full Text
The Full Text of this article is available as a PDF (135.3 KB).
Selected References
These references are in PubMed. This may not be the complete list of references from this article.
- Ashton C. M., Kuykendall D. H., Johnson M. L., Wun C. C., Wray N. P., Carr M. J., Slater C. H., Wu L., Bush G. R. A method of developing and weighting explicit process of care criteria for quality assessment. Med Care. 1994 Aug;32(8):755–770. doi: 10.1097/00005650-199408000-00001. [DOI] [PubMed] [Google Scholar]
- Bateman D. N., Eccles M., Campbell M., Soutter J., Roberts S. J., Smith J. M. Setting standards of prescribing performance in primary care: use of a consensus group of general practitioners and application of standards to practices in the north of England. Br J Gen Pract. 1996 Jan;46(402):20–25. [PMC free article] [PubMed] [Google Scholar]
- Bozzette S. A., Asch S. Developing quality review criteria from standards of care for HIV disease: a framework. J Acquir Immune Defic Syndr Hum Retrovirol. 1995;8 (Suppl 1):S45–S52. [PubMed] [Google Scholar]
- Brook R. H., Chassin M. R., Fink A., Solomon D. H., Kosecoff J., Park R. E. A method for the detailed assessment of the appropriateness of medical technologies. Int J Technol Assess Health Care. 1986;2(1):53–63. doi: 10.1017/s0266462300002774. [DOI] [PubMed] [Google Scholar]
- Campbell S. M., Roland M. O., Shekelle P. G., Cantrill J. A., Buetow S. A., Cragg D. K. Development of review criteria for assessing the quality of management of stable angina, adult asthma, and non-insulin dependent diabetes mellitus in general practice. Qual Health Care. 1999 Mar;8(1):6–15. doi: 10.1136/qshc.8.1.6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Eccles M., Clapp Z., Grimshaw J., Adams P. C., Higgins B., Purves I., Russell I. North of England evidence based guidelines development project: methods of guideline development. BMJ. 1996 Mar 23;312(7033):760–762. doi: 10.1136/bmj.312.7033.760. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fraser R. C., Khunti K., Baker R., Lakhani M. Effective audit in general practice: a method for systematically developing audit protocols containing evidence-based review criteria. Br J Gen Pract. 1997 Nov;47(424):743–746. [PMC free article] [PubMed] [Google Scholar]
- Hadorn D. C., Baker D. W., Kamberg C. J., Brooks R. H. Phase II of the AHCPR-sponsored heart failure guideline: translating practice recommendations into review criteria. Jt Comm J Qual Improv. 1996 Apr;22(4):265–276. doi: 10.1016/s1070-3241(16)30230-9. [DOI] [PubMed] [Google Scholar]
- James P. A., Cowan T. M., Graham R. P., Majeroni B. A., Fox C. H., Jaén C. R. Using a clinical practice guideline to measure physician practice: translating a guideline for the management of heart failure. J Am Board Fam Pract. 1997 May-Jun;10(3):206–212. [PubMed] [Google Scholar]
- Johnson N. Feasibility of developing and selecting criteria for the assessment of clinical performance. Br J Gen Pract. 1993 Dec;43(377):499–502. [PMC free article] [PubMed] [Google Scholar]
- Lanska D. A public/private partnership in the quest for quality: development of cerebrovascular disease practice guidelines and review criteria. Am J Med Qual. 1995 Summer;10(2):100–106. doi: 10.1177/0885713X9501000207. [DOI] [PubMed] [Google Scholar]
- McKnight J., Edwards N., Pickard L., Underwood J., Voorberg N., Woodcox V. The Delphi approach to strategic planning. Nurs Manage. 1991 Apr;22(4):55–57. doi: 10.1097/00006247-199104000-00015. [DOI] [PubMed] [Google Scholar]
- Murphy M. K., Black N. A., Lamping D. L., McKee C. M., Sanderson C. F., Askham J., Marteau T. Consensus development methods, and their use in clinical guideline development. Health Technol Assess. 1998;2(3):i-iv, 1-88. [PubMed] [Google Scholar]
- Naylor C. D., Guyatt G. H. Users' guides to the medical literature. XI. How to use an article about a clinical utilization review. Evidence-Based Medicine Working Group. JAMA. 1996 May 8;275(18):1435–1439. doi: 10.1001/jama.275.18.1435. [DOI] [PubMed] [Google Scholar]
- Shekelle P. G., Kahan J. P., Bernstein S. J., Leape L. L., Kamberg C. J., Park R. E. The reproducibility of a method to identify the overuse and underuse of medical procedures. N Engl J Med. 1998 Jun 25;338(26):1888–1895. doi: 10.1056/NEJM199806253382607. [DOI] [PubMed] [Google Scholar]