Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 Jun 3.
Published in final edited form as: Nurs Outlook. 2003 Mar-Apr;51(2):65–69. doi: 10.1016/s0029-6554(02)05459-3

Practice Guidelines and Measurement: State-of-the-Science

Patricia C Dykes 1
PMCID: PMC3107711  NIHMSID: NIHMS259505  PMID: 12712141

Abstract

Background

Restructuring of the health care system has exposed widespread evidence of practice variability and has highlighted the benefits associated with nurses embracing interdisciplinary, best practice solutions to health care delivery. Clinical practice guidelines have emerged as a valuable interdisciplinary evidenced-based tool.

Purpose

This article explores the state of the science of guideline measurement and evaluates the strengths and weaknesses of measurement approaches.

Method

A computerized search of Cumulative Index of Nursing and Allied Health Literature, Health and Psychosocial Instruments, Medline, and PubMed for the search term “practice guidelines” was combined with the following key words: attitudes, adherence, effect, impact, instrument, and measurement.

Discussion

Measurement issues identified in this analysis are related to the manner in which guidelines are written and the lack of a standard methodology for measurement.

Conclusions

The challenge remains to establish sound measures of adherence and impact while controlling for confounding variables. Questions remain as to the format of practice guidelines to best grant autonomy while offering recommendations that are clear and measurable.

The restructuring of the health care system during the past 20 years has spurred an interest in identifying and implementing practices associated with quality and desired clinical and cost outcomes. Much of this focus has highlighted widespread evidence of practice variability, even within specialized domains.1-6 This understanding has led to an interest in identifying best practice and applying those patterns known to improve patient outcomes. The current health care environment presents an unremitting challenge to health care organizations and providers with an interest in improving effectiveness of care. Competition in health care brought about by cost constraints and increasing demands of patients, payers, and accreditors is guiding how health care is delivered and documented.7 Providers are called on to document the efficacy of interventions from multiple perspectives, including clinical, financial, and consumer satisfaction. This broadening of perspective compels nurses to champion an interdisciplinary, evidence-based approach to patient care that integrates research findings and tailors those findings to meet the needs of targeted populations. In essence, traditional practice patterns are called into question as pressure mounts to include evidence-based practice as a means of demonstrating the efficacy of care.

Evidence-based practice has been defined as the integration of “individual clinical expertise with the best available external clinical evidence from systematic research.”8 One prominent strategy for introducing evidence-based practice is the implementation of clinical practice guidelines. Clinical practice guidelines are defined by the Institute of Medicine as “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances” (see http://www.iom.edu/IOM/IOMHome.nsf). Much evidence exists that supports the relationship between practitioner adherence to guidelines and positive clinical and financial outcomes.9-17 However, studies1-6 to date demonstrate that practitioner adherence with core recommendations within practice guidelines is inconsistent and highly variable.

Given the limited resources available in the health care marketplace and existing evidence regarding the association between adherence with practice guidelines and positive patient and cost outcomes, strategies to objectively quantify clinician adherence to guidelines and the impact of guideline adherence are desirable. To identify existing measures related to practice guidelines and measurement of attitudes, guideline adherence, and the impact of practice guidelines, a computerized search of the Cumulative Index of Nursing and Allied Health Literature, Health and Psychosocial Instruments, Medline, and PubMed was conducted. The search term “practice guidelines” was combined with the following key words: attitudes, adherence, effect, impact, instrument, and measurement. Initially, more than 300 references were identified in this search. Preference was given to primary sources and articles that specifically used a measurement technique or tool in evaluating variables related to practice guidelines. In addition, selected articles detailed the method of inquiry used to evaluate pertinent variables. Articles that simply described the use of guidelines in practice without measuring some variable related to those guidelines were eliminated. Although there were many articles published about the use of guidelines within and related to nursing practice, few focused on measurement or use of a measurement tool in their evaluation process and thus were not included in this article.

Under these more restrictive criteria, 47 references—published between 1994 and 2002 in 41 journals—were identified. The majority was found in specialized medical journals, but several included nurses as first authors18,19 or members of the research team and co-authors.20,21 The authors were largely health services researchers and clinicians affiliated with major medical and research institutions. Much of the work was funded by public or private health services research grants. The trend in the number of published research articles that addressed measurement issues related to practice guidelines was upward, from 3 in 1994 to 13 in 2001. The instruments or measurement techniques described in the aforementioned articles generally fell into the following 3 categories, according to what they were designed to measure: clinician adherence, the impact of guidelines on cost or patient outcomes, or a combination of variables.

CLINICIAN ADHERENCE

The majority of the articles identified describe techniques or tools to measure clinician adherence with practice guideline recommendations. For the purposes of this article, guideline adherence is defined as a condition in which the prescribed treatment, as noted by self-report questionnaire or by data in the clinical data base, conforms to treatment recommended in the identified practice guidelines. The most common technique used to study clinician adherence was a descriptive study that used a cross-sectional or retrospective chart review to measure adherence with practice guideline recommendations.22-27 Another method used was the before-and-after observational study design with or without a concurrent control group.19,28 In these studies, clinician adherence was measured by documentation in the clinical data base before and after implementation of an intervention designed to promote guideline adherence. In 1 study,29 referral guidelines were randomly distributed to general dental practitioners, and referral practices were monitored after guideline distribution. In each of these studies, specific guideline recommendations served as measurement criteria for adherence. With the exception of the randomized controlled trials, all these studies lacked sufficient control to allow the researchers to draw any definitive conclusions from their findings. The lack of randomization to a group and absence of a control group made selection bias a key threat to validity. In addition, the observational studies did not control for risk factors or other variables related to patient status that might affect adherence. Finally, these studies did not control for the effect of time on adherence to guidelines. Because adherence can change during the course of a study, it is important to address this variable to get an unbiased estimate of adherence.30 Although measurement issues exist in many of the studies of adherence, they are useful for performance improvement purposes, in which the results will be used to improve practices within an organization, or to serve as pilot work or the foundation for larger, more controlled studies.

The most common type of instrument used to measure clinician adherence with guidelines was the self-report questionnaire.20,31-37 In this sample, response rates ranged from 31% to 66%. In every case, multiple mailings and other strategies were implemented to improve the rate of response. Most investigators reported the inability to use some data due to incomplete questionnaires.

In addition to clinician self-report questionnaires, adherence was measured in this sample with physician questionnaire and chart review38 and patient report questionnaire.39 Three articles28,40,41 in this sample described the process and associated difficulties of the use of practice guidelines for analysis of adherence to evidence-based clinical guidelines. Most of the difficulties are related to the general nature of practice guideline recommendations and identifying objective measures of adherence to these recommendations.

IMPACT ON PATIENT OUTCOMES

Measurement of the impact of practice guidelines on patient outcomes has had a delayed appearance in the literature. At this time, there is no one standard instrument that measures the impact of practice guidelines on patient outcomes. Impact is generally evaluated by investigating multiple clinical and cost outcomes before and after guideline implementation or by comparing the outcomes of 2 groups of patients who have been randomized to guideline-based interventions or usual care. Two randomized control trials,42,43 3 quasiexperimental designs,3,44,45 and 3 observational studies21,46,47 were identified that investigated the impact of practice guidelines on clinical and cost outcomes. These studies generally measured clinician adherence with practice guidelines and compared patient outcomes in the experimental verses control groups. Two additional articles30,48 were identified that examined methodologic issues related to evaluating the impact of practice guidelines on patient outcomes.

MEASUREMENT OF MULTIPLE VARIABLES

A number of articles were identified that described techniques or tools to measure multiple variables—the most common being attitudes regarding practice guidelines and self-reported use of these guidelines.18,49,50 Other variables measured include knowledge, skills, decision-making, and barriers to guideline use.49,51-53 Four recent articles49,52-54 described the use of a randomized control trial to evaluate these variables. One study54 used a quasiexperimental method, and 5 studies used a survey method to evaluate outcome variables related to practice guidelines.18,51,55-57. Each study in this category used a self-report questionnaire to explore the variables under consideration after the dissemination of practice guidelines. Individuals were randomized to an intervention designed to provide education related to guideline content and specific recommendations or to a control group that received the guideline without a follow-up educational intervention. In addition to the self-report survey, the randomized control trials also used other methods to assess these variables, such as chart review, referral pattern records, and clinical scenarios with questions to evaluate decision-making. The findings of these studies suggest that dissemination of guidelines coupled with interventions to increase awareness of content and specific recommendations is associated with improved decision-making skills and referral patterns.49,50,53

The advantages of a survey questionnaire are its efficiency and ease with which it can be distributed to large numbers of geographically distant participants.58 Disadvantages of survey questionnaires are related to characteristic low response rates and the inability of researchers to follow-up with participants to ensure questions are adequately interpreted and completed in full.58 In this sample, the response rate ranged from 22% to 65%. In every case, despite multiple mailings and other strategies implemented to improve the rate of response, incomplete data on returned surveys resulted in limiting the sample size.

The sources identified in this article acknowledge many unresolved issues related to measuring adherence to practice guidelines and the impact of guidelines on practice patterns and patient outcomes.

An issue that emerged regarding measurement of outcome variables related to practice guidelines was the number of available guidelines and the differences in guideline focus and content (even within disciplines). There is a general lack of agreement regarding what should be measured. However, the references identified for this article demonstrate consensus that the ability to measure adherence with guideline recommendations and the impact of those guidelines on practice patterns and patient outcomes is vital. Tools and techniques designed to evaluate these variables are increasingly prevalent in the literature. The recent appearance of articles that identify and examine methodologic issues related to evaluating the impact of practice guidelines on practice patterns and patient outcomes30,48 is encouraging. Recognition is the first step toward setting a standard and assisting clinicians and researchers with designing reliable and valid tools to measure these outcome variables.

MEASUREMENT ISSUES AND RECOMMENDATIONS RELATED TO PRACTICE GUIDELINES

The sources identified in this article acknowledge many unresolved issues related to measuring adherence to practice guidelines and the impact of guidelines on practice patterns and patient outcomes. These issues fall into the following 2 general categories: issues related to the manner in which guidelines are written and issues related to the lack of a standard method for measurement of adherence and impact of practice guidelines. The intent of clinical practice guidelines is to offer assistance or guidance to clinicians for specific clinical situations, rather than to firmly dictate practice (see http://www.iom.edu/IOM/IOMHome.nsf). As a result, many guidelines lack explicit recommendations that can be implemented into objective measurement criteria. This inserts a degree of uncertainty into interpretation of recommendations and, in some cases, leaves interpretation up to individuals. At present, the strength of guideline recommendations is rated on the basis of the source of supporting evidence. Some consensus is needed regarding the implementation of recommendations and establishing with whom the responsibility lies for this task. The advantage of granting this responsibility to nationally recognized professional organizations is that standard criteria for measurement would be developed to promote consistent measurement across settings and that the data would be aggregated and benchmarked. The disadvantage of this method is the prospect of alienating clinicians who may interpret such recommendations as dictation of practice.

Marshall et al30 categorized the prominent issues regarding measuring the impact of practice guidelines on clinician practice and patient outcomes into 5 areas of complexity (Table 1). These issues underscore the major difficulties identified in the literature regarding measuring impact. Impact is not a single measurement but requires the use of multiple and often complex assessments and measures. To accurately capture the impact of guidelines on practice patterns and patient outcomes, clinician adherence also must be measured. Careful consideration of the length of time from guideline distribution to integration into workflow and practice patterns is necessary. Statistical methods must be applied to control for time sensitivity of interventions. For example, how does one control for the effect of clinician adherence over time on outcomes of a group of patients who may have been exposed to variable degrees of clinician adherence? Attention also is needed regarding the multiple variables that may affect integration and adherence. An operational definition for guideline adherence, tailored to a specific guideline and setting, would promote accurate measurement of adherence and the impact of adherence on patient outcomes. Length of exposure to guideline adherence could have a significant effect on patient outcomes and would likely impact data analysis and findings. Because practice guidelines are generally diagnosis- and discipline-specific, it is doubtful that 1 tool could be developed to universally measure impact and control for all necessary variables. It is possible that specific methodologic issues identified in preliminary work will facilitate the development of a template that could be used to generate multiple diagnosis-specific tools that consider these issues and adequately capture the impact of guideline adherence on practice patterns and patient outcomes.

Table 1.

Issues Related to the Measurement of Impact of Practice Guidelines

Area of Concern Issue
1. Selection Of Patients

  • Do guidelines apply to all patients with a particular problem or diagnosis?

  • How does comorbid diagnosis affect selection?

  • What measures are taken to avoid selection bias?
2. Baseline Risk

  • How does one control for level of risk?

  • What is the impact of comorbid diagnosis on data collection and measurement?
3. Duration Models For The Outcomes Of Interest

  • What statistical techniques are applied to analysis to control for the time sensitivity of specific interventions?

  • What was the duration of adherence and how is this factored into analysis?
4. Time-Varying Models For The Independent Variables

  • Did adherence to guideline recommendations change over time? How is this captured?
5. Exposure

  • Measures of exposure for guideline adherence

  • What is the minimum period of time that adherence needs to be present after which the “benefits” of adherence become apparent?
Open in a new tab

Adapted from: Marshall DA, Simpson KN, Norton EC, Biddle AK, Youle M. Measuring the effect of clinical guidelines on patient outcomes. Int J Technol Assess Health Care 2000;16:1015-6.

Some consensus is needed regarding the implementation of recommendations and establishing with whom the responsibility lies for this task.

Another important issue identified in this exercise is the general disregard for the impact of nurses and other disciplines on patient outcomes in existing measures. With the exception of the Nursing Guidelines Interview Questionnaire, all the instruments identified in this computerized search measured outcomes related to physician and primary care provider attitudes, adherence, and practice and associated patient outcomes. The interdisciplinary nature of practice guidelines demands an instrument that is capable of capturing and measuring outcomes that are often interdisciplinary in nature. This issue has not yet been addressed in the literature and will need to be resolved before a useful and generalizable methodology is adopted.

The measurement of clinician adherence to practice guidelines and the impact of those guidelines on practice patterns and patient outcomes remain areas of great interest in health care. With limited health care resources and evidence that decreased practice variation is associated with improved patient outcomes at reduced costs, this trend will likely continue. The challenge for clinicians and researchers is to establish sound measures of adherence and impact and methods of controlling for confounding variables. In addition, questions remain as to the most useful format of practice guidelines to grant both sufficient clinician autonomy and recommendations that are clear and measurable.

Acknowledgments

The author wishes to thank Elaine Larson, RN, PhD, EAAN, CIC for her assistance with this manuscript.

References

  • 1.Lin EH, Katon WJ, Simon GE, et al. Achieving guidelines for the treatment of depression in primary care: is physician education enough? Med Care. 1997;35:831–42. doi: 10.1097/00005650-199708000-00008. [DOI] [PubMed] [Google Scholar]
  • 2.Mor V, Laliberte LL, Petrisek AC, et al. Impact of breast cancer treatment guidelines on surgeon practice patterns: results of a hospital-based intervention. Surgery. 2000;128:847–61. doi: 10.1067/msy.2000.109530. [DOI] [PubMed] [Google Scholar]
  • 3.Roghmann MC, Sexton M. Adherence to asthma guidelines in general practices. J Asthma. 1999;36:381–7. doi: 10.3109/02770909909068231. [DOI] [PubMed] [Google Scholar]
  • 4.Srinivasan R, Fisher RS. Early initiation of post-PEG feeding: do published recommendations affect clinical practice? Digest Dis Sci. 2000;45:2065–8. doi: 10.1023/a:1005552227119. [DOI] [PubMed] [Google Scholar]
  • 5.Taylor DM, Auble TE, Calhoun WJ, Mosesso VN., Jr Current outpatient management of asthma shows poor compliance with International Consensus Guidelines. Chest. 1999;116:1638–45. doi: 10.1378/chest.116.6.1638. [DOI] [PubMed] [Google Scholar]
  • 6.Wang PS, Berglund P, Kessler RC. Recent care of common mental disorders in the United States: prevalence and conformance with evidence-based recommendations. J Gen Intern Med. 2000;15:284–92. doi: 10.1046/j.1525-1497.2000.9908044.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.McMahon LF, Jr, Edward AM, Bernard AM, et al. The integrated inpatient management model’s clinical management information system. Hosp Health Serv Admin. 1994;39:81–92. [PubMed] [Google Scholar]
  • 8.Sackett DL, Rosenberg WM. On the need for evidence-based medicine. J Public Health Med. 1995;17:330–4. [PubMed] [Google Scholar]
  • 9.Tai-Seale M, Croghan TW, Obenchain R. Determinants of antidepressant treatment compliance: implications for policy. Med Care Res Rev. 2000;57:491–512. doi: 10.1177/107755870005700405. [DOI] [PubMed] [Google Scholar]
  • 10.Sood N, Treglia M, Obenchain RL, et al. Determinants of antidepressant treatment outcome. Am J Manage Care. 2000;6:1327–36. [PubMed] [Google Scholar]
  • 11.Schulberg HC, Katon W, Simon GE, Rush AJ. Treating major depression in primary care practice: an update of the Agency for Health Care Policy and Research Practice Guidelines. Arch Gen Psychiatry. 1998;55:1121–7. doi: 10.1001/archpsyc.55.12.1121. [DOI] [PubMed] [Google Scholar]
  • 12.Mancuso CA. Impact of new guidelines on physicians’ ordering of preoperative tests. J Gen Intern Med. 1999;14:166–72. doi: 10.1046/j.1525-1497.1999.00308.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Marrie TJ, Lau CY, Wheeler SL, et al. A controlled trial of a critical pathway for treatment of community-acquired pneumonia. CAPITAL Study Investigators. Community-acquired pneumonia intervention trial assessing levofloxacin. JAMA. 2000;283:749–55. doi: 10.1001/jama.283.6.749. [DOI] [PubMed] [Google Scholar]
  • 14.Frankel HL, FitzPatrick MK, Gaskell S, et al. Strategies to improve compliance with evidence-based clinical management guidelines. J Am Coll Surg. 1999;189:533–8. doi: 10.1016/s1072-7515(99)00222-7. [DOI] [PubMed] [Google Scholar]
  • 15.Fiel S. Guidelines and critical pathways for severe hospital-acquired pneumonia. Chest. 2001;119(2 Suppl):412S–418S. doi: 10.1378/chest.119.2_suppl.412s. [DOI] [PubMed] [Google Scholar]
  • 16.Feely J. The therapeutic gap—compliance with medication and guidelines. Atherosclerosis. 1999;147(Suppl 1):S31–7. doi: 10.1016/s0021-9150(99)00253-1. [DOI] [PubMed] [Google Scholar]
  • 17.Engstrom C, Hernandez I, Haywood J, Lilenbaum R. The efficacy and cost effectiveness of new antiemetic guidelines. Oncol Nurs Forum. 1999;26:1453–8. [PubMed] [Google Scholar]
  • 18.Lia-Hoagberg B, Schaffer M, Strohschein S. Public health nursing practice guidelines: an evaluation of dissemination and use. Public Health Nurse. 1999;16:397–404. doi: 10.1046/j.1525-1446.1999.00397.x. [DOI] [PubMed] [Google Scholar]
  • 19.Andrews JO, Tingen MS, Waller JL, Harper RJ. Provider feedback improves adherence with AHCPR smoking cessation guideline. Prevent Med. 2001;33:415–21. doi: 10.1006/pmed.2001.0907. [DOI] [PubMed] [Google Scholar]
  • 20.LaClair BJ, Reker DM, Duncan PW, et al. Stroke care: a method for measuring compliance with AHCPR guidelines. Am J Phys Med Rehabil. 2001;80:235–42. doi: 10.1097/00002060-200103000-00017. [DOI] [PubMed] [Google Scholar]
  • 21.Pilon CS, Leathley M, London R, et al. Practice guideline for arterial blood gas measurement in the intensive care unit decreases numbers and increases appropriateness of tests. Crit Care Med. 1997;25:1308–13. doi: 10.1097/00003246-199708000-00016. [DOI] [PubMed] [Google Scholar]
  • 22.van der Weijden T, Hutten JB, Brandenburg BJ, et al. Cholesterol management in Dutch general practice. A comparison with national guidelines. Dutch College of General Practitioners. Scand J Primary Health Care. 1994;12:281–8. doi: 10.3109/02813439409029254. [DOI] [PubMed] [Google Scholar]
  • 23.Maviglia SM, Teich JM, Fiskio J, Bates DW. Using an electronic medical record to identify opportunities to improve compliance with cholesterol guidelines. J Gen Intern Med. 2001;16:531–7. doi: 10.1046/j.1525-1497.2001.016008531.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Lim PZ, Tunis SL, Edell WS, et al. Medication prescribing patterns for patients with bipolar I disorder in hospital settings: adherence to published practice guidelines. Bipolar Disorder. 2001;3:165–73. [PubMed] [Google Scholar]
  • 25.Kaplan JE, Parham DL, Soto-Torres L, et al. Adherence to guidelines for antiretroviral therapy and for preventing opportunistic infections in HIV-infected adults and adolescents in Ryan White-funded facilities in the United States. J Acquired Immune Defic Syndrome. 1999;21:228–35. doi: 10.1097/00126334-199907010-00008. [DOI] [PubMed] [Google Scholar]
  • 26.Glasgow RE, Boles SM, Calder D, et al. Diabetes care practices in primary care: results from two samples and three measurement sets. Diabetes Educ. 1999;25:755–63. doi: 10.1177/014572179902500508. [DOI] [PubMed] [Google Scholar]
  • 27.Chodoff P, Bischof R, Nash D, Laine C. The AHCPR guidelines on heart failure: comparison of a family medicine and an internal medicine practice with the guidelines and an educational intervention to modify behavior. Clin Perform Qual Health Care. 1996;4:179–85. [PubMed] [Google Scholar]
  • 28.James PA, Cowan TM, Graham RP, et al. Using a clinical practice guideline to measure physician practice: translating a guideline for the management of heart failure. J Am Board Fam Pract. 1997;10:206–12. [PubMed] [Google Scholar]
  • 29.O’Brien K, Wright J, Conboy F, et al. The effect of orthodontic referral guidelines: a randomised controlled trial. Br Dentist J. 2000;188:392–7. doi: 10.1038/sj.bdj.4800492. [DOI] [PubMed] [Google Scholar]
  • 30.Marshall DA, Simpson KN, Norton EC, et al. Measuring the effect of clinical guidelines on patient outcomes. IntJ Technol Assess Health Care. 2000;16:1013–23. doi: 10.1017/s0266462300103083. [DOI] [PubMed] [Google Scholar]
  • 31.Di Iorio D, Henley E, Doughty A. A survey of primary care physician practice patterns and adherence to acute low back problem guidelines. Arch Fam Med. 2000;9:1015–21. doi: 10.1001/archfami.9.10.1015. [DOI] [PubMed] [Google Scholar]
  • 32.Dickerson JE, Garratt CJ, Brown MJ. Management of hypertension in general practice: agreements with and variations from the British Hypertension Society guidelines. J Human Hypertension. 1995;9:835–9. [PubMed] [Google Scholar]
  • 33.Hermens RP, Hak E, Hulscher ME, et al. Adherence to guidelines on cervical cancer screening in general practice: program elements of successful implementation. Br J Gen Pract. 2001;51:897–903. [PMC free article] [PubMed] [Google Scholar]
  • 34.Legorreta AP, Christian-Herman J, O’Connor RD, et al. Compliance with national asthma management guidelines and specialty care: a health maintenance organization experience. Arch Intern Med. 1998;158:457–64. doi: 10.1001/archinte.158.5.457. [DOI] [PubMed] [Google Scholar]
  • 35.Meng YY, Leung KM, Berkbigler D, et al. Compliance with US asthma management guidelines and specialty care: a regional variation or national concern? J Eval Clin Pract. 1999;5:213–21. doi: 10.1046/j.1365-2753.1999.00177.x. [DOI] [PubMed] [Google Scholar]
  • 36.Pathman DE, Konrad TR, Freed GL, et al. The awareness-to-adherence model of the steps to clinical guideline compliance. The case of pediatric vaccine recommendations. Med Care. 1996;34:873–89. doi: 10.1097/00005650-199609000-00002. [DOI] [PubMed] [Google Scholar]
  • 37.Finkelstein JA, Lozano P, Shulruff R, et al. Self-reported physician practices for children with asthma: are national guidelines followed? Pediatrics. 2000;106(4 Suppl):886–96. [PubMed] [Google Scholar]
  • 38.Halm EA, Atlas SJ, Borowsky LH, et al. Understanding physician adherence with a pneumonia practice guideline: effects of patient, system, and physician factors. Arch Intern Med. 2000;160:98–104. doi: 10.1001/archinte.160.1.98. [DOI] [PubMed] [Google Scholar]
  • 39.Kogan MD, Alexander GR, Kotelchuck M, et al. Comparing mothers’ reports on the content of prenatal care received with recommended national guidelines for care. Public Health Rep. 1994;109:637–46. [PMC free article] [PubMed] [Google Scholar]
  • 40.Metfessel BA. An automated tool for an analysis of compliance to evidence-based clinical guidelines. Medinfo. 2001;10(Pt 1):226–30. [PubMed] [Google Scholar]
  • 41.Tremaine WJ, Sandborn WJ. Practice guidelines for inflammatory bowel disease: an instrument for assessment. Mayo Clin Proceed. 1999;74:495–501. doi: 10.4065/74.5.495. [DOI] [PubMed] [Google Scholar]
  • 42.Naughton BJ, Mylotte JM, Ramadan F, et al. Antibiotic use, hospital admissions, and mortality before and after implementing guidelines for nursing home-acquired pneumonia. J Am Geriatr Soc. 2001;49:1020–4. doi: 10.1046/j.1532-5415.2001.49203.x. [DOI] [PubMed] [Google Scholar]
  • 43.Rossignol M, Abenhaim L, Seguin P, et al. Coordination of primary health care for back pain. A randomized controlled trial. Spine. 2000;25:251–8. doi: 10.1097/00007632-200001150-00018. discussion 258-9. [DOI] [PubMed] [Google Scholar]
  • 44.Bailey R, Weingarten S, Lewis M, Mohsenifar Z. Impact of clinical pathways and practice guidelines on the management of acute exacerbations of bronchial asthma. Chest. 1998;113:28–33. doi: 10.1378/chest.113.1.28. [DOI] [PubMed] [Google Scholar]
  • 45.Yates S, Annis L, Pippins J, Walden S. Does a lipid clinic increase compliance with National Cholesterol Education Program Treatment Guidelines? Report of a case-matched controlled study. South Med J. 2001;4:907–9. [PubMed] [Google Scholar]
  • 46.Mostafa G, Sing RF, Matthews BD, et al. The economic benefit of practice guidelines for stress ulcer prophylaxis. Am Surg. 2002;68:146–50. [PubMed] [Google Scholar]
  • 47.Vernon SA, Ghosh G. Do locally agreed guidelines for optometrists concerning the referral of glaucoma suspects influence referral practice? Eye. 2001;15(Pt 4):458–63. doi: 10.1038/eye.2001.155. [DOI] [PubMed] [Google Scholar]
  • 48.Eytan TA, Goldberg HI. How effective is the computer-based clinical practice guideline? Effect Clin Pract. 2001;4:24–33. [PubMed] [Google Scholar]
  • 49.Kerry S, Oakeshott P, Dundas D, Williams J. Influence of postal distribution of the Royal College of Radiologists’ guidelines, together with feedback on radiological referral rates, on x-ray referrals from general practice: a randomized controlled trial. Fam Pract. 2000;17:46–52. doi: 10.1093/fampra/17.1.46. [DOI] [PubMed] [Google Scholar]
  • 50.Smeele IJ, Grol RP, van Schayck CP, et al. Can small group education and peer review improve care for patients with asthma/chronic obstructive pulmonary disease? Qual Health Care. 1999;8:92–8. doi: 10.1136/qshc.8.2.92. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Tunis SR, Hayward RS, Wilson MC, et al. Internists’ attitudes about clinical practice guidelines. Ann Intern Med. 1994;120:956–63. doi: 10.7326/0003-4819-120-11-199406010-00008. [DOI] [PubMed] [Google Scholar]
  • 52.Gifford DR, Holloway RG, Frankel MR, et al. Improving adherence to dementia guidelines through education and opinion leaders. A randomized, controlled trial. Ann Inter Med. 1999;131:237–46. doi: 10.7326/0003-4819-131-4-199908170-00002. [DOI] [PubMed] [Google Scholar]
  • 53.Holloway RG, Gifford DR, Frankel MR, Vickrey BG. A randomized trial to implement practice recommendations: design and methods of the Dementia Care Study. Control Clin Trials. 1999;20:369–85. doi: 10.1016/s0197-2456(99)00006-9. [DOI] [PubMed] [Google Scholar]
  • 54.Chen RS, Rosenheck R. Using a computerized patient database to evaluate guideline adherence and measure patterns of care for major depression. J Behav Health Serv Res. 2001;28:466–74. doi: 10.1007/BF02287776. [DOI] [PubMed] [Google Scholar]
  • 55.Flores G, Lee M, Bauchner H, Kastner B. Pediatricians’ attitudes, beliefs, and practices regarding clinical practice guidelines: a national survey. Pediatrics. 2000;105(3 Pt 1):496–501. doi: 10.1542/peds.105.3.496. [DOI] [PubMed] [Google Scholar]
  • 56.Watkins C, Harvey I, Langley C, et al. General practitioners’ use of guidelines in the consultation and their attitudes to them. Br J Gen Pract. 1999;49:11–15. [PMC free article] [PubMed] [Google Scholar]
  • 57.Crain EF, Weiss KB, Fagan MJ. Pediatric asthma care in US emergency departments. Current practice in the context of the National Institutes of Health Guidelines. Arch Pediatr Adolesc Med. 1995;149:893–901. doi: 10.1001/archpedi.1995.02170210067012. [DOI] [PubMed] [Google Scholar]
  • 58.Waltz CF, Strickland O, Lenz ER. Measurement in nursing research. 2. Philadelphia: FA Davis; 1991. [Google Scholar]

RESOURCES