Abstract
A variety of computer-based applications, including computerized clinical reminders, are intended to increase adherence to evidence-based clinical guidelines. The value of these systems in clinical practice is still unclear. One reason for the limited success of clinical reminders may be physicians’ low tendency to adhere to their advice. We studied the determinants of physicians’ adherence to clinical advice regarding the management of dyslipidemia. Overall, the clinical reminders increased physicians’ adherence to the clinical guidelines. Physicians were more compliant with the reminders when they experienced a greater patients’ load, when they were less acquainted with the patient, and when more time has passed since the last major cardiac event. These findings can help to predict physicians’ adherence and to improve the usage of clinical reminders for the benefit of patients, physicians and HMOs.
Introduction
Adherence to evidence-based clinical guidelines is widely considered to be a clinically important goal. Yet there is a large gap between what such guidelines promise and what they deliver1. Although risk factors contributing to the development of Coronary Artery Disease (CAD) are well known and effective interventions exist, the majority of patients are suboptimally treated2–4.
Clinical Reminders (CR) are a technological solution to promote physicians’ adherence with the clinical guidelines. Many studies have shown positive effects on clinical management and outcomes5–14, while others pointed to the systems being only marginally useful in clinical practice, and having only limited and variable effects15. CRs impact on diagnosis and patient outcomes were not clear5, their benefits appear to deteriorate over time12, and there is significant variation between clinics and physicians in adherence to CRs16. It seems that understanding the determinants of physicians’ adherence to CRs can contribute to improved intervention systems.
Only few studies examined the factors affecting physicians’ adherence to CR systems. Patterson and her colleagues17–20 reported barriers to effective use of clinical reminders. Issues mentioned include physicians’ workload, time to remove inapplicable reminders, false alarms, lack of training, reduced eye contact with the patient, the use of paper forms rather than software, ease of use issues, and accessibility of workstations. Militello19 provided a strong indication that workload and time constraints constitute major barriers to reminder use. Another study20 identified barriers, such as lack of coordination between nurses and providers, using the reminders while not with the patient, workload, lack of CR flexibility and poor interface usability.
A successful prevention chain is constructed from (1) established evidence-based clinical guidelines; (2) abstraction of clinical guidelines into CR or decision-support systems; (3) physicians’ adherence to the CRs; and (4) patient adherence to treatment, which is reflected in clinical outcomes. In this paper we focus on a major component in this chain, which is physicians’ adherence to CRs. A key condition for successful implementation of reminder systems is to motivate the primary care teams to use these reminders. This can be achieved if the reminders provide valid information and impose minimal additional burden on the GPs. Physicians’ adherence with such computerized reminders may be affected by the physicians’, the patients’, the clinics’, and the reminders’ characteristics. We aim to evaluate the effect of these characteristics on adherence with CRs regarding patients requiring secondary prevention of dyslipidemia. The innovation in this study is in suggesting some properties which affect adherence, based on a large-scale dataset over a long time period.
Method
A computer-based clinical decision support system was implemented in the “Computerized Community Cholesterol Control” (4C) intervention program which is aimed at secondary prevention of dyslipidemia. This project was initiated and is maintained by Soroka University Medical Center (SUMC), a 1000-bed tertiary care academic medical center serving a population of 500,000 people in the southern district of Israel, belonging to “Clalit Health Services”, the largest HMO in Israel. The project is active since 2001 and is currently being implemented nationwide. A comprehensive database of patients’ health records was created, including demographics, diagnosis, hospitalizations, laboratory tests and pharmacotherapy. A decision-support algorithm produces patient-specific reminders, based on the NCEP-III21 guidelines for the management of dyslipidemia. The reminders were sent every four months to primary care physicians. Each reminder included a clinical summary of the patient and one of the following recommendations: (a) screening and monitoring of lipoprotein levels; (b) pharmacotherapy of lipid-lowering drugs; or (c) metabolic expert consultation in overcomplicated clinical status. If the patient was within target values, no reminder was sent.
97,064 reminders were produced between January 2001 and May 2005, regarding 14,018 patients treated by 216 physicians in 112 primary care clinics (56 intervention clinics and 56 demographically matched control clinics). Patients hospitalized at Soroka University Medical Center from 1993 were screened for dyslipidemia related diagnoses (ICD-9: 410–414, 428, v36.0, v45.81, and v45.82). Reminders were sent only to physicians in the intervention group. For the control group, the reminders were generated by the algorithm, but were not sent to the physicians.
In terms of clinical outcomes, the reminders improved lipids screening and monitoring, helped to optimize pharmacotherapy, and reduced the number of major cardiovascular events and re-hospitalizations22. Nevertheless, we were interested to understand the factors affecting the physicians’ adherence with the CRs.
For this paper we analyzed physicians’ responses to 7,980 reminders on screening and monitoring of lipoprotein levels, sent to 95 doctors in 29 clinics (14 intervention, 15 control), in a 16-month period (September 2002 to January 2004), regarding 4,460 4C patients.
We built a SQL-Server-based engine for temporal causal reasoning, to assess physicians’ adherence with the reminders. Each reminder was categorized by the engine as “adhered” or “not adhered”, according to the correspondence between the reminder and the actual treatment that was given by the corresponding physician. Since the reminders were sent as hardcopies, there was a great challenge to causally relate between a specific reminder and a specific treatment action taken by the physician. Automated quality assessment of clinician actions and patient outcomes is a central problem in guideline- or standards-based medical care23. We documented “adherence” when a lipid panel test was conducted no more than 4 months after a reminder indicated the need for such a test. Based on this calculation, we used a logistic regression with backward-stepwise elimination to statistically correlate the adherence with properties of the reminder, the physician and the patient. The predictors were:
Physicians’ properties: age, gender, experience with the 4C project (in months), average reminders load in each batch, and morbidity-adjusted patients’ load. The morbidity adjustment weighed the actual number of patients registered to the physician, with morbidity characteristics of the patients’ population. The adjustment was determined by the primary care HMO. A patients’ load higher than 1 indicates that the physician treats clinically complex population and hence, his or her actual workload is relatively high.
Patients’ properties: age, gender, co-morbidity (by Charlson co-morbidity index24), familiarity with corresponding physician (measured by the number of visits to the physician in the study period).
Reminders properties: the intervention group, and the proximity (in months) of the reminder to the previous Major Cardiac Event (MACE), such as Acute Myocardial Infraction, CABG (bypass graft), PTCA (catheterization), etc.
Due to the hierarchical nature of the data, we employed separate logistic regression models for each group of predictors. We were especially interested in the interactions of these predictors with the intervention group, since the interactions indicate different behavior in the two groups, which is probably a result of the intervention.
Results
The adherence rate was greater in the intervention group than in the control group (47% vs. 41%, OR=1.289, 95% CI=[1.179,1.408]). The logistic model with the physicians’ properties is shown in table 1. The interaction of the patients’ load with the intervention group indicates that there were differences between physicians in the intervention and control groups. Indeed, figure 1 shows that the physicians in the intervention group adhered with the CRs when they experienced greater patients’ load, whereas the physicians in the control group were not affected by the patients’ load.
Table 1.
Logistic regression model with physicians’ predictors (only p<.05 significant effects are shown).
Predictor | B | S.E. | Sig. |
---|---|---|---|
Experience in project | −0.013 | 0.005 | 0.005 |
Patients’ load by Intervention | 0.180 | 0.031 | 0.000 |
Constant | −0.285 | 0.045 | 0.000 |
Figure 1.
Patients’ load by Intervention group
The logistic model with the patients’ properties is shown in table 2. An interesting interaction is the interaction patient-physician familiarity and intervention group. This interaction, presented in figure 2, indicates that when there was a low familiarity with the patients, the physicians in the intervention group were more compliant with the CRs relatively to the control group. When the physicians in the intervention group were more familiar with their patients, they were less compliant with the CRs relatively to the control group.
Table 2.
Logistic regression model with patients’ predictors (only p<.05 significant effects are shown).
Predictor | B | S.E. | Sig. |
---|---|---|---|
Intervention group | 0.387 | 0.089 | 0.000 |
Age | 0.007 | 0.002 | 0.002 |
Familiarity | 0.014 | 0.001 | 0.000 |
Familiarity by Intervention | −0.003 | 0.001 | 0.033 |
Constant | −1.636 | 0.152 | 0.000 |
Figure 2.
Familiarity by Intervention group
The logistic model with the reminders’ properties is shown in table 3. There was an interaction of the proximity to the previous Major Cardiac Event (MACE) with the intervention group. This interaction, presented in figure 3, indicates that the physicians in the intervention group adhered more with the CRs when the last event was a long time ago, whereas the physicians in the control group were not affected by the proximity to the event.
Table 3.
Logistic regression model with reminders’ predictors (only p<.05 significant effects are shown).
Predictor | B | S.E. | Sig. |
---|---|---|---|
Intervention group | 0.329 | 0.056 | 0.000 |
MACE Proximity by Intervention | 0.003 | 0.001 | 0.021 |
Constant | −0.375 | 0.032 | 0.000 |
“B” stands for the predictor coefficient; “S.E.” is the standard error; “Sig.” is the p. value.
Figure 3.
MACE proximity by Intervention group
Discussion
The greater adherence rate in the intervention group indicates that the CR system was effective in terms of changing physicians’ behavior. The CR system increased screening and monitoring of LDL levels by 6% (p<0.05) compared to control, which is a positive change in physicians behavior since more patients are screened and monitored for dyslipidemia as is recommended by the guidelines. This result conforms to many studies which evaluated CRs as an efficient clinical tool5. It seems that the physicians in the intervention group were more compliant with the CRs when they experienced greater patients’ load, whereas the physicians in the control group were not affected by the patients’ load. This result is somewhat different from other studies which indicated workload as a barrier for implementation of CRs17–20. One explanation for this can be that especially when the physicians experienced overload, they had less time to consider the whole clinical picture and preferred to rely on the computerized reminder. The relatively high adherence in the intervention group when the patients were less familiar to the physicians suggest that the CRs were more valuable to the physicians when they were less acquainted with their patients. When they knew the patient well, the reminder was less valuable to them, resulting in lower adherence. We found no previous reference to the increased adherence to CRs when treating relatively unfamiliar patients. This explanation may be strengthened by the result of relatively greater adherence in the intervention group when the last major event was a long time ago. General practitioners are usually aware of such events and of the general clinical condition of patients undergoing such events, and hence the reminder was probably less valuable to them, resulting in lower adherence.
Limitations.
First, we should note that greater adherence does not necessarily indicate better treatment, whereas lower adherence does not necessarily indicate worse treatment. Physicians may know many details about their patients which are not available to the computerized algorithm. Indeed, the adherence was relatively lower when physicians were more familiar with patients. Second, our algorithm temporally matched reminders and treatment actions; yet, we cannot know for sure if the physician’s action was a direct result of the reminder or a routine practice.
Conclusions
The CR system was effective in terms of positively changing physicians’ behavior. The intervention physicians were more compliant with the CRs when they experienced greater patients’ load, when they were less acquainted with the patient, and when the last major event was a long time ago. We are currently enriching the adherence model by looking at additional factors which can influence the physicians’ adherence with the CRs.
References
- 1.Smith SC. Bridging the Treatment Gap. Am J Cardiol. 2000;85:3E–7E. doi: 10.1016/s0002-9149(00)00944-9. [DOI] [PubMed] [Google Scholar]
- 2.Bowker TJ, Clayton TC, Ingham J, McLennan NR, Hobson HL, Pyke SD, Schofield B, Wood DA. A British Cardiac Society survey of the potential for secondary prevention of coronary disease: ASPIRE (Action on Secondary Prevention through Intervention to Reduce Events) Heart. 1996;75:334–342. doi: 10.1136/hrt.75.4.334. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Sueta CA, Chowdhury M, Boccuzzi SJ, Smith SC, Jr, Alexander CM, Londhe A, Lulla A, Simpson RJ., Jr Analysis of the degree of undertreatment of hyperlipidemia and congestive heart failure secondary to coronary artery disease. Am J Cardiol. 1999;83:1303–7. doi: 10.1016/s0002-9149(99)00117-4. [DOI] [PubMed] [Google Scholar]
- 4.Schrott HG, Bittner V, Vittinghoff E, Herrington DM, Hulley S. Adherence to National Cholesterol Education Program Treatment goals in postmenopausal women with heart disease. The Heart and Estrogen/Progestin Replacement Study (HERS). The HERS Research Group. JAMA. 1997;277:1281–1286. [PubMed] [Google Scholar]
- 5.Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA. 1998;280(15):1339–46. doi: 10.1001/jama.280.15.1339. [DOI] [PubMed] [Google Scholar]
- 6.Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive preventive care in the ambulatory setting. JAMIA. 1996;3:399–409. doi: 10.1136/jamia.1996.97084513. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Kuperman GJ, Teich JM, Tanasijevic MJ, Ma’Luf N, Rittenberg E, Jha A, Fiskio J, Winkelman J, Bates DW. Improving response to critical laboratory results with automation: results of a randomized controlled trial. J Am Med Inform Assoc. 1999;6(6):512–22. doi: 10.1136/jamia.1999.0060512. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Cannon DS, Allen SN. A comparison of the effects of computer and manual reminders on compliance with a mental health clinical practice guideline. J Am Med Inform Assoc. 2000;7(2):196–203. doi: 10.1136/jamia.2000.0070196. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Dexter PR, Perkins S, Overhage JM, Maharry K, Kohler RB, McDonald CJ. A computerized reminder system to increase the use of preventive care for hospitalized patients. N Engl J Med. 2001;345(13):965–70. doi: 10.1056/NEJMsa010181. [DOI] [PubMed] [Google Scholar]
- 10.Oniki TA, Clemmer TP, Pryor TA. The effect of computer-generated reminders on charting deficiencies in the icu. J Am Med Inform Assoc. 2003;10:177–187. doi: 10.1197/jamia.M1175. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Phillips LS, Ziemer DC, Doyle JP, et al. An endocrinologist-supported intervention aimed at providers improves diabetes management in a primary care site. Diabetes Care. 2005;28:2352–2360. doi: 10.2337/diacare.28.10.2352. [DOI] [PubMed] [Google Scholar]
- 12.Demakis JG, Beauchamp C, Cull WL, et al. Improving residents’ compliance with standards of ambulatory care: results from the va cooperative study on computerized reminders. JAMA. 2000;284(11):1411–6. doi: 10.1001/jama.284.11.1411. [DOI] [PubMed] [Google Scholar]
- 13.Raschke RA, Gollihare B, Wunderlich TA, et al. A computer alert system to prevent injury from adverse drug events: development and evaluation in a community teaching hospital. JAMA. 1998;280(15):1317–20. doi: 10.1001/jama.280.15.1317. [DOI] [PubMed] [Google Scholar]
- 14.Bogucki B, Jacobs BR, Hingle J, et al. Computerized reminders reduce the use of medications during shortages. J Am Med Inform Assoc. 2004;11(4):278–280. doi: 10.1197/jamia.M1531. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Sequist TD, Gandhi TK, Karson AS, et al. A randomized trial of electronic clinical reminders to improve quality of care for diabetes and coronary artery disease. J Am Med Inform Assoc. 2005;12(4):431–8. doi: 10.1197/jamia.M1788. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Agrawal A, Mayo-Smith MF. Adherence to computerized clinical reminders in a large healthcare delivery network. MEDINFO. 2004:111–4. [PubMed] [Google Scholar]
- 17.Patterson ES, Nguyen AD, Halloran JP, Asch SM. Human factors barriers to the effective use of ten hiv clinical reminders. J Am Med Inform Assoc. 2004;11:50–59. doi: 10.1197/jamia.M1364. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Patterson ES, Doebbeling BN, Fung CH, Militello L, Anders S, Asch SM. Identifying barriers to the effective use of clinical reminders: bootstrapping multiple methods. J Biomed Inform. 2005;38:189–199. doi: 10.1016/j.jbi.2004.11.015. [DOI] [PubMed] [Google Scholar]
- 19.Militello L, Patterson ES, Tripp-Reimer T, et al. Clinical reminders: why don’t they use them?. Proc. of the Human Factors and Ergonomics Society 48th Annual Meeting; 2004. pp. 1651–1655. [Google Scholar]
- 20.Saleem JJ, Patterson ES, Militello L, et al. Exploring barriers and facilitators to the use of computerized clinical reminders. J Am Med Inform Assoc. 2005;12(4):438–48. doi: 10.1197/jamia.M1777. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Expert panel on detection, evaluation, and treatment of high blood cholesterol in adults. Executive summary of the third report of the National Cholesterol Education Program (NCEP) (Adult treatment Panel III) JAMA. 2001;285(19):2486–97. doi: 10.1001/jama.285.19.2486. [DOI] [PubMed] [Google Scholar]
- 22.Gilutz H, Zelinger J, Henkin Y, Bonneh DY, Liss Z, Peleg R, Mayslus M, Ilia R, Porath A.A Computerized Community Cholesterol Control (4C) Journal of the American College of Cardiology 2002395Supplements. [Google Scholar]
- 23.Advani A, Goldstein M, Shahar Y, Musen MA. Developing quality indicators and auditing protocols from formal guideline models: knowledge representation and transformations. AMIA Annu Symp Proc. 2003:11–5. [PMC free article] [PubMed] [Google Scholar]
- 24.Charlson ME, Pompei P, Ales KL, McKenzie CR. A new method of classifying prognostic co-morbidity in longitudinal studies: development and validation. J Chron Dis. 1987;40(5):373–383. doi: 10.1016/0021-9681(87)90171-8. [DOI] [PubMed] [Google Scholar]