Abstract
Background
Many individuals with diabetes, hypertension and hyperlipidemia have difficulty achieving control of all three conditions. We assessed the incidence and duration of simultaneous control of hyperglycemia, blood pressure, and low-density lipoprotein (LDL) cholesterol in patients from two health care systems in Colorado.
Methods and Results
Retrospective cohort study of adults at Denver Health (DH) and Kaiser Permanente Colorado (KP) with diabetes, hypertension, and hyperlipidemia from 2000 through 2008. Over a median of 4.0 and 4.4 years, 16% and 30% of individuals at DH and KP achieved the primary outcome (simultaneous control with a glycosylated hemoglobin (HbA1c) < 7.0%, blood pressure < 130/80 mmHg and LDL cholesterol < 100 mg/dL), respectively. With less strict goals (HbA1c < 8.0%, BP < 140/90 mmHg, and LDL cholesterol < 130 mg/dL), 44% and 70% of individuals at DH and KP achieved simultaneous control. Socio-demographic characteristics (increasing age, white ethnicity), and the presence of cardiovascular disease or other comorbidities were significantly but not strongly predictive of achieving simultaneous control in multivariable models. Simultaneous control was less likely as severity of the underlying conditions increased, and more likely as medication adherence increased.
Conclusions
Simultaneous control of diabetes, hypertension, and hyperlipidemia was uncommon and generally transient. Less stringent goals had a relatively large effect on the proportion achieving simultaneous control. Individuals who simultaneously achieve multiple treatment goals may provide insight into self-care strategies for individuals with comorbid health conditions.
Keywords: Diabetes mellitus, hypertension, hypercholesterolemia, epidemiology
In the United States, over 75 million people have two or more chronic medical conditions.1 Individuals with multiple health conditions must maintain good nutrition and physical activity, manage complex medication regimens, and monitor themselves for achievement of treatment goals and complications of their conditions.2 Diabetes is a prototype for this problem, since persons with diabetes must attain glycemic control and manage common, comorbid cardiovascular disease risk factors such as blood pressure and hyperlipidemia. Few are successful in simultaneously achieving these goals. In the 2003-2006 National Health and Nutrition Examination Survey (NHANES), only 12% of individuals with diabetes achieved a glycosylated hemoglobin (HbA1c) less than 7.0%, systolic blood pressure less than 130 mmHg, diastolic blood pressure less than 80 mmHg, and a fasting low-density lipoprotein (LDL) cholesterol level less than 100 mg/dL.3 Other studies in the United States and Europe confirm a comparably low rate of simultaneous control.4-11 Because all these studies were cross-sectional, they assessed only the prevalence of simultaneous control, and not the incidence or maintenance of simultaneous risk factor control, or the severity of the three conditions. Furthermore, they did not assess the impact of behaviors such as medication adherence on achievement of these goals. Finally, most of the previous studies did not examine the effect of different threshold values for HbA1c, blood pressure, and LDL cholesterol, which have important implications for the assessment of quality of care.
We analyzed data from two health care delivery systems in Colorado, an inner-city integrated health care delivery system in Denver (Denver Health), and a large managed care organization (Kaiser Permanente Colorado), to address four questions: 1) Among individuals with concurrent diabetes, hypertension, and hyperlipidemia, what is the incidence of and proportion maintaining simultaneous control for all three conditions? 2) What is the effect of changing goals for HbA1c, blood pressure, and LDL cholesterol on estimates of simultaneous control? 3) Are easily measurable socio-demographic and clinical characteristics, severity of the underlying conditions, or medication adherence associated with simultaneous risk factor control? 4) Do the incidenceor predictors of simultaneous control differ between the health care systems?
Methods
Study Populations
We conducted this retrospective cohort study using two registries: 1) the hypertension registry of Denver Health (DH), a nationally recognized, integrated safety-net delivery system in inner-city Denver, Colorado12, 13 and 2) the diabetes registry of Kaiser Permanente Colorado (KPCO), a large managed care organization with extensive disease management programs.14
Denver Health
Denver Health consists of a 500-bed hospital and eight neighborhood-based primary care clinics coupled with public health and emergency medical services systems. DH provided care to more than 140,000 individuals in Denver County in 2007. A clinical information system integrates information from all DH community health centers, emergency services, in-patient services, pharmacies and the clinical laboratory. Clinical data on blood pressure, smoking status, height, and weight were present in the electronic record for individuals who received care after January 2005. Participants for this study were drawn from a registry of individuals with hypertension who received care at Denver Health between January 1, 2000, and December 31, 2008. The hypertension registry included all DH patients with one or more ICD9-CM codes for hypertension on any outpatient or inpatient claim.13 Within this cohort, we identified individuals with diabetes using ICD-9 codes: 250.×× (diabetes), 357.2× (polyneuropathy in diabetes), 362. ×× (diabetic retinopathy), and 366.41 (diabetic cataract). In this population the hypertension case definition had a sensitivity of 88% and a specificity of 78%, and the diabetes definition had a sensitivity of 92% and specificity of 86%, compared to a comprehensive medical record review.12 To identify individuals with hyperlipidemia, we required laboratory evidence of a fasting LDL cholesterol level greater than 100 mg/dL or a filled prescription for a lipid-lowering drug. We defined the cohort for the current study as all individuals in the hypertension registry who also had diagnoses of diabetes and hyperlipidemia. We excluded individuals less than 21 years of age and those with less than one year of care in the DH system.
Kaiser Permanente Colorado
Kaiser Permanente Colorado is an integrated, group model, not-for-profit HMO which served approximately 450,000 enrollees in the six-county Denver/Boulder area in 2008. Electronic data on blood pressure, medication dispensing, laboratory test results, diagnoses, and health care utilization was available from electronic health records and administration databases from January 2000. The cohort of participants with diabetes was based on membership in a validated diabetes registry. In addition, we required a minimum of two years of continuous enrollment and at least two diabetes diagnoses (ICD-9 codes of 250 with a fifth digit of 0 or 2) at any point between January 1998 and September 2008.14,15 To identify individuals with hypertension, we used a previously validated algorithm based on ICD-9 codes, dispensed medications, and blood pressure measurements.16 Previous studies have found high accuracy with these definitions.15,16 Hyperlipidemia was defined as in the DH cohort. We defined the cohort for the current study as all individuals in the diabetes cohort who also had diagnoses of hypertension and hyperlipidemia.
Study Measures
Since diabetes, hypertension, and hyperlipidemia were typically diagnosed at different times, we identified the date of diagnosis of the third of these diseases, and excluded individuals who did not have at least one measurement of HbA1c, blood pressure, and LDL cholesterol after that time. The beginning of cohort membership was the date of the diagnosis of the third condition or the date when blood pressures and laboratory results became available in the electronic record (January 2005 for DH and January 2000 for KP), whichever was later. We defined the end of follow-up as the date of the last clinical measurement of HbA1c, blood pressure, or LDL cholesterol prior to December 31, 2008.
The study outcome was the occurrence of simultaneous control of HbA1c, systolic and diastolic blood pressure, and LDL cholesterol. We defined risk factor control using the 2002 guidelines from the American Diabetes Association, which were in place for the majority of the study period: HbA1c less than 7.0%, systolic blood pressure less than 130 mmHg and diastolic blood pressure less than 80 mmHg, and fasting LDL cholesterol less than 100 mg/dL.17 To estimate the severity of each individual risk factor, we recorded the highest value of each risk factor. We assessed whether individuals achieved control of each risk factor, and if so, whether they subsequently lost and later regained control.
Blood pressure and laboratory measurements were not necessarily obtained at the same time. Accordingly, we identified time periods during which all risk factor measurements were considered to be guideline-concordant. Any individual who had at least one guideline-concordant measurement of each risk factor (HbA1c, blood pressure, and LDL cholesterol) within a 90-day period, without any intervening non-guideline-concordant measurement of any risk factor, was defined as having “simultaneous” risk factor control. We defined the beginning of an interval of simultaneous control as the time of the first of the three guideline-concordant measurements.
Potential socio-demographic and clinical predictors of simultaneous control were derived from registration files, visit claims, laboratory databases, and pharmacy records. There was a substantial amount of missing race information in the KP cohort due to lack of systematic collection of race/ethnicity information during the earlier portions of the study period. Imputation of missing race information using a Bayesian algorithm originally developed by the RAND Corporation based on surnames and geocoded addresses did not substantially change the overall racial distribution.18 Substance abuse was defined using diagnosis codes; in DH laboratory toxicology screens were also included. We counted the overall number of comorbid diagnoses using the Quan version of the Elixhauser index,19 eliminating the three conditions of interest (diabetes, hypertension, and hyperlipidemia) since they were present in all cohort members. We divided the Quan index into a variable for any cardiovascular disease (based on the ICD-9 codes in the Quan index for cardiac arrhythmia, congestive heart failure, valvular disease, peripheral vascular disease, and coronary artery disease) and a variable for the other comorbidities in the Quan index.
To assess the intensity of medication treatment, we counted the number of oral hypoglycemics, antihypertensives, and lipid-lowering medications dispensed by the DH or KP pharmacies in the 90 days prior to the last measurement of any risk factor. We calculated adherence for each medication as the total days' supply dispensed, divided by the number of days between the first fill and the end of the supply provided in the last fill for that drug, and capped at 1.20 Adherence for each medication class was calculated as the time-weighted mean of adherence for each drug prescribed. We also recorded whether insulin was dispensed in the 90 days prior to the last clinical measurement of any risk factor, but did not calculate adherence.
Statistical Analysis
All analyses were stratified by health care delivery system. We conducted bivariate analyses to identify associations between all candidate predictors and the achievement of simultaneous risk factor control, using t-tests or Wilcoxon rank-sum tests for continuous variables, and chi-square or Fisher exact tests for dichotomous or categorical predictors. Using logistic regression to estimate the odds of ever achieving simultaneous risk factor control, we first examined socio-demographic variables and clinical diagnosis with a p-value of < 0.25. Backwards elimination was performed, checking at each step to assure that odds ratios did not change significantly, until all remaining variables in the model had a p-value of <0.05 in at least one of the study cohorts.21 We then added the number of medications for all three conditions, medication adherence, and the highest value of each target conditions, as a proxy for severity of the underlying conditions. All models were adjusted for length of follow-up. We assessed model discrimination with the c-statistic.22
This study was approved by the Colorado Multiple Institutional Review Board and the institutional review board of Kaiser Permanente Colorado. Analyses were conducted with SAS Versions 9.1 and 9.2 (SAS Institute, Inc., Cary, NC).
Results
The characteristics of the study populations are shown in Table 1. There were 5,269 individuals in the DH cohort and 23,458 individuals in the KP cohort, with a median follow-up time of 4.0 and 4.4 years, respectively. Compared to the KP cohort, the DH cohort was younger (mean age of 56.4 years vs. 62.0 years), and had a smaller proportion of men (39.0% vs. 52.2%) and a higher proportion of racial minorities (81.5% vs. 31.4% of individuals without missing race information). In the DH cohort, 28.7% reported Spanish as their primary language, while in the KP cohort only 2% requested interpreters.
Table 1.
Characteristics of individuals with diabetes, hypertension, and hyperlipidemia by achievement of simultaneous control of HbA1c blood pressure (BP), and LDL cholesterol
| Denver Health (n=5,269) | Kaiser Permanente Colorado (n=23,458) | |||||||
|---|---|---|---|---|---|---|---|---|
| No simultaneous control | Simultaneous control | Total | p-value | No simultaneous control | Simultaneous control | Total | p-value | |
| n=4,413 | n=856 | n=5,269 | n=16,341 | n=7,117 | n=23,458 | |||
| Socio-demographic characteristics | ||||||||
| Mean (sd) / median age in years | 56.2 (11.5) / 56.3 | 57.7 (10.8) / 58.2 | 56.4 (11.4) / 56.6 | <0.0001 | 60.7 (12.2) / 60.6 | 65.0 (10.6) / 66.3 | 62.0 (11.9) / 62.7 | <0.0001 |
| Male gender (%) | 38.8 | 40.1 | 39.0 | 0.49 | 51.0 | 55.1 | 52.2 | <0.0001 |
| Race (%) | ||||||||
| African-American | 19.9 | 13.0 | 18.8 | <0.0001 | 5.6 | 3.3 | 4.9 | <0.0001 |
| Hispanic | 58.8 | 62.5 | 59.4 | 14.4 | 12.0 | 13.7 | ||
| White | 17.8 | 20.8 | 18.3 | 45.3 | 61.9 | 50.4 | ||
| Other | 2.2 | 2.3 | 2.2 | 4.4 | 4.6 | 4.5 | ||
| Missing | 1.4 | 1.4 | 1.4 | 30.2 | 18.2 | 26.6 | ||
| Mean (sd) / median duration of follow-up (in years) after all 3 conditions became concurrent | 4.2 (2.5) / 4.0 | 4.5 (2.4) / 4.3 | 4.2 (2.5) / 4.0 | 0.006 | 4.2 (2.4) / 3.9 | 5.2 (2.2) / 5.3 | 4.5 (2.4) / 4.4 | <0.0001 |
| Baseline medical comorbidities | ||||||||
| Mean (sd) / Median body mass index | 32.6 (7.6) / 31.5 | 32.7 (6.9) / 31.8 | 32.6 (7.5) / 31.5 | 0.23 | 32.7 (7.0) / 31.7 | 31.9 (6.4) / 30.9 | 32.5 (6.8) / 31.4 | <0.0001 |
| Tobacco use (%) | ||||||||
| Never | 55.7 | 53.3 | 55.3 | 0.58 | 46.1 | 47.2 | 46.4 | <0.0001 |
| Former | 21.8 | 24.2 | 22.2 | 27.8 | 39.2 | 31.3 | ||
| Current | 21.6 | 22.4 | 21.8 | 11.3 | 9.1 | 10.7 | ||
| Missing | 0.9 | 0.1 | 0.8 | 14.8 | 4.5 | 11.7 | ||
| Substance Abuse (%) | 9.7 | 9.7 | 9.7 | 0.97 | 12.3 | 11.6 | 12.1 | 0.13 |
| Any cardiovascular comorbidity (%)* | 24.2 | 30.7 | 25.3 | <0.0001 | 26.4 | 39.2 | 30.3 | <0.0001 |
| Additional comorbidities (%)* | ||||||||
| 0 | 30.6 | 24.7 | 29.7 | <0.0001 | 29.1 | 25.4 | 27.9 | <0.0001 |
| 1 | 29.1 | 28.5 | 29.0 | 31.4 | 29.7 | 30.9 | ||
| 2 | 19.2 | 18.6 | 19.1 | 20.2 | 20.8 | 20.4 | ||
| 3 or more | 21.1 | 28.3 | 22.2 | 19.4 | 24.1 | 20.9 | ||
| Highest values, mean (sd) / median HbA1c (%) | 10.8 (2.9) / 10.9 | 9.4 (2.6) / 8.9 | 10.6 (2.9) / 10.6 | <0.0001 | 10.0 (2.4) / 9.7 | 8.7 (2.1) / 8.2 | 9.6 (2.4) / 9.2 | <0.0001 |
| Systolic BP (mmHg) | 162 (24) / 158 | 152 (20) / 151 | 160 (23) / 157 | <0.0001 | 164 (21) / 161 | 162 (19) / 160 | 164 (20) / 160 | <0.0001 |
| Diastolic BP (mmHg) | 90 (11) / 90 | 87 (9) / 86 | 90 (11) / 89 | <0.0001 | 95 (10) / 94 | 93 (9) / 92 | 95 (10) / 94 | <0.0001 |
| LDL cholesterol (mg/dL) | 139 (39) / 134 | 131 (35) / 129 | 137 (39) / 133 | <0.0001 | 142 (38) / 138 | 133 (34) / 129 | 139 (37) / 135 | <0.0001 |
As measured using the Quan index.18
The percentages ever meeting the goals for the individual risk factors ranged from 61.0% to 89.1%, with slightly higher percentages in the KP cohort (Table 2). Fluctuations in control of each risk factor were common. Only 16.2% of the DH cohort and 30.3% of the KP cohort ever achieved simultaneous control of all three risk factors (Figure 1, Table 2). Once achieving simultaneous control, few individuals were able to maintain it. Among those with at least 90 days of follow-up after achieving simultaneous control, 23% of the DH cohort and 39% of the KP cohort subsequently lost and then regained control, while 64% of the DH cohort and 56% of the KP cohort lost control and never regained it, and only 13% of the DH cohort and 5% of the KP cohort never lost control. In both cohorts, loss of simultaneous control was most commonly due to elevated blood pressure (82% for KP intervals, 92% for DH intervals), followed by HbA1c (12% for KP, 20% for DH), and LDL cholesterol (7% for KP, 8% for DH; percentages do not sum to 100% because loss of simultaneous control could be due to more than one risk factor). Increasing the allowed period for achieving simultaneous control from 90 to 365 days only slightly increased the proportion ever achieving simultaneous control (34.1% for KP, 18.1% for DH). With the 90 day definition, almost all individuals had at least one opportunity to achieve simultaneous control (98.2% for KP; 92.3% for DH).
Table 2.
Individual risk factor control in individuals with diabetes, hypertension and hyperlipidemia
| Risk factor | n (%) ever achieving control | Median duration in days (IQR) of initial interval of control | n (%) with ≥ 90 days of follow up after achieving control | Median percentage of subsequent time period controlled* | n (%) ever losing control* | n (%) regaining control of risk factor after losing control† |
|---|---|---|---|---|---|---|
| Denver Health | ||||||
| HbA1c | 3,215 (61.0%) | 78 (1-441) | 2,897 (90.1%) | 20.6% | 2,299 (79.4%) | 977 (42.5%) |
| Blood pressure | 4,108 (78.0%) | 1 (1-108) | 3,904 (95.0%) | 12.1% | 3,697 (94.7%) | 2,539 (68.7%) |
| LDL cholesterol | 4,416 (83.8%) | 178 (1-796) | 3,790 (85.8%) | 53.2% | 2,702 (71.3%) | 1,235 (45.7%) |
| Simultaneous control | 856 (16.2%) | 48 (3-132) | 691 (80.7%) | 17.0% | 601 (87.0%) | 158 (26.3%) |
| Kaiser Permanente Colorado | ||||||
| HbA1c | 16,084 (68.6%) | 201 (1-696) | 14,731 (91.6%) | 40.4% | 10,185 (69.1%) | 5,699 (56.0%) |
| Blood pressure | 20,894 (89.1%) | 1 (1-57) | 19,729 (94.4%) | 12.7% | 19,023 (96.4%) | 16,696 (87.8%) |
| LDL cholesterol | 19,946 (85.0%) | 311 (1-932) | 17,932 (89.9%) | 78.8% | 9,978 (55.6%) | 7,747 (77.6%) |
| Simultaneous control | 7,117 (29.9%) | 76 (24-183) | 6,658 (93.6%) | 15.7% | 6,320 (94.9%) | 2,599 (41.1%) |
Among individuals with ≥ 90 days follow-up after achieving control.
Among individuals ever losing control of that condition.
Figure 1.
Achievement and maintenance of simultaneous control of HbA1c, blood pressure, and LDL cholesterol in two Colorado cohorts of individuals with diabetes, hypertension, and hyperlipidemia.
Using less stringent risk factor cut points, over twice as many people simultaneously achieved a HbA1c < 8%, blood pressure < 140/90 mmHg, and LDL cholesterol < 130 mg/dL than achieved the stricter ADA guidelines (Figure 2).
Figure 2.
Effect of varying goals for HbA1c blood pressure (BP), and LDL cholesterol (LDL-C) on the percent ever achieving simultaneous control.
Bivariate comparisons between individuals ever achieving simultaneous risk factor control and those who did not were largely similar between the two cohorts (Table 1). Individuals who achieved simultaneous control had longer follow-up time, were older, and were more likely to be white, have diagnosed cardiovascular disease, and had more comorbid conditions. Individuals in the DH cohort who achieved simultaneous control were also more likely to be Hispanic, while individuals in the KP cohort who achieved simultaneous control were more likely to be male, less likely to be current smokers and were slightly leaner. In the DH cohort, primary language was not associated with achievement of simultaneous control. Individuals who achieved simultaneous control had lower maximum values for HbA1c, blood pressure, and LDL cholesterol.
Medication information was unavailable in 25.3% of the DH cohort and 6.9% of the KP cohort (Table 3). These individuals represent a mix of individuals who obtained their medications at pharmacies external to the health systems and individuals who were not taking any medications for diabetes, hypertension, or hyperlipidemia. After excluding individuals with missing medication information, those who received fewer medications for diabetes or hypertension or were not receiving insulin were more likely to achieve simultaneous control, while those receiving any medications for hyperlipidemia were more likely to attain simultaneous control. Medication adherence was higher in the KP cohort than in the DH cohort, and in individuals who achieved simultaneous control than in those who did not.
Table 3.
Pharmacological treatment and adherence of individuals with diabetes, hypertension, and hyperlipidemia by achievement of simultaneous control of HbA1c blood pressure, and LDL cholesterol.
| Denver Health (n=5,269) | Kaiser Permanente Colorado (n=23,458) | |||||||
|---|---|---|---|---|---|---|---|---|
| No simultaneous control | Simultaneous control | Total | p-value | No simultaneous control | Simultaneous control | Total | p-value | |
| n=4,413 | n=856 | n=5,269 | n=16,341 | n=7,117 | n=23,458 | |||
| Medication adherence*, mean (sd) / median | 61.2 (23.3) / 62.6 | 72.3 (18.7) / 74.9 | 63.0 (23.0) / 65.2 | <0.0001 | 81.1 (17.9) / 86.5 | 86.1 (13.6) / 90.1 | 82.7 (16.8) / 87.8 | <0.0001 |
| Any medication dispensed in the last 90 days of follow-up* (n(%)) | 3,301 (74.8%) | 635 (74.2%) | 3,936 (74.7%) | 0.70 | 15,117 (92.5%) | 6,713 (94.3%) | 21,830 (93.1%) | <0.0001 |
| Number of hypertensive medications (%)† | ||||||||
| 0 | 8.5 | 9.0 | 8.6 | 0.006 | 14.1 | 10.5 | 13.0 | <0.0001 |
| 1 | 30.9 | 26.3 | 26.3 | 28.4 | 25.2 | 27.4 | ||
| 2-3 | 45.8 | 52.8 | 46.9 | 44.0 | 49.6 | 45.7 | ||
| 4 or more | 14.8 | 12.0 | 14.4 | 13.4 | 14.8 | 13.9 | ||
| Number of oral hypoglycemic medications (%)† | ||||||||
| 0 | 28.1 | 27.1 | 27.9 | <.0001 | 33.4 | 39.8 | 35.4 | <0.0001 |
| 1 | 32.4 | 42.7 | 34.1 | 39.2 | 41.7 | 40.0 | ||
| 2 or more | 39.5 | 30.2 | 38.0 | 27.4 | 18.5 | 24.7 | ||
| Any cholesterol-lowering medication (%)† | 69.0 | 80.2 | 70.8 | <.0001 | 71.4 | 83.3 | 75.0 | <0.0001 |
| Any insulin (%)† | 34.0 | 15.6 | 31.1 | <.0001 | 32.4 | 20.0 | 28.6 | <0.0001 |
Any medication for diabetes, hypertension, or hyperlipidemia.
Assessed in the 90 days prior to last clinical measurement of any of the three conditions, among individuals with at least one medication for diabetes, hypertension, or hyperlipidemia dispensed in the last 90 days of observation.
Clinical utilization and risk factor measurement rates are shown in Table 4. Some of the rate distributions are skewed, and thus the median is the better measure of their central tendencies. For both populations, primary care visits were somewhat more frequent among those who achieved simultaneous control. The frequency of measurement of individual risk factors was slightly more frequent among those who achieved simultaneous control than those who did not.
Table 4.
Utilization rates and frequency of risk factor measurement of individuals with diabetes, hypertension, and hyperlipidemia by achievement of simultaneous control of HbA1c blood pressure, and LDL cholesterol.
| Denver Health (n=5,269) | Kaiser Permanente Colorado (n=23,458) | |||||||
|---|---|---|---|---|---|---|---|---|
| No simultaneous control | Simultaneous control | Total | p-value | No simultaneous control | Simultaneous control | Total | p-value | |
| n=4,413 | n=856 | n=5,269 | n=16,341 | n=7,117 | n=23,458 | |||
| Annual utilization rates, mean (sd) / median * | ||||||||
| Primary care visits | 4.8 (5.2) / 4.0 | 4.9 (3.1) / 4.4 | 4.8 (4.9) / 4.0 | 0.0001 | 6.6 (16.8) / 4.6 | 6.1 (4.3) / 5.2 | 6.4 (14.2) / 4.8 | <.0001 |
| Specialty (cardiology, endocrinology, and renal) | 0.4 (1.1) / 0.0 | 0.5 (1.1) / 0.0 | 0.4 (1.1) / 0.0 | <.0001 | 0.6 (1.7) / 0.0 | 1.0 (1.9) / 0.2 | 0.8 (1.8) / 0.0 | <.0001 |
| ED visits | 1.0 (3.7) / 0.4 | 0.9 (2.4) / 0.4 | 1.0 (3.5) / 0.4 | 0.82 | 0.5 (1.1) / 0.2 | 0.6 (0.9) / 0.3 | 0.5 (1.1) / 0.2 | <.0001 |
| Inpatient visits | 0.4 (1.8) / 0.0 | 0.5 (2.0) / 0.0 | 0.4 (1.8) / 0.0 | 0.13 | 1.1 (3.2) / 0.0 | 1.4 (3.2) / 0.4 | 1.2 (3.2) / 0.18 | <.0001 |
| Annual measurement rates, mean (sd) / median * | ||||||||
| HbA1c | 2.1 (3.1) / 1.9 | 2.1 (0.9) / 2.0 | 2.1 (3.4) / 1.8 | <.0001 | 2.6 (13.2) / 1.8 | 2.0 (2.3) / 1.9 | 2.8 (15.8) / 1.8 | <.0001 |
| Blood pressure | 3.4 (4.5) / 2.5 | 3.5 (2.6) / 2.9 | 3.3 (4.8) / 2.4 | <.0001 | 6.0 (13.7) / 4.4 | 6.0 (4.3) / 5.0 | 6.0 (16.1) / 4.1 | <.0001 |
| LDL cholesterol | 1.4 (3.1) / 1.1 | 1.3 (0.6) / 1.2 | 1.4 (3.4) / 1.1 | <.0001 | 2.4 (13.2) / 1.6 | 2.0 (2.3) / 1.8 | 2.5 (15.7) / 1.5 | <.0001 |
Assessed from date when all three conditions were first present until date of last clinical measurement of any of the three conditions
In a multivariable model that included only socio-demographic risk factors and clinical diagnoses, age, race/ethnicity, and the presence of cardiovascular and non-cardiovascular comorbidities were associated with simultaneous control (Table 5, Model 1). In addition, men were more likely to achieve simultaneous control. Despite the statistical significance of these predictors, the model discrimination was only fair. Limiting Model 1 to individuals with available medication information did not substantially change the odds ratios or c-statistics (data not shown). Inclusion of maximum risk factor values and medication information increased the c-statistics (Table 5, Model 2).
Table 5.
Predictors of achieving simultaneous risk factor control among individuals with diabetes, hypertension, and hyperlipidemia.
| Denver Health | Kaiser Permanente Colorado | |||
|---|---|---|---|---|
| Model (odds ratios with 95% confidence intervals)* | 1 n=5,269 |
2 n=3,861 |
1 n=23,458 |
2 n=21,381 |
| Age (per 10 years) | 1.12 (1.04-1.20) | 1.10 (1.00-1.22) | 1.27 (1.24-1.31) | 1.14 (1.10-1.18) |
| Race | ||||
| African-American | 0.60 (0.46-0.77) | 0.91 (0.64-1.30) | 0.53 (0.45-0.61) | 0.73 (0.62-0.87) |
| Hispanic | 1.02 (0.84-1.23) | 1.57 (1.20-2.05) | 0.74 (0.67-0.80) | 0.91 (0.82-1.00) |
| White | Ref. | Ref. | Ref. | Ref. |
| Other | 1.03 (0.62-1.72) | 1.27 (0.63-2.54) | 0.89 (0.78-1.03) | 0.96 (0.82-1.13) |
| Unknown | 0.94 (0.49-1.80) | 1.02 (0.42-2.47) | 0.66 (0.61-0.72) | 0.80 (0.73-0.87) |
| Male | 1.10 (0.94-1.28) | 1.06 (0.87-1.30) | 1.19 (1.12-1.27) | 1.08 (1.00-1.15) |
| Any cardiovascular comorbidities† | 1.14 (1.05-1.24) | 1.08 (0.97-1.20) | 1.41 (1.33-1.52) | 1.43 (1.33-1.54) |
| Other comorbidities† | ||||
| 0 | Ref. | Ref. | Ref | Ref |
| 1 | 1.25 (1.02-1.53) | 1.09 (0.85-1.40) | 1.03 (0.95-1.11) | 1.03 (0.94-1.12) |
| 2 | 1.24 (0.99-1.57) | 1.07 (0.80-1.44) | 1.11 (1.02-1.21) | 1.15 (1.05-1.27) |
| 3 or more | 1.76 (1.41-2.21) | 1.79 (1.34-2.38) | 1.32 (1.21-1.44) | 1.49 (1.35-1.65) |
| Highest HbA1c (per 1% increase) | 0.90 (0.86-0.93) | 0.79 (0.78-0.81) | ||
| Highest systolic BP (per 10 mmHg increase) | 0.78 (0.74-0.82) | 0.85 (0.83-0.86) | ||
| Highest LDL cholesterol (per 10 mg/dL increase) | 0.96 (0.93-0.98) | 0.93 (0.92-0.94) | ||
| Insulin‡ | 0.38 (0.29-0.50) | 0.54 (0.49-0.59) | ||
| Oral hypoglycemic medication‡ | ||||
| 0 | Ref. | Ref | ||
| 1 | 1.20 (0.94-1.54) | 0.95 (0.88-1.02) | ||
| 2 or more | 0.60 (0.45-0.79) | 0.57 (0.52-0.63) | ||
| Any cholesterol medication‡ | 1.75 (1.38-2.22) | 1.80 (1.66-1.96) | ||
| Blood pressure medication‡ | ||||
| 0 | Ref. | Ref | ||
| 1 | 0.73 (0.50-1.05) | 0.90 (0.80-1.01) | ||
| 2 or more | 0.91 (0.64-1.31) | 0.98 (0.88-1.10) | ||
| Medication adherence for BP, oral hypoglycemic and cholesterol medication (per 10 percentage points) | 1.26 (1.20-1.32) | 1.14 (1.11-1.17) | ||
| Model discrimination (c-statistic) | 0.60 | 0.78 | 0.69 | 0.77 |
Model 1: Demographics and clinical diagnoses only; Model 2: Demographics, clinical diagnoses, medication adherence, number of medications, and severity of target conditions. All models adjusted for duration of observation and smoking status.
As measured using the Quan index.19
Dispensed in the 90 days prior to the last clinical measurement of any of the three conditions.
Discussion
In this study of patients receiving care in two integrated delivery systems, we found that 16% and 30% of individuals with diabetes, hypertension and hyperlipidemia achieved simultaneous control of all three conditions, as defined by 2002 ADA guidelines,17 over a median of 4.0 and 4.4 years of follow-up. Among those with at least 90 days of follow-up after achieving simultaneous control, only 13% and 5% of the DH and KP cohorts, respectively, maintained simultaneous control until the end of the observation period. The predictors of ever achieving simultaneous control were similar in the two populations. Socio-demographic and clinical characteristics did not discriminate accurately between individuals who attained simultaneous control and those who did not, while maximum risk factor values, medication patterns, and medication adherence improved model discrimination.
Prior studies of simultaneous risk factor control among individuals with diabetes have reported prevalence rates, while we determined incidence rates. Incidence data has the advantage of giving a fuller picture of what a cohort of individuals is able to achieve over time, as well as allowing examination of maintenance of control. Previously reported prevalence rates from several countries have been low, mostly in the 5-20% range.3-11 For example, a study using NHANES data found the prevalence of simultaneous achievement of a HbA1c < 7.0%, blood pressure < 130/80 mmHg, and LDL cholesterol < 100 mg/dL was 7.0% in 1999-2002 and 12.2% in 2003-2006.3 A study from the VA National Diabetes Registry in 1999-2000, using the same criteria, found a 3.9% rate of simultaneous control.11
Most previous studies have only examined one set of criteria for achievement of simultaneous control. The study from the VA National Diabetes Registry also examined less stringent criteria (HbA1c < 9%, blood pressure < 140/90 mmHg, and LDL cholesterol < 130 mg/dL) and found a prevalence of 30.7%.11 In the last several years, the optimal level of risk factor control to prevent macrovascular outcomes has been a matter of increasing debate.23, 24 In addition, there has been an ongoing discussion about whether item-by-item measurements, composite measurements, or all-or-none measurements are the most effective for judging quality of care and motivating improvement in quality.25, 26 All-or-none measurements, such as our assessment of simultaneous risk factor control in diabetes, takes the patient's perspective, as the patient is the unit of analysis.25 It is also a more sensitive measure of quality improvement than item-by-item measurements.25 Our study illustrates that when continuous measurements (such as HbA1c, blood pressure, and LDL cholesterol) are transformed into dichotomous threshold-based measures, relatively small differences in cut-points can have large effects on conclusions concerning quality of care. Selection of an appropriate threshold can be difficult, especially with the greater emphasis on individualized clinical goals in diabetes.26, 27 Using high threshold goals ensures that the goals are appropriate for almost all individuals and focuses attention on individuals that are the furthest from the optimal levels and who therefore have the most to gain. However, it does not encourage the health care system to help most individuals achieve optimal levels. In contrast, using stricter threshold goals means the risks of the resulting aggressive treatment will exceed the potential benefits for some individuals.26
The frequency of risk factor measurement has implications for our assessment of simultaneous control. Individuals could only be in simultaneous control if they had all three risk factors measured within 90 days. The differences in the risk factor measurement rates between those who did and did not achieve simultaneous control were relatively small. Increasing the allowed period for achieving simultaneous control did not substantially change our findings. The frequency of risk factor measurement also has implications for the maintenance of simultaneous control, since individuals could only fall out of simultaneous control when they had risk factors measured. The specific timing of risk factor measurements is the result of a number of complex factors, including the risk factor levels themselves, patients' level of engagement with the healthcare system, and both clinicians' and patients' perception of underlying risk. Most people lost simultaneous control because of elevated blood pressure, which likely reflects both the greater frequency of blood pressure measurements and the greater intraperson variability of blood pressure.
Prior studies found that males,7, 11, 28 whites,6, 11, 28 older individuals,7, 28 those with greater education28 or income,7 and individuals with cardiovascular disease and lower body mass index6, 7, 11, 28 were more likely to achieve simultaneous control. Prior studies also found that achieving simultaneous control is associated with receiving cholesterol-lowering medications and fewer antihypertensive medications, and not receiving insulin.6, 11, 28 These findings are largely consistent with ours.
The higher rate of simultaneous control in KP compared to DH is likely the result of both patient level and system level factors. Compared to the KP population, the DH population has a much higher proportion of individuals with low socioeconomic position; the reasons for poorer health in socioeconomically disadvantaged individuals are complex but likely result from more than just limited access to healthcare.29 Second, based on our multivariable models, the KP cohort would be predicted to have a higher rate of simultaneous control based on its age, gender, disease severity, and comorbidity profile. Third, the higher medication adherence of the KP cohort than the DH cohort likely explains some of the differences. Fourth, differences in physician practices between the two systems, such as different degrees of treatment intensification or different goal setting, could potentially explain some of the differences. Finally, KP has been able to devote substantially more resources to population based management systems over a period of many years. Population-based mechanisms for identifying individuals who have not achieved simultaneous control and treating elevated risk factors could potentially help improve achievement and maintenance of simultaneous control. Focusing on blood pressure, in particular, would be most likely to increase achievement of simultaneous control.
Our findings contribute to the literature in several ways. First, we assessed achievement of simultaneous control in two disparate health care systems within the same region. Second, our longitudinal study could assess incidence rather than simply prevalence of simultaneous control. As a result, we determined that sustained simultaneous risk factor control was rare and brief, especially using stringent guidelines. Third, no prior study has assessed either the severity of the individual diseases, medication adherence, or health care utilization as correlates of simultaneous control. Finally, although prior studies had identified socio-demographic and clinical variables associated with simultaneous control, they did not report the discrimination of their statistical models. The low c-statistics of Model 1 suggests that clinicians and researchers need to look beyond these conventional and easily obtainable measures if they are to identify useful predictors of self-management for diabetes.30
This study has several limitations. First, because we took advantage of existing cohorts, the inclusion and exclusion criteria differed slightly for the two different health care systems. Second, since all data were obtained in routine clinical practice, the number of measurements of clinical and timing of outcomes was variable, and there was missing data on some variables (most notably medication use in DH and race information in KP). Third, we were not able to determine the pre-treatment severity of diabetes, hypertension or hyperlipidemia, and had to use the highest available measurement, which could be confounded by a number of factors, as a proxy. Fourth, individuals could receive services from other health care providers and systems, although this was likely limited. For all these reasons, our findings may not be generalizable to other populations or settings.
Individuals who are able to achieve simultaneous treatment goals can be viewed as “positive deviants,”31, 32 whose strategies for self-care may provide important lessons for other individuals. The degree of statistical discrimination provided by a multivariable model based only on socio-demographic factors and clinical diagnoses suggests that assessment of self-care behaviors may be necessary to explain the ability of these individuals to attain control of their conditions. Although our ability to measure such behavioral characteristics was limited, medication adherence emerged as a strong predictor of simultaneous control, while tobacco use and substance abuse were unrelated to treatment outcomes. Assessment of the behavioral strategies that these individuals use to facilitate their adherence with medications or other elements of self-care will require additional quantitative and qualitative research.
In summary, we found that in two large cohorts 16-30% of individuals were able to achieve simultaneous, but generally transient, control of diabetes, hypertension, and hyperlipidemia over a median of 4.0-4.4 years of follow-up. Small changes in the treatment goals had a relatively large effect on the proportion considered to be at goal. Efforts to understand the strategies that such individuals, particularly those with durable control, use to balance the demands of their multiple health conditions may help define interventions to improve self-care and health outcomes among the increasing population of individuals with multiple, chronic health conditions.
What is Known
-
-
Individuals with diabetes must manage multiple cardiovascular risk factors, such as glucose, blood pressure, and cholesterol levels, at the same time.
-
-
Previous studies have found low levels of simultaneous control of these factors, but have been cross-sectional and thus unable to follow risk factor changes over time.
What the Study Adds
-
-
In these two health systems over a median follow-up of over 4 years, simultaneous control of diabetes, hypertension, and hyperlipidemia was uncommon and generally transient.
-
-
Small changes in the treatment targets had large effects on the proportion of individuals achieving simultaneous control of glucose, blood pressure, and cholesterol levels.
Acknowledgments
Funding Sources: This study was supported by the National Heart, Lung, and Blood Institute Cooperative Agreement 1 U01 HL079208, the Agency for Healthcare Research and Quality grant 1 R21 HS017627-01, and the Colorado Clinical Translational Sciences Institute 1 U54 RR025217. Dr. Schroeder was supported by the National Institute of Diabetes and Digestive and Kidney Diseases Training Grant 5 T32 DK007446 and a grant from the Endocrine Fellows Foundation.
Footnotes
Conflict of Interest Disclosures: None
References
- 1.Parekh AK, Barton MB. The challenge of multiple comorbidity for the US health care system. JAMA. 2010;303:1303–1304. doi: 10.1001/jama.2010.381. [DOI] [PubMed] [Google Scholar]
- 2.Bayliss EA, Bosworth HB, Noel PH, Wolff JL, Damush TM, Mciver L. Supporting self-management for patients with complex medical needs: recommendations of a working group. Chronic Illn. 2007;3:167–175. doi: 10.1177/1742395307081501. [DOI] [PubMed] [Google Scholar]
- 3.Cheung BM, Ong KL, Cherny SS, Sham PC, Tso AW, Lam KS. Diabetes prevalence and therapeutic target achievement in the United States, 1999 to 2006. Am J Med. 2009;122:443–453. doi: 10.1016/j.amjmed.2008.09.047. [DOI] [PubMed] [Google Scholar]
- 4.Saydah SH, Fradkin J, Cowie CC. Poor control of risk factors for vascular disease among adults with previously diagnosed diabetes. JAMA. 2004;291:335–342. doi: 10.1001/jama.291.3.335. [DOI] [PubMed] [Google Scholar]
- 5.Vouri SM, Shaw RF, Waterbury NV, Egge JA, Alexander B. Prevalence of achievement of A1c, blood pressure, and cholesterol (ABC) goal in veterans with diabetes. J Manag Care Pharm. 2011;17:304–312. doi: 10.18553/jmcp.2011.17.4.304. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Bertoni AG, Clark JM, Feeney P, Yanovski SZ, Bantle J, Montgomery B, Safford MM, Herman WH, Haffner S, Look AHEAD Research Group Suboptimal control of glycemia, blood pressure, and LDL cholesterol in overweight adults with diabetes: the Look AHEAD Study. J Diabetes Complications. 2008;22:1–9. doi: 10.1016/j.jdiacomp.2006.10.003. [DOI] [PubMed] [Google Scholar]
- 7.Guthrie B, Emslie-Smith A, Morris AD. Which people with Type 2 diabetes achieve good control of intermediate outcomes? Population database study in a UK region. Diabet Med. 2009;26:1269–1276. doi: 10.1111/j.1464-5491.2009.02837.x. [DOI] [PubMed] [Google Scholar]
- 8.Rossi MC, Nicolucci A, Arcangeli A, Cimno A, De Bigontina G, Giorda C, Meloncelli I, Pellegrini F, Valentini U, Vespasiani G, Associazione Medici Diabetologi Annals Study Group Baseline quality-of-care data from a quality-improvement program implemented by a network of diabetes outpatient clinics. Diabetes Care. 2008;31:2166–2168. doi: 10.2337/dc08-0469. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Shubrook JH, Jr, Snow RJ, McGill SL, Brannan GD. “All-or-none” (bundled) process and outcome indicators of diabetes care. Am J Manag Care. 2010;16:25–32. [PubMed] [Google Scholar]
- 10.Varma S, Boyle LL, Varma MR, Piatt GA. Controlling the ABCs of diabetes in clinical practice: a community-based endocrinology practice experience. Diabetes Res Clin Pract. 2008;80:89–95. doi: 10.1016/j.diabres.2007.10.029. [DOI] [PubMed] [Google Scholar]
- 11.Jackson GL, Edelman D, Weinberger M. Simultaneous control of intermediate diabetes outcomes among Veterans Affairs primary care patients. J Gen Intern Med. 2006;21:1050–1056. doi: 10.1111/j.1525-1497.2006.00519.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Hanratty R, Estacio RO, Dickinson LM, Chandramouli V, Steiner JF, Havranek EP. Testing electronic algorithms to create disease registries in a safety net system. J Health Care Poor Underserved. 2008;19:452–465. doi: 10.1353/hpu.0.0027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Steiner JF, Ho PM, Beaty BL, Dickinson LM, Hanratty R, Zeng C, Tavel HM, Havranek EP, Davidson AJ, Magid DJ, Estacio RO. Sociodemographic and clinical characteristics are not clinically useful predictors of refill adherence in patients with hypertension. Circ Cardiovasc Qual Outcomes. 2009;2:451–457. doi: 10.1161/CIRCOUTCOMES.108.841635. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Bayliss EA, Blatchford PJ, Newcomer SR, Steiner JF, Fairclough DL. The effect of incident cancer, depression and pulmonary disease exacerbations on type 2 diabetes control. J Gen Intern Med. 2011;26:575–581. doi: 10.1007/s11606-010-1600-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Zgibor JC, Orchard TJ, Saul M, Platt G, Ruppert K, Stewart A, Siminerio LM. Developing and validating a diabetes database in a large health system. Diabetes Res Clin Pract. 2007;75:313–319. doi: 10.1016/j.diabres.2006.07.007. [DOI] [PubMed] [Google Scholar]
- 16.Selby JV, Lee J, Swain BE, Tavel HM, Ho PM, Margolis KL, O'Connor PJ, Fine L, Schmittdiel JA, Magid DJ. Trends in time to confirmation and recognition of new-onset hypertension, 2002-2006. Hypertension. 2010;56:605–611. doi: 10.1161/HYPERTENSIONAHA.110.153528. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.American Diabetes Association Standards of medical care for patients with diabetes mellitus. Diabetes Care. 2002;25:213–229. doi: 10.2337/diacare.25.1.213. [DOI] [PubMed] [Google Scholar]
- 18.Elliott MN, Fremont A, Morrison PA, Pantoja P, Lurie N. A new method for estimating race/ethnicity and associated disparities where administrative records lack self-reported race/ethnicity. Health Serv Res. 2008;43:1722–1736. doi: 10.1111/j.1475-6773.2008.00854.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi JC, Saunders LD, Beck CA, Feasby TE, Ghali WA. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43:1130–1139. doi: 10.1097/01.mlr.0000182534.19832.83. [DOI] [PubMed] [Google Scholar]
- 20.Steiner JF, Prochazka AV. The assessment of refill compliance using pharmacy records: methods, validity, and applications. J Clin Epidemiol. 1997;50:105–116. doi: 10.1016/s0895-4356(96)00268-5. [DOI] [PubMed] [Google Scholar]
- 21.Hosmer D, Lemeshow S. Applied Logistic Regression. 2nd. New York, NY: Wiley John and Sons, Inc.; 2000. [Google Scholar]
- 22.Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007;115:928–935. doi: 10.1161/CIRCULATIONAHA.106.672402. [DOI] [PubMed] [Google Scholar]
- 23.Montori VM, Fernandez-Balsells M. Glycemic control in type 2 diabetes: time for an evidence-based about-face? Ann Intern Med. 2009;150:803–808. doi: 10.7326/0003-4819-150-11-200906020-00008. [DOI] [PubMed] [Google Scholar]
- 24.Skyler JS, Bergenstal R, Bonow RO, Buse J, Deedwania P, Gale EA, Howard BV, Kirkman MS, Kosiborod M, Reaven P, Sherwin RS, American Diabetes Association; American College of Cardiology Foundation; American Heart Association Intensive glycemic control and the prevention of cardiovascular events: implications of the ACCORD, ADVANCE, and VA Diabetes Trials: a position statement of the American Diabetes Association and a Scientific Statement of the American College of Cardiology Foundation and the American Heart Association. J Am Coll Cardiol. 2009;53:298–304. doi: 10.1016/j.jacc.2008.10.008. [DOI] [PubMed] [Google Scholar]
- 25.Nolan T, Berwick DM. All-or-none measurement raises the bar on performance. JAMA. 2006;295:1168–1170. doi: 10.1001/jama.295.10.1168. [DOI] [PubMed] [Google Scholar]
- 26.O'Connor PJ, Bodkin NL, Fradkin J, Glasgow RE, Greenfield S, Gregg E, Kerr EA, Pawlson LG, Selvy JV, Sutherland JE, Taylor ML, Wysham CH. Diabetes performance measures: current status and future directions. Diabetes Care. 2011;34:1651–1659. doi: 10.2337/dc11-0735. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.American Diabetes Association Standards of medical care in diabetes--2011. Diabetes Care. 2011;34(Suppl 1):S11–S61. doi: 10.2337/dc11-S011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Selby JV, Swain BE, Gerzoff RB, Karter AJ, Waitzfelder BE, Brown AF, Ackermann RT, Duru OK, Ferrara A, Herman W, Marrero DG, Caputo D, Narayan KM, TRIAD Study Group Understanding the gap between good processes of diabetes care and poor intermediate outcomes: Translating Research into Action for Diabetes (TRIAD) Med Care. 2007;45:1144–1153. doi: 10.1097/MLR.0b013e3181468e79. [DOI] [PubMed] [Google Scholar]
- 29.Pincus T, Esther R, DeWalt DA, Callahan KF. Social conditions and self-management are more powerful determinants of health than access to care. Ann Intern Med. 1998;129:406–411. doi: 10.7326/0003-4819-129-5-199809010-00011. [DOI] [PubMed] [Google Scholar]
- 30.Steiner JF. Can we identify clinical predictors of medication adherence… and should we? Med Care. 2010;48:193–195. doi: 10.1097/MLR.0b013e3181d51ddf. [DOI] [PubMed] [Google Scholar]
- 31.Bradley EH, Curry LA, Ramanadhan S, Rowe L, Nembhard IM, Krumholz HM. Research in action: using positive deviance to improve quality of health care. Implement Sci. 2009;4:25. doi: 10.1186/1748-5908-4-25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Marsh DR, Schroeder DG, Dearden KA, Sternin J, Sternin M. The power of positive deviance. BMJ. 2004;329:1177–1179. doi: 10.1136/bmj.329.7475.1177. [DOI] [PMC free article] [PubMed] [Google Scholar]