Abstract
Objective:
As part of a multidisciplinary team managing patients with type-2 diabetes, pharmacists need a consistent approach of identifying and prioritizing patients at highest risk of adverse outcomes. Our objective was to identify which predictors of adverse outcomes among type-2 diabetes patients were significant and common across 7 outcomes and whether these predictors improved the performance of risk prediction models. Identifying such predictors would allow pharmacists and other health care providers to prioritize their patient panels.
Research Design and Methods:
Our study population included 120,256 adults aged 65 years or older with type-2 diabetes from a large integrated health system. Through an observational retrospective cohort study design, we assessed which risk factors were associated with 7 adverse outcomes (hypoglycemia, hip fractures, syncope, emergency department visit or hospital admission, death, and 2 combined outcomes). We split (50:50) our study cohort into a test and training set. We used logistic regression to model outcomes in the test set and performed k-fold validation (k = 5) of the combined outcome (without death) within the validation set.
Results:
The most significant predictors across the 7 outcomes was: age, number of medicines, prior history of outcome within the past 2 years, chronic kidney disease, depression, and retinopathy. Experiencing an adverse outcome within the prior 2 years was the strongest predictor of future adverse outcomes (odds ratio range: 4.15–7.42). The best performing models across all outcomes included: prior history of outcome, physiological characteristics, comorbidities and pharmacy-specific factors (c-statistic range: 0.71–0.80).
Conclusions:
Pharmacists and other health care providers can use models with prior history of adverse event, number of medicines, chronic kidney disease, depression and retinopathy to prioritize interventions for elderly patients with type-2 diabetes.
Keywords: diabetes, pharmacist, risk factors, prediction, model, adverse events
BACKGROUND
Diabetes remains very prevalent in the United States with over 30 million people living with the disease and costing over $245 billion in medical costs and lost wages.1 Specifically, type-2 diabetes accounts for ~95% of all adult diabetes cases.1 The management of patients with type-2 diabetes is an interdisciplinary task typically involving physicians, pharmacists, nurses and other diabetes educators. Each of these providers have unique responsibilities and evolve their practice with emerging evidence. Studies suggest that when pharmacists take a greater role in the management of type-2 diabetes through pharmacist-led programs, patients in such programs more readily attain and maintain glycemic goals than those in usual care.2,3 Among other activities, pharmacists offer oversight over patients’ medication regimens through interventions such as comprehensive or targeted medication reviews, dose adjustments based on renal and hepatic function, and lifestyle counseling.4 Such interventions have shown efficacy in managing patients with type-2 diabetes who are typically complex and with varying treatment needs.5,6 However, what remains poorly understood is which predictors of serious adverse outcomes among patients with type-2 diabetes are most relevant to a pharmacy department. Identifying such predictors would allow pharmacy departments to specifically prioritize those patients for pharmacist-led interventions while also efficiently managing the limited number of pharmacists within a health care organization.7 Risk models predicting outcomes among patients with diabetes are discussed in literature,8-13 but none specifically identify risk factors that are most relevant to pharmacists. Also, to our knowledge, no other studies have examined predictor significance across multiple outcomes in a large type-2 diabetes population.
In this study our objective was to identify which predictors of adverse outcomes among type-2 diabetes patients were significant and common across all outcomes and whether these predictors improved the performance of risk prediction models. Our study also outlined how a medical specialty could examine such predictors in an effort to provide its practitioners with efficient prioritization strategies.
RESEARCH DESIGN AND METHODS
Study Setting and Population
The study population was derived from the Surveillance, PREvention, and ManagEment of Diabetes Mellitus (SUPREME-DM) DataLink project, a large diabetes registry created from the electronic health record and other clinical and administrative data of 11 health care systems.14-17 Our study specifically included patients with type-2 diabetes from three Kaiser Permanente regions participating in the DataLink project: Kaiser Permanente Northern California (KPNC), Kaiser Permanente Colorado (KPCO), and Kaiser Permanente North West (KPNW). Patients were eligible to be in our study if they were 65 years or older as of January 1, 2010. We also only included patients with > 1 month of enrolment in 2010 in our study-over 98% of our population had 12 months of enrolment. This study was reviewed and approved by the KPNC institutional review board (IRB); KPCO and KPNW ceded IRB oversight to KPNC.
Outcomes
We examined 7 outcomes: hypoglycemia, hip fracture, syncope, emergency department (ED) visit or hospital admission, death, combined outcome (including death), and combined outcome (without death). Hypoglycemia, syncope, ED visits or hospitalization, and death were chosen because patients with diabetes take glucose lowering drugs (eg, sulfonylureas) that increase the risk these 5 outcomes.18 We based the outcomes on 2011 data extracted from the electronic health record. We determined cases of hypoglycemia, hip fracture, and syncope using the following ICD-9 diagnoses codes: hypoglycemia—251.0, 251.1, 251.2, 962.3, and 250.8 (but excluded if used in conjunction with these codes: 259.8, 272.7, 681.xx, 682.xx, 686.9x, 707.xx, 709.3, 730.0–730.2, 731.8), hip fracture—820.0x, 820.2x, 820.8x, and 733.14, syncope—458.0 and 780.2.19 For hypoglycemia only, we required a principal inpatient diagnosis. Death was confirmed from multiple sources, for example, social security administration, state death records, tumor registry, encounter and membership data (Appendix I, Supplemental Digital Content 1, http://links.lww.com/MLR/B829). The confidence on the death event was then ranked from poor to excellent. We only used events with excellent confidence ranking.
Independent Variables
We grouped predictors in our models as: demographics (race, age, sex, census block education level defined as percent of adult population with a bachelor’s degree or higher, and census block median household income); pharmacy-specific (30 d copay for generic drugs, count of unique medications, and number of fills of 3 mainstay diabetes treatment drug classes: angiotensin-converting enzyme inhibitors and angiotensin receptor blockers, oral anti-hyperglycemics, and statins), physiological characteristics (hemoglobin A1C, low-density lipoprotein cholesterol, and systolic blood pressure), and 18 comorbidities (including prior history of an adverse outcome). We identified all comorbidities through ICD-9 and procedure codes. For example, alcohol abuse and drug abuse were acquired from the Enhanced Elixhauser algorithm.20 Pharmacy-specific and physiological predictors were based on data from 2010.
We categorized the count of medications into <5 medications, 5–9 medications (polypharmacy), and 10 or more medications (hyper-polypharmacy).21-26 The number of fills were categorized into 0 fills, 1 fill, and 2 or more fills. For oral anti-hyperglycemics, there was an additional category: 2 or more fills plus insulin. The 18 comorbidities included in our study were: history of outcome within prior 2 years, alcohol abuse, anxiety, arthritis, asthma, atrial fibrillation, cancer, chronic kidney disease (CKD), chronic obstructive pulmonary disease, dementia, depression, drug abuse, heart failure, ischemic heart disease, osteoporosis, retinopathy, stroke, and visual impairment. For each outcome (hypoglycemia, hip fracture, syncope, and ED visit or hospital admission), we included history of that outcome in 2009 or 2010 or both years.
Statistical Analyses
We split (50:50) our cohort into a test and validation set. Given that we did not have time-to-event survival data we used multivariate logistic regressions to model our outcomes within the test set. We then applied the resulting coefficients on the validation set through k-fold (k = 5) validation. We also identified the strength of association and significance level for each predictor with multivariate logistic regression models for each outcome. We examined significance levels at P < 0.001, <0.01, and <0.05. To quantify the incremental contributions of each group of predictors (pharmacy-specific predictors, physiological characteristics, and comorbidities) to our models, we sequentially added each of these predictor groups to a reference model that controlled only for demographics. We assessed the change in model performance by calculating the area under the receiver operating curve (AUROC or AUC or c-statistic27) as we sequentially added the following predictor groups: history of outcome in prior 2 years, physiological characteristics, comorbidities and pharmacy-specific factors. We also calculated the work-up to detection ratio (number needed to evaluate—NNE28) for the models with the highest c-statistics. All analyses were performed in R version 3.2.3.
RESULTS
Our study included 120,256 elderly adult patients with diabetes with a mean age of 73 years (SD: 6.8 y). Our population was evenly split among males and females and mostly white (58.5%). Comorbidity prevalence in the study population ranged from 0.8% (drug abuse and visual impairment) to 37.7% (CKD). Mean low-density lipoprotein cholesterol and hemoglobin A1C were 80.4% (SD: 28.7) and 6.9% (SD: 1.1), respectively. The average total number of medications per patient was 5 (SD: 3.4). The prevalence rates of our study outcomes were: hypoglycemia (0.2%), hip fracture (0.6%), syncope (4.3%), ED visit or hospital admission (3.9%), and death (5.1%) (Table 1). The majority of all medication fills in the cohort were over 90 days with the median day supply of the entire cohort being 100 days.
TABLE 1.
Study Participant Characteristics
| Derivation Cohort | Validation Cohort | Entire Cohort | |
|---|---|---|---|
| No. participants | 60,128 | 60,128 | 120,256 |
| Age [median (mean ± SD)] (y) | 73 (74.3 ± 6.8) | 73 (74.2 ± 6.8) | 73 (74.3 ± 6.8) |
| Sex* | |||
| Male | 30,371 (51) | 30,335 (50) | 60,706 (50.4) |
| Female | 29,757 (49) | 29,791 (50) | 59,548 (49.5) |
| Race/ethnicity* | |||
| Asian or Hawaiian/Pacific Islander | 6639 (11) | 6499 (11) | 13,138 (10.9) |
| Black | 4693 (8) | 4692 (8) | 9385 (7.8) |
| Hispanic | 7742 (13) | 7680 (13) | 15,422 (12.8) |
| White | 35,147 (58) | 35,210 (58) | 70,357 (58.5) |
| Native American or unknown | 5907 (10) | 6047 (10) | 11,954 (9.9) |
| Census block level % of adult population with bachelor’s degree or higher* | |||
| < 15% | 12,028 (20) | 11,897 (20) | 23,925 (19.9) |
| 15%–24% | 12,229 (20) | 12,360 (20) | 24,589 (20.4) |
| 25%–49% | 23,899 (40) | 23,755 (40) | 47,654 (39.6) |
| 50%–100% | 11,890 (20) | 12,037 (20) | 23,927 (19.9) |
| Comorbidities* | |||
| Arthritis | 15,620 (26) | 15,758 (26) | 31,378 (26) |
| Asthma | 6093 (10) | 6145 (10) | 12,238 (10) |
| Atrial fibrillation | 5772 (10) | 5734 (10) | 11,506 (10) |
| Cancer | 4500 (7) | 4704 (7) | 9204 (7) |
| Chronic kidney disease | 22,589 (38) | 22,554 (38) | 45,143 (38) |
| Chronic obstructive pulmonary disease | 5236 (9) | 5249 (9) | 10,485 (9) |
| Dementia | 661 (1) | 600 (1) | 1261 (1) |
| Depression | 7637 (13) | 7826 (13) | 15,463 (13) |
| Drug abuse | 483 (1) | 504 (1) | 987 (1) |
| Heart failure | 7073 (12) | 6898 (12) | 13,971 (12) |
| Ischemic heart disease | 13,927 (23) | 13,836 (23) | 27,763 (23) |
| Osteoporosis | 3734 (6) | 3732 (6) | 7466 (6) |
| Retinopathy | 9369 (16) | 9275 (16) | 18,644 (16) |
| Stroke | 3002 (5) | 3077 (5) | 6079 (5) |
| Visual impairment | 506 (1) | 497 (1) | 1003 (1) |
| Pharmacy-specific predictors | |||
| Count of medications [median (mean ± SD)] | 5 (5.3 ± 3.4) | 5 (5.3 ± 3.4) | 5 (5 ± 3.4) |
| 30-day copay for generic drugs [median (mean ± SD)] | 10 (7.6 ± 6.1) | 10 (7.6 ± 5.5) | 10 (7.6 ± 5.8) |
| LDL cholesterol (mean ± SD) | 80.4 ± 28.6 | 80.4 ± 28.9 | 80.4 ± 28.7 |
| Hemoglobin A1c (mean ± SD) | 6.9 ± 1.1 | 6.9 ± 1.1 | 6.9 ± 1.1 |
| Outcomes (adverse events) in 2011* | |||
| Hypoglycemia | 116 (0.2) | 123 (0.2) | 239 (0.2) |
| Hip fracture | 391 (0.6) | 382 (0.6) | 773 (0.6) |
| Death | 3013 (5.1) | 3084 (5.1) | 6097 (5.1) |
| Syncope | 2690 (4.3) | 2617 (4.3) | 5307 (4.3) |
| Had ED visit or hospitalization with diabetes as principal diagnosis | 2333 (3.9) | 2432 (3.9) | 4765 (3.9) |
| Composite outcome | 8569 (14.6) | 8968 (14.6) | 17,537 (14.6) |
The number of participants (% of total population).
ED indicates emergency department; LDL, low-density lipoprotein.
Tables 2 and 3 show adjusted odds ratios (OR) of model predictors across the 7 outcomes. We found that the most consistently significant predictors across the 7 outcomes were age, medication count, prior history of outcome, CKD, depression and retinopathy. Increasing age was associated with increasing odds of experiencing each outcome. For example, patients aged 85 years and older had significantly increased risk of hypoglycemia [OR, 2.57; confidence interval (CI), 1.50–4.33), hip fracture (OR, 5.11; CI, 3.76–6.98), syncope (OR, 1.95; CI, 1.74–2.19), ED visits or hospital admission (OR, 1.58; CI, 1.36–1.82), death (OR, 4.85; CI, 4.35–5.41), combined outcome including death (OR, 2.91; CI, 2.72–3.12), and combined outcome excluding death (OR, 1.65; CI, 1.51–1.81) compared with patients 65–69 years old. Comorbid depression and retinopathy, likewise, were associated with an increased risk across all adverse outcomes. Having a prior history of the adverse event was the strongest predictor (OR range, 4.15–11.0) across the 5 outcomes for which prior history applies—hypoglycemia, hip fracture, syncope, ED visit or hospitalization, and combined outcome without death. For example, the odds of a repeat hypoglycemic episode were 7 times higher for those who experienced it in the past 2 years compared with those who did not. History of the combined outcome without death had the highest OR of 11.0. Although not statistically significant in the hip fracture models, both comorbid CKD and number of medications were associated with increased risk of the other 5 study outcomes.
TABLE 2.
Adjusted Odds Ratios and 95% Confidence Intervals From Logistic Regression Models
| Hypoglycemia | Hip Fracture | Syncope | ED Visit or Hospital Admission | |
|---|---|---|---|---|
| Demographics | ||||
| Race/ethnicity | ||||
| White | Reference | Reference | Reference | Reference |
| Black | 2.56 [1.70–3.78]*** | 0.51 [0.34–0.75]** | 0.98 [0.87–1.10] | 1.65 [1.46–1.86]*** |
| Hispanic | 1.45 [0.97–2.13] | 0.81 [0.63–1.04] | 0.84 [0.76–0.93]*** | 1.14 [1.02–1.28]* |
| Asian or Hawaiian/Pacific Islander | 1.59 [1.01–2.45]* | 0.52 [0.36–0.72]*** | 0.93 [0.83–1.03] | 0.92 [0.80–1.05] |
| Native American or unknown | 0.19 [0.03–0.59]* | 0.15 [0.06–0.30]*** | 0.51 [0.43–0.59]*** | 0.55 [0.45–0.67]*** |
| Age (y) | ||||
| 65–69 | Reference | Reference | Reference | Reference |
| 70–74 | 1.35 [0.88–2.07] | 1.55 [1.14–2.11]** | 1.21 [1.11–1.33]*** | 1.15 [1.04–1.28]** |
| 75–79 | 1.62 [1.05–2.51]* | 2.42 [1.81–3.27]*** | 1.46 [1.33–1.60]*** | 1.19 [1.07–1.34]** |
| 80–84 | 2.37 [1.51–3.73]*** | 4.21 [3.16–5.66]*** | 1.83 [1.66–2.02]*** | 1.45 [1.28–1.64]*** |
| 85 plus | 2.57 [1.50–4.33]*** | 5.11 [3.76–6.98]*** | 1.95 [1.74–2.19]*** | 1.58 [1.36–1.82]*** |
| Sex | ||||
| Female | Reference | Reference | Reference | Reference |
| Male | 1.16 [0.86–1.57] | 0.60 [0.50–0.72]*** | 1.16 [1.09–1.24]*** | 1.02 [0.94–1.11] |
| % of population with bachelor’s degree or higher | ||||
| < 15% | 2.04 [1.16–3.70]* | 0.93 [0.68–1.26] | 1.06 [0.99–1.30] | 0.99 [0.86–1.15] |
| 15%–24% | 1.36 [0.77–2.47] | 0.78 [0.58–1.05] | 1.05 [0.96–1.22] | 0.95 [0.83–1.09] |
| 25%–49% | 1.61 [0.99–2.70] | 0.88 [0.69–1.11] | 0.98 [0.88–1.14] | 0.96 [0.86–1.08] |
| 50% or higher | Reference | Reference | Reference | Reference |
| Median household income | ||||
| < $30,000 | 1.16 [0.54–3.25] | 1.17 [0.74–1.80] | 0.89 [0.74–1.06] | 1.29 [1.05–1.56]* |
| $30,000–49,999 | 1.33 [0.81–2.20] | 1.15 [0.87–1.52] | 1.01 [0.91–1.13] | 1.16 [1.02–1.33]* |
| $50,000–69,999 | 1.14 [0.71–1.84] | 1.09 [0.84–1.41] | 0.95 [0.86–1.04] | 1.13 [1.00–1.27] |
| $70,000–89,999 | 1.15 [0.72–1.86] | 0.96 [0.74–1.24] | 1.05 [0.96–1.15] | 1.03 [0.91–1.16] |
| ≥ $90,000 | Reference | Reference | Reference | Reference |
| Pharmacy-specific predictors | ||||
| 30-day copay for generic drugs | 1.00 [0.98–1.01] | 1.01 [1.01–1.02]* | 1.00 [1.00–1.01] | 1.01 [1.01–1.02]** |
| < 5 medications | Reference | Reference | Reference | Reference |
| 5–9 medications | 1.30 [0.90–1.89] | 1.14 [0.94–1.39] | 1.09 [1.02–1.18]*** | 1.11 [1.02–1.22]*** |
| ≥ 10 medications | 2.46 [1.59–3.82]*** | 1.29 [0.98–1.70] | 1.36 [1.23–1.51]*** | 1.31 [1.15–1.48]*** |
| No. fills for ACEI/ARB | ||||
| 0 fills | Reference | Reference | Reference | Reference |
| 1 fill | 2.04 [1.16–3.46]** | 0.84 [0.57–1.22] | 1.07 [0.92–1.24] | 0.94 [0.78–1.13] |
| 2 or more fills | 0.91 [0.64–1.31] | 0.73 [0.61–0.89]** | 0.92 [0.85–0.99]* | 0.96 [0.87–1.05] |
| No. fills for oral anti-hyperglycemics | ||||
| 0 fills | Reference | Reference | Reference | Reference |
| 1 fill | 1.22 [0.53–2.44] | 0.93 [0.65–2.06] | 0.92 [0.78–1.09] | 0.81 [0.65–1.00] |
| 2 or more fills | 1.15 [0.81–1.64] | 0.86 [0.75–1.26] | 0.82 [0.76–0.88]*** | 0.83 [0.75–0.90]*** |
| 2 or more fills including insulin | 2.13 [1.44–3.16]*** | 0.93 [1.12–1.59] | 0.88 [0.79–0.97]* | 1.25 [1.13–1.39]*** |
| No. fills for statins | ||||
| 0 fills | Reference | Reference | Reference | Reference |
| 1 fill | 0.91 [0.43–1.82] | 1.53 [1.07–2.15]* | 1.03 [0.88–1.20] | 1.10 [0.91–1.32] |
| 2 or more fills | 1.03 [0.68–1.64] | 0.84 [0.67–1.07] | 0.99 [0.81–1.08] | 0.98 [0.87–1.10] |
| Physiological characteristics | ||||
| LDL | 1.00 [0.99–1.01] | 1.00 [1.00–1.00] | 1.00 [1.00–1.00] | 1.00 [1.00–1.00]] |
| Hemoglobin A1C | 1.07 [0.96–1.19] | 0.95 [0.88–1.03] | 1.03 [1.01–1.06]* | 1.26 [1.23–1.30]*** |
| Systolic blood pressure | 1.00 [0.99–1.01] | 1.00 [0.99–1.00] | 0.99 [0.98–0.99]*** | 1.00 [1.00–1.00] |
| Comorbidities | ||||
| History of outcome (within past 2 y) | 7.42 [3.67–13.6]*** | 4.81 [3.43–6.60]*** | 4.61 [4.28–4.97]*** | 4.15 [3.74–4.60]*** |
| Alcohol abuse | 0.42 [0.02–1.89] | 1.26 [0.57–2.40] | 1.20 [0.93–1.53] | 1.05 [0.74–1.45] |
| Anxiety | 1.13 [0.60–1.95] | 0.93 [0.63–1.32] | 1.16 [1.01–1.33]* | 1.13 [0.95–1.33] |
| Arthritis | 0.94 [0.69–1.28] | 0.87 [0.72–1.04] | 1.01 [0.94–1.08] | 0.97 [0.89–1.05] |
| Asthma | 1.36 [0.91–1.97] | 0.87 [0.66–1.14] | 1.08 [0.97–1.18] | 1.06 [0.94–1.19] |
| Atrial fibrillation | 0.83 [0.51–1.30] | 1.43 [1.14–1.79]** | 1.06 [0.97–1.18] | 1.05 [0.93–1.18] |
| Cancer | 0.82 [0.46–1.35] | 1.08 [0.82–1.41] | 1.07 [0.92–1.24] | 1.02 [0.89–1.17] |
| Chronic kidney disease | 1.95 [1.43–2.67]*** | 1.14 [0.96–1.36] | 1.09 [1.02–1.17]** | 1.64 [1.51–1.78]*** |
| COPD | 1.24 [0.80–1.86] | 1.14 [0.87–1.45] | 1.06 [0.95–1.17] | 1.09 [0.96–1.23] |
| Dementia | 1.04 [0.25–2.82] | 1.26 [0.66–2.19] | 1.08 [0.81 –1.42] | 1.06 [0.73–1.48] |
| Depression | 1.49 [1.03–2.11]* | 1.37 [1.10–1.68]** | 1.21 [1.11–1.32]*** | 1.26 [1.13–1.39]*** |
| Drug abuse | 1.35 [0.33–3.70] | 1.43 [0.60–2.87] | 0.90 [0.63–1.24] | 1.10 [0.74–1.57] |
| Heart failure | 1.19 [0.81–1.73] | 1.09 [0.86–1.37] | 1.19 [1.09–1.31]*** | 1.24 [1.11–1.38]*** |
| Ischemic heart disease | 0.95 [0.68–1.31] | 1.07 [0.88–1.30] | 1.21 [1.13–1.30]*** | 1.14 [1.05–1.25]** |
| Osteoporosis | 1.66 [1.03–2.57]* | 1.49 [1.16–1.90]** | 0.98 [0.86–1.11] | 1.10 [0.95–1.28] |
| Retinopathy | 1.68 [1.23–2.29]*** | 1.70 [1.40–2.07]*** | 1.28 [1.18–1.39]*** | 1.67 [1.53–1.81]*** |
| Stroke | 1.63 [1.01–2.50]* | 1.28 [0.95–1.69] | 1.12 [1.00–1.26] | 1.13 [0.97–1.30] |
| Visual impairment | 1.10 [0.27–2.93] | 1.49 [0.76–2.62] | 1.07 [0.79 –1.41] | 1.38 [1.01–1.85]* |
Bold values indicate statistical significance
ACEI indicates angiotensin-converting enzyme inhibitors; ARB, angiotensin receptor blockers; COPD, chronic obstructive pulmonary disease; ED, emergency department; LDL, low-density lipoprotein.
P < 0.05 significance level.
P < 0.01 significance level.
P < 0.001 significance level.
TABLE 3.
Adjusted Odds Ratios and 95% Confidence Intervals From Logistic Regression Models
| Death | Combined Outcome (Including Death) |
Combined Outcome (Excluding Death) |
|
|---|---|---|---|
| Demographics | |||
| Race/ethnicity | |||
| White | Reference | Reference | Reference |
| Black | 0.98 [0.86–1.10] | 1.22 [1.14–1.31]*** | 1.24 [1.14–1.36]*** |
| Hispanic | 0.93 [0.84–1.02] | 0.98 [0.92–1.03] | 1.01 [0.94–1.09] |
| Asian or Hawaiian/Pacific Islander | 0.82 [0.73–0.92]*** | 0.86 [0.80–0.92]*** | 0.89 [0.82–0.97]** |
| Native American or unknown | 0.42 [0.34–0.50]*** | 0.38 [0.34–0.42]*** | 0.54 [0.48–0.61]*** |
| Age (y) | |||
| 65–69 | Reference | Reference | Reference |
| 70–74 | 1.32 [1.19–1.47]*** | 1.21 [1.14–1.28]*** | 1.13 [1.06–1.21]*** |
| 75–79 | 1.85 [1.67–2.05]*** | 1.56 [1.47–1.65]*** | 1.27 [1.18–1.37]*** |
| 80–84 | 2.68 [2.42–2.99]*** | 2.02 [1.90–2.15]*** | 1.53 [1.42–1.66]*** |
| 85 plus | 4.85 [4.35–5.41]*** | 2.91 [2.72–3.12]*** | 1.65 [1.51–1.81]*** |
| Sex | |||
| Female | Reference | Reference | Reference |
| Male | 1.18 [1.10–1.26]*** | 1.10 [1.05–1.14]*** | 1.04 [0.99–1.10] |
| % of population with bachelor’s degree or higher | |||
| < 15% | 1.17 [1.03–1.32]* | 1.05 [0.97–1.12] | 1.09 [0.99–1.20] |
| 15%–24% | 1.07 [0.96–1.20] | 1.03 [0.97–1.11] | 1.04 [0.95–1.14] |
| 25%–49% | 1.07 [0.98–1.18] | 1.00 [0.95–1.06] | 1.00 [0.93–1.07] |
| 50% or higher | Reference | Reference | Reference |
| Median household income | |||
| < $30,000 | 1.09 [0.91–1.29] | 1.17 [1.06–1.30]** | 1.00 [0.87–1.14] |
| $30,000–49,999 | 1.11 [0.99–1.24] | 1.10 [1.03–1.18]** | 1.05 [0.96–1.14] |
| $50,000–69,999 | 1.08 [0.98–1.19] | 1.07 [1.01–1.14]* | 0.96 [0.89–1.04] |
| $70,000–89,999 | 1.04 [0.95–1.15] | 1.07 [1.01–1.13]* | 1.06 [0.98–1.14] |
| ≥ $90,000 | Reference | Reference | Reference |
| Pharmacy-specific predictors | |||
| 30-day copay for generic drugs | 1.00 [0.99–1.00] | 1.00 [0.99–1.00] | 1.00 [1.00–1.01] |
| < 5 medications | Reference | Reference | Reference |
| 5–9 medications | 1.25 [1.16–1.35]*** | 1.15 [1.10–1.21]*** | 1.13 [1.07–1.20]*** |
| ≥ 10 medications | 1.58 [1.43–1.76]*** | 1.48 [1.39–1.58]*** | 1.46 [1.35–1.59]*** |
| No. fills for ACEI/ARB | |||
| 0 fills | Reference | Reference | Reference |
| 1 fill | 1.12 [0.98–1.29] | 1.10 [1.01–1.20]* | 1.03 [0.93–1.16] |
| 2 or more fills | 0.72 [0.67–0.77]*** | 0.76 [0.73–0.79]** | 0.84 [0.79–0.90]*** |
| No. fills for oral anti-hyperglycemics | |||
| 0 fills | Reference | Reference | Reference |
| 1 fill | 1.18 [1.01–1.38]* | 1.16 [1.05–1.28]** | 1.06 [0.94–1.21] |
| 2 or more fills | 0.81 [0.75–0.87]*** | 0.77 [0.74–0.81]*** | 0.82 [0.77–0.87]*** |
| 2 or more fills including insulin | 1.00 [0.91–1.11] | 1.10 [1.04–1.17]** | 1.16 [1.08–1.25]*** |
| No. fills for statins | |||
| 0 fills | Reference | Reference | Reference |
| 1 fill | 1.05 [0.90–1.21] | 1.07 [0.98–1.18] | 1.07 [0.95–1.20] |
| 2 or more fills | 0.69 [0.63–0.75]*** | 0.88 [0.83–0.93]*** | 0.93 [0.86–1.00] |
| Physiological characteristics | |||
| LDL | 0.99 [0.99–0.99]*** | 0.99 [0.99–0.99]*** | 1.00 [0.99–1.00] |
| Hemoglobin A1C | 0.99 [0.96–1.02] | 1.04 [1.03–1.06]*** | 1.11 [1.09–1.13]*** |
| Systolic blood pressure | 0.99 [0.98–0.99]*** | 0.99[0.99–0.99]*** | 1.00 [0.99–1.01] |
| Comorbidities | |||
| History of outcome (within past 2 y) | NA | NA | 11.0 [10.4–11.6]*** |
| Alcohol abuse | 1.69 [1.35–2.10]*** | 1.55 [1.34–1.80]*** | 1.39 [1.14–1.68]*** |
| Anxiety | 0.92 [0.79–1.07] | 1.12 [1.03–1.22]** | 1.11 [0.99–1.23] |
| Arthritis | 0.88 [0.82–0.94]*** | 0.99 [0.95–1.04] | 1.01 [0.96–1.07] |
| Asthma | 0.97 [0.88–1.07] | 1.01 [0.95–1.07] | 1.06 [0.98–1.14] |
| Atrial fibrillation | 1.29 [1.18–1.40]*** | 1.31 [1.24–1.39]*** | 1.04 [0.98–1.15] |
| Cancer | 1.71 [1.56–1.88]*** | 1.25 [1.17–1.33]*** | 1.02 [0.92–1.11] |
| Chronic kidney disease | 1.38 [1.29–1.47]*** | 1.30 [1.25–1.36]*** | 1.23 [1.17–1.29]*** |
| COPD | 1.61 [1.48–1.75]*** | 1.19 [1.13–1.27]*** | 1.00 [0.92–1.09] |
| Dementia | 2.42 [1.95–2.99]*** | 2.37 [2.01–2.78]*** | 1.22 [0.96–1.55] |
| Depression | 1.17 [1.07–1.27]*** | 1.46 [1.38–1.54]*** | 1.14 [1.06–1.22]*** |
| Drug abuse | 0.79 [0.54–1.10] | 1.05 [0.86–1.27] | 1.09 [0.85–1.39] |
| Heart failure | 2.04 [1.88–2.21]*** | 1.58 [1.50–1.67]*** | 1.25 [1.17–1.34]*** |
| Ischemic heart disease | 1.12 [1.04–1.20]** | 1.20 [1.15–1.25]*** | 1.12 [1.06–1.18]*** |
| Osteoporosis | 1.18 [1.05–1.33]** | 1.27 [1.18–1.36]*** | 1.01 [0.92–1.12] |
| Retinopathy | 1.27 [1.18–1.38]*** | 1.45 [1.39–1.52]*** | 1.54 [1.44–1.63]*** |
| Stroke | 1.37 [1.23–1.53]*** | 1.66 [1.55–1.78]*** | 1.15 [1.05–1.26]*** |
| Visual impairment | 1.48 [1.15–1.89]** | 1.40 [1.18–1.66]*** | 1.19 [0.95–1.49] |
Bold values indicate statistical significance.
ACEI indicates angiotensin-converting enzyme inhibitors; ARB, angiotensin receptor blockers; COPD, chronic obstructive pulmonary disease; ED, emergency department; LDL, low-density lipoprotein; NA, not applicable.
P < 0.05 significance level.
P < 0.01 significance level.
P < 0.001 significance level.
Having > 10 medications was universally predictive of experiencing an adverse event (except for hip fractures) compared with having fewer than 5 medications (Tables 2, 3). For example, patients with > 10 medications were 2 and a half times more likely to experience hypoglycemia versus those with <5 medications (OR, 2.46; CI, 1.59–3.82).
We reported the k-fold validation AUC results in Appendix II (Supplemental Digital Content 2, http://links.lww.com/MLR/B830). The k-fold validation AUCs ranged from 0.643 to 0.813 (hypoglycemia), 0.728 to 0.811 (hip fracture), 0.613 to 0.708 (syncope), 0.598 to 0.744 (ED visit or hospital admission), and 0.606 to 0.790 (combined outcome without death). The calibration plot for the combined outcome without death model showed good model calibration (Appendix III, Supplemental Digital Content 3, http://links.lww.com/MLR/B831).
In Table 4, we reported c-statistics (AUCs) to show the incremental effect of adding a select group of predictors on model performance over a reference model of demographic characteristics (age, sex, race, census block level education, and median household income). The reference models consistently had the lowest AUCs, except for death, while our most complex models that included all predictors had the highest AUCs.
TABLE 4.
C-statistics (Area Under the Curve With 95% Confidence Intervals)
| Hypoglycemia | Hip Fracture | Syncope | Death | ED/Hospital Visit | Combined Outcome (Including Death) |
|
|---|---|---|---|---|---|---|
| Reference* | 0.675 [0.651–0.700] | 0.740 [0.726–0.754] | 0.630 [0.624–0.636] | 0.705 [0.699–0.712] | 0.604 [0.597–0.613] | 0.652 [0.648–0.656] |
| Reference+history of outcome in prior 2 y† | 0.693 [0.662–0.726] | 0.756 [0.740–0.773] | 0.680 [0.672–0.688] | NA | 0.649 [0.640–0.657] | NA |
| Reference+history of outcome in prior 2 y +clinical characteristics‡ | 0.713 [0.679–0.747] | 0.756 [0.738–0.776] | 0.683 [0.675–0.691] | 0.698* [0.691–0.707] | 0.683 [0.674–0.691] | 0.649* [0.643–0.653] |
| Reference+history of outcome in prior 2 y +clinical characteristics +comorbidities§ | 0.772 [0.742–−0.801] | 0.780 [0.763–0.797] | 0.702 [0.695–0.711] | 0.763* [0.757–0.771] | 0.745 [0.738–0.753] | 0.709* [0.705–0.714] |
| Reference+history of outcome in prior 2 y +clinical characteristics +comorbidities +pharmacy-specific factors∥ | 0.802 [0.774–0.830] | 0.786 [0.768–0.803] | 0.706 [0.698–0.715] | 0.772* [0.765–0.779] | 0.761 [0.753–0.769] | 0.718* [0.713–0.722] |
In all death models, history of outcome could not be included.
Reference model: age, sex, race, education, median household income.
History of outcome in prior 2 years.
Clinical characteristics: systolic blood pressure, hemoglobin A1c, and low-density lipoprotein.
Comorbidities: alcohol abuse, anxiety, arthritis, asthma, atrial fibrillation, cancer, chronic kidney disease, chronic obstructive pulmonary disease, dementia, depression, drug abuse, heart failure, ischemic heart disease, osteoporosis, retinopathy, stroke, and visual impairment.
Pharmacy-specific factors: 30-day generic drug copay, count of medications, number of fills in 3 drug classes (angiotensin-converting enzyme inhibitors/angiotensin-receptor blockers, oral anti-hyperglycemics and insulin, and statins).
ED indicates emergency department; NA, not applicable (because history of death was excluded).
Table 5 shows the work-up to detection ratio or NNE for the 6 outcomes at a 95% specificity level. In this case, the NNE is the number of patients a pharmacist would have to evaluate to capture 1 patient who is experiencing an adverse outcome.28 For example, at a model specificity level of 95%, a pharmacist would have to evaluate 6 patients to encounter 1 incidence of syncope. Similarly, at the same specificity level, for every 3 patients a pharmacist evaluates, they encounter one case of the combined outcome including death.
TABLE 5.
Work-up to Detection Ratio (Number Needed to Evaluate)
| Number Needed to Evaluate | ||||||
|---|---|---|---|---|---|---|
| Hypoglycemia | Hip Fracture | Syncope | ED Visit/Hosp Admission |
Death | Combined Outcome (Including Death) | |
| 95% specificity | 88.9 | 34 | 5.6 | 5.7 | 5.3 | 2.6 |
ED indicates emergency department.
DISCUSSION
Pharmacists and other clinicians may benefit from a systematic approach that prioritizes their patient panels based on patients’ risk of experiencing adverse outcomes such as hypoglycemia, hip fractures, syncope, ED visits or hospital admissions, and death. Given that operationalizing 5 separate outcome models in a pharmacy department would be a stretch of resources, we created a combined outcome that was a composite of the 5 outcomes. This composite outcome would allow for a central source of risk stratification of patients with type-2 diabetes.
To our knowledge, this study is the first to assess which patient characteristics are consistently predictive across a panel of these adverse outcomes among patients with type-2 diabetes.
In general, we found that age, the number of medications, prior history of adverse event (within past 2 y), CKD, depression and retinopathy were significantly associated with most or all the outcomes. This suggested that these predictors could be used by pharmacists and other health care providers as markers of increased risk of our outcomes. These isolated predictors could also be subject to further studies where they can be examined in a prognostic model.
Among the pharmacy-specific predictors, our study showed that the number of medications a patient was taking was strongly associated with increased odds of experiencing all the outcomes except hip fracture. We found that patients with > 10 medicines were at higher risk that those with 5–9 medicines and <5 medicines. The number of medicines or polypharmacy has been associated with falls22-25 and death26 in other studies, but to our knowledge, our study was the first to find a strong association with hypoglycemia, syncope, ED visit or hospital admissions, and a combined outcome in patients with type-2 diabetes. Given that pharmacists already perform comprehensive medication reviews, our study highlighted that prioritizing those patients with 10 or more medications for their interventions may help prevent hypoglycemia, syncope, ED visit or hospital admissions, and death. In the oral anti-hyperglycemic drug group, having 2 or more fills including insulin was strongly associated with hypoglycemia and ED visits or hospital admissions while only having 2 or more fills without insulin was associated with lower odds of these outcomes. These findings were consistent with recent literature29,30 and emphasizes the need for pharmacists to target patients on insulin with adherence, dosing, and other insulin-related interventions.
In our study, the consistently significant comorbid conditions across the 5 outcomes were: having a prior history of an outcome within the past 2 years, CKD, depression, and retinopathy. The strong association between having a prior history of hypoglycemia and experiencing future hypoglycemic episodes was consistent with the published literature,29,31,32 but our study also showed that the history of hip fractures, syncope, and ED visits or hospital admissions were also important in predicting recurrences of these adverse events among patients with type-2 diabetes. Our study highlighted depression and retinopathy as the other universal predictors of all our outcomes. This emphasized the opportunity for pharmacists to also prioritize patients with type-2 diabetes with comorbid depression and retinopathy for interventions such as referrals to psychiatry for uncontrolled depression, medication reviews focused on making drug labels legible, identifying drugs with visual and cognitive impairment properties, and general lifestyle assessments, for example support structures at home, and need for prefilled pill boxes, etc.
Through presenting the work-up to detection ratios, we provided pharmacy departments with the approximate impact of implementing our risk prediction models. For example, our findings suggested that using our combined outcome model, pharmacists may only need to review 3 patients flagged by the algorithm to capture 1 incidence of a patient at risk of any of the 5 outcomes. Such information may assist pharmacy departments in budgeting for staffing needs and other components of an intervention program.
Our study was subject to a few limitations that included: our data was from an integrated health system and may not be represent findings in nonintegrated systems; however, our large sample size from multiple states allowed for a diverse population. Our study included only adult patients with type-2 diabetes and therefore may not be generalizable to patients with type-1 diabetes or pediatric patients with type-2 diabetes. Nevertheless, our study’s findings point to a very specific set of risk factors that can be used to prioritize type-2 diabetes patient panels by pharmacists and other clinicians.
CONCLUSIONS
Pharmacists can use the predictors we identified in our study (age, number of medicines, prior 2-y history of adverse events, comorbid CKD, depression, and retinopathy) to prioritize type-2 diabetes patients that are at highest risk of hypoglycemia, hip fractures, syncope, ED visit or hospital admission, and death. Our study also provided pharmacy departments with metrics (work-up to detection ratios) to assess the implementation of our risk prediction models within their health systems.
Finally, while our study focused on pharmacists, we believe that this information is relevant to other providers such as physicians, nurses and diabetes educators. This is because even though each provider may have a different approach of intervening on the patients, our models provide a consistent way to identify and prioritize patients who are at highest risk of the selected adverse outcomes.
Supplementary Material
ACKNOWLEDGMENTS
The authors would like to acknowledge Drs Lynn Deguzman, PharmD and Dean Fredriks PharmD from Kaiser Permanente Northern California for their valuable support on this study.
Supported by grant number 1R21DK103146-01A1 from the National Institute of Diabetes and Digestive and Kidney Disorders.
S.K. and J.S. receive funding from The Permanente Medical Group (TPMG) Delivery Science Fellowship Program. J.S. was supported by the Health Delivery Systems Center for Diabetes Translational Research (CDTR) (NIDDK grant 1P30-DK092924). V.L. was supported by the NIH Grant# NIH R35GM128672 and Grant# NIH K23 GM112018. V.L. also received unrelated funding from the National Institute of General Medical Sciences, and the National Heart, Lung and Blood Institute. G.N. received unrelated research funding from Sanofi, Merck, Janssen, Amarin and Boehringer Ingelheim. The remaining authors declare no conflict of interest.
Footnotes
Supplemental Digital Content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website, www.lww-medicalcare.com.
REFERENCES
- 1.Centers for Disease Control and Prevention, National Diabetes Statistics Report 2017, September 17, 2018. Available at: www.cdc.gov/diabetes/pdfs/data/statistics/national-diabetes-statistics-report.pdf. Accessed September 17, 2018.
- 2.Prudencio J, Cutler T, Marin S, et al. The effect of clinical pharmacist-led comprehensive medication management on chronic disease state goal attainment in a patient-centered medical home. J Manag Care Spec Pharm. 2018;24:423–429. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Greer N, Bolduc J, Geurkink E, et al. Pharmacist-led chronic disease management: a systematic review of effectiveness and harms compared with usual care. Ann Intern Med. 2016;165:30–40. [DOI] [PubMed] [Google Scholar]
- 4.Bukhsh A, Khan TM, Lee SW, et al. Efficacy of pharmacist based diabetes educational interventions on clinical outcomes of adults with type 2 diabetes mellitus: a network meta-analysis. Front Pharmacol. 2018;10:339–340. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.O’Connor PJ, Rush WA, Pronk NP, et al. Identifying diabetes mellitus or heart disease among health maintenance organization members: sensitivity, specificity, predictive value, and cost of survey and database methods. Am J Manag Care. 1998;4:335–342. [PubMed] [Google Scholar]
- 6.Zgibor JC, Orchard TJ, Saul M, et al. Developing and validating a diabetes database in a large health system. Diabetes Res Clin Pract. 2007;75:313–319. [DOI] [PubMed] [Google Scholar]
- 7.Wang EH, Michelle C, Man D, et al. Should there be a cap on the number of patients under the care of a clinical pharmacist. Can J Hosp Pharm. 2012;65:319–321. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Lee AK, Lee CJ, Huang ES, et al. Risk factors for severe hypoglycemia in black and white adults with diabetes: the Atherosclerosis Risk in Communities (ARIC) study. Diabetes Care. 2017;40:1661–1667. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Karter AJ, Lipska KJ, O’Connor PJ, et al. High rates of severe hypoglycemia among African American patients with diabetes: the surveillance, prevention, and Management of Diabetes Mellitus (SUPREME-DM) network. J Diabetes Complications. 2017;31:869–873. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Rubin DJ, Recco D, Turchin A, et al. External validation of the diabetes early readmission risk indicator. Endocr Pract. 2018;24:527–541. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Karter AJ, Warton EM, Lipska KJ, et al. Development and validation of a tool to identify patients with type 2 diabetes at high risk of hypoglycemia-related emergency department or hospital use. JAMA Intern Med. 2017;177:1461–1470. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Berkowitz SA, Karter AJ, Lyles CR, et al. Low socioeconomic status is associated with increased risk for hypoglycemia in diabetes patients: the Diabetes Study of Northern California (DISTANCE). J Health Care Poor Underserved. 2014;25:478–490. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Lipska KJ, Warton EM, Huang ES, et al. HbA1c and risk of severe hypoglycemia in type 2 diabetes: the Diabetes and Aging Study. Diabetes Care. 2013;36:3535–3542. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Nichols GA, Desai J, Elston Lafata J, et al. Construction of a multisite DataLink using electronic health records for the identification, surveillance, prevention, and management of diabetes mellitus: the SUPREME-DM project. Prev Chronic Dis. 2012;9:110–119. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Raebel MA, Xu S, Goodrich GK, et al. Initial antihyperglycemic drug therapy among 241 327 adults with newly identified diabetes from 2005 through 2010: a surveillance, prevention, and management of diabetes mellitus (SUPREME-DM) study. Ann Pharmacother. 2013;47:1280–1291. [DOI] [PubMed] [Google Scholar]
- 16.Schmittdiel J, Raebel M, Dyer W, et al. Prescription medication burden in patients with newly-diagnosed diabetes: the SUrveillance, PREvention, and ManagEment of Diabetes Mellitus (SUPREME-DM) cohort. J Am Pharm Assoc. 2014;54:374–382. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Raebel MA, Ellis JL, Schroeder EB, et al. Intensification of antihyperglycemic therapy among patients with incident diabetes: a Surveillance Prevention and Management of Diabetes Mellitus (SUPREME-DM) study. Pharmacoepidemiol Drug Saf. 2014;23:609–618. [DOI] [PubMed] [Google Scholar]
- 18.Budnitz DS, Lovegrove MC, Shehab N, et al. Emergency hospitalizations for adverse drug events in older Americans. N Engl J Med. 2011;365:2002–2012. [DOI] [PubMed] [Google Scholar]
- 19.Lipska KJ, Ross JS, Wang Y, et al. National trends in US hospital admissions for hyperglycemia and hypoglycemia among Medicare beneficiaries, 1999 to 2011. JAMA Intern Med. 2014;174:1116–1124. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Quan H, Sundararajan V, Halfon P, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43:1130–1139. [DOI] [PubMed] [Google Scholar]
- 21.Nishtala PS, Salahudeen MS. Temporal trends in polypharmacy and hyperpolypharmacy in older New Zealanders over a 9-year period: 2005-2013. Gerontology. 2015;61:195–202. [DOI] [PubMed] [Google Scholar]
- 22.Alenazi AM, Alshehri MM, Alothman S, et al. Functional reach, depression scores, and number of medications are associated with number of falls in people with chronic. Stroke PMR. 2017;10:806–816. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Masumoto S, Sato M, Maeno T, et al. Potentially inappropriate medications with polypharmacy increase the risk of falls in older Japanese patients: 1-year prospective cohort study. Geriatr Gerontol Int. 2018;18:1064–1070. [DOI] [PubMed] [Google Scholar]
- 24.Ming Y, Zecevic A. Medications & polypharmacy influence on recurrent fallers in community: a systematic review. Can Geriatr J. 2018;21:14–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Masnoon N, Shakib S, Kalisch-Ellett L, et al. What is polypharmacy? A systematic review of definitions. BMC Geriatrics. 2017;17:230–240. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Schottker B, Muhlack DC, Hoppe LK, et al. Updated analysis on polypharmacy and mortality from the ESTHER study. Eur J Clin Pharmacol. 2018;74:981–982. [DOI] [PubMed] [Google Scholar]
- 27.Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21:128–138. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Romero-Brufau S, Huddleston JM, Escobar GJ, et al. Why the C-statistic is not informative to evaluate early warning scores and what metrics to use. Crit Care. 2015;19:285–291. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Misra-Hebert AD, Pantalone KM, Ji X, et al. Patient characteristics associated with severe hypoglycemia in a type 2 diabetes cohort in a large, integrated health care system from 2006 to 2015. Diabetes Care. 2018;41:1164–1171. [DOI] [PubMed] [Google Scholar]
- 30.McCall AL. Insulin therapy and hypoglycemia. Endocrinol Metab Clin North Am. 2012;41:57–87. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Pacaud D, Hermann JM, Karges B, et al. Risk of recurrent severe hypoglycemia remains associated with a past history of severe hypoglycemia up to 4 years: results from a large prospective contemporary pediatric cohort of the DPV initiative. Pediatr Diabetes. 2017;19:493–500. [DOI] [PubMed] [Google Scholar]
- 32.Silbert R, Salcido-Montenegro A, Rodriguez-Gutierrez R, et al. Hypoglycemia among patients with type 2 diabetes: epidemiology, risk factors, and prevention strategies. Curr Diab Rep. 2018;18:53–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.