An Interrupted Time Series Analysis to Determine the Effect of an Electronic Health Record–Based Intervention on Appropriate Screening for Type 2 Diabetes in Urban Primary Care Clinics in New York City

Abstract

OBJECTIVE

To determine the impact of a health system–wide primary care diabetes management system, which included targeted guidelines for type 2 diabetes (T2DM) and prediabetes (dysglycemia) screening, on detection of previously undiagnosed dysglycemia cases.

RESEARCH DESIGN AND METHODS

Intervention included electronic health record (EHR)–based decision support and standardized providers and staff training for using the American Diabetes Association guidelines for dysglycemia screening. Using EHR data, we identified 40,456 adults without T2DM or recent screening with a face-to-face visit (March 2011–December 2013) in five urban clinics. Interrupted time series analyses examined the impact of the intervention on trends in three outcomes: 1) monthly proportion of eligible patients receiving dysglycemia testing, 2) two negative comparison conditions (dysglycemia testing among ineligible patients and cholesterol screening), and 3) yield of undiagnosed dysglycemia among those tested.

RESULTS

Baseline monthly proportion of eligible patients receiving testing was 7.4–10.4%. After the intervention, screening doubled (mean increase + 11.0% [95% CI 9.0, 13.0], proportion range 18.6–25.3%). The proportion of ineligible patients tested also increased (+5.0% [95% CI 3.0, 8.0]) with no concurrent change in cholesterol testing (+0% [95% CI −0.02, 0.05]). About 59% of test results in eligible patients showed dysglycemia both before and after the intervention.

CONCLUSIONS

Implementation of a policy for systematic dysglycemia screening including formal training and EHR templates in urban academic primary care clinics resulted in a doubling of appropriate testing and the number of patients who could be targeted for treatment to prevent or delay T2DM.

Introduction

Despite recent surveillance data showing decreased incidence of type 2 diabetes (T2DM) in the U.S., the numbers still indicate an epidemic, with a substantial proportion of individuals with T2DM remaining unaware of their condition (1). Efficacious treatments exist and the consequences of untreated T2DM can be severe (2,3). Additionally, 86 million Americans have prediabetes, a condition in which blood glucose is elevated, increasing the risk of developing T2DM (4–6). Lifestyle and pharmacological interventions in people with prediabetes decreases T2DM incidence (7–10). Early intervention has additional incremental benefits for improving lipid and blood pressure outcomes (8,10,11). Although improved detection of dysglycemia (T2DM or prediabetes) is likely to have a positive impact on outcomes (10,12,13), and despite available screening guidelines (14,15), screening is inconsistent and underutilized in routine clinical practice (16).

In 2012, a formal, health system–wide diabetes management system (DMS) was introduced in primary care clinics of an urban academic health system in New York City (17), as part of a broader reorganization of primary health care delivery in the center, under New York City’s patient-centered medical home (PCMH) initiative. A PCMH is a health care delivery model for primary care intended to provide comprehensive and continuous medical care that focuses on prevention, early detection, and obtaining optimal health outcomes. The American Diabetes Association (ADA) recommends screening for undiagnosed dysglycemia for patients considered to be “at risk” according to a specific algorithm (14,15). Appropriate screening tests for the diagnosis of dysglycemia include the fasting glucose test, the oral glucose tolerance test, and, since 2010, the hemoglobin A_1c (HbA_1c) test; the random glucose test was not considered an appropriate option because it is not a recommended test for the diagnosis of prediabetes (15,18). We have previously shown that although a random glucose test was often ordered in routine primary care, the use of one of the three appropriate tests to detect undiagnosed dysglycemia was not routinely applied (17).

The DMS included a screening component aimed at identifying patients with previously undiagnosed dysglycemia. DMS screening implementation consisted of 1) formal provider and office staff training and 2) establishment of an electronic health record (EHR)–based decision support system to facilitate the determination of screening eligibility and appropriate testing of eligible patients. This study aimed to determine the effect of the DMS screening component implementation on patterns of dysglycemia screening in these primary care clinics. Specifically we tested whether 1) the proportion of patients eligible for dysglycemia screening who received an appropriate screening test was significantly higher after implementation than before; 2) the proportion of patients eligible for dysglycemia screening who received other screening tests not specifically targeted by the DMS screening component, such as cholesterol testing, would not change after implementation; and 3) the proportion of patients ineligible for dysglycemia screening who received a screening test decreased after implementation.

RESEARCH DESIGN AND METHODS

Study Population and Setting

Our study population comprised patients attending five primary care clinics in one academic hospital system of a large health care network in New York City (two private practices and three federally qualified community health centers). All clinics served mainly low- and middle-income populations.

Our sample included all patients aged 18 years or older who had at least one qualifying face-to-face primary care visit in any of the five clinics between 1 March 2011 and 31 December 2013. Patients were excluded if they had a prior diabetes diagnosis or dysglycemia screening in the past 24 months (for those with a prior normal test) or in the past 6 months (for those with a prior abnormal test) as recommended by ADA (15,18). A qualifying visit was one in which vital signs or other assessments (e.g., weight and height) were recorded in the EHR. Only the first qualifying visit each month for a given patient was included. The analysis consisted of a total of 117,589 qualifying visits for 40,456 unique patients.

Study Design

We used an interrupted time series design and segmented regression analysis to test whether the proportion of eligible patients tested for dysglycemia on a monthly basis increased after the implementation of the DMS.

Training on the Screening Component of the DMS

Implementation of the targeted screening program included formal training of providers and office staff regarding 1) the ADA guidelines for appropriate testing in primary care settings, including eliciting the risk factors that determine screening eligibility, and 2) the use of newly developed EHR modifications that guide providers to collect and evaluate risk information from patients and specifically facilitate orders of HbA_1c tests. The EHR modifications included alerts for possible screening based on patient age and BMI, a DM risk factor questionnaire based on the ADA criteria for identifying high-risk patients, and an HbA_1c order prompt. The training was documented in a manual (see training material in the Supplementary Data). The screening component’s implementation process was completed over a 6-month period (1 January–30 June 2012).

Measurements and Case Definitions

The main independent variable was timing of implementation of the DMS’s screening component. The main dependent variable was the monthly proportion of the appropriate tests for dysglycemia in patients eligible for screening, before and after DMS implementation.

All data were extracted from the outpatient EHR. Screening eligibility was determined for each patient at the first visit of each month. Appropriate tests for dysglycemia were counted if they occurred at or after that first visit, at any time during the month.

We counted patients as eligible if they 1) lacked a diabetes diagnosis prior to or at the visit; 2) were at risk based on ADA guidelines, as we could determine from available outpatient EHR data (Table 1); and 3) had no recent appropriate testing for dysglycemia, as described above for patient selection. A diabetes diagnosis was based on ICD-9 codes (249–250.93 or 648.01–648.03) or prescribed medications for diabetes (16). At-risk screening eligibility made use of the following data (15): sociodemographic data (age during the target month, race/ethnicity, and sex); clinical data (average of recorded BMIs and systolic and diastolic blood pressure data during the index visit or two prior months); laboratory data (average HDL cholesterol and triglyceride levels during the 2 months prior to the index visit); ICD-9 codes for hypertension (HTN), cardiovascular disease (CVD), and polycystic ovarian syndrome (PCOS); and prescription medications related to HTN and hypertriglyceridemia for the 24 months prior to the index visit. Patients were counted as eligible for screening if they were 1) at least 45 years old or 2) had a BMI >25 kg/m² plus at least one additional risk factor: nonwhite race/ethnicity, systolic blood pressure ≥140 mmHg or diastolic blood pressure ≥90 mmHg, ICD-9 diagnosis of HTN or antihypertensive medication, HDL cholesterol <0.9 mmol/L (35 mg/dL), triglycerides >2.82 mmol/L (250 mg/dL) or medication for high triglycerides, or ICD-9 diagnosis of PCOS or CVD. Cutoffs followed the current ADA recommendations for testing (15,18).

Table 1.

Criteria for testing for T2DM in asymptomatic adult individuals (and corresponding available EHR data fields) used to determine “at risk” for the purposes of the analyses in this paper

1. All adults (age >18 years) who are overweight (BMI ≥ 25 kg/m2) and have one additional risk factor as below:

• Members of a high-risk race/ethnicity (e.g., African American, Latino, Native American, Asian American, Pacific Islanders) (nonwhite race/ethnicity)

• HTN (≥140/90 mmHg or on therapy for HTN [systolic blood pressure ≥140 mmHg or diastolic blood pressure ≥90 mmHg; ICD-9 diagnosis of HTN or antihypertensive medication])

• HDL cholesterol level <35 mg/dL (0.90 mmol/L) and/or a triglyceride level >250 mg/dL (2.82 mmol/L [HDL cholesterol <0.9 mmol/L (35 mg/dL); triglycerides >2.82 mmol/L (250 mg/dL) or medication for high triglycerides])

• Women with PCOS (ICD-9 diagnosis of PCOS)

• History of CVD (ICD-9 diagnosis of CVD)

2. Age 45 years or older

Data Analysis

We used segmented regression analysis to assess the extent to which the DMS screening policy implementation was associated with changes in testing rates among eligible and ineligible patients. The periods before and after the implementation period (excluding the 6-month implementation period) constitute the two segments of our regression models; these periods include 10 monthly time points before and 18 monthly time points after the policy implementation. The average monthly number of observations was 3,347 (range 2,442–4,472), including 2,588 (range 1,835–3,616) among eligible patients and 759 (range 555–1,074) among ineligible patients.

We first plotted the monthly number of tests among eligible patients and visually compared the patterns of monthly proportions of patients tested before and after the implementation. We fit segmented least squares regression models to the monthly series, with parameters for intercept, baseline trend, and changes in level and trend after the intervention, assuming linearity of the trend lines within each segment. We tested for up to six-order autocorrelation, using the Durbin Watson statistic as a measure of autocorrelation. Using the parameter estimates resulting from the model, we estimated the difference between observed and expected proportions of people screened at the end of the implementation period and after 18 months of follow-up that occurred after the 6-month implementation. We report the mean difference between observed and expected proportions and 95% CIs.

Potential confounding in time series studies is limited to factors that are related to the outcome of interest and that change at the time of the intervention. To test whether changes in the composition of the study population could account for any observed shift, we included the percentages within discrete age, sex, and race/ethnicity categories and the proportion of patients eligible for screening in our models. As none of these factors changed substantially over time, they were not included in final models. We also stratified our analyses by site to test whether the intervention had different effects at the different sites. The results were similar by site so the final results are reported by pooling the data from all the sites.

The analyses were repeated using two negative comparison conditions: 1) the monthly proportion of cholesterol tests among patients eligible for dysglycemia screening (we expected to find no change associated with the DMS screening policy implementation) and 2) the monthly proportion of dysglycemia tests among ineligible patients (we expected to find a decrease, if the algorithm for determining eligibility for screening was implemented correctly).

To better understand reasons for testing of ineligible subjects, we conducted a post hoc analysis by manually reviewing 150 randomly selected charts of patients classified as ineligible who received testing. Data not available from the extracted EHR fields were manually recorded and included eligibility for screening beyond the 2-month window around the face-to-face visit, family history of diabetes, prior diagnosis of prediabetes beyond the 24-month window, and symptoms, other illnesses, and medications not included in the ADA eligibility criteria.

Finally, we calculated the proportion of eligible patients whose testing results indicated values consistent with dysglycemia (either T2DM or prediabetes), according to current ADA criteria (18). Using the same time series analyses described above, we examined whether this proportion changed after implementation.

The study was approved by the institutional review board of the St. Luke’s Roosevelt Hospital Center’s Health Sciences Institute.

Results

Table 2 shows the overall number of visits in the periods before and after DMS screening policy implementation, as well as the distribution of age, sex, race/ethnicity, BMI, HTN, and CVD in the two periods. There were no meaningful differences in the average sociodemographic or clinical characteristics of the patients seen in the clinics before and after policy implementation.

Table 2.

Demographic and clinical characteristics of patients seen during face-to-face visits before and after implementation of the screening component of the DMS

	All visits (n = 117,589)	Visits before implementation (n = 40,322)	Visits after implementation (n = 53,401)
Age, years
Mean (SD)	49.3 (18.0)	50.2 (17.9)	48.5 (18.1)
Median	49	50	48
Sex
Female	73,763 (63)	25,213 (63)	33,399 (63)
Male	43,826 (37)	15,109 (37)	20,002 (37)
Race/ethnicity
Hispanic	46,241 (39)	16,413 (41)	20,281 (38)
Non-Hispanic white	18,308 (16)	6,317 (16)	8,250 (15)
Non-Hispanic black	40,265 (34)	13,090 (32)	19,009 (36)
Non-Hispanic Asian	2,640 (2)	917 (2)	1,198 (2)
Other	10,135 (9)	3,585 (9)	4,663 (9)
Baseline BMI (kg/m2)
<25	37,948 (34)	12,069 (33)	18,190 (36)
25–29.9	36,819 (34)	12,247 (34)	16,968 (33)
>30	34,542 (32)	11,792 (33)	15,564 (31)
Any HTN
No	68,762 (58)	22,380 (56)	32,682 (61)
Yes	48,827 (42)	17,942 (44)	20,719 (39)
Any CVD
No	109,152 (93)	37,347 (93)	49,691 (93)
Yes	8,437 (7)	2,975 (7)	3,710 (7)

Data are presented as n (%) unless otherwise indicated.

The proportion of eligible patients who received appropriate testing was stable during the 10 months before the intervention (slope = 0% screened per month [95% CI −0.2, 0.2]), with an estimated proportion of 9.1% (95% CI 8.0, 10.2) in the month immediately before implementation of the DMS screening component (Fig. 1). During the 18 months after implementation, there was a doubling in the proportion of eligible patients who received appropriate testing for dysglycemia, with an average increase of 11.0% (95% CI 9.0, 13.0) in the proportion of eligible patients who received testing.

Figure 1.

Time series of proportions (%) of those receiving glucose testing among those eligible or ineligible for dysglycemia screening (triangles and circles, respectively) based on ADA-endorsed criteria and proportions (%) of those receiving cholesterol testing among those eligible for dysglycemia screening (squares) (y-axis), by month (x-axis). Fitted trend lines show predicted values from the segmented regression analysis without the intervention. Glucose testing in the figure refers to any of the three ADA-endorsed screening tests for undiagnosed T2DM and prediabetes (dysglycemia).

The proportion of eligible patients whose test results showed dysglycemia was 59% (9% for T2DM and 50% for prediabetes); pre- versus postimplementation differences were not statistically significant (all P > 0.05) (not shown) and estimates were similar to our previous reports in this population (16).

Figure 1 also shows the results of screening for the ineligible comparison group. The monthly proportions of ineligible patients screened was substantially lower than in patients eligible for screening, but dysglycemia screening among ineligible patients also doubled after policy implementation (level change = 5.0% [95% CI 3.0, 8.0], slope change = 0.4% [95% CI −0.2, 1.0]). Cholesterol testing among patients eligible for glucose testing did not change after the dysglycemia screening policy implementation (level change = 0% [95% CI −0.02, 0.05], slope change = 0% [95% CI −0.9, 0.9]).

When analyses were repeated in subgroups defined by age categories (<45 years of age, between 45 and 70 years of age, and >70 years of age), sex (female/male), and race/ethnicity categories (as in Table 1), to explore whether patterns differed by these potential modifying factors, the results were unchanged and are therefore not presented here.

Because of the increased testing among patients who were ineligible, we conducted a validation review of 150 randomly selected charts of ineligible but tested patients. We found that 84.7% (n = 127) of these patients were confirmed to be ineligible for testing based on ADA guidelines. However, among them, 49 patients had diagnoses of depression or other major psychiatric illnesses or were treated for HIV infection or other infections or had symptoms other than polyuria and polydipsia (for example, erectile dysfunction, fatigue, weight loss, etc.). The remaining 15.3% (n = 23) were potentially eligible to be screened according to the ADA criteria. For example, 13 patients had sufficient criteria to be screened if BMI in the algorithm was extended beyond the 2-month window. Additionally, two patients had information listed in their chart that was not captured in a structured field, such as family history of diabetes or prior diagnosis of prediabetes, which would have made them eligible for testing.

Conclusions

Implementation of an EHR system with decision support targeting ADA recommendations resulted in a doubling of screening of eligible patients. This evaluation of a large-scale dysglycemia screening program in primary care clinics of a U.S. academic urban health care system is, to our knowledge, the first report of a systematic dysglycemia screening applied in such a setting. Our interrupted time series design is considered the strongest quasi-experimental design to evaluate longitudinal effects of a time-delimited intervention (19–21). Our findings suggest that this relatively easily implemented policy can result in improvements in dysglycemia identification and management on a large scale. In the eligible urban primary care population in our study, the proportion of test results in the T2DM and prediabetes range was very high, at 9 and 50%, respectively, and higher than reported in other populations (22,23). The increased testing therefore should provide an opportunity to intervene, to prevent or delay T2DM or its complications.

Rigorous training of study personnel and/or physician training and follow-up of physician behavior to ensure proper identification of appropriate patients for T2DM prevention and treatment protocols has been previously reported in the context of randomized controlled trials (7,22–29). However, the use of ADA screening criteria by primary care physicians in day-to-day primary care practice and after specific training, as we report here, has not been previously examined. Results from the 2007–2012 National Health and Nutrition Examination Survey showed that 51% of individuals surveyed self-reported being tested for dysglycemia (14); however, this estimate did not exclude random glucose tests and thus likely overestimated levels of appropriate testing (15,16). Previous reports of opportunistic screening using one of the three ADA-recommended tests or previously undiagnosed dysglycemia in primary care practices in the U.S. ranged from 6 to 21% (16,30–32). As in these previous studies, although our program doubled screening rates, the resulting 24% remains inadequate to address the ongoing diabetes epidemic in the U.S.

Screening for dysglycemia is controversial and debated by some (33,34); however, most expert groups have recommended appropriate glucose testing for individuals deemed at risk according to various criteria (13–16,18). These recommendations have been strengthened (16) by findings that 1) prediabetes carries a large predictive risk for the development of T2DM, 2) lifestyle intervention for the treatment of prediabetes is effective and efficacious in preventing or delaying T2DM, and 3) there is extremely low awareness of prediabetes among the general public (12,35–37). Our results support the guidelines to broadly screen for dysglycemia in the U.S. (16); in addition, these findings can be used to study the feasibility of including such widespread screening in new state-based health cost reimbursement programs (38).

Although the policy we evaluated was associated with a doubling of eligible patient screening, three-fourths of eligible patients did not receive screening. Possible reasons are as follows: 1) we may have underestimated the actual number screened since we considered only each patient’s first qualifying visit and many may have been screened in later visits, or 2) patients who received referrals for testing may not have followed up with blood draw/screening recommendations during the 1-month review period. Further exploration of physician and patient behavior related to dysglycemia screening should be undertaken to identify possible areas for improvement.

The program did not result in increases in screening for cholesterol but did result in increased testing for patients who were ineligible for diabetes screening. Our manual chart review confirmed that most tested patients classified as ineligible were truly ineligible according to the ADA risk criteria. We found that many of these patients had one isolated ADA criterion, such as BMI in the overweight/obese range, nonwhite race, PCOS, presence of HTN, or a family history of T2DM. Physicians may have made a conscious decision to test them based on the presence of just one criterion or they may have followed the earlier U.S. Preventive Services Task Force recommendation on glucose screening among people with HTN (16). In addition, others had risk factors, such as major psychiatric illnesses, HIV infection, or other infections, or had symptoms other than polyuria and polydipsia (for example, erectile dysfunction, fatigue, weight loss, etc.), which were not specifically included in the 2010 ADA screening guidelines (15).

Although this study is limited by its reliance on EHR data of uncertain accuracy to apply risk stratification criteria and to measure the occurrence of testing, it is likely to represent the knowledge that providers had about their patients at the time when the decision about ordering a screening test was made. We could not capture family history, gestational diabetes, level of physical activity, or presence of acanthosis, risk factors that are noted in the ADA guidelines but not consistently available in our EHR.

Conclusion

Although in absolute terms, the majority of eligible clinic patients seen over a month period were not tested, broadly applied, relatively simple policies, which included formal staff training and EHR modifications aimed to facilitate screening for undiagnosed dysglycemia, resulted in a doubling of appropriate testing and the number of patients who could be targeted for treatment to prevent or delay T2DM. The effect of such policies on the delivery of diabetes prevention treatment and ultimately on T2DM incidence should be further studied.

Appendix

Collaborators. Dr. Brenda Matti (Division of General Medicine at St. Luke’s Roosevelt Hospital Center; data collection), Dr. Carolyn Chu (William F. Ryan Center Health Care Team; data collection), Dr. Francisco Perez Mata (Division of Endocrinology, Department of Medicine, Icahn School of Medicine at Mount Sinai; data collection), Julian Botta (NY Obesity Research Center, St. Luke’s Roosevelt Hospital Center; data collection and data anaylsis), Pindan Hao (Division of Endocrinology, Department of Medicine, Icahn School of Medicine at Mount Sinai; data collection and data analysis), and Dr. Carolina Hurtado (Division of Endocrinology, Department of Medicine, Icahn School of Medicine at Mount Sinai; data collection).