Introductory Overview of the Natural Experiments for Translation in Diabetes 2.0 (NEXT-D2) Network: Examining the Impact of US Health Policies and Practices to Prevent Diabetes and Its Complications

Abstract

Purpose of Review

Diabetes incidence is rising among vulnerable population subgroups including minorities and individuals with limited education. Many diabetes-related programs and public policies are unevaluated while others are analyzed with research designs highly susceptible to bias which can result in flawed conclusions. The Natural Experiments for Translation in Diabetes 2.0 (NEXT-D2) Network includes eight research centers and three funding agencies using rigorous methods to evaluate natural experiments in health policy and program delivery.

Recent Findings

NEXT-D2 research studies use quasi-experimental methods to assess three major areas as they relate to diabetes: health insurance expansion; healthcare financing and payment models; and innovations in care coordination. The studies will report on preventive processes, achievement of diabetes care goals, and incidence of complications. Some studies assess healthcare utilization while others focus on patient-reported outcomes.

Summary

NEXT-D2 examines the effect of public and private policies on diabetes care and prevention at a critical time, given ongoing and rapid shifts in the US health policy landscape.

Introduction

Approximately 30.3 million Americans (9.4% of the population) have diabetes and another 84 million have prediabetes and are at risk for progression to type 2 diabetes [1]. Estimates place the total cost of diabetes to the USA at $245 billion per year in direct and indirect costs, including reduced productivity, disability, and premature mortality [2, 3]. One of every 3 Medicare dollars and 1 in 5 healthcare dollars are spent providing care for someone with diabetes [2]. The diabetes epidemic is truly a societal problem, affected by decisions, programs, and policies within a broad socio-ecological landscape at all levels of American society, from individuals and their social networks to providers, health systems, and government. Consequently, in responding to the epidemic, it is essential to examine the real-world effectiveness of key policies, programs, and interventions that are directly or indirectly aimed at entire populations using a systematic approach (e.g., natural experiments). Natural experiments are exposures or changes not directly manipulated by researchers, but are rather the result of policy or program interventions that vary in their implementation along a number of possible dimensions, such as time, geography, or content [4]. Well-designed natural experimental studies can provide robust evidence about effects on diabetes-related outcomes, including potential unintended consequences such as worsening healthcare disparities or increased utilization in the absence of improved outcomes. This evidence can inform future decisions about program and policy implementation, with the ultimate goal of improving health and quality of life for individuals with diabetes or those at risk for diabetes.

Unfortunately, public health research has fallen short in this critical undertaking. Some health-related programs and policies have not been comprehensively evaluated in peer-reviewed publications (e.g., the Aetna Diabetes Leap health plan limited to patients with diabetes, [5] state-level Woman, Infants and Children Farmers’ Market Nutrition Programs that provide coupons for fresh fruits and vegetables [6], city by city adoption of “Complete Streets” programs designed to enhance safety and convenience for all users [7]). Others are evaluated with poor research designs highly susceptible to bias and confounding, which can result in flawed results (e.g., published studies assessing the health impact of influenza vaccination, or the introduction of electronic health records) [8••]. While controlled trials by a team of researchers that randomize at the individual level are the gold standard for clinical efficacy research, they are usually impractical or inappropriate to test real-world policy effectiveness due to logistical, ethical, and/or political constraints [9–11].

The next best available option is to apply rigorous analytic methods to observational data to emulate a hypothetical randomized trial [12]. We now have convincing evidence that well-designed observational data studies provide very comparable effect estimates when compared to randomized trials examining the same outcomes [13]. The Cochrane Collaboration specifies several observational designs as meeting this level of rigor, including quasi-randomized trials, controlled before-and-after studies, and interrupted time series [14]. A recent Cochrane publication comparing reviews of observational studies and randomized controlled trials (RCTs) that examined equivalent research questions found no significant difference in effect sizes between the two types of study designs [15•]. When the investigators limited observational analyses under consideration to cohort studies and non-pharmacological interventions, the effect estimates for observational studies were even closer to RCT results.

Glass and colleagues provide several recommendations to maximize the validity and usefulness of results from observational data analyses, as such analyses should be carefully planned, executed, and reported [12]. First, impactful observational data analyses must study policies or programs that are potentially modifiable, as opposed to fixed factors that cannot readily be changed. Second, these analyses should have specific, well-defined research questions analogous to an RCT, e.g., comparing patients exposed to a new, discrete policy change with other patients who are not exposed, in order to simulate the impact of randomization to this new policy using a “counterfactual” scenario. This recommendation aligns closely with evaluations of natural experiments which use pragmatic research designs and readily available data sources to evaluate and compare a new or existing policy to other alternatives or to what may have happened in the absence of any intervention [16–19]. Third, observational data analyses must adjust for confounders using state-of-the-science approaches such as propensity score techniques and inverse probability weighting, or marginal structural modeling in the case of time-varying confounders, to minimize the problem of non-comparability between groups [12, 20, 21]. Fourth, analyses of observational data must be transparent in all phases including reporting of inclusion/exclusion criteria, data missingness, and analytic methods and results, to enable informed assessments of study quality by outside observers. Transparent and detailed descriptions of study designs and analytic methods guard against the greatest threats to the validity of observational data analyses, since others will be able to determine whether there are additional potential confounders that are not being measured, or whether the confounders that are included may be mismeasured [22].

Description of the Network

The Natural Experiments for Translation in Diabetes 2.0 (NEXT-D2) Network is a research collaboration of eight academic centers using rigorous quasi-experimental study designs to evaluate opportune naturally occurring experiments in healthcare policy and practice with a focus on diabetes-related outcomes. NEXT-D2 builds upon activities of the original NEXT-D Network of five centers funded by the Centers for Disease Control and Prevention (CDC) and the National Institutes of Health (NIH) that launched in 2010, called “Natural Experiments and Effectiveness Studies to Identify the Best Policy and System Level Practices to Prevent Diabetes and Its Complications” [19]. Several NEXT-D reports have described the impact of population-targeted policies (e.g., electronic health record screening decision prompts, lifestyle modification programs, targeted copayment reductions for diabetes care, high deductible health plans) on preventive behaviors and diabetes outcomes, quantity and quality of care, morbidity and costs, and unintended consequences of these policies [9, 23–36].

NEXT-D2 is funded by the CDC, the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) at the NIH, and the Patient-Centered Outcomes Research Institute (PCORI) to evaluate a new set of natural experiments across three areas: health insurance expansion; healthcare financing and payment models; and innovations in care coordination (Table 1). Each of the funding agencies is closely integrated within NEXT-D2 activities, working directly with the academic centers to ensure that the research is conducted and results disseminated in ways that maximize impact for individuals with or at risk for diabetes. Additional details regarding NEXT-D2 including information for patients, researchers, and policy makers are available on the network website: [33].

Table 1.

Summary of the eight projects in NEXT-D2

Team/project	Policy/program	Data sources	Setting and population	Main outcomes	Main analytic method
Northwestern University	ACA Medicaid expansion	Electronic health records from 2009 to 2019 from the Chicago Area Patient-Centered Outcomes Research Network (CAPriCORN) and the Greater Plains Collaborative Clinical Data Research Network	10 million patients in 5 Medicaid expansion and 4 non-expansion states	Diabetes diagnosis, treatments, HbA1c levels, hypoglycemic, and other diabetes-related medications	Difference-in-difference methods
Oregon Health & Sciences University/Prevent-D	ACA Medicaid expansion	EHR data from the ADVANCE clinical data research network, which has data from > 700 community health centers and Oregon Medicaid claim data	Patients aged 19–64 with diabetes, prediabetes, at risk for type 2 diabetes, or no diabetes	Insurance status, healthcare delivery, diabetes biomarkers (e.g., HbA1c), Medicaid expenditures	Difference-in-difference
Harvard Medical School and Harvard Pilgrim Health Care Institute /HDHP impacts	Employer-mandated transition to high-deductible insurance coverage	OptumInsight (Eden Prairie, MN) Clinformatics commercial insurance claims database	Patients with diagnosis of diabetes from 2005 to 2017	Inpatient, emergency room, and outpatient service utilization, out-of-pocket costs, and total healthcare service consumption	Interrupted time series design and difference-in--difference
Pennsylvania State University College of Medicine at Hershey Medical Center	Universal preventive coverage for obesity screening and counseling	EHR and claims data from the PaTH Clinical Research Data Network (CDRN), a partnership of six health systems in 3 states	Patients with overweight and obesity with diabetes or who are at risk for type 2 diabetes	Weight loss, patient-reported outcomes (e.g., self-rated health, depression, physical activity, fruit and vegetable consumption), diabetes incidence, diabetes-related processes of care	Multi-level mixed effects models, difference-in--difference
Tulane University	Introduction of a CMS CPT billing code for non-face-to-face chronic care management services	EHR and claims data from the Research Action for Health Network (REACHnet)	Patients aged 35–94 with diabetes plus at least one other chronic condition	Uptake and use of the new CMS billing code, diabetes quality indicators (e.g., HbA1c < 7%, blood pressure < 140/90 mmHg, etc.)	Regression discontinuity, difference-in--difference
University of California, Berkeley	CMS State Innovation Model (SIM) Initiative	Population health survey data (BRFSS, HCUP), provider and system survey data (NSPO, N-SHOS)	Physician organizations, adult patients with diabetes	Organizational: adoption of diabetes care management, health IT, implementation of core PCMH components. Individual: diabetes-related health behaviors (e.g., physical activity, tobacco use, etc.)	Stepped-wedge, difference-in--difference
Icahn School of Medicine at Mt. Sinai	New York State’s Health Home program	EHR data from the NYC-CDRN, which includes 7 major health systems, Medicaid claims, Medicaid HH program data, key informant interviews, focus groups	Adults with diabetes who have been insured by Medicaid during the study period	-Quality of care (e.g., nephropathy screening) -Utilization (e.g., diabetes-related preventable admissions) -Intermediate outcomes (e.g., HbA1c, blood pressure) -Diabetes complications	Interrupted time series with comparison series and difference-in--differences
University of California, Los Angeles	UnitedHealthcare Medicaid practice-level innovation (Accountable Care Communities) and Medicaid Health Home program	UnitedHealthcare Medicaid claims data	Adult Medicaid patients with diabetes or at risk for type 2 diabetes	Diabetes-related health outcomes (e.g., HbA1c), processes of care (e.g., microalbuminuria screening), utilization and cost of care	Interrupted time series design, regression discontinuity, and difference-in--difference

ACA Affordable Care Act, EHR electronic health record, HDHP high-deductible health plan, CMS Centers for Medicare and Medicaid Services, CPT current procedural terminology, BRFSS Behavioral Risk Factor Surveillance System, HCUP Healthcare Cost and Utilization Project, NSPO National Study of Physician Organizations, N-SHOS National Survey of Health Organizations and Systems

The original NEXT-D Network developed a conceptual socio-ecologic framework to help researchers and policy makers carefully consider the wide spectrum of programs and policies designed to impact diabetes prevention and control. We have placed each of the eight studies in NEXT-D2 on this framework at the level/s where the natural experiment under study has its intended mechanism of action, although there may be additional effects at other levels (Fig. 1). Programs can be implemented by and within individual health systems, represented by the innermost circle. Other programs are initiated by suppliers and purchasers of healthcare such as private employers and private or public health insurers, represented by the next surrounding circle. As depicted in the third circle, institutions in the broader community, including religious, social service, civic, and commercial organizations, provide resources for and support programs designed to prevent type 2 diabetes or to improve care for patients with diabetes. Finally, the laws and regulations that shape the economic, physical, social, and cultural environments of Americans with or at risk for diabetes are represented in the outermost circle.

Fig. 1.

Theoretical framework for Natural Experiments for Translation in Diabetes 2.0 (NEXT-D2). Levels of analysis by lead academic centers: 1, Northwestern University; 2, Oregon Health & Science University; 3, Harvard Medical School and Harvard Pilgrim Health Care Institute; 4, Pennsylvania State University College of Medicine at Hershey Medical Center; 5, Tulane University; 6, University of California, Berkeley; 7, Icahn School of Medicine at Mt. Sinai; 8, University of California, Los Angeles. (Reprinted from American Journal of Preventive Medicine Volume 48, Ackermann et al., Evaluating Diabetes Health Policies Using Natural Experiments: The Natural Experiments for Translation in Diabetes Study, pp. 747–54, copyright 2015, with permission from Elsevier) [19]

Another key feature of NEXT-D2 is an explicit focus on stakeholder engagement in natural experiment research, including patients, providers, and policy makers. There is a growing consensus that stakeholder engagement is an important aspect that spans the research process, from study conception and design through the analytic and reporting phases [37]. Stakeholder engagement and involvement may result in research that is more patient centered, useful, and trustworthy and will ultimately lead to greater use and uptake of research results by patients and the broader community [38, 39]. This is particularly important for NEXT-D2, to ensure that the full spectrum of socio-ecologic influences of natural experiments (Fig. 1) is being considered from the patient’s perspective. The importance of strong quasi-experimental designs and scientific rigor in research to maximize causal inference and minimize flawed conclusions is not necessarily intuitive to patients, communities, and other stakeholders in diabetes care and prevention. Working with patients, healthcare systems, and other key stakeholders to identify effective ways to convey the advantages of strong natural experiment research designs, including patient-centered outcomes, is an important goal of NEXT-D2. To formalize the NEXT-D2 commitment to stakeholder engagement, an engagement committee was formed that includes NEXT-D2 researchers together with patients who are partnering with the individual research studies, which meets monthly with an agenda to share best practice strategies across the network. This goal of stakeholder engagement has been operationalized by inclusion of patient partners and other relevant stakeholders in the engagement committee, attending in-person meetings, participating in manuscript preparation, and delivering external presentations to patient peers and national meetings.

A recent systematic review conducted by the Agency for Healthcare Research and Quality (AHRQ) identified several recommended practices for working with stakeholders in research, many of which are being employed across studies within the network [40]. These methods help to ensure the key responsibility of stakeholder engagement in meeting the research engagement principles of reciprocal relationships, co-learning, partnership, and trust, transparency and honesty [38]. Individual studies within the network highlight different levels and types of stakeholder engagement. For studies funded by PCORI, there is an increased emphasis on the incorporation of patient and stakeholder partner engagement throughout the study period. For example, one study has identified patients as study co-investigators on the study protocol for institutional review board submission.

We describe below the individual research projects in NEXT-D2, each of which focuses on the care or prevention of diabetes. Each project evaluates a policy or program using a specific, well-defined research question and employs statistical methods to reduce bias from confounding. The projects fall under three major themes—health insurance expansion, value-based healthcare and financing models, and innovations in care coordination, as detailed in Table 1.

Health Insurance Expansion

Northwestern University—the Effect of the Affordable Care Act Medicaid Expansion on Diabetes Treatment and Outcomes

The Affordable Care Act (ACA) provided federal support for individual states to expand Medicaid eligibility as of 2014 [41]. Some states implemented the expansion while others did not, creating a natural experiment in which outcomes can be compared in “expansion” and “non-expansion” states. The Northwestern University study will analyze electronic health records from the Chicago Area Patient Centered Outcomes Research Network (CAPriCORN) and the Greater Plains Collaborative Clinical Data Research Network which include a sample of over 10 million patients in five Medicaid expansion and four non-expansion states from 2009 (pre-treatment baseline) through 2019. This study window will provide sufficient time following the Medicaid expansion, to evaluate measures of diabetes diagnosis, diabetes treatment, and importantly also health-related outcomes. This study will use difference-in-difference methods, with the large sample providing sufficient power to detect small changes in hemoglobin A1c (HbA1c) levels and other outcomes and to quantify adherence to hypoglycemic and other diabetes-related medications.

Oregon Health & Science University—Post ACA Reform: EValuation of community hEalth ceNTer care of Diabetes (PREVENT-D)

As a key front-line source of care for low-income Americans, community health centers (CHCs) in the safety net are providing services to many patients who gained Medicaid coverage through the ACA expansion [42, 43]. This study will analyze electronic health record data from the ADVANCE clinical data research network [44] including 470 CHCs in 12 Medicaid expansion states (n = 1,242,823 patients) and 248 CHCs in 9 non-expansion states (n = 830,399 patients), to assess changes in receipt of services and health-related outcomes pre- and post-expansion. [45] As with the study led by Northwestern University, the findings of the PREVENT-D study will evaluate whether the Medicaid expansion affects processes of care for diabetes as well as health-related outcomes. Additionally, econometric analyses will inform the impact of the ACA Medicaid expansion on Medicaid expenditures.

Healthcare Financing and Payment Models

Harvard University—Effects of Tailored High-Deductible Health Plans on Diabetes Outcomes: Informing the Future of Health Insurance Benefit Design

High-deductible health plans (HDHPs) that require patients to pay up to $1000–$6000 in out-of-pocket costs per year are rapidly replacing low-cost-sharing insurance plans [46, 47]. Fourteen percent of all Americans with commercial health insurance are now covered by health savings accounts (HSAs), a component of some HDHPs that allows for pre-tax dollars to be used for medical expenses [48]. Employers can tailor HSAs to minimize the potential financial burden on patients by purchasing additional coverage to make key preventive medicines, including those to manage diabetes-related comorbidities, free to patients and by depositing annual contributions into HSAs in order to offset out-of-pocket costs. Using an interrupted time series design and difference-in-difference analyses, this study will analyze data from > 60 million health plan members within a 12-year rolling cohort (2005–2017) to determine the impacts of HSA-HDHPs on healthcare utilization and health outcomes of patients with diabetes.

Pennsylvania State University—a Patient-Centered PaTH to Address Diabetes: Impact of Obesity Counseling

In 2012, the Centers for Medicare and Medicaid Services (CMS) introduced full insurance coverage for intensive behavioral therapy (IBT) as counseling for obesity, implementing a specific healthcare procedure billing code for this purpose [49]. Other insurers moved to provide universal coverage of IBT in 2013 for adults of all ages with obesity, also without any out-of-pocket cost. The overarching goal of this study is to understand the effectiveness of IBT for obesity as covered by CMS in improving weight loss for adults either with or at high risk of type 2 diabetes. Since overweight patients are at highest risk for type 2 diabetes, improved weight management services could prevent type 2 diabetes and its negative health outcomes. This study will compare weight and diabetes outcomes in patients across three states using electronic health records and claims data. Using multi-level mixed effects models and difference-in-difference analyses, this study will analyze data from 2.2 million overweight patients and patients with obesity between 2009 and 2019, before and after the 2013 implementation of IBT coverage.

Tulane University—Louisiana Experiment Assessing Diabetes outcomes (LEAD) study: Impact of Medicare Reimbursement Policy Supporting Chronic Care Management

Chronic care management (CCM) services target patients with two or more chronic conditions, with the aim of better coordinating their healthcare. In 2015, CMS implemented non-face-to-face CCM services with a specific billing and reimbursement code requiring that the services consume at least 20 min of clinic staff time per month may be supervised by physicians and certain non-physician practitioners and must comply with the CCM scope of services [50]. This study will evaluate the impacts of CCM reimbursement on clinical outcomes, utilization rates, and patient-reported outcomes in diabetes care. The study will also assess the uptake of non-face-to-face CCM services within health systems and examine barriers and facilitators to implementing and maintaining CCM services. We expect that the final study cohort will include over 150,000 individuals within three Louisiana health systems, who have diabetes and at least one other chronic condition. This study will use both regression discontinuity and difference-in-difference models.

University of California, Berkeley—the CMS State Innovation Model (SIM) Initiative as an Accelerator of Delivery System Transformation: The SIM-Diabetes Study

The State Innovation Model (SIM) Initiative has invested nearly one billion federal dollars in state efforts to transform the healthcare payment and delivery system since 2013 [48]. These reforms include the implementation of core components of the patient-centered medical home model as well as payment reforms incentivizing care management, but there have been no systematic assessments to date of the SIM Initiative. This study will assess the impact of the State Innovation Model Initiative on diabetes-related health behaviors and hospitalizations at the county level, as well as the related economic impact. The three-phase rollout of the State Innovation Model Initiative funding allows for examination of 18 purposively sampled states using a stepped wedge quasi-experimental research design that compares early adopter states with “waiting control” states. The results will inform state and local health officials and public health and healthcare delivery stakeholders on the effectiveness of these reforms.

Innovations in Care Coordination

Icahn School of Medicine at Mount Sinai—the Impact of Medicaid Health Homes on Patients with Diabetes

New York State’s Medicaid Health Home program, established in 2012, seeks to enhance care and outcomes for low-income patients with complex chronic conditions such as diabetes, through care management and community-health system collaborations. This study will utilize electronic health records from the New York City Clinical Data Research Network (which includes seven major health systems), Medicaid data, key informant interviews, and focus groups to study the impact of this policy on processes of care, health outcomes, and racial and ethnic health disparities among patients with diabetes who also have other chronic conditions. Using interrupted time series and difference-in-differences analyses and propensity score matching, the study will analyze data from an estimated 80,000 patients with diabetes and compare outcomes from 2007 to 2017 for patients with diabetes enrolled in the Health Home program and a control group of patients with diabetes who qualify for Health Homes but were not enrolled.

University of California, Los Angeles—Evaluation of New Models of Coordinated Care Among Medicaid Enrollees with Diabetes and Prediabetes

University of California, Los Angeles, and UnitedHealthcare (UHC) are partnering to evaluate three new interventions designed to better care for UHC Medicaid members, including those with diabetes or prediabetes. Accountable Care Communities is a UHC-led practice-level initiative that provides selected safety net community practices in 21 states with training, real-time clinical dashboards, and, in some cases, the potential for shared savings to improve care delivery for high-risk members. The Accountable Care Communities study will analyze 2009–2016 data using person-months within an interrupted time series analysis, with approximately 2 million person-months in a combined 46 intervention and 166 control practices. Care Coordination Organizations and Health Homes are designed for the most complex, vulnerable patients within any practice in the UHC Medicaid population, focusing on improving the effectiveness of the care team as well as on delivering individualized, patient-centered support services. Care Coordination Organizations launched in 2014 across the national network of UHC Medicaid coverage and have enrolled over 400,000 patients to date. UHC Health Home programs began in 2012 and are limited to a few states including Washington State, New York, and Kansas. Analyses of the latter two programs will use interrupted time series or regression discontinuity models examining data from before and after the programs were initiated, analyzing the impacts on medication use and adherence, diabetes processes of care, and utilization of healthcare.

Conclusion

The mission of NEXT-D2 is to provide rigorous evaluations of public healthcare-related policies and programs as they affect patients with diabetes and those at risk of type 2 diabetes in order to expand the evidence base and support better healthcare decision-making. The activities of NEXT-D2 come at a critical time. The prevalence of diabetes in the USA is reaching a plateau overall but continues to rise among vulnerable populations, including African-American and Latino patients as well as those with a high school education or less [51]. In addition to ongoing changes in population health, the US health policy landscape is changing rapidly such that the need for evidence-based policy decisions is greater than ever [52]. The collaborative structure of NEXT-D2 will ensure that the included studies of naturally occurring experiments address important policy-relevant research questions, using meticulous and carefully planned analytic approaches as described by Glass and colleagues [12].

The importance of actively communicating findings from these rigorous evaluations of existing policies and programs to health policy leaders and other stakeholders cannot be overstated. The NEXT-D2 Network structure will facilitate deliberate group planning of how best to create and disseminate “actionable” messages from NEXT-D2 studies, how to work closely with patients, practitioners, and policy makers in the process of study design, and how to build credibility as a source of impartial, balanced information. Study results will be published in traditional academic channels including peer-reviewed journals. However, this approach of “passive diffusion” in the academic literature may have limited influence on diabetes-related public policy [53, 54]. Supplementary dissemination approaches involving active, interpersonal communication of clearly expressed, brief research summaries with end-users (e.g., legislators, health delivery system leaders) or their trusted intermediaries are likely to be more impactful [55, 56]. An important goal of the NEXT-D2 Network is to organize a results symposium in this format, for these and other stakeholders.

Our hope is that rigorous evaluation of natural experiments now and in future years will create a compelling evidence base demonstrating the effectiveness or lack of effectiveness of these programs in real-world settings where Americans live, work, and receive care. Such evidence will support the strongest possible decision-making at the health policy level, with prioritization of programs and policies with proven benefits. Some research studies within NEXT-D2 focus on evaluating whether Medicaid expansion or the activities of Medicaid Health Homes have improved health outcomes for vulnerable populations with diabetes. Other NEXT-D2 studies will examine the impact of changes in Medicare reimbursement policy, the CMS-supported State Innovation Model Initiative, and new models of health insurance coverage on health outcomes for patients with diabetes. These studies, evaluated in the context of a broad socio-ecological framework, may help to inform discussions and next steps about the path forward to improve care for Americans with diabetes or at risk of type 2 diabetes.