Integrating Social Determinants of Health into Primary Care Clinical and Informational Workflow during Care Transitions

Abstract

Context:

Care continuity during transitions between the hospital and home requires reliable communication between providers and settings and an understanding of social determinants that influence recovery.

Case Description:

The coordinating transitions intervention uses real time alerts, delivered directly to the primary care practice for complex chronically ill patients discharged from an acute care setting, to facilitate nurse care coordinator led telephone outreach. The intervention incorporates claims-based risk stratification to prioritize patients for follow-up and an assessment of social determinants of health using the Patient-centered Assessment Method (PCAM). Results from transitional care are stored and transmitted to qualified healthcare providers across the continuum.

Findings:

Reliance on tools that incorporated interoperability standards facilitated exchange of health information between the hospital and primary care. The PCAM was incorporated into both the clinical and informational workflow through the collaboration of clinical, industry, and academic partners. Health outcomes improved at the study practice over their baseline and in comparison with control practices and the regional Medicaid population.

Major Themes:

Current research supports the potential impact of systems approaches to care coordination in improving utilization value after discharge. The project demonstrated that flexibility in developing the informational and clinical workflow was critical in developing a solution that improved continuity during transitions. There is additional work needed in developing managerial continuity across settings such as shared comprehensive care plans.

Conclusions:

New clinical and informational workflows which incorporate social determinant of health data into standard practice transformed clinical practice and improved outcomes for patients.

Introduction

Communication across health care settings and sectors is critical when coordinating care for individuals with chronic disease, particularly during care transitions, to prevent unnecessary readmissions to the hospital [1]. The circumstances influencing an individual’s ability to live a healthy life, also known as the social determinants of health (SDH), may enhance their risk for hospitalization and include factors such as social support networks, the home and neighborhood environment, health literacy and well-being, and economic forces [2,3]. Therefore, systematically assessing for understanding, and communicating SDH risk factors throughout the care continuum is an essential component of comprehensive health care, particularly for low-income adult populations with complex chronic disease, limited resources, and behavioral health or substance abuse problems.

Strong evidence supporting the efficacy of transitional care in reducing readmission has been limited since Naylor’s and Coleman’s early clinical trials that focused on elderly patients with heart failure [4,5]. Rennke and colleagues [6,7] systematically reviewed 47 clinical trials to evaluate the impact of hospital-initiated transitional care on adverse events such as readmission and emergency department (ED) visit rates. The authors cited failure to report context, implementation, and cost as weaknesses of the research. An important contribution of their work was the development of a taxonomy of transitional safety intervention types into predischarge, postdischarge, and bridging between care settings. Bridging interventions, which included some elements of both pre- and postdischarge planning, were effective in reducing ED visits.

Burke and colleagues [8,9] expanded the concept of bridging interventions and proposed an idealized transitional care process that included timely availability of information, care coordination, communication with the care team, and inclusion of the social context. This model informed our work in our Coordinating Transitions (CT) project, which employed risk stratification, real-time care alerts, care coordination outreach, and the systematic assessment of SDH, resulting in higher-value postdischarge utilization with fewer inpatient (IP) and ED visits than expected.

The widespread use and availability of health information technology (HIT), including the electronic health record (EHR) and Regional Health Information Organizations (RHIO), have presented new opportunities to exchange critical SDH information at the systems, provider, and patient levels. However, health information exchange (HIE) tools, intended to facilitate real-time communication and improve care coordination during transitions, have provided only limited benefit because of interoperability and data management issues [1,10]. The Coordinating Transitions project aimed to overcome these barriers by using existing HIE to bring the critical information about high risk discharges to the care coordinator in real time. Low-cost telephone outreach resulted in immediate identification of problems and early outpatient (OP) follow-up. In this pilot study, HIE enabled a redesigned approach to transitional care.

Our objective was to introduce an assessment of SDH into an intervention intended to improve care coordination for chronically ill individuals during care transitions. The long-term goal of this project is to integrate this information into comprehensive shared care plans (CSCP) that reflect social, behavioral, and physical health needs as well as identifying readmission risk that can be shared across settings and providers [11].

Innovation

The CT project incorporated innovations for integrating SDH into the practice’s clinical workflow and EHR by utilizing existing interoperable HIE systems and existing electronic data to identify the population most at risk for readmission. These innovations resulted in actionable and timely care alerts that were delivered to the primary care practice to be used to prioritize outreach efforts. Care Transition Alerts (CTA) for high-risk individuals trigger an outreach phone call that includes an SDH assessment. A web-based version of the Patient-Centered Assessment Method (PCAM), a brief social needs assessment tool, was developed for scoring and standardizing methods to identify social problems that may place the individual at risk for hospitalization [12,13]. These problems are included in clinical documents transmitted via existing, and interoperable, HIE through the RHIO. Outcomes were analyzed using de-identified claims from the Medicaid Data Warehouse (MDW) to compare effectiveness in study and comparison practices. The CT project leveraged existing technologies, staff, and electronic data to develop a low-cost and sustainable intervention that resulted in fewer admissions and ED visits.

Case Description

The CT project employed a unique collaboration between the University at Buffalo School of Nursing (SON), the Department of Industrial and Systems Engineering (ISE), the Department of Family Medicine, the RHIO “HEALTHeLINK,” and Elmwood Health Center, a Patient-Centered Medical Home (PCMH). The organizing principle was that improving care continuity would require significant changes in the clinical workflow supported by HIE and clinical decision support (CDS). The clinical workflow underwent a redesign that included proactive outreach to people who were recently discharged from the hospital. Using the standardized PCAM assessment tool, it was possible to incorporate SDH information into the care plan to better understand the challenges individuals faced during care transitions. We collaborated with researchers from the Department of Family Medicine and Community Health at the University of Minnesota in implementing the PCAM into the clinical and informational workflow. The research team developed informatics solutions using an iterative process informed by both technical and observational data collected in the clinical setting. The entire team met quarterly to evaluate progress and to refine the process.

Setting

The CT intervention was implemented in a single PCMH in a large metropolitan area in upstate New York with a roster of about 6,000 individuals. The primary care practice provided services for a large Medicaid population with behavioral health comorbidity. Furthermore, the PCMH enrolled nearly all of their patients in the RHIO and automatically received clinical data from across the care continuum. Two comparison practices were selected for inclusion in the outcomes analysis based on similar baseline care practices and proportion of Medicaid recipients. The PCMH had a Registered Nurse (RN) in the role of care coordinator prior to implementation who followed patients with diabetes to insure adherence to clinical guidelines. The role was expanded to include telephone outreach to recently discharged persons with chronic disease, and the PCMH received salary support for the expanded duties.

Population

The intervention was limited to adult patients with pre-existing chronic disease who were discharged from the hospital to the community setting. The outcomes analysis was limited to Medicaid recipients over the age of 18 who were not dually eligible for Medicare with a pre existing chronic condition. Individuals were assigned to study or comparison practices based on the provider identifier assigned during outpatient office visits over the past three years. The three-year time frame was used to exclude patients who had not seen a primary care provider and individuals were assigned to the most recently seen provider

Timeline

Originally designed as a two-year pilot study, the project has been extended to further integrate SDH assessment into HIE across settings. Figure 1 illustrates the project timeline divided into periods: Design (6 months), Implementation (12 months), Evaluation (6 months), Enhancement (6 months), and Future. It further divides the 12-month Implementation period into three 4-month periods.

Figure 1.

Coordinating Transitions Timeline for Implementation

Redesigned Clinical Workflow

The clinical workflow was designed to minimize changes in staff roles and overall care processes. Initially we held a two-day workshop with the creators of the PCAM tool to discuss clinical and technical implementation. Based on that discussion, the plan was to expand the RN care coordinator role to include outreach phone calls to individuals who generated a CTA after discharge from the hospital. After the call, the nurse would record the PCAM assessment findings and include problems identified in the Transitions of Care note.

The research assistant (RA) from ISE made detailed observations of the clinical and documentation workflow throughout the design and implementation phases. Changes in PCMH clinical workflow were tracked by a SON RA who made weekly visits to the practice to review transitional care cases and discuss problems encountered. The nursing RA reviewed the EHR of each person who triggered an alert, and tracked his or her utilization for 90 days after discharge. In addition, both assistants kept field notes of their observations for analysis using NVivo 11 Plus software.

Patient-Centered Assessment Method (PCAM)

The PCAM is a 12-item, four-point Likert scale assessment tool that includes four domains: physical and mental health, social support, health literacy and engagement with services [13,14]. The tool is designed to provide action-based assessment that can be applied by professional staff in community settings. After completing a postdischarge telephone call with the patient or caregiver, the nurse selects the response that most closely matches his or her perception for each question. The PCAM is administered by professional staff without extensive psychiatric background, and has been used in settings around the world with good ability to identify serious social issues affecting health [13].

Data Sources

The research relied on two completely separate sources of data: a clinical HIE database, and a de identified claims database from the MDW that was used to analyze health outcomes. The clinical HIE followed established practices for data sharing through the RHIO, and was visible only to clinicians. An RA from the SON was allowed to retrospectively review the EHR of patients who triggered a care alert, to determine fidelity of the intervention. Cases were given a unique case number, and no identifying information was collected. The University at Buffalo Institutional Review Board approved this protocol.

The PCMH provided a monthly report with the medical record number and the list of diagnoses from billing records for the prior three-year period. The clinical algorithm described below is used to identify chronic conditions and divide the population into segments based on chronic disease complexity. The resulting risk stratification table stored within the clinical data repository (CDR) is used to identify which discharged individuals have pre-existing chronic disease, and to list the pre-existing chronic conditions and the relative risk of hospitalization for the hierarchical disease category in the CTA.

Two data sets were extracted from the MDW for the outcome analysis. The first was a demographic table that included the research case identifier, age, sex, insurer, and county. The second table included the research case identifier; a subset of flagged OP, ED, and IP claims; disease codes; procedure codes; provider identifier; and date the service was delivered. A risk stratification table was created that summarizes total OP, ED and IP visits annually; creates flags for chronic diseases; and a hierarchy of comorbidity. Additionally, utilization after each IP visit was tracked for 90 days after the discharge, including the type of utilization and the days since discharge. This table was used in the readmission analysis.

Risk Stratification

The study used risk stratification based on the International Classification of Disease (ICD) codes contained in the claims database (either the MDW or the PCMH report). Clinical Classification Software (CCS) developed by the Healthcare Cost and Utilization Project [15] provides a way to classify diagnoses into a limited number of categories, and has recently been updated to include ICD 10 codes. A clinical algorithm developed at the SON creates a hierarchy of 31 chronic diseases using data definitions in the CCS. The hierarchy divides the population into segments, defined as individuals without major chronic disease, those with chronic disease, and those with system failure. The algorithm also counts the number of OP, ED and IP events in a single year.

Outcome Analysis

The study was conducted under real-world conditions, which is typical in comparative effectiveness research. Within the region, there are multiple concurrent interventions designed to reduce readmissions that make it difficult to determine if improved outcomes are related to the intervention. In order to improve the internal validity the study employed a pretest-posttest design comparing the rates of OP, ED and IP utilization during the years before and after implementation. In addition to identifying the study and comparison practices, comparison was made with the regional Medicaid population. The assumption was that the intervention’s success would be measured by the actual rate and magnitude of change in rates of utilization. High value postdischarge utilization consists of low IP rates, low ED rates, and high OP rates [16,17].

Findings

The CT project developed risk stratification, realtime alerts, web-based assessment of SDH, and HIE to support telephone outreach by an RN care coordinator in a primary care practice to improve transitional care continuity. SDH assessments were incorporated into transitional care and care planning activities in a primary care setting by integrating the PCAM into clinical and informational workflows. Observational comparative effectiveness research evaluated the impact of the SDH enhanced transitional care intervention on IP, ED, and OP utilization in chronically ill Medicaid adults.

Health Information Exchange

The intervention relied on existing HIE that incorporated standards for interoperability outlined by the Office of the National Coordinator for Health Information Technology (ONC) [18]. Figure 2 and 3 demonstrates the information flow. When a patient moves within the health care system, an automated electronic admission, discharge, or transfer (ADT) notification is sent to the RHIO. These notifications are stored in the CDR and are pushed to the PCMH if the patient has provided consent to share their data. Within the CDR, the ADT notice for discharges from the hospital to the community is compared with a risk stratification table to identify discharged patients who have pre-existing chronic conditions. The risk stratification table is based on disease codes (ICD 9 or 10) from the PCMH billing records, which are classified into 31 chronic disease categories using the CCS developed by the Healthcare Cost and Utilization Project [15]. The algorithm to create the table was developed at the University at Buffalo SON, and has been used to analyze the impact of transitional care interventions in the region [19,20]. The CDR creates a CTA that includes information about the reason for the admission, the pre-existing chronic conditions, and the relative risk of hospitalization based on disease complexity.

Figure 2.

Information Flow Pathways Within the Coordinating Transitions Project

Figure 3.

Timeline for Incorporating Social Determinants of Health Using the PCAM

Notes: PCAM = Patient-Centered Assessment Method, SW = Social Worker, ePCAM = web-based version of paper PCAM, CCD = Continuity of Care Document, LOINC = Logical Observation Identifiers Names and Codes, CSCP = Comprehensive Shared Care Plan

A secure email is used to send the PCMH care coordinator the alert, which triggers an outreach phone call and completion of the PCAM. Discrete results of the PCAM assessment of SDH are stored in the practice’s EHR (Allscripts) as a laboratory value, and problems identified are included in the transitions of care note. This note is transmitted to the RHIO and stored by the CDR as a continuity of care document (CCD) where it is available to be retrieved by providers caring for the individual across the region. Thus information about the care transition links the hospital to the primary care setting, and new, actionable knowledge about SDH is available to providers across the health care continuum.

Incorporating Social Determinants of Health

Our initial efforts to incorporate SDH into the clinical workflow focused on having the nurse care coordinator complete the paper-based PCAM tool. The care coordinator entered the numeric value of each response into a laboratory report. This approach allowed storing of discrete values; however, limitations in the number of characters allowed in the descriptors made it difficult to interpret the results. Furthermore, there was no way to calculate a summary score, and putting the discrete numeric response into the report did not support the identification of problems for incorporation into the EHR.

Early implementation focused on refining the technology for supporting the process, including the need to filter discharge notifications to identify only those alerts generated from IP stays. During the middle implementation phase, the practice saw value in the intervention and started to develop tracking mechanisms to ensure that individuals received needed follow-up in a timely manner. This breakthrough occurred in response to a case where the CT intervention prevented readmission and a crisis in the home of a behavioral health client. In the late phase, the practice began to see both clinical and financial benefits from the intervention and began to expand the ADT notifications to include follow-up on ED visits. The practice also decided to colocate a behavioral health specialist with the care coordinator to assist with development of interventions for those with mental health and substance abuse conditions.

Comparative Effectiveness Analysis

The CT project sought to reduce hospitalizations and ED visits in the adult Medicaid population with chronic disease. The intervention targeted recent IP discharges and was expanded by the practice to include ED visits. Table 1 presents the utilization rates per 1,000 cases for the study PCMH, two control practices (also PCMHs) and the regional Medicaid population at baseline (2014) and implementation (2015). The difference between the expected utilization rate (the rate at baseline) and the actual rate in 2015 was used to calculate improvement and avoided events. Avoided events were calculated by multiplying the difference in rates by the population size in 2015.

Table 1.

Comparison of IP, ED, and OP Utilization Rates in Adult Medicaid Rosters of Study and Control Practices (2014, 2015)

GROUP	POPULATION SIZE (2015)	EVENT TYPE	2014 RATE PER 1,000	2015 RATE PER 1,000	DIFFERENCE IN RATE (2015–2014)	AVOIDED EVENTS
Study PCMH	419	IP	338	255	–83	–35
		ED	2,038	1,327	–711	–298
		OP	6,996	8,907	1,911	801
Control A	963	IP	306	282	–23	–22
		ED	1,608	1,178	–430	–414
		OP	7,741	9,033	1,292	1,244
Control B	2,085	IP	279	287	9	18
		ED	1,711	1,551	–160	–333
		OP	8,328	9,285	957	1,995
Regional Medicaid	38,612	IP	358	297	–61	–2,341
		ED	1,754	1,434	–320	–12,354
		OP	7,925	8,253	–328	12,674

Notes: IP = Inpatient, ED = Emergency Department, OP = Outpatient utilization, PCMH = Patient-Centered Medical Home.

The study PCMH has the lowest OP rate and highest ED rate at baseline. In addition, the IP rate is much higher than either control practice, although it is slightly lower than the regional Medicaid population. The pattern would be classified as low value [17]. In 2015, the study PCMH has the greatest reduction in IP and ED visits and also the greatest increase in OP utilization. This pattern, but not the extent of change, is mirrored in control practice A and the regional Medicaid populations, but in control practice B during the implementation year there was an increase in admissions.

Statistical analysis using the two-sample test for equality of proportions with continuity correction demonstrates that there is no significant difference in IP utilization between the study and control populations in either 2014 or 2015 (p value set at .05). However, the difference in IP rates between 2014 and 2015 in the study population is statistically significant (x² = 5.07, 95% CI = 0.009 – 0.157, p = 0.02). The change is not significant in either control population, but in the other Medicaid population the difference is statistically significant ((x² = 301.46, 95% CI = 0.054 – 0.068, p < .001). The reduction in the IP rate is greatest in the study population where there were 83 fewer IP stays per 1,000 individuals than expected based on 2014 rates (see Appendix A for details of statistical analysis).

Limitations

There are a number of biases and threats to internal and external validity in this project. The study practice was selected because it was sophisticated in the development, integration, and use of HIE systems in clinical practice, with an established nurse care coordinator role and data sharing support through the RHIO. Ability to generalize results to other regional practices or outside of the western New York region, where there is not extensive RHIO support and regional HIE integration, may be limited. In this observational study we have attempted to use both baseline and implementation year utilization and multiple control practices to minimize threats to internal validity. However, there are multiple concurrent incentives to reduce readmissions in the region, and the effects of these interventions cannot be isolated from the effects of the study intervention. Furthermore, there was a large increase in the Medicaid population at the study practice in 2015, and this may have affected the IP and ED rates.

Major Themes

Our study strove to improve care continuity by including social factors in integrated data streams shared across care settings. This work aligns with current research that highlights the need for care coordination supported by effective information sharing and system integration. System approaches such as accountable care organizations (ACOs) are working toward these goals, but have not yet successfully developed clinical workflows capable of sustaining this change over time. For example, recent systematic reviews [21,22] point toward postdischarge care as having a greater impact on readmissions than does care quality in the hospital. ACOs that integrate data across settings have had a mixed impact on hospitalization rates and reduced health care costs in Medicare populations [23,24,25,26]. However, Cantor and colleagues [27], in an analysis of Medicaid populations in New Jersey, felt there is the potential for substantial savings. Hewner and colleagues [19,20] demonstrated the potential of accountable care, managed care, and care coordination in reducing readmissions in the Medicaid population. An extensive review of articles integrating social factors into care planning demonstrates that very little work has been done on this in the United States, and international efforts have been limited to seeking interprofessional consensus on future directions for integrating SDH into CSCP [28].

Lessons Learned

The CTA was a critical CDS tool to teach the care coordinator how to prioritize outreach to patients. Monitoring fidelity of the intervention supported this initial learning through case reviews and followup. Initially the care coordinator felt uncomfortable dealing with behavioral health issues and needed reinforcement to understand the importance of including them as part of a comprehensive screening for SDH. The care coordinator also needed assistance early on to develop a plan of care that addressed the problems that were identified using the PCAM. Once the nurse started to see success in her role of managing complex cases with both behavioral and physical chronic conditions, the care alerts became less important and the practice transitioned to filtering the ADT notifications directly, using the CTA as a reminder It is possible that the principle benefit in the alerts is in training the staff in new population-based models of care.

Our original plan was to create a care transitions dashboard within the EHR. However we found that modifying our approach to use existing technology and incorporate ONC standard tools for interoperability significantly enhanced our solution and will facilitate dissemination of the intervention both regionally and nationally in areas with RHIO support. Further, each step in the development and implementation process required exceptional commitment on the part of both the clinical site and the RHIO to make the intervention successful. For example, it wasn’t until we were finally able to transmit a CCD that we identified the need to add Logical Observation Identifiers Names and Codes (LOINC) to the information on the PCAM so that it would be communicated accurately. The collaborative team was invaluable in developing creative solutions to facilitate dissemination of the intervention.

Incorporation of SDH was a significant challenge for the clinical practice, and it took months before the staff saw the value of systematically collecting this information. However once tracking mechanisms were in place, and after experiencing a few success stories, the staff embraced the concept and took initiative to modify the interventions to improve outreach to ED discharges. Technical problems related to recording the PCAM results and extracting meaningful information about the individual’s level of risk limited the SDH data that were available in the EHR. The team continues to work to improve the process and make it seamless.

Evaluation of health care outcomes was limited to the Medicaid population. Access to de identified claims data from the MDW was especially useful in unobtrusively identifying comparison practices and in examination of secular trends in the region. Although incorporation of SDH is especially relevant in the Medicaid population with limited economic resources, SDH certainly play a role in low-value postdischarge utilization in other populations as well.

Although quantifying the savings to the health care system through avoided events is useful in demonstrating effectiveness, it falls short of meeting the Triple Aim [29]. We hope to incorporate the patient experience of care coordination in future studies and to compare the impact of SDH in younger and older populations to develop customized interventions for all individuals. Development of a CSCP is critical for managerial continuity during care transitions and patient-centered care. Future initiatives should focus on key data elements that should be incorporated into this plan using questions required in the Continuity Assessment Record and Evaluation (CARE) item set that standardizes data collection across acute- and postacute settings as part of the IMPACT Act of 2014 [30,31].

Conclusion

The intervention demonstrates the potential of using a systems approach, bridging acute and primary care, to improve clinical practice during care transitions. Most notably it demonstrates that HIE allows primary care to be actively and proactively engaged at the time of discharge. CTAs filter out unactionable information and provide background on the history of chronic disease, behavioral health problems, and risk of hospitalization so that outreach can be prioritized. Further, systematically incorporating SDH into the intervention adds timely, useful knowledge at the point of care, while storage in the EHR facilitates analysis of the impact of SDH. Finally, using de-identified claims data for the entire population allows evaluation of the impact of the intervention. The intervention advances knowledge and practice by expanding the interprofessional team that is working to avoid rehospitalization to primary care, by coordinating both care and information across settings, and by avoiding unnecessary hospital utilization; thereby reducing overall health care costs, improving care quality, and patient experience of care.

The CT project demonstrated the impact of incorporating SDH assessment into the primary care workflow, especially when supported by timely HIE. However, our efforts to assimilate the knowledge gained into an interprofessional care plan that could be shared across settings met with many challenges. Unfortunately, there is very little scientific evidence available at this time to guide us in our efforts. Our comprehensive review of the literature demonstrated that foundational work remains to be done, such as the development of a mutually agreed upon definition for the concept of “shared care planning.” [28,29,30,31,32] Nevertheless, our CT project and planned future work aligns well with the national “vision for making the CSCP a reality.” [11] Internationally, consensus building workgroups have identified similar priorities such as identifying the need to promote EHR interoperability for linking health and social care needs into a CSCP that is holistic and places the person’s “story” at the center of care [28,29,30,31,32,33].

To this end, we have begun the process of creating LOINC codes for indexing both the responses to the PCAM questions and for the summary scores to facilitate the automatic exchange of key SDH information between providers and settings. We plan to use SNOMED-CT codes to standardize the nomenclature for identified problems, facilitating the integration of this information into interoperable problem lists used by the entire interprofessional team. Once these steps are completed, we can consider the challenges of consolidating SDH into CSCP that provides managerial continuity for individuals as they move across settings. These innovations could be incorporated into transitional care planning at multiple points in the process and may point to ways to better engage individuals in their own care planning. Future directions focus on disseminating our intervention to other care settings both regionally and nationally.

Appendix A

1. Within Group Comparison: Study Versus Control for 2014 and 2015

Two-sample test for equality of proportions with continuity correction, between study group with each of the three control groups, for 2014 and 2015 shows no significant differences.

Table A1.1.

2014
	CONTROL A		CONTROL B		MEDICAID
	chi-square	p-value	chi-square	p-value	chi-square	p-value
Study PCMH	0.07	.79	3.70	.054	0.35	.55

Table A1.2.

2015
	CONTROL A		CONTROL B		MEDICAID
	chi-square	p-value	chi-square	p-value	chi-square	p-value
Study PCMH	0.94	.33	1.60	.21	3.26	.07

2. Comparison of Practice Site 2014 to 2015

Table A2.1.

	14 VERSUS 15
	CHI-SQUARE	P VALUE
Study	5.07	.02 *
Control A	3.66	.06
Control B	0.29	.59
Medicaid	301.46	<.001 ***

Additional information is included in Table A2.2.

Table A2.2.

	SAMPLE ESTIMATES OF PROPORTION: 14	SAMPLE ESTIMATES OR PROPORTION: 15	95 PERCENT CONFIDENCE INTERVAL: (14–15)	P VALUE
Study	0.338	0.255	[0.009, 0.157]	.02 *
Control A	0.327	0.282	[–0.001, 0.090]	.06
Control B	0.279	0.287	[–0.038, 0.021]	.59
Medicaid	0.358	0.291	[0.054, 0.068]	<.001 ***

3. Mann-Whitney U Test

We utilized a Mann-Whitney U test to test the null hypothesis that there is no statistically significant difference in ED and OP utilization between 2014 and 2015 in the study population. The results show that there were statistically significant differences between 2014 and 2015 in the study population.

Two-Sample Wilcoxon Rank Sum Test (Mann-Whitney test)

Table A3.

	2014 VERSUS 2015 ED
	ED	OP
W statistic	64172	48661
p-value	<.001	.0025