Does participation in health information exchange improve hospital efficiency?

Abstract

The federal government allocated nearly $30 billion to spur the development of information technology infrastructure capable of supporting the exchange of interoperable clinical data, leading to growth in hospital participation in health information exchange (HIE) networks. HIEs have the potential to improve care coordination across healthcare providers, leading ultimately to increased productivity of health services for hospitals. However, the impact of HIE participation on hospital efficiency remains unclear. This dynamic prompts the question asked by this study: does HIE participation improve hospital efficiency. This study estimates the effect of HIE participation on efficiency using a national sample of 1017 hospitals from 2009 to 2012. Using a two-stage analytic design, efficiency indices were determined using the Malmquist algorithm and then regressed on a set of hospital characteristics. Results suggest that any participation in HIE can improve both technical efficiency change and total factor productivity (TFP). A second model examining total years of HIE participation shows a benefit of one and three years of participation on TFP. These results suggest that hospital investment in HIE participation may be a useful strategy to improve hospital operational performance, and that policy should continue to support increased participation and use of HIE. More research is needed to identify the exact mechanisms through which HIE participation can improve hospital efficiency.

1 Introduction

Hospitals in the United States are under constant pressure to improve the quality and safety of the care they deliver to a growing population with increasing insurance coverage, all while restraining cost growth [1, 2]. However, the care delivery system in the U.S. is highly fragmented, and as a result meeting these demands requires greater alignment and coordination across disparate entities in the healthcare system. A key strategy in creating this coordination has been through the development of health information exchange (HIE) networks. HIEs provide a platform for the transfer of interoperable patient level clinical, demographic, and health related information between distinct entities within the U.S. healthcare setting, and are believed by many policy makers, health services researchers, and healthcare providers to be a critical tool in achieving cost, quality, and efficiency goals [3, 4]. Indeed, a fully interoperable healthcare system is projected to save nearly $80 billion annually on U.S. healthcare spending [5].

The development of HIE networks have largely been supported by three different components of the Health Information Technology for Economic and Clinical Health (HITECH) act passed in 2009 [6, 7]. First, HITECH allocated significant funding for states to develop statewide HIEs through the State Health Information Cooperative Grant Program. Second, HITECH funded the establishment of Beacon community programs that used interoperable health information in innovative ways [8]. Third, HITECH's largest component was the Meaningful Use (MU) incentive programs that aims to increase the adoption and utilization of certified Electronic Health Records (EHRs) capable of exchanging interoperable data [9]. The MU program offers a total of nearly $30 billion in incentive payments allocated through Medicare and Medicaid for providers that attest to specified criteria, and penalizes providers that fail to adopt robust EHR systems. Currently, both the State Cooperative Grant and Beacon programs have ended, and while the MU program remains intact for hospitals, it is being phased out and combined with larger payment reform efforts for eligible professionals. Interoperability appears to remain an important priority for health policy [10], yet its funding and support is at an important moment of transition as stakeholders try to assess the value of this technology.

Nonetheless, these policies have enabled the proliferation of HIEs and over 100 HIEs exist today with nearly a third of hospitals participating in these entities [11]. Notably, these HIEs employ different technological architectures and governmental structures [12–14]. For example, while grants from the HITECH program initially encouraged centralized, state-wide HIEs open to all providers (e.g. Regional Health Information Organizations), the federal government shifted its support to the Direct model for HIEs [12]. The Direct model is email like exchange directly between two providers. Still, other models have also emerged in the HIE landscape, such as vendor-mediated and enterprise models that function as federated HIEs available only to providers using the same EHR vendor (e.g. Epic Care Everywhere) [15]. Furthermore, HIEs cover different geographic areas – some focused on connecting providers across a state, others regionally focused, and still others (e.g. vendor-mediated) less defined by market boundaries [16]. Adding to this complexity, these different approaches to HIE are not mutually exclusive, as a single provider can participate in multiple HIEs, and HIEs themselves can connect to each other. While these different approaches likely have consequences in terms of the impact of HIE for both providers and population health [17], the broader question of whether information exchange is beneficial remains largely open.

Research on hospital HIE participation has predominantly focused on factors associated with adoption of HIE and has identified that HIE participating hospitals are more likely to be non-profit owned, have a large market-share, and be located in markets with low competition [18–22]. Barriers to adoption include lack of sustainable business models and available funding, concerns over privacy and security of data, costs, and competitiveness with other providers [20, 23–27]. Health care providers view patients' health information as a proprietary asset and freely exchanging this information with competitors may reduce any strategic advantage this information allows them, such as patient and physician retention [25, 28, 29].

Several recent studies examine the management factors that contribute to utilization of HIE technology, a key mediator in the pathway to financial and clinical value creation [30–34]. Obtaining user buy-in through workflow analysis and integration into clinical workflow, as well as providing adequate training appear fundamental to gain user acceptance and maximize derivative benefits. Other literature on HIEs has examined both the potential and realized financial and quality impacts of participation [35, 36]. Kaelber and Bates [37] outline how HIE can improve patient safety, but evidence on the actual safety and quality impact of HIE remains limited [38]. A few studies examine hospitals participating within single HIEs and find that HIE can significantly reduce healthcare spending and resource utilization, but these benefits are highly context dependent [35, 39–42]. More recently, research has begun to examine the effect of HIE participation on larger samples of hospitals: Lammers et al. [43] found that HIE participating hospitals in California and Florida had reduced probability of repeat imaging in their emergency departments.

2 New contribution

Despite the limited, yet emerging evidence regarding the impact of HIE on cost and quality, the effect of HIE on efficiency – a related dimension of organizational performance - remains unclear. Quality and cost analyses, while important, do not capture the entire spectrum of organizational performance. This study addresses this gap in the literature by examining the efficiency effects associated with hospital participation in health information exchange using a national sample of hospitals over a critical time period in the growth of HIEs. Efficiency has been a focus of several previous studies on the effect of EHR on hospital performance, yet the evidence remains mixed [44–48]. However, experts believe that interoperable health information exchange can bring about efficiency gains beyond that of EHR alone due to its ability to create greater care coordination across distinct providers [49]. This analysis thus addresses a new dimension of hospital performance that HIE participation may be impacting, providing an indication of the ability for the HITECH interoperability funding programs to produce greater efficiency in the healthcare system. Furthermore, this analysis provides assistance for hospital managers to justify both initial and continued HIE investment decisions. For researchers, the results from this study can provide rationale for further investigation into the mechanisms through which HIE may be driving efficiency change, and for investigation into the interplay between HIE architecture and provider performance.

3 Conceptual framework

In addition to the HITECH act, new reimbursement programs such as bundled payments, accountable care organizations (ACOs), and alternative payment models (APMs), are encouraging greater coordination and alignment between providers across the healthcare system [50, 51]. HIE participation is one strategy hospital are deploying to achieve these goals, as it can provide hospitals with access to accurate and timely information through clinical information sharing. Hospitals can use this information to improve their existing work practices and routines, or to innovate and redesign their workflow and conduct new tasks. These two uses of the newly accessible health information contribute to an organization's ability to coordinate tasks with providers external to the focal organization and make effective clinical decisions [52]. This coordination capability allows hospitals to improve their care coordination for patients, and their service coordination with other providers. For patients, this care coordination capability will result in reduced patient intake time, facilitation of faster and more accurate diagnoses, reduced medication errors, or improved patient safety [3, 37, 53]. Service coordination will lead to reduced redundant imaging [43]. This coordination will result in cost savings, quality improvements, and organizational efficiency gains [1, 54, 55]. This logic model is adapted from Huerta et al. (2016) and is presented in Fig. 1 [54].

Fig. 1. Logic model describing value creation from hospital health information exchange participation.

Though the logic model shows how HIE participation can result in cost and quality benefits, the focus of this present study is on efficiency. As described in the logic model, HIE can enable overall efficiency change, also known as total factor productivity (TFP), through two mechanisms: conducting the same tasks with greater efficiency, or innovating and conducting entirely new tasks that allow for achievement of an organizational goal, such as a greater volume of health service production, to be reached with a lower ratio of outputs to inputs. The former pathway is typically viewed as a result of management choices regarding allocation of resources to incorporate a new technology into existing routines and work practices, and is termed technical efficiency change (TEC). Alternatively, efficiency gains can also come from innovating around a new technology, such as HIE participation. These changes typically occur through workflow redesign, changing job roles, or revising organizational policies and procedures. These innovations are termed technological change (TC). All three types of efficiency can improve over time as the accumulation of organizational knowledge allows for better utilization of existing resources and more innovation centered on the capabilities offered by a new technology [56]. This learning effect is particularly relevant for HIEs, as this technology is considered a network technology, whereby more users allow for greater knowledge spread and greater value accrual for network participants [19].

Given the dynamics around HIE participation and efficiency change, two hypotheses are derived:

H1: Hospital HIE participation will result in improved technical efficiency change, technological change, and total factor productivity.

H2: Hospitals that participate in HIE longer will realize greater efficiency change than those that do not participate at all.

4 Methods

4.1 Data and selection

This study uses three data sources: the 2009–2012 American Hospital Association (AHA) Annual surveys; the 2009–2012 AHA Information Technology (AHA IT) Supplement; and the 2009–2012 Center for Medicare and Medicaid Services (CMS) Case-Mix Index (CMI) Public Data Files. The AHA annual survey reports information about the organizational characteristics of nearly 90% of hospitals in the U.S. The AHA IT supplement has been issued to respondents of the annual survey since 2008 and contains information about hospitals' information technology capabilities. Between 55 and 65% of respondents to the AHA annual survey respond to the annual AHA IT supplement. The AHA IT supplement represents hospital IT capabilities as of the preceding year, i.e. the 2009 AHA IT supplement represents hospital IT characteristics as of 2008. The analytic sample will be a balanced panel of all non-federal, acute-care general hospitals within the U.S. with available data.

4.2 Dependent variables

The dependent variables for this analysis were determined using the Malmquist algorithm, an extension of the linear programming technique data envelopment analysis (DEA) to assess efficiency changes over time. DEA has been used in a diverse range of industries, including healthcare, to assess efficiency [54, 57–63]. Typical regression models compare each hospital to the average relationship between a set of inputs and outputs. Alternatively, DEA assesses the relative relationship between the ratio of inputs to outputs between organizations, effectively comparing organizations directly to their peers. DEA determines this relative relationship by identifying a ‘leading edge’ of organizations with the most efficient use of inputs and outputs. Other organizations are compared to this ‘leading edge’ to determine their (in)efficiency. Organizations can either have the wrong level of inputs (technical inefficiency) or the wrong mixture of inputs (allocative inefficiency). However, estimation of allocative inefficiency requires input cost data. In the context of healthcare, these costs are set both exogenously and locally, and are subject to distortion given market dynamics in healthcare [64]. Despite the absence of cost data, technical inefficiency can be estimated by taking differences in input prices across markets into consideration, since these differences result in different resource allocation strategies [65]. A market segmented analysis thus indirectly controls for input cost differences, and allows for estimation of technical inefficiency while accounting for allocative inefficiency. In this current study, markets were defined as the combination of both division and metropolitan core based statistical areas (CBSAs), and suburban and rural CBSAs, within each of the nine census regions—yielding a total of eighteen different markets. In preliminary work, we examined different market (e.g. each CBSA separately) definitions as a robustness check, and based on the lack of significant differences observed, we selected the combined CBSA approach as a balance between granularity and parsimony.

The Malmquist algorithm is a specification of DEA used to measure changes in efficiency over time [57, 66]. The Malmquist algorithm estimates the three component indices of efficiency change: technical efficiency change (TEC), technological change (TC), and total factor productivity (TFP) change. This study utilizes an input-oriented Malmquist algorithm under the assumption of constant returns to scale (CRS). The CRS specification assumes a linear change in outputs for changes in inputs. DEA specified as a CRS model has been previously used to compare hospital efficiency from EHR and electronic lab order entry implementation, and this specification remains appropriate in this study due to the expected input minimization of hospitals [44, 45]. Furthermore, we chose the input-oriented specification of the model because it stresses management's control of the selection of inputs. All data for the Malmquist algorithm was determined from the 2009–2012 AHA annual surveys, and the 2009–2012 CMS CMI data files. No data from the AHA IT surveys was required for this analysis, and this broader inclusion criteria insures greater representation within each market [67].

Model inputs and outputs were determined from a literature review of studies concerning HIT adoption in hospitals [44, 45, 68, 69]. This literature review suggested using a model with three inputs and five outputs (Table 1). The inputs include assets that management can control: total licensed beds and two measures of staffing. Total licensed beds stands as a proxy for facility size. Staffing is measured using the total number of licensed nursing staff and the total number of other full time employees. Nursing staff is separated due to its importance as an input that hospital administrators manage [68]. Other full time employees include all non-nurse personnel, excluding residents, interns, and trainees.

Table 1. Variables included in the Malmquist algorithm.

Inputs	Outputs
Total licensed beds	Surgical outpatient procedures
Licensed nursing staff	Medicare case-mix index adjusted admissions
Other full-time employees	Average daily census Emergency room visits Outpatient load

The five model outputs provide an assessment of volume: surgical outpatient procedures, Medicare Case-Mix Index (MCMI) adjusted admissions, length of stay, volume of emergency room visits, and outpatient load. Surgical outpatient procedures identifies hospitals with a heavy outpatient surgical load. MCMI adjusted admissions accounts for the complexity of the patient population served by each hospital and including this measure is consistent with previous studies using DEA [68]. Length of stay combined with the MCMI adjusted admissions give a measure of the average daily census, an indicator of the volume of inpatients treated at the hospital. Finally, emergency room visits and outpatient load consume hospital resources and are thus considered to be hospital outputs [44, 68].

The Malmquist equation was estimated using DEAP software [70]. This software estimates the TFP index, as well as the TEC and TC indices. Index values can be less than, equal to, or greater than one depending on whether a hospital decreased, remained the same, or improved efficiency between years relative to other hospitals within the market. Malmquist outcomes can be interpreted as a percentage change (e.g. 1.03 would imply a 3% efficiency gain). The geometric mean of each year-to-year change for these indices will be divided into quintiles to be used as dependent variables in subsequent analyses. The geometric mean is used because it is robust to the compounding effects of efficiency change across years.

4.3 Independent variables

The key independent variables of interest are any HIE participation, and total cumulative duration of HIE participation. These variables were determined from responses to the 2009–2012 AHA IT supplements. Recall that the AHA IT supplement represents hospital IT capabilities in the preceding year of the survey. Thus, the 2009–2012 AHA IT supplements represent hospital IT status in 2008–2011. This approach allows IT status to be lagged by one year from the efficiency changes. This lag is commonly used in studies assessing the impact of IT on hospital performance, and is indicative of the time between adoption of IT and impact of that technology [71, 72]. The AHA IT supplement asks hospitals to “indicate [their] level of participation in a regional health information exchange (HIE) or regional health information organization (RHIO)” [73]. Only hospitals with responses of “actively exchanging data” were considered to be participating. This definition has been widely used to assess hospital HIE participation [18, 74, 75]. A variable assessing cumulative HIE participation was calculated based on the total number of years the hospital has been participating in HIE. Given that HIE functions as a network technology and our prediction of a learning effect, we did not assume a monotonic relationship between HIE participation and efficiency and chose to model cumulative HIE as a categorical rather than a continuous variable.

Control variables were determined from the 2012 AHA annual and IT surveys and include system membership, ownership type, teaching status, EHR status, and Hirschman-Herfindahl Index (HHI). Besides EHR status, these factors are predominantly fixed over time. System membership may affect alliance management capabilities. Ownership type and teaching status have been previously found to affect HIE adoption [18, 24, 74]. These factors may also lead to differences on utilization and management of HIE, leading to an effect on efficiency. EHR status is included to provide a control for underlying differences in IT capabilities between hospitals. EHR status was assessed using a previously developed scale of EHR capabilities categorizing hospitals as having no EHR, a basic EHR, or a comprehensive EHR based on both the quantity of electronic functionalities and the breadth of implementation across hospital units [76]. Finally, HHI serves as an indicator of the competition in a focal hospital's local market. Competition has been previously shown to affect adoption of HIE, and may also contribute to how efficient a hospital operates. HHI was calculated from hospital level beds within a market defined as a CBSA [77].

Preliminary analysis revealed the presence of missing data for two key variables: HIE and EHR status. Missing data for either HIE or EHR status was imputed using a rule-based approach. This rule-based approach provides a conservative estimation of HIE participation or EHR status, yet allows hospitals that logically may possess a technology to maintain that status. This approach to dealing with missing data took advantage of the longitudinal nature of the data.

Only hospitals missing two or fewer years of data had values imputed. For imputation of EHR status, EHR was aggregated to the none, basic, or comprehensive scale and this scale was used for imputation. Imputation of all variables proceeded in the following ordered rules:

1. Observations missing data in three or four years were dropped.

2. Missing values in 2009 were imputed as not participating, or no EHR.

3. Missing values in 2010 were imputed as not participating, or no EHR, unless the hospital was participating in both 2009 and 2011, or had a basic (or comprehensive) EHR in both 2009 and 2011.

   1. In the case where a hospital had a comprehensive EHR in 2009, but a basic EHR in 2011, the missing 2010 was imputed as basic EHR.

4. Missing values in 2011 were imputed as not participating, or no EHR, unless the hospital was participating in HIE in both 2010 and 2012, or had either a basic or comprehensive EHR in both 2010 and 2012.

   1. Hospitals with missing 2011 values, and a comprehensive EHR in 2010, but a basic EHR in 2012, were imputed as basic EHR.

5. Missing values in 2012 were imputed with the same value as 2011.

An indicator for an observation with imputed values was created and included in multivariate analyses to test whether data imputation affected model results.

4.4 Analytic approach

All analysis was conducted using Stata 12. The Malmquist outcomes from each market were combined together, and regressed on the independent variables in a series of ordinal logistic regression equations. This approach has been previously used in an analysis of HIT vendor selection strategy and efficiency [68]. However, selection bias and endogeneity issues needed to be addressed for all analyses. Previous studies using the AHA IT supplement data have suggested that small, but significant differences exist between hospitals that respond to the supplement and those that only respond to the annual survey. Supplement responders are more likely to be large, teaching hospitals, and located in the Northeast or Midwest [74]. That study, and others that used the AHA IT supplement as a panel, have applied inverse probability weighting (IPW) to help correct for this non-response bias [74, 75]. This study applies this same weighting technique to correct for non-response bias.

A second issue requiring adjustment is the possible endogeneity of HIE participation. Hospitals that participate in HIE may have greater changes in efficiency due to unobserved characteristics. This study uses a propensity score adjustment similar to previous studies examining the impact of HIT adoption on hospital performance [78, 79]. Matching using propensity scores can remove up to 90% of bias due to unobserved differences in the adopters and non-adopters, to the extent that unobservables are correlated with observables [80, 81]. The application of this approach estimates a propensity score for HIE adoption at any time based on 2012 hospital characteristics of system membership, ownership type, teaching status, and EHR status, bed size, and census region. Propensity scores were then divided into quintiles to be used as categorical predictors in subsequent analyses [81, 82].

The geometric means of each of the three outcomes derived from the Malmquist equation (TFP, TC, TEC) were divided into quintiles, as these outcomes are not normally distributed and may not have a monotonic relationship with the independent variables [68]. Robustness tests with tertiles, quartiles, and quintiles revealed similar findings, so quintiles were chosen due to their greater level of granularity and comparability to previous studies [68]. To address H1 a separate ordinal logistic regression equations was used for each of the TEC, TC, and TFP outcomes. The key independent variable was any HIE use. Control variables were system membership, ownership, teaching status, EHR status, and HHI. Sampling weights and propensity score quintiles were also included. Testing for H2 followed an identical approach, with the key independent variable of interest being total years of HIE participation.

5 Results

A total of 1017 hospitals met the selection criteria and were included in the analytic sample. The Malmquist algorithm analysis did not require data from the AHA IT supplement. Not matching the AHA annual survey to the AHA IT supplement before matching to the CMS case-mix index files resulted in a much greater representation of hospitals from the entire population. Thus, the Malmquist algorithm can estimate efficiency change relative to a better approximation of all other hospitals in a market. The sample selection for the Malmquist algorithm yielded a total of 3035 hospitals.

Hospital characteristics of the analytic sample are presented in Table 2. The characteristics presented in Table 2 are adjusted using inverse probability weights to account for the selection bias due to survey non-response. Overall, 44.3% of hospitals included in the analytic sample participated in HIE for at least one of the four years of the survey data, but only 4.4% of hospitals participated for all four years. Most hospitals that participated in HIE at some point participated for only one year.

Table 2. Inverse probability weighted sample characteristics.

Variable	Sample (n = 1017) f (%) or Mean (SE)
Any HIE participation	450 (44.3)
Years of HIE:
None	567 (55.7)
1 Year	221 (21.7)
2 Years	128 (12.5)
3 Years	56 (5.5)
≥ 4 Years	45 (4.4)
System membership	582 (57.2)
Teaching	113 (11.1)
Ownership type:
Gov't, Non-Fed	185 (18.2)
Non-Profit	716 (70.4)
Private	116 (11.4)
Bed size:
Small (<99)	270 (26.6)
Medium (100–399)	573 (56.4)
Large (>400)	174 (17.1)
EHR Status:
None	93 (9.1)
Basic	227 (22.3)
Comprehensive	697 (68.5)
HHI	0.34 (0.01)

f frequency, SE Standard Error, HIE Health Information Exchange, EHR Electronic Health Record, Gov't Government, HHI Hirschman-Herfindahl Index

Summary statistics, including means, standard deviations, and ranges, of the Malmquist algorithm results are presented in Table 3. These results are of the more representative sample used for the Malmquist algorithm. Of hospitals included in the analytic sample (n = 1017), 51.2% increased their technical efficiency, 58.1% increased their technological efficiency; and 62.2% increased their total factor productivity.

Table 3. Descriptive statistics for dependent variables derived from the Malmquist algorithm by combined division/metropolitan and suburban/rural within census regions (n = 3035).

Efficiency index	Mean	SD	Min	Max
Technical efficiency change	1.01	0.08	0.73	1.50
Technological change	1.01	0.07	0.65	1.43
Total factor productivity	1.02	0.07	0.64	1.90

SD Standard Deviation, Min Minimum, Max Maximum

H1 posited that hospital HIE participation will result in improvements in technical efficiency change, technological change, and total factor productivity. This hypothesis was tested using three separate ordinal logistic regression equations for each of the three Malmquist indices. The results from these models are presented in Table 4. HIE participants were 1.29 times more likely to be in a higher quintile of TEC (95% CI: 1.0–1.64). Non-profit status was also found to significantly increase TEC relative to non-federal government hospitals, as was teaching status and system membership.

Table 4. Ordinal logit results for the effect of any HIE participation on three measures of efficiency change over time calculated from the Malmquist algorithm.

Variable	Outcome
	TEC OR (95% CI)	TC OR (95% CI)	TFP OR (95% CI)
Any HIE participation	1.29 (1.03–1.64)*	1.06 (0.84–1.33)	1.32 (1.05–1.67)*
Teaching	2.01 (1.27–3.17)*	0.52 (0.31–0.89)*	0.92 (0.62–1.37)
System	1.72 (1.34–2.22)*	0.62 (0.47–0.81)*	1.27 (0.98–1.63)
Ownership:
Gov”t, non-federal	Ref.	Ref.	Ref.
Non-Profit	1.56 (1.11–2.18)*	0.48 (0.34–0.68)*	0.69 (0.50–0.96)*
For-Profit	0.95 (0.58–1.39)	1.36 (0.81–2.28)	1.13 (0.64–1.99)
Bed Size:
Small (<99)	Ref.	Ref.	Ref.
Medium (100–399)	1.05 (0.80–1.68)	1.05 (0.79–1.39)	1.19 (0.85–1.66)
Large (>400)	1.06 (0.67–1.59)	0.80 (0.49–1.31)	0.89 (0.59–1.35)
EHR Status:
None	Ref.	Ref.	Ref.
Basic	1.21 (0.92–1.35)	0.99 (0.74–1.34)	1.36 (1.04–1.77)*
Comprehensive	0.96 (0.68–1.35)	0.90 (0.60–1.35)	0.87 (0.59–1.28)
HHI	1.09 (0.76–1.55)	1.26 (0.90–1.76)	1.54 (1.07–2.22)*
Propensity score quintile:
Quintile 1	Ref.	Ref.	Ref.
Quintile 2	0.63 (0.42–0.96)*	2.08 (1.40–3.10)*	1.11 (0.74–1.66)
Quintile 3	0.39 (0.26–0.60)*	3.58 (2.28–5.60)*	1.20 (0.77–1.88)
Quintile 4	0.46 (0.30–0.69)*	3.41 (2.23–5.23)*	1.55 (1.03–2.32)*
Quintile 5	0.15 (0.09–0.26)*	14.06 (8.11–24.37)*	1.85 (1.14–3.00)&ast;

TEC Technical Efficiency Change, TC Technological Change, TFP Total Factor Productivity, OR Odds Ratio, CI Confidence Interval, Ref. Reference Category, HIE Health Information Exchange, EHR Electronic Health Record, Gov't Government, HHI Hirschman-Herfindahl Index. All models include inverse probability weights

^*p < .05

Results from the ordinal logit model for TC show that any HIE participation was not significantly related to TC. In this model non-profit hospitals had a reduced likelihood of being in higher quintiles of TC compared to non-federal government hospitals, as did both teaching hospitals and system members. The model for the effect of any HIE participation on TFP showed that any HIE participation significantly increased the likelihood of being in a higher quintile of TFP by a factor of 1.32 (95% CI: 1.06–1.67). Hospitals with a basic EHR were also found to have a significantly increased likelihood of being in a higher category of TFP. Hospitals in markets with lower competition (i.e. greater HHI value) had a greater likelihood of being in the highest quintile of TFP.

H2 stated that longer hospital HIE participation will lead to greater changes in TEC, TC, and TFP. Model results are presented in Table 5. The ordinal logit model testing the effect of cumulative years of HIE participation on TEC shows that none of the categories of cumulative years had a significant effect on TEC. Non-profits were found to have an increased probability of being in a higher quintile of TEC, as were teaching and system members. Similar to the results of the H2 model for TEC, no cumulative years of HIE participation were significant in the H2 model for TC. Opposite the results of TEC, non-profit hospitals were found to be less likely to be in a higher quintile of TC, as were teaching hospitals and system members.

Table 5. Ordinal logit results for the effect of cumulative years of HIE participation on three measures of efficiency change over time calculated from the Malmquist algorithm.

Variable	Outcome
	TEC OR (95% CI)	TC OR (95% CI)	TFP OR (95% CI)
Years of HIE:
None	Ref.	Ref.	Ref.
1 Year	1.25 (0.93–1.68)	1.10 (0.83–1.46)	1.33 (1.00–1.77)*
2 Years	1.29 (0.91–1.84)	0.87 (0.62–1.23)	1.17 (0.82–1.67)
3 Years	1.46 (0.97–2.20)	1.53 (0.95–2.45)	1.73 (1.17–2.56)*
≥ 4 Years	1.32 (0.78–2.23)	0.96 (0.55–1.67)	1.26 (0.58–2.71)
Teaching	2.02 (1.28–3.20)*	0.53 (0.31–0.91)*	0.92 (0.61–1.38)
System	1.72 (1.33–2.22)*	0.62 (0.47–0.81)*	1.26 (0.98–1.63)
Ownership: Gov”t, Non-Federal	Ref.	Ref.	Ref.
Non-Profit	1.56 (1.11–2.20)*	0.47 (0.33–0.67)*	0.69 (0.50–0.95)
For-Profit	0.96 (0.58–1.58)	1.34 (0.79–2.25)	1.13 (0.64–1.99)
Bed Size:
Small (<99)	Ref.	Ref.	Ref.
Medium (100–399)	1.05 (0.79–1.39)	1.03 (0.78–1.37)	1.17 (0.84–1.64)
Large (>400)	1.05 (0.66–1.67)	0.79 (0.48–1.31)	0.89 (0.58–1.35)
EHR Status:
None	Ref.	Ref.	Ref.
Basic	1.21 (0.92–1.60)	1.00 (0.74–1.35)	1.35 (1.04–1.77)*
Comprehensive	0.96 (0.68–1.35)	0.89 (0.59–1.33)	0.86 (0.58–1.27)
HHI	1.09 (0.77–1.56)	1.26 (0.89–1.77)	1.55 (1.08–2.23)*
Propensity Score Quintile:
Quintile 1	Ref.	Ref.	Ref.
Quintile 2	0.63 (0.42–0.95)*	2.09 (1.40–3.11)*	1.10 (0.73–1.66)
Quintile 3	0.40 (0.26–0.61)*	3.67 (2.34–5.77)*	1.23 (0.79–1.93)
Quintile 4	0.46 (0.30–0.70)*	3.48 (2.27–5.32)*	1.57 (1.05–2.36)*
Quintile 5	0.15 (0.09–0.26)*	14.24 (8.19–24.74)*	1.88 (1.15–3.05)*

TEC Technical Efficiency Change, TC Technological Change, TFP Total Factor Productivity, OR Odds Ratio, CI Confidence Interval, Ref. Reference Category, HIE Health Information Exchange, EHR Electronic Health Record, Gov't Government, HHI Hirschman-Herfindahl Index. All models include inverse probability weights

^*p < .05

In the H2 model for TFP, both one (OR: 1.33; 95% CI: 1.00–1.77) and three (OR: 1.73; 95% CI: 1.17–2.56) year participants were found to have a significantly greater likelihood of being in the highest quintile of TFP relative to non-participants. Both hospitals with basic EHR systems and those in areas with low competition had an increased chance of being in higher quintiles of TFP.

6 Discussion

The recent policy efforts to use interoperable health information exchange to create greater coordination across distinct entities in the healthcare system has spurred growth in hospital participation in these entities. However, many hospitals continue to question the value to their organization of exchanging clinical data. Few studies, with limited generalizability, document the impact of HIE participation on organizational performance. The goal of this study was to contribute to the growing body of evidence concerning the value proposition of hospital HIE participation by examining the effect of HIE participation on hospital efficiency. In brief, it was found that hospitals participating in HIE at any point in the study experienced a greater likelihood of being in the highest quintile of both TEC change and TFP change. The results also indicate that hospitals are not seeing a uniform learning effect from HIE participation, as evidenced by the finding that hospitals only experienced improvements in TFP for one and three years of HIE participation.

The finding that any HIE participation effects both TEC and TFP suggests that HIE enables hospitals to conduct their routine tasks with fewer labor resources relative to volume outputs. Several pathways either predicted or identified by existing literature could account for these efficiency gains. For example, HIE may alter the hospital environment by reducing information retrieval time, or allowing hospitals to make more accurate and faster diagnoses, resulting in fewer inpatient admissions from the ED [40]. Overall, these reductions in resource requirements may produce TFP gains that benefit the HIE participating organizations. However, the lack of significant findings for the TC suggests that hospitals may be taking a more cautious approach towards innovation around HIE. Previous research suggests that hospitals may need to redesign workflow, or modify their information gathering roles in order to maximize the benefit from HIE [30, 31, 33, 83]. Despite these findings, best practices for integrating HIE into workflow are yet to emerge, and hospitals may require more time than observed in this study to change their care processes. Similar to other health information technology (e.g. EHRs), hospital managers may not yet understand the best approach to implementation for HIE [84], and as a result are reluctant to invest in innovation without more certain return. Alternatively, HIE platforms may need modification and maturation in order to benefit participating hospitals [85].

It was predicted that not only would hospitals improve their efficiency due to any HIE participation, but that longer HIE participation would yield greater efficiency gains. This hypothesis comes from previous research that shows that organizations take time to both routinely use a new technology and learn how best to use that asset [86]. Managerial aims such as culture change or workflow redesign may take several years to implement. However, neither hospital TEC or TC benefited from additional years of HIE usage. The TEC changes discussed above may be too small when broken down into the year categories to observe any effects. Similar to the H1 findings, innovation related activities may require additional time than observed in this study to produce efficiency gains. Despite these insignificant findings, TFP did increase for hospitals participating in HIE for one and three years, but not for two and four or more years. This finding does offer a clear answer regarding the relationship between length of HIE participation and TFP, as it suggests a potential step-wise pattern of benefit over time. However, further research is needed to understand the shape of this relationship over time.

Secondary findings from the analyses are relatively consistent across both H1 and H2 models. Both H1 and H2 models found that teaching status, system membership, and non-profit hospitals exhibited positive changes in TEC, yet a negative likelihood of increasing their TC. TEC and TC may in fact be organizational trade-offs to some extent: focusing resources to optimize routine tasks may take away from managerial efforts to identify beneficial innovations. The dynamism of the hospital environment may deter certain types of organizations (e.g. system members or non-profits) from attempting to redesign workflow. Future studies may consider examining the trade-offs organization's make in their efforts to improve performance.

Another interesting finding from this study was that only basic EHR systems were found to improve TFP, but not comprehensive systems. Appari et al. [72] found a similar result in a study on the effect of EHR adoption on quality: basic EHR systems can improve quality, yet more advanced EHR systems do not yield additional quality benefits. It remains possible that until hospitals are able to innovate and redesign workflow using EHR capabilities, the operational benefits may be limited at more basic technology adoption levels [46]. Alternatively, Adler-Milstein et al., found that EHR can improve process adherence and patient satisfaction, but not efficiency [87], whereas other research on efficiency has found that EHR systems can produce small efficiency gains [45, 46, 88]. Each study, including this present study, used different measures of efficiency, different data sources, and examined overlapping time periods. Given the sensitivity of the effect of EHR on efficiency to these differences, it may be that the effect of EHR is highly context dependent, and is more associated with unmeasured variables concerning the implementation of the technology rather than its presence alone.

6.1 Management implications

Results from this study suggest that hospitals can derive TC and TFP benefits from HIE participation. For managers, this result supports the decision to join an HIE, but to do so with careful planning efforts regarding how to fit the technology into existing workflow. This planning may include assessing both nurse and clinician current technological facility, and designing and implementing the HIE with this level of know-how in mind [89]. Managers could also work with vendors to design HIE systems that match current workflow [30, 83]. Additionally, managers should work to align their HIE efforts with other ongoing efforts to improve care coordination and provide population health, such as through ACOs [90].

The implications of our findings on continued investment in HIE participation for current participants is less clear, as we did not find a clear association between length of participation and efficiency gains. However, given that we did not find a negative association, continued investment at current levels in HIE may be a prudent approach. In the long run, additional investment in integration efforts such as training or workflow redesign may help optimize HIE performance affects, but until best practices regarding HIE emerge, managers may be better served in the near term deploying limited resources to insure fit between the technology and existing work practices.

6.2 Policy implications

The policy environment surrounding interoperability is rapidly shifting, as uncertainty remains around the future of the MU program for hospitals. Many experts believe that the MU program for hospitals should be combined with APMs and quality improvement initiatives as has been done with the MU program for eligible professionals through the Medicare Access and Children's Health Insurance Program Reauthorization (MACRA) act [51]. On its surface, the results reported in this study suggest that continued investment from public sources in HIE development is a worthwhile mechanism to achieve increased efficiency. Moreover, the results also suggest that these investments should be purposefully long-term to account for the time it may take to realize benefits from HIE participation.

Digging deeper into the results reported here suggests that much still remains to be learned about how hospitals are using HIE networks, and what comprises best practices for implementation of HIE services. Moreover, it additionally remains unknown how different HIE architecture may exacerbate or alleviate existing health disparities – an important public policy goal. The federal government, and specifically AHRQ and the Office of the National Coordinator for Health Information Technology, have an important role to play in answering these questions. Those agencies can continue to provide research funding to understand the effects of interoperable health information. They may also advocate for expanding the purview of components of MACRA that encourage providers to send, receive, and integrate health information from outside sources to cover hospitals as well [91]. Those agencies have been successful in working with vendors to create national interoperability standards and best practices [92], but continued partnership with enterprise and vendor-mediated, as well as state HIEs, to facilitate research will help improve the benefits and implementation of HIEs.

6.3 limitations

This study faces several key limitations. First, while the inverse probability weighting helps reduce the selection bias present due to incomplete responses to the AHA IT survey over the four years, some bias likely remains and the generalizability of the results reported in this study should be considered with this bias in mind.

Second, the propensity score adjustment used to address the possible endogeneity of HIE participation may not entirely eliminate this source of bias. This method has been previously applied, however, other approaches such as using an instrumental variable or a fixed effects model are often viewed as the preferred way to eliminate endogeneity. No reasonable instrumental variable could be identified for this study, and the fixed effects model would not account for the compounding effect of efficiency change captured by using the geometric mean of the year-to-year changes. Thus, the propensity score approach was the best available.

A third limitation arises due to technical flaws in the determination of the Malmquist, making inferences derived from this approach potentially invalid due to the complicated determination of the frontier, the assumption of no measurement error, and unknown serial correlation [93, 94]. The use of these estimates in the second-stage regression introduces additional error because the deterministic nature of the Malmquist produces serial correlation in the regression model. More advanced methodologies, such as the Simar-Wilson two-stage bootstrapping approach, use a data generating process that is robust to this bias [93, 94]. However, this methodology is not commonly applied in the field of health services, and it remains unknown the most appropriate specification of this approach in the context of hospitals. Exploration of best practices for applying the Simar-Wilson approach, or other bootstrapping techniques, for the field of health services research is an important area for future research.

Additional limitations arise from the lack of available granular data on HIE participation. This study did not account for HIE usage within a given hospital. Previous research reports very low rates of HIE usage within participating hospitals [31, 34, 39], and usage remains an important mediator to efficiency change. Likewise, no data was available on the type of information being exchanged, making identification of the mechanism(s) responsible for increasing efficiency via HIE participation undetectable. Lastly, the lack of data on the actual start date of HIE participation could have resulted in sample hospitals participating in both technologically and developmentally different HIEs.

Related to the lack of start date information, a major limitation comes from the lack of inclusion of information on the HIE itself. The AHA IT survey asks hospitals specifically about HIE or RHIO participation. Hospitals may interpret this question broadly, but the specificity of this question could bias the participating hospitals to the more community oriented ones. Additionally, as mentioned in the introduction, HIEs exist in several different technological or governmental forms, such as vendor-mediated, direct, or central repository models [12, 13]. HIEs also currently exist in different maturation states—with some being heavily used by a wide variety of provider types, and others being infrequently used by possibly a limited number of different provider organizations. These HIE level factors are critical components in the ability for hospital participants to derive benefit from being a member of a particular HIE [85]. The effect of network level factors on HIE impact remains an unexplored area.

7 Conclusion

The future of interoperable health information networks in the U.S. is at a critical point, as public funding to support these networks is drying up and the value of participation remains unclear for hospitals. Despite these conditions, interoperability is still viewed as a critical component for achieving better coordination in health services. This study adds to the body of evidence regarding the impact of exchange capabilities for hospitals, and finds that HIE participation can improve some aspects of hospital efficiency. The identification of this relationship at the macro level stimulates additional questions regarding the specific mechanisms involved in the effect of HIE on hospital performance, and the best practices for using HIE. Addressing these issues, identifying the new capabilities that HIE enables, and disseminating this information are critical in the effort to achieve maximum value from HIE participation for organizations.