Changes in Health Information Exchange Use Behavior After Introduction of a Fast Healthcare Interoperability Resources (FHIR) Application

Abstract

The aim of our study was to characterize emergency department clinicians’ health information exchange (HIE) use patterns after the implementation of a Fast Healthcare Interoperability Resources (FHIR) application. Using longitudinal electronic health record log data, we categorized HIE use behavior as: no HIE use (0), Web-based viewer use only (1), FHIR application use only (2), or Web-based viewer and FHIR application use (3). We sequenced HIE use behavior from September 2019 to February 2023, then employed hierarchical agglomerative clustering to identify clinician characteristics associated with each HIE use pattern. Our results showed four usage patterns representing (1) clinicians who “lagged” in HIE use and continued as sporadic HIE users (n=66, 46.1%), (2) “late adopters” who had more consistent usage over time (n=32, 22.4%), (3) “legacy users” whose preferred modality was the Web-based viewer (n=25, 17.5%), and (4) “mixed modality users” who displayed frequent changes in HIE access modality (n=20, 14.0%).

Introduction

A prominent focus of national health information technology (HIT) policies and advances in interoperability has been increased use of health information exchange (HIE).^1,2 HIE, the interorganizational sharing of patient-level information,³ has a myriad of applications in the improvement of individual care delivery, population, and public health outcomes, as well as organizational performance.⁴ Numerous rigorous studies have shown HIE to improve safety, reduce duplicative services, and generate cost savings.^5,6 For individual end users, HIE is also often associated with faster access to relevant patient information.⁷

Despite increasing organizational adoption, several studies have documented barriers to individual usage such as incompatible workflows, insufficient time, and fragmented systems that require multiple log-ons and passwords.⁸ Additionally, clinicians have reported finding the volume of information available from HIE systems difficult to navigate⁹ and not always organized in a manner that supports their decision-making processes.^10,11 While it is highly unlikely that accessing the HIE is useful for every patient visit, the available evidence indicates usage is more often the exception than the norm.¹² In US healthcare facilities with HIE available to clinicians, objective measures of usage typically are less than 10% of encounters: 5% of hospitalizations,¹³ 7% of emergency department visits,¹⁴ or 5% or lower for all encounter types.^15,16

To address the barriers to accessing HIE identified in prior studies,¹⁷ our preliminary work developed an application (Health Dart) using the Fast Health Interoperability Resources (FHIR) standard to present selected data elements from the HIE directly in the electronic health record (EHR).¹⁸ Additional details can be found in Rivera & Hosler, et al. (2023).¹⁹ FHIR, the latest information exchange protocol, improves upon existing Health Level 7 (HL7) standards by combining previous versions of HL7 (i.e., version 2 and version 3), and was designed to advance interoperability and integration of diverse data elements.⁴ As a result of FHIR’s popularity, policymakers have been quick to mandate regulations pressuring HIT vendors to support this new protocol.^20,21 Furthermore, FHIR adoption is required to participate in the US Centers for Medicare & Medicaid Services (CMS) payment rules,²⁰ yet several studies have cited implementation challenges resulting in low adoption rates.^4,22

In our study, Health Dart was implemented across 14 Indiana University (IU) Health emergency departments (EDs) in four waves. The first wave began in December 2019 and each subsequent wave began two months after the prior wave, with the last wave beginning August 2020. Waves 2 and 3 were delayed as a result of the COVID-19 pandemic. CareWeb, a Web-based HIE viewer for accessing HIE data in a manual search fashion,²³ was the sole means for accessing HIE data prior to implementation of Health Dart and continued to be available to all clinicians throughout the study. Given the availability of both modalities to access the HIE data (Web-based viewer CareWeb, and EHR-embedded FHIR application Health Dart), this study sought to examine how the modality of accessing the HIE data changed among clinicians. Cluster analysis methods were used to uncover patterns of HIE use behavior before and after the introduction of Health Dart. These findings carry significant implications for policy and practice, highlighting the need to optimize implementation strategies that promote widespread adoption and maximize the impact of FHIR-based solutions.

Methods

Data

First, encounter-level log data from IU Health’s native EHR system, Cerner, were extracted and used as the primary source to identify the modality (CareWeb and/or Health Dart) by which clinicians accessed the HIE. Second, operational staff in each ED provided clinician shift schedules for each month presented in the log data. These two data sources were merged and aggregated at the clinician-month level. Supplementary data from CMS’ National Plan and Provider Enumeration System were used to obtain clinician characteristics such as primary specialty, sex, and credential such as Medical Doctor (MD), Doctor of Osteopathic Medicine (DO), Physician Assistant (PA), and Nurse Practitioner (NP). These data included 254 unique clinicians, or 5,853 clinician-month observations.

Based on expert opinion from co-author PM, we restricted our analyses to clinicians who worked a minimum of 100 shifts from September 2019 to February 2023. The purpose of this decision was to capture clinicians who had an opportunity to regularly access the HIE and allowed us to identify data retrieval modality changes over time. After we implemented this exclusion criteria, our final sample for analysis included 5,720 clinician-month observations representing 143 unique clinicians.

The stepped-wedge, cluster non-randomized controlled trial design where Health Dart was rolled out to IU Health facilities in four waves¹⁹ enabled us to extract two months of EHR log data immediately before Health Dart implementation to represent the “pre” time period, and 30 months immediately after Health Dart implementation to represent the “post” time period. The intent of this was to standardize the pre- and post-implementation data across all waves. Lastly, if a clinician had encounters in the log data from multiple IU Health facilities, we attributed the clinician to the facility where they had the most encounters.

Sequencing & Cluster Analyses

We stratified the data into the pre- and post-implementation periods to identify the HIE data retrieval modalities used in each period. Our variable of interest was a nominal variable representing the different data retrieval modalities a clinician used each month: no HIE use (0), Web-based viewer (CareWeb) use only (1), FHIR-application (Health Dart) use only (2), or both CareWeb and Health Dart use (3). To condense the distinct patterns (sequences) of the HIE data retrieval for each clinician into a limited number of groups, we employed state sequencing analysis. First, we computed the pairwise optimal-matching (OM) distance between pairs of sequences using an insertion/deletion cost of 1.²⁴This enabled us to obtain the calculated dissimilarities between sequences (i.e. distance matrix).²⁴ We then used hierarchical agglomerative clustering (Ward’s criterion) to identify groups of clinicians in the pre- and post-implementation periods. Hierarchical agglomerative clustering is an algorithm that iteratively merges similar sequences based on the calculated dissimilarity between sequences.²⁵ Thus, small dissimilarities between sequences are clustered together. This method partitioned each clinician into mutually exclusive groups having similar HIE data retrieval use behavior. The optimal number of clusters was determined using the hierarchical structure of the dendrogram, review of sequence blots, and an expert group discussion to reach consensus. All analyses were conducted in R Studio (Posit, Boston, MA) version 2022.7.1 Build 554 using the TramineR package.²⁴ This study was reviewed and approved by the Indiana University Institutional Review Board (protocol #1905749709).

Results.

Table 1 depicts the characteristics of our overall sample stratified by the type of HIE user prior to Health Dart implementation. The study sample was primarily comprised of clinicians who were male (n=91, 63.6%), had a specialty of Emergency Medicine (n=126, 88.1%) with a MD or DO credential (n=141, 98.6%). One quarter of the sample worked at one Indianapolis suburban site, IU Health West ED (n=36, 25.2%). Because CareWeb was the only modality available to access the HIE prior to Health Dart implementation, clinicians fell into one of two types: Type 1, defined as no regular use of the HIE prior to Health Dart (n=108, 75.5%); or Type 2, defined as regular users of CareWeb (n=35, 24.5%).

Table 1.

Descriptive Statistics of Study Sample by Pre-Health Dart Implementation User Type.

Characteristic	Study Sample n=143 (%)	Pre- Health Dart Implementation, n (%)
		Type 1 Non-HIE User 108 (75.5)	Type 2CareWeb User 35 (24.5)
Sex
Male	91 (63.6)	69 (48.3)	22 (15.4)
Female	52 (36.4)	39 (27.3)	13 (9.1)
Hospital Site
IUH West	36 (25.2)	23 (16.1)	13 (9.1)
IUH North	24 (16.8)	20 (14.0)	4 (2.8)
IUH Ball Memorial	19 (13.3)	16 (11.2)	3 (2.1)
IUH Bloomington	18 (12.6)	17 (11.9)	1 (0.7)
IUH Arnett	9 (6.3)	4 (2.8)	5 (3.5)
IUH Jay	6 (4.2)	6 (4.2)	0 (0.0)
IUH Saxony	6 (4.2)	3 (2.1)	3 (2.1)
IUH Morgan	5 (3.5)	2 (1.4)	3 (2.1)
IUH Paoli	5 (3.5)	5 (3.5)	0 (0.0)
IUH Bedford	4 (2.8)	4 (2.8)	0 (0.0)
IUH Tipton	4 (2.8)	3 (2.1)	1 (0.7)
IUH Blackford	3 (2.1)	3 (2.1)	0 (0.0)
IUH Frankfort	2 (1.4)	1 (0.7)	1 (0.7)
IUH White Memorial	2 (1.4)	1 (0.7)	1 (0.7)
Credential
MD/DO	141 (98.6)	106 (74.1)	35 (24.5)
NP/PA	2 (1.4)	2 (1.4)	0 (0.0)
Primary Specialty
Emergency Medicine	126 (88.1)	95 (66.4)	31 (21.7)
Other	17 (11.9)	13 (9.1)	4 (2.8)
Shifts worked per month, mean	11.3	11.2	11.3

Our hierarchical clustering algorithm of the post-implementation sample identified four types of clinicians after Health Dart was implemented (Figure 1). Type 1 (n=25, 17.5%) users were characterized by consistent use of the HIE with CareWeb as the preferred modality. We labeled this type, legacy users. Type 2 (n=66, 46.1%) users generally lagged in any access of the HIE and, when they finally used the HIE, tended to display intermittent HIE use with differing modalities. We labeled this group as lagging. Type 3 (n=32, 22.4%) users were also characterized by lagging adoption, but having adopted the HIE with more consistent usage and indications that Health Dart alone or in combination was the preferred modality. We labeled this group late adopters. Last, Type 4 (n=20, 14.0%) users were distinguished by consistent HIE use via either Health Dart and/or CareWeb. Reflective of this frequent mix or changing between modalities, we called this type mixed modality users.

Figure 1.

Clinician HIE Use Behavior Types After Health Dart Implementation.

Table 2 presents counts and frequencies showing how clinicians transitioned from their pre-implementation type to their post-implementation type. Most clinicians (n=63, 58.3%) who did not use the HIE before Health Dart implementation typically transitioned into the lagging group after implementation. Additionally, most clinicians who were in the CareWeb user group before Health Dart implementation, became legacy users (n=20, 57.1%) or mixed modality users (n=12, 34.3%) after implementation.

Table 2.

Clinician HIE Use Behavior Changes Pre- and Post- Health DartImplementation.

HIE User Type,n(%)	Post-Implementation
	Type 1 Legacy 25(17.5)	Type 2 Lagging 66 (46.1)	Type 3 Late Adopter 32 (22.4)	Type 4 Mixed Modality 20 (14.0)
Pre-Implementation
Type 1 Non-HIE User	5 (4.6)	63 (58.3)	32 (29.6)	8 (7.4)
Type 2 CareWeb User	20 (57.1)	3 (8.6)	0 (0.0)	12 (34.3)

In Table 3, we compared the proportion of clinicians categorized in each post-implementation user type for each characteristic. Results demonstrate that most clinicians (n=66, 46.2%) fell into the lagging type after Health Dart was implemented. Lagging clinicians also worked on average, fewer monthly shifts (10.3) as compared to clinicians in the mixed modality user type (12.7). Additionally, a higher proportion of clinicians who worked at the IU Health Bloomington ED fell in the late adopter type as compared to the other 3 types.

Table 3.

Clinician Characteristics Stratified by Post-Health Dart Implementation Type.

Characteristic, n(%)	Type 1 Legacy 25 (17.5)	Type 2 Lagging 66 (46.1)	Type 3 Late Adopter 32 (22.4)	Type 4 Mixed Modality 20 (14.0)
Sex
Male	20 (22.0%)	37 (40.7%)	26 (28.6%)	8 (8.8%)
Female	5 (9.6%)	29 (55.8%)	6 (11.5%)	12 (23.1%)
Emergency Department
IUH West	7 (19.4%)	18 (50.0%)	3 (8.3%)	8 (22.2%)
IUH Arnett	5 (55.6%)	1 (11.1%)	2 (22.2%)	1 (11.1%)
IUH North	3 (12.5%)	12 (50.0%)	6 (25.0%)	3 (12.5%)
IUH Saxony	3 (50.0%)	2 (33.3%)	0 (0.0%)	1 (16.7%)
IUH Morgan	2 (40.0%)	1 (20.0%)	0 (0.0%)	2 (40.0%)
IUH Ball Memorial	1 (5.3%)	12 (63.2%)	3 (15.8%)	3 (15.8%)
IUH Bloomington	1 (5.6%)	4 (22.2%)	11 (61.1%)	2 (11.1%)
IUH Frankfort	1 (50.0%)	1 (50.0%)	0 (0.0%)	0 (0.0%)
IUH Tipton	1 (25.0%)	1 (25.0%)	2 (50.0%)	0 (0.0%)
IUH White Memorial	1 (50.0%)	1 (50.0%)	0 (0.0%)	0 (0.0%)
IUH Bedford	0 (0.0%)	4 (100.0%)	0 (0.0%)	0 (0.0%)
IUH Blackford	0 (0.0%)	2 (66.7%)	1 (33.3%)	0 (0.0%)
IUH Jay	0 (0.0%)	4 (66.7%)	2 (33.3%)	0 (0.0%)
IUH Paoli	0 (0.0%)	3 (60.0%)	2 (40.0%)	0 (0.0%)
Credential
MD/DO	24 (17.0%)	65 (46.1%)	32 (22.7%)	20 (14.2%)
NP/PA	1 (50.0%)	1 (50.0%)	0 (0.0%)	0 (0.0%)
Primary Specialty
Emergency Medicine	22 (17.5%)	57 (45.2%)	29 (23.0%)	18 (14.3%)
Other	3 (17.6%)	9 (52.9%)	3 (17.6%)	2 (11.8%)
Shifts worked per month, mean	11.6	10.3	12.2	12.7

Discussion

Several studies have identified the positive impact HIE has on an organization’s patient outcomes, financials, and healthcare resources.^5,26 With patients regularly seeking care from multiple healthcare facilities,⁵ HIE has the potential to be a valuable mechanism to address information needs in light of such fragmented care. Modalities that use FHIR (Health Dart) or HL7 (CareWeb) provide necessary access to HIE data; however, few studies examine clinician usage patterns of FHIR or HL7 longitudinally.^15,27,28 To fill this gap, our study examined how ED clinicians’ modality in accessing HIE data changed after the implementation of Health Dart, a FHIR-based application. To our knowledge, this is a novel application of sequence analysis to examine ED clinicians’ HIE behaviors longitudinally.

We identified four distinct types of clinicians whose adoption patterns were similar to the classifications of adoption as defined in Rogers’ Diffusion of Innovation Theory.²⁹ Overall, we observed two broad types of clinicians: those who found utility in using the HIE and those who did not. Over half of the clinicians in our study did not see the utility in the HIE until halfway through the 30-month post-implementation period and were described as either lagging in adoption (i.e., Rogers’ “laggard” group), displaying sporadic patterns of usage, or were late adopters (i.e., Rogers’ “late majority”), who preferred using Health Dart to access the HIE data. Late adoption of the HIE may be a result of multiple factors or barriers. For example, both modalities require password changes every three months. In the event that a clinician forgets to change their password, they are unable to access either system and must acquire assistance from their information technology department. A second factor that may have influenced delayed uptake was that during the initial roll-out, Health Dart was placed near the bottom of the Cerner EHR navigation menu, a location less likely to be noticed by clinicians. As identified by our previous study, we found that moving Health Dart to the top of the EHR navigation menu was associated with increased use of the HIE, thus confirming that an application’s placement in the EHR affects usage.¹⁹ Lastly, for various reasons, clinicians may have forgotten Health Dart was available which may have contributed to low adoption. While we hypothesized reasons for delayed use, it would be important to conduct qualitative studies examining users’ experience with Health Dart to understand barriers to use.

The second type of clinician we identified were those who did see the utility in the HIE. Among these clinicians, we identified two distinct types: legacy users and mixed modality users. Legacy users were those clinicians who consistently used the legacy system, CareWeb. Mixed modality users exhibited a similar proclivity to use HIE, but largely varied in the modality used to access the HIE. This finding becomes especially important when considering pre-implementation patterns. On average, clinicians who used the legacy system (CareWeb) during pre-implementation continued to use one of the two modalities (CareWeb and/or Health Dart) to access the HIE information post-implementation. Similarly, clinicians who were not accessing the HIE pre-implementation, were also stable in their behaviors and were more reluctant to adopt the HIE. While our results largely parallel the types of users identified by Rogers,³¹ our study goes one step further and identifies the characteristics of clinicians who were hesitant to adopt Health Dart. The ability to identify these clinician user types enables health systems to develop tailored interventions that help clinicians to overcome barriers they face when accessing the HIE.

From a theoretical perspective, these results also suggest that pre-existing habits may have a role when implementing new HIT, indicating a need to expand theoretical frameworks to more accurately capture HIT adoption. Health systems often implement new technology alongside their legacy systems to ensure adequate development and testing prior to full scale implementation. While our study aligned with this approach, Health Dart was not intended to replace CareWeb and our results demonstrate that habit may be more influential in the presence of multiple HIT options. To better understand how habits will influence the implementation of new HIT with legacy systems, organizations could first conduct small-scale pilots and closely monitor usage behavior prior to full-scale implementations. This would provide the preliminary evidence needed to understand how an organization’s clinical staff will adapt to the new product and serves as an opportunity to get feedback from users who will spend the most time in the system. Conversely, if small-scale implementation is found to be unsuccessful, health systems have the opportunity to de-implement without expending the significant amount of time and money that a large-scale implementation would incur.

In summary, usage behavior is an intermediate step in understanding the true impact of new HIT.³⁰ FHIR, developed to fulfill the need of easier, faster, and improved methods to exchange large amounts of health information, was the impetus of interoperability and integration.²² Despite federal policies^20,21 mandating adoption of FHIR, our findings suggest that increasing HIE usage may be the first step to monitoring and realizing FHIR’s full potential. This could be accomplished in several ways. Given that the majority of clinicians lagged in use of the HIE, there is potential for developing tailored interventions for each type of user for health systems that have already implemented HIE technologies. For organizations that have yet to implement HIE technology, state or federal guidelines that incentivize HIT vendors to offer small-scale pilot implementations prior to full-scale roll-out may lay the foundation for more streamlined and efficient roll-out processes that enable close monitoring. Furthermore, while the intent of Health Dart implementation was to streamline HIE access, our results do not provide strong evidence that clinicians prefer FHIR over the standard modality of accessing the HIE, CareWeb. Additional studies should examine more specific factors, such as the quality of information or how well the application performs, that are influential in determining whether a clinician decides to use the application. This would provide information on how best to design the application so that it is the least burdensome to the user.

Our study offers several strengths which can enhance the applicability of our findings. First, our study allowed us the opportunity to examine FHIR adoption in a well-resourced health system in Indiana. While our study was implemented in a single health system in Indiana, our FHIR-based solution was implemented in 14 different EDs located in rural, urban, and suburban areas. As a result, our study inherently considered the complexity of health systems which included differences in leadership, staffing across sites, availability of HIT support and training, and the patient population, among others.

Our study is not without limitations. First, we implemented a FHIR-based solution within a well-resourced health system and our results may not translate to under-resourced health systems that have yet to implement, or are struggling to implement, FHIR-based solutions. Considering the setting of our study, under-resourced health systems likely need even more support so that federal policies do not create further division in addressing the health of vulnerable populations. Our findings also suggest that federal policies be examined and refined so as not to detract from the progress that FHIR-based solutions have made. Additionally, because Health Dart was implemented in waves, there was increased awareness and education around the use of the HIE data. Thus, we are unable to distinguish whether the FHIR standard or Health Dart’s user-centered design contributed most to changes in clinician behavior.

Conclusion

Our findings provide evidence that most clinicians are not using the HIE to its fullest extent. Furthermore, our results do not provide strong evidence that clinicians prefer FHIR over existing HIE modalities already available. While our findings establish a solid foundation for future FHIR-based technology implementation research, future efforts should aim to evaluate organizational readiness to implement FHIR-based solutions. Our findings also support the recommendation to develop robust implementation plans which focus on conducting small-scale pilots prior to full-scale implementation of HIT. This would enable close monitoring of implementation and allow healthcare organizations to adapt and develop local processes that ensure high utilization of new HIT.

Author Contributions:

HMK: Formal analysis, Writing – original draft, Validation, Visualization, Conceptualization, Software, Methodology, Investigation, Data Curation, Writing – review & editing, Project Administration RLR: Investigation, Supervision, Funding Acquisition, Resources, Writing – review & editing SP: Validation, Writing – review & editing AH: Data Curation SR: Conceptualization, Methodology, Writing – review & editing PM: Supervision JTS: Conceptualization, Methodology, Software Design, Software Application & Utilization DK: Supervision, Writing – review & editing JRV: Data Curation, Methodology, Conceptualization, Supervision, Writing – original draft, Writing – review & editing TKS: Conceptualization, Investigation, Supervision, Funding Acquisition, Resources, Writing – review & editing