Leveraging health information exchange for clinical research: Extreme underreporting of hospital service utilization among patients with substance use disorders.

Abstract

Background:

Substance use disorders are associated with inefficient and fragmented use of healthcare services. The Chesapeake Regional Information System for Our Patients, Inc. (CRISP) is a Health Information Exchange (HIE) linking disparate systems of care in the mid-Atlantic region.

Methods:

This article describes applications of HIE for tracking hospital service utilization in substance use disorder clinical and services research, drawing upon data from one of the first studies approved to access the CRISP HIE. Participants were 200 medical/surgical inpatients with comorbid opioid, cocaine, and/or alcohol use disorder (45.5% female; 56.5% black; 77.5% opioid use disorder; 42.0% homeless). This study compared HIE-identified hospital service utilization with conventional methods of participant self-report during in-person research follow-ups (3-, 6-, and 12-months post-discharge) and electronic health record (EHR) review from the hospital system of the index admission.

Results:

This sample exhibited high levels of hospital utilization, which would have been underestimated using conventional methods. Relying exclusively on self-report in the 12-month observation period would have identified only 33.8% of 429 inpatient hospitalizations and 9.0% of 1,287 ED visits, due to both loss-to-follow-up and failure to report events. Even combining self-report with single-system EHR review identified only 66.2% of inpatient hospitalizations and 59.8% of ED visits.

Conclusions:

CRISP HIE data were superior to conventional methods for ascertaining hospital service utilization in this sample of patients exhibiting high-volume and fragmented care. The use of HIE holds implications for improving rigor, safety, and efficiency in research studies.

1. Introduction

Substance use disorders can adversely affect health through multiple pathways, spanning direct causes (e.g., infections from injecting behavior; intoxication-related injury), worsening of comorbid conditions (e.g., exacerbation of asthma from smoke inhalation), as well as exerting myriad indirect effects (e.g., general disorganization leading to failure to keep medical appointments and adhere to medications). Substance use disorders are associated with fragmented, high-volume, and high-cost health care utilization (Schrag et al., 2006; Walley et al., 2012; Gryczynski et al., 2016, Ronan and Herzig, 2016; Gupta et al., 2018; Mejia de Grubb et al, 2019).

Utilization of acute healthcare services, including emergency department and inpatient hospitalization, represent critical data points for clinical trials gauging safety and efficacy of pharmacotherapy and behavioral interventions in addiction medicine and other fields. Due to the high cost of acute care, such utilization also plays a prominent role in health economic analyses that weigh the relative costs and benefits of treatments and services. However, many clinical trials of interventions and economic analyses rely on patient self-report to gauge health service utilization. More sophisticated studies sometimes track such service utilization through healthcare records or administrative claims data, although due to the fragmentation of the US healthcare system, these are often limited to a specific provider, provider network, or payor. This can be particularly challenging in urban areas with high population density, multiple hospitals, and a variety of payors.

Recently, the maturing of health information exchange (HIE) networks has created new opportunities to more accurately and comprehensively track health service utilization. HIE has grown substantially over the last decade (Furukawa et al., 2013; Hersh et al., 2015; Holmgren and Adler-Milstein, 2017). Spurred by the Health Information Technology for Economic and Clinical Health (HITECH) Act, many regions set up independent health information organizations to remove barriers to information sharing and promote interoperability of electronic health records (EHR). Some of these efforts were stymied by technical challenges, concerns about legal barriers related to privacy, and unclear financial sustainability (Vest and Gamm, 2010; Adler-Milstein et al., 2011, Mello et al., 2018), leading some HIEs to fall short of their supporters’ aspirations (Adler-Milstein et al., 2011; Holmgren and Adler-Milstein, 2017; Holmgren et al., 2017). However, in recent years, some of these barriers have been surmounted, making HIE data available for research under certain conditions.

1.1. Evolution of the CRISP HIE

The Chesapeake Regional Information System for our Patients (CRISP) (Horrocks et al. 2016) became a Maryland state-designated HIE and launched in 2009. By 2012, all 47 acute care hospitals in Maryland were connected to CRISP. Clinicians could look up their patients’ hospital service utilization through a query portal. Later in 2012, CRISP activated an “encounter notification service”, whereby providers could get real-time admission and discharge updates for a defined patient panel. In 2013, CRISP became the central access point for the state prescription drug monitoring program, now mandatory for all opioid and benzodiazepine prescribers. CRISP further enhanced its reporting capabilities and analytics, expanded coverage to Washington, DC, provided technical/operational services to the West Virginia Health Information Network, and linked to additional networks in adjacent states. In recent years, CRISP has become widely used for clinical care, maintaining a Master Patient Index of over 18 million and routing more than 250 million documents, messages, and clinical data elements annually.

1.2. CRISP Research Initiative

Although the primary purpose of the HIE is to enhance the efficiency and quality of clinical care, there are considerable opportunities in leveraging it for research. Given the large scale of HIE, epidemiological and population health studies are an obvious application. Clinical studies of interventions (including behavioral interventions, medications, devices, and procedures) could also benefit from the HIE. Many clinical research studies collect information about health service utilization, often for the purposes of tracking patient outcomes, monitoring serious adverse events, and conducting economic analyses. Ascertaining health care utilization at hospitals – both inpatient admissions and emergency department (ED) visits – is critical for gauging how patients fare. Utilization information is typically ascertained through patient self-report and/or review of medical records at the originating clinical research site. This approach can be limited, especially in areas with multiple hospitals and in populations that exhibit high-volume and fragmented care.

Throughout its early history, the regulations governing the CRISP HIE restricted its use to treatment and healthcare operations as permitted under HIPAA, limiting access to those with a direct patient-provider relationship and prohibiting access to CRISP for research purposes. During this time, our team was preparing to launch a clinical research study, entitled Navigation Services to Avoid Rehospitalization (NavSTAR), focused on reducing hospital readmissions in a high-risk group of medical and surgical inpatients with comorbid substance use disorders. The original method to track inpatient admissions and ED visits was through a combination of EHR review at the index hospital where participants were recruited, and through participant self-report at research follow-ups. We also contacted CRISP leadership and began discussions about leveraging the more comprehensive HIE-mediated encounter information for research while preserving necessary privacy protections.

In 2016, Maryland modified regulations to permit state-designated HIEs to allow research under certain conditions (Code of Maryland Regulations, 2016). CRISP made changes to its participation agreement, established a governance framework, and created procedures for making data accessible for IRB-approved studies in which patients expressly consent to let researchers access their CRISP records (https://crisphealth.org/services/crisp-research-initiative/).

At the time of launching the NavSTAR trial, CRISP’s approval for research remained uncertain. Language vetted by CRISP’s legal department was included in consent forms, but the research approval process was still being established. There was no guarantee that data would be available, complete, or usable. Hence, our team proceeded to collect healthcare utilization data according to the original plan. With the trial well underway, NavSTAR was officially approved by CRISP, and we added HIE abstraction to our methodology. Thus, we are now able to compare hospital utilization data obtained via CRISP with that obtained by conventional methods.

2. Methods

The current study uses healthcare utilization data collected as part of the NavSTAR study, a randomized trial comparing a patient navigation intervention with treatment-as-usual for hospitalized medical/surgical patients with comorbid opioid, cocaine, and/or alcohol use disorders. The study recruited inpatients at an urban academic hospital who had received assessment by the hospital’s substance use consultation service. Research staff met privately with patients at the bedside to screen for eligibility, obtain informed consent, conduct the baseline assessment, and subsequently randomize participants to Patient Navigation or Treatment-as-Usual. For the present study, data were pooled across study arms (without any between-arm comparisons). The focus of the present study is on comparing data on hospital service utilization from the HIE, to that ascertained via conventional methods of single-system EHR review and self-report.

2.1. Participants

The current study used data from the first 200 NavSTAR trial participants who completed the 12-month study period, representing half of the total sample in the trial. Study inclusion criteria were: age ≥18 years; current opioid, cocaine, and/or alcohol use disorder (American Psychiatric Association, 2013); and able to provide written informed consent. Comprehension was tested via a brief consent quiz, which patients could take up to three times. Exclusion criteria were: enrollment in substance use disorder treatment 30 days prior to hospitalization; primary residence outside city limits; pregnancy; planned discharge to a long-term inpatient facility (e.g., assisted living, hospice); hospitalization for a suicide attempt; and inability to pass the consent quiz (indicative of not comprehending the study, or possible cognitive impairment). The current sample was recruited after 248 patients were referred to research staff for screening by the hospital-based addiction consultation service. Of the patients screened, 13 declined or were discharged prior to completing enrollment, and 35 were ineligible (14 failed the consent quiz, 6 did not meet diagnostic criteria for substance use disorder, 6 currently in substance use disorder treatment, 5 living outside of the city, 1 suicide-related hospitalization, 1 with planned discharge plan to a long-term care facility, 1 enrolled in another trial of a similar intervention, and 1 excluded due to concerns for research staff safety). Of the 200 participants, 99 were randomized to Patient Navigation and 101 to Treatment-as-Usual. All participants provided written informed consent, including access to EHR and HIE data.

2.2. Data Collection

Hospital inpatient and ED utilization data were collected for a 12-month period following discharge from the index hospitalization. Data were abstracted from the index hospital’s EHR, the CRISP HIE, and via self-report during in-person follow-up interviews conducted at 3-, 6-, and 12-months post-discharge. Information abstracted included admission and discharge dates; length of stay; type of admission; chief complaint; and name of hospital. Cumulative death rates were 4.5% (3 months), 9.5% (6 months), and 13.5% (12 months). Overall rates of follow-up among those available (i.e., alive, not incarcerated, not medically incapacitated) were 75.7% (3 months), 75.9% (6 months), and 67.3% (12 months).

2.2.1. Self-Report

At each follow-up, research participants were queried about any recent inpatient admissions or ED visits since the last study visit using Timeline Follow-Back interview techniques. This approach has been widely used to facilitate reporting of specific behaviors and events, and is frequently used to gauge specific day-by-day information on substance use and healthcare utilization (Sobell and Sobell, 1992; Hoffmann and Ninonuevo, 1994; Robinson et al., 2014). The approach makes use of calendars and situational prompts to enhance recall (Belli, Stafford, and Alwin, 2009). If the participant reported a hospital event, research staff probed for details. Participants were asked to complete a release of information for the relevant hospital, to which staff subsequently submitted requests for discharge summaries.

2.2.2. Index Hospital System EHR

Research staff reviewed the index hospital’s EHR and abstracted any inpatient admissions and ED visits occurring during the 12-month follow-up period. These records covered the index hospital plus nine other non-pediatric hospital facilities in the same medical system (three of which operate within city limits, two others within several miles of city limits).

2.2.3. Health Information Exchange

Research staff accessed each participant’s record in the CRISP HIE, cross-checked and confirmed known hospital events, and abstracted any additional encounters. The CRISP HIE covered all 47 acute-care hospitals in Maryland (with the exception of the federal VA Medical Centers), and additional facilities in Washington DC and adjacent states.

2.3. Data Analysis

For the purpose of this analysis, the combination of HIE and EHR records were considered the most complete objective measure of hospital encounters. HIE+EHR records were compared with self-report, single-system EHR review, and HIE, alone and in selected combinations. Self-report of hospital-related events were considered ‘correctly reported’ if the self-report was verified by a record in the HIE or EHR within a two-week period before or after the reported event date. Number of events and unique participants were compiled for each data source. Rates of correct identification were computed, representing the detection sensitivity against the best objective data source of HIE+EHR. In addition, multivariable logistic regression analyses examined associations of participant and event-level characteristics with non-disclosure of (a) inpatient and (b) emergency department utilization during a completed interview. Independent variables in the models were participant gender, age, race, whether or not overdose was listed as a reason for the event, and (for inpatient hospitalizations) length of stay. Standard errors for these event-level models were adjusted for clustering within participants.

3. Results

3.1. Sample

Participants were 45.5% female, 56.5% black, 40.0% white, and with a mean (SD) age of 45.4 (12.0) years. The sample was 42.0% homeless (based on notes in the EHR). Substance use disorders specific to opioids, cocaine, or alcohol were an inclusion criterion, and 77.5% of participants met criteria for opioid use disorder, while 51.0% met criteria for cocaine use disorder, and 35.0% met criteria for alcohol use disorder. Overall rates of severe substance use disorders (i.e., ≥ 6 of 11 diagnostic criteria) were 72.0% for opioids, 46.5% for cocaine, and 30.0% for alcohol; thus, among participants who met criteria for a specific substance use disorder, a severe disorder was by far the most common. During the baseline interview, participants reported a mean (SD) of 9.1 (15.3) lifetime medical hospitalizations.

3.2. Hospital Service Utilization

Table 1 shows detection of hospital service utilization by different methods. Based on the best objective data of HIE+EHR, there were a total of 1,716 hospital events experienced by 183 unique participants (91.5% of the sample). This included 429 inpatient admissions (among 145 participants) and 1,287 ED visits (among 158 participants) that did not culminate in an inpatient stay.

Table 1.

Detection of hospital utilization by different methods.

	Self-report alone	EHR review alone	Self-report + EHR review	HIE alone	HIE+HER combined
Any Hospital Utilization (Inpatient or ED)
Number of Events	261	954	1053	1666	1716
Unique Participants	105	159	168	181	183
% of events accurately identified	15.2%	55.6%	61.4%	97.1%	100%
Inpatient Hospitalizations
Number of Events	145	234	284	421	429
Unique Participants	83	114	129	143	145
% of events accurately identified	33.8%	54.6%	66.2%	98.1%	100%
ED visits
Number of Events	116	720	769	1245	1287
Unique Participants	58	126	133	157	158
% of events accurately identified	9.0%	55.9%	59.8%	96.7%	100%

Note: EHR= Electronic health record, HIE= Health information exchange, ED= Emergency Department. Rates are computed using HIE+EHR combination as the best objective standard. EHR review at the index hospital system covered a system of 9 non-pediatric hospital facilities. HIE covered 47 hospital facilities statewide, plus facilities in Washington DC and adjacent states. Numbers and rates cover a 12-month period post study entry.

The HIE alone captured a substantial majority of confirmed hospital events (97.1% of all known events), although there remained a relatively small number of events identified only in the single-system EHR review. Reasons for non-identification via HIE are unclear, but 28.6% of unidentified ED events were brief visits in which the patient left mid-care following initial triage. Additionally, one participant opted-out of the HIE (separately from the study), but some data on this individual’s hospital utilization was available in the EHR.

Although the index hospital’s EHR covered nine facilities statewide, relying on single-system EHR review would have identified only 55.6% of hospital events, with similar rates of detection for inpatient admissions and ED visits.

Self-report alone had the lowest detection rate, and would have identified only 15.2% of hospital events. Even the combination of self-report and EHR review showed a large detection gap, identifying only 61.4% of events.

3.3. Performance of Self-Report

Figure 1 shows a breakdown of event determination by self-report, for 429 inpatient hospitalizations identified via the best objective data of HIE+EHR. Overall, only one-third of known inpatient hospitalizations were correctly reported in the interview. Of all inpatient hospitalizations known to have occurred, 40.1% of events were not disclosed by the participant even though an interview was conducted. The remaining events could not be ascertained via self-report due to death prior to the interview due date, incarceration during the interview window, or inability to locate the participant for follow-up.

Figure 1.

Self-report of inpatient hospital service utilization vs. objective records (HIE + EHR).

Note: HIE = Health information exchange. EHR = Electronic health record from the index hospital system.

The performance of self-report was even worse for ED visits (Figure 2). Only 9.0% of all ED visits known to have occurred from HIE+EHR data were correctly identified via self-report during the interview, while 53.5% of ED events were not disclosed even though an interview was completed. Nearly one-third of ED visits could not be ascertained via self-report due to inability to locate the participant for follow-up, with the remaining events left unascertained via self-report due to participant death prior to, or incarceration during, the interview window.

Figure 2.

Self-report of hospital emergency department visits vs. objective records (HIE + EHR).

Note: HIE = Health information exchange. EHR = Electronic health record from the index hospital system.

A more generous standard that considered all self-reports valid (regardless of mismatched time frame or facility, and ‘over-reports’ lacking any verification in records) identified only 14 additional events, improving self-report’s identification rate less than 1%.

The process of requesting discharge summaries from hospitals following participants’ self-report was time-consuming and uncomprehensive. Of requested discharge summaries (prompted by self-report for a specific date/date range, but requested for a broader time period to maximize inclusivity), 56.2% were provided, 6.7% were sent back with an indication of “no record”, and 37.5% of requests went unfulfilled with no response.

3.3.1. Predictors of non-disclosure.

Logistic regression analyses examining which hospital events were identified via HIE+EHR but were not disclosed during a completed interview found no significant associations between non-disclosure of inpatient hospitalizations and participant gender, age, race, overdoserelated event, or length of stay (all ps >.05). For emergency department visits, only older age was significantly (albeit weakly) associated with non-disclosure (Adjusted Odds Ratio= 1.03, 95% Confidence Interval= 1.00, 1.05; p= .03).

4. Discussion

This study examined the utility of Health Information Exchange in identifying hospital service utilization over a 12-month period, as compared with conventional methods of self-report during research follow-ups and single-system electronic health record review. Using HIE data yielded considerably more information about hospital service utilization than would have otherwise been available. Participants in this study were adults with opioid, cocaine, and/or alcohol use disorders who were initially recruited for a study of patient navigation services during an inpatient medical hospitalization. This sample exhibited very high levels of hospital service utilization in the year after enrolling in the study. In the current sample, we found 429 inpatient hospitalizations for 200 people – the equivalent of 2,145 hospital stays per 1,000 population. For comparison, the annual rate of hospital stays in the US general population during this approximate time period was 104.2 per 1,000 population (Freeman et al., 2018).

The current study illustrates an important application of HIE for clinical research, which yielded high-quality data on participants’ hospital service utilization. This approach was far superior to conventional methods of relying on self-report, even when augmented by single-system EHR review. Gathering data by self-report can be burdensome for participants, particularly if they have a number of events to report. Problems of incomplete recall aside, this time burden may create a disincentive to report events. Gathering data by self-report is also labor-intensive for research staff, who must track participants in the community and conduct interviews. Interviewers in the current study used well-validated strategies to promote recall about events and behaviors (Sobell and Sobell, 1992; Robinson et al., 2014). After report of a hospitalization, medical records had to be requested, and monitored as to what was requested and received. As is the case in many studies, a sizable number of participants were lost-to-follow-up. Without the CRISP HIE, their service utilization would have remained unknown. Importantly, incorporating single-system EHR review yielded more comprehensive data than self-report alone, but still fell far short of the HIE, missing 1 in 3 hospital events (despite single-system EHR covering three hospitals locally and nine statewide).

The literature on the accuracy of self-reported hospital service utilization is mixed. One study with homeless adults in Toronto found that participants reported ED visits and hospitalizations fairly accurately compared to administrative healthcare records (Hwang et al., 2016). A study comparing self-report with state billing records in South Carolina found evidence of over-reporting of inpatient hospitalizations, but not ED visits (Saunders et al., 2016). Another study with treatment-seeking individuals with alcohol use disorder compared self-report with service records, finding good agreement for inpatient hospitalizations but underreporting for emergency department visits (Killeen et al., 2004). A comparison of Medicare claims data against the Medical Expenditure Panel Survey found that households underreported ED events by one-third, but accurately reported inpatient hospitalizations (Zuvekas and Olin, 2009). A systematic review of 42 studies on the accuracy of self-reported healthcare utilization (broadly defined) found substantial variability in the ability of self-report to accurately capture service utilization and that accuracy of self-report can be affected by various factors spanning sample characteristics (e.g., cognitive ability), type and frequency of service utilization, and data collection procedures (Bhandari and Wagner, 2006). Thus, while some studies have found that self-report can sometimes provide adequate information about hospital service utilization, others have found it to be lacking. The extent to which self-report accurately reflects service utilization may be subject to various complexities surrounding the study methodology, circumstances of data collection, and population of focus.

HIE promises to become a powerful tool for research by addressing some of the shortcomings of self-report and single-system EHR. One useful way to think of HIE data is as an EHR aggregator. While a number of studies have linked clinical research data to administrative and insurance claims records to measure patients’ health service utilization (Bright et al., 1989; Iezzoni, 1997; Riley, 2009), such data have limitations. They may not be suitable for samples with a diverse payer mix, or who experience frequent lapses in coverage. Moreover, HIE data includes rich detail that cannot be found in claims, including detailed discharge summaries, progress notes, lab results, and consultations.

As one of the first research projects approved to access the CRISP HIE, the NavSTAR study is in a unique position to offer insights into its utility for clinical research. Fortuitous timing of the approval process presented an opportunity to compare different methods of obtaining service utilization data. This study supports the use of HIE data to track hospital service utilization in the context of clinical research. Moreover, it calls into question the validity of self-report data, at least for this sample of patients with high-volume and fragmented utilization.

4.1. Limitations

A potential limitation of this study is the focus on a high-risk group, most with opioid use disorder and nearly half lacking stable housing. This is an important group of patients, especially given the nation’s ongoing opioid crisis, the important role that hospital EDs can play in addressing it (D’Onofrio et al., 2018), and their participation in behavioral, pharmacological, and economic studies of OUD treatment and treatment linkage. Nevertheless, these sample characteristics likely elevated the difficulty of locating participants for follow-up, while perhaps also accentuating problems with recall. The accuracy of self-report likely differs across patient populations, and the chasm between self-report and objective records may have been especially wide in this sample. The CRISP HIE continues to expand its scope, recently adding sub-acute skilled nursing facilities. However, it currently does not have full coverage of ambulatory and urgent care centers, and thus cannot provide a full picture of healthcare utilization. Likewise, there are limits on substance use disorder treatment data sharing due to federal confidentiality regulations. Despite these limitations, the current study illustrates an important application of HIE data for clinical research.

5. Conclusion

There are a number of potential ways that HIE could enhance clinical and services research. First, HIE can offer a reliable and comprehensive approach for tracking hospital service utilization. Second, HIE data can be useful for monitoring serious adverse events in studies of medications, medical devices, procedures, and behavioral interventions. Third, HIE data could reduce the time, cost, and potential risks to research staff tracking participants in the community. Finally, HIE data can inform economic evaluations of services, especially those aimed at reducing readmissions or high-cost healthcare utilization.

Depending on the study, HIE data could enhance research in ways ranging from the marginal to the revolutionary. The advantages of HIE may be especially pronounced for studies focusing on health service utilization as an outcome, and for participants likely to access care across multiple providers and systems. In the case of the NavSTAR trial, the primary outcome of interest is hospitalization, and an economic cost-benefit analysis is planned. Relying exclusively on conventional approaches would have resulted in considerable undercounting of hospital utilization.

As regional HIEs mature, the substantial benefits of secondary uses such as research and public health surveillance could help to justify the investment in creating and maintaining a robust HIE. The experience with the CRISP HIE in the mid-Atlantic region demonstrates that HIE data can successfully be made available to the research community in a way that enhances methodological rigor of research and advances the primary mission of HIE to improve patient care.

Highlights.

• Health information exchange (HIE) can yield high-quality data on health services

• HIE identified inpatient, ED visits in a sample with high-volume, fragmented care

• Utilization review via HIE was far superior to self-report and single-system EHR

• HIE has considerable utility for clinical and services research