A framework for assessing patient crossover and health information exchange value

Abstract

Objective

To evaluate the benefit of a health information exchange (HIE) between hospitals, we examine the rate of crossover among neurosurgical inpatients treated at Emory University Hospital (EUH) and Grady Memorial Hospital (GMH) in Atlanta, Georgia. To inform decisions regarding investment in HIE, we develop a methodology analyzing crossover behavior for application to larger more general patient populations.

Design

Using neurosurgery inpatient visit data from EUH and GMH, unique patients who visited both hospitals were identified through classification by name and age at time of visit. The frequency of flow patterns, including time between visits, and the statistical significance of crossover rates for patients with particular diagnoses were determined.

Measurements

The time between visits, flow patterns, and proportion of patients exhibiting crossover behavior were calculated for the total population studied as well as subpopulations.

Results

5.25% of patients having multiple visits over the study period visited the neurosurgical departments at both hospitals. 77% of crossover patients visited the level 1 trauma center (GMH) before visiting EUH.

Limitations

The true patient crossover may be under-estimated because the study population only consists of neurosurgical inpatients at EUH and GMH.

Conclusion

We demonstrate that detailed analysis of crossover behavior provides a deeper understanding of the potential value of HIE.

Introduction

Over the course of a lifetime, individuals may receive treatment from a variety of providers and non—affiliated hospitals, even within the same geographic area, or may migrate, as in any marketplace, to facilities and providers they feel can best deliver services that meet their healthcare needs. This leads to each patient's medical data between different provider offices and healthcare systems, which can contribute to errors in diagnosis or treatment and create unnecessary costs due to duplicate testing.^1–3

Ideally, when a patient is treated at any hospital, providers should have access to all prior relevant medical records of the patient. As pointed out in a series of studies in Indiana, patients can have multiple emergency department (ED) visits to different hospitals.^4–6 This patient crossover along with a recognized deficit in information sharing and focus on transition from paper to electronic medical records has created a favorable climate for regional and national health information exchanges (HIE). Indeed, HIE for sharing records between medical institutions have the potential to greatly improve the quality of patient care including medical error reduction,^7–9 better surveillance of infectious diseases,^{10 11} and improved delivery of standardized care.¹² A standardized national HIE has the potential to save tens of billions of dollars a year by reducing duplicate testing, inpatient hospitalizations, and length-of-stay.^{13 14} There are a number of regional and state HIE at different stages of development and functionality.¹⁵ For example, the Indiana Health Information Exchange is a well-integrated regional system, where hospitals in the greater Indianapolis area share information through a central server.¹⁶

As the creation of HIE requires significant funding and involves multiple stakeholders with potentially different interests and objectives, demonstrating the extent and direction of crossover can provide useful data for HIE financing and decision making.^17–19 While previous studies have examined the total rate of patient crossover in health systems, little work has been completed to provide a more in-depth analysis of the crossover behavior including consideration of crossover patterns, time between visits, and the clinical classification of patients. To underscore the extent to which an HIE would be useful in Atlanta, and to develop a methodology which can be easily applied to larger patient groups in future studies, we examined the crossover of patients seen by the neurosurgery services at two hospitals in the Atlanta metropolitan service area: Emory University Hospital (EUH) and Grady Memorial Hospital (GMH).

Background

Both EUH and GMH are busy neurosurgical centers offering a full spectrum of services, but each has specific subspecialty practices that are primarily based at each facility. Among Atlanta area hospitals, EUH and GMH ranked sixth and fourth, respectively, for inpatient discharges in 2009.²⁰ During the time period of this retrospective study, GMH was the major level 1 trauma center in the Atlanta metropolitan area, and therefore the majority of neurosurgical trauma patients visited this facility, while the core cerebrovascular neurosurgery and endovascular neuro-interventional practices were based at EUH, as was the Emory Pituitary Center. This centralization of expertise: (i) provides the best return on invested capital; (ii) maximizes capacity utilization of specialized fixed equipment; (iii) clusters thought leaders and other human capital with relevant talent and expertise; and (iv) supports the provision of the highest quality of patient care with the best outcomes possible. Undoubtedly this also has an influence on patient flow, particularly when patients have non—emergent complex cases that can be safely sent for care at the designated center of excellence.

The Department of Neurosurgery at Emory University School of Medicine provides neurosurgical physician services at EUH and GMH. Consistent with the distribution of faculty, the annual volume of neurosurgical patients seen at EUH is higher than at GMH. During our study period, EUH had approximately 50% more neurosurgical inpatient visits than GMH. Although the neurosurgery services at GMH and EUH are provided by one common group of neurosurgeons, the two hospital facilities are independent entities. Each has its own funding, management, administration, and healthcare information systems.

The common group of neurosurgery providers between EUH and GMH presented a rare opportunity to analyze the crossover of patients between two healthcare facilities utilizing administrative databases maintained at each facility to support clinical operations. Leveraging this data, the Institutional Review Board approved a pilot effort, limited to the neurosurgical inpatients at EUH and GMH, undertaken to inform the design of further studies.

Materials and methods

Neurosurgical inpatient visit data from Microsoft Access databases, used to maintain physician patient rounding lists at EUH and GMH between June 2000 and September 2009, were compiled, and included patient identifiers, admit date, diagnoses, and operative procedures. Patient identifiers in this data set include first name, last name, middle name, and age at the time of visit.

To establish a unique identifier for each patient, we used SQL queries in Microsoft Access to standardize names, and then compared names and birth date ranges (determined from age and admit date). Entries with identical names and overlapping birth date ranges were assumed to belong to the same patient. Entries with identical patient names that could not be validated as the same individual with birth dates were treated as different patients.

After eliminating duplicate records, we identified the patient crossover between EUH and GMH from the unique patient list and the corresponding visit records. We also analyzed the diagnosis fields to determine if there was a different rate of crossover among patients with certain diagnoses. With the intention of choosing diagnoses hypothesized to have high crossover rates, we focused on 10 diagnosis groups including non-traumatic subarachnoid hemorrhage (ntSAH), traumatic subarachnoid hemorrhage (tSAH), trauma (TR), arteriovenous malformation (AVM), aneurysm (RYSM), intraparenchymal or intraventricular hemorrhage (IH), extra-axial (subdural and epidural) hematoma (SE), cerebrospinal fluid leak (CSFlk), shunt related (SH), and pituitary related (PIT). Due to the free text characteristics of the diagnosis fields, with large variations in nomenclature, a detailed methodology was developed to assign each patient visit to its corresponding diagnosis group(s). This methodology is explained in the supplementary online appendix/table A1.

Using a χ² test, we identified diagnosis groups at each hospital having significantly higher rates of crossover than the remaining population of patients not included in the diagnosis group. Due to the non-continuous (binary) nature of the data, the Yates χ² was used, which corrects for continuity.^{21 22} Fisher's test, which also corrects for continuity, was utilized instead of Yate's χ² when the sample size was small.²³ A one-tailed hypothesis test was used, testing whether the crossover rate among those with a diagnosis was greater than among those never receiving the diagnosis at a particular hospital.

For patients who visited both hospitals, we used Microsoft Excel to order visit data chronologically and identify patient movement patterns and time between visits. We investigated movement patterns among crossover patients receiving similar diagnoses at the same hospital. For these patients, we examined the proportion whose crossover occurred before the diagnosis as opposed to after the diagnosis to identify how flow patterns relate to the timing of the diagnosis. We also identified crossover patterns among patients receiving similar diagnoses at both hospitals. Previous patient crossover studies have used similar methods to construct unique patient lists and determine crossover.^{4–6 10} Other studies identified the number of return visits within specific time periods (eg, 24 h)⁶ and focused on crossover patients having a specific diagnosis.¹⁰

Results

Patient crossover

We identified 24 115 unique patients between the databases from the two hospitals, accounting for 29 589 total visits over the 9-year period studied. Within the study period, 3704 patients had multiple visits, and 202 unique patients visited both hospitals. These 202 patients account for 1.399% of the total number of neurosurgical inpatients treated at EUH, and 2.13% of neurosurgical inpatients treated at GMH.

As shown in table 1, 5.45% of all neurosurgical inpatients with multiple visits were seen at both hospitals during the 9-year period. This crossover population accounts for 0.8377% of the total inpatient neurosurgical population visiting EUH and/or GMH.

Table 1.

Neurosurgical inpatient crossover between Emory University Hospital (EUH) and Grady Memorial Hospital (GMH): the number of patients who visited only EUH, only GMH, and both EUH and GMH over the study period

	Neurosurgical inpatients (N=24 115)	Neurosurgical inpatients having 2+ visits (N=3704)
EUH only	14 438 (59.87%)	2936 (79.3%)
GMH only	9475 (39.29%)	566 (15.3%)
Both EUH and GMH	202 (0.8377%)	202 (5.45%)

Patient crossover by diagnosis group

We calculated the crossover rates for subsets of patients, based on diagnoses over the course of the study and the hospitals at which the diagnoses occurred. For example, we examined the crossover rate among patients who had an aneurysm (RYSM) diagnosis at EUH as well as the crossover rate among patients with a RYSM diagnosis at GMH. The crossover rates and statistical results (p value from χ² test, with a standard α=0.05 for assessing significance) for each diagnosis group and hospital are provided in table 2. Note that a single patient meeting the qualifications for multiple diagnosis groups is represented in each group. As a result, patient groups reported in table 2 are not disjoint.

Table 2.

Patient crossover by diagnosis group

	EUH		GMH
	Proportion of patients with diagnosis at EUH who crossover (CrossD)	p Value: HA: CrossD>CrossW	Proportion of patients with diagnosis at GMH who crossover (CrossD)	p Value: HA: CrossD>CrossW
Intraparenchymal or intraventricular hemorrhages (IH)	16/1391 (1.15%)	0.7391	23/1306 (1.76%)	0.778
Extra-axial (subdural and epidural) hematomas (SE)	11/936 (1.2%)	0.6564	22/1605 (1.37%)	0.9805
Arteriovenous malformation (AVM)	11/458 (2.4%)	0.0477	7/33 (21%)	0.0001
Aneurysm (RYSM)	26/1712 (1.52%)	0.3421	33/144 (23%)	0.0001
Cerebrospinal fluid leak (CSFlk)	9/314 (2.9%)	0.0232	3/45 (7%)	0.0603
Pituitary related (PIT)	16/881 (1.8%)	0.1639	26/864 (3.0%)	0.0349
Trauma (TR)	15/790 (1.9%)	0.1338	46/5314 (0.87%)	0.9999
Non-traumatic subarachnoid hemorrhage (ntSAH)	43/1958 (2.20%)	0.1338	40/325 (12%)	0.0001
Traumatic subarachnoid hemorrhage (tSAH)	6/201 (3.0%)	0.0008	18/1184 (1.52%)	0.9069
Shunt related (SH)	15/1046 (1.43%)	0.0525	15/318 (4.7%)	0.0012

Crossover rates and χ² p values for Emory University Hospital (EUH) and Grady Memorial Hospital (GMH): comparing the crossover rate for patients who are and who are not in the diagnosis group at the respective hospital. All patients included in this analysis have at least one visit with the indicated diagnosis.

Cross^D: (patients with the diagnosis at the given hospital who crossover)/(patients with the diagnosis at the given hospital).

Cross^W: (patients without the diagnosis at the given hospital who crossover)/(patients with the diagnosis at the given hospital).

The alternative hypothesis, H_A: the crossover proportion is higher among those with the diagnosis than those without the diagnosis for the respective group.

Patients who have an AVM, CSFlk, or tSAH diagnosis at EUH have a significantly higher probability of also visiting GMH than EUH patients without these diagnoses. Similarly, among GMH patients, those with AVM, RYSM, PIT, ntSAH, or SH diagnoses have a significantly higher rate of crossover than patients at each of these facilities without these diagnoses. At EUH, crossover rates for patients with a TR diagnosis are similar to those for EUH patients without a TR diagnosis, while GMH patients experiencing a TR diagnosis are less likely to crossover than GMH patients without a TR diagnosis (p=0.9999).

Patient flow

Two types of inpatient movement were identified between the hospitals: uni-directional and bi-directional. Figure 1 presents the movement patterns observed and percentages of crossover patients exhibiting each.

Figure 1.

Patient movement patterns: the various movement patterns between Emory University Hospital (EUH) and Grady Memorial Hospital (GMH) along with the respective numbers and percentages of patients exhibiting each. The patient groups are disjoint.

Of the 202 patients visiting both hospitals, eight (4.0%) had indiscernible movement patterns because visits at different hospitals took place on the same day and inpatient visit data lacked time stamps. Overall, 175 (87%) crossover patients had uni-directional movement, never returning to the hospital they had first visited. In this same patient group, 130/175 (74%) were treated at GMH first. Note that the first visit for a given patient may not be their actual first visit due to unavailable patient visit data prior to the start date of the study period. For more detailed flow results, see the online supplementary appendix/table A2.

Movement patterns among crossover patients receiving similar diagnoses at the same hospital are reported in supplementary online appendix/table A3. The majority of crossover patients visit GMH prior to EUH, regardless of diagnosis. However, of the 15 crossover patients having a TR diagnosis at EUH, eight (53%) visit GMH before the diagnosis at EUH, while nine (60%) visit GMH after the diagnosis at EUH. Likewise, among crossover patients having an SH diagnosis at GMH, 60% and 53% of patients visit EUH before and after the diagnosis at GMH, respectively. The before and after percentages for each case may not sum to 100% due to bi-directional patient movement.

Time between visits

Table 3 reports the number of days between visits for all movement patterns reported. For example, for movement direction GMH→EUH, the time between a visit to GMH and a subsequent visit to EUH is included for all uni-directional and bi-directional flow patterns with GMH visited before EUH. Note that movement directions GMH→EUH and EUH→GMH are disjoint with respect to the times between visits, but are not patient disjoint sets due to bi-directional movement patterns.

Table 3.

Patient movement patterns and time between visits

Movement direction	Number of patients	Time between visits (days)
		Mean	Median
GMH→EUH	149 (77%)	275	34
EUH→GMH	64 (33%)	470	248
All directions	194	334	57

The number of patients exhibiting each direction of patient movement between Emory University Hospital (EUH) and Grady Memorial Hospital (GMH) and the time between visits, reported at the mean and median.

Of crossover patients treated at both GMH and EUH with discernible movement patterns, 77% of patients treated at GMH were later treated at EUH. Figure 2 reports the frequency of times between visits for movement directions reported in table 3. For 40 patients, the time between a visit to GMH and a visit to EUH, in that order, is 5 days or less.

Figure 2.

Days between visits to different hospitals: the number of days between visits to Emory University Hospital (EUH) and Grady Memorial Hospital (GMH), reported by direction of patient movement and frequency of occurrence.

Flow within diagnosis groups

The movement patterns of patients who received similar diagnoses at both hospitals are shown in table 4. Of the 50 patients who visited both hospitals and were diagnosed with an ntSAH at either hospital, 30 (60%) had an ntSAH diagnosis at both hospitals. When an ntSAH diagnosis first occurred at GMH, more than 50% of patients had a second visit at EUH within 1 week.

Table 4.

Patient crossover by diagnosis group: mean and median time between visits with similar diagnoses at Emory University Hospital (EUH) and Grady Memorial Hospital (GMH), based on direction of movement, and including the percentage of crossover patients receiving the same diagnosis at both hospitals.

	Time between visits						Crossover patients with diagnosis at both hospitals*
	GMH→EUH			EUH→GMH
	Mean time (days)	Median time (days)	Patients	Mean time (days)	Median time (days)	Patients
Non-traumatic subarachnoid hemorrhage (ntSAH)	7	1	27	94	9	5	30/50 (60%)
Aneurysm (RYSM)	34	3	11	86	9	5	15/43 (35%)
Pituitary related (PIT)	255	92	11	247	36	3	12/30 (40%)
Arteriovenous malformation (AVM)	35	5	6	–	–	–	6/12 (50%)
Shunt related (SH)	334	309	4	520	231	4	7/23 (30%)
Cerebrospinal fluid leak (CSFlk)	18	18	2	–	–	–	2/10 (20%)
Trauma (TR)	20	7	5	660	702	5	9/52 (20%)
Traumatic subarachnoid hemorrhage (tSAH)	269	269	2	702	702	1	1/21 (5%)

^*Patient groups corresponding to movement patterns GMH→EUH and EUH→GMH are not disjoint due to bi-directional movement. The denominator corresponds to the count of crossover patients having the respective diagnosis at GMH, EUH, or both hospitals.

Among inpatients in diagnosis groups identified to have significant crossover rates for GMH (AVM, RYSM, PIT, ntSAH, and SH), at least 30% had the same diagnosis at EUH.

For patients with an AVM, RYSM, ntSAH, or TR diagnosis at both hospitals, the median time between a visit to GMH followed by a visit to EUH is less than 1 week, implying that the visits may be for related incidents. For patients with SH diagnoses, the mean and median time between visits to different hospitals is more than 7 months. This is consistent with the nature of shunts, which are implanted foreign bodies subject to malfunctions or infection any time after they are implanted.

Discussion

Crossover population

In this study, only 5.45% of neurosurgical inpatients with multiple visits in the 9-year period studied visited both hospitals' neurosurgical departments. Factors contributing to patients visiting different hospitals may include emergency needs, second opinions, referral to specialty services, economic hardship, and patient preference. At least 50% of consecutive patient visits to different hospitals occurred within 57 days of each other, supporting the need to share information about recent visits including diagnoses, diagnostic testing results, and treatments. More efficient information sharing between hospitals facilitated by HIE can have many positive consequences including cost savings through the reduction of duplicate tests and improved patient outcomes due to better information.^{13 14 24}

While improved patient outcomes are likely to occur for all crossover patients, although difficult to prove,²⁵ the reduction of duplicate tests is more likely to be realized for patients with acute conditions who transfer or visit both hospitals within a short time period, since it is possible that without HIE, the same test might be repeated. As an example, for patients diagnosed with ntSAH at both hospitals, the time between visits is short, due to the need to transfer to a subspecialized care setting and possibly the need for urgent, but not emergent, intervention. For these types of patients, efficient sharing of medical records can be useful in the prevention of duplicate testing.

We hypothesize that the population of patients presenting with acute cerebrovascular neurosurgical disease (eg, ruptured intracranial aneurysm patients) who were transferred from GMH to EUH for further specialized management likely underwent duplicate diagnostic testing including imaging, and laboratory and other diagnostic studies (eg, electrocardiogram, admission chest x-ray, etc) due to the lack of healthcare information systems integration across the two facilities. Whenever unneeded duplicate testing occurs, this can result in increased costs and potentially unnecessary and avoidable risk to patients.²⁶

HIE is also valuable for patients with chronic medical issues for whom the time between visits to different hospitals is longer since medical history and historical diagnostic results may be significant in identifying treatment needs. For example, within the neurosurgical patient population, for patients with SH diagnoses, it is important to be able to compare current and historical imaging to help determine whether a shunt has indeed malfunctioned. Thus, ease of access to prior imaging and other health records from outside medical facilities made available through HIE can be critical to the provider's ability to (i) make an accurate diagnosis, and (ii) present these patients with the best information and treatment options.

Our analysis also demonstrates the significance of the crossover behavior exhibited by patients in certain diagnosis groups. We conclude that for patients with certain diagnoses, the probability of receiving treatment at both hospitals is greater than for a standard patient. An understanding of the nature of these conditions can highlight the ways and the settings in which HIE implementation can have the greatest and least benefit.

The crossover rate among AVM patients at GMH is higher than that for the standard neurosurgical GMH patient and each of these patients crossover after the AVM diagnosis at GMH. More importantly, six of these seven patients identified in our crossover patient data set had a subsequent AVM diagnosis at EUH as well, with a median time of 5 days between visits. All patients with an AVM diagnosis at both EUH and GMH have their first diagnosis at GMH, and these patients account for 50% of all crossover patients who have an AVM diagnosis at either hospital. While the crossover rate for patients diagnosed with AVM at EUH is significantly higher than for the remaining population, 90% of these crossover patients visit GMH first. Thus, there is a strong flow pattern (GMH→EUH) among patients receiving an AVM diagnosis at either hospital. Many of these patients receive an AVM diagnosis at both EUH and GMH, suggesting these are inpatient stays related to the same acute incident. Such information is useful; for example, if a regional HIE were considering a staged roll-out with a pilot phase, it would be helpful to know which patient populations demonstrate the highest crossover to identify the best patient groups for testing the prototype system.

As demonstrated by the analysis of AVM patients, both the nature of an acute diagnosis as well as the patient flow patterns provide a better understanding of the potential value of HIE. Similar insights can also be drawn for chronic patient populations. Approximately half of all crossover patients with an SH diagnosis at GMH were treated at EUH before visiting GMH, and a similar percentage were treated at EUH after the diagnosis at GMH. Similar results exist for crossover patients with an SH diagnosis at EUH as well. Given that the movement between hospitals for crossover patients with this diagnosis flows in both directions nearly equally, both GMH and EUH would likewise benefit from the mutual sharing of information in this case.

Our analysis reveals that the crossover for patients with TR diagnoses is no more frequent than for the typical patient. Thus, we demonstrate that a local HIE between EUH and GMH is less valuable for patients with traumatic injuries than for the standard neurosurgical patient population examined in this study. Trauma patients transported to GMH, a level 1 trauma center, may not be local to Atlanta, and therefore may receive their routine medical care elsewhere. Thus, if trauma patient crossover was examined more broadly including additional medical facilities in Georgia, there may be more trauma patient crossover observed.

Study limitations

We only investigated patient crossover among neurosurgical patients at two of the many hospitals in the Atlanta region. In a study involving two hospital systems in the Indianapolis region, a 20% crossover was identified among ED patients having multiple visits in a 1-year period.⁴ The fact that a greater overlap was seen in the ED patient population in a 1-year time span as compared to a 9-year time span in the present study is not surprising. The ED sees a general patient population and many EDs serve as a primary care provider of last resort, particularly for the indigent and uninsured. Patients present to EDs with medical problems of any sort. In contrast, significantly fewer patients have medical problems requiring neurosurgical management and as a result, the probability of patient overlap is likely significantly lower. Among five hospital systems in the Indianapolis region, a 25% crossover was identified among ED patients having multiple visits in a 1-year period,⁵ increasing to 34% in a companion study covering ED visits over 4 years for the same hospital system.⁶ These results lead us to believe that further expansion of this study to include neurosurgical patients presenting to other Atlanta area hospitals would show that the neurosurgical patient crossover is higher.

Neurosurgery is a highly specialized clinical service that cares for a small patient population as compared to other clinical services, and therefore we hypothesize that the patient crossover rate would likely be significantly higher if assessed at EUH and GMH in other less specialized clinical services and services that care for patients with diseases that have high incidence and prevalence. Such services may include primary care, internal medicine, emergency medicine, pediatrics, general surgery, and even cardiology, given the high incidence and prevalence of coronary artery disease in the US population.

Although the Microsoft Access database is used to maintain a list of patients admitted to and consulted upon by the neurosurgery service for the purpose of physician rounds, it is not a hospital admissions or billing database. Thus, it is possible that some patient visits are not captured. In addition, due to inconsistencies in name entry formatting, it is possible that a patient may be mistaken for two different patients.

The subgroup analysis herein was based on our interpretations of the free text diagnosis fields in the Access database. Given that the diagnosis information is free text rather than categorical and that a continuum of abbreviations and language is used by the neurosurgery physicians to describe similar concepts, the categorization of patients into subgroups is imperfect at best. However, we feel that the groupings are sufficient to provide insight about and a reasonable estimation of the crossover among neurosurgery patients between EUH and GMH.

Finally, crossover rates, flow patterns, and trends with regards to time between visits may vary by age and/or gender. Unfortunately, gender was not available in the patient database. Due to the long time span of the study, in addition to patients having multiple visits at different ages, an accurate age distribution of the total patient population in our study was not possible.

Conclusions

There is a cost associated with the infrastructure, development, and integration required to implement HIE between healthcare facilities. Better information leads to better decisions, but the perceived lack of return on investment to the stakeholders providing the capital needed to implement HIE has been a barrier to the adoption and acceleration of HIE.^{24 27} One solution to this impasse is the development of rigorous analysis frameworks that help examine how HIE might create value and the magnitude of this potential value.

Examination of crossover rates within a population is a crucial first step to understanding the value of HIE. While the crossover rate for our study population may not be large, possibly indicating that HIE is unnecessary, we provide a detailed analysis of patient flow patterns and patient characteristics which may help highlight particular patient populations for which HIE may create the most value and other subpopulations for which HIE may offer less value. This methodology allows for identifying different ways in which HIE benefits different groups (eg, patients with acute vs chronic conditions).

Based on the framework herein presented, knowledge about specific patient characteristics that are drivers of crossover behavior are likely to inform decision making with regards to HIE. This methodology can help stakeholders identify where capital expenditures on HIE may not only be worthwhile financially but also clinically warranted and beneficial to patients.

While this pilot study focused on a neurosurgical inpatient population, the analysis methods presented are valuable for extension into studies of larger and more general patient populations. Further study is needed to develop additional methodologies useful for assessing the value of HIE.