Health Information Exchange in the ED: What Do ED Clinicians Think?

Abstract

Objectives

Our regional health information exchange (HIE), known as Carolina eHealth Alliance (CeHA)-HIE, serves all major hospital systems in our region and is accessible to emergency department (ED) clinicians in those systems. We wanted to understand reasons for low CeHA-HIE utilization and explore options for improving it.

Methods

We implemented a 24-item user survey among ED clinician users of CeHA-HIE to investigate their perceptions of system usability and functionality, the quality of the information available through CeHA-HIE, the value of clinician time spent using CeHA-HIE, the ease of use of CeHA-HIE, and approaches for improving CeHA-HIE.

Results

Of the 231 ED clinicians surveyed, 51 responded, and among those, 48 reported having used CeHA-HIE and completed the survey.

Conclusions

Results show most ED clinicians believed that CeHA-HIE was easy to use and added value to their work, but they also desired better integration of information available from CeHA-HIE into their system’s electronic medical record.

Until recently the availability of shared health information to clinicians in emergency departments (EDs) has been fragmented, inefficient, and when omitted, has led to unnecessary duplication of resources, tests and procedures, increased costs, and potentially compromised quality of care.¹ Federal initiatives have incentivized the use of health information exchange (HIE)^2,3 to allow routine, real-time sharing of electronic medical record (EMR) information from disparate healthcare systems. The immediate availability of patient data from multiple institutions at the point of care would seem to be of significant value for clinicians; however, HIE adoption and use remain low despite documentation of the potential value. Barriers to HIE growth include the cost of HIE operations, the perceived impediment of HIE to competition,^4,5 gaps in data and workflow, and usability issues.⁶

Reported potential HIE benefits include improved quality of care and lower costs achieved by reducing ED length of stay and avoiding redundancies in laboratory, radiographic imaging, consultations, and unnecessary admissions.^7–9 Despite demonstrated benefits, ED clinician HIE use remains unexpectedly low, at 2% to 26% of ED encounters.^10,11 Factors other than technology have been important to HIE adoption; hospitals in areas with high levels of competition for patients were less likely to implement an HIE, whereas public hospitals, nonprofit hospitals, and network members were more likely to adopt an HIE.¹² ED clinicians have reported workflow disruption, the need to use multiple systems, time constraints, not anticipating helpful information in the HIE, and “bugs in the system” as their reasons for not using available HIEs. Other issues include the disruptive nature of new technologies, forgotten passwords, user interface problems, printer problems, and physicians’ lack of trust in HIE.¹³ Whatever the cause, the sustainability of an HIE is under threat despite its demonstrated capacity to improve healthcare quality and achieve cost savings.

Although patients are the principal beneficiaries of information shared via an HIE, the principal end users are ED clinicians. Understanding end users’ perspectives on HIE technology is crucial to its long-term success and to the adoption, use, and improvement of HIE systems such as the Carolina eHealth Alliance (CeHA)-HIE.¹⁴

The objectives of our study were to characterize the perceptions of ED clinicians about the functionality and usability of the CeHA-HIE system and to describe the reasons that ED clinicians do or do not use it. This information may inform the design, implementation, and improvement of HIE systems. We sought to determine how ED clinicians viewed the HIE, for whom they considered the HIE useful, and which design characteristics they considered most valuable. We investigated quality issues and explored options for improving the functionality and usability of the CeHA-HIE.

Methods

Study Period, Setting, Sample Size, and Design

The study was conducted from May 2014 to October 2014 in a medium-sized metropolitan area in the southeastern United States and the surrounding region (2012 census of approximately 700,000 people). Study sites included all CeHa-HIE participating sites in 11 EDs operated by 4 hospital systems: 1 academic ED, 5 community hospital EDs, 4 freestanding EDs, and 1 ED/chest pain center. Eligible participants included 231 attending emergency physicians, residents, and advanced practice providers collectively referred to as ED clinicians practicing at CeHA-HIE study sites.

We used an observational, cross-sectional study design with a voluntary, anonymous, online survey (Appendix) consisting of 25 items to investigate HIE use and its usability and functionality, the quality of the information available, the value of time spent using the CeHA-HIE, and its ease of use; and to describe issues relevant to using the HIE, including circumstances under which the HIE was found to be useful. We used the HIE system name, CeHA, in all survey items because it was the term used to identify our regional HIE during the study period and because survey items applied only to CeHA, not to an HIE in general. Item numbers in this text match those in column 1 of the survey. Per contractual agreement with affiliated systems, we were unable to collect any data, including ED clinician characteristics that may allow readers or data users to identify individual clinicians, EDs, or health systems.

Usability

We defined usability as “a set of attributes that bear on the effort needed for use, and on the individual assessment of such use by a stated or implied set of users.”¹⁵ Degree of usability depends on whom the users are and on their expectations regarding the system. To assess the usability of CeHA, we adapted 6 questions from the 10-item System Usability Scale (SUS) to create 8 usability survey items and developed 2 additional items based on prior, informal assessments of user perceptions. Evaluations of various surveys note advantages to adapting SUS questions.¹⁶ SUS is the only questionnaire whose questions address different aspects of user reaction to a Web site or system as a whole as opposed to asking the user to assess specific features of the Web site/system (eg, visual appearance, organization of information). Survey items 4, 6 through 11, 14, and 24 assess respondent agreement with queries about CeHA usability with SUS items 4, 6 through 9, and 11.

Functionality

Functionality is “a set of attributes that bear on the existence of a set of functions and their specified properties.”¹⁵ The intent of functionality questions is to determine whether required functions are available in the system and satisfy stated or implied system needs. Survey items 5, 12, 13, 15 through 17, 20, 21, and 23 assessed respondent agreement with CeHA functionality as described in prior publications.^1,7,9,12,13 For example, a stated system need was that available information from participating healthcare systems should be present in CeHA. The corresponding survey item was “Information about my patient was present most of the time when I queried CeHA.” Other functionality attributes included whether CeHA was perceived as easy to use, whether the user was able to quickly find what he or she was looking for, and whether available CeHA data improved the quality of care provided.

Patient and Clinical Issues and Overall Assessment

Informal interaction with ED clinicians using CeHA-HIE indicated that they were more likely to use CeHA-HIE if they anticipated the presence of relevant information in the HIE or suspected that they were dealing with a patient with a complex diagnosis or who was seeking narcotics for inappropriate reasons. Survey items 18, 19, and 22 asked about patient characteristics and clinical issues that may prompt CeHA use. Respondents also were required to answer item 25, “What is your opinion of the functionality and quality of CeHA?”

Survey Administration

The survey was implemented and distributed via a Research Electronic Data Capture (Vanderbilt University, Nashville, TN) installation at the Medical University of South Carolina.¹⁷ ED medical directors at each hospital system agreed to serve as “site champions,” who promoted survey participation and supervised study operation at their respective sites. We invited ED clinicians to participate in the study via e-mails sent by site champions that contained a link to the survey. No incentives for ED clinicians to use the HIE or to complete the survey were offered. Two reminders to participate were sent to all ED clinicians at 1-month intervals following the initial notification of the survey’s online availability.

Each individual respondent answered question 1, “Have you ever used the CeHA system?” Respondents answering “no” (CeHA nonusers) were directed to two additional questions about why they had not used CeHA and what could be done to make it more likely that they would use it in the future. Potential choices for answering these questions were presented in checkbox format with space for writing “other” responses.

Participants answering “yes” to item 1 (CeHA users) were directed to 22 additional survey items. Respondents used a visual analog scale (VAS) from 1 (strongly disagree) to 100 (strongly agree) to indicate their level of agreement with each of 21 items. Comments were solicited via prompts following each item regarding ways in which the HIE could be improved. The final item required a written response regarding overall functionality and quality of CeHA. A response of “none” was allowed.

Scoring

For survey items 4 through 23, we defined “strong agreement” as a mean VAS score of ≥70 for positively framed items (meaning the higher the score, the better) or a mean VAS score of ≤30 for negatively framed items (meaning the lower the score, the better). Weak agreement was indicated by the obverse, particularly mean values <70 for positively framed items and >30 for negatively framed items.

Human Subjects

No identifiers for participants were collected and no protected health information was recorded. Approval was obtained from the institutional review board or human subjects committee of all four participating hospital systems. The approval of the institutional review board committee was received for this work through the research proposal (Pro00021569) entitled “Sustaining Emergent Access to Information Systems Linking EDs (Sea Isles)” that was considered and reviewed by the institutional review board and approved with respect to the study of human subjects as adequately protecting the rights and welfare of the individuals involved, using adequate methods of securing informed consent from these individuals, and not involving undue risk in the light of potential benefits to be derived therefrom.

Results

Data were extracted directly from Research Electronic Data Capture. Survey responses were received from 51 individuals, or 22.1% of those surveyed. Three respondents reported not having used the CeHA. We report summary qualitative results for CeHA nonuser responses (items 2 and 3). The sample size, mean, and standard deviation are reported for each of the 21 survey items separately and by categories of usability, functionality, and patient and clinical issues. Of the 21 items, 11 were answered by 48 CeHA users and 10 by 47 CeHA users. Comments were offered for 14 of 21 survey items (Table 1) and item 25 (Table 2). We used a theme-based, qualitative approach to prepare and present comment summaries.

Table 1.

Survey responses: system use, usability, functionality, and patient and clinical issues

Item no.	Survey item	No. respondents	Mean	SD	Comments
1	Have you ever used the CeHA system?	51	—	—	None
2	Please give at least two reasons for why you have not used CeHA. Please check or write in as many reasons as you would like.	3	—	—	“Retired from clinical practice” “All of the above” “No username or login set up”
3	Despite the potential usefulness of CeHA, clinician adoption is a barrier to sustaining it. Please let us know two things we could do to make it more likely that you will use CeHA in the future. Please check or write in as many reasons as you would like.	2	—	—	None
Usability constructs
4	I would like to access CeHA for every patient I see.	48	74.7	21.8	“Some minor complaints is not necessary. Same for patients new to area.” “Not every patient but one with complicated history/physical examination, multiple visits, pain/medication requests.” “Only if it was autoaccessed; not having to take the time to look up every patient.” “If it has useful information.” “It should be integrated into Epic so I don’t have to go out and search.” “No need for low- acuity patients.”
6	I thought CeHA was easy to use.	47	71.9	22.2	“Yes and no” “It would be nice if we could login on the top of Epica like for OACIS; frequently have to go to multiple computers to find a CeHA link that’s functional.” “Another log in code!” “When CeHa access works, it is easy to use.”
7	I found CeHA cumbersome to use.b	48	38.0	24.3	“At first it was; it is better now.” See above. “Login code needs to be streamlined.”
8	I found the information from the various hospital systems in CeHA to be well integrated.	48	57.6	23.4	“Depends on hospital; some hospitals have been sketchy, but getting better.” “Some hospitals are lacking in ED notes, while others are lacking in d/cA summaries.” “Need better and more data sharing.” “Sometimes it seems sparse for a recent visit. Would be great to get other hospitals on it.”
9	I felt very confident using CeHA.	47	75.3	17.5	None
10	I felt very confident using the information I found in CeHA.	48	83.4	14.3	None
11	I needed to learn a lot of things before I could get going with CeHA.b	47	19.9	14.5	None
14	I often found the information in CeHA to be incomplete and to be missing important data.b	47	51.7	21.3	“Sometimes.” “Never can get the ED physicians’ actual chart from the hospital.” “Some of the newer tests might be missing.” “It is improving, but at times different findings in ED records.” “Often a visit was so proximate to my visit with a patient, the information from outlying hospital hasn’t been loaded.”
24	The information I get from CeHA usually makes the time it takes to log on and look worth the effort.	48	75.8	20.0	None
Functionality constructs
5	If CeHA provided good information and was easy to use, I would use it for every patient I see.	48	79.8	21.5	“Not needed for every patient.” “Would use for anyone who had history of recent admittance/ED evaluation at another hospital.” “Not every but ones where I am trying to avoid repetitive testing.” “Record of CeHA use could be built into Epic as a check box, again as a reminder.”
12	Information about my patients was present most of the time when I queried CeHA.	48	59.5	21.8	“If patients had been to the other facilities.”
13	I found the information I was looking for when I used CeHA.	47	64.2	19.5	“Great in patients who not only hospital shop but also receive treatment at one hospital and then present to one of the others in town for further treatment or complications.”
15	The charts from recent visits to other EDs were usually available to me in CeHA.	48	57.6	23.4	“Usually only get the radiology and laboratory values; never get EKG.” See above. “Charts from hospital usually are only discharge instructions; charts from hospital are generally not available.” “If the patient had visited the other EDs.”
16	Recent discharge summaries from other hospitals were usually available to me in CeHA.	48	59.1	23.1	“Not as reliable as ED charts.” “If hospitalization had occurred.”
17	The information about my patient in CeHA was usually sufficient to be useful clinically.	47	71.3	16.6	None
20	I believe CeHA is a very valuable addition to my practice and improves my efficiency.	47	84.1	14.2	None
21	I believe CeHA improves the quality of the care I deliver.	47	83.5	14.0	“When information needed and available.” “Without any doubt.”
23	I would prefer CeHA to be embedded in the electronic medical record at our hospital.	48	90.8	10.6	“This would be a huge help!” “Single sign on.”
Patient and clinical issues
18	I used CeHA more often when I anticipated there would be information available on my patient.	47	86.2	15.3	None
19	I am more likely to use CeHA for diagnoses that are more complex.	47	78.6	17.5	“Lots of crossover patients from hospital.”
22	I am more likely to use CeHA if I believe a patient is seeking narcotics for inappropriate reasons.	48	74.3	23.6	“Very helpful.” “More likely to check DHEC site.”

^APls define d/c.

CeHA, Carolina eHealth Alliance health information exchange; DHEC, South Carolina Department of Health and Environmental Control; ED, emergency department; EKG, electrocardiogram; OACIS, Online Application and Classification Information System.

^aEpic Systems, Verona, Wisconsin.

^bA lower score that signifies an agreement with the statement. All hospital names were removed from all of the comments.

Table 2.

Respondent comments on overall functionality and quality of CeHA

Respondent no.	Comments
Positive comments about CeHA
A1	“I like it a lot. Could be made better, but it’s pretty good now. Saves from repeating studies and can get very useful records especially for complicated patients.”
A11	“I am new to it, but it is a great resource.”
A15	“Works well. Links frequently nonfunctional.”
A2	“Works great! Very helpful but not all hospitals have information up in timely manner.”
A20	“Very good.”
A21	“Think it’s needed and certainly hope it will continue.”
A22	“A wonderful resource!”
A23	“Great overall. Very helpful.”
A25	“Great resource, but still needs improvements.”
A29, A40	“Excellent.”
A31	“Great.”
A4, A33	“Good.”
A34	“Good overall system. Some charts seem incomplete, but most of the time, I find what I am looking for.”
A35	“Love it.”
A36	“Excellent overall and almost always helpful.”
A43	“Excellent—generally best when expecting to find information (patient states had visited ED or was hospitalized).”
A44	“The CeHA system is a great asset to ED providers in being able to provide quality care to our patients. The functionality seems to vary highly, but has improved over the last year.”
A47	“Not bad, not great.”
Benefits of CeHA
B5	“A single login at the beginning of my shift gives me access for an entire day’s worth of patients.”
B6	“Quality is improving, and the answer to a clinical question is present in the CeHA about 50% of the time.”
C12	“Recent CeHA updates have continued to improve functionality and quality.”
Issues with CeHA
C13	“Works well. Occasionally doesn’t have all the information I want. Also would be nice if it made it more clear which facility the patient was seen in. For example, a hospital where the record is from may require you to read closely in the summary to figure which of their facilities patient was in.”
C24	“Could be easier to use, and lots of times I find the information is not updated frequently.”
C3	“Wish the Web site was easier to use and information more easily obtainable.”
C48	“Overall good functionality; having not received any orientation on how to use it, I found it relatively user-friendly.”
C7	“When it has information, it helps, especially in complex patients that bounce back and forth between hospitals.”
Proposed improvements
D14	“Needs some work and won’t reach its potential until it is embedded into the EMR.”
D16	“Think it’s a great resource, but would be nice to be able to print lab results.”
D18	“Very limited in its ability to provide recent notes from other EDs. Discharge instructions are usually present, but instructions do not reflect the content of the case.”
D19	“Need to make it easier to use and would be best if it was integrated into medical record.”
D27	“When fully complete will be invaluable.”
D28	“The information is invaluable. The design/interface are poor and represent the greatest hurdle to using CeHA.”
D30	“Quality is decent, could provide even more information to be more helpful. Functionality is fine, would like to be able to print findings!”
D32	“Better if built into Epic.”
D37	“Pretty good, but lacking in many areas.”
D38	“Very nice to have, but could be improved. Would also be very helpful to include records from other hospitals.”
D41	“ED records need to be available. Old EKGs need to be available. This is a very useful tool that needs to be supported and made more robust.”
D45	“Has potential but very buggy. Wish it were more consistent in terms of availability of information.”
D46	“When I am able to find information, I usually find it extremely helpful and useful. There are often times when I know that a patient has had recent studies, tests, or ER visits, but the information is not present in CeHA”
D8	“I think having it integrated in to our EMR would be a big help! One less password and log on to remember!”
D9	“It works well, but would be better if embedded in Epic.”
Response of none
E10, E17, E39, E42	None
No response
F26

All hospital names were removed from all of the comments.

CeHA, Carolina eHealth Alliance health information exchange; ED, emergency department; EKG, electrocardiogram; EMR, electronic medical record.

Experience with the CeHA-HIE

A total of 51 ED clinicians answered the item about having used the CeHA-HIE. Almost all respondents, 94.1% (n = 48), reported having used CeHA, with 5.9% (n = 3) answering that they had not used it. ED clinicians answering “no” to this item were queried about their reasons for not using it. One respondent chose all of six reasons listed for not using the system (Appendix), another indicated not having a user name or login for CeHA, and another reported being retired from clinical practice.

Usability

Respondents were in strong agreement regarding usability items 4, 6, 9 through 11, and 24. They indicated that they would like to use CeHA for every patient they saw, that CeHA was easy to use, that they felt confident using CeHA and the information they found in it, that they did not need to learn a lot of things before they could begin using the CeHA, that they would prefer that CeHA be embedded in the EMR at their hospital, and that the information received from CeHA usually made the time it took to log on and look worth the effort. Respondents were in weak agreement with items 7, 8, and 14, indicating that they found CeHA cumbersome to use and that they were concerned about the integration of information from various hospital systems and about information being incomplete or missing.

Major themes identified in respondent comments included their preference that CeHA be embedded within the EMR at each healthcare system so that a separate login would not be required (n = 5) and that each participating healthcare system include more detailed information such as ED clinician notes, full patient charts, images of electrocardiograms and x-rays, discharge summaries, and all test results (n = 3). Respondents acknowledged that accessing CeHA may not be necessary for all patients, especially those presenting with minor complaints or those who were new to the area (n = 3). Responses mentioned only once recommended adding more hospitals to the CeHA and performing more frequent uploading of information.

Functionality

Responses to functionality items 5, 17, 20, 21, and 23 indicated strong agreement among users that if CeHA provided good information and was easy to use, they would use it for every patient they saw, that the information about their patients in CeHA usually was sufficient to be useful clinically, that CeHA was a valuable addition to their practice and improved their efficiency, that CeHA improved the quality of care they delivered, and that they would prefer that CeHA be embedded in the EMR at their hospital. Items 12, 13, 15, and 16 showed weak agreement about whether information on their patients was present most of the time when they queried CeHA, whether they found the information they were looking for, and whether charts or discharge summaries from recent visits to other EDs usually were available in CeHA.

Major themes identified in respondent comments were consistent with those mentioned in the usability portion of the survey. Respondents tended not to use CeHA for every patient, but rather for those who had a recent, local hospital admission or ED visit, those who presented with a need for further treatment or with complications from prior treatment, or those who needed testing that may prove to be repetitive (n = 5). Criticisms and recommendations regarding the HIE included not having all of the relevant data from other healthcare systems (n = 3) and expressing a preference for embedding CeHA in the EMR along with a prompt or reminder to use it (n = 2).

Patient and Clinical Issues and Overall Assessment

Respondents indicated strong agreement with items that may prompt their use of CeHA, in particular its use for patients with more complex diagnoses or for patients believed to be seeking narcotics for inappropriate reasons. They also were in agreement that their anticipation of information about their patients being present in CeHA prompted its use. Given that there were only three comments for this set of items, we report no major themes.

Responses to the last survey item (“What is your opinion of the functionality and quality of CeHA?”) were grouped into six categories: positive comments about CeHA (n = 20), benefits of CeHA (n = 2), issues with CeHA (n = 6), proposed improvements (n = 15), response of none (n = 4), or no response (n = 1). Respondent comments were consistent with levels of agreement on specific survey items and in general were positive regarding the availability of CeHA and its value, its potential to improve care and efficiency, and its ease of use (Table 2). Needs for improvement also were consistent across the survey, including calls for changes in functionality by embedding CeHA in the EMR, creating a single logon, and making system improvements to ensure timely availability of charts and discharge summaries from other systems for recent ED visits.

Discussion

Previous studies reported higher rates of HIE use when caring for patients who made repeated visits, patients with comorbidities, patients known to have data in the exchange, and at sites providing HIE access to nurses as well as physicians.¹⁰ We found that factors perceived to make HIE consultation more valuable were clinical presentations indicative of complex disease, suspicion of prescription drug seeking, perception of the patient as a frequent ED user, and ED clinician anticipation that information would be present in the HIE.

ED clinicians perceived CeHA to be an asset to the practice of emergency medicine. The majority of providers strongly agreed that CeHA was a valuable addition to their practice and that it improved their efficiency as well as the quality of care they delivered. Respondents rated CeHA as easy to use and likely to contain good clinically useful information. They strongly agreed the information they obtained made the time taken to log on and look was worthwhile and they would like to access the HIE for every patient they saw with only a few exceptions (ie, patients with minor complaints and those new to the area).

The learning curve for the clinician interface was good; users found the system easy to use without orientation or engagement of information technology personnel. Software for CeHA remained a problem, with there being lesser agreement among respondents about items that assessed the integration of information from different hospital systems and items addressing incomplete or missing information. Charts and discharge summaries from recent visits to other EDs in the system were not perceived to be available on a timely basis. We expect these issues to resolve as compatibility between our participating hospital EMR systems and our HIE is improved. There was strong agreement that the HIE could be improved by embedding it in the EMR of participating hospital systems along with having a single log on. We concur that these features are necessary to improve HIE adoption and use, especially because the need for a separate login simply to determine whether information is present is a major disincentive to a busy clinician.

Online survey advantages included low cost, anonymous and automated respondent input, automatic data capture and storage, and use of site champion e-mail at each healthcare system to prompt participation in the survey. Online administration was convenient for busy ED clinicians and minimized time to send reminders.

The limitations of our study included the low sample size and the low overall response rate. Because our HIE was relatively new at the time of this survey, not all ED clinicians may have been aware of the CeHA-HIE and therefore were less likely to participate in the survey. Selection bias was caused by the inclusion of only those ED clinicians who volunteered to complete a survey and from their subjectivity in deciding about the utility of the HIE. Observer bias was inherent because our design focused on ED clinician opinions about the positive and negative aspects of using the CeHA-HIE. Survey responses were not adjusted for the amount of experience that ED clinicians had with the HIE, the number of years they had been in practice, or other user characteristics. We chose to recruit participants using e-mails from site champions. This likely resulted in most ED clinicians completing the survey when they received the e-mail, which was independent from the last time they had used the HIE, thereby resulting in recall bias. Our adaptation of SUS questions, use of a VAS rather than the SUS Likert scale, and the scoring of individual responses limit comparison of our results with other HIE evaluations using SUS.

Conclusions

We found that although ED clinicians were in strong agreement about the usability and functionality of CeHA, they also indicated the need for improvements in the timely availability of information and in some functional characteristics. We found variability among ED clinicians in their approach to the HIE and their assessment of which clinical circumstances made HIE more or less valuable at the point of care. We discovered operational issues with HIE that impeded its use. There are still software issues that limit the effectiveness of CeHA; however, we are hopeful there will be reductions in these barriers as systems are improved and technology matures.

Key Points.

• Health information exchange (HIE) adoption and use remain low despite its potential value for patient care and cost reduction.

• We surveyed emergency department clinicians accessing our regional HIE to understand how we might best improve its adoption, use, and functionality.

• Most emergency department clinicians believed that our HIE was easy to use and added value to their work, but they also desired better integration of the information available from the HIE into their system’s electronic medical record.