Workarounds and Test Results Follow-up in Electronic Health Record-Based Primary Care

Summary

Background

Electronic health records (EHRs) have potential to facilitate reliable communication and follow-up of test results. However, limitations in EHR functionality remain, leading practitioners to use workarounds while managing test results. Workarounds can lead to patient safety concerns and signify indications as to how to build better EHR systems that meet provider needs.

Objective

To understand why primary care practitioners (PCPs) use workarounds to manage test results by analyzing data from a previously conducted national cross-sectional survey on test result management.

Methods

We conducted a secondary data analysis of quantitative and qualitative data from a national survey of PCPs practicing in the Department of Veterans Affairs (VA) and explored the use of workarounds in test results management. We used multivariate logistic regression analysis to examine the association between key sociotechnical factors that could affect test results follow-up (e.g., both technology-related and those unrelated to technology, such as organizational support for patient notification) and workaround use. We conducted a qualitative content analysis of free text survey data to examine reasons for use of workarounds.

Results

Of 2554 survey respondents, 1104 (43%) reported using workarounds related to test results management. Of these 1028 (93%) described the type of workaround they were using; 719 (70%) reported paper-based methods, while 230 (22%) used a combination of paper- and computer-based workarounds. Primary care practitioners who self-reported limited administrative support to help them notify patients of test results or described an instance where they personally (or a colleague) missed results, were more likely to use workarounds (p=0.02 and p=0.001, respectively). Qualitative analysis identified three main reasons for workaround use: 1) as a memory aid, 2) for improved efficiency and 3) for facilitating internal and external care coordination.

Conclusion

Workarounds to manage EHR-based test results are common, and their use results from unmet provider information management needs. Future EHRs and the respective work systems around them need to evolve to meet these needs.

1. Background

Electronic health records (EHRs) facilitate reliable communication of test results to clinicians [1]. However, communication regarding test results involves more than just information transfer; it also involves action upon the transmitted information. Previous research has shown that workarounds, such as paper reminders, manual overrides, and digital files outside the EHR, are commonly used by clinicians to manage important information [2–4]. Many of these actions would not have been expected or intended by EHR designers. Workarounds are defined as use of technology in ways for which it was not designed, avoiding its use and relying on an alternate means of accomplishing work [5] or developing an interim solution when a required feature or function does not exist [6]. These informally designed work processes may resolve immediate demands and crises and compensate for poorly designed systems, but may also introduce inaccuracies and serve as poor information tracking mechanisms [7, 8].

Follow-up of abnormal test results is a high-risk clinical process, and its failure can lead to substantial patient safety implications and care delays [9]. Most EHRs transmit important patient-related information, such as abnormal laboratory test results, as electronic notifications to clinicians (called “View Alerts” in this system, and sometimes just “alerts”). Despite EHR-based delivery of abnormal test results, failure to follow-up on test results is common [10]. A study of test result management systems in 4 family medicine offices found that 28% to 55% of abnormal tests did not have documented follow-up [11]. In our previous work we found that almost 8% of abnormal imaging results and 7% of laboratory test notifications sent to providers through EHRs lacked timely follow–up [9, 12]. To improve timely follow-up of abnormal tests, it is important to examine the challenges that EHR-based test result management poses to providers’ workflows and the way providers adapt their work practices to overcome such challenges. Our previous EHR-based test result management-related qualitative work at the Department of Veterans Affairs (VA) suggested that providers often resort to workarounds, including use of paper to follow-up on notifications [2]. For instance, we found that the EHR system did not have a convenient self-reminder feature, prompting providers to use handwritten notes as prospective reminders to ensure that follow-up of test results was completed in a timely manner. Because workarounds are often used to “fix” evident or perceived system inefficiencies [13-15], this knowledge can help build safer EHR-based work systems that meet provider needs. Although several studies have documented workarounds in computerized documentation of medication administration [16–18] and electronic consult-management [19], no previous studies have specifically focused on understanding frequency and characteristics of workarounds in management of test results. This gap in understanding of workarounds related to test results management processes calls for an in-depth evaluation of strategies used by providers to ensure timely follow-up of abnormal test results.

2. Objective

Our study objective was to understand why primary care practitioners (PCPs) use workarounds to manage test results. We analyzed data from a previously conducted, national cross-sectional survey on test result management to identify the types of workarounds used and then examined the association of key sociotechnical factors that could affect test results follow-up (e.g., both technology-related and those unrelated to technology, such as organizational support for patient notification) and workaround use.

3. Methods

3.1 Study Design and Survey/Participants

We conducted a secondary analysis of quantitative and qualitative data from a national, cross-sectional, web-based survey of all Department of Veterans Affairs (VA) PCPs (N = 5290). All VA facilities use Computerized Patient Record System (CPRS), an integrated EHR, which enables communication of test results to PCPs. Most routine and abnormal laboratory and imaging test results are delivered to PCPs through a notification system in the EHR known as the “View Alert” system (► Figure 1). The original survey data were collected between June and November 2010, with a response rate of 51.8%. Details of the 105-item survey development and data collection are published elsewhere [20]. Survey development was guided by an 8-dimension sociotechnical model. Several social (e.g., organizational policies and procedures, workflow and people) and technical (e.g., EHR notification software, its ease of use and EHR user interface) aspects of EHR-related test result notifications were assessed, including 5 questions (► Appendix I) on PCPs use of alternate methods to follow-up on test results. Respondents were 51.9% female, 28.5% non-white, and 29.6% non-physician providers (e.g., physician assistants, nurse practitioners); and 75.4% had 2 or more years of practice in the Veterans Health Administration (► Appendix II). We conducted a secondary analysis, focusing on these 5 questions, to examine factors associated with use of workarounds. The primary study and secondary analysis were approved by the local institutional review board.

Fig. 1.

Alert notification window in Computerized Patient Record System

Appendix I.

Workarounds Related Survey Items

Survey Items

I use the VistA -DHCP system to process alert notifications: For some notifications For all notifications I do not use VistA (DHCP), I only use CPRS to process notifications Processing alert notifications in VistA-DHCP is better than CPRS because it prevents alerts from disappearing. Please list any other reasons why you continue to use VistA-DHCP to process some alert notifications: In addition to CPRS and/or VistA-DHCP, I am using additional methods to help me follow my test results (e.g., paper-based methods). What methods, other than CPRS and/or VistA-DHCP, do you use to help follow your test results?

Appendix II.

Characteristics of survey respondents (n=2590)

Characteristic	n (%)
Age    20–39    40–49    50–59    60 and over    Missing	338 (13.1)    685 (26.4)    961 (37.1)    402 (15.5)    204 (7.9)
Gender    Male    Female    Missing	1080 (41.7)    1343 (51.9)    167 (6.4)
Race    White    Black    Asian    Other    Missing	1630 (62.9)    118 (4.6)    431 (16.6)    188 (7.3)    223 (8.6)
Job classification    Physician, academic    Physician, non-academic    Nurse practitioner    Physician assistant    Missing	438 (16.9)    1228 (47.4)    561 (21.7)    204 (7.9)    159 (6.1)
Years at VA    <2    2–10    11–20    >20    Missing	437 (16.9)    1219 (47.1)    589 (22.7)    2144 (5.6)    01 (7.8)

3.2 Analyses

Quantitative Analysis. Frequencies for types of workarounds were calculated based on responses to the following survey question – “What methods, other than the EHR (i.e., CPRS and/or VistADHCP [Decentralized Hospital Computer Program]), do you use to help follow your test results?” Two researchers, a sociologist (SM) and an internal medicine physician (DRM), performed a content analysis [21] to categorize workarounds used by providers to follow-up on test results. To examine factors associated with use of workarounds, we conducted a multivariate logistic regression analysis. We used the 5-point Likert-type scale with response options ranging from “strongly disagree” to “strongly agree.” Survey item, “In addition to CPRS and/or VistA-DHCP, I am using additional methods to help me follow my test results (e.g., paper-based methods),” was used to define the primary outcome variable: use of workarounds. Scale items 1, 2 and 3 were combined to represent nonusers of alternate methods (Disagree), and 4 and 5 were combined to represent users (Agree). The predictor variables were derived from answers to survey items related to provider perceptions about missed test results, workload, volume of notifications, organizational support to facilitate patient notification and perceived effectiveness of notification system (► Appendix III). We used descriptive statistics to summarize the types of workarounds used by providers.

Appendix III.

Survey items loaded to each factor

Items that cluster on the same factor	Survey items
Factor 1 Perceived Alert Noise or Overload	Noise (Overload): Too Many (Q: I receive too many alert notifications per day.) Noise (Overload): Too Many FYI (Q: I get too many FYI (For Your Information) only alert notifications that require my signature, even though no action on my part is required.) Noise (Overload): Too Many Unnecessary (Q: I often receive alert notifications where I am unsure as to why they were sent to me.) Alert Fatigue: More Than Manageable (Q: The number of alert notifications I receive on any given day exceeds what I can effectively manage.) LOG Transformation of Signal to Noise (% of alerts necessary for quality care) Noise (Overload): Too Many Duplicative (Q: I receive too many lab alert notifications that are duplicative, i.e. pertain to the same panel.)
Factor 2 Perceived Effectiveness of Alert System	Performance Expectancy: Alerts Increase My Productivity (Q: Using alert notifications in CPRS increases my productivity.) Q: My colleagues believe the alert notifications in CPRS help them get their job done effectively.) Performance Expectancy: Alerts Enhance My Effectiveness (Q: Using alert notifications in CPRS enhances my effectiveness on the job.) Ease of Use (Q: I find the alert notification system in CPRS easy to use.)
Factor 3 Patient Notification Support”	Patient Notification Support: Have Help (Q: I have the help I need for notifying patients of test results.) Patient Notification Support: Have Staff (Q: I have support staff to assist with management of test result alert notifications (i.e. acting upon and following-up) Patient Notification Support: CPRS Has Features (Q: CPRS has convenient features for notifying patients of test results.)
Factor 4 Represents “Experience”	Years as PCP at VA Age Ever Used EMR Other Than CPRS (Q: Have you ever used an electronic medical record system other than CPRS?)
Factor 5 Outcomes of Patient Safety	Outcomes of Patient Safety: I Missed Alerts Leading to Poor Patient Care (Q: In the past year, I missed abnormal lab or imaging test results that led to delayed patient care.) Outcomes of Patient Safety: Colleagues Missed Alerts Leading to Poor Patient Care (Q: In the past year, colleagues I work with missed abnormal lab or imaging test result alert notifications that led to delayed patient care.
Factor 6 Volume of Alerts	LOG Transformation of Time spent on alerts (Hours per week spent managing alerts) LOG Transformation of volume (Number of alerts received per day) % of abnormal imaging results lacked follow-up within 14 days.

On the basis of literature review and discussions with a multidisciplinary research team and input from EHR subject-matter experts, we chose 31 items from the original survey potentially related to use of workarounds. We also included key provider demographic variables, such as age and years of experience. Factor Analysis was used to reduce these 31 items into a smaller set of dimensions. A principal axis factor analysis was conducted on the 31 items with direct oblimin rotation (since factors are likely to be correlated) [22].

Qualitative Analysis. To gain a better understanding of providers’ motivation for relying on alternative methods to follow-up test results, we used free-text survey data related to the following question- “In addition to CPRS and/or VistA-DHCP I am using additional methods to help me follow my test results (e.g., paper-based methods).” Two investigators (SM and DRM) reviewed the responses to identify quotations with stated reasons for use of workarounds.

We used two approaches to analyze these quotations. First we used an emergent coding approach to code the data inductively. Second, we used an a priori coding scheme to categorize stated reasons for use of workarounds. By using an a priori coding approach, we were able to corroborate our findings with a pre-existing framework for use of workarounds [3]. An emergent coding approach allowed us to account for new themes pertinent to the test result management process. Methods of qualitative content analysis [23] were used to analyze these data. We read the statements line-by-line and used an open-coding method to assign initial codes. Two investigators, (SM and DRM) independently reviewed the responses to familiarize themselves with the range of reasons stated by the providers. The investigators met regularly to discuss the emerging codes. This process ensured that the coders were challenged and were able to gain awareness of their biases during the important phase of code development. On the basis of these discussions, we developed a codebook; and both coders used the codebook to independently code stated reasons for use of workarounds. To establish inter-rater reliability, we merged the coded data set to identify discrepancies in coding. We resolved inter-coder discrepancies (<10% of codes) through discussion until consensus was reached. During the next step of data synthesis, codes with similar content were grouped into higher-level categories and given more abstract labels. In the following step, we examined relationships between these higher-level categories to identify themes. The investigators (SM and DRM) met regularly to discuss thematic organization. The emerging themes were then discussed with the study team. Additionally, we categorized stated reasons for use of workarounds by applying a framework of 11 reasons for use of paper- and computer-based workarounds [3]. One investigator (SM) indexed each stated reason to this framework. Another member of the research team (DFS) validated accuracy of the indexed codes by reviewing a sample (10%) of randomly selected quotations with codes mapped to the framework of Saleem et. al [3]. Some quotations provided more than one reason for use of workarounds. In such cases, more than one code was assigned. We performed all coding and qualitative data management in ATLAS.ti 6.2 (Atlas.ti Scientific Software Development GmbH, Berlin, Germany).

4. Results

4.1 Types of Workarounds

A total of 2554 (51%) PCPs answered the survey question regarding use of alternate methods. Of these, 1104 (43%) respondents reported using workarounds outside the traditional EHR-based workflow to follow-up on test results. Of these 1104 respondents, 1028 (93%) provided information about the type of workaround they were using; and 298 (27%) also provided their reasons for using them (► Appendix IV). Most respondents (70%) indicated that they use some form of paper method (► Figure 2), which included paper lists (22%) and sticky notes (27%); printouts/faxes (14%); and writing notes on clinical artifacts, such as patient routing sheets (7%). Over 22% of respondents used a combination of paper and electronic methods, including electronic calendars (11%) and digital tickler files (2%), either to follow-up on test results or to generate reminders for future follow-up action by sending reminder emails to themselves (9%).

Appendix IV.

Workarounds Related Survey Items – Number of responses

Fig. 2.

Workarounds used by primary care providers to follow-up on test results

4.2 Factors Related to Use of Workarounds

A principal axis factor analysis with direct oblimin rotation of 31 survey questions from the PCP survey was conducted on data gathered from 2554 PCPs. An initial analysis was run to obtain eigenvalues for each factor in the data. An examination of the Kaiser-Meyer-Olkin measure verified the sampling adequacy for the analysis and suggested that the sample was factorable (Kaiser-Meyer-Olkin = .72). Ten factors had eigenvalues over Kaiser’s criterion of one, and, in combination, explained over 31% of the variance. The scree plot showed inflexions that would justify retaining 6 factors that each cluster around a single dimension (explaining 25% of the variance). The Factor Analysis was then rerun, creating “scores” for each of the 6 factors. These 6 factors (► Table 1) were used in the subsequent multivariate logistic regression with use of workarounds as the outcome (► Appendix I for survey items loaded into each factor).

Table 1.

Description of factors and number of survey items per factor

Factor	Description of what each factor/dimension represents	# of Survey items loaded into factor*
1	Perceived alert noise or overload (too many unnecessary alerts)	6
2	Perceived effectiveness of alert systems (alert system increases productivity/ effectiveness of system)	4
3	Patient notification support (have staff help/ helpful system support)	3
4	Experience (demographic characteristics)	3
5	Outcomes of Patient Safety (missing alerts leading to poor patient care/ witnessing others missing alerts)	2
6	Volume of Alerts (number of alerts received per day)	3

*Note: see Appendix I for survey items loaded into each factor.

In the multivariate analysis we found that PCPs who reported having limited patient-notification support, such as protected administrative time and help from other staff members to notify patients about test results, were more likely to use workarounds (p=0.02). Additionally, PCPs who had either personally missed test results or reported awareness of a colleague missing results previously were more likely to use workarounds (p=0.001). Our analysis revealed no significant association between “Perceived alert overload” and workaround use (p=0.70). Furthermore, neither volume of notifications received nor years of expertise with the EHR system was associated with workaround use (► Table 2).

Table 2.

Relationship between factors obtained based on factor analysis of survey items and workaround use using multivariate logistic regression

Variable	Coefficient (β)	Standard Error	Wald X2	P value	Odds Ratio	95% CI
Perceived Alert Noise or Overload	-.015	.040	.146	.702	.985	.912–1.064
Perceived effectiveness of alert system	.012	.038	.100	.751	1.012	.939–1.091
Patient Notification Support	.101	.042	5.724	.017	1.107	1.018–1.203
Experience	.000	.035	.000	.999	1.000	.933–1.072
Outcomes of Patient Safety	-.171	.038	19.898	.001	.843	.782-.909
Volume of Alerts	.160	.106	2.261	.053	1.173	.953–1.444

4.3 Qualitative Results

Analysis of stated reasons for use of workarounds revealed that PCPs use various paper and electronic methods to overcome perceived system inadequacies. We identified three main themes to explain the use of workarounds: 1) as a memory aid to the PCPs, 2) for improved efficiency, and 3) for facilitating internal and external care coordination. The main themes and sub-themes are presented in ► Table 3. We used a coding framework of 11 categories of reasons for use of paper-based workarounds [3] and found 9 of these categories mapped to our 2 themes: 1) memory aid and 2) improved efficiency. ► Table 4 presents the main themes, categories of reasons for use of workarounds (based on the framework of Saleem et al.); and their frequencies and examples.

Table 3.

Reasons for use of workarounds – Overview of themes and sub-themes based on qualitative content analysis

Main Themes	Sub-themes
Memory Aid	Workarounds used as prospective memory tool Workarounds used as work tracking method
Improved Efficiency	Workarounds used to improve efficiency Workarounds used for better data organization
Facilitate Internal and External Care Coordination	Workarounds used due to external workflow Workarounds used to facilitate team-based care delivery

Table 4.

Reasons for use of workarounds to follow-up on diagnostic test results: Main themes and categorization

Main Themes	Categories*1 (Saleem et al.)	Freq.	Saleem et al.’s Description	Examples
Memory Aid	Memory	62	Reminder about “old” or existing information.	I keep a notepad to remind myself if a patient had a test done or followed up on treatment recommendations.
	Awareness	26	Recognize new/important information: notify, alert, trigger; adjusting “signal to noise” ratio.	Nurses at my clinic and the other clinics make a list of patients that have had INR done for that particular day and give that to me to remind me to take action the next day.
Improved efficiency	Efficiency	68	Using a workflow process that improves actual or perceived efficiency.	There are quite a few other things such as coding and identifying to the computer multiple times who I am when I’m obviously signed in, that are time consuming, tedious, detract my attention and inefficient. Writing the info on paper as I am informed of completed tests yet no results are available for viewing until days later--no other way of tracking (a potential safety issue)
	Data organization	28	Data layout issues; need to view existing data differently, accessing data.	When I need to follow up on an alert that I have already reviewed (once reviewed you cannot go back) such as test result that was scanned into vista imaging but it is not visible yet, I print out the alert.
	Knowledge/ skill/ease of use	3	Training/support/experience/ease of finding needed information.	I keep a notebook of specific things that I need to follow up on that I cannot easily tag in CPRS.
	Task specificity	3	Need specificity or ability to customize to patient, provider, department, etc.; some signal/ noise issues.	I cannot request a subset of labs, say for testosterone and urinalysis, without getting ALL the labs (which I don’t need), so I have to follow some patients on paper.
	Task complexity	5	Complexity of task dictates workflow issues or functionality issues.	I write notes on routing slips or scrap sheets of paper. I have a notebook that I write some things down on that require multiple follow ups or coordination of multiple services.
	Sensory-motor preferences	29	Preferred sensory input for task: “hear,” “tangible,” easily modified (i.e., hand notes); mobility, something to “deliver.”	I HATE lab alerts on CPRS unless they are panic values. I try to keep alerts to one screen at all times so I can see new ones as they pop up. Lab alerts TOTALLY clog this up. I prefer printed copies on paper which I can look at, organize and prioritize on my own. I really want to continue this.
	Longitudinal data processes	1	Task requires processing multiple data points across time.	Paper Coumadin flowsheet for each of my Coumadin pts – I need to see the past trends, dose/ INR correlations, as I am deciding on management.
Facilitate Internal and external workflow*2	N/A	61	N/A	I receive paper copies of critical labs and tests sent outside the VA system.

*¹ The categorization presented in the table is based on applying framework of 11 reasons for use of paper- and computer-based workarounds developed by Saleem et al. [3]

*²“Facilitate Internal and external workflow” – this theme does not map to Saleem’s framework.

4.3.1 Workarounds Used as Memory Aid

We found that various paper-based workarounds were most commonly used as prospective memory tools to remind providers to take follow-up actions on important test results. As one PCP stated, “I keep a list of labs or tests I need to follow-up on certain patients, if they are important I want to double-check them.” Paper-based workarounds were also used to make PCPs aware of new information. For example, staff members highlighted critical information (abnormal test results) and left the printouts on the PCP’s desk for review. Additionally, PCPs reported using a number of electronic tools and strategies, such as Microsoft Outlook (Redmond, WA) calendar appointment reminders, a digital Microsoft Word or Notepad file, or leaving a progress note unsigned so that it is indefinitely visible in their notification window. In addition to serving as prospective memory tools to take future follow-up actions, these tools served as a task list to organize their work for the day or week and ensure that all necessary actions were completed.

Several factors appeared to contribute to the need for prospective memory tools. PCPs expressed concern about notifications “disappearing” from the EHR-based notification system, as well as certain abnormal results being lost (Pap smear and pathology biopsy results). Fear of losing information motivated some PCPs to use paper and electronic workarounds as a secondary safeguard to ensure that important information necessary for follow-up was available.

“Once I click on an alert, it goes away. I need a way to ‘remind me’ of things I need to follow-up on since clicking on an alert doesn’t mean I can take care of it immediately, or am waiting for more information, etc.”

In the current system, once a certain type of notification was acknowledged, it was removed from the PCP’s inbox. Because the system did not have features that would allow providers to “go back” to retrieve an acknowledged notification, some PCPs relied on workarounds. In some cases, paper copies of notifications served as a memory aid and allowed PCPs to go back to the information whenever they needed it.

“I use lists to keep track of pap smears and pathology biopsy results because, historically, these got ’lost’ and I was never alerted. I track pertinent scans, consults, and testing for some patients due to appointment cancellations that are not rescheduled by the consultant/radiology.”

Memory aids were also used to cue PCPs that certain information should be tracked. For example, PCPs had to keep track of a patient who failed to complete diagnostic testing. Additionally, they were responsible for follow-up actions in the distant future after the initial notification that required their immediate attention. To ease cognitive overload resulting from tracking numerous notifications with various follow-up time frames, some PCPs used workarounds as memory aids.

“Sometimes I will leave a future paper note in my calendar if a patient needs important follow-up in the future, say in 1–2 month, for an abnormal radiology study.”

“Reminder notebook or pad for long-term follow-up. Post-it notes for short-term follow-up to remind me to look later for result.”

4.3.2 Workarounds for Improved Efficiency

Poor data organization emerged as an important reason for use of workarounds. PCPs were overwhelmed with information, as expressed in the quote: “I have too many alerts and can’t stay overtime to process them.” To process notification-related information under considerable time pressure, PCPs reported needing well-integrated system features to improve efficiency, such as the ability to manage notifications according to their level of urgency. However, in the EHR-based notification system used at VA facilities, clinically relevant information (e.g., critically abnormal test results) was often received together with messages containing non-urgent and redundant test results and other information, requiring PCPs to manually sift through all notifications on a crowded computer screen to locate the urgent and actionable information.

“The EHR creates a deluge of notifications, alerts, follow-ups, and tele-care and nursing notes. Because of this deluge, the relevant information I need to follow-up on is buried. I use my patient print out from daily visits to notate what I need to check.”

“Paper: I end up printing labs sometimes when I have so many view alerts so I can clear the computer screen from all of the meaningless alerts.”

To overcome such challenges, PCPs reported developing workarounds, such as printing important notifications to clear the screen, stating that printed copies of notification allow a more efficient review than locating the same information in the electronic system. Similarly when PCPs did not have time to process all notifications, printing them to highlight abnormal results and creating paper logs/lists to place high-priority notifications on the top of the list helped them manage them more efficiently.

“I prefer printed copies on paper, which I organize and prioritize on my own. I really want to continue this.”

“Have all lab test results printed as hard copy for review, as I find getting lab results on CPRS overloads the system.”

“I HATE lab alerts on the EHR unless they are panic values. I try to keep alerts to one screen at all times so I can see new ones as they pop up. Lab alerts TOTALLY clog this up.”

4.3.3 Use of Paper to Facilitate Internal and External Care Coordination

PCPs reported using paper-based workarounds due to pre-existing workflows for which there was no EHR-based method. To avoid lengthy travel to VA hospitals, patients often used local (non-VA) resources for diagnostic tests and other services. This external workflow generated numerous paper records of imaging and lab results and physical therapy notes, as well as all notes from non-VA home health agencies. Although such “dual-utilization” (VA and private laboratories) was part of an established routine, electronic exchange of data was not possible. To share findings of tests done outside VA facilities, test results and notes were faxed to the ordering provider. Additionally, patients routinely received paper copies of the tests done outside VA to share with their PCPs, for which the only reasonable method of recording these data into the EHR was to have them scanned. Nonetheless, PCPs continued to use such “unavoidable” [6] workarounds to facilitate pre-existing external workflow.

“Non-VA sources of paper results, including imaging and lab, physical therapy results and non-VA home agencies that are reviewed and signed off by me. Labs from non-VA physician, all notes from home health nurses, outside providers, including imaging and what not. If I ask for a lab to be done at a local (distant) hospital using private insurance, then I get results via fax.”

“I use paper. Many tests are done locally, as I work in a VA clinic and patients do not like the long drive to the main VA; they use private insurance for local studies. I also have many paper results to review, notify patients of, and then request scanning.”

Additionally, several paper-based workarounds were part of the daily routine. For example, at certain facilities, test results were configured to be sent both electronically to the PCPs, as well as to a printer in the clinic. Similarly, laboratory staff kept paper records of tests ordered for the day and passed that information to the PCPs. In many cases, PCPs’ responses implied that they were simply following previously established workflows, and that they had not put forth an effort or had an interest to change these routines.

5. Discussion

We conducted a secondary analysis of a national survey of PCPs and found that PCPs continued to use paper and electronic workarounds to support follow-up of test results in a fully integrated EHR-based test results communication system. We conducted a quantitative analysis to examine the association of key sociotechnical factors that could affect test results follow-up and use of workarounds. Qualitative analysis focused on understanding why providers use workarounds to follow-up on test results. We found that PCPs frequently transferred electronically communicated information regarding test results to paper printouts, sticky notes, paper lists, and logs, as well as employed use of digital equivalents, such as electronic calendar events, text files, and unsigned progress notes. Our findings indicate extensive use of paper to follow-up on test results, which is consistent with reports of “paper-persistence” in other aspects of the EHR, including Computerized Physician Order Entry (CPOE) [16, 24] and consult-management systems [19].

While it appears that PCPs use workarounds to improve efficiency, and they may seem necessary in some situations, using them as a routine practice can potentially introduce new risks to the test-result management process. Our analysis of stated reasons for use of workarounds revealed that PCPs developed alternate paper processes to manage test results to overcome suboptimal system features. For example, they developed a method to send “self-reminders” regarding upcoming follow-up tasks and future follow-up dates because the EHR did not have this functionality. Lack of such “self-reminder” features in the current system prompted some PCPs to rely on risky temporary solutions, such as leaving reminder messages for themselves on Post-it notes. All of these issues are useful for design of future EHRs.

Beyond volume of notifications, we found that the manner in which test results were delivered was a major source of dissatisfaction prompting workaround use. PCPs often worked under considerable time pressure and used workarounds to improve efficiency [25]. Previous research has shown that use of EHR technologies can alter pre-existing workflows and place increased demand on providers’ time. For example, Ash et al. found that implementation of CPOE resulted in more or new work for providers by forcing them to enter required information, respond to alerts, deal with multiple passwords, and expend extra time [24]. Another study of the effect of CPOE on physician time found that computer order entry takes physicians about twice as long as paper ordering [26] Highlighting the role of heavy workload and time pressure, Halbesleben et. al. [15] have noted that in busy health care settings, providers are more likely to use workarounds to overcome perceived barriers to their workflow.

We also found that not all workarounds were developed by users. Due to lack of electronic data transfer with external facilities, PCPs were compelled to use paper records of tests done outside VA facilities.

Lack of support for notifying patients of their test results was associated with use of workarounds. This is not surprising because PCPs are spending a substantial amount of time on non-face-to-face communication with patients. An analysis of EHR-based alert notifications received by PCPs at 10 VA clinics found that PCPs received a mean of 56.4 alerts with new clinical information each day and spent an estimated 50 minutes processing them [27]. Although the volume of electronic notifications received by providers has increased, there appears to be limited administrative support for managing them. Just over one third in our survey reported having administrative support for notifying patients of test results. PCPs use several strategies, such as staying after hours or coming in on weekends to address notifications [20]. It appears that use of workarounds, such as keeping a paper-based list of patient names to contact at a later time, is a strategy used by PCPs to manage their workload.

PCPs who report having a personal history of missed test results or having witnessed others missing alerts were more likely to use workarounds. It is likely that awareness regarding missed test results may prompt some PCPs to use paper and electronic-based workarounds as a secondary safeguard. Our qualitative analysis revealed that PCPs were concerned about alerts “disappearing” from the system, and used printed copies of abnormal test results to ensure that important information was not lost. Additionally, having personally missed a test result may heighten the need for alternate methods to keep track of alerts. For example, we found that PCPs used various paper-based methods as prospective memory aid. Ferneley and Sobreperez [28] have described such action as “Harmless Workarounds.” Such workarounds, used mainly to ensure that an essential task is completed, do not significantly affect the workflow or the accuracy of information. Nonetheless, use of workarounds creates a parallel “hybrid” paper and electronic system with numerous nonstandard workflows and follow-up practices that are difficult to track. Casalino et al. [7] have found that partial EHR systems (paper-based progress notes and electronic test results or vice versa) were associated with higher follow-up failure rates than not having EHRs. Furthermore, an analysis of malpractice claims with EHR-related factors found that use of hybrid health record was the second most common contributory factor [9]. Another study of over 3000 EMR-related safety incidents found that hybrid EMR-paper workflows were attributed to 6 of 31 types of incidents classified by the Pennsylvania Safety Authority [29]. Use of unsystematic practices [11] may lead to a higher risk of important information being missed. Furthermore, PCPs used handwritten notes and other parallel-system workarounds to manage test result notifications, risking reintroducing unsafe processes by means of the same shortcomings the EHR systems were meant to overcome.

The results of our study highlight several aspects of provider EHR use that lead to workarounds, and could, therefore, be targeted to reduce workaround use, improve standardization of workflow, and potentially improve safety related to test result follow-up. First, a commonly stated precipitating factor for workaround use is the lack of EHRs’ ability to facilitate creating a reminder for future follow-up action. Thus, the addition of robust EHR-based features that allow task tracking or self-reminders may significantly reduce reliance on workarounds. Second, PCP workload was a common theme in many responses, and several studies have documented their dissatisfaction with information overload and time burden [15, 24]. Thus, additional work is needed for system redesign to reduce transmission of non-actionable information, improving efficiency of test result management and the use of administrative support for notification processing.

Our study has several limitations. First, since it is based on a cross-sectional survey, only associations between variables, not causation, can be inferred from the data. Second, this study is focused on a single EHR used by providers in a single health care system. Because of differences in EHR functionalities and organizational policies regarding test result management, some findings reported here may not be generalizable to other EHRs or practices outside the VA system. Nonetheless, workarounds are common in all health care organizations; thus, understanding the types of workarounds and the factors associated with their prevalence can be of use to health care systems outside the VA. To analyze reasons for use of workarounds, we had to rely on free-text survey data with varying levels of detail provided by 27% of respondents. We could not confirm validity of the self-reported responses by asking any additional probing questions. Nonetheless, the free-text responses provided valuable information about strategies used by providers to overcome system deficiencies.

6. Conclusion

Our study identified the types of workarounds used by PCPs to follow-up test results notifications in EHRs. We found that paper-based workarounds are common. The main reasons for use of workarounds are to improve efficiency, trigger memory and improve care coordination. Future EHRs and their respective work systems around them will need to evolve to meet these needs.