Department-focused electronic health record thrive training

Abstract

A novel approach of department-focused electronic health record (EHR) training was implemented to improve efficiency and time management of EHR use. Based off baseline log data, 5 in-person training sessions were designed, focusing on the common inefficiencies of 6 chosen participants. Log data of 4 key metrics and 2 efficiency scores were analyzed 4 months post-training. A survey was conducted to assess self-reported EHR competence. Individually, several participants had improved efficiency scores. There was a reduced average time spent in the inbox per day, in notes per dictation, and in notes per day. This translated to an average of 8.9 min saved per day (range 0–29.1 min/day) and 37.1 hours saved per year (range 0–116.2 hours/year). From the post-training surveys, all participants felt more efficient in their use of the EHR. This study demonstrates an example of department-focused EHR training and log-based analysis improving time management and efficiency.

Lay Summary

This was a pilot study analyzing the use of log-based data to monitor for improvements in electronic health record (EHR) efficiency, after 6 participants completed hands-on, interactive training sessions. These training sessions focused on metrics that were commonly deficient amongst radiation oncology providers at a large academic institution. Four months after the training, a post-training survey demonstrated that all participants felt more efficient in their EHR use. Additionally there was a reduced average time spent on both communication and documentation metrics. This study demonstrates an example of department-focused EHR training and log-based analysis improving time management and efficiency.

INTRODUCTION

Adoption of the electronic health record (EHR) has never been higher throughout the country with 90% of hospitals and 80% of office-based clinics adopting the EHR by 2019.^1–3 The EHR provides many advantages, including increased patient record accessibility and efficiency, improved clinical workflow and communications, and improved quality and safety.^4–6

However, with increased use of the EHR, physicians spend one-fourth of their time in direct contact with patients and the remainder spent on “desktop medicine.”^7,8 This increase has led to approximately 70% of EHR users reporting health information technology (HIT)-related stress and 2.5 times the odds of burnout, leading to reduced quality of care and increased medical errors.⁹ Furthermore, burnout can lead to reductions in physician clinical work hours annually, costing healthcare systems $4.6 billion nationwide, or $7600 per physician.^9–12

Some of the most common contributors to HIT-related stress include lack of standardization, computerized order entry, inbox management, click burden, excessive data entry requirements, and perceived sense of inefficiency.^1,4,13 Several of these sources of EHR frustration have previously been explored, including mitigation studies focusing on reducing click burden, increasing standardization, hiring scribes, and improving EHR education.^1,11,14 In order to understand where inefficiencies lie in providers’ interactions with the EHR, a recent spotlight has been placed on analyzing EHR software log files. This data can serve as a benchmark for providers and health system leadership to compare metrics at individual, departmental, or institutional levels. This also allows for review of physician workloads, work flows, and areas for efficiency improvement.¹⁵

Studies have suggested that HIT-related stress analyses and specific solutions should be stratified by specialty, as EHR tasks and associated stressors can vary between medical specialties.⁹ This is certainly true within the specialty of radiation oncology, which relies heavily on multiple EHR systems for daily clinical tasks.¹⁶ A survey found that <30% of radiation oncologists were satisfied, to very satisfied with the EHR.¹¹ An ASTRO work force study in 2017 found that radiation oncologists spend approximately 48% of their time in direct patient contact, which decreased about 5% compared with a survey 3 years prior, while their other clinical work, mainly EHR management, increased by 5% to 40.9%.¹⁷

Objective

We focused on EHR use of physicians within radiation oncology at a large academic institution. We sought to determine the most common inefficiencies of providers by analyzing log data from Epic (Epic Systems Corporation). We then provided department-focused, hands-on training sessions on these inefficiencies. The goal was improving performance of our participants as measured by this same data.

MATERIALS AND METHODS

Baseline assessment

Signal is an Epic UserWeb program available to Epic customers that provides efficiency data of all ambulatory Epic users, allowing comparisons of multiple groups at the system, department, or individual provider level. Signal focuses on 4 key metrics: In Basket, Orders, Notes and Letters, and Clinical Review. Subcategories within each metric were compared within radiation oncology as a whole and with each provider. Two baseline scores were collected per provider. The first, PEP score, measures efficiency by looking at provider workload and compares that against system usage. The second, Proficiency score, measures individual’s personalization of tools to enhance efficiency (Table 1).

Table 1.

Signal metric definitions

Metric	Definition
PEP score	Score of a normal distribution average indicating the difference between time the provider spent in the system according to User Action Log data and expected time in the system based on the provider’s workload. A higher score means the provider spent less time in the system compared with the amount of work they completed in the system
Proficiency score	A composite score calculating how frequently the provider uses efficiency tools. A higher score indicates more use of efficiency tools
Time in In Basket per day	Average amount of time spent in In Basket per provider per day. Numerator: Total number of minutes spent per provider in an In Basket activity or navigator section within the reporting period. Denominator: Total number of days that the provider logged in and worked during the reporting period
Time in In Basket per appointment	Time spent in In Basket per scheduled appointment. To be included, providers need at least 5 appointments scheduled per week in the reporting period. Numerator: Total minutes providers spent in an In Basket activity or navigator section within the reporting period. Denominator: Number of appointments during the reporting period
Time in notes per notes	Average time spent writing a note for each note written per provider. Numerator: Total number of minutes spent in a notes activity or navigator section per provider within the reporting period. Denominator: Number of notes written during the reporting period
Time in notes per day	Average amount of time each provider spends in notes each day. Numerator: Total number of minutes spent per provider in a notes activity or navigator section within the reporting period. Denominator: Total number of days that the provider logged in and worked during a reporting period
Time in notes per appointment	Time spent writing notes per appointment. To be included, providers need at least 5 appointments scheduled per week in the reporting period. Numerator: Total number of minutes providers spent in a notes activity or navigator section within the reporting period. Denominator: Number of appointments during the reporting period
Time in orders per day	Average amount of time each provider spends in orders per day. Numerator: Total number of minutes providers spent in an orders activity or navigator section within the reporting period. Denominator: Total number of days that the provider logged in and worked during the reporting period
Time in orders per appointment	Time spent in orders per scheduled appointment. To be included, providers need at least 5 appointments scheduled per week in the reporting period. Numerator: Total number of minutes providers spent in an orders activity or navigator section within the reporting period

From the collected data, 6 participants were chosen to participate in a custom, hands on, training course to address these deficiencies. We chose participants with a wide spectrum of efficiency scores in order to analyze for improvements in all user types. A total of 6 was felt to be a practical number of participants for a pilot study. It enabled interaction during the training sessions, which could have been diminished with increasing participant number. Prior to this study, all participants received one Epic training course, either when Epic was first implemented or when the participant was employed with the organization. Additional participant characteristics and baseline efficiencies are reported in Table 2.

Table 2.

Individual user characteristics and metrics from the pre-training baseline to 4 post-training months (all times in minutes)

				User #1	User #2	User #3	User #4	User #5	User #6	Average
User characteristics
Years using Epic at our organization				7	18	12	5	3	4	8.17
Metrics
In Basket	Time in In Basket per day	Baseline		14.98	9.85	2.74	3.30	9.57	8.43	8.15
		1 month post-training		14.12	10.96	4.11	5.43	6.79	9.22	8.44
		2 months post-training		10.71	9.53	6.11	2.18	6.87	8.35	7.29
		3 months post-training		10.38	7.19	4.17	2.78	5.95	9.52	6.66
		4 months post-training		6.89	7.98	4.04	2.49	7.72	10.13	6.54
	Time in In Basket per appointment	Baseline		2.08	1.59	0.48	0.85	1.56	1.37	1.32
		1 month post-training		2.65	1.77	0.66	1.24	1.93	1.82	1.68
		2 months post-training		2.11	1.86	0.83	0.53	2.34	1.51	1.53
		3 months post-training		2.53	1.69	0.43	0.81	1.88	2.12	1.58
		4 months post-training		1.00	1.14	0.50	0.71	2.73	1.27	1.22
Notes	Time in notes per note	Baseline		13.32	0.79	4.65	6.14	5.88	5.32	6.02
		1 month post-training		15.49	1.33	2.66	6.31	8.24	7.72	6.96
		2 months post-training		11.96	1.24	3.35	5.66	7.62	5.43	5.87
		3 months post-training		13.78	1.83	3.29	6.51	8.90	5.39	6.62
		4 months post-training		9.14	1.77	2.64	6.38	7.34	4.62	5.32
	Time in notes per day	Baseline		51.89	4.67	11.64	23.40	22.67	27.33	23.60
		1 month post-training		50.17	7.23	9.23	27.72	26.28	35.49	26.02
		2 months post-training		36.66	4.67	16.46	23.00	18.13	26.71	20.94
		3 months post-training		44.09	7.49	14.19	22.43	25.23	20.81	22.37
		4 months post-training		49.37	12.25	11.61	20.63	16.31	28.56	23.12
	Time in notes per appointment	Baseline		7.20	0.75	2.05	5.99	0.81	4.45	3.54
		1 month post-training		9.41	1.17	1.47	6.31	1.59	6.99	4.49
		2 months post-training		7.24	0.91	2.23	5.61	1.43	4.82	3.71
		3 months post training		10.75	1.77	1.48	6.51	1.19	4.62	4.39
		4 months post-training		7.19	1.75	1.44	5.89	2.23	3.57	3.68
Orders	Time in Orders per Day	Baseline		9.59	4.20	3.46	3.27	5.00	4.79	5.05
		1 month post-training		6.18	2.04	2.90	3.59	5.59	4.42	4.12
		2 months post-training		5.06	1.98	3.39	3.53	4.20	4.93	3.85
		3 months post training		4.49	1.81	2.77	2.52	3.76	4.08	3.24
		4 months post-training		4.96	2.91	3.11	3.15	6.31	5.36	4.30
	Time in orders per appointment	Baseline		1.33	0.68	0.61	0.84	0.81	0.78	0.84
		1 month post-training		1.16	0.33	0.46	0.82	1.59	0.87	0.87
		2 months post-training		1.00	0.39	0.46	0.86	1.43	0.89	0.84
		3 months post training		1.10	0.43	0.29	0.73	1.19	0.91	0.77
		4 months post-training		0.72	0.42	0.39	0.90	2.23	0.67	0.89
Clinical review	Time in clinical review per day	Baseline		25.05	12.28	13.76	10.92	12.46	16.76	15.21
		1 month post-training		19.30	12.38	13.23	13.29	13.32	16.01	14.59
		2 months post-training		20.03	8.81	9.06	10.92	12.36	14.77	12.66
		3 months post training		16.53	10.05	13.22	12.32	8.77	13.99	12.48
		4 months post-training		16.07	12.34	13.16	13.44	11.98	18.94	14.32
Scores	Proficiency score	Baseline		3.33	3.22	3.36	2.95	3.66	3.43	3.32
		1 month post-training		2.98	3.46	5.14	5.04	3.51	3.47	3.93
		2 months post-training		5.27	3.82	3.75	5.06	3.39	3.68	4.16
		3 months post training		3.22	3.65	3.82	3.20	3.51	3.52	3.49
		4 months post-training		3.12	3.68	3.72	3.12	3.21	3.67	3.42
	PEP score	Baseline		3.82	6.59	4.79	4.81	4.78	5.78	5.10
		1 month post-training		3.53	6.41	5.17	4.15	4.45	4.98	4.78
		2 months post-training		3.66	6.57	5.36	4.79	4.54	5.51	5.07
		3 months post training		4.24	6.56	5.42	4.76	4.65	5.45	5.18
		4 months post-training		4.39	6.79	4.90	4.30	4.25	5.05	4.95
Total time per day	Based off 4 key metrics	Baseline		101.51	30.99	31.60	40.91	49.49	57.32	51.97
		1 month post-training		89.76	32.62	29.47	50.03	50.81	65.15	52.97
		2 months post-training		72.46	25.00	35.02	39.63	42.29	54.76	44.86
		3 months post training		75.49	26.53	34.36	40.06	44.03	48.40	44.81
		4 months post-training		77.29	35.48	31.91	39.71	41.37	63.00	48.13
Time saved	Time per day			29.05	5.99	0	1.27	8.12	8.92	8.89
	Time per month			581.06	119.83	0	25.44	162.4	178.38	177.85
	Time per year			6972.7	1438	0	305.33	1948.85	2140.55	2134.24

Note: At an individual level, while 4 of the participants overall maintained their PEP scores, 2 of the participants, who were the most and least proficient at baseline, saw an increase in their PEP scores (6.6–6.8 and 3.8–4.4, respectively). The average Proficiency Score remained overall stable from baseline to completion (3.3–3.4). However, 2 months post-training, the value peaked at 4.2. At an individual level, two-thirds of participants had an increased score from baseline to completion. Additionally, 5 participants had their maximum score 2 or 3 months post-training.

Instructional design

We collaborated with clinical informatics and content experts to develop custom group training sessions based upon baseline efficiency metrics and key areas of deficiencies of the participants. The education content focused on documentation and inbox management within the EHR, with the aim to enhance high-quality and efficient documentation and communication, while decreasing major stressors such as click burden, nonstandardized notes, and extensive data entries. Five hour-long training sessions were developed. The first 2 sessions focused on documentation. Standardized templates for typical radiation oncology notes were provided and customized for each user. The use of voice recognition tools for dictation was encouraged. The subsequent 2 sessions concentrated on inbox and EHR communication management. The final session was both a review and an interactive period, allowing for individualized questions and access to customization from content experts. Each participant was provided personalization guides for documentation and inbox tools, as well as a radiation oncology-specific list of applicable SmartTools.

Evaluation methods

EHR performance data were collected monthly for a total of 4 months post-training. We selected metrics relevant to clinical practice and feasible to trend (Table 1). A 7 question, 4 month post-training survey was conducted. This consisted of “yes, no, or maybe” responses qualitatively assessing self-reported competence, performance, and the perceived impact of educational intervention on participants’ clinical practices.

RESULTS

The 6 chosen participants were present for all 5 of the in-person training sessions. All participants completed the survey 4 months after completion of training.

EHR performance data

With regards to the PEP score, there was no overall change in the average value for the participants from baseline to 4 months post-training (5.1–4.9). However, there was a peak average value of 5.2 3 months post-training (Table 2).

For the inbox metrics, there was a reduction in the average time spent in the In Basket per day (8.1–6.5 min) and a reduction in time in In Basket per appointment (1.3–1.2 min). Individually, 4 of the 6 reduced both their time in In Basket per day and per appointment.

For the documentation metrics, there was an average reduction in time spent in notes per note (6.0–5.3 min), reduction in time in notes per day (23.6–23.1 min), and an increase in time in notes per appointment (3.5–3.7 min). Half of the participants had reduced time in notes per note, with the largest decrease of 13.3–9.14 min per note. Three of the participants improved their time in notes per day and 4 improved their time in notes per appointment.

By comparing the summation of “per day” metrics from baseline to 4 months post-training, the estimated number of minutes saved per day was determined. For each individual, the estimated minutes saved per day were 0, 1.3, 6.0, 8.1, 8.9, and 29.1, with an average of 8.9 min saved per day. Expanding this to hours saved per year, and extrapolating that physicians work 5 days per week in the 12-month interval, the users respective values were 0, 5.1, 24.0, 32.5, 35.7, and 116.2, with an average of 37.1 h saved per year.

Post-training surveys

Results from the survey conducted 4 months after the training noted all participants felt more efficient with their use of the EHR or more efficient in their time interacting with the EHR. Half felt more efficient in the inbox and all felt more efficient in note composition. Two-thirds also felt more efficient in orders. All participants reported they were still using the provided documentation templates. The majority continued to use efficiency tools and dictation.

DISCUSSION

This was a unique intervention seeking to improve radiation oncologists’ EHR efficiencies by tracking multiple variables through Signal log data through the Epic UserWeb following focused training sessions. EHR-time savings varied from 0 to 29.1 min/day translating to a possible 116.2 EHR hours saved per year. This was a desired outcome, as increased time in the EHR has been linked to stress and burnout. Additionally, the participant survey responses showed self-reported improved efficiency.

The average PEP and Proficiency scores remained similar from baseline to 4 months post-training. The stability of the PEP score was not unexpected, as this score was a normal distribution within the entire healthcare system, across many different EHR users and departments. Although the average PEP score remained similar, the least and most efficient users at baseline had an increase in their PEP score, indicating improvements in efficiency can be seen no matter the baseline level.

Training sessions focused on documentation and inbox inefficiencies of the radiation oncology department. The sessions led to improvements in the time spent within various subcategories of the key metrics of documentation and inbox management. Interestingly, the metric of time in notes per note improved, despite the introduction of smart dictation tools and new standardized, department-specific documentation templates. It was expected that this metric would have increased with these changes. However, just the opposite was seen, and, in fact, one participant dramatically improved the documentation time from approximately 13 to 9 min per note. Individually, this may appear as a modest improvement, but when translated to each note written per day, a notable amount of time is saved.

Enhancing EHR efficiencies and addressing EHR challenges is an imperative component for both patient care and employee satisfaction. In response to the noted frustrations and burnout associated with the EHR, healthcare systems are devoting considerable resources to optimize EHR proficiency. Studies have focused on various modalities for improvement, spanning from repeat EHR training to hiring additional employees to handle the EHR workload. Most studies, including this one, are limited to short-term outcomes. Long-term data will be beneficial to determine the sustainability of behavior.

There are several other limitations. First, Signal only acquires data on providers who have a schedule of patient appointments. Our study only focused on attending-level physicians with a broad range of baseline proficiencies. One future direction would be to assess efficiencies of medical residents, advanced practitioners, nurses, and medical assistants. Their productivities and proficiencies have a downstream effect on the remainder of the department staff. Additionally, with future training sessions, it would be interesting to compare how improvements vary based on different participant ages or different levels of technology competence. Second, we acknowledge that these measures are fitting only for the ambulatory setting and would need to be altered for inpatient practices. Third, while time is an important measure, we recognize that time alone does not capture all dimensions of EHR efficiency or providers’ satisfaction. Finally, while we acknowledge that our participant population was small, it allowed for an interactive group training session between participants and trainers. For future training sessions, a balance between group interaction and typical department resource constraints must be considered.

Next steps include expansion of the training sessions to the remainder of our department. Other general applications include expanding specialty-specific efficiency training to other departments and/or offering this training to new hires. Finally, individual user access to Signal data could allow for self-directed evaluations of deficiencies and areas of improvement after training. This could decrease reversion to pre-training habits as suggested in this study.

CONCLUSION

This study demonstrates an example of department-focused, hands-on EHR training aimed to improve user efficiency and satisfaction with EHR interaction. The results demonstrate improved efficiency in time management within the metrics of documentation, inbox management, and orders. This work also highlights the benefits of using log-based data to monitor efficiency status and progress within the EHR to assess the quality of training.

ACKNOWLEDGMENTS

We acknowledge Erin Banaszak, MEd RTR and Madeline Hehemann, BSN, RN for their contribution in designing and conducting each training session. We acknowledge Heather Holsworth, MHA, BS RTT and Victoria Sergent, BSN, RN-BC, DC for their contribution in organizing and designing the training sessions.

CONFLICT OF INTEREST STATEMENT

None declared.

DATA AVAILABILITY

The data underlying this article will be shared on reasonable request to the corresponding author.