Deploying ambient clinical intelligence to improve care: A research article assessing the impact of nuance DAX on documentation burden and burnout

Highlights

• •Ambient clinical intelligence (ACI) significantly reduced provider documentation burden, frustration and burnout. ACI use reduced daily documentation time, and 2.5 h per week of ‘Pajama Time’ or off-hours documentation.

Abstract

Introduction

Occupational burnout among clinical care providers, due in part to documentation burden, has reached crisis level. This study measured the effect of using new clinical documentation software, an ‘ambient clinical intelligence’ (ACI) program, to reduce the documentation workload and improve provider wellbeing.

Methods

This was a randomised, control study with a step-wedge design. Providers were randomly assigned to use ACI early or late in the study. Medical records metadata captured time spent on documentation. Measures of burden and burnout were collected monthly.

Results

ACI significantly reduced documentation burden, provider frustration and burnout. Providers spent less documentation time each day, and 2.5 h less per week of off-hours documentation.

Discussion

This study demonstrates that the use of ACI does indeed relieve the documentation burden and had both subjective and objective benefits. The widespread use of ACI has the potential to alleviate the crisis of physician burnout.

Introduction

Occupational burnout is a physical and/or emotional exhaustion brought about by chronic workplace stress. Healthcare providers regularly contend with long hours, inflexible schedules, and the expectation that they will sacrifice personal needs. This has been compounded by COVID pandemic stress and strong economic headwinds. This confluence of factors has had a deleterious impact on caregivers’ wellbeing, leading them to over-extend themselves, damaging relationships, losing personal investment in the profession, and often leaving the profession entirely.^1,2 Burnout has become a crisis for care delivery. The World Medical Association in 2018 declared a global pandemic of physician burnout³ and, in 2022, the United States surgeon general issued an advisory pointing to a burnout crisis that ‘not only harms individual workers but also threatens the nation’s public health infrastructure’.⁴

In addition, the disability and resignation of caregivers causes significant financial strain. A pre-COVID pandemic analysis using a conservative model estimated ‘approximately $4.6 billion in costs related to physician turnover and reduced clinical hours attributable to burnout each year in the United States’.⁵ A recent analysis of burnout-driven retention issues projected 11,339 primary care physicians to leave their practices each year, resulting in more than $979 million in annual excess healthcare expenditures due to turnover.⁶ Financial stress on healthcare systems and practices further compounds burnout in the caregivers who stay.

Burnout is increasing in frequency and severity. A recent survey of >2,400 physicians noted a dramatic increase in burnout, with 62.8% of respondents manifesting at least one symptom.⁷ A systemic review published in 2020 considered 43 articles and research in 19 countries. It revealed widespread symptoms including ‘reduced empathy towards patients, reduced participation in professional development activities, and intentions to leave the job or profession’. In addition, there was moral and physical distress and increased potential for substance abuse. Importantly, this review noted burnout as a ‘loss spiral rather than an end-state’ and called for specific, targeted interventions rather than broad measures.⁸

Clinical documentation takes up a significant portion of a shift and reduces the amount of time that can be spent diagnosing and treating patients. For providers, this necessarily creates a stressful conflict of values. It has been suggested that we reframe the conversation around healthcare burnout as ‘moral injury’, as providers are increasingly forced to spend precious hours on tasks unrelated to the reasons they went into healthcare.⁹ Furthermore, clerical burden has been posited as a leading cause of burnout and as much as 80% of burnout could be attributed to workflow issues.¹⁰ In a study auditing medical records, it was noted that residents logged 9 h of their 20-h shifts working on documentation, approximately 2 h of documentation for every 1 h with patients.¹¹ Unfortunately, the documentation burden does not end at the end of a shift. Most providers work on documentation on unscheduled days, especially weekends and holidays, during ‘Pajama Time’, an American Medical Association term referring to time after work in the office or at home on days with appointments.¹² Interventions that reduce the documentation burden are desperately needed to mitigate burnout.

Healthcare information technology developers have created tools to reduce documentation burden, and modern computing is advancing this work. Speech recognition software applications have been trained with meta-linguistic knowledge through natural language processing and natural language generation.¹³ Text-to-speech applications are now being empowered with these computing technologies to create ambient clinical intelligence (ACI). ACI uses microphones to capture patient interactions and then applies ‘conversational artificial intelligence (AI), machine learning, speech synthesis, natural language understanding, and cloud computing’,¹⁴ ACI interprets rhetorical cues made by speakers that indicate topic relationships. The precision of ACI and specific voice-to-text tools is improving, and proliferation of these tools is accelerating.^15,16 These ACI-enabled tools potentially remove steps from the documentation process and save time.

Historically, in our healthcare system, there are few structured input protocols in our existing EMR systems to sort contextual information; providers must sort and edit transcription records or write notes entirely in prose. In a small pilot study, providers looking to decrease documentation time used AI-enabled clinical documentation transcription software to capture, contextualise and systematise encounter information. This captured the patient encounter, analysed relationships, and displayed the output for the provider’s review. Providers’ feedback was positive. However, the impact on their documentation burden was limited to non-standardised subjective impressions. To collect empirical data, we designed a randomised, control study using medical chart metadata as well as monthly self-reported burnout measures.

Methods

The study was a randomised, control study that used a step-wedge design. Providers were randomly assigned to either an Early Implementation group that began using clinical documentation software immediately after baseline data collection, or to a Late Implementation group that abstained from using such software until the end of the study. Because the Late Implementation group did not have exposure to the automated clinical documentation software, they functioned as a control group. The software chosen for this study was the Dragon Ambient eXperience (DAX) Solution® from Nuancea.

The health system’s Institutional Review Board reviewed and approved the study (STUDY2022000071) and consent form prior to the initiation of study activities, and informed consent was prospectively obtained from all study participants.

This study was conducted and reported in accordance with the ‘Strengthening the Reporting of Observational studies in Epidemiology’ (STROBE) guidelines (Appendix 1).¹⁷

We identified 109 family medicine providers across the seven states of our health system which were most likely to benefit from automated clinical documentation. Using Epic Signal Data, we found providers that were struggling in more than one area related to documentation burden. These providers were experiencing high documentation burden, spending many hours outside of the workday on patient visit documentation, and were taking more than a week to close visit notes. The 109 providers were invited to attend one of three 90-minute virtual information sessions to learn about DAX® 2021 and the research study.

Following the information session, 24 interested providers signed up to participate and were randomly assigned, 12 to each implementation group. The study took place from 1 June 2022 to 31 December 2022. There was a 2-month baseline period before any of the providers received the software, during which baseline data collection was completed. Following this, the Early Implementation group received training and began using ACI software, while the Late Implementation group continued with documentation as usual. After 4 months, providers in the Late Implementation group were invited to take part in training and provided access to the software.

Given the step-wedge design, some of the Late Implementers began using the automated clinical documentation before others, although this occurred within the bounds of the final month of the study. Following the end of research study, both groups were able to continue using the software. Two from the Late Implementation group erroneously received the software tool at the same time as the Early Implementation group. These two providers remained in the Late Implementation group for our analysis of documentation burden, thus our analyses reflect an intent-to-treat approach.

Measures

Automated clinical documentation usage

Weekly automated clinical documentation usage for each provider was calculated by dividing the number of visits using ACI software by the total number of visits. Usage was captured through Epic visit notes.

Provider reported measures

To capture provider-reported documentation frustration, burden and burnout, we administered a monthly four-item pulse survey to all 24 providers; providers were sent an invitation to complete the survey via email monthly for 7 months. The items included:

• 1.In the past 30 days, I felt burned out from my work.
• 2.In the past 30 days, I felt frustrated with the patient visit documentation process.
• 3.In the past 30 days, I felt I was not able to connect with my patients during their visits.
• 4.In the past 30 days, I spent too much time documenting patient encounters.

Providers rated how often they felt about each statement using the scale ‘Never’, ‘Sometimes’, ‘Usually’ or ‘Always’. For analysis, these were collapsed these into binary responses: Yes (always and usually) and No (sometimes and never).

Documentation burden

Encounter metadata was obtained via Epic’s Signal Data and used to assess changes in provider productivity metrics. Signal data were obtained monthly throughout the study and included:

• -Minutes in notes per appointment: Average number of minutes per month a provider spent writing notes per appointment.
• -Minutes in notes per note: Average number of minutes per month a provider spent writing a note.
• -Minutes in notes per day: Average number of minutes per month a provider spent on notes each day.
• -Minutes in system per day: Average number of minutes per month a provider was logged into the system per day.
• -Pajama Time: Average number of minutes per month a provider spent in charting activities on weekdays outside the hours of 7am or 5:30pm or outside scheduled hours on weekends or non-scheduled holidays.
• -Time outside 7am to 7pm: Average number of minutes a provider spent in the system on scheduled days outside 7am to 7pm.
• -Time outside scheduled hours: Average number of minutes a provider spent in the system outside scheduled hours.

Data analysis

All data were analysed using R v.4.3.0, a free software environment for statistical computing and graphics from the R Foundation, Vienna University of Economics and Business.¹⁸

Self-reported measures: To assess changes in provider-reported measures, we grouped surveys taken before software implementation and surveys taken after implementation, independent of the study group. We compared provider surveys before software usage as one sample and surveys received after software usage as the other sample. The Late Implementation group had only pre-usage data because they did not use the software during the study, while the Early Implementation group had only one pre-usage survey before they started using the software. We then conducted logistic regression using a modified Poisson distribution to estimate risk ratios, clustered by provider to account for repeated measures (seen in Table 1).

Table 1.

Provider-reported experience survey.

Agreement with the statement	Prior to DAX® implementation N=63		After DAX® implementation N=71		Logistic regression (clustered by provider)
	%	95% CI	%	95% CI	RR	95% CI	p-value
I felt burned out from my work	57.1%	(44.7–68.7)	26.8%	(17.8–38.2)	0.5	(0.3–0.9)	<0.01
I felt frustrated with the patient visit documentation process	88.9%	(78.5–94.6)	39.4%	(28.8–51.2)	0.4	(0.3–0.6)	<0.001
I felt I was not able to connect with my patients during their visit	23.8%	(14.9–35.8)	4.2%	(1.4–12.3)	0.3	(0.1–0.7)	<0.001
I spent too much time documenting patient encounters	96.8%	(88.2–99.2)	45.1%	(34.0–56.7)	0.4	(0.3–0.5)	<0.001

Provider surveys from both groups were treated as one sample for analysis. The Late Implementation group only had pre-DAX-usage data because they did not use DAX during the study, while the Early Implementation group only had one pre-DAX-usage survey before they started use.

Objective measures: To assess changes in objective measures of documentation burden, we conducted a difference-in-difference analysis using linear regression to compare the baseline and follow-up periods of Early Implementers to the baseline and follow-up periods of Late Implementers. The analysis was conducted at an individual provider level (seen in Table 2).

Table 2.

Documentation burden.

Time taken to perform clinical documentation	Early implementation providers		Late implementation providers		Difference-in-difference
	Months 1–2	Months 3–7	Months 1–2	Months 3–6
	N=12	N=12	N=12	N=12
	Mean ± SD	Mean ± SD	Mean ± SD	Mean ± SD	Est*	p-value
Minutes of Pajama Time	107.0±63.7	80.9±38.6	77.4±48.0	80.6±55.3	−29.33	0.0100
Minutes in notes per appointment	12.3±4.2	10.6±3.9	10.6±5.7	10.6±6.4	−1.73	0.0070
Minutes in notes per day	97.7±28.3	80.7±24.4	86.0±33.3	85.9±37.4	−16.91	0.0011
Minutes in notes per note	6.8±3.0	6.5±3.5	4.5±2.4	5.0±2.3	−0.83	0.0375
Minutes in system per day	242.0±47.7	219.2±37.9	223.0±57.3	219.9±61.9	−19.75	0.0183
Minutes outside 7am to 7pm	46.2±27.5	40.1±18.5	38.6±21.9	42.1±20.8	−9.61	0.0363
Minutes outside scheduled hours	78.8±37.2	66.6±27.5	59.7±36.8	60.5±38.9	−12.94	0.0127

^⁎Note. This is a difference-in-difference model using multiple linear regression. Est is the estimate for the difference in difference between the two groups for the corresponding period.

To assess the relationships between objective documentation burden and provider-reported measures, we used Pearson correlations and aligned monthly burden with the provider-reported measures (seen in Table 3).

Table 3.

Correlations between provider experience and documentation burden.

Provider experience	DAX usage	Documentation burden
Survey responses	DAX visits (%) N=12	Time in notes per day N=24	Pajama Time N=24	Time outside 7am to 7pm N=24	Time outside of scheduled hours N=24
I felt frustrated with the patient visit documentation process	−0.50*	0.28*	0.19*	0.15	0.21*
I felt burned out from my work	−0.33*	0.22*	0.38*	0.24*	0.38*

Cells present correlation coefficients (r) (range −1.0 to 1.0). The percentage of DAX visits was calculated only for the Early Implementation group. Other comparisons include all providers. Correlations are between documentation burden and provider experience survey results for the same month.

^⁎Significant at p<0.05.

Results

ACI documentation software usage

For Early Implementers, ACI documentation software usage ranged from 20–40% of visits. On average, Early Implementers used the tool for an average of 32% of weekly visits. For Late Implementers, usage was zero during the study period.

Provider-reported measures

Provider response rates on the monthly surveys were high: 100% response rate at months 1, 2, and 3; 90% response rate at months 4 and 6; and 80% response rate at month 5. Because providers served as their own controls, provider-reported measures were compared pre- implementation and post-implementation, regardless of group assignment.

Prior to software implementation, the majority of providers reported frustration with the visit documentation process (88.9%) and too much time documenting patient encounters (96.8%). More than half of the providers felt burned out from their work (57.1%). Almost a quarter of providers felt they were unable to connect with their patients during their visit (23.8%) (Table 1).

After software implementation, all provider-reported measures significantly and meaningfully improved. Providers reported 30.3% less burnout, 49.5% less frustration with the documentation process, and 51.7% less time spent on documentation. Lack of connection with patients also significantly decreased by 19.6% (Table 1, all p<0.01).

Documentation burden

Given the nature of Epic’s Signal Data, which is collected for all users, there were no missing data for any of the Documentation Burden measures.

Notably, Early Implementers had an average reduction in Pajama Time of 26 min per day, while Late Implementers had an increase of 3.2 min per day. Similarly, Early Implementers had an average reduction of 22.8 min ‘In System’ per day compared to Late Implementers, who had a reduction of only 3.0 min per day (Table 2).

The difference-in-differences multiple regression analysis revealed significant differences in documentation burden between the Early and Late Implementation groups. Early Implementers had a significantly lower documentation burden at 3–7 months for all burden indicators, while Late Implementers showed no significant changes (Table 2, all p<0.05).

ACI software usage reduced frustration with documentation and burnout

There was a significant correlation between ACI software usage (% DAX visits) and provider-reported burnout and frustration with the documentation process. ACI software usage was significantly associated with reduced provider-reported burnout (r=−0.33, p<0.05) and with provider-reported frustration with the documentation process (r=−0.50, p<0.05). Burnout and frustration with the documentation process were also significantly correlated with documentation burden indicators (Table 3).

Correlation between DAX usage, Pajama Time, and burnout

Post-hoc regression analyses, which included ACI software usage, and multiple measures of documentation burden as predictors of self-reported burnout, revealed ACI software was significantly and negatively predictive of burnout (OR=0.96; 95% CI=0.94–0.98, p=0.0003), whereas Pajama Time was significantly and positively associated with burnout (OR=1.01; 95% CI=1.00–1.03, p=0.036).

Discussion

Ambient AI clinical documentation packages

There are numerous versions of AI-assisted clinical documentation. Indeed, in the USA the Food and Drug Administration (FDA) has approved over 1,000 medical devices incorporating AI, including clinical documentation packages. This study used DAX (Dragon Ambient eXperience) developed by Nuance (now part of Microsoft)¹⁹ that uses ambient AI to automatically document and transcribe patient encounters in real time and integrates with Electronic Health Record (EHR) systems such as Epic and Cerner. Our Providence healthcare system uses Epic, with which DAX is seamlessly compatible.

Other ambient AI packages in use in the USA are Suki,²⁰ which integrates with all the major EHR systems, and has automated transcriptional function, Augmedics²¹ that uses remote medical scribes, and Ambience Healthcare²² which has been adopted by some major healthcare systems because of its compatibility with most EHR systems.

In the UK, a Nuance Dragon Medical One package is used in the National Health Service (NHS) and private clinics, being compatible with NHS digital health record systems. Others include Heidi Health²³ which supports multilingual documentation, Lyrebird Health,²⁴ TORTUS²⁵ and Clinical Notes AI.²⁶

Regulation of AI-based clinical documentation software

Regulation of AI applications varies worldwide and is based on different criteria. European Union regulations are prescriptive and risk based, aiming to harmonise regulations across member states. In the UK, the Medicines and Healthcare products Regulatory Agency (MHRA) oversees medical devices that incorporate AI clinical software using a pro-innovation approach. In the USA, the FDA regulates AI clinical software, but comprehensive federal AI regulations are lacking and different states can implement their own AI-related laws. Worldwide, the safety (including confidentiality) and effectiveness of AI clinical documentation are the major regulatory concerns.

Effectiveness of AI clinical documentation

Despite the widespread use of ambient AI clinical documentation packages, there are few studies of their influence on caregiver stress and burnout. Hence, the value of this study. One study, from the University of Michigan, observed the effect of AI ambient documentation on the patient–physician experience. There were demonstrated benefits in subjective but not in objective patient satisfaction.²⁷ By contrast, the perspective of our study was different. It was focused on clinician satisfaction and workload, and we identified both subjective and objective benefits.

The introduction of ACI clinical documentation software was associated with several beneficial changes. These changes are highly significant, despite the limited group size in the study and the use of the software during only 32% of clinical visits.

The response rate to the subjective provider surveys was notably high (80–100%). The greatest frustration, among 97% of providers, was the amount of time needed to document patient encounters. This frustration was lessened by half after the introduction of the ACI software. The other measures were impacted, too, and all were statistically significant (Table 1). The reduction in the stressors reported by the clinicians in our study after the introduction of ACI documentation software clearly suggests that this technological approach can contribute to decreasing provider burnout.

The data concerning the documentation burden was complete and entirely objective, being captured in the Epic Signal Data system. The Early Implementation group saw many reductions in documentation time after the introduction of the software, especially in the minutes in the system, outside scheduled hours and Pajama Time (Table 2). By contrast, the Late Implementation group, which did not have the benefit of using the ACI software, showed no reduction of the documentation burden. This absence of change suggests that an unmeasured variable, such as seasonality, changing hospital procedures or documentation guidelines, is not responsible for the lessening of the documentation burden in the Early Implementer group. The difference in the documentation times between the Early and Late Implementer groups was statistically significant for all measures. It seems likely that the relief from out-of-hours working would contribute to the improved responses in provider-reported surveys.

Measures of the effectiveness of ambient AI clinical documentation

ACI software usage resulted in objective changes including: (i) reductions in time outside work hours, (ii) reduction in overall time per appointment and (iii) improvements in providers’ subjective experiences with burnout and frustration.

Subjective measures showed that the use of ACI software: (i) improved providers’ experiences with documenting encounters, (ii) lessened frustration with process, (iii) improved connections with patients, and (iv) decreased burnout overall.

Potential benefits of employing ambient AI clinical documentation

The improvements we identified were for care providers rather than patients. Reduction in clinician burnout and improved patient–clinician experience could result in significant financial savings. The cost of advertising, recruiting, interviewing and hiring to replace providers who resign is substantial. Those who suffer burnout but do stay in position often require additional support and remediation of burnout symptoms that tax system resources. Widespread use of documentation-relieving technology would likely improve providers’ work experience, reduce turnover and, potentially, reduce costs and save resources.

Study limitations

This study was limited by the small numbers of providers in this pilot and the limited time. Greater improvements in the longer term may be expected. The strengths of this study, which include objective metadata analysis, multiple measurements and a high response rate, offset the size limitation of this pilot study. Additionally, this study was conducted within a single healthcare system, minimising documentation differences that are often a barrier to analysis of data collected from hospitals and clinics in different systems.

Conclusion

Providers’ documentation burden is high and is likely to increase further. Routine documentation is expanding to include additional patient information about genetic testing, social medicine and personalised treatments, and adding to the time burden. Given the inverse relationship between documentation time and caregiver retention, it is imperative to reduce provider burden, ease burnout, and thus improve retention and our providers’ wellbeing. We found that ACI documentation software, in this study Nuance DAX®, is a useful tool to reduce documentation time, lessen burnout and reduce frustration. Compared to other strategies that require a more significant investment of time and resources, we found that ACI software is a useful tool to save provider time and directly address burnout. Ultimately, this approach offers a way to improve healthcare for both our providers and patients.

Funding

This study was not supported by external funds and grants. However, ACI clinical documentation software (DAX®) licenses were provided by Nuance (Microsoft). The company was not involved in study design, implementation, analysis or writing.

Data availability statement

The raw/processed data required to reproduce the above findings cannot be shared at this time due to legal and ethical reasons, including compliance with HIPAA regulations.

Ethics approval and consent to participate

The Providence Health System’s Institutional Review Board reviewed and approved the study (STUDY2022000071) Written informed consent was obtained from all participants prior to the commencement of study procedures.

CRediT authorship contribution statement

Staci J. Wendt: Methodology, Investigation, Conceptualization. Catherine T. Dinh: Project administration. Michael Sutcliffe: Writing – original draft. Kyle Jones: Methodology, Formal analysis. James M. Scanlan: Writing – review & editing. J. Scott Smitherman: Writing – review & editing.

Declaration of competing interest

Providence St. Joseph Health received a limited number of software licenses from Nuance, a Microsoft Company, to complete this research study. The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.