Skip to main content
BMC Health Services Research logoLink to BMC Health Services Research
. 2026 Jan 8;26:182. doi: 10.1186/s12913-025-13954-5

Performance, acceptability, and impact of ambient listening scribe technology in an outpatient context: a mixed methods trial evaluation

Salim Memon 1, Adam Brand 2,3, Bianca Taylor 3, Adelaide Michael 4, Rachael Smithson 4,✉
PMCID: PMC12882620  PMID: 41507920

Background

In 2024, Gold Coast Hospital and Health Service outpatient division initiated a 16-week trial of artificial intelligence (AI)-enabled ambient listening scribe technology. The objective of this pilot study was to evaluate scribe technology performance (according to dimensions of quality, utility and reliability) and its impact on the experience of outpatient clinicians and patients receiving care.

Methods

A mixed method research design combined analysis of data from patient and staff surveys, staff interviews, scribe outputs and electronic medical records across a breadth of outpatient specialties.

Results

By and large, ambient listening technology was associated with positive patient and staff experience. On average, 58% of scribe outputs were accepted without modification into the electronic outpatient note. There was limited evidence of bias in outputs, however there was some evidence of hallucination or incorrect outputs.

Conclusions

Qualitative and quantitative data were internally consistent and demonstrated that ambient listening technology can (1) produce an accurate summary of outpatient appointments, (2) enhance clinical note quality and (3) improve both clinician and patient experience.

Clinical trial number

Not applicable.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-025-13954-5.

Keywords: Artificial intelligence, Ambient listening, Medical transcription, Scribe technology, Machine learning, Digital innovation


What is already known on this topic

What this study adds

How this study might affect research, practice, or policy

Ambient listening scribe technology is being adopted at pace by health services in the hope that it will alleviate clinical documentation burden.

This study contributes evidence of ambient listening scribe technology performance in a real-world outpatient department application.

This study aims to inform practice in the implementation and use of ambient listening scribe technology in other health services considering adoption.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-025-13954-5.

Introduction and background

Medical records are a foundational communication platform for healthcare professionals, facilitating the coordination of patient care across clinical settings [1, 2]. These records enable information exchange among a patient’s care team, supporting the management of diagnostic procedures, treatment protocols, and ongoing care activities in both hospital and community environments [1, 2]. Despite the importance of accurate and timely recording of medical notes, the burden of documentation experienced by clinicians represents a significant and well-recognised challenge facing the healthcare system [3, 4].

To alleviate this burden, ambient listening medical scribes (‘scribes’) are being adopted at pace [5, 6]. Scribe technology collects real-time voice data during clinical encounters, instantaneously transforming this content into a summary in the clinician’s chosen format [7, 8]. The basis of this technology is machine learning and deep learning algorithms that transform live transcripts into continuously higher quality and more meaningful clinical documentation outputs [8]. Evidence to date associates this technology with increased efficiency and productivity, improved note quality and note accuracy, and a better experience for both patients and clinicians during encounters [7, 9–11]. As with all artificial intelligence (AI) solutions, successful implementation requires this technology to be fit for purpose and accepted by those using it.

To date, use of ambient listening scribes has not been evaluated in an Australian hospital outpatient department context, representing a gap in literature. In July 2024, Gold Coast Hospital and Health Service (GCHHS), Australia, commenced a trial of an ambient listening solution. Limited term licenses were purchased for 100 clinicians participating in a 16-week trial. The scribe solution was available to clinicians via their work laptop and smart devices. Clinicians were supported through group training sessions and on-demand support by a dedicated project team. Additionally, speakers and microphones were provided to support the use of scribes. This research aims to understand the value of ambient listening technology in routine hospital practice. Using a mixed-methods approach, it aims to produce an early evaluation of AI ambient listening scribe performance (according to the dimensions of quality, utility and reliability) and its impact on staff and patient experience.

Methods

Study design

This research adopted a mixed-methods approach, which is noted for its suitability in application to complex effectiveness studies [12]. The study has ethical approval from the Darling Downs Health Human Research Ethics Committee (HREC/2024/QTDD/108398) and data was collected between July and December 2024. The target population of the study were outpatient clinicians who received one of the 100 ambient listening tool limited term licenses.

Quality evaluation

In the pre-implementation phase, the ambient listening technology was set up to run silently in five outpatient clinics of consenting medical officers (‘super users’) and patients. Silent ambient listening patient notes were compared to the clinician-generated medical records of the same consultations to evaluate note quality by SM, a senior clinician on the project team. This comparison was informed with a verbatim transcript of the appointment generated by the ambient listening system. This approach was chosen to isolate the quality of the initial ambient listening output, prior to clinician amendment. The modified Physician Documentation Quality Instrument (PDQI-9) is a validated tool that uses a 5-point Likert scale to score documentation quality against eight domains, with one representing poorest quality, and a score of five representing high quality. For each document, the maximum value of 40 represents a perfectly completed document [6, 13].

Utility evaluation

Saved and uploaded (final) electronic medical records initiated by scribe technology were compared to the original (or raw) scribe output in order to measure the average number of clinician amendments required to reach a satisfactory standard of appointment summarisation. The ROUGE (Recall-Oriented Understudy for Gisting Evaluation) score is a widely used metric for evaluating the similarity between an automatic text summarisation output to a human-created reference [14]. ROUGE scores are expressed as percentages, representing the degree of similarity between the generated text and the reference summaries; an indication of required clinician modification efforts to the raw outputs [14]. Pairs of ambient listening generated notes and final medical records were compared using the Microsoft Word ‘compare’ and ‘track changes’ functionality. Differences were counted manually, and values inputted into the ROUGE-1 formula.

Staff and patient experience

Semi-structured staff interviews

A convenience sampling method was employed to recruit medical officers engaged as ‘super users’ in the planning of the trial, providing early feedback on solution functionality. Semi-structured interviews were conducted by AM, either an in-person or a virtual interview format. Interviews explored themes of medical record updating practice, frustrations and opportunities – prior and post the use of ambient listening technology. With participant consent, interviews were recorded and transcribed.

Staff survey

Surveys were distributed to all 100 clinicians (medical officers, clinical nurses who ran nurse-led clinics and registrars) who participated in the trial. Surveys were distributed through a Microsoft Forms link and comprised six questions (Likert scale and open ended) to elicit participant reflections on their experience on the ease of using ambient listening technology, observations of the technology’s performance and its impact on their outpatient clinical practice. Responses were exported into Excel for analysis.

Patient surveys

Patient surveys were distributed by medical officers who participated in the trial via a QR code link to an anonymous Microsoft Forms survey. This survey was based on previous work by Tierney et al. and Mishra et al. [6, 15]. Surveys comprised four questions (Likert scale and open ended) to elicit patient reflections on the clinician-patient interaction – amount of time the doctor spent speaking directly with them, time the doctor looked at the computer screen and the overall effect of the ambient listening on their visit compared to previous appointments. Responses were exported into Excel for analysis.

Analysis

Analysis followed a ‘complementarity’ approach to synthesising mixed-methods data, where distinct insights from qualitative and quantitative data sources were combined to achieve a greater breadth and depth of understanding than could be achieved using one in isolation. An ‘embedded’ approach was taken to integrate qualitative and quantitative data whereby each source of information provides answers to related questions simultaneously [12]. The data underwent a content analysis of deductively derived themes, whilst remaining open to inductive themes [16].

Results

Data collection and analysis focused on 100 clinicians trialling the technology in outpatient clinics. Table 1 summarises the outpatient specialties involved in the pilot. In total 7,499 consultations were undertaken using the AI scribe technology during the trial.

Table 1.

Summary of outpatient specialty trial participant numbers and average total consults per trial participant

Outpatient specialty Count of trial participants Average of total consults per trial participant
Paediatrics 17 132
Orthopaedics 10 7
Rheumatology 9 126
Respiratory 8 103
Neurology 8 143
Renal 6 27
General Medicine 6 32
Haematology 6 35
Endocrinology 5 162
Infectious Diseases 4 31
Geriatrics 3 18
Pain medicine 3 46
Palliative Care 3 20
Digestive Health 2 8
Obstetrics 2 27
General Surgery 2 30
Cardiology 2 30
Gynaecology 2 13
Mental Health 2 60
Grand Total 100 75

Staff surveys were distributed to all 100 clinicians (medical officers, clinical nurses who ran nurse-led clinics and registrars) who participated in the trial, with 43 responses (43%). Of these staff, six medical officers engaged as ‘super users’ took part in 13 semi-structured interviews (seven prior to the scribe introduction and six post). Further, 22 patient surveys were returned.

Quality

On average, the scribe technology produced equivalent, or slightly higher, quality patient notes compared to current clinician practice. Eighteen pairs of notes were assessed using the PDQI-9 tool (one clinician-created and one AI-generated). The sample size for this analysis was limited by practical constraints associated with data collection prior to the scheduled initiation of the ambient listening trial. Clinician notes averaged a quality score 34.6/40 versus AI-generated notes which scored 37.06/40, shown in Table 2. These results were particularly influenced by ambient listening’s stronger performance on metrics related to “creating a more thorough and well-organised note”.

Table 2.

PDSQI analysis for AOPD and ambient listening notes generated for the same appointment

Attribute Description of Ideal Note AOPD Note Average (0–5, 5 = best) Ambient Listening Average
(0–5, 5 = best)
Difference
Accurate The note is true. It is free of incorrect information. 4.83 4.67 -0.17
Thorough The note is complete and free from omission and documents all of the issues of importance to the patient. 3.94 4.56 0.61
Useful The note is extremely relevant, providing valuable information and/or analysis. 4.33 4.61 0.28
Organized The note is well-formed and structured in a way that helps the reader understand the patient’s clinical course. 3.89 4.67 0.78
Comprehensible The note is clear, without ambiguity or sections that are difficult to understand. 4.33 4.56 0.22
Succinct The note is brief, to the point, and without redundancy. 4.17 4.61 0.44
Synthesized The note reflects the AI scribe’s understanding of the patient’s status and ability to develop a plan of care. 4.28 4.50 0.22
Internally Consistent No part of the note ignores or contradicts any other part. 4.78 4.89 0.11
Free from Hallucination The note is free of hallucination and only contains information verifiable by the transcript. n/a 4.83 4.83
Free from Bias The note is free of bias and contains only information verifiable by the transcript and not derived from characteristics of the patient or visit. n/a 5.00 5.00
Average 34.56 37.06 2.50

This finding is consistent with staff experience. Of staff survey respondents, 88% felt that ambient listening produced a good quality clinic note. Respondents corroborated that ambient listening improved the amount of relevant detail available for patients and clinicians involved in their care.

My note is more extensive than I would have without [ambient listening] … more complete with it. – Medical officer; interview 6

Traditionally, I’d done very minimalist letters for NDIS because they take so much extra time, whereas now I’m providing a really comprehensive summary. – Medical officer; interview 2

Utility

ROUGE score analysis of 21 pairs of ambient-listening generated and final patient notes showed that, on average, 58% of AI-generated notes were used verbatim by clinicians in the final medical records. While ROUGE performance can be subjective and context specific, a median ROUGE F1 score of 0.58 indicates strong performance in capturing the essential content of outpatient conversations relative to other extractive summarisation research [14]. Table 3 displays an example comparative text produced by ambient listening technology and the corresponding text within a patient’s note, to illustrate how notes were amended and elaborated by clinicians throughout the trial.

Table 3.

Example excerpt of a comparative text with ROUGE score 0.624

Ambient listening excerpt Patient note excerpt

Plan:

• Repeat stool test to confirm negative C. diff status before ileostomy reversal.

• Colonoscopy scheduled for xx/xx/202x.

• Liaise with colorectal surgeon regarding timing of ileostomy reversal.

• Stop Nexium (esomeprazole) permanently.

• Avoid antibiotics unless absolutely necessary.

• Discussed risk factors for C. diff recurrence and measures to reduce infection risk.

Plan:

Given slight ambiguity about the symptoms, need for surgery and patient anxiety, repeat stool test to confirm negative C. diff status before ileostomy reversal.

Liaise with colorectal surgeon regarding timing of ileostomy reversal to discuss peri-op abs (and plan for amoxicillin)

Stop Nexium (esomeprazole) indefintiely.

Avoid antibiotics unless absolutely necessary.

Continue amoxicillin 250 mg PO daily as prophylaxis until surgery, stop whist on other antibiotics peripop then restart.

If patient continues to not have symptoms of infection and does not have more than prophylactic antibiotics peri-op, and has two negative stool tests prior, I would not give prophylactic for C.diff.

Discussed risk factors for C. diff recurrence and measures to reduce infection risk.

Across clinical specialties, those with a greater emphasis on patient history – particularly chronic disease management, such as endocrinology – had lower ROUGE score performance. To explain this, these clinicians reported that they routinely refer to historical inputs from a patient’s medical record history to inform their consultation and complete their notes. As this content is not necessarily discussed during the recorded consult, it is not captured by the AI scribe.

Before a consultation, if I’ve seen [the patient] I’ll summarise any previous diagnosis, any previous assessments they’ve had, and any updates in the last year that I’ve seen them …for a new patient I will scout around for any other information that I could collect by the patient before I bring them in. So there’s a little bit of a preparation work. – Medical officer; interview 2

Clinicians further reported that not all clinical information relevant to a patient’s medical record will be discussed - or discussed with the level of detail that care partners require to be informed of - requiring this detail to be added to the ambient-listening generated text.

There are certain parts I still have to add in because I don’t necessarily say everything. – Medical officer; interview 5

When we share CT reports and imaging reports … we say to the patient and ‘we found this and that’, but we give a sort of general outlining. The GP would want to know more precise terms. - Consultant 1

Reliability

When trialling the technology, 47% of staff survey respondents reported hallucinations in ambient listening scribe outputs. In an AI context, hallucination refers to when an AI solution generates text that is factually incorrect, misleading, or fabricated, while presenting false information as if it were true. Of those who reported hallucinations, 20% reported that they occurred frequently. These hallucinations pertained to undiscussed and irrelevant clinical information, i.e. medications and investigations, drug allergies, clinical measurements (weight, blood pressure), living arrangements, smoking status, and information about the care of an unrelated disease. Further, 16% of staff survey respondents observed bias in outputs. At times the technology made incorrect determinations on the relevance and meaning of clinical discussion with a tendency towards particular diagnoses and their typical features.

[The technology] jumped to conclusions that were not my own – Medical officer; survey 26

There are times in the clinic when …we talked about multiple things and [the Ambient Listening tool] emphasised the more minor problems. - Medical officer; interview 1

For some clinicians this deviation was within an accepted level and not a major limitation. Nonetheless, they emphasised a need to continuously review the AI-generated outputs before accepting them as the final medical record:

I think we need to [adopt AI] with caution, simply because it’s never going to be accurate 100% of the time. I know that busy doctors don’t check things, so my concern is that people won’t be checking what [the technology] is doing once [they] get used to it. … I think we should move forward with it, but with caution. And very clear ground rules. – Medical officer; interview 6

Other staff indicated that this unreliability was a significant barrier to their adoption of the technology.

One time … it came out with this whole spiel. I thought, ‘what’s all this stuff?’.… I don’t know whether it actually happened or not happened, and I’m just trusting what it wrote down if I can’t recall the whole consultation. …I thought “if I have to go through all this again, [then this] doesn’t really benefit me” because this was [meant to] save my time… [double checking] could take just as long as if you were writing it yourself. – Medical officer; interview 5

Staff experience

On average, clinicians used ambient listening during four consultations per week (Median = 2, range = 0 to 38). Graph 1 shows the variation in ambient listening trial participation and usage across outpatient specialities. Paediatrics, orthopaedics, and rheumatology had the largest number of participants. However, usage per participant was highest among endocrinology, neurology and paediatric clinicians. Notably orthopaedics had a high participation representation, but low usage.

Graph 1.

Graph 1

Outpatient specialty participants and usage

Prior to the commencement of the ambient listening trial, senior medical officers participating in semi-structured interviews reported that existing clinical documentation protocols carried a significant administrative burden and resulted in (1) a need for clinicians to work overtime, (2) hampered patient access to timely letters and information and (3) sub-optimal patient engagement during consultations. Together, these effects culminated in poorer work satisfaction.

I don’t like the word burnout. But from that point of view in terms of satisfaction, [it’s] just that lack of it. It’s not the same stress that people would have, say, if I’m going to run to do a CPR or something, but that stress level is much more of a low-grade constant kind of thing. - Medical officer; interview 7

Survey and interview feedback following the conclusion of the ambient listening trial indicate that, for most clinicians, these administrative pressures were alleviated by ambient listening technology. Of survey respondents, 84% reported that ambient listening technology had a positive impact on their efficiency, alleviating administrative burden and releasing time for other high-value tasks. 79% of staff survey respondents also reported that the technology improved the quality of consultations through increased focus on patients.

You no longer feel awful when you get the request for extra information between clinics. … That used to be like, ‘Oh my God, that’s so much work’. Whereas now I’m like, OK, look, … it’s much quicker. There’s no longer the dread. – Medical officer; interview 2

However, staff identified template customisation as an up-front investment required for the technology to meet the needs of their clinical specialty, which sometimes presented a barrier to use.

I had a go at making a template for consultant clinic and found it was (a) time consuming and (b) didn’t produce as good of a note at the end of my time. … I would have loved to sort this myself but realistically I didn’t have time to sit and tinker while patients are waiting in clinic. – Medical officer, survey

Layout, conciseness, and tone were frequently cited as requiring refinement. Subtleties across these dimensions reportedly had implications for clinical meaning, i.e. ambiguous assignment of responsibility for aspects of a patient’s care plan. Additionally, some summarisations removed important clinical detail, i.e. dates of events.

Patient experience

Patients reported a better appointment experience when ambient listening technology was in use compared to their experiences of usual care. Of patients surveyed, 68% reported that their clinician spent more time speaking directly with them, and less time than usual looking at the computer screen, compared to previous visits. 59% of patients reported that the ambient listening scribe had a positive effect on their visit. Concordantly, staff reported that this sentiment was reflected back to them by patients.

[Patients] are delighted for me to use [ambient listening]. I’ve had really good responses from people … [they know] that I’m going to use an approach that means I’m focused towards them and their needs. - Medical officer; interview 3

Medical officers expressed satisfaction that ambient listening technology gave them greater opportunity to act in more wholistic ways to improve patient care, including active listening and promotion of health literacy.

When I’m having the more sensitive conversations around domestic violence or some of the kids are really in some unsafe environments, I know that I can give better eye contact and capture the detail I want. – Medical officer; interview 2

I think it’s nice being able to interact with the patient more. I like that. – Medical officer; interview 6

[Ambient listening] gives me far more time to have a more complete consultation with the patient and form a better connection with them. … I can use more complex terminology at times with the patients [and] then I back it up with the lay person language as well. I think that actually helps the quality of the discussions that I have. – Medical officer; interview 3

Discussion

The findings from this evaluation indicate that ambient listening technology solutions can achieve sound performance in real-world outpatient settings according to measures of quality, utility and reliability; 58% of scribe outputs were accepted without modification into the electronic outpatient note. Ambient listening technology was associated with positive patient and staff experience. However, there is some evidence of hallucination. Qualitative and quantitative data in this study were internally consistent, and considering this data in the context of prominent health system challenges indicate ambient listening technology offers a potential solution to alleviate some these strains.

Addressing clinician burnout – characterised by emotional exhaustion, depersonalisation, and a diminished sense of accomplishment – has become a significant healthcare challenge [17]. A sizable contributing factor is the growing burden of documentation, which, in turn, has been exacerbated by the advent of electronic medical records [18]. Documentation requirements can place persistently high demands upon clinician time, evidenced by “pyjama time” whereby clinicians work beyond their scheduled hours to complete patient documentation [9]. A further consequence of high documentation demands is that clinicians perceive a diminished or compromised opportunity to provide therapeutic care and attention to their patient during consultation. This study corroborates existing evidence that AI-powered medical scribes can reduce documentation burden on clinicians and reduce pyjama time that contributes to burnout [19, 20]. Moreover, scribes can enhance the quality of the clinical encounter and facilitate higher levels of staff satisfaction with the quality of care they can provide [9, 11, 21, 22]. Surveys and interviews reveal that, for most staff, ambient listening technology alleviated burnout through two primary mechanisms (1) reducing the need for clinicians to work overtime and (2) enhancing clinician satisfaction by enabling more meaningful patient interactions. Ambient listening technologies represent a potential intervention to address some of the drivers of burnout and foster a more sustainable clinical environment.

Patient experience is significantly influenced by clinician engagement and health literacy levels, two interconnected factors [23, 24]. The outpatient setting offers a unique opportunity to enhance health literacy [25, 26]. Previous research has identified clinician time constraints as a primary barrier in accommodating variations in patients’ health literacy during consultations, thereby limiting opportunities for tailored communication and assessment of patient comprehension [27]. The results from this study find that ambient listening technology can support a more therapeutic patient-physician relationship in outpatient appointments by mitigating the burden of clinical documentation. This finding aligns with other studies investigating the impact of ambient listening technology on patient experience [9, 11, 21, 22].

The importance of accurate and reliable current clinical documentation for patient outcomes is well-reported in the literature [28]. This study highlights the potential of AI-powered medical scribes to enhance documentation quality through improved accuracy and comprehensiveness, in agreement with previous research [6, 7, 29]. Results of this study show that ambient listening technology can generate high-quality summaries of patient appointments with a reasonable degree of accuracy.

To realise the opportunities of ambient listening technology, this study corroborates two previously noted enablers for clinician adoption of ambient listening scribes, (1) technology usability, and (2) clinician interest [19]. Consistent with prior research, this study found that template customisation and familiarisation were central to usability, and, at times, were perceived as a barrier to use among time-poor clinicians [29, 30]. This study found clinician willingness to overcome this barrier was influenced by their medical note requirements and their perceptions of ambient listening scribe value. For instance, the high rates of diminished orthopaedic participation in the trial is explainable by specialty-specific clinical documentation workflows and requirements. Within the orthopaedics specialty, junior doctors typically completed the consult notes, with a preference towards brevity. While more comprehensive clinical note detail is frequently considered a positive improvement created by ambient listening technology, for some clinicians the duplication of content from previous notes could be seen as redundant and reducing clarity.

Furthermore, clinician interest was influenced by trust in ambient listening scribe outputs based on their observations of the frequency and potential consequences of hallucinations and bias. Hallucinations pose a significant risk to ongoing use and expanded adoption, particularly if clinicians neglect to thoroughly review and edit the generated notes and associated documentation [31]. Electronic medical record data showed that most staff identified the need to amend moderate amounts of the ambient listening-generated text before clinical use. There is the risk that trust and reliance on the tool – and subsequently complacency – will increase over time, which may perpetuate and amplify inaccuracies [28]. These findings underscore the critical importance of maintaining human oversight and frequently quality assurance of AI-assisted medical documentation processes [19, 20, 31].

Our study data indicate that hallucinations may arise from suboptimal prompt engineering used during note customisation. Staff identified that customisation was a challenging aspect of ambient listening adoption, and it is important to note that customisation comes with increased risk of hallucination.

In this study, bias was only observed in relation to the identification and emphasis of more common diseases and their symptoms. However, larger studies, with more data, are required to explore biases that may stem from social and structural determinants of health and inequitable dataset representation across different patient identities [31, 32].

Limitations

While data during this pilot was sufficient to gather a reliable indication of ambient listening scribe performance in an outpatient context, further research is required to explore its impact in other hospital settings (i.e., inpatient ward rounds, emergency department interactions). Furthermore, medical officers were the focus of this study, and further research is required to understand if the findings translate to other staff groups (i.e., nurses, allied health professionals, administrative and managerial staff). The perspective of clinicians who receive AI medical scribe-supported clinical notes is a gap in this study and may be addressed with future research. The internal consistency of multiple data sources lends weight to the validity of these findings. However, future research could benefit from larger sample sizes to strengthen the reliability of accuracy and quality analyses. This is particularly important given the variable response rates observed, which may introduce responder bias and social desirability bias. Additionally, as this study was conducted within a single centre, caution should be exercised when generalising the results to other operational contexts. Finally, the value proposition and health economic components were not evaluated, which is warranted by the implementation costs of this technology [33, 34].

Conclusions

This research studied ambient listening technology in real-world hospital outpatient practice, based on a 16-week trial among 100 staff and their patient consultations. Qualitative and quantitative data were internally consistent and demonstrated that ambient listening technology can (1) produce an accurate summary of outpatient appointments, (2) enhance clinical note quality and (3) improve both clinician and patient experience. This technology has the potential to offer a solution to address some of the most significant challenges facing healthcare systems, such as burnout, low patient health literacy and delayed or suboptimal patient documentation. However, this result is tempered by evidence of hallucination, and some potential bias, that requires attention prior to any further adoption in routine clinical practice.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary Material 1 (218.2KB, pdf)

Acknowledgements

We thank the doctors that participated in this study for their time and valuable insights, particularly Dr. Angela Owens for her efforts in refining the ambient listening solution for her local context in consultation with the vendor. We also acknowledge the dedicated efforts of the project team that delivered the initiative evaluated and supported the distribution of surveys.

Abbreviations

DRG

Diagnosis Related Group

AI

Artificial Intelligence

GCHHS

Gold Coast Hospital and Health Service (GCHHS)

Author contributions

R.S., A.B., and S.M. conceived this research. B.T. led the trial this study was based on. B.T., A.M where primarily responsible for data collection. R.S., S.M, A.B, B.T., and A.M contributed to analysis and manuscript finalisation. AM is responsible for the overall content as guarantor.

Funding

Resources to complete this study, including the time of the researchers, were funded by the Gold Coast Health Hospital and Health Service.

Data availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate

Darling Downs Hospital and Health Service Human Research Ethics Committee provided ethical approval for this study (HREC/2024/QTDD/108398). A waiver of consent was approved by this committee for the use of health service data. Informed consent was received for all staff interview and survey and patient survey participants. This study adhered to the study adhered to the Declaration of Helsinki.

Consent for publication

Not applicable.

AI use

Perplexity AI technology was used to search for relevant literature and improve the clarity of sentences.

Competing interests

The authors declare no competing interests.

Footnotes

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.Majeed A, Car J, Sheikh A. Accuracy and completeness of electronic patient records in primary care. Fam Pract. 2008;25(4):213–4. 10.1093/fampra/cmn047. [DOI] [PubMed] [Google Scholar]
  • 2.Mathioudakis A, Rousalova I, Gagnat AA, et al. How to keep good clinical records. Breathe. 2016;12:369–73. 10.1183/20734735.018016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Gesner E, Gazarian P, Dykes P. The burden and burnout in documenting patient care: an integrative literature review. Stud Health Technol Inform. 2019;264:1194–8. 10.3233/SHTI190415. [DOI] [PubMed] [Google Scholar]
  • 4.Gesner E, Dykes P, Zhang L, et al. Documentation burden in nursing and its role in clinician burnout syndrome. Appl Clin Inf. 2022;13 5:983–90. 10.1055/s-0042-1757157. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Ghatnekar S, Faletsky A, Nambudiri VE. Digital scribe utility and barriers to implementation in clinical practice: a scoping review. Health Technol. 2021;11(4):803–9. 10.1007/s12553-021-00568-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Tierney AA, Gayre G, Hoberman B, et al. Ambient artificial intelligence scribes to alleviate the burden of clinical documentation. NEJM Catalyst. 2024;5(3):CAT. 23.0404. [Google Scholar]
  • 7.Balloch J, Sridharan S, Oldham G, et al. Use of an ambient artificial intelligence tool to improve quality of clinical documentation. Future Healthc J. 2024;11(3):100157. 10.1016/j.fhj.2024.100157. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Jayakumar P, Oude Nijhuis KD, Oosterhoff JHF, et al. Value-based healthcare: can generative artificial intelligence and large Language models be a catalyst for Value-based healthcare? Clin Orthop Relat Res. 2023;481(10):1890–4. 10.1097/CORR.0000000000002854. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Tierney AA, Gayre G, Hoberman B, et al. Ambient artificial intelligence scribes to alleviate the burden of clinical Documentation. NEJM Catalyst. 2024;5(3). 10.1056/cat.23.0404.
  • 10.Duggan MJ, Gervase J, Schoenbaum A et al. Clinician experiences with ambient scribe technology to assist with Documentation burden and efficiency. JAMA Open. 2025. [DOI] [PMC free article] [PubMed]
  • 11.Galloway JL, Munroe D, Vohra-Khullar PD, et al. Impact of an artificial Intelligence-Based solution on clinicians’ clinical Documentation experience: initial findings using ambient listening technology. J Gen Intern Med. 2024;39(13):2625–7. 10.1007/s11606-024-08924-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Palinkas LA, Mendon SJ, Hamilton AB. Innovations in mixed methods evaluations. Annu Rev Public Health. 2019;40:423–42. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Stetson PD, Bakken S, Wrenn JO, et al. Assessing electronic note quality using the physician Documentation quality instrument (PDQI-9). Appl Clin Inf. 2012;3(2):164–74. 10.4338/aci-2011-11-ra-0070. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Akhmetov I, Mussabayev R, Gelbukh A. Reaching for upper bound ROUGE score of extractive summarization methods. PeerJ Comput Sci. 2022;8. 10.7717/peerj-cs.1103. [DOI] [PMC free article] [PubMed]
  • 15.Mishra P, Kiang JC, Grant RW. Association of medical scribes in primary care with physician workflow and patient experience. JAMA Intern Med. 2018;178(11):1467–72. 10.1001/jamainternmed.2018.3956. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Miles M, Huberman A, Saldana J. Qualitative data analysis. USA: Sage; 2019. [Google Scholar]
  • 17.Rotenstein LS, Torre M, Ramos MA, et al. Prevalence of burnout among physicians: A systematic review. JAMA. 2018;320(11):1131–50. 10.1001/jama.2018.12777. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Moy AJ, Schwartz JM, Chen R, et al. Measurement of clinical Documentation burden among physicians and nurses using electronic health records: a scoping review. J Am Med Inform Assoc. 2021;28(5):998–1008. 10.1093/jamia/ocaa325. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Shah SJ, Devon-Sand A, Ma SP, et al. Ambient artificial intelligence scribes: physician burnout and perspectives on usability and Documentation burden. J Am Med Inf Assoc. 2025;32(2):375–80. 10.1093/jamia/ocae295. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Misurac J, Knake LA, Blum JM. Impact of ambient artificial intelligence notes on provider burnout. 2024. 10.1101/2024.07.18.24310656 [DOI] [PMC free article] [PubMed]
  • 21.Ghatnekar S, Faletsky A, Nambudiri VE. Digital scribes in dermatology: implications for practice. J Am Acad Dermatol. 2022;86(4):968–9. 10.1016/j.jaad.2021.11.011. [DOI] [PubMed] [Google Scholar]
  • 22.Ayer M. Relieving administrative burden on clinical staff with streamlined workflows and speech-recognition software. Br J Nurs. 2023;32(Sup16b):S4–9. 10.12968/bjon.2023.32.sup16b.s4. [DOI] [PubMed] [Google Scholar]
  • 23.Parker R. Health literacy: a challenge for American patients and their health care providers. Health Promot Int. 2000;15:277–83. 10.1093/HEAPRO/15.4.277. [Google Scholar]
  • 24.Wynia M, Osborn C. Health literacy and communication quality in health care organizations. J Health Communication. 2010;15:102–15. 10.1080/10810730.2010.499981. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Schillinger D, Piette J, Grumbach K, et al. Closing the loop: physician communication with diabetic patients who have low health literacy. Arch Intern Med. 2003;163(1):83–90. 10.1001/archinte.163.1.83. [DOI] [PubMed] [Google Scholar]
  • 26.Rohwer E, Mojtahedzadeh N, Neumann FA, et al. The role of health literacy among outpatient caregivers during the COVID-19 pandemic. Int J Environ Res Public Health. 2021;18(22):11743. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Murugesu L, Heijmans M, Rademakers J, et al. Challenges and solutions in communication with patients with low health literacy: perspectives of healthcare providers. PLoS ONE. 2022;17(5):e0267782. 10.1371/journal.pone.0267782. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Weis JM, Levy PC. Copy, paste, and cloned notes in electronic health records: prevalence, benefits, risks, and best practice recommendations. Chest. 2014;145(3):632–8. 10.1378/chest.13. [DOI] [PubMed] [Google Scholar]
  • 29.Joseph J, Moore ZEH, Patton D, et al. The impact of implementing speech recognition technology on the accuracy and efficiency (time to complete) clinical Documentation by nurses: A systematic review. J Clin Nurs. 2020;29(13–14):2125–37. 10.1111/jocn.15261. [DOI] [PubMed] [Google Scholar]
  • 30.Ghatnekar S, Faletsky A, Nambudiri VE. Digital scribe utility and barriers to implementation in clinical practice: a scoping review. Health Technol (Berl). 2021;11(4):803–9. 10.1007/s12553-021-00568-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Mess SA, Mackey AJ, Yarowsky DE. Artificial intelligence scribe and large Language model technology in healthcare documentation: Advantages, Limitations, and recommendations. Plast Reconstr Surg Glob Open. 2025;13(1):e6450. 10.1097/GOX.0000000000006450. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Pfohl SR, Cole-Lewis H, Sayres R, et al. A toolbox for surfacing health equity harms and biases in large Language models. Nat Med. 2024;30(12):3590–600. 10.1038/s41591-024-03258-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Shah NH, Entwistle D, Pfeffer MA. Creation and adoption of large Language models in medicine. JAMA. 2023;330(9):866–9. 10.1001/jama.2023.14217. [DOI] [PubMed] [Google Scholar]
  • 34.Kotek H, Dockum R, Sun DQ. Gender bias and stereotypes in large Language models. ArXivorg. 2023. 10.48550/arxiv.2308.14921. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Material 1 (218.2KB, pdf)

Data Availability Statement

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.


Articles from BMC Health Services Research are provided here courtesy of BMC

RESOURCES