Thinking critically about AI documentation quality in primary care

The introduction of artificial intelligence (AI)-driven clinical documentation has taken healthcare by storm. According to many leaders and clinicians, there has never been a technology that has been so impressive and rapidly adopted by clinicians.¹ Arising on the soil of wide-spread clinician burnout and frustrations, including with the time consumed by charting clinical encounters, AI vendors have developed modules that facilitate clinical documentation.² These tools work by recording the encounter (with patient consent) as a digital audio file, which is then transcribed using speech-recognition technology, and then processed by AI to produce a formatted note for the clinician to review and sign. This perspective piece is written by a primary care clinician and patient safety researcher using AI documentation for nearly 2 years (GS), and a PhD communication specialist (MK) directly observing hundreds of encounters with this physician.

From a patient safety perspective, especially diagnostic safety, this new technology offers a myriad of improvement opportunities for transforming patient-clinician interactions and documentation workflow.³ Given the important role clinical notes play in formulating, recording and communicating key information, we argue that healthcare must go beyond being dazzled by the magic of instant note production to more critically evaluate this leading healthcare AI application. We identify key issues that warrant addressing and suggest potential mitigation strategies (table 1) keeping in mind a historical perspective of past health information technologies shortcomings/disappointments (eg, clinical decision support, problem lists, copy/pasted notes, diagnosis software).^{4 5} We examine several issues that have arisen in our work with AI-documented notes, as well as touch on broader issues raised by this new technology.

Table 1.

Issues warranting addressing for AI-clinical documentation

Issue	Potential strategies to address
Need to better characterise qualities of a good clinical note, particularly related to diagnostic assessment.	► Convening discussions and promoting research among clinicians, educators, patients and quality, safety and informatics experts to leverage AI to better define, standardise and optimise note content, organisation and workflow efficiencies.
Balance note production efficiency versus clinician tailoring of notes in own voice. Reliance on assessment plans generated by AI may contribute to cognitive biases and diagnostic errors.	► Time-saving/efficiencies of AI notes should be partially reallocated to enable dedicated reflection of clinical encounters and crafting/editing time. ► Use of direct dictation of assessment (eg, by clinician using voice recognition after visit) rather than relying on AI capture during visit. User-friendly ways to customise AI note templates to match the clinician’s narrative style and practice needs.
Errors in AI notes: how to minimise, detect and mitigate.	► While generally rare, occasionally serious; requires vigilance to ensure careful MD review of notes before signing. ► Potential of advanced AI review tools to help detect and correct. ► Patients can play a role in identifying errors in notes, thereby improving note accuracy and patient safety.22
How will trainees learn to write good notes? Risk of atrophy of note-writing skills.	► Standardised curriculum and training requirements (including minimum number of supervised notes) on clinical documentation best practices before learners access AI to generate notes.23 ► Developing and teaching new paradigms for how to write notes in the AI era.
Omission of important elements (eg, richer social history (‘chit chat’), physical examination general description, diagnostic assessments) that may not be verbalised during encounters.	► Vendor attention to this design shortcoming. ► Workflow during and after encounters to ensure missing elements are captured. Retraining clinicians to better verbalise for patients during encounters and ways to add post encounters.24
Loss of information continuity from prior notes if starting with blank AI notes each visit.	► Listening to and understanding clinicians who copy-pasted prior notes. ► Engineering AI note encounter workflow (including prepopulating information) to ensure prior problems are not lost.
Limitations of ‘one-size fits all’ AI notes	► Deployment of new AI features that allow customisable templates, editing, pausing to insert dictated corrections. ► Making such customisation easier for users to unlock such features.
Balance/optimise note comprehensiveness and succinctness to achieve readability and usefulness. Achieve patient-friendly language while preserving medical precision/accuracy.	► Collapsible note synoptic sections to view more versus less detail. ► Leverage AI capabilities to produce note summaries and versions tailored to patient’s health literacy level.
Patient privacy and trust (eg, risk of privacy breaches, erosion of patient trust with robotised AI notes)	► Enhanced data privacy protections (regulatory, institutional, monitoring). ► Attention to AI note language and careful clinician editing of the AI-drafted notes before signing off is especially important in the era of open notes. ► User-friendly and responsive systems for patient feedback on any concerns/questions they may have regarding the content of their notes.
Dominance of for-profit commercial products limiting cooperative learning.	► Public reporting of errors and transparency about shortcomings, including collating experience and understanding why some clinicians who have tried (often briefly) but abandon AI documentation. ► Regulation and oversight of standards.
Climate impacts of AI electricity and water demand.	► While not unique to AI documentation, quality and safety informed healthcare need to be mindful of environmental impacts, thus work with broader efforts to minimise harms.25
AI, artificial intelligence.

In 2021 two key developments threw the doors wide open for rethinking outpatient clinical notes in the USA and beyond. The first was a revision of coding requirements by the US Centres for Medicare and Medicaid which previously required documentation of a specific number of history and physical examination components. The new regulations simplified note-writing, making notes less about check-box documentation, thereby providing flexibility for more meaningful narrative and recording of medical decision-making. This liberated notes from prior constraints and provided an opportunity to rethink and improve recording of diagnostic assessments and plans.⁶

A second change in the USA, with similar movements in countries in the Americas, Europe, Africa and Asia, mandated opening notes to patients— giving patients access to their medical record and clinical notes.^{7 8} No longer were notes solely for communication among healthcare staff or documentation as a ‘cover’ against malpractice liability or for physician reimbursement justification. Rather, each note also serves as an important communication channel with the patient. Thus, verbal explanations and instructions, often easily forgotten in the stress of an encounter, can now be later reviewed by patients and families.⁹ Notes need to be more sensitive to language (especially stigmatising language), patients’ health literacy and anxieties, the nuances of diagnostic uncertainty and patient-centred follow-up instructions.

IMPACT ON CLINICIAN REFLECTION AND COGNITIVE PROCESSES

Although this transformed role of clinical notes afforded an opportunity to rethink note content and documentation workflow, clinicians are mainly being driven by feeling overburdened by electronic medical records (EMRs) and overall productivity demands. From a diagnosis safety standpoint, we have an opportunity to rethink and redesign notes—especially the assessment section—to make it richer in terms of reflecting differential diagnosis, weighing uncertainties and probabilities, grappling with urgency and crafting and operationalising follow-up plans. Instead, clinicians have sought relief and shortcuts through approaches such as copy/pasting older notes, use of templates and human scribes to offload documentation burden.¹⁰ Now, commercial vendors have seized on advances in speech recognition and large language model AI to create AI-documentation tools to streamline note-writing worldwide.¹¹

Beyond overlooking the opportunity to collectively, rigorously and intelligently redesign notes for safety, quality, efficiency and better communication, we now have a new set of challenges. While AI documentation competently compiles what is said aloud in the examination room (with some exceptions below), something important is also lost—the time for the clinician to reflect on and craft their assessment. Previously, this typically occurred after the patient left the examination room, often later in the day or at night, perhaps after test results had returned. While the miracle of having a completed note written within a few seconds after the encounter is an undeniable time-saver for the clinician and a gift for more family time at home, it might also be viewed as a theft of reflection time from the patient. Although there is nothing preventing clinicians from taking the time to carefully edit the AI notes to bring back such reflection moments, we suspect that the temptation to quickly sign the note will be irresistible.¹

Relatedly, a more profound problem that we have observed in the notes the author (GS) has been ‘writing’ is the complete absence of an assessment—generally the most important component of the note.⁶ The AI tool capably summarises the history and the plan, but many of the elements of an actual diagnostic assessment may not be spoken aloud (or in some cases the AI discards them). A quick sign-off of the AI note can overlook that the ‘assessment/plan’ of an AI note does not contain any real ‘assessment’. We illustrate two examples (figure 1) of AI notes lacking any assessments that the author (GS) almost quickly signed. These AI-generated notes that initially lacked any assessment; instead, they had to be added manually. Once this assessment reflection space disappears, how will trainees and clinicians learn to craft and maintain good assessments—skills that were historically honed by repeatedly practising writing thoughtful notes.

Figure 1.

Illustration of lack of meaningful assessment in ‘assessment/plan’ section of AI-generated note requiring manual addition. Current AI-generated notes basic templates have three main sections: history, physical examination, assessment/plan. The left-hand column is a screenshot of the assessment/plan section from two AI-Generated Notes illustrating that there is no actual assessment of the patients’ problems. The right-hand column illustrates the revised notes with the added text (in bold) manually revised by the clinician to add meaningful assessment of the problem. AI, artificial intelligence.

OTHER CONSIDERATIONS OF AI-DOCUMENTED NOTES

We have also noticed omissions of other important elements of a clinical encounter. For example, aspects of the physical examination such as general description (eg, patient is disoriented, confused, has pressure of speech, was anxious or inappropriate, appears chronically ill, in mild respiratory distress) are unlikely to be spoken aloud in front of the patient and risk disappearing from examination documentation. And as discussed elsewhere,¹² there is important psychosocial history information that the AI bot has decided is unimportant ‘chit-chat’. Instead, the AI tool often eliminates such ‘chit-chat’, despite this being information primary care clinicians would (and should) record given its importance for relationship building.¹³ Examples of such information eliminated from the author’s own notes include: patient reporting their daughter has overdosed on drugs, planned a trip to Haiti, or having a stressful job situation—topics important to ask about at the next visit inquiring how their daughter was doing, how was their trip, or were things any better on the job.¹²

Another unresolved issue with AI-documented notes is the problem of how to incorporate older historical data. Currently, many clinicians copy and paste their prior notes to maintain continuity from past visits. While some disparage this copy/pasting (claiming it is a lazy shortcut, brings forward out-of-date information, adds length/‘note bloat’), it serves an important function in creating a thread to carry forward important prior information to ensure it is not overlooked.¹⁴ With AI documentation, while copy/paste could still work, the recommended best practice is to start with a blank slate and let the AI produce a clean note each visit. Otherwise, the AI notes become cluttered and redundant, and the AI tool we use has trouble figuring out where to insert the new encounter information to make it clear what is new versus old. Thus, these AI notes might be characterised as a ‘note without a memory’. This has led many of our specialist colleagues, for example, oncologists, who wish to have a running thread of patients’ cancer history and treatments, to prefer their current manual charting approaches over AI note-taking.

Two final issues warrant additional consideration for safety and quality. The first is the readability and usefulness of clinical notes. Unfortunately, even the most carefully crafted (by human or machine) notes are useless if not used. Here we refer to being read by both other clinicians and patients. Studies show that many notes are never read. For example, when patients present to the emergency department (ED), although primary care clinicians’ notes may contain important information, busy ED doctors often do not review these notes.¹⁵ And patients, despite now having access in many jurisdictions, may not open the notes or have problems understanding the language. We should not lose the opportunity to explore what AI documentation can do to help address these problems, perhaps by deploying summarisation tools (for the ED doctors) or customising note language to align with patients’ health literacy or spoken language to minimise disparities.

Finally, there is the problem of patient trust. Trust is an essential element in healthcare.¹⁶ If notes contain errors (which AI notes occasionally do, with added risk when clinicians skip careful proofreading before signing), it is easy to see how already declining levels of trust might be further diminished. More subtly— and more importantly—when patients read notes that lack the personalisation of the language their own clinicians might use to craft the note, how will they respond? Deference to AI-generated phrasing risks patients perceiving a disregard for nuances or subtleties a physician might write based on patients’ individual preferences and understandings.

LACK OF TRANSPARENCY LIMITING LEARNING IN COMMERCIALISED AI PRODUCTS

Addressing the questions we raise here calls for concerted efforts to study and learn from how this AI tool can be most safely and wisely deployed. At the very least, the metrics evaluating these tools need to get beyond how many minutes it saves clinicians or clinician satisfaction or burnout ratings.^{1 17} Much of the knowledge about how these systems work, the errors they make and opportunities for improvement currently reside exclusively with the commercial vendors.¹⁸ There needs to be much more transparency to compare the various products and test their claims and performance in the real world. Going beyond proprietary black boxes, there needs to be transparent learning and feedback at multiple levels, from the clinicians and patients, but also what the vendors learn as they compare their notes to the encounters’ voice files, speech-generated transcripts, the AI-drafted notes and clinician edits to each note. These companies are now engaged in mining and analysing this data, but our health systems, clinicians and researchers currently lack access to this data and analyses. We need to be able to compare performance of the competing products in standardised ways, lest the least expensive, though perhaps most flawed, product be foisted on financially strapped institutions and clinicians.

WAY FORWARD

Independent researchers need to be funded and given access to data to evaluate how AI-documentation products are performing. Clinicians will require training on verbalisation practices during encounters to facilitate conveying and capture of important information.⁶ We need to apply systems engineering, human factors, ethnographic and qualitative research to study work flows and outputs. Even simply studying the clinicians that have tried but abandoned using AI documentation,¹⁹ or patients who either do or do not like AI-generated notes, would be extremely valuable.²⁰

Notes are a critical cognitive and communication component of healthcare safety. To what extent have clinicians lost control over the means of production of their notes and how can we take them back to ensure they are used for better, safer care, improved thinking and better communication? Recognising shortcomings in ‘old fashioned’ EMR notes²¹ cannot be an excuse to not proactively take responsibility for improving AI notes. The next generation of AI documentation will be orders more powerful. This future will likely include vendors not only being able to listen to our conversations but also reading our prior notes to further enhance their products’ deep learning.¹⁸ We need to be deeply learning as well.