Improving the Quality of Electronic Medical Record Documentation: Development of a Compliance and Quality Program

Abstract

Background  Introducing an electronic medical record (EMR) system into a complex health care environment fundamentally changes clinical workflows and documentation processes and, hence, has implications for patient safety. After a multisite “big-bang” EMR implementation across our large public health care organization, a quality improvement program was developed and implemented to monitor clinician adoption, documentation quality, and compliance with workflows to support high-quality patient care.

Objective  Our objective was to report the development of an iterative quality improvement program for nursing, midwifery, and medical EMR documentation.

Methods  The Model for Improvement quality improvement framework guided cycles of “Plan, Do, Study, Act.” Steps included design, pre- and pilot testing of an audit tool to reflect expected practices for EMR documentation that examined quality and completeness of documentation 1-year post-EMR implementation. Analysis of initial audit results was then performed to (1) provide a baseline to benchmark comparison of ongoing improvement and (2) develop targeted intervention activities to address identified gaps.

Results  Analysis of 1,349 EMR record audits as a baseline for the first cycle of EMR quality improvement revealed five out of nine nursing and midwifery documentation components, and four out of ten medical documentation components' completion and quality were classified as good (>80%). Outputs from this work also included a framework for strategies to improve EMR documentation quality, as well as an EMR data dashboard to monitor compliance.

Conclusion  This work provides the foundation for the development of quality monitoring frameworks to inform both clinician and EMR optimization interventions using audits and feedback. Discipline-specific differences in performance can inform targeted interventions to maximize the effective use of resources and support longitudinal monitoring of EMR documentation and workflows. Future work will include repeat EMR auditing.

Background and Significance

Electronic medical records (EMRs) are comprehensive health information systems that have recently replaced paper-based records and manual processes for clinical information and patient care documentation in many large Australian health care organizations. Introducing EMR systems into complex health care environments fundamentally changes the work, workflows, and documentation processes of nursing, midwifery, and medical staff. 1 EMR systems are expected to provide improved patient care by providing repositories for comprehensive clinical information and facilitating inter- and intradisciplinary communication, clarity of and access to clinical information, and improved medication safety. However, these new systems and workflows must be accepted and adopted as intended by clinicians in order for expected benefits to be achieved. 2 3

Effective adaptation of international vendor-built EMR programs and systems to the Australian health care environment requires extensive consultation with end users to ensure that workflows, language, and setup of the system are suitable for use by clinicians. 4 Due to the necessary adaptation of systems, each health care organization or jurisdiction is expected to individually tailor EMR system documentation needs. Other requirements for EMR acceptance and adoption by clinicians include ensuring workflows and hardware fit local requirements.

At the study sites, this was achieved by developing EMR device principles, 5 monitoring workflow compliance and documentation completeness using clinician feedback and small, discipline-specific assessment of EMR documentation (not published).

The EMR was developed in conjunction with clinicians across the health care organization and adapted for the Australian context for implementation in 2019. Documentation and workflows within the EMR were designed to align with the health care organization's workflows, policies, procedures, and Australian National Safety and Quality Health Service Standards (National Standards). The National Standards is a framework that supports health care organizations to meet and deliver expected safe and quality care standards nationally. 6 The National Standards also provided useful criteria to guide EMR content and workflows at implementation.

Despite significant work to tailor EMR systems to meet organizational context and needs, many health care organizations have encountered issues with EMR workflow compliance, largely due to the increased documentation burden on clinicians. 7 Examination of EMR documentation burden and quality has most often focused on facilitating EMR use by clinicians by improving workflows or examining condition- or process-specific documentation. 8 9 EMR usability is multifactorial; hence, usability and adoption must be examined in the context of multiple and complex work demands on clinicians. Improvement strategies need to consider human responses to using EMR in their work to ensure sustained delivery of safe and high-quality patient care. 10

In order to examine documentation compliance and quality of a newly implemented EMR system throughout our health care organization, an iterative, ongoing quality improvement program was developed. 11 The Model for Improvement quality improvement framework incorporates three questions (“What are you trying to accomplish?,” “How will you know a change is an improvement?,” and “What change can you make that will result in improvement?”) and cycles of “Planning, Doing, Studying and Acting” 12 . This framework was selected as it has been successfully used in health care settings and provides a clear framework for customizable and iterative quality improvement projects. 13 The cycle allowed the development of an iterative process, specific to our setting, and was imperative to ensure model practicality and measurement development. 14 The multidisciplinary (nursing, midwifery, and medical) informatics teams were included at each cycle stage.

Significance

The implementation of an EMR system dramatically changes clinicians' work and workflows. EMR adoption can be variable and may lead to unintended consequences. Planning for EMR implementation and ongoing optimization must consider human behaviors and change management to support clinician adoption of new documentation processes. To ensure sustained quality of patient care delivery, adherence to changed workflows by clinicians must also be supported.

Objective

The aim of this study was to develop an iterative quality improvement program to address nursing, midwifery, and medical EMR documentation by clinicians.

Methods

The Model for Improvement quality improvement framework guided this iterative study. The multidisciplinary informatics team worked together to answer the three foundational questions of the framework and outline the cycle steps of planning, doing, studying, and acting. 12

• What are you trying to accomplish: The objective was to develop an iterative quality improvement framework to assess and support EMR clinician documentation.

• How will you know a change is an improvement: There was a need to develop and undertake an audit to obtain benchmarking data to evaluate clinician documentation post-EMR implementation (as part of the ongoing quality improvement project).

• What change can you make that will result in improvement: Ongoing education and training interventions to address specific gaps in EMR documentation.

Planning: Design, Pre- and Pilot Testing of an Audit Tool to Capture Documentation Completeness and Quality within the Health Care Organization

The EMR audit tools were developed by the informatics teams to focus on key areas of organizational risk and EMR workflows aligned with the National Standards. 6 Nursing and midwifery audits focused on the documentation of inpatient initial admission assessment, risk assessments, care plans, handover, rounding and discharge planning, as well as nutrition, fluid balance, blood or blood products, and medication administration. These components aligned with four of the National Standards: (1) medication safety, (2) comprehensive care, (3) communicating for safety, and (4) blood management. The medical audit focused on EMR documentation of diagnoses, problem list completion, goals of care, allergy documentation, admission documentation, inpatient reviews, operative notes, discharge documentation, medication reconciliation, venous thromboembolism care pathway, results endorsement, and emergency department presentation and discharge. These aligned with three National Standards: (1) medication safety, (2) comprehensive care, and (3) communicating for safety. The nursing and midwifery and medical audit questions, as well as corresponding National Standards, are presented in Tables 1 and 2 , respectively.

Table 1. Nursing and midwifery audit questions and corresponding National Standards.

Concept	Questions	Corresponding National Standard
Inpatient assessment	• Completion of required documentation  • Timing of completed documentation postadmission to the ward	Standard 5: Comprehensive Care
Risk assessments	Completion of required risk assessments as per patient's clinical condition, nursing assessment or age  • Falls risk  • Nutrition risk screen  • Cognitive impairment  • Pressure injury risk Timing of risk assessments completion in relation to admission
Care plans	The presence or absence of any suggested care plans (from risk assessments)
	Number and type of care plans were active for the patient  • Number of interventions (including 'not done' interventions and patient-specific documentation)  • Variances documented against interventions as necessary
Handover	Evidence of nurse–nurse handover documented within last 24 h	Standard 6: Communicating for Safety
Rounding and nursing care	• Nurse rounding within last 16 h (including introducing self, pain assessments, interventions related to risk assessments, environmental scans, assistance with patient care activities, call bell within reach)  • Nursing care documented including appropriate workflows for patients who have had a fall in hospital, patients with wounds, patient mobility documented,  • The presence or absence of a nursing note	Standard 5: Comprehensive Care
Nutrition	Appropriate documentation related to patient nutrition including diet orders
Lines and devices	Adherence to workflow and appropriate documentation completed for each line or device present (insertion, site condition, and removal)
Fluid balance chart	Appropriate documentation related to patient input and output on fluid balance chart
Blood or blood product administration	Adherence to workflow for blood or blood product administration	Standard 7: Blood Management
Medication administration	Adherence to workflows for medication administration (with and without barcode scanning)	Standard 4: Medication Safety
Early discharge planning	Appropriate early discharge planning documentation completed	Standard 5: Comprehensive Care and Standard 6: Communicating for Safety

Table 2. Medical audit questions and corresponding National Standards.

Concept	Questions	Corresponding National Standard
Use of key shared repositories of clinical information	Problems list (diagnoses and problems)	Standard 6: Communicating for Safety
	Recording diagnoses on discharge  • Use of the principal diagnosis being documented on discharge summary
	Recording diagnoses and chronic conditions in a patient's chart  • Chronic medical problems and this visit diagnoses updated into central repository vs. free text into notes only
	Treatment limitations or lack thereof documented within 24 h of a patient's admission	Standard 5: Comprehensive Care
	Complete documentation of patient's allergies or ADRs (description and all fields completed)	Standard 4: Medication Safety
Documentation activities	Emergency department presentation and discharge	Standard 6: Communicating for Safety
	Inpatient/Critical care admission, inpatient stay (review) and discharge  • Admission notes clearly documented with a plan  • Ward rounds notes (review of patients during inpatient stay) documented regularly with a plan  • Discharge summaries completed in a timely manner with the correct note (to be sent electronically), and including a principal diagnosis, discharge medication reconciliation and plan
	Theatre operation notes and codes  • Operation notes documented in correct workflow, with the indication(s), codes and post-operative plan present
Medication reconciliation	Home medications documentation for patients on arrival to key areas (emergency department, inpatient wards, critical care)	Standard 4: Medication Safety and Standard 6: Communicating for Safety
	Transfer medication reconciliation completion for those stepping down from critical care to other acute areas
Use of venous thromboembolism care pathway	Use of venous thromboembolism care pathway  • Utilizing the care pathway to make an assessment of a patient's risk of venous thromboembolism	Standard 5: Comprehensive Care
Results endorsement	Completion of this workflow to formally acknowledge diagnostic imaging results	Standard 6: Communicating for Safety

Abbreviation: ADRs, adverse drug reactions.

Pretesting of the audit tools included clinician, researcher, and informatics experts examining each item for clarity for auditors (e.g., preference for simple yes/no responses and counting), relevance to the workflows of interest, and usability of the EMR documentation forms, capturing documentation adherence with organizational policies and procedures and understanding adoption of workflows. Changes enhanced item clarity and consistent use between auditors (i.e., wording of questions and guidance about where to look within EMR). Pilot testing assessed item order and consistency of data captured across auditors, with no changes required.

Doing: Undertaking a Retrospective Audit of Nursing, Midwifery, and Medical Documentation within the Electronic Medical Record

All audit data were collected retrospectively by trained (nursing, midwifery, and medical) clinician superusers and subject matter experts; these expert EMR users had supported EMR implementation and provided ongoing staff support. The training was conducted in small groups either in-person or online depending on clinician availability and included the demonstration of where to find the required clinical documentation for the audit questions within EMR, providing clinicians with a data dictionary of where to look for the documentation within the EMR. These same clinicians undertook the pre- and pilot-testing of the audit.

All audit data were nonidentifiable, retrospective, and captured using handheld devices in a secure form. Audit data were analyzed for quality (i.e., data fields completed/omitted) and completeness (i.e., frequency of expected forms used/completed) in a patient's EMR documentation.

Inclusion and Exclusion Criteria

Audit data were collected from randomly selected EMR records of patients present in wards at the time of the audit. A random number generator was used to select the beds for the patient record audit. Audits involved patient records from wards and departments on six acute hospital sites. Care settings included critical care, acute medical, surgical, and emergency settings. The nursing and midwifery audits excluded operating theatre as hybrid computer and paper EMR workflows were in place. Medical audits excluded mental health as the EMR workflow differed from other care settings. Both the nursing and midwifery and medical audits excluded records of inpatient stays less than 24-hour duration (reduced documentation) and maternity areas (hybrid paper and EMR documentation workflows).

Studying: Analysis of Audit Results as Baseline Measures for Informing Expected Standards of Documentation and Set-Up Benchmarking for Comparison in Future Work

Results analysis was completed by two members of the research team who had not completed the audit. Statistical analyses (frequencies and descriptives) were completed using IBM Statistical Package for the Social Sciences (Version 27) for Windows.

Acting: Development of Targeted Education and Training Activities

Based on the audit results and gaps identified in EMR documentation, targeted education and training activities were developed and delivered both in-person and online to capture as many staff as possible.

Study Setting

A large Victorian tertiary public health care organization that provides care across the entire lifespan was the setting for this study. A “big-bang” style EMR system implementation (i.e., change from paper to EMR system for all inpatient clinical documentation, medications, orders, and scheduling) was used at three-time points across seven hospital sites between August and November 2019. Nursing, midwifery, and medical informatics teams collaborated on governance, communication, education, training, and research to support adoption and sustainability as the EMR system was embedded into the organization.

Ethics Approval

Ethics approval was granted by the healthcare organization (reference numbers QA/64509/MonH-2020-213787(v1) and QA/76441/MonH-2021-266966(v1)).

Results

Audit Results

Of the 1,349 records audited between May and August 2020, 629 were nursing and midwifery documentation audits (audit completed over 12 weeks) and 720 were medical documentation audits (audit completed over 4 weeks). The duration of auditing timeframes was determined by the availability of clinical staff to conduct the audits.

Audit data were reported using the percentage of documentation quality (fields completed) and completeness (forms used) of the questions (detailed in Tables 1 and 2 ) as follows: good (80% or higher), moderate (60–80%), and poor (less than 60%). The nursing, midwifery, and medical informatics teams, in collaboration with the training and adoption team, determined the percentage ranges for documentation quality and completeness for ease of understanding and benchmarking within the large organization. The nursing and midwifery audit results are presented in Table 3 . Five components of nursing and midwifery documentation were rated as good: care planning (88%), lines and devices (85%), blood or blood products documentation (91%), handover (84%), and rounding (83%). Diet orders (69%), fluid balance charting (60%), and early discharge planning (64%) were rated as moderate, and one component was rated as poor (risk assessments [21%]). Two components were assessed on documentation quality as well as timeliness: inpatient assessments were initiated in a timely manner; however, their rates of completion were poor (33%), while the documentation of medication administration documentation was good (91%) and use of patient barcode scanning was moderate (73%).

Table 3. Nursing and midwifery audit results.

Audit component		Score	Grade (good >80%, moderate 60–80%, poor <60%)
Inpatient assessment	Documentation	33%	Poor
	Timeliness	>80%	Good
Risk assessments documentation		21%	Poor
Care plans documentation		88%	Good
Handover documentation		84%	Good
Rounding and nursing care documentation		83%	Good
Early discharge planning documentation		64%	Moderate
Nutrition—diet orders documentation		69%	Moderate
Lines and devices documentation		85%	Good
Fluid balance chart documentation		60%	Moderate
Blood or blood product administration documentation		91%	Good
Medication administration	Documentation	91%	Good
	Patient barcode scanning	73%	Moderate

The medical audit results are presented in Table 4 . Four components of the medical audit were rated as good: allergies with mandatory field documentation (98%), inpatient review documentation (99%), admission documentation workflow (81%), and operation documentation workflow (88%). One component was rated moderate (discharge documentation workflow [72%]), and five components were rated as poor: venous thromboembolism pathway workflow (42%), medical reconciliations workflow (<50%), problems list documentation workflow (65%), goals of care documentation workflow (61%), and diagnostic imaging results requiring endorsement (68%).

Table 4. Medical audit results.

Audit component		Score	Grade (Good >80%, Moderate 60–80%, Poor <60%)
Use of key shared repositories of clinical information	Problems list documentation workflow	65%	Moderate
	Goals of care documentation workflow	61%	Moderate
	Allergies with mandatory fields documentation	98%	Good
Documentation activities	Admission documentation workflow	81%	Good
	Inpatient reviews documentation	99%	Good
	Operation documentation workflow	88%	Good
	Discharge documentation workflow	72%	Moderate
Medication reconciliation	Medical reconciliations workflow	<50%	Poor
Use of venous thromboembolism care pathway	Venous thromboembolism pathway workflow	42%	Poor
Results endorsement		68%	Moderate

Development and Delivery of Targeted Training Interventions

The areas that scored poorly in the audits were the foci for targeted improvement interventions. Specific education and training interventions included discipline-specific communications to managers and EMR superusers throughout the organization; a dedicated period of in-person supernumerary shifts (including EMR subject matter experts, EMR trainers, and members of informatics teams) across all six acute hospital sites and clinical areas to answer questions and address specific workflows that scored poorly for each discipline and answer questions from clinical staff; review and update of existing education and training materials on the organization's learning portal and intranet sites for ease of use and clarity; and review and update of new staff member education and training materials.

A framework was also developed collaboratively between the informatics, education, training, and adoption teams. The framework identified five essential elements to enhance EMR documentation adoption and quality.

• Communication: Communication includes informing health care organization staff (both clinical and managerial) about key performance gaps, proposed solutions, and available EMR resources via screensavers, clinical, and educational forums, and manager-specific communications.

• EMR Optimization: EMR optimization included using the audit findings as opportunities to review and evaluate EMR design and build. This included clarity of information and optimization of workflows, as well as minimization of documentation requirements, was possible (e.g. elimination of duplication).

• Targeted training and adoption activities: Targeted training and adoption activities elaborated on the foundational EMR training and online resources for staff and were used to empower staff with the skills and capabilities required for optimal EMR use. Programs developed included ongoing learning to address EMR documentation completeness and quality gaps identified in the audit results. Training and adoption materials were made available to all staff via the organization's intranet and education portals.

• User support: User support involved clinicians working collaboratively at executive, managerial, and direct-care levels to continuously assess staff needs and redirect resources and support as required.

• Governance, engagement, and ownership: Governance, engagement, and ownership involved building on this study's development of key priorities for EMR documentation and quality improvement, engaging with and empowering key enablers, and embedding EMR excellence through role modeling and clinical dialogue.

To support the five elements of this new framework and track ongoing compliance, an EMR data dashboard was also established to monitor quality and compliance rates for nursing, midwifery, and medical EMR documentation expectations alongside the National Standards. The dashboard extracts EMR clinical documentation data related to the audited areas to examine documentation completeness and quality. The dashboard, therefore, supports sharing of information on clinicians' use of the EMR to the informatics, training, and adoption teams, clinical managers, and educators, as well as executive and leadership teams. Furthermore, the EMR dashboard data facilitate reporting across organizational governance structures. The health care organization executive, clinical lead, and governance frameworks were also updated.

Discussion

This study used the Model for Improvement quality improvement framework 12 to develop an ongoing, iterative quality improvement program to address nursing, midwifery, and medical EMR documentation by clinicians within our health care organization.

In order to address the need for benchmarking data for EMR documentation by clinicians, an audit was developed and used to identify key workflows requiring improvements in EMR documentation quality and completeness. Though audit results were not comparable across disciplines due to differences in work and workflows, valuable data were obtained to assist with the evaluation of EMR documentation components to ensure clarity and streamlining of workflows where possible. Targeted education and training interventions were developed to address gaps in EMR documentation compliance and quality for each health care group. By doing so, this study addressed the need to provide personalized education and training that corresponds to clinicians' work and workflows, thereby improving EMR use by clinicians. 15 Conducting the audit supported evaluation of timeliness of EMR documentation as part of admission and ongoing care workflows, in addition to documentation quality. This also fills a gap previously identified in Australian research on EMR admission documentation. 16

Positive responses from clinicians receiving in-person and personalized EMR support are in agreement with previous literature that showed EMR demonstrations improved multidisciplinary clinicians' attitudes and knowledge. 17 Additionally, reassessment of the EMR system, as well as targeted interventions based on the audit results, supports the need to consider that clinical documentation information is not only valued by clinicians but integral to support them in providing safe and quality patient care. 3

Two further outcomes were the result of the development and evaluation of audit data in this study: the first included a framework with five elements to ensure ongoing EMR documentation and use by clinicians are monitored, and EMR adoption and quality are enhanced, while the second includes a new EMR data dashboard that provides clinical, managerial, and leadership staff with up-to-date information on EMR documentation quality and compliance. These two initiatives support the health care organization's ongoing EMR improvement program. It is hoped these initiatives will also minimize documentation-related cognitive burden by improving the organization's understanding of EMR documentation requirements and, in turn, facilitate clinicians' work and EMR documentation by providing resources and targeted interventions to minimize resource wastage. 18 19 Governance frameworks were also updated to reflect the need for a coordinated approach to EMR optimization, including updated EMR training for new or returning staff to include online learning and webinars on workflows deemed to be at-risk for poor adoption or compliance.

The audit of EMR documentation only 1-year postimplementation will serve the health care organization as a baseline for EMR use and documentation evaluation. As the organization's EMR use matures, examination of ongoing EMR adoption will include repeating audits on nursing, midwifery, and medical staff documentation, and comparing results to assist in understanding the consequences of the targeted interventions. 20 This study helps to fill the gap in Australian literature on EMR implementations and quality improvement initiatives related to EMR documentation.

The EMR data dashboard incorporates multidisciplinary EMR documentation quality and compliance, as well as feedback from clinicians about targeted training programs. The dashboard will help identify areas requiring further remediation and assess intervention effectiveness. An ongoing program has been developed for the targeted training of specific areas of EMR documentation. Targeted training alternates each quarter to ensure sufficient resources for the multiple sites of the large health care organization and to minimize the burden on clinicians learning or relearning of workflows.

Strengths and Limitations

This study has several strengths and limitations. Strengths include a multidisciplinary team approach, providing both in-person and online EMR support to clinicians. The recent implementation of EMR systems throughout Australian health care organizations, compared to our international colleagues, means that context-specific interventions to facilitate learning and adoption of EMR are less available or not relevant to the Australian health care context. The accessibility of the informatics, training, and adoption teams to resources made available by the health care organization for ongoing EMR quality improvement postimplementation included allocation of staff including an EMR researcher, nursing, midwifery and medical EMR superusers, and subject matter experts. This strength may not be feasible or possible in smaller organizations.

Due to the nature of providing in-person support to a large health care organization's workforce over multiple sites, some clinicians may not have been able to access the in-person workflow walkthroughs. The exclusion of some clinical areas that are utilizing a mix of paper and electronic documentation may also be seen as a limitation in this study.

The iterative nature of this quality improvement study can also be seen as both a limitation and a strength. Only obtaining informal feedback at this stage of the study in regard to clinicians' perceptions of effectiveness is a limitation; however, multiple outcomes (audit results and development of a data dashboard and framework for ongoing adoption interventions) are promising strengths.

Conclusion

This study provides the foundation for an ongoing and iterative quality improvement program to improve EMR documentation. Development and evaluation of an EMR audit supported targeted training and education interventions to address gaps in the quality and compliance of clinicians' EMR documentation. Barriers to EMR usability were addressed by both in-person and online measures to assist clinicians including understanding EMR documentation requirements as well as patient care implications.

A framework to support ongoing EMR adoption-related interventions for the organization was developed, as well as an EMR data dashboard that provides information about multidisciplinary clinicians' use (including completeness and quality) of EMR documentation.

Clinical Relevance Statement

An audit of EMR documentation quality and completeness can inform targeted training and education interventions for EMR adoption improvement. Ongoing assessment of EMR use and documentation compliance and quality can assist in understanding the workflows, needs, and education and training requirements of nursing, midwifery, and medical staff. This approach should be considered as part of any large-scale EMR implementation or optimization.

Multiple Choice Questions

1. Which clinician groups were included in the phase one audit?

   1. Nursing and Midwifery

   2. Nursing

   3. Nursing, Midwifery, and Medical

   4. Medical

   Correct Answer: The correct answer is option c: Nursing, Midwifery, and Medical. All three clinician groups' EMR documentation was audited in phase one of the study in order to obtain a breadth of clinical areas and clinician workflows for examination of EMR documentation compliance and quality.

2. Which National Standards were the nursing and midwifery audit based on?

   1. Medication safety, comprehensive care, and communicating for safety and blood management

   2. Medication safety and comprehensive care

   3. Comprehensive care and communicating for safety

   4. Medication safety, comprehensive care, and communicating for safety

   Correct Answer: The correct answer is option a: Medication safety, comprehensive care, and communicating for safety and blood management. The alignment of the nursing and midwifery audit to four of the National Standards supported the multidisciplinary informatics teams and training and adoption team in their preparation of the targeted training materials and information provided to clinicians to support existing workflows and clinical practices.