Impact of Implementing Electronic Nursing Records on Quality and Safety Indicators in Care

ABSTRACT

Electronic Health Records (EHR) have been adopted to improve the quality of care. Despite the growing use of health information technology, nursing documentation remains a challenge. In Tunisia, the implementation of the Electronic Medical Record (EMR) system is gaining momentum as part of national initiatives to modernize healthcare. However, nursing documentation is still largely paper-based, and no studies in Tunisia have adressed this topic. This research aims to assess the effect of the Electronic Nursing Record (ENR) on the quality and safety of care indicators (QSCI). This is an interventional study structured in four phases: development, pre-implementation, implementation, and evaluation, integrating the principles of the ‘Standards for Reporting Implementation Studies’ (StaRI). A list of QSCI was defined and validated through a literature review and Delphi consensus. The impact of the ENR on these indicators was evaluated in a Tunisian university hospital through a quasi-experimental study. Indicators were measured before ENR, one month after, and six months after. Data analyses was conducted using SPSS with statistical tests. Initially, the study led to the identification and validation of seventeen QSCI. Subsequently, a quasi-experimental study was conducted to evaluate the impact of ENR implementation on these indicators. The results revealed a significant improvement in the intervention group (using ENR), particularly in the traceability of vital signs (p < 10⁻³) and infusion administration (p = 0.027). Conversely, the control group (using paper-based documentation) performed better in terms of traceability of inter-team handovers (95.1% compared to 71.9% for the intervention group). The electronic documentation system is seen as a major transformation in healthcare in many hospitals worldwide. Moreover, electronic nursing documentation is crucial for patient safety. Its implementation in our hospital revealed a positive impact of the ENR on certain aspects of care quality while highlighting gaps in inter-team handovers.

1. Introduction

Over the past decade, healthcare systems around the world have begun implementing advanced Electronic Health Records (EHR) [1], offering benefits in cost management and improved care quality [2,3]. In the USA, more than 4,000 hospitals use these systems [4], although few studies have explored their direct impact on the quality and safety of care [5,6]. In Tunisia, despite digitization efforts through the e-health project [7,8], nursing documentation remains largely paper-based in public healthcare facilities, leading to issues with traceability and errors [9]. While the private sector has begun adopting digital systems, the impact of an Electronic Nursing Record (ENR) on the quality and safety of care has not yet been studied.

It is essential to highlight that the implementation of a well-maintained, comprehensive, and continuously updated ENR would lead to a sustainable improvement in quality and safety of care indicators [10–12]. Indeed, digitization, by requiring instant traceability of adverse events affecting patient safety and care quality, allows real-time intervention [13]. This facilitates quick decision-making to address issues and prevent similar incidents in the future, contributing to the continuous long-term improvement of quality and safety of care indicators [14,15].

Currently, no studies in Tunisia have evaluated the impact of implementing an ENR on the quality and safety of care. This gap leaves the potential effects of digitizing nursing care on traceability, risk management, and the improvement of quality and safety of care indicators unexplored. Therefore, the objective of this research is to assess the effect of ENR on these indicators, to fill this research gap and provide relevant data to improve the quality of nursing care in the Tunisian healthcare system.

2. Materials and Methods

This is an interventional study aimed at evaluating the impact of implementing an Electronic Nursing Record (ENR) on the quality and safety of care. We followed the principles of the ‘Standards for Reporting Implementation Studies’ (StaRI) [16] to ensure transparency and rigor in the study’s execution and reporting of results.

To define and validate the quality and safety of care indicators (QSCI), we used the Delphi method, a structured consensus process based on the opinions of an expert panel [17]. This process involved eight meetings during which the experts assessed the feasibility and relevance of the indicators in our hospital setting.

Our study was structured into four phases: development, pre-implementation, implementation, and evaluation, while integrating both StaRI principles and the Delphi method. This approach ensured a gradual adaptation of the intervention to the hospital context. The development phase focused on designing the intervention, specifically the implementation of the ENR, according to the needs of healthcare professionals. The pre-implementation phase concentrated on preparing for the introduction of this intervention, while the implementation phase supervised its integration into clinical practice. Finally, the evaluation phase measured the impact before the ENR implementation, as well as at 1 month and 6 months after the intervention.

In the methodology section, we will detail the methodological aspects specific to each of these phases, explaining the processes and approaches used to adapt the intervention to the hospital’s specific context. Subsequently, in the results chapter, we will approach these same phases from a different perspective, presenting the results obtained at each stage. This dual approach will allow us to understand not only how the intervention was implemented but also its concrete effects on the quality and safety of care.

• I. Phase 1: Development

  1. Objective: To describe the process of defining and validating the QSCI.
  2. Methodology

2.1. Literature Review

The QSCI were identified through an in-depth literature review, combining a scoping review of scientific articles and an analysis of the accreditation framework from National Instance for Evaluation and Accreditation in Health (NIEAH) (grey literature).

2.1.1. Scoping Review

We followed the methodology proposed by Arksey and O’Malley to conduct a scoping review, identifying existing literature on the impact of Electronic Health Records (EHR) on the quality and safety of care [18]. This approach clarified the main concepts, identified gaps in the literature, and pinpointed relevant quality and safety of care indicators (QSCI). The search was conducted in the Medline and Embase databases, using a search strategy focused on computerized nursing records and their impact on the quality and safety of care. The search equations used were: (‘Medical Records Systems, Computerized’[Mesh]) AND ‘Quality Indicators, Health Care’[Mesh] and (‘Medical Records Systems, Computerized’[Mesh]) AND ‘Patient Safety’[Mesh]. We included studies published in English or French over the past 15 years.

• Inclusion and Exclusion Criteria

We screened the titles and abstracts retrieved based on the above criteria. Articles were included if a quantitative or qualitative study related to the use of electronic medical records in hospital settings, focusing on patient safety and/or care quality.We excluded studies conducted in community settings, primary care, neonatal/maternity care, psychiatry, radiology, and oncology departments. Studies were also excluded if they solely focused on paper documentation or did not report specific results related to patient safety, care quality, or nursing documentation quality. Additionally, duplicates, non-original texts, titles, and abstracts irrelevant to the research question, as well as articles unavailable in full text, were excluded. Disagreements were resolved through discussion among the authors.

• Data Extraction

A data extraction table was created to collect the following information: authors, year of publication, country of origin, study description, and results. We extracted data from the included studies. Any differences were discussed, and consensus was reached among the authors.

2.1.2. Review of the NIEAH Accreditation Manual for Second and Third-Line Healthcare Facilities:

NIEAH was created in Tunisia to assess and improve the quality of healthcare services in the country. Its main objective is to promote the continuous improvement of healthcare quality, focusing on several aspects, including: the evaluation of healthcare institutions, the accreditation of healthcare facilities, quality development, and training and support. The accreditation manual for second and third-line healthcare facilities is the result of a process initiated in 2016 and completed in several stages. It is intended for both public and private second and third-line healthcare institutions [19]. This accreditation framework was reviewed to identify criteria related to the quality and safety of care. This analysis helped complete the list of QSCI.

2.1.3. Delphi Method

The information from the scoping review and the criteria from the NIEAH accreditation manual were synthesized into a list of potential quality and safety of care indicators. To validate these indicators, we applied the Delphi method, involving a panel of experts selected from the hospital’s quality control unit. This panel consisted of eight members: the hospital’s general director, the head of the prevention and care safety department, the head of the nursing department, the head of the pharmacy department, and four visiting experts from NIEAH (a nurse, a hygienist, and two administrators).

The first step of the Delphi process involved sending the potential indicators list via email to the experts, asking them to assess the relevance and feasibility of each indicator. This was followed by a series of eight meetings, where the experts discussed the results of these assessments, enabling them to refine and validate the indicators considered relevant for patient safety and/or care quality.

• (3) Synthesis

The literature review and the iterative validation process using the Delphi method led to the selection and validation of a list of QSCI. These indicators were chosen for their measurability and feasibility within our hospital. The validated QSCI will be used to assess the impact of the ENR, with measurements taken at three distinct points: before the ENR implementation, one month after, and six months after its deployment.

• (4) Operational definition of study variables [see Table 1]:

• II. Phase 2: Pre-implementation

1. Objective: Prepare the groundwork for the implementation of the ENR.

2. Activities:

   • Planning: The implementation planning involved defining specific objectives, identifying necessary resources, and developing a timeline. A study was conducted in both intervention departments to identify needs regarding equipment and materials.
   • Training: Training and awareness sessions were organized for nurses. These sessions focused on the utility and use of the ENR and were provided to all three teams (morning, afternoon, and evening) in the intervention departments. The training included workshops and practical sessions, with continuous support during the implementation phase. It was conducted by the doctoral candidate and the hospital’s IT engineering department. A training organization chart was established to ensure that all nurses received the necessary training along with personalized follow-up.

• Technical Preparation:

  • -Equipment Acquisition: Following the needs assessment, a computer was purchased for each intervention department, funded by the hospital’s budget. This computer is dedicated exclusively to the use of the Electronic Nursing Record (ENR) by the nurses.
  • -Installation and Activation of the ENR: The ‘Dossier Médical Informatisé’ (DMI) application was installed, and the nursing component, called ‘ENR,’ was activated on the computers in both departments. Each nurse received a personal password via SMS to ensure secure access to the DMI.
  • -Composition of the ENR: The ENR consists of three main interfaces: the monitoring sheet, the treatment and infusion prescription, and the instructions. The ‘instructions’ interface provides a dedicated space for nurses to communicate various information. During training sessions, particular attention was given to the importance of documenting critical information such as blood transfusions, patient falls, pressure ulcers, infections associated with care, and the information provided to patients regarding care procedures. A list of specific recommendations was distributed to each supervisor of the intervention departments to ensure that this information was properly documented.
  • -Accountability: Each supervisor of the intervention departments was designated to oversee the proper use and continuity of the ENR from the moment of its implementation.

Table 1.

Operational definition of study variables.

Indicators	Operational definition of the indicator
Average length of service stay	It is calculated by subtracting the patient’s admission date from the discharge date.
Patient satisfaction rate with their level of information	Number of patients satisfied with their level of information during period t divided by the total number of patients surveyed during the same period, multiplied by one hundred.
Patient satisfaction rate with pain processing	Number of patients satisfied with pain management during period t divided by the total number of patients surveyed during the same period, multiplied by one hundred.
Compliance rate of instructions between teams	Number of patients for whom nurses conducted inter-team instructions during period t divided by the total number of patients admitted to the hospital during the same period, multiplied by one hundred.
Compliance rate of instructions between services	The number of patients for whom nurses conducted inter-service handover instructions during period t, divided by the total number of patients admitted to the hospital during the same period, then multiplied by one hundred.
Compliance with the vital sign measurement rate	Number of patients for whom nurses documented measured vital signs during period t divided by the total number of patients admitted to the hospital during the same period, multiplied by one hundred.
Rate of traceability of information given to the patient regarding the care protocol	Number of patients for whom nurses documented the information provided regarding the care protocol during period t divided by the total number of patients admitted to the hospital during the same period, multiplied by one hundred.
Mortality rate in the service	It is calculated by dividing the number of deaths that occurred in the hospital during a period t by the number of patients admitted to the hospital during the same period, and then multiplying the result by one hundred.
Readmission rate	Number of patients readmitted to the hospital during a period t, divided by the number of patients admitted to the hospital during the same period
Compliance rate with hospital patients’ fall documentation instructions	It is calculated by dividing the number of documented falls during period t by the total number of falls occurring during the same period.
Compliance rate with hospital patients’ pressure ulcer processing documentation instructions	It is calculated by dividing the number of documented pressure ulcers that occurred in the hospital during period t by the total number of pressure ulcers that occurred during the same period.
Compliance rate with the directive regarding documentation of healthcare-associated infections (HAI).	It is calculated by dividing the number of documented HAIs that occurred in the hospital during period t by the total number of HAIs that occurred during the same period
Compliance rate with traceability of treatment dispensing	It is calculated by dividing the number of treatments administered and documented in the hospital during period t by the total number of treatments administered during the same period.
Compliance rate with traceability of infusion dispensing	It is calculated by dividing the number of infusions dispensed and documented in the hospital during period t by the total number of infusions dispensed during the same period.
Compliance rate with traceability of transfusion act	It is calculated by dividing the number of transfusions performed and traced at the hospital during a period t out of the total number of transfusions performed during the same period.
Post-transfusion vital sign measurement rate	Number of transfusions followed by measurement of vital signs performed and traced during a period t out of the total number of transfusions performed during the same period.
Rate of reporting of adverse events related to care	It is calculated by dividing the number of adverse events associated with care that occurred and were documented in the hospital during period t by the total number of adverse events associated with care that occurred during the same period.

(3) Preliminary Evaluation:

A satisfaction survey was conducted among the nurses and supervisors of the intervention departments at the end of the training sessions and preparation for the implementation of the ENR. This quantitative study aimed to evaluate the relevance of the training, the clarity of the information conveyed, and the understanding and adoption of the ENR by the healthcare staff.

• III. Phase 3: Implementation

1. Objective: Describe the process of implementing the ENR.

2. Activities:

   • Deployment: During the initial period, nurses received support sessions from the doctoral candidate, an experienced nurse, and the IT engineer for the three teams: morning, afternoon, and night shifts.
   • Initial Monitoring: The doctoral candidate monitored the early stages of the implementation by accessing the ENR and comparing the computerized data with the paper records.
3. Methodology: The implementation of the ENR was evaluated by:

   • Calculating the ENR Utilization Rate at t₁ and t₂: The utilization rate of the ENR is a variable in the study. It was calculated by the doctoral candidate by accessing the DMI application and then the ENR section. This rate corresponds to the percentage of patients whose ENR was completed by the nurses during a given period, relative to the total number of hospitalized patients recorded in the DMI during the same period.
   • Conducting a ENR Acceptance Survey Among Users: An acceptance survey for the ENR was conducted three months after its launch. This survey aimed to gather impressions and feedback from users, particularly nurses, on various aspects of the ENR, such as its ease of use, utility, impact on nursing work, and user satisfaction. The results of this survey allowed us to identify the strengths of the system as well as areas needing improvement. By analyzing user feedback, we were able to adjust our approach to better meet their needs and optimize the use of the ENR within the hospital.

• IV. Phase 4: Evaluation

1. Objective: Evaluate the impact of the ENR on the QSCI.

2. Activities:

   • Study Type:

This is a quasi-experimental pilot study conducted over an eleven-month period (from October 2022 to August 2023) at a university hospital in the central region of Tunisia. It is a public administrative institution with legal personality and financial autonomy, while being under the supervision of the Ministry of Health. Its primary activities include providing healthcare, as well as ensuring prevention, education, and research in the medical and health fields. The establishment has thirty-two multidisciplinary services and a capacity of 608 budgeted beds. Regarding human resources, 1703 people work at the hospital, divided among medical, nursing, labor, and administrative staff.

The study consists of two groups:

• Intervention Group (G1): This group includes a medical service (Med 1) and a surgical service (Surg 1) that benefited from the implementation of the ENR. An evaluation of the validated QSCI was conducted at three time points: t₀, t₁, and t₂.

• Control Group (G0): This group comprises a medical service (Med 2) and a surgical service (Surg 2) that did not implement the ENR. Similarly, an evaluation of the validated QSCI was conducted at three time points: t₀, t₁, and t₂.

Measurements of the QSCI, were taken at three time points:

• Baseline Measurement of QSCI, (t₀): Before the implementation of the ENR: October 2022.

• Second Measurement of QSCI, (t₁): From 16 January 2023, to 16 February 2023: one month after the implementation of the ENR.

• Third Measurement of QSCI, (t₂): From 16 July 2023, to 16 August 2023: six months after the implementation of the ENR [see Figure 1].

• Study Population:

• This work was ‘exhaustive’ and included all patients hospitalized in the study departments at t₀, t₁, and t₂. We included patients who were able to communicate and had given prior consent to participate in the study. Patients with clinical conditions preventing them from responding to the questionnaire were excluded.

• Additionally, our research involved all nurses working in the study departments. We included those providing nursing care who were present during the study period and had voluntarily given prior consent to participate. We excluded those who refused to participate in the study and those absent during the research period (on long-term leave, etc.).

• Data Collection: To measure QSCI (quality and safety of care indicators) pre- and post-intervention, the following instruments and methods were used:

Figure 1.

Diagram of the study process.

Collection of Patient-Related Indicators:

• A questionnaire for patients identifying gender, age, and patient satisfaction regarding their level of information and pain management. This questionnaire was filled out by the doctoral student through a direct interview with the patient.

• An observation grid completed by the doctoral student, based on:

  • The EMR accessed via the ‘Patient Management Department,’ to determine the length of stay, mortality rate, and readmission rate of patients.

Collection of Nurse-Related Indicators

• A questionnaire for nurses identifying gender, age, rank, work seniority, as well as the occurrence during the patient’s hospitalization of: an adverse event, a healthcare-associated infection, a fall, or a pressure ulcer.

• An observation grid completed by the doctoral student, based on the patient’s paper file and the ENR, to evaluate nursing documentation (for the intervention group, both paper and electronic formats were used to ensure maximum information was captured).

• Data Analysis:

Data were analyzed using SPSS software version 26.0. For the descriptive study, quantitative variables were expressed by their means and standard deviations, while qualitative variables were presented as absolute and relative frequencies. For comparisons, Pearson’s chi-square test (or Fisher’s exact test) was used for qualitative variables, and the Student’s t-test was used for comparing means. A significance level of p-value less than 0.05 was considered statistically significant.

• Ethical Considerations:

Data collection took place after obtaining approval from the hospital's ethics committee (number: HS 23-2022), the department heads, and the supervisors of the departments involved in the study, as well as after receiving informed consent from the study participants while explaining to them the framework and purpose of the research. Participation was strictly confidential and anonymous; no personal data or names were mentioned in the reports derived from this research. We declare that there is no potential conflict of interest related to this research.

3. Results

The results chapter is structured around four phases: development, pre-implementation, implementation, and evaluation, while integrating the StaRI principles and the Delphi method, ensuring consistency with the structure of our methodology.

• I. Phase 1: Development

1. Results of the Scoping Review:

The initial search in Medline yielded 1153 articles. After excluding duplicates and non-original texts, followed by title and abstract review, 1112 articles were rejected, and 41 articles were selected for further analysis. Finally, discussions among the authors led to the inclusion of 21 studies in the review [see Figure 2].

Figure 2.

Search process and outputs.

Characteristics of the Included Studies: Of the 21 articles, sixteen were conducted in the USA, one in England, one in Korea, one in Ireland, one in Jordan, and one in Sweden. In terms of research design, 16 studies were quantitative and 5 studies were qualitative. The results and key QSCI, as well as detailed findings from each study, are presented in [Table 2].

• (2) Results of the Review of the Accreditation Manual for Second and Third-Level Healthcare Institutions:

Table 2.

Summary of data from included studies.

Author, year, country	Study description	Results
Kutney-Lee and Kelly, 2011, US [20]	A cross-sectional, secondary analysis of 16,352 nurses working in 316 hospitals in 4 states was conducted. The aim of this study was to examine the effect of having a basic electronic health record (EHR) on nurse-assessed quality of care, including patient safety.	Nurses in hospitals with fully implemented basic EHRs were consistently and significantly less likely to report unfavorable outcomes in their hospitals than did nurses in hospitals without fully implemented basic EHRs.
Selvaraj et al, 2018, American [3]	Examination of participants from the Get With The Guidelines (N = 21 222), using various degrees of EHR implementation. A multivariable logistic regression was performed to determine the relation between EHR status and several in-hospital quality metrics and outcomes.	Increasing EHR implementation was associated with improved patient outcomes (higher rates of discharge home and less frequent length of stay >4 days). However, increasing EHR implementation was also associated with worse rates of several achievement measures.
Soriano and all, 2019, California [21]	Semi structured interviews describe the role of nurse managers in quality monitoring, their experience in using the EHR to monitor nurse-sensitive quality measures, and their related training.	Participants described the EHR as the primary source of information for monitoring documentation and of observing clinical procedures.
McCarthy et all, Ireland, 2019 [22]	To review the evidence on the effects/impact of electronic nursing documentation interventions on promoting or improving quality care and/or patient safety in acute hospital settings.	This systematic review found limited evidence on the effects of electronic nursing documentation interventions on promoting or improving quality care and patient safety in acute hospital settings.
Ward et al, 2014, US [23]	A retrospective before/after analysis, to study the effect of electronic health record implementation on the operational metrics of a diverse group of emergency departments.	There is no meaningful difference in 8 measures of operational performance for EDs experience EHR implementation between a baseline and steady state period.
Encinosa et al, America, 2012 [24]	Use of three data sets: examination of all hospital claims incurred within 90 days after the surgery admission date; the American Hospital Association’s 2007 Annual Survey; the AHA 2007 Annual Survey	Electronic medical records (EMRs) do not reduce the rate of patient safety events. However, once an event occurs, EMRs reduce death by 34%, readmissions by 39%, and spending by $4,850 (16%). Thus, EMRs contain costs by better coordinating care to rescue patients from medical errors once they occur.
DesRoches et al, U.S. hospitals, 2010 [25]	Examination of electronic health record adoption in U.S. hospitals and the relationship to quality and efficiency.	There was no significant relationship between EHR and: mortality, length of stay and readmission rates. However, hospitals with DSE performed better in preventing surgical complications.
Mitchell et al, US, Texas hospitals, 2012 [26]	Primary analysis of the relationship between EHR implementation, quality, and safety was performed using secondary data collected and compiled from three data sources.	Path analysis was performed and revealed that no statistically significant relationship exists between EHR access and patient safety or quality of care for small hospitals.
Yoo et al, South Korea, 2014 [27]	This study analyzed the clinical indicators that have been developed to improve and monitor the quality of care and patient safety.	With the continuous monitoring and feedback activities, the system enabled to maintain the optimum level of patient care.
Spellman Kennebeck et al, USA, 2012 [28]	Evaluate the effect of EHR implementation on overall patient length of stay (LOS), time to medical provider, and provider productivity during and after implementation of EHR.	Overall LOS and time to doctor increased during EHR implementation. On average, admitted patients’ LOS was 6–20% longer. For discharged patients, LOS was 12–22% longer.
Zlabek et al, USA, 2011 [29]	The authors studied the effects of an inpatient EHR system with computerized provider order entry on selected measures of cost of care and safety.	Laboratory and radiological examinations per week per hospitalization decreased. Medication errors per 1000 hospital days decreased, while near-misses per 1000 hospital days increased.
Baernholdt et al, America, 2018 [30]	A qualitative study using focus groups was conducted.	In all 4 groups, 5 themes were represented: Meaningful measurement, Gaps in measurement, Structural characteristics, Act of measuring and Utility of measurement.
Nowinski et al, USA, 2007 [31]	The purpose is to evaluate changes in organizational culture and quality of care over time, in order to better understand the relationships between the EHR, organizational culture, and quality improvement.	Employees perceived the organizational culture as becoming more, rather than less, hierarchical. The hypothesis was «quality indicators would show improvement due to enhanced information flow and ease of information retrieval.
Schall et al, Iowa, 2017 [32]	Description of a dashboard designed to promote quality care through display of evidence-based quality indicators within an EHR.	Use of the dashboard led to reduced task completion times and error rates in comparison to the conventional EHR for quality indicator–related tasks.
Chan et al, USA, 2010 [33]	A search on articles between January 2004 and June 2009 related to EHRs, quality measures, and data quality on PubMed.	Many studies evaluated multiple aspects of data quality or comparability, with 23 studies (66%) evaluating data accuacy, 20 studies (57%) data completeness …
Tubaishat et al, Jordan, 2019 [34]	An exploratory qualitative study using semi-structured interviews with nurses working in hospitals using the same EHR system in Jordan, to explore the effect of EHRs on patient safety, as perceived by nurses.	EHRs have directly or indirectly improved patient safety by reducing medication errors, improving data completeness and durability. Respondents expressed concern about data entry errors, technical problems, minimal clinical alerts and poor use of the system’s communication channels.
Stevenson et al, southeast of Sweden, 2012 [35]	Focus group interviews were conducted with 21 Nurses. This was a qualitative study to explore nurses’ perceptions of using electronic patient records in everyday practice, in general ward settings.	There were mainly positive views from the nurses on the medication module about patient safety and this led to fewer errors. None of the nurses wanted to return to paper records but strongly verified that they wanted a much improved system.
Fuller et al, USA, 2018 [36]	Literature review, to map the extent, range, and nature of research on the effectiveness, level of use, and perceptions about electronic medication administration records (eMARs).	Nine studies reported on improvements to safety and quality of care as a result of eMAR, which integrated decision support systems resulted in improvements in staff adherence to medication monitoring and reduced missed lab tests and other orders.
Whalen et al, USA, 2018 [37]	Description of medication safety and implementation challenges and solutions of a large academic center transitioning its EHR to Epic.	There was a 5-fold increase in the overall number of medication safety reports; by the third month the rate of reported medication errors had returned to baseline.
Dixon-Woods et al, 2013, England [38]	A case study was based on ethnographic observations of clinical settings and meetings, informal conversations with clinical staff, semi structured interviews, and the collection of relevant documents.	Technovigilance was-based on the hospital’s own evidence-highly effective in improving specific indicators. Measures such as the rate of omitted doses of medication showed marked improvement.
Roth et al, California, 2009 [39]	A review of the data requirements for the indicators in the Quality Assessment Tools system (QA Tools)	A review of the data suggests that only about a third of the indicators would be readily accessible from EHR data.

The authors examined the 560 criteria from the accreditation manual, and 14 criteria related to the quality and safety of care were selected. Subsequently, a preliminary list of QSCI was developed.

• (3) Identification and Validation of QSCI:

After conducting 8 meetings using the Delphi method, the expert panel from our hospital validated a list of seventeen QSCI, as shown in [Table 3]. This validation is a crucial step to ensure that the selected indicators meet the specific needs of our institution.

• II. Phase 2:Pre-Implementation

Table 3.

The validated list of quality and safety indicators in relation to the electronic nursing documentation.

No	Quality and safety of care indicators:
1	Mortality rate in the service
2	Readmission rate
3	Compliance rate with hospital patients’ fall documentation instructions
4	Compliance rate with hospital patients’ pressure ulcer processing documentation instructions
5	Compliance rate with the instruction on the documentation of healthcare associated infections
6	Patient satisfaction rate with their level of information
7	Patient satisfaction rate with pain processing
8	Compliance rate of instructions between teams
9	Compliance rate of instructions between services
10	Compliance with the vital sign measurement rate
11	Compliance rate with traceability of treatment dispensing
12	Compliance rate with traceability of infusion dispensing
13	Rate of traceability of information given to the patient regarding the care protocol
14	Compliance rate with traceability of transfusion act
15	Post-transfusion vital sign measurement rate
16	Rate of reporting of adverse events related to care

The pre-implementation phase focused on training the nurses to ensure a successful transition to using the Electronic Nursing Record (ENR).

1. Training of Nurses and Supervisors: All nurses in the intervention departments and both supervisors received training sessions, during which the EHR, and more specifically the ENR, was clarified. Several workshops and support sessions were conducted to strengthen their understanding and commitment.

2. Nurses’ Satisfaction with the Training Sessions: At the end of the training sessions, a satisfaction survey was conducted among the nurses and the two supervisors in the intervention departments (n = 27). The survey assessed the relevance of the training, the clarity of the information provided, and the participants’ adoption of the ENR. The results showed that 96.3% of the healthcare staff were satisfied with the overall atmosphere of the sessions, 85.2% with the scientific value of the content, and 77.8% with the interactivity during the training. Additionally, 62.9% found the sessions beneficial, and 85.2% stated that the training would lead to changes in their practices.

   • III. Phase 3:Implementation

1. ENR Usage Rate at t₁ and t₂: One month after the intervention (t₁), the usage rate of the ENR was 67.5%. By t₂, six months after implementation, this rate had dropped to 51.4%.

2. User Acceptance of the ENR: To assess the acceptance of the ENR by nurses in their professional practices, a quantitative study was conducted with the nurses in the two intervention departments (n = 25), three months after the introduction of the ENR. Among them, 56% were between 33 and 50 years old, with a sex ratio (male/female) of 0.086. Additionally, more than half (52%) had between 5 and 10 years of work experience.

• Regarding their perception of the system’s ease of use, 70% of the nurses found the system easy to use, 40% considered it reliable, 45.8% thought it was efficient, and 62.5% found it secure. In terms of the perceived usefulness of the ENR, 54.2% responded that the digital system made their work more satisfying, half of the nurses said the ENR facilitated their work, and 37.5% believed that it increased productivity.

• Overall, 41.7% of the nurses expressed they were ‘somewhat in favor’ of using digital systems. Only 16.6% preferred to return to the old methods of working, while 58.4% believed they should use digital systems more in their daily tasks.

• IV. Phase 4: Evaluation

1. Objective:

• The objective was to evaluate the effect of implementing the ENR on the QSCI by comparing the results before and after the implementation of the ENR.

• (2) Sociodemographic Characteristics of Nurses (n = 46):

• The average age of nurses working in the four departments concerned was 56.20 ± 11.15 years in the ‘Surgery 2’ department, while it ranged between 38.20 ± 7.16 and 39.40 ± 13.82 years in the other departments. Regarding years of service, it was 21.30 ± 12.05 years in the ‘Surgery 2’ department and ranged from 9.67 ± 8.06 to 15.80 ± 11.46 years in the other departments.

• (3) Comparison of QSCI Before (t0) and After ENR Implementation (t1 and t2):

  • T0: Before ENR Implementation

At t₀, before the implementation of the ENR, the average age in the control group was 44 ± 23 years with a sex ratio of 1.3 (male/female). In the intervention group, the average age was 45 ± 21 years with a sex ratio of 1.03 (male/female). Prior to ENR implementation, there was no statistically significant difference in QSCI between the two groups, except for the average length of stay, which was higher in the intervention group (p = 0.006), and better traceability of vital signs measurement in the same group (p < 10⁻³) [see Table 4].

• T1: One Month After ENR Implementation

Table 4.

Comparison of validated quality and safety care indicators between the control group and the intervention group at t₀, t₁, and t_2.

	t0			t1				t2
				G0 n = 169	G1 n = 120			G0 n = 143	G1 n = 138
The quality/safety indicators of care n (%)	G0 n = 138	G1 n = 120	p	paper	paper	ENR	p	paper	paper	ENR	p
The average length of stay in the department (days ± SD)	4,1 ± 5,7	6,6 ± 8,3	0,006	5,8 ± 8,3	7,1 ± 10,6	–	0,257	4,9 ± 5,8	7,9 ± 10,4	–	0,003
The mortality rate in the department	0 (0,0)	1 (0,8)	0,465	0 (0,0)	1 (0,8)	–	0,415	1 (0,7)	2 (1,4)	–	0,617
The readmission rate	4 (2,9)	5 (4,2)	0,737	2 (1,2)	2 (1,7)	–	1,000	7 (4,9)	3 (2,2)	–	0,218
The patient satisfaction rate regarding their level of information	123 (89,1)	104 (86,7)	0,544	160 (94,7)	117 (97,5)	–	0,235	133 (94,0)	135 (97,8)	–	0,055
The patient satisfaction rate regarding pain management	129 (93,5)	108 (90,0)	0,308	158 (93,5)	116 (96,7)	–	0,230	136 (95,1)	135 (97,8)	–	0,218
The rate of compliance with handover procedures between teams	127 (92,0)	105 (87,5)	0,228	163 (96,4)	112 (93,3)	34 (28,3)	0,224	136 (95,1)	99 (71,9)	54 (39,1)	<10−3
The rate of compliance with handover procedures between departments	0 (0,0)	0 (0,0)	–	0 (0,0)	0 (0,0)	0 (0,0)	–	0 (0,0)	0 (0,0)	0 (0,0)	–
The rate of compliance with tracking of measured vital signs	111 (80,4)	120 (100,0)	<10−3	151 (89,3)	119 (99,2)	78 (65,0)	0,001	114 (79,7)	138 (100,0)	70 (50,7)	<10−3
The rate of compliance with tracking of medication dispensation	138 (100,0)	120 (100,0)	-	169 (100,0)	120 (100,0)	18 (15,0)	–	143 (100,0)	137 (99,3)	44 (31,9)	0,491
The rate of compliance with tracking of information given to the patient regarding the care protocol	0 (0,0)	0 (0,0)	–	0 (0,0)	0 (0,0)	0 (0,0)	–	0 (0,0)	0 (0,0)	0 (0,0)	–
The frequency of infusions.	13 (9,4)	40 (33,3)	<10−3	25 (14,8)	39 (32,5)	–	<10−3	12 (8,4)	59 (42,8)	–	<10−3
The rate of compliance with tracking the dispensation of infusions.	10 (76,92)	37(92,5)	0,124	22(88)	39 (100,0)	8 (20,51)	0,027	9 (75,0)	57 (96,61)	14 (23,73)	0,008
The frequency of falls	0 (0,0)	1 (0,8)	0,465	0 (0,0)	0 (0,0)	–	–	0 (0,0)	2 (1,4)	–	0,240
The compliance rate with the instruction regarding documentation of patient falls	–	0 (0,0)	–	–	–	–	–	–	0 (0,0)	0 (0,0)	–
The frequency of pressure ulcers.	0 (0,0)	1 (0,8)	0,465	0 (0,0)	0 (0,0)	–	–	0 (0,0)	1 (0,7)	–	0,491
The compliance rate with the instruction regarding documentation of pressure ulcer management.	0 (0,0)	1 (100,0)	–	–	–	–	–	–	1 (100,0)	0 (0,0)	–
The frequency of Healthcare-Associated Infections (HAIs)	0 (0,0)	0 (0,0)	–	0 (0,0)	1 (0,8)	–	0,415	0 (0,0)	2 (1,4)	–	0,240
The compliance rate with the instruction regarding documentation of Healthcare-Associated Infections (HAIs).	–	–	–	–	0 (0,0)	0 (0,0)	–	–	0 (0,0)	0 (0,0)	–
The frequency of transfusions	1 (0,7)	3 (2,5)	0,341	5 (3,0)	15 (12,5)	–	0,002	5 (3,5)	28 (20,3)	–	<10−3
The compliance rate with the instruction regarding documentation of the transfusion procedure.	1 (100,0)	3 (100,0)	–	5 (100,0)	15 (100,0)	6 (5,0)	–	5 (100,0)	28 (100,0)	3 (2,2)	–
The compliance rate with tracking post-transfusion vital signs.	1 (100,0)	3 (100,0)	–	4 (80,0)	11 (73,3)	1 (6,7)	1,000	4 (80,0)	23 (82,1)	0 (0,0)	1,000
The frequency of adverse events associated with care	0 (0,0)	0 (0,0)	–	0 (0,0)	2 (1,7)	–	0,172	0 (0,0)	0 (0,0)	–	–
The rate of reporting adverse events associated with care	–	–	–	–	0 (0,0)	0 (0,0)	–	–	–	–	–

ENR = Electronic Nursing Record, G0= Control Group, G1= Intervention Group, SD= standard deviation

One month after the intervention (t₁), the frequency of ENR usage was 67.5%. The control group had an average age of 38 ± 21 years and consisted of 85 men (50.3%) with a sex ratio of 1.01(male/female). In the intervention group, the average age was 40 ± 22 years, with 61 men (50.8%). QSCI. analysis showed that the traceability rate of vital signs measurement was higher in the intervention group at 99.2% compared to 89.3% (p = 0.001), as well as the traceability rate of infusion administration, which was 100% in the intervention group compared to 88% in the control group (p = 0.027) (on paper). However, for the intervention group, traceability using the ENR was lower than on paper for all QSCI.

• t₂: Six Months After ENR Implementation

At t₂, six months after the implementation of the ENR, the usage frequency of the ENR had dropped to 51.4%. The control group was predominantly male (85, 59.4%) with an average age of 43 ± 24 years, while the intervention group consisted of 50.7% men with an average age of 44 ± 21 years.

At t₂, the average length of stay in the intervention group was longer at 7.9 ± 10.4 days compared to 4.9 ± 5.8 days in the control group (p = 0.003). The traceability rate of measured vital signs was 100% in the intervention group versus 79.7% in the control group (p < 10⁻³). Additionally, the traceability rate of infusion administration was higher in the intervention group at 96.6% compared to 75.0% in the control group (p = 0.008) (on paper). However, the rate of inter-team handover compliance was higher in the control group at 95.1% compared to 71.9% in the intervention group (p < 10⁻³). Overall, the traceability frequency using the ENR was lower than on paper for all QSCI.

4. Discussion

4.1. Summary of the main results of this study and interpretation in light of relevant literature

A clear definition of the QSCI is essential for evaluating the effectiveness of the ENR in clinical practice. Our results have established a list of 17 indicators, including measures such as the average length of stay, mortality rates, and readmission rates, as well as specific criteria related to nursing documentation, such as the traceability of vital parameters and adherence to care instructions.

These indicators are crucial for monitoring the quality of care, as they provide measurable criteria for identifying areas needing improvement [40,41]. By incorporating elements related to documentation, our study addresses a gap often found in the literature, where the importance of documentation quality is frequently underestimated [42,43].

In comparison with previous studies, our results confirm that indicators such as length of stay [3,25,27–29] and in-hospital mortality [3,24–26] are recognized standards in evaluating the quality of care. Previous studies have also highlighted relevant indicators, such as premature discharges before treatment completion and overall patient satisfaction. Other indicators include rates of pressure ulcers [22,30], falls [22,30,31], and readmissions [24,25,30,32], which are commonly used to assess patient safety.

Furthermore, some studies have reported indicators related to nursing documentation, such as the frequent loss of important information during shift changes, loss of information during patient transfers between units [20], documentation monitoring [21], the rate of post-transfusion vital signs measurement [27], co-administration of medications and blood, as well as pain assessment prior to surgery, classification of transfusion reactions, and the accuracy, completeness, and comparability of data [5,33,34], as well as rates of medication errors and medication administration [5,34,36–38].

However, our list is enriched by highlighting aspects of documentation that are often overlooked, such as the traceability of vital parameters, documentation of care, and monitoring of blood transfusions. These elements are particularly critical in complex clinical situations, where precise and comprehensive documentation is essential to ensure continuity of care and minimize errors. Indeed, documenting transfusions, for example, not only ensures a rigorous follow-up of the actions performed but also helps prevent incidents related to incompatibilities or transfusion reactions, thereby enhancing patient safety.

The analysis of the results of this study showed that at t₀, prior to the implementation of the ENR, the comparison of the QSCI between the two groups did not reveal any statistically significant differences, except for the average length of stay, which was longer in the intervention group. This difference may be explained by the fact that one of the two services treated patients suffering from chronic and severe illnesses, requiring more complex and prolonged care. Furthermore, the intervention group showed a higher adherence to the traceability of measured vital parameters, although these parameters were recorded on monitoring sheets rather than in a paper nursing record.

However, the parameters must be ‘well done and well recorded.’ Additionally, this data must be easily viewable so that trend changes can be instantly identified by all clinicians. It is therefore crucial that this clinical documentation is presented comprehensively [5].

In our study, the nurses did not perform inter-service handovers during the three periods (t₀, t₁, t₂), particularly during patient transfers to other services. Effective communication among caregivers is essential to ensure a smooth and successful transfer of information [44,45]. Documentation plays a central role in this process, as it allows for capturing, sharing, and disseminating the necessary data for continuity of care. Any gaps in documentation, whether in terms of accuracy, availability, or updating of information, can compromise the transfer of information, leading to treatment errors, delays in care, or even a deterioration in patient care coordination [46,47].

Previous studies have also shown that gaps in inter-service communication can lead to an increase in adverse events related to care [48,49]. Health information technologies can be effective in preventing adverse events. They improve communication, provide point-of-care references, facilitate calculations, perform monitoring functions, and offer decision support [50].

In our study, one month after the intervention (t₁), the frequency of ENR usage reached 67.5%. This usage frequency, while moderate, is encouraging in a context where the hospital was preparing for accreditation by NIEAH. The nursing staff, heavily engaged in this quality initiative, demonstrated a use of the ENR that, although lower than that of paper documentation for some indicators, remains promising for a first experience with this system.

The analysis of the QSCI reveals that the intervention group achieved significantly higher results than the control group in terms of traceability of vital parameters (99.2% vs. 89.3%, p = 0.001) and administration of infusions (100% vs. 88%, p = 0.027). These figures indicate a positive impact of the ENR. However, it is important to note that traceability on the ENR remains lower than that of the paper system for most indicators, suggesting that an adaptation period is necessary for staff to become accustomed to the new practices.

According to our study, at t₂, six months after the implementation of the ENR, the frequency of ENR usage dropped to 51.4%, compared to 67.5% at t₁. Nevertheless, the adherence rate for the traceability of measured vital parameters remains higher in the intervention group (100%) compared to the control group (79.7%, p < 10⁻³). The adherence rate for the traceability of infusion administration is also higher in the intervention group (96.61% vs. 75%, p = 0.008). However, the handover of information between teams is more respected in the control group (95.1% vs. 71.9%, p < 10⁻³), which may reflect shortcomings in the adoption of the ENR for inter-team communications.

The decrease in ENR usage between t₁ and t₂ could be explained by several factors, including staff shortages during the summer period and an increased workload. Additionally, nurses are required to document both on paper and in the ENR, leading to work overload and confusion. This dual documentation system may explain why, although certain traceability indicators have improved with the ENR, overall traceability on paper remains superior.

According to a Moroccan study, the hospital information system HOSIX is well accepted by staff, but other users report ongoing issues primarily related to its usability, technical quality, functionalities, added value of computerization, and the time-consuming nature of working within the hospital information system [51].

A mixed-methods study conducted on a sample of 240 nurses in a university hospital where electronic health records (EHRs) had recently been introduced found that the majority of participants had EHR software (62.8%), internet access (84.2%), and desktop computers (76.3%). However, the effective use of EHRs remained low, with only 27.3% of participants using it correctly [52]. These results illustrate the challenges faced during the initial implementation of EHRs by healthcare providers.

Furthermore, a study by Selvaraj et al. in 2018 compared outcomes for patients admitted to hospitals without EHRs (N = 1484), with partial EHRs (N = 13,473), and with complete EHRs (N = 6265). The results showed that increased implementation of EHRs was associated with significant improvements in patient outcomes, including higher rates of discharge home and reduced length of stay [3].

According to our results, the adherence rate for traceability of measured vital parameters at t₂ (on paper) was higher than at t₁ (100% vs. 99.2%) for the intervention group. In contrast, on the ENR, the rate dropped from 65.0% to 50.7%. As for the handover of information between teams, although it increased at t2 compared to t₁ on the ENR (39.1% vs. 28.3%), this frequency remains concerning due to its low level.

A qualitative study from 2018 identified two main reasons for the inadequacy of documentation on vital signs. First, the absence of clear guidelines led to inconsistencies in recording. Secondly, the EHR lacked adequate functionalities for recording vital signs, which led to poor presentation of this data and the creation of alternative paper solutions [53]. This raises questions about the effectiveness of information systems and underscores the importance of improving documentation guidelines and tools to optimize the quality of care.

In our study, after the implementation of the ENR, the adherence rate for the traceability of treatment dispensation on paper was 100%. In contrast, the digital traceability was low, with only 15% at t₁ and 31.9% at t₂. Regarding the traceability of infusions, it was 100% at t₁ and 96.61% at t₂ on paper, while on the ENR, it was only 20.51% at t₁ and 23.73% at t₂. These results suggest that, despite the introduction of the ENR, its use remains insufficient to ensure complete traceability.

The introduction of electronic medication administration systems has fundamentally transformed how nurses administer treatments. This has led to more accurate documentation regarding the timing, dosage, and route of administration of medications [4,54]. For example, Zlabek et al. observed a reduction in medication errors, from 17.9 to 15.4 per 1000 hospital days, illustrating the positive impact of electronic systems on patient safety [29].

Studies have confirmed that EHRs improve patient safety by reducing medication errors, falls, and infections. They have also enhanced data completeness and sustainability while optimizing healthcare professionals’ time [20,22,34,55–57].

Although EHRs have notable effects on reducing hospital costs and improving the quality of care, several issues have been reported by nurses. In various studies, they indicated that documentation often took more time due to repeated data entry in multiple locations, which limited their ability to focus on other essential aspects of their work. Additionally, teamwork was moderately linked to unintended consequences, including risks to patient safety, as well as challenges related to system design and technological barriers. Communication with patients also tended to decrease when these risks were high [56,58–62].

The analysis of the results from other studies revealed the absence of a statistically significant relationship between access to EHRs and patient safety or quality of care [24,26,63]. However, once an adverse event occurs, EHRs have been associated with a 34% reduction in mortality, a 39% decrease in readmissions, and savings of $4,850 (16%), highlighting their crucial role in coordinating care and reducing medical errors [24].

The use of technological innovations to improve patient safety remains a challenge. The ENR is considered a major transformation in healthcare, offering more benefits than drawbacks. Its implementation must meet the needs of patients, nurses, and other clinicians involved in care. As the ‘voice’ to the IT department, nurses play an essential role in integrating direct patient care with technology, enabling the transformation of data into wisdom and improving outcomes [54,61].

4.2. Strengths and Limitations of the Study

The study stands out for its rigorous approach, based on a solid literature review and addressing a current theme. It is framed as an interventional study, adhering to the ‘Standards for Reporting Implementation Studies’ (StaRI), and is structured into four phases: development, pre-implementation, implementation, and evaluation.

The validation of QSCI was carried out using the Delphi method, involving a panel of experts to specifically tailor these indicators to the establishment. The evaluation was based on a quasi-experimental study, allowing for a comparison between an intervention group and a control group, facilitated by the use of a validated list of QSCI. Furthermore, multidisciplinary collaboration among various stakeholders enriched the research and strengthened the robustness of the results obtained.

However, this study has several limitations. First, the lack of national studies on the impact of Electronic Nursing Information Systems on the quality of care and patient safety limits our ability to contextualize our results. Moreover, the validation of indicators focused on elements already integrated into the EMR, which may have reduced the scope of our evaluation by excluding important indicators related to the autonomous role of nurses. The study also overlooked physicians, thus limiting the generalizability of the results.

Additionally, due to time and resource constraints, the intervention group was restricted to two departments, which affected the size and representativeness of the sample. The COVID-19 pandemic also caused delays, impacting the dynamics of the study.

Furthermore, the heterogeneity of the groups presents a limitation. Although we selected departments with similar functions, differences still exist. For instance, in some departments, staff may be accustomed to paper systems and show reluctance to adopt EHR, due to concerns about the speed, security, or reliability of the new methods.

A thorough discussion on these points could provide targeted recommendations for improving the adoption of EHR in various contexts. It would also be beneficial to consider future research with a strengthened methodology, comparing different hospitals to assess the impact of EHR on the quality and safety of care.

4.3. Recommendations

To enhance the quality and safety of care through EHR, it is essential to strengthen cybersecurity with robust solutions and provide training for staff on secure data management. The integration of decision-support systems based on artificial intelligence can also optimize treatments. Furthermore, harmonizing practices across institutions is crucial to ensure effective continuity of care. Continuous training for healthcare professionals on the use of digital tools, combined with interdisciplinary collaboration, is also necessary. Finally, future research should evaluate the impact of EHR and explore new technologies to support the digitization of the healthcare system.

It is noteworthy that improving QSCI requires a longer implementation period for the ENR, spanning several years. This reality emphasizes the importance of thorough follow-up, which should be addressed within another program dedicated to evaluating this gradual implementation. Such a program can measure the long-term impacts of ENR implementation on QSCI, ensuring that the expected improvements are fully realized.

4.4. Conclusion

Nurses, as end users of EHR, play a crucial role in identifying opportunities to improve the efficiency and safety of care. They can leverage data from clinical documentation systems to optimize the quality of patient care. Our findings highlight the effectiveness of the ENR in enhancing certain aspects of care quality while also revealing gaps in the transmission of inter-team communication. This underscores the necessity for ongoing training and adaptation time to ensure a successful transition to this new system.

Funding Statement

The research laboratory ‘Emerging Bacterial Resistance and Safety of Care (LR 20 SP 06), Sousse, Tunisia’ contributed to the funding of the article’s publication.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Ethical Considerations:

Data collection took place after obtaining approval from the hospital’s ethics committee (number: HS 23-2022), the department heads, and the supervisors of the departments involved in the study, as well as after receiving informed consent from the study participants while explaining to them the framework and purpose of the research. Participation was strictly confidential and anonymous; no personal data or names were mentioned in the reports derived from this research. We declare that there is no potential conflict of interest related to this research.