Abstract
Background
Electronic medical records (EMR) are considered a key component of the health care system’s digital transformation. The implementation of an EMR promises various improvements, for example, in the availability of information, coordination of care, or patient safety, and is required for big data analytics. To ensure those possibilities, the included documentation must be of high quality. In this matter, the most frequently described dimension of data quality is the completeness of documentation. In this regard, little is known about how and why the completeness of documentation might change after the implementation of an EMR.
Objective
This study aims to compare the completeness of documentation in paper-based medical records and EMRs and to discuss the possible impact of an EMR on the completeness of documentation.
Methods
A retrospective document analysis was conducted, comparing the completeness of paper-based medical records and EMRs. Data were collected before and after the implementation of an EMR on an orthopaedical ward in a German academic teaching hospital. The anonymized records represent all treated patients for a 3-week period each. Unpaired, 2-tailed t tests, chi-square tests, and relative risks were calculated to analyze and compare the mean completeness of the 2 record types in general and of 10 specific items in detail (blood pressure, body temperature, diagnosis, diet, excretions, height, pain, pulse, reanimation status, and weight). For this purpose, each of the 10 items received a dichotomous score of 1 if it was documented on the first day of patient care on the ward; otherwise, it was scored as 0.
Results
The analysis consisted of 180 medical records. The average completeness was 6.25 (SD 2.15) out of 10 in the paper-based medical record, significantly rising to an average of 7.13 (SD 2.01) in the EMR (t178=–2.469; P=.01; d=–0.428). When looking at the significant changes of the 10 items in detail, the documentation of diet (P<.001), height (P<.001), and weight (P<.001) was more complete in the EMR, while the documentation of diagnosis (P<.001), excretions (P=.02), and pain (P=.008) was less complete in the EMR. The completeness remained unchanged for the documentation of pulse (P=.28), blood pressure (P=.47), body temperature (P=.497), and reanimation status (P=.73).
Conclusions
Implementing EMRs can influence the completeness of documentation, with a possible change in both increased and decreased completeness. However, the mechanisms that determine those changes are often neglected. There are mechanisms that might facilitate an improved completeness of documentation and could decrease or increase the staff’s burden caused by documentation tasks. Research is needed to take advantage of these mechanisms and use them for mutual profit in the interests of all stakeholders.
Trial Registration
German Clinical Trials Register DRKS00023343; https://drks.de/search/de/trial/DRKS00023343
Keywords: clinical documentation, digital transformation, document analysis, electronic medical record, EMR, Germany, health services research, hospital, implementation
Introduction
The digital transformation of the health care system is considered an essential subject to meet current and future societal challenges such as an aging population or rising health care expenditures while at the same time maintaining a high quality of care [1]. An important early step in hospitals’ digitalization and a fundamental requirement for expanding digital maturity is the implementation of an electronic medical record (EMR) [2]. This EMR is considered to be an “electronic record of health care information of an individual that is created, gathered, managed, and consulted by authorized clinicians and staff within 1 health care organization” [3] and replaces the internal clinical documentation on preprinted paper-based charts. Studies show that the implementation of an EMR can lead to various improvements in the clinical context (eg, in the availability of information [4], coordination of care [5], or patient safety [6]). Moreover, the EMR facilitates the secondary usage of the documented data for research purposes through its digital accessibility [7]. To reach those benefits, it is indispensable that the EMR contain documentation that is of high quality. However, there are varying definitions regarding the quality of documentation. In that matter, the Institute of Medicine defined completeness, legibility, accuracy, and meaning as the main aspects of a medical record’s data quality [8]. For those, the completeness of documentation was shown to be the most common dimension of data quality when empirically analyzing the documentation in EMRs [9], and it was highlighted to be especially important for secondary uses such as big data analyses [10].
Our recent systematic review also stated the completeness of documentation as the state of the art for the comparison of paper-based and EMRs [11]. This comparison is important since the implementation of an EMR and the associated transition from handwritten documentation to digital documentation can heavily affect the documentation subject since the transition offers the possibility to adjust which information has to be documented in which way [12]. For example, digitization enables the adoption of certain functionalities that can alter the completeness of documentation, like automatically transferring information from other digital devices to the EMR [13]. Moreover, when working with the EMR, information can be documented remotely, while the paper-based medical record had to be located and physically accessed first. In this matter, several studies conducted in the inpatient setting showed increased completeness in the EMR compared to the paper-based medical record, for example, for the documentation of signs and symptoms [13,14], weight and height, or malnutrition screening [15]. This suggests that the implementation of an EMR might lead to improvements in the completeness of documentation in general. It is therefore the main purpose of this study to evaluate the change in completeness due to the implementation of an EMR in an inpatient setting. Literature already provides proof of a change of completeness in regard to some specific documented information that is analyzed in this work (eg, the documentation of vital signs) [13,14]. Those empirical results might thus be validated for the presented work’s specific setting and discipline. In addition, some of the information that is analyzed in this work is not described in literature yet (eg, the documentation of pain). It is examined for the first time with regard to changes in completeness after the implementation of an EMR.
The knowledge gained can not only support the implementation of new EMRs but could also help understand and optimize arising changes in documentation when existing EMRs need to be adapted [16,17]. This is an important aspect, as the implementation of new EMRs is described as one of the most important interventions to improve the quality of documentation [18]. In this process, mechanisms affecting the completeness of documentation in medical records are not completely understood [10]. On the other hand, this knowledge is needed to fulfill reported educational needs regarding how to reach the optimum quality of documentation [19]. In this context, this study contributes to a more comprehensive understanding of the impact of an EMR on the quality of documentation.
Methods
Overview
This study follows the “Strengthening the Reporting of Observational Studies in Epidemiology” (STROBE) statement [20] whenever it is applicable. It offers reporting standards to ensure the reporting of any important information in empirical research studies. A checklist with details, where the STROBE information is mentioned in the manuscript, can be found in Multimedia Appendix 1.
Ethical Considerations
The study has been approved by the ethics committee of the Medical Faculty of the University of Cologne, Germany (20-1349). All data was anonymized at all times during the scientific analysis. No compensation was paid.
Setting and Participants
The study took place as part of the research project eCoCo, which Beckmann et al [21] described in detail. Within the eCoCo project, the researchers collected various types of data (observations, surveys, interviews, documents, and administrative data) to investigate a possible change in interprofessional collaboration and clinical workflows following the implementation of an inpatient EMR. This study is part of the related work package on documentation content and quality, which took place in a large academic teaching hospital in Germany. The hospital replaces its internal documentation on preprinted paper-based charts with a commercial EMR system (Meona; Mesalvo Freiburg GmbH). The EMR runs on multiple computers that can be moved flexibly over the ward on trolleys. The study follows a pre-post design, retrospectively analyzing the content of the medical records before and after the implementation of the EMR on the hospitals’ orthopaedical ward. Within the first measuring phase, the paper-based medical records were provided as a digital copy of the paper sheets. Those paper-based records represent all patients who were treated on the ward during the last 3 weeks in November 2020 (t0). After 6 months, employees received training on how to use the EMR before the implementation of the EMR took place in May 2021. The EMRs were again provided as a digital copy within a second measuring phase, representing all patients who were treated on the same ward during the first 3 weeks of August 2022 (t1). This resulted in a gap of 15 months between the first and second measuring phases. The complete data set was available to the research team in November 2022 (Figure 1). The hospital provided anonymized medical records to the research team after the records were archived and cleared of sensitive personal data (eg, the patient’s name or date of birth) in the hospital’s internal processes. Any assignment of the patient data or linking of the records’ contents to any individual patient was therefore impossible for the research team, which is, thus, in compliance with the European Union General Data Protection Regulation. This also implies the absence of sociodemographic information for describing the compared samples. The hospital’s mandatory annual quality report, which is available to the public through a designated database [22], is therefore used to describe the ward’s patient sample and the performed treatments in general. This allows an approximation to a description and comparison of the compared samples in terms of their International Classification of Diseases (ICD)–diagnoses distribution.
Figure 1.
Data collection. EMR: electronic medical record.
Study Objective
To answer the question of a possible change in completeness, the records were analyzed by content [23]. The change from paper-based documentation to EMRs always offers the opportunity to fundamentally change the structure of the records. This was shown exemplarily by Montagna et al [12] when the documentation as a continuously written text in the paper-based record was changed to a list of events in the EMR. It is therefore important to ensure the comparability between the 2 record types for the purpose of analyzing a possible change in their completeness. To achieve comparability, the medical records progress note was selected as a specific object of interest for this study’s analyses since it retained the same structure and format in both record types. Part of this progress note is the fever chart (Figures 2 and 3), which includes basic details about vital signs, personal health data, etc [24].
Figure 2.
Paper-based fever chart.
Figure 3.
Electronic fever chart.
All information that was commonly documented in both of the 2 record types (paper-based and electronic) became part of this work. Weiskopf and Weng [9] described this selection mechanism for assessing data quality based on the parallels between the EMR and the paper-based record. This procedure resulted in a total of 10 key items that were analyzed for completeness in this work: blood pressure, body temperature, diagnosis, diet, excretions, height, pain, pulse, reanimation status, and weight. The documentation of this information is equally possible and performed by nurses and physicians. However, there is no information available about who specifically entered the information.
All of those items should be documented immediately when patient care begins on the ward [25]. However, while the documentation of vital signs can take place up to several times a day, the documentation of the patient’s diet usually occurs once a day, and the documentation of the reanimation status (patient’s preference regarding a possible resuscitation) is probably documented only once per hospital stay. Because of these varying documentation practices and to ensure comparability, the analysis focuses on certain documentation in the progress notes that was entered on the first day of patient care on the ward. With regard to the documentation of a diagnosis, it is therefore the diagnosis with which a patient is admitted to the hospital. This diagnosis is mainly responsible for the allocation to specific medical specialties as well as a certain ward and does not necessarily have to match the final diagnosis at the time of discharge, which is important for reimbursement purposes.
Statistical Analysis of Completeness
For every record, each of the 10 items received a dichotomous score of 1 if it was documented on the first day of patient care on the ward; otherwise, it was scored as 0. This resulted in a percentage of completeness for each item per record type. Chi-square tests for independence were used to assess statistically significant differences in the percentage of completeness per item between the 2 record types. Relative risks were calculated for the association between the electronic record type and a possible increase in completeness. To improve the reliability of the associated confidence intervals, they were calculated with 5000 bootstrap replications since the original sample sizes are unbalanced. Moreover, the overall completeness was assessed as sum of the 10 items, resulting in a mean score of completeness per record type ranging from 0 (no item documented) to 10 (all 10 items documented). Those mean scores of completeness per record type were analyzed for equality of variance and statistical difference using unpaired, 2-tailed t tests. Assumptions were checked using several methods (normal distribution: QQ plots and Shapiro-Wilk test; homogeneity of variances: Levene test; and linearity: scatter plot). The level of significance was set to be P<.05 for all calculations. The data were stored in Microsoft Excel (Microsoft Corp) and analyzed in December 2022 using SPSS software (version 29; IBM Corp).
Results
Participants
During the first measuring phase (November 2020), a total of 44 patients (paper-based) were treated on the orthopaedical ward. They were encountering a total of 136 treated patients (electronic) during the second measuring phase (August 2022). This resulted in a total of 180 medical records that became part of this analysis. Due to the data protection regulation and the accompanied anonymization of the records data, there is no information regarding the demographics of the specific study population. Therefore, the ward’s ICD-diagnosis distribution is given as an approximation of a sample description. In 2020, the 3 most frequently coded diagnoses for the orthopaedical ward were complications of internal orthopedic prosthetic devices, implants and grafts (ICD-T84), dorsalgia (ICD-M54), and fracture of shoulder and upper arm (ICD-S42). This report is not yet published for 2022, but the top 3 treated diagnoses in 2019 or 2021 were similar to those in 2020 (Table 1). It can therefore be expected that the treated diagnoses will be similar in 2022, too. Another supporting fact is that the most frequently performed procedure (surgical access to the lumbar spine, the sacrum, or the coccyx [coded as OPS-5-032 in the German adaptation of the International Classification of Procedures in Medicine which is part of the coding system for hospitals reimbursement]) was the same in all 3 years (2019-2021).
Table 1.
Most frequently coded diagnoses.
| ICDa Code | Values, nb/Nc (%) | |
| 2019 | ||
|
|
Dorsalgia (ICD-M54) | 213/3147 (6.77) |
|
|
Other spondylopathies (ICD-M48) | 133/3147 (4.23) |
|
|
Fracture of forearm (ICD-S52) | 131/3147 (4.16) |
| 2020 | ||
|
|
Complications of internal orthopedic prosthetic devices, implants and grafts (ICD-T84) | 166/2912 (5.7) |
|
|
Dorsalgia (ICD-M54) | 148/2912 (5.08) |
|
|
Fracture of shoulder and upper arm (ICD-S42) | 121/2912 (4.16) |
| 2021 | ||
|
|
Complications of internal orthopedic prosthetic devices, implants and grafts (ICD-T84) | 164/3091 (5.3) |
|
|
Dorsalgia (ICD-M54) | 163/3091 (5.27) |
|
|
Fracture of forearm (ICD-S52) | 159/3091 (5.14) |
aICD: International Classification of Diseases.
bFrequency of coded diagnosis.
cTotal inpatient cases.
Change of Completeness
The mean number of documented items was 6.25 (SD 2.15) out of 10 in paper-based medical records and 7.13 (SD 2.01) out of 10 in EMRs. The Levene test confirmed the homogeneity of variances. The Shapiro-Wilk test did not confirm normal distributions, but the QQ plots show an approximation to a normal distribution and a comparable degree of normality (Multimedia Appendix 2). The unpaired t test confirmed the EMRs were statistically significantly more complete than the paper-based medical records under equal variances in the 2 record types (t178=–2.469; P=.01; d=–0.428). When looking at the 10 items separately, data from chi-square tests showed that the documentation of diet increased from being present in 30% (13/44) of the paper-based medical record to 75% (102/136; P<.001) in the EMR, height from 27% (12/44) to 85.3% (116/136; P<.001), and weight from 27% (12/44) to 86% (117/136; P<.001). At the same time, documentation of diagnosis decreased from being present in 100% (44/44) of the paper-based medical records to 49% (66/136; P<.001) in the EMR, excretions from 86% (38/44) to 68% (92/136; P=.02), and pain from 95% (42/44) to 78% (106/136; P=.008). The documentation of vital signs such as blood pressure (P=.47), body temperature (P=.497), and pulse (P=.28) remained unchanged on a high level of completeness, while the documentation of reanimation status (P=.73) remained unchanged on a low level of completeness (Table 2). Positive relative risks (Figure 4) illustrate the association of the electronic record type (exposure) with complete documentation (outcome). The confidence intervals represent 5000 bootstrap replications.
Table 2.
Change of completeness.
| Variable | Type of record | Chi-square (df) | P value | RRa (95% CI) | |
|
|
Paper (n=44), n (%) | Electronic (n=136), n (%) |
|
|
|
| Blood pressure | 37 (84) | 120 (88.2) | 0.5 (1) | .47 | 1.05 (0.92-1.23) |
| Body temperature | 36 (81.8) | 117 (86) | 0.5 (1) | .497 | 1.05 (0.91-1.25) |
| Diagnosis | 44 (100) | 66 (48.5) | 37.1 (1) | <.001 | 0.48 (0.40-0.57) |
| Diet | 13 (29.6) | 102 (75) | 29.8 (1) | <.001 | 2.54 (1.70-4.69) |
| Excretions | 38 (86.4) | 92 (67.7) | 5.8 (1) | .02 | 0.78 (0.66-0.93) |
| Height | 12 (27.3) | 116 (85.3) | 54.5 (1) | <.001 | 3.13 (2.06-5.91) |
| Pain | 42 (95.4) | 106 (77.9) | 7.0 (1) | .008 | 0.82 (0.73-0.91) |
| Pulse | 36 (81.8) | 120 (88.2) | 1.2 (1) | .28 | 1.08 (0.94-1.29) |
| Reanimation status | 5 (11.4) | 13 (9.6) | 0.1 (1) | .73 | 0.84 (0.33-3.33) |
| Weight | 12 (27.3) | 117 (86) | 56.5 (1) | <.001 | 3.15 (2.07-6.00) |
aRR: relative risk.
Figure 4.
Forest plot of relative risks.
Discussion
Principal Findings and Comparison to Previous Work
The main findings of this study confirm an improved completeness of the analyzed information in the EMR on average. This provides further evidence for the suggestion that the general completeness of documentation can improve after the implementation of an EMR. The findings align with the results of similar studies, showing improvements in other data quality dimensions like the accuracy [26] or legibility [27] of documentation. However, when looking at the completeness of the analyzed 10 items in detail, the improvements can only be seen in 3 out of 10 items (diet, height, and weight), while 3 different items exhibited a deterioration in completeness (diagnosis, excretions, and pain). This links to the results of Coffey et al [28], who found 5 of their 11 analyzed items to be more complete while also proving 1 of their elements to be less complete. The reason for the variation in the change in completeness may lie in the mechanism of how information reaches the record. In the paper-based medical records, all information was documented by hand by the various professional groups. EMRs, on the other hand, offer technical features, for example, automatically obtaining information from other digital sources, like patients’ health insurance data [29]. This was manifested as a possible mechanism by Jang et al [30], who showed improved completeness in the EMR for the automatically filled information but not for the manually documented ones.
The analysis shows that roughly every second EMR was missing the documentation of a diagnosis. This is a remarkable change, as it was present in every paper-based record (44/44, 100% vs 66/136, 48.5%). In the first place, it must be clarified that the diagnosis is determined by a physician who enters it into an independently run hospital information system (HIS). This documented diagnosis can also be a preliminary diagnosis, which is used for distribution to the clinical disciplines and is present for every admitted patient. The HIS was already in operation when medical staff was still using the paper-based preprints for documentation purposes. After the EMR’s implementation, the HIS was still in operation along with the EMR. That being said, it is undisputed that during the paper-based period as well as the electronic period, a diagnosis was indeed present for the patients. In the paper-based period, the diagnosis was transferred manually from the HIS into the paper-based preprints, when a record for a recently admitted patient was prepared by a nurse. Since the HIS and the EMR are produced by different software developers, the diagnosis cannot be transferred automatically from the HIS into the EMR. Due to this noninteroperability of the 2 independent digital systems, the manual transfer is still necessary in the electronic period. With the drop of completeness in mind, this double documentation was accepted and carried out in the period of the paper-based record. In the electronic period, the described double documentation has decreased. One possible explanation is that the HIS was not automatically accessible, when an employee had the paper-based record at hand. With the introduction of the EMR, the availability of the EMR became synonymous with the availability of the HIS, since both are accessible from a computer. Therefore, the transfer of the diagnosis from the HIS to the EMR may no longer have been considered necessary. Nevertheless, the reason for this difference remaining unclear illustrates that the sole analyzation of completeness of the documentation alone does not provide sufficient information about the actual quality of the provided treatment. In that matter, it must also be highlighted that the record can contain additional qualitative data entries, like free texts, which might complement the analyzed quantitative information. This underlines that an insufficient quality of documentation does not necessarily allow conclusions to be drawn about the quality of care, and vice versa.
Brown [31] emphasizes this by cautioning people to always consider the circumstances under which people put information into the record before drawing conclusions. This is a major issue because the completeness of documentation might be biased due to aspects that do not directly derive from clinical care. On the one hand, the hospital’s reimbursement for the delivered care depends on what is documented and might cause a possible strengthened thorough filling of certain fields [32]. On the other hand, the burden caused by documentation tasks is critically heavy. It is responsible for a high prevalence of burnout among physicians and nurses [33]. Therefore, clinically or legally unnecessary documentation might be evaded [34]. However, even though complete documentation might neither necessarily arise from nor be essential for the delivery of excellent clinical care, it is likewise of concern under the aspect of big data analytics. In this regard, it would be desirable for the discussed diagnosis to indeed be present in the EMR, even if it already exists in the HIS. An automatic transfer of this information could help to prevent the burden on staff resulting from manual transmission and ensure a complete data set. This is an important point, as the insights gained from analyzing big data offer numerous opportunities, like data-based personalized care in diagnostics and therapy or the support of scientific activities, both with the chance of saving lives and reducing health care costs at the same time [7,35]. It is therefore indispensable to recognize the possibility of changes in documentation due to the implementation or adaptation of EMRs. Only with this attention will it become possible to optimize the documentation process with a focus on the various benefits for all stakeholders, like patients [6], practitioners [36], organizations [5], and society [7].
Strengths and Limitations
The German health care system, in which the study was conducted, was heavily strained by the high number of COVID-19 cases and the associated use of intensive care units during the study period. Especially the first measuring phase (November 2020) fell into the first pandemic year when many planned procedures were suspended to increase hospital capacities. For the first lockdown period in Germany (March 2020), a decrease in orthopedic surgeries is described by approximately 80% [37]. A lockdown-like situation was again declared during the first measuring phase [38], which probably explains the difference in treated patients over the 2 measuring phases (nPaper=44 vs nElectronic=136). However, the similarity between the coded ICD diagnoses over different years (Table 1) suggests that the proven changes in completeness of documentation are not due to significant changes in the studied patient sample, but a detailed sample description based on socioeconomical data is missing due to data protection regulations. On the other hand, there is a study assuming a positive influence of the pandemic on the completeness of documentation since an incomplete documentation might have led to repetitive contacts with the patient, which could have been avoided if the documentation would have been complete in the first place [39]. However, this cannot be verified in this paper due to the lack of further measuring phases. Within this given context, the generalizability of the presented results remains limited.
Further, limitations regarding the analyzed data set have to be stated. The chosen unpaired t test is theoretically based on the assumption of normal distributions. This could not be confirmed statistically for the mean completeness scores by the Shapiro-Wilk test. Although t test has been shown to be robust to a missing normal distribution [40] and the QQ plots (Multimedia Appendix 2) indicate an approximation to a normal distribution, the results could still be biased by the broken assumption.
Moreover, the analyzed data set is missing any information on which person was entering the documentation regarding which patient. On the one hand, it might be arguable that the same physicians or nurses were documenting during the first and also the second measuring phases. This circumstance would make the 2 compared measuring phases dependent samples, having an impact on the chosen statistical model. Since the analyzed data set is missing this information, the results might be biased regarding a possible dependent or independent sample. However, the time passed between the 2 measuring phases might have led to a change of the employees since the teaching status of the hospital results in many young physicians or nurses who do not necessarily stay on the same ward for a long time. Moreover, the hospital in which the study was conducted has a rotation system in which clinicians rotate hospital-wide across different wards of the same discipline. Those 2 facts let us assume that the 2 compared samples are indeed independent. However, the lack of information regarding the documenting individual is preventing the use of advanced tests like mixed effect models. These could equally consider the record type on the one hand and the possible documenting individuals on the other hand, potentially advancing the results’ reliability. However, the 15-month interval from the implementation date of the EMR to the second data collection signifies that there is only little risk of any possible changes in documentation due to a bias from the described effects of preimplementational documentation training [41] since the employees indeed underwent software training before they were allowed to use the EMR. Therefore, the shown changes in completeness are, in fact, most likely due to the implementation of the EMR.
Conclusions
The results show that implementing EMRs can influence the completeness of documentation. A demonstrated improved completeness might also facilitate an improvement of the described outcomes that depend on documentation that is of high quality, like the availability [4] and analyzability of information [7,35], the coordination of care [5], or patient safety [6]. However, at the same time, the results show that a deterioration of completeness is also conceivable with the accompanied risks. This highlights the importance of understanding the underlying mechanisms that determine these changes. The knowledge may help stakeholders manage the implementation of new EMRs or the optimization of existing EMRs. Future research should address mechanisms that can improve documentation while simultaneously reducing the burden on practitioners caused by documentation tasks.
Acknowledgments
This work is funded by the German Federal Ministry of Education and Research (grant 01GP1906B). The sponsor had no influence on study design, data collection, analysis, or the writing process.
Abbreviations
- EMR
electronic medical record
- HIS
hospital information system
- ICD
International Classification of Diseases
- STROBE
Strengthening the Reporting of Observational Studies in Epidemiology
STROBE Checklist.
Q-Q-Plots.
Data Availability
The data sets generated and analyzed during this study are available from the corresponding author on reasonable request.
Footnotes
Authors' Contributions: FW and UK conceptualized the article. Data collection and analysis were performed by FW, supervised by UK. The original draft of the manuscript was written by FW, and all authors reviewed and edited previous versions of the manuscript and contributed to the interpretation of the data. All authors read and approved the final manuscript.
Conflicts of Interest: None Declared.
References
- 1.Gopal G, Suter-Crazzolara C, Toldo L, Eberhardt W. Digital transformation in healthcare - architectures of present and future information technologies. Clin Chem Lab Med. 2019;57(3):328–335. doi: 10.1515/cclm-2018-0658. https://www.degruyter.com/document/doi/10.1515/cclm-2018-0658 .cclm-2018-0658 [DOI] [PubMed] [Google Scholar]
- 2.Mangiapane M, Bender M. EMR Adoption Model (EMRAM) In: Mangiapane M, Bender M, editors. Patientenorientierte Digitalisierung im Krankenhaus. Wiesbaden: Springer Vieweg; 2020. pp. 33–39. [Google Scholar]
- 3.Jacob PD. Chapter 3 - Management of patient healthcare information: healthcare-related information flow, access, and availability. In: Gogia S, Novaes M, Basu A, Gogia K, Gogia S, editors. Fundamentals of Telemedicine and Telehealth. London: Academic Press; 2020. pp. 35–57. [Google Scholar]
- 4.Embi PJ, Weir C, Efthimiadis EN, Thielke SM, Hedeen AN, Hammond KW. Computerized provider documentation: findings and implications of a multisite study of clinicians and administrators. J Am Med Inform Assoc. 2013;20(4):718–26. doi: 10.1136/amiajnl-2012-000946. https://europepmc.org/abstract/MED/23355462 .amiajnl-2012-000946 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Vos JFJ, Boonstra A, Kooistra A, Seelen M, van Offenbeek M. The influence of electronic health record use on collaboration among medical specialties. BMC Health Serv Res. 2020;20(1):676. doi: 10.1186/s12913-020-05542-6. https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-020-05542-6 .10.1186/s12913-020-05542-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Yanamadala S, Morrison D, Curtin C, McDonald K, Hernandez-Boussard T. Electronic health records and quality of care: an observational study modeling impact on mortality, readmissions, and complications. Medicine (Baltimore) 2016;95(19):e3332. doi: 10.1097/MD.0000000000003332. https://europepmc.org/abstract/MED/27175631 .00005792-201605100-00010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Raghupathi W, Raghupathi V. Big data analytics in healthcare: promise and potential. Health Inf Sci Syst. 2014;2:3. doi: 10.1186/2047-2501-2-3. https://europepmc.org/abstract/MED/25825667 .14 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Dick RS, Steen EB, Detmer DE. The Computer-Based Patient Record: An Essential Technology for Health Care, Revised Edition. Washington: National Academies Press; 1997. [PubMed] [Google Scholar]
- 9.Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. J Am Med Inform Assoc. 2013;20(1):144–151. doi: 10.1136/amiajnl-2011-000681. https://europepmc.org/abstract/MED/22733976 .amiajnl-2011-000681 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Kahn MG, Callahan TJ, Barnard J, Bauck AE, Brown J, Davidson BN, Estiri H, Goerg C, Holve E, Johnson SG, Liaw ST, Hamilton-Lopez M, Meeker D, Ong TC, Ryan P, Shang N, Weiskopf NG, Weng C, Zozus MN, Schilling L. A harmonized data quality assessment terminology and framework for the secondary use of electronic health record data. EGEMS (Wash DC) 2016;4(1):1244. doi: 10.13063/2327-9214.1244. https://europepmc.org/abstract/MED/27713905 .egems1244 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Wurster F, Fütterer G, Beckmann M, Dittmer K, Jaschke J, Köberlein-Neu J, Okumu MR, Rusniok C, Pfaff H, Karbach U. The analyzation of change in documentation due to the introduction of electronic patient records in hospitals: a systematic review. J Med Syst. 2022;46(8):54. doi: 10.1007/s10916-022-01840-0. https://europepmc.org/abstract/MED/35781136 .10.1007/s10916-022-01840-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Montagna S, Croatti A, Ricci A, Agnoletti V, Albarello V, Gamberini E. Real-time tracking and documentation in trauma management. Health Informatics J. 2020;26(1):328–341. doi: 10.1177/1460458219825507. https://journals.sagepub.com/doi/10.1177/1460458219825507?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub0pubmed . [DOI] [PubMed] [Google Scholar]
- 13.Zargaran E, Spence R, Adolph L, Nicol A, Schuurman N, Navsaria P, Ramsey D, Hameed SM. Association between real-time electronic injury surveillance applications and clinical documentation and data acquisition in a South African trauma center. JAMA Surg. 2018;153(5):e180087. doi: 10.1001/jamasurg.2018.0087. https://europepmc.org/abstract/MED/29541765 .2674877 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Thoroddsen A, Ehnfors M, Ehrenberg A. Content and completeness of care plans after implementation of standardized nursing terminologies and computerized records. Comput Inform Nurs. 2011;29(10):599–607. doi: 10.1097/NCN.0b013e3182148c31. https://journals.lww.com/cinjournal/abstract/2011/10000/content_and_completeness_of_care_plans_after.11.aspx .00024665-201111000-00011 [DOI] [PubMed] [Google Scholar]
- 15.McCamley J, Vivanti A, Edirippulige S. Dietetics in the digital age: The impact of an electronic medical record on a tertiary hospital dietetic department. Nutr Diet. 2019;76(4):480–485. doi: 10.1111/1747-0080.12552. https://onlinelibrary.wiley.com/doi/10.1111/1747-0080.12552 . [DOI] [PubMed] [Google Scholar]
- 16.Karp EL, Freeman R, Simpson KN, Simpson AN. Changes in efficiency and quality of nursing electronic health record documentation after implementation of an admission patient history essential data set. Comput Inform Nurs. 2019;37(5):260–265. doi: 10.1097/CIN.0000000000000516. https://journals.lww.com/cinjournal/fulltext/2019/05000/changes_in_efficiency_and_quality_of_nursing.5.aspx .00024665-201905000-00005 [DOI] [PubMed] [Google Scholar]
- 17.Meier-Diedrich E, Davidge G, Hägglund M, Kharko A, Lyckblad C, McMillan B, Blease C, Schwarz J. Changes in documentation due to patient access to electronic health records: protocol for a scoping review. JMIR Res Protoc. 2023;12:e46722. doi: 10.2196/46722. https://www.researchprotocols.org/2023//e46722/ v12i1e46722 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Wiebe N, Varela LO, Niven DJ, Ronksley PE, Iragorri N, Quan H. Evaluation of interventions to improve inpatient hospital documentation within electronic health records: a systematic review. J Am Med Inform Assoc. 2019;26(11):1389–1400. doi: 10.1093/jamia/ocz081. https://europepmc.org/abstract/MED/31365092 .5542034 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Emekli E, Coscun Ö, Budakoglu I, Kiyak YS. Clinical record keeping education needs in a medical school and the quality of clinical documentations. Konuralp Med J. 2023;15(2):257–265. doi: 10.18521/ktd.1259969. https://dergipark.org.tr/en/pub/ktd/issue/78097/1259969 . [DOI] [Google Scholar]
- 20.von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, STROBE Initiative The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med. 2007;147(8):573–577. doi: 10.7326/0003-4819-147-8-200710160-00010. https://www.acpjournals.org/doi/abs/10.7326/0003-4819-147-8-200710160-00010?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub0pubmed .147/8/573 [DOI] [PubMed] [Google Scholar]
- 21.Beckmann M, Dittmer K, Jaschke J, Karbach U, Köberlein-Neu J, Nocon M, Rusniok C, Wurster F, Pfaff H. Electronic patient record and its effects on social aspects of interprofessional collaboration and clinical workflows in hospitals (eCoCo): a mixed methods study protocol. BMC Health Serv Res. 2021;21(1):377. doi: 10.1186/s12913-021-06377-5. https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-021-06377-5 .10.1186/s12913-021-06377-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Referenzdatenbank der Qualitätsberichte der Krankenhäuser. Gemeinsamer Bundesausschuss. 2023. [2023-12-22]. https://qb-referenzdatenbank.g-ba.de/
- 23.Prior L. Repositioning documents in social research. Sociology. 2008;42(5):821–836. doi: 10.1177/0038038508094564. https://journals.sagepub.com/doi/abs/10.1177/0038038508094564 . [DOI] [Google Scholar]
- 24.Ranegger R, Hackl WO, Ammenwerth E. Implementation of the Austrian Nursing Minimum Data Set (NMDS-AT): a feasibility study. BMC Med Inform Decis Mak. 2015;15:75. doi: 10.1186/s12911-015-0198-7. https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-015-0198-7 .10.1186/s12911-015-0198-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Toney-Butler TJ, Unison-Pace WJ. Nursing Admission Assessment and Examination. Treasure Island (FL): StatPearls Publishing; 2018. [PubMed] [Google Scholar]
- 26.Yadav S, Kazanji N, Narayan KC, Paudel S, Falatko J, Shoichet S, Maddens M, Barnes MA. Comparison of accuracy of physical examination findings in initial progress notes between paper charts and a newly implemented electronic health record. J Am Med Inform Assoc. 2017;24(1):140–144. doi: 10.1093/jamia/ocw067. https://europepmc.org/abstract/MED/27357831 .ocw067 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Muallem YA, Dogether MA, Househ M, Saddik B. Auditing the completeness and legibility of computerized radiological request forms. J Med Syst. 2017;41(12):199. doi: 10.1007/s10916-017-0826-0. https://link.springer.com/article/10.1007/s10916-017-0826-0 .10.1007/s10916-017-0826-0 [DOI] [PubMed] [Google Scholar]
- 28.Coffey C, Wurster LA, Groner J, Hoffman J, Hendren V, Nuss K, Haley K, Gerberick J, Malehorn B, Covert J. A comparison of paper documentation to electronic documentation for trauma resuscitations at a level i pediatric trauma center. J Emerg Nurs. 2015;41(1):52–56. doi: 10.1016/j.jen.2014.04.010. https://www.jenonline.org/article/S0099-1767(14)00177-9/fulltext .S0099-1767(14)00177-9 [DOI] [PubMed] [Google Scholar]
- 29.Seroussi B, Bouaud J. The (Re)-Relaunching of the DMP, the French shared medical record: new features to improve uptake and use. In: Ugon A, Karlsson D, Klein GO, editors. Building Continents of Knowledge in Oceans of Data: The Future of Co-Created EHealth. Amsterdam: IOS Press; 2018. pp. 256–260. [PubMed] [Google Scholar]
- 30.Jang J, Yu SH, Kim CB, Moon Y, Kim S. The effects of an electronic medical record on the completeness of documentation in the anesthesia record. Int J Med Inform. 2013;82(8):702–707. doi: 10.1016/j.ijmedinf.2013.04.004. https://www.sciencedirect.com/science/article/abs/pii/S1386505613001007?via%3Dihub .S1386-5056(13)00100-7 [DOI] [PubMed] [Google Scholar]
- 31.Brown ML. Can't you just pull the data? The limitations of using of the electronic medical record for research. Paediatr Anaesth. 2016;26(11):1034–1035. doi: 10.1111/pan.12951. https://onlinelibrary.wiley.com/doi/10.1111/pan.12951 . [DOI] [PubMed] [Google Scholar]
- 32.Pruitt Z, Pracht E. Upcoding emergency admissions for non-life-threatening injuries to children. Am J Manag Care. 2013;19(11):917–924. https://www.ajmc.com/pubMed.php?pii=85249 .85249 [PubMed] [Google Scholar]
- 33.Gesner E, Gazarian P, Dykes P. The burden and burnout in documenting patient care: an integrative literature review. Stud Health Technol Inform. 2019;264:1194–1198. doi: 10.3233/SHTI190415.SHTI190415 [DOI] [PubMed] [Google Scholar]
- 34.Saravi BM, Asgari Z, Siamian H, Farahabadi EB, Gorji AH, Motamed N, Fallahkharyeki M, Mohammadi R. Documentation of medical records in Hospitals of Mazandaran University of medical sciences in 2014: a quantitative study. Acta Inform Med. 2016;24(3):202–206. doi: 10.5455/aim.2016.24.202-206. https://europepmc.org/abstract/MED/27482136 .AIM-24-202 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Batko K, Ślęzak A. The use of big data analytics in healthcare. J Big Data. 2022;9(1):3. doi: 10.1186/s40537-021-00553-4. https://europepmc.org/abstract/MED/35013701 .553 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Ommaya AK, Cipriano PF, Hoyt DB, Horvath KA, Tang P, Paz HL, DeFrancesco MS, Hingle ST, Butler S, Sinsky CA. Care-Centered clinical documentation in the digital environment: solutions to alleviate burnout. National Academy of Medicine. 2018. [2023-12-22]. https://nam.edu/care-centered-clinical-documentation-digital-environment-solutions-alleviate-burnout/
- 37.Kapsner LA, Kampf MO, Seuchter SA, Gruendner J, Gulden C, Mate S, Mang JM, Schüttler C, Deppenwiese N, Krause L, Zöller D, Balig J, Fuchs T, Fischer P, Haverkamp C, Holderried M, Mayer G, Stenzhorn H, Stolnicu A, Storck M, Storf H, Zohner J, Kohlbacher O, Strzelczyk A, Schüttler J, Acker T, Boeker M, Kaisers UX, Kestler HA, Prokosch HU. Reduced rate of inpatient hospital admissions in 18 German University Hospitals during the COVID-19 lockdown. Front Public Health. 2020;8:594117. doi: 10.3389/fpubh.2020.594117. https://europepmc.org/abstract/MED/33520914 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Videokonferenz der Bundeskanzlerin mit den Regierungschefinnen und Regierungschefs der Länder am 28. Oktober 2020. Presse- und Informationsamt der Bundesregierung. 2020. [2023-12-22]. https://www.bundesregierung.de/resource/blob/997532/1805024/5353edede6c0125ebe5b5166504dfd79/2020-10-28-mpk-beschluss-corona-data.pdf?download=1 .
- 39.Curtis CA, Nguyen MU, Rathnasekara GK, Manderson RJ, Chong MY, Malawaraarachchi JK, Song Z, Kanumuri P, Potenzi BJ, Lim AKH. Impact of electronic medical records and COVID-19 on adult goals-of-care document completion and revision in hospitalised general medicine patients. Intern Med J. 2022;52(5):755–762. doi: 10.1111/imj.15543. https://europepmc.org/abstract/MED/34580964 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Wilcox RR. Introduction to Robust Estimation and Hypothesis Testing. Amsterdam: Academic Press; 2011. [Google Scholar]
- 41.Prokosch HU, Ganslandt T. Perspectives for medical informatics. reusing the electronic medical record for clinical research. Methods Inf Med. 2009;48(1):38–44.09010038 [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
STROBE Checklist.
Q-Q-Plots.
Data Availability Statement
The data sets generated and analyzed during this study are available from the corresponding author on reasonable request.
