Accuracy and completeness of electronic medical records obtained from referring physicians in a Hamilton, Ontario, plastic surgery practice: A prospective feasibility study

Increased computing power, and advances in telecommunication and information technology over the past 30 years have significantly improved health care delivery, and have facilitated allocation of resources and collaboration among health care professionals. In addition, the emergence of electronic medical records and increased access to health care databases has opened new avenues for research and potentially improved patient care. However, the efficacy of electronic medical records with regard to patient outcomes has, paradoxically, not undergone extensive study.

Abstract

OBJECTIVE:

To assess the feasibility of auditing electronic medical records (EMRs) in plastic surgery for future large-scale research studies. The secondary objective was to ascertain the accuracy and completeness of EMRs accompanying referral requests by physicians for plastic surgery consultation between July and December 2013.

METHODS:

EMRs of 30 patients were reviewed and crosschecked independently by two reviewers and subsequently verified by a third reviewer using predefined criteria to determine whether they were accurate and/or complete. Descriptive analysis was performed to calculate the frequency of inaccuracies and incompleteness for each EMR information field. Information fields were compared to assess whether the frequency of inaccuracies and incompleteness varied.

RESULTS:

Of the 270 information fields reviewed, four (1.48%) were inaccurate and 66 (24.4%) were incomplete. The most common field of inaccuracy was current medications, followed by medical history and medical allergies. The most common field of incompleteness was history of presenting illness followed by surgical history.

CONCLUSION:

Despite their purported benefits, inaccuracies and incompleteness are a frequently occurring problem in EMRs. A large-scale study may be beneficial in determining the efficacy of EMRs in the future.

There has been mounting recognition of the role of information technology systems in the provision of health care. Electronic medical records (EMRs) – an initiative of the 1970s – have measurably enhanced the quality of health care. They facilitate access, availability and legibility, which encourage collaboration among health care professionals (1). Arguably more important, EMRs decrease medical errors by providing ‘complete’ patient information and history (1).

Despite the purported benefits, evidence regarding the effect of EMRs on patient and health care team outcomes is limited (2). Furthermore, specific individual experiences concerning EMRs are poorly documented, contributing to challenges in determining their efficacy (2). Assessing their strengths and weaknesses is important because EMRs are associated with substantial financial implications, and ineffective EMRs would represent a waste of the vast resources required to implement and maintain EMR systems (3). More importantly, ineffective records can seriously undermine patient safety and lead to a significant number of medical errors.

Most studies suggest that EMRs provide accurate and complete patient records (1,4,5). However, we had noticed both quantitative and qualitative inaccuracies (eg, wrong medications) and incompleteness (eg, incomplete documentation of medications and allergies) in EMRs obtained from referring physicians, and decided to evaluate the accuracy and completeness of EMRs. Tse and You (6) have previously noted discrepancies; however, this was a pilot study with several methodological limitations. The purpose of our study was to assess the feasibility of auditing EMRs in plastic surgery for a future large-scale research study. The secondary objective was to ascertain the accuracy and completeness of EMRs accompanying referral requests by physicians for plastic surgery consultation between July 15 and December 15, 2013.

METHODS

The present analysis was a prospective cohort study. The Hamilton Integrated Research Ethics Board (Hamilton, Ontario) provided approval (Project Number: 13-526-C).

Data collection

The setting for the present study was the practice of the senior author (AT), a plastic surgeon practicing in Hamilton, Ontario. This is a diverse plastic surgery practice that includes both reconstructive and cosmetic adult patients. His patients are referred from Hamilton and the surrounding area, specifically the Hamilton Niagara Haldimand Brant Local Health Integration Network. In the course of the patient consultation, a complete up-to-date medical record was documented to include patient demographics, reason for referral, history of presenting illness, previous surgical/medical history, current medications and medical allergies. A standardized form designed a priori that included these information fields was used to collect data by the senior author and, subsequently, to compare against EMRs obtained from referring physicians. To minimize recording patient information that had changed between the time patients were referred and seen at the plastic surgery clinic, the senior author asked each patient whether any information had changed during the wait period. This was performed for each information field.

Inclusion and exclusion criteria

In the present study, all patients were included provided they had accompanying EMRs from their referring physicians.

Data abstraction

Using a data abstraction form in Excel (Microsoft Corporation, USA), data were extracted by two reviewers (CH, MK) independently. The reviewers crosschecked the EMRs of 30 patients referred between July 15 and December 15, 2013 with the senior author’s medical records to assess their accuracy and completeness. The definition used for an inaccurate record was one that had a qualitative or quantitative error, while the definition used for an incomplete record was one that omitted information necessary to gain a full perspective on the patient and/or the nature of patient’s disease. These definitions did not apply to patient information that had changed between the time patients were referred and seen at the plastic surgery clinic, as determined by the senior author at the time of patient consultation. In addition, the definition of an inaccurate record did not include spelling errors.

Data extracted/reviewed included patient demographics (ie, health insurance number, age, date of birth, sex), reason for referral, history of presenting illness, previous surgical/medical history, current medications and medical allergies. Each patient was given a unique identifier number to ensure confidentiality and privacy of the patients. Discrepancies in data abstraction were settled through consensus and, in the case of persistent discrepancies, discussion with a third reviewer (AT). In addition, the senior author reviewed every EMR information field that was deemed to be inaccurate or incomplete by the reviewers. To ensure the uniformity of data abstraction, each reviewer completed a calibration exercise before beginning data abstraction. The kappa statistic was calculated to determine concordance between the reviewers.

Sample size calculation and statistical analyses

Because the present analysis was a feasibility study, sample size was not calculated. Based on the data collected and extracted, descriptive analysis was performed to calculate the frequency of inaccuracies and incompleteness for each EMR information field. Information fields were compared to assess whether the frequency of inaccuracies and incompleteness varied.

RESULTS

In the present study, the level of concordance for the calibration exercise was determined to be ≥90%. Table 1 summarizes the demographics of the 30 patients, along with the specialties of referring physicians (ie, family physicians versus specialists) and time from when EMR was received to when the patient was seen. Notably, the majority of patients were referred by family physicians and the mean wait period was 95.6 days. Table 2 is a descriptive analysis demonstrating the frequency of inaccuracies and incompleteness for each EMR information field. The data revealed that all EMRs had accurate and complete age/ date of birth, reason for referral, health insurance number and sex. However, inaccuracies and/or incompleteness were found in all other information fields. Inaccuracies were found in the following information fields: current medications (n=2), medical history (n=1) and medical allergies (n=1) (Figure 1). Figure 2 presents a pie chart of incompleteness in EMRs, with each segment representing an information field. The most common field of incompleteness was history of presenting illness (n=22), followed by surgical history (n=19) and medical history (n=12).

TABLE 1.

Patient demographics

Patients, n	30
Referring physician
Family physician	25 (83.3)
Specialist	5 (16.7)
Time from when EMR was received to when patient was seen, days
Mean ± SD	95.6±57.7
Median (minimum, maximum)	106.5 (7, 216)

Data presented as n (%) unless otherwise indicated. EMR Electronic medical record

TABLE 2.

Accuracy and completeness of electronic medical records

	Accurate		Complete
	Yes	No	Yes	No
Age/date of birth	30 (100)	0 (0)	30 (100)	0 (0)
Reason for referral	30 (100)	0 (0)	30 (100)	0 (0)
History of presenting illness	30 (100)	0 (0)	8 (26.7)	22 (73.3)
Medical history	29 (96.7)	1 (3.3)	18 (60.0)	12 (40.0)
Surgical history	30 (100)	0 (0)	11 (36.7)	19 (63.3)
Current medications	28 (93.3)	2 (6.7)	24 (80.0)	6 (20.0)
Medical allergies	29 (96.7)	1 (3.3)	23 (76.7)	7 (23.3)
Health insurance number	30 (100)	0 (0)	30 (100)	0 (0)
Sex	30 (100)	0 (0)	30 (100)	0 (0)

Data presented as n (%)

Figure 1).

Inaccuracies in electronic medical records. Each segment represents a corresponding information field

Figure 2).

Incompleteness in electronic medical records. Each segment represents an information field

DISCUSSION

The present analysis was the first comprehensive feasibility study to evaluate the frequency of inaccuracies and incompleteness in EMRs obtained from referring physicians over a five-month period. Despite the commonly held belief that EMRs decrease medical errors by providing ‘complete’ patient information and history, our results indicate that inaccuracies and incompleteness are a frequently occurring problem. Of the 270 EMR information fields that we reviewed, four (1.48%) were inaccurate and 66 (24.4%) were incomplete.

The critical errors identified in the present study are important because they potentially affect patient safety. For instance, only 22 of 30 EMRs analyzed had accurate and complete current medications and medical allergies, meaning that in eight cases, wrong medications could have been administered with possible serious consequences. Furthermore, 22 EMRs had incomplete history of presenting illness, which delayed the assessment and treatment of these patients. This suggests that there is a considerable discordance between an ideal setting in which EMRs reduce medical errors and enhance quality of care, and a practical setting in which EMRs are often associated with inaccuracies and incompleteness.

Interestingly, there was a considerable variation among EMR information fields, with several having no inaccuracies or incompleteness, and others, such as history of presenting illness, having frequencies of incompleteness as high as 22 per 30 medical records. This may be explained, in part, by the fact that in many cases referring physicians used EMRs to simply denote their reason for referral rather than documenting complete patient information and history. EMRs are highly user dependent, and reported clinical benefits of EMRs are likely highly dependent on meaningful use and longer EMR experience by referring physicians (5).

Comparison with previous publication

In 2001, Tse and You (6) evaluated the frequency of inaccuracies and incompleteness in EMRs and found that 39% and 51% of information on allergies and medications, respectively, were inaccurate. Furthermore, they found a relatively high percentage (36%) of incomplete information. The frequency of inaccuracies and incompleteness found in the present study was considerably lower compared with the frequency reported by Tse and You (6). The difference in our findings was likely the result of variance in study design, with the criteria applied in our study being more thorough and stringent.

Strengths and weaknesses

There were several limitations to the present study. First, because it relied on patient interviews to obtain data, recall bias was a possibility. In addition, because the senior author conducted patient interviews, he may have been prone to asking questions and documenting patient history more thoroughly. He likely collected more accurate and more complete data. However, in contrast, the senior author’s thorough interviews and more accurate/complete data were advantageous because they were considered to be the gold standard in our study. The subjective nature of ‘completeness’ is another important limitation: what a referring physician considers to be ‘complete’ may not be ‘complete’ for a plastic surgeon in a tertiary care centre. Finally, the study was underpowered to draw definitive conclusions. To increase precision and internal validity, we needed to review at least 292 EMRs to achieve 95% CIs of ±2.5% for accuracy and completeness of ≥95%.

CONCLUSION

Our findings suggest that, despite the purported benefits of EMRs, they often contain inaccurate and incomplete patient information and history. Evidence from the present study suggests that a future large-scale study auditing EMRs to assess their efficacy is feasible.