Efficiency, Comprehensiveness and Cost-effectiveness when comparing Dictation and Electronic Templates for Operative Reports

Abstract

Surgeons typically document operative events using dictation services. Dictated reports are frequently incomplete or delayed. Electronic note templates could potentially improve this process. Using a study design of alternating four week blocks, we compared the timeliness and comprehensiveness of operative notes created through the use of electronic templates (structured text documents for reuse) versus dictation services for five surgical procedures. Templates resulted in dramatically faster times to the presence of a verified operative report in the medical record compared to dictation services (mean 28 v. 22,440 minutes). Templates increased overall compliance with national standards for operative note documentation and avoided transcription costs. Documentation with templates took slightly more time than dictation (mean 6.77 v. 5.96 minutes; P=0.036), not including the additional time necessary to subsequently verify dictated reports. We conclude that electronic note templates can improve the timeliness and comprehensiveness of operative documentation, while decreasing transcription costs and requiring minimal additional effort on the part of surgeons.

BACKGROUND

Documenting the events of a surgery as an operative report is standard practice for surgeons. Certain elements must be included to ensure that a report satisfies institutional and national standards.¹ After transcribing, the surgeon must review, correct errors, and ultimately “verify” the dictated report (Figure 1). There may be delays to the time of initial dictation by the surgeon, transcription of the dictated report, or final verification by the surgeon. Delays or lack of structure in dictation may also increase the likelihood that important elements may be missing from the final report.

Figure 1.

Time Frame for Operative Report Documentation

Structured electronic note templates could improve operative documentation. Templates are structured text documents that are created and stored for reuse and modification.² Such templates are particularly suited for well-structured and high-volume activities. Templates have been compared to hand-written notes to complete nursing assessments in the outpatient setting, ³ and were found to improve consistency in documentation, led to immediacy in availability, and resulted in better compliance with documentation guidelines.

We hypothesized that physician entry of structured electronic note templates could similarly improve operative reporting in the inpatient setting. In this study, we evaluated the effects of templates on timeliness to various stages of operative report completion, comprehensiveness and costs of the final verified documents, and the effort compared to the standard process of dictation.

METHODS

This study was conducted at Wishard Memorial Hospital (Indianapolis, IN), a university-affiliated, 264-bed, county-managed hospital from July to December 2004. Study subjects included physician housestaff and faculty of the Obstetrical and Gynecology (Ob-Gyn) Department assigned to one of 4 inpatient teams. We excluded Ob-Gyn housestaff and faculty who were on a team for less than 7 days (e.g., weekend only, holiday only, cross coverage) or did not receive initial orientation to the study as explained below.

Physicians routinely enter all hospital orders and can document patient encounters directly in the “Gopher” physician order entry system⁴ on workstations located throughout the hospital. The institutional review board of Indiana University approved the study.

Development of templates

We chose to construct templates for the following procedures: cesarean section, tubal ligation post-partum, total abdominal hysterectomy, vaginal hysterectomy, and laparoscopic tubal ligation. We chose these procedures on the basis of Ob-Gyn faculty preferences, as well as their beliefs regarding those procedures most commonly performed by their department. We confirmed their beliefs about common procedures by reviewing recent Ob-Gyn billing office records. The procedures chosen for templates accounted for approximately 75% of all major gynecological procedures.

We created the template content in coordination with Ob-Gyn faculty based upon previously dictated reports, departmental standards, health information management, and the Joint Commission of Accreditation and Healthcare Organizations (JCAHO) standards regarding operative report documentation.¹

Subject assignment

The subjects had rotations on the inpatient service that varied between 5 and 7 weeks, depending on their level of training. By examining each rotation schedule ahead of time, we determined a single 4 week timeframe during which team faculty and housestaff remained constant. These 4 week blocks served as our study periods. We randomized the first block with a coin flip and then subsequently alternated experimental and control blocks (a total of 4 four-week blocks). At the beginning of all study rotations, subjects received a 20 minute orientation about the study and instructions regarding the electronic templates. We alerted subjects to the study through the use of posted signs, a departmental memo, and email messages.

During “dictation blocks,” the subjects were instructed to dictate reports as they would for any other procedure. We prevented access during dictation blocks to templates within the order entry system.

Conversely, during “template blocks,” the subjects were instructed to avoid dictation and document all appropriate operative events using the 5 study templates. Since not all Ob-Gyn procedures could be documented using note templates, the subjects continued to have access to the dictation service during the “template blocks.” Electronic note templates were embedded into the “Procedure Note” section of the order entry system. The five available templates were included on the choice list. As depicted in Figure 2, templates consisted of a series of fill-in-the-blanks (FITBs). For FITBs with a self-limited number of answers, there were drop-down menus, but the subjects had the option of typing in their own answers. After template completion, the text was presented to the subjects in a text window that allowed direct editing immediately prior to “signing” the note.

Figure 2.

Screenshot of template from “Gopher”

The template required completion of 85% of FITBs - those judged critical to a complete operative note (Table 2). If required FITBs were not filled, the display would return to the first neglected required FITB, not allowing completion of the template until answered. “Optional” sections of the note template in which the subject chose not to complete the FITBs were automatically deleted prior to final review by the subject (including the “fixed text” of the optional section).

Table 2.

Required sections and comparisons of Operative Reports

Section Title	Dictation Group(%)	Template Group(%)
Date of procedure a,b,d	100	100
Primary surgeon	98	100
(1st assistant) a,b,c,d	99	99
Preoperative diagnosis a,d	100	99
Postoperative diagnosis a,b,c,d	100	100
Procedures performed a,c,d	100	100
Anesthesia a,d	98	100
Estimated blood loss (EBL) a,b,c,d	96	94
Intravenous fluids (IVF) d	91	100
Urinary output (UO) d	31	62
Findings a,b,c,d	89	100
Indications b,d	73	100
Complications a,b,d	80	100
Description of the procedure a,b,c,d	100	100
Specimens removed a,b,c,d	60	99
Overall	85	97

^a- Interpretive guidelines for the operative report¹

^b- Sections required from the study institution

^c- Sections required for post-operative notes⁵,⁶

^d- Sections for determining completeness of operative report

The subject could then choose to electronically sign the order session, with the completed template report immediately stored into the patient’s electronic medical record and available for review by other health care providers. At the end of “template blocks,” subjects completed surveys comparing templates and dictation.

Data Collection

The billing office provided a list of patients that contained the patient medical record numbers (MRNs), dates of service, surgeons who verified the individual reports, and associated CPT codes corresponding to the 5 procedures in the study. For dictated reports, the dictation service (MedQuist, Nashville, TN) provided the patient’s MRNs, dates and times of dictation, and the total time that the physician spent dictating. Finally, we retrieved information from the Gopher order entry system, including the date and time that documents created using electronic templates were stored and how long it took the surgeons to complete the notes using templates.

In the case of dictations, the dictation service was unable for technical reasons, to provide the total time spent dictating for 8 (4%) dictations. In the case of electronic templates, for other technical reasons, we were unable to collect the time spent completing the template for 24 (17%) documents. These technical problems were unrelated to characteristics of the subjects, intervention, or outcomes. But, we confirmed that there were no significant differences in either length or overall comprehensiveness of documents completed by dictation with or without duration. We similarly found no such differences when comparing documents completed by templates with or without durations.

All time intervals in this study for both groups are relative to the operation end time. We collected operation end times from surgical record. If data about procedures were missing from these records, then we reviewed anesthesia records for the necessary information. The operation end time, as established by protocol, is when the operation was completed and the patient is transferred from the operating room to recovery. It was possible for housestaff to complete the operative report prior to completion of the operation end time, therefore “negative” times are possible.

All operative reports were reviewed by one of the investigators (MRL) to confirm that they were appropriately assigned relevant study CPT codes. By determining if section titles and the content of the sections were found anywhere in the report, the reports were evaluated for completeness and then calculated as a percentage. Those sections and section titles that were not present or did not provide enough information were considered incomplete.

Statistical analysis

We collected data for the first 4 blocks of this study (2 control, 2 experimental) and calculated means, medians, standard deviations, and confidence intervals. In 4 of the reports from the control group, some time intervals were greater than 5 standard deviations above the mean and were trimmed to the next highest value to lessen their influence on the descriptive statistics and tests. When the outliers were included, the data supported the results even more for the experimental group. Differences between the control and experimental groups were tested using generalized linear mixed models that accounted for multiple procedures and/or blocks per subject. Line counts of reports were calculated using Perl script. All calculations were made using Stata statistical software (Release 7.0. College Station, TX: Stata corporation) and linear mixed models were used with SAS statistical software (Version 9.0. Cary, NC: SAS Institute, Inc.).

RESULTS

During the 4 four-week study blocks, billing office records indicated that Ob-Gyn housestaff performed a total of 478 procedures on 440 individual inpatients that corresponded to the 5 study procedures (Figure 3). We also removed 3 procedures (3 patients) where a CPT code was collected but the procedure was never documented. We also removed 10 duplicate reports from the “dictation group” and 13 duplicate reports from the “template group.”

Figure 3.

Distribution of procedures

Forty-five unique housestaff subjects participated in the study. The subjects rotated an average of 1.58 times on the inpatient service: 17 (38 percent) were only in a “dictation group,” 13 (29 percent) were only in a “template group,” while the remaining 15 (33 percent) were found in both groups. Twelve (27 percent) subjects were faculty while the remaining were housestaff. As previously noted, we could not prevent housestaff from dictating during template rotations. Consequently, 33 procedures that would have been most appropriately completed through the use of templates were dictated during “template group” blocks. We found no major differences in our results whether we included or excluded these 33 dictated procedures in the “template group.”

The breakdown of reports and average times for the “dictation group” and “template groups” are listed in Table 1. The mean time from the end of the operation to when the report was entered was considerably less when using electronic templates than dictation (mean 0.43 v. 9.7 hours; P<0.001). The mean time from the end of the operation to when the information system stored the operative report (“preliminary document”) was two orders of magnitude less for the “template group” than the “dictation group” (mean 0.46 v. 123 hours; P<0.001). The mean time for electronic verification in the “dictation group” was over 14 days (in many hospital systems, the operative report would not be widely accessible until this final verification step).

Table 1.

Time intervals and length of documents completed by dictation and template groups ^*

	Template Group N = 138		Dictation Group N = 198†
	Mean* (hours)	Median (hours)	Mean* (hours)	Median (hours)	p-value
Time to completion of initia dictation (hours)‡	N/A	N/A	9.7	0.43
Time to preliminary document (hours)‡	N/A	N/A	123	72
Time to “verified” completed document (hours)	0.46	0.16	374	194	<0.001
Lines of text	45.9	47.0	51.9	48.5	<0.001

^*- All times recorded from the end of the operation.

^†- Four of the outliers (with values greater than 5 standard deviations above the mean) were trimmed to the next highest value

^‡- The use of templates results in a complete, “verified” document. Time to initial dictation or a preliminary is not applicable (N/A) to the template group.

The mean time needed to produce an operative report was slightly longer in the “template group” than the “dictation group” (mean 6.77 v 5.96 minutes; P=0.036), a 13% or 49 second increase compared to dictation. However, we did not attempt to assess the following in the “dictation group”: the amount of time needed to write the mandatory abbreviated postoperative note, the subsequent time to proofread the transcribed report, or to verify it electronically. (In the template group, completion of the template obviated the need for an abbreviated post-operative note and there is no subsequent separate process of verification).

We evaluated the comprehensiveness of reports by examining whether key elements were found anywhere in the report text (not simply in certain sections). Overall inclusion of these key elements (Table 2) for the “template group” was higher than the “dictation group” (97 v. 85 percent; P<0.001.

“Dictation group” reports were modestly longer than “template group” reports (52 v. 46 lines of text; P<0.001). For dictation services, the cost at our institution is 12.5 cents per line of text. For the approximately 100 reports produced by 12 housestaff and 4 faculty for 4 weeks, adoption of these templates corresponds to an average monthly savings of $650.

DISCUSSION

We found that the use of templates resulted in the presence of a verified operative report in the patient’s medical record (on average) within less than half of an hour compared to over 14 days for the standard dictation process (an 800-fold difference). This is clinically important insofar as patients may seek medical care within such a time window for either routine post-operative management, or on occasion, for post-operative complications. While the difference in times could have been predicted with almost certitude, the study verified that there was a true difference. In addition, the study also quantified this large difference. Consequently, JCAHO also has requirements for timely operative reports.⁷

A number of causes likely underlie the delays associated with dictation of operative reports. First, given that the mean time to completion of initial dictation was 9.7 hours, dictation is not consistently performed in the immediate post-operative period (although the median of 0.43 hours indicates that it was commonly done in this period). In contrast, the relatively short mean and median times for electronic note templates indicate that they were consistently completed immediately after the surgery. We believe that the surgeons used the opportunity afforded by mandatory electronic post-operative orders (required by hospital policy) to simultaneously complete the appropriate electronic template. Finally, we observed long delays in both transcription and verification (signing) of the reports. The verification process is complicated by the fact that housestaff often rotate to different hospital, with faculty too often learning of unverified reports weeks after the procedure was performed.⁸

Template-based operative reports were found to be significantly more likely to include the elements deemed important by national guidelines. We confirmed prior findings that electronic templates can improve documentation^3,9 and increase availability.³ Previously published articles suggest that using additional sections assist health care providers in identifying additional drains, ensure identical documentation with nursing and anesthesia records,⁶ in recording data for clinical studies,¹⁰ and in providing thorough documentation for medico-legal and billing¹¹ purposes. Our intervention directly leads to cost-savings through the reduction of dictation services. There are also likely indirect financial benefits, since payment for procedures may be deferred or denied altogether when certain components of the report are not fully documented.¹¹ One published study determined that 76% of dictated operative notes were deficient from a billing standpoint.¹¹

Our findings supplement prior descriptions of the shortcomings of dictation systems. The housestaff must dial the number, and enter the correct prompts for identifiers (e.g., MRN, housestaff), work types, etc. though the touch pad with no visual feedback. In one study, over 10% of dictated discharge summaries and operative reports had incorrect MRNs or patient names.¹² Recorded voice patterns (e.g., mumbling, mastication, verbal anecdotes unrelated to procedures, etc.) may be unrecognizable. With templates, subjects entered information directly into the electronic record without confusion about MRNs; subjects are able to validate all sections and correct mistakes at the time of note entry. In addition, subjects in the “template group” who had to write post operative orders in the computer did not have to switch from computer to the telephone for dictation.

After operative reports were dictated, transcribed and accepted by the system, they were considered “complete” only when housestaff verified the electronic copies, corrected mistakes and signed them electronically. With unverified reports, housestaff frequently needs to be contacted, but returning pages are not always certain. When documents are not “completed” or cannot be located, there are delays in coding and, eventually, reimbursement. JCAHO management standards are breached if charts are not completed within 30 days from the date of discharge.¹³

There are likely a number of reasons why some subjects did not use templates during “template group” blocks. Although subjects were given a study presentation and reminders, the subjects failed to either recall the study dates or did not recall that they were participating in any study. In reviewing some of the surgical cases, templates may not have been occasionally used because cases were particularly complex. For example, dictation services were sometimes employed if another service was assisting (e.g., simultaneous cystocoele repair by Urology).

CONCLUSIONS

Although slightly more time was necessary for completion, electronic templates resulted in dramatically decreased delays and increased comprehensiveness of operative documentation, with consequent cost savings.