Use of Computer-based Records, Completeness of Documentation, and Appropriateness of Documented Clinical Decisions

Abstract

Objective: To investigate whether using a computer-based patient record (CPR) affects the completeness of documentation and appropriateness of documented clinical decisions.

Design: A blinded expert panel of four experienced internists evaluated 50 progress notes of patients who had chronic diseases and whose physicians used either a CPR or a traditional paper record.

Measurements: Completeness of problem and medication lists in progress notes, allergies noted in the entire record, consideration of relevant patient factors in the progress note's diagnostic and treatment plans, and appropriateness of documented clinical decisions.

Results: The expert reviewers rated the problem lists and medication lists in the CPR progress notes as more complete (1.79/2.00 vs. 0.93/2.00, P < 0.001, and 1.75/2.00 vs. 0.91/2.00, P < 0.001, respectively) than those in the paper record. The allergy lists in both records were similar. Providers using a CPR documented consideration of more relevant patient factors when making their decisions (1.53/2.00 vs. 1.07/2.00, P < 0.001), and documented more appropriate clinical decisions (3.63/5.00 vs. 2.50/5.00, P < 0.001), compared with providers who used traditional paper records.

Conclusions: Physicians in our study who used a CPR produced more complete documentation and documented more appropriate clinical decisions, as judged by an expert review panel. Because the physicians who used the CPR in our study volunteered to do so, further study is warranted to test whether the same conclusions would apply to all CPR users and whether the improvement in documentation leads to better clinical outcomes.

The medical record has been used for more than a century as a tool to assist clinicians in the care of patients. Today, the medical record has a comprehensive purpose: “to recall observations, to inform others, to instruct students, to gain knowledge, to monitor performance, and to justify interventions.”¹ Whether the record has evolved to meet these contemporary goals is subject to debate. An early critic of the utility of the record, Florence Nightingale, found the record lacking in many respects. In her 1873 essay Notes on a Hospital, she wrote

In attempting to arrive at the truth, I have applied everywhere for information, but in scarcely an instance have I been able to obtain hospital records fit for any purposes of comparison. If they could be obtained, they would enable us to decide many other questions besides the one alluded to. They would show subscribers how their money was being spent, what amount of good was really being done with it, or whether the money was not doing mischief rather than good.

Today her words still ring true. Managed care, outcome research, cost-effectiveness studies, and database mining are contemporary vernacular for the information needs that Nightingale articulated so clearly more than a century ago.

Subsequent commentators have documented the limitations of the current record-keeping system. Several investigators have quantified the problem of data missing from the record itself.^2,3,4,5 One study examined the ability of physicians to find information in the record: In 81 percent of return visits to an ambulatory care clinic, because relevant patient information was inaccessible clinicians either postponed their decisions or made decisions based on incomplete information; on average, four pieces of information relevant to the decisions made during the encounter were not available to the physicians.⁶ Such empirical studies validate clinicians' impressions that the current paper record is an inadequate tool to support the task of providing patient care.

Not only is the paper record a poor repository of patient information, but also the tasks associated with such record keeping consume up to 38 percent of the physician's time associated with an outpatient visit.⁷ With mounting pressures on physicians to improve the cost and time efficiency of patient-care encounters, tasks that consume more than one third of the physicians' time are logical targets for analysis and improvement. In one study, physicians were better able to predict their patients' future symptoms and laboratory test results when they had access to a computer-based flowsheet of summary data.⁸ In this study, we compared the completeness of documentation and the appropriateness of documented decisions of physicians who volunteered to use a CPR with those of physicians using traditional paper records in the same clinic.

Methods

A panel of four expert physicians analyzed medical records from an internal medicine practice in which approximately one half of the clinicians (15 of 33) had been using a CPR since July 1996. The expert panel members were board-certified in internal medicine and had been in practice for an average of 28 years. No expert panel member was involved in the CPR project or was a member of the practice studied. The CPR system (EpicCare, developed by Epic Systems, Madison, Wisconsin) allowed providers to enter directly problems, medications, allergies, progress notes, and orders. Any active problems, medications, and allergies that are entered into a patient's record are automatically included in progress notes. Laboratory test results are entered into the record automatically through electronic interfaces and can be displayed in flowsheet or graphic formats. During the study period, the CPR did not provide clinical alerts.

All study providers practicing in the internal medicine clinic were board-certified attending physicians. Approximately half the physicians elected to participate in a pilot project that involved using a CPR in daily practice. The group of physicians using the CPR (CPR users) included 11 men and 4 women. The remaining physicians, who continued to use the paper record, included 6 men and 8 women. The average number of years in practice for the physicians using a CPR was 11; that for physicians using a paper record was 12.

To assess the possible effects of a CPR on clinical documentation, patient records were selected as the units of analysis. Included in the pool of eligible records were those associated with patients diagnosed with one or more chronic diseases (e.g., asthma, hypertension, diabetes, congestive heart failure, hypothyroidism, chronic renal failure, or rheumatoid arthritis) who were seen in the internal medicine clinic four or more times between September 1996 and August 1997. There were 491 records of patients whose physicians used the CPR and 435 records of patients whose physicians used the traditional paper record. We randomly selected patient records from the eligible pools. We conducted a power analysis to determine an appropriate sample size of records for the study cohorts.⁹

For each randomly selected medical record, progress notes were transcribed into a standard format. A transcribed version of the medical record face sheet (containing the problem, medication, and allergy lists, when available), four consecutive internal medicine clinic progress notes, and the results of diagnostic tests ordered during the covered encounters were assembled as a “mini-record” for the expert panel.

Each expert reviewed an identical set of 56-mini-records. Twenty-five mini-records represented those of providers using a CPR; 25 represented those of providers using paper records. Six duplicate records—three from each group—were added to the study sets so that we could assess intracoder reliability. The experts were blinded to whether a note was recorded by a provider using a CPR or a paper record. All reviewers evaluated five aspects of each of the 56 medical records. They were instructed to rate the completeness of the problem list (on a scale from 0, indicating “does not exist,” to 2, “itemizes most major problems for this patient”) and medication list (on a scale from 0 to 2, where 2 indicates “itemizes most active medications and instructions”) in the most recent progress note by using information in the entire mini-record. Thus, the reviewer is evaluating the documentation of what transpired during the most recent encounter in the context of information from recent previous visits. This is intended to be similar to the information context in which the provider operated during the actual encounter. Reviewers rated the completeness of allergy documentation anywhere in the mini-record (on a scale from 0 to 2, where 2 indicates “itemizes most allergies”). The reviewers were then instructed to rate the degree to which relevant patient factors were considered in the assessment and plan (on a scale from 0, indicating “considered none of the relevant patient factors,” to 2, “considered most of the relevant patient factors”) of the most recent progress note in the mini-record using the information in the entire mini-record. Similarly, the reviewers were asked to rate the appropriateness of the assessment and diagnostic and treatment decisions documented during the most recent encounter (on a scale from 0, indicating “completely inappropriate,” to 5, “completely appropriate”) considering information in the entire mini-record. All data were entered into SPSS version 7.0 for statistical analysis.

Results

Each reviewer completed a review of 56-mini-records. We evaluated intracoder reliability using a test—retest reliability model. The calculated intraclass correlation coefficient was 0.94 (using the Kendall coefficient of concordance). Coefficients above 0.90 are considered highly reliable.¹⁰ Characteristics of the patients whose records were reviewed appear in ▶.

Table 1.

Characteristics of Patients Whose Charts Were Reviewed

Characteristic	CPR Group	Paper-record Group
Male	17	14
Female	8	11
Mean age (years)	55.8	56.8
Average no. of problems	6.2	5.6
Average no. of current medications	7.4	5.8
Note: CPR indicates computer-based patient record.

We compared (Using the Mann-Whitney U-test) the experts' ratings of medical records from the two groups of providers in each of the five aspects evaluated (▶). The expert reviewers rated the problem lists and medication lists in the CPR progress notes as significantly more complete than those in the progress notes in the paper record (1.79/2.00 vs. 0.93/2.00, P < 0.001, and 1.75/2.00 vs. 0.91/2.00; P < 0.001, respectively). The allergy information in the entire CPR and paper record were similar (1.97/2.00 vs. 1.85/2.00, P = 0.07). Physicians who used a CPR provided more evidence in their documented assessment that they had considered relevant patient factors in making their decisions (1.53/2.00 vs. 1.07/2.00, P < 0.001) and documented more appropriate decisions (3.63/5.00 vs. 2.50/5.00, P < 0.001) than did those who used traditional paper patient records. Examples of progress notes from a CPR user and from a paperrecord user are shown in Figures ▶ and ▶.

Table 2.

Comparison of Completeness of Documentation and Appropriateness of Medical Decisions Made, Between Providers Who Used Computer-based Patient Records (CPRs) and Those Who Used Paper Records

Rating Dimension	n	CPR Users (Mean)/Highest Possible Score	Paper-Record Users (Mean)/Highest Possible Score	Significance
Major problems documented in problem list	169	1.79/2.00	0.93/2.00	<0.001
Medications documented in medication list	171	1.75/2.00	0.91/2.00	<0.001
Allergies documented in mini-record	169	1.97/2.00	1.85/2.00	0.07
Relevant clinical factors considered in assessment and plan	172	1.53/2.00	1.07/2.00	<0.001
Appropriateness of assessment and plan	172	3.63/5.00	2.50/5.00	<0.001

Figure 1.

Sample progress note from a computer-based patient record user. This record was assigned an average score of 4.0 for appropriateness by the expert review panel.

Figure 2.

Sample progress note from a paper record user. This progress note was assigned an average score of 2.3 for appropriateness by the expert review panel.

More thorough documentation of consideration of relevant patient factors was independently and significantly correlated with a more complete progress note problem list (P < 0.01) and with a more complete progress note medication list (P < 0.01), by a twotailed Kendall tau-b analysis. Furthermore, there was a statistically significant correlation between documentation of consideration of relevant factors in the assessment and plans and the appropriateness of the documented medical decisions (P < 0.01), as judged by the expert panel.

Discussion

As noted in the introduction, prior studies have documented several limitations of the paper medical record. In our study, a blinded expert panel judged the documented decisions of physicians using a CPR as more appropriate than those using traditional paper records in the same clinic.

At least three possibilities could explain part of the differences in appropriateness of documented decisions when physicians used a CPR. First, having complete information available during patient encounters may have directly improved the decisions made. Second, the improved legibility and organization of data may have facilitated physicians' ability to find information. Third, physician use of a CPR may have affected documentation habits.

Since the CPR system automatically included previously entered problems, medications, and allergies in the progress note, one would expect that CPR users' progress notes would be more complete (assuming the physician entered the information in the past). Obviously, this is a benefit of using a CPR. Theoretically, if the face sheet in a paper record were accurately maintained and providers referred to it during the course of an encounter, paper-record users and CPR users would have access to similar information. However, in the opinion of the blinded reviewers, the documented consideration of relevant problems and medications was less complete and the documented decisions were less appropriate.

During the early implementation of a CPR at Duke, investigators evaluated the extent to which previously noted problems, findings, and laboratory results were overlooked when a CPR was used, compared with a manual record.¹¹ The investigators randomized patients to be seen either with a CPR or with a paper record. During encounters using the CPR, providers received a preprinted encounter form, which included a flowsheet of previous subjective, objective, and laboratory data. After the provider wrote notes on the encounter form, clerks entered the data into the computer system. After each visit, one of the physician investigators examined the chart to determine whether any information in the record had not been acted on. They found that physicians acted more frequently on information recorded in the CPR compared with the paper record. Our study extends these findings by showing an improvement in the completeness of documentation and the appropriateness of documented decisions, based on the judgment of an expert panel. Also, in this study the entire encounter was conducted using an online CPR. The providers reviewed prior patient information on the computer and entered documentation and orders directly into the computer. One might be concerned that the additional effort expended by the physician users to enter data themselves might negatively affect the documentation habits. We found the opposite to be true.

Other studies have demonstrated that computerbased reminders can change physician behavior.^{12,13,14,15,16,17,18,19}

During the study period, our CPR system did not employ a rules-driven reminder system. Consequently, we were able to isolate the effect on documentation of using a CPR from the possible additional positive effects of computer-based decision support in the form of reminders and alerts.

The reader should keep our choice of reference standard in mind when judging the clinical importance of our findings. We chose to construct a reference standard in the form of a mini-record derived from either the CPR or the paper record. Only with full knowledge of a patient's complete history and every evaluative finding (physical sign and laboratory test result) could a true “gold standard” be established. Even then, whether a patient has a firm diagnosis or a notable problem is subject to interpretation. Also, there are often inconsistencies in documented historical and physical findings. Consequently, in actual practice each clinician is faced with the task of gleaning accurate information from the available medical record, and our reference standard mimics this situation. The actual clinical decision maker may have recalled information from prior encounters with the patient that were not reflected in our reference standard. Similarly, we chose appropriateness of documented decisions as a measurement because of the difficulty of determining whether other decisions had been made but not documented in the record.

Finally, the generality of our findings may be limited by a self-selection bias due to voluntary participation as a CPR user in our study. Inherent differences between those who chose to use the CPR and those who did not may account for some of the differences measured. Several factors tend to diminish the impact of the potential self-selection bias on our results: all the physicians in the study were hired into the same division (using similar hiring criteria); each group constituted approximately half of the division (i.e., not a small minority of the division); and the physicians were part of the same group practice environment. Further study is warranted to test whether the same results would apply to all CPR users.

Conclusion

Although there are significant challenges to overcome (e.g., terminology standards, capital requirements, organizational change) before CPR systems can be deployed on a broad scale and put into routine use,²⁰ systems such as the one used in this study are becoming increasingly available. As in many information-intensive industries, a compelling argument can be made that providing effective information management tools to high-level professionals will improve the likelihood that these people will make appropriate decisions. Before health care professionals and the health care industry will act on the Institute of Medicine's recommendation to adopt CPRs for routine use in patient care,²¹ we need to prove that the potential benefits are obtainable. This study shows that the CPR improves the completeness of documentation and that documented decisions are more appropriate when evaluated by an expert panel on the basis of information contained in the record. We argue that an improvement in documentation is an improvement in practice. Additional study is needed to determine whether this improvement in practice results in an improvement in clinical outcomes.