Relationship Between Documentation Method and Quality of Chronic Disease Visit Notes

Summary

Objective

To assses the relationship between methods of documenting visit notes and note quality for primary care providers (PCPs) and specialists, and to determine the factors that contribute to higher quality notes for two chronic diseases.

Methods

Retrospective chart review of visit notes at two academic medical centers. Two physicians rated the subjective quality of content areas of the note (vital signs, medications, lifestyle, labs, symptoms, assessment & plan), overall quality, and completed the 9 item Physician Documentation Quality Instrument (PDQI-9). We evaluated quality ratings in relation to the primary method of documentation (templates, free-form or dictation) for both PCPs and specialists. A one factor analysis of variance test was used to examine differences in mean quality scores among the methods.

Results

A total of 112 physicians, 71 primary care physicians (PCP) and 41 specialists, wrote 240 notes. For specialists, templated notes had the highest overall quality scores (p≤0.001) while for PCPs, there was no statistically significant difference in overall quality score. For PCPs, free form received higher quality ratings on vital signs (p = 0.01), labs (p = 0.002), and lifestyle (p = 0.002) than other methods; templated notes had a higher rating on medications (p≤0.001). For specialists, templated notes received higher ratings on vital signs, labs, lifestyle and medications (p = 0.001).

Discussion

There was no significant difference in subjective quality of visit notes written using free-form documentation, dictation or templates for PCPs. The subjective quality rating of templated notes was higher than that of dictated notes for specialists.

Conclusion

As there is wide variation in physician documentation methods, and no significant difference in note quality between methods, recommending one approach for all physicians may not deliver optimal results.

Introduction

The rate of adoption of electronic health records (EHRs) has been rapidly increasing under the new meaningful use and accountable care requirements [1]. Many have argued that EHR adoption and use can lead to significant improvements in the cost and quality of care [2, 3]; and, 82% and 85% of physicians currently using EHRs report that the system has had a positive effect on delivery of (1) preventive care and (2) chronic illness care, respectively [4]. However there have been concerns raised about the quality of visit note documentation in EHRs as well as the time expended to perform electronic clinical documentation [5–8].

Since physicians spend a significant part of their day on clinical documentation [9], it is crucial to ensure that the transition from paper documentation to electronic documentation optimizes the quality and efficiency benefits of implementing EHRs. At present, there is no single best method of visit note documentation, and physicians are likely to continue to use a range of documentation methods in the future [4]. Although 85% of physicians use one particular method for more than three-quarters of all documented notes, their method of choice varies based on clinical and non-clinical factors [10]. 49% of providers use structured templates for at least three-quarters of all visit notes authored, while about 22% still use dictation for the majority of their notes and 13% use free form [10].

Thus, while physicians seem to have a preference for the particular documentation method they use most often, it is important that their chosen methods create clinical notes that provide accurate, complete, and useful documentation of the visit. Effective clinical documentation may be important to support quality of care not only at the time of the visit, but throughout the duration of a patient’s healthcare experience, and especially for those with a chronic disease. Visit notes can be especially important for coordinating care across the healthcare spectrum for patients with a chronic disease.

Although structured documentation has been linked to more thorough and complete notes, disadvantages include a loss of expressivity and thoughtfulness [11]. In addition, templates and the ability to import text have been shown to lead to additional errors [12]. One study investigated documentation method and quality of care for diabetes and CAD, showing that physicians that use structured EHR documentation in comparison to dictation seem to have a higher quality of care [13]. While we know that structured data can support downstream uses such as automated alerts, decision support and quality assessments, additional factors may be involved in choice of documentation method such as documentation time efficiency, expressivity, and note quality [11, 14]. Additionally, physicians reported that some automated features of electronic notes, such as template generation, created issues such as increase in length, redundancy, or poor formatting [8].

2. Objectives

Because care for patients with chronic disease is responsible for more than 75% of our nation’s health expenditures, it is particularly important to evaluate the quality of notes for chronic care of patients written using different documentation methods [3]. Our goal was to determine the relationship between methods of documenting visit notes and the quality of notes of primary care providers (PCPs) and specialists, as well as the variables that contribute to higher quality notes for patients with two chronic diseases, diabetes and coronary artery disease (CAD).

3. Methods

3.1 Setting

Massachusetts General Hospital and Brigham and Women’s Hospital are affiliated with Partners HealthCare, a large integrated healthcare delivery system in Massachusetts. The primary ambulatory electronic health record in use at these medical centers is the Longitudinal Medical Record (LMR), an internally developed, Certification Commission for Healthcare Information Technology (CCHIT) certified system that includes problem lists, medication lists, primary care and specialty notes, allergies, immunizations, and family history. The LMR supports electronic note documentation, medication ordering, and results review.

3.2 Sample

We identified Brigham and Women’s Hospital (BWH) and Massachusetts General Hospital (MGH) PCPs, endocrinologists and cardiologists who used the outpatient electronic health record in 2010. These physicians were categorized based on the documentation method (free form, dictate, template) they used most often to document their visit notes based on the total number of notes authored in 2010. The method of entry was determined by metadata within the note. In contrast to free-form text entry or dictation, templates can contain a varying number of pre-defined structured fields, such as medications, allergies, physical exam and past medical history, and may include custom text previously created [10]. Templates can be created by practice staff or individual providers and maintain some flexibility so that fields can be modified by the provider. In order to compare between documentation methods, we identified notes that would have a similar purpose and complexity. We retrieved visit note encounter data from the Research Patient Data Registry for all BWH and MGH patients with an outpatient visit characterized as moderately complex (CPT code 99214) and with an ICD-9-CM diagnosis code of diabetes (250), CAD (414), or both. We excluded notes that were identified as patient letters or preceptor notes in order to standardize the type of note as much as possible. We then matched the encounter data to our sample of physicians and filtered out encounters with providers not in our sample population. The final sample of notes from the encounters identified was selected for chart review based on whether the method of documentation used for that note matched the predominant method of the provider. We selected at least 30 notes of each documentation method for both PCPs and specialists. In cases where the number of unique providers in the category was low and we expected the within-physician correlation to have a larger impact, we selected up to 50 notes in that category in order to improve our power. We excluded multiple visit notes per provider/patient pair. We calculated that with 80% power at a significance level (alpha) of 0.05, we could detect a. 66 point difference in note quality between the documentation methods for PCPs and specialists (assuming a standard deviation of 0.89).

For PCPs and specialists, we selected a subset of 10 notes from each of the physician documentation method categories (free form, dictate, template) to pilot the chart review.

3.3 Instrument Development

We reviewed the literature related to quality documentation tools, methodology for assessing quality of documentation and quality indicators for diabetes and CAD. From this review, we generated a list of note content items and potential variables related to note quality to create an electronic abstraction form for the chart review. The abstraction form was developed through research team consensus meetings with the assistance of clinician experts (DB, GS, HR) with significant combined experience in quality and patient safety research. The form included six sections that our experts identified as major content areas of a note (vital signs, medications, lifestyle, labs, symptoms, assessment & plan) with specific questions related to note content and subjective quality. This paper reports on only the subjective ratings for quality that were included as part of the form. Each section required the reviewer to indicate an overall quality rating for that section by responding to the question “Please rate the overall quality of documentation of (vitals) in this note only” on a scale from one (very poor) to five (excellent). The reviewer was also asked to respond to “Please rate the overall quality of this note” on a scale from one (very poor) to five (excellent). Two additional subjective statements were included that asked the reviewers to rate their agreement with: “The note reflects the writer’s understanding of the patient’s overall status and synthesizes the main problems and goals into a clear assessment and plan” and “The note is formatted and laid out in a way that makes it easy to read and find relevant and critical information.” The final section of the form was the 9 item Physician Documentation Quality Instrument (PDQI-9) [15] that we included as an additional subjective measure of note quality. The 9 PDQI attributes were Up-to-date, Accurate, Thorough, Useful, Organized, Comprehensible, Succinct, Synthesized and Consistent.

We also collected any patient data (medications, allergies, problem list etc.) in the electronic health record that would have been available to the physician in the EHR on the date of the review note. We presented these data to the chart reviewers within the abstraction tool.

Two internal medicine residents conducted the chart review. These residents both had previous experience with the electronic health record but were not currently practicing at a Partner’s affiliated practice. We developed a set of written instructions for the reviewers to use during the chart review which included clarification and definitions for some of the elements, in addition to general information on using the form. The reviewers had not created any of the notes themselves. They were not blinded to the provider writing the note but were told to inform us if they felt they had any conflicts of interest so that we could remove that note and replace it with another.

We piloted the form with the two reviewers. The reviewers used the form to review a subset of 10 records. The reviewers were told to ‘overuse’ the comment fields in the form to identify any issues with the form, the patient data, or the questions. We debriefed the reviewers following the pilot to discuss issues with the form and suggestions for improvement to further revise the chart review instrument, written instructions and process before they began reviewing the final sample. We also checked in with the reviewers periodically during their review of the final sample and reviewed any issues or questions they had, making sure to provide consistent instruction to both reviewers.

3.4 Data Analysis

Each reviewer assessed all notes in the final sample so that two sets of scores were obtained. The nine PDQI attributes that were rated on a scale from one (not at all) to five (extremely) were totaled for an overall PDQI-9 score ranging from 9–45. Subjective ratings given for overall quality and PDQI-9 scores (overall and for each attribute) were averaged for both reviewers to determine a single rating for each note. The correlation between the two reviewer’s overall quality score was good (Pearson’s r 0.5). Physician assessment of overall quality score varied by 1 point or less in over 88% of the responses. The quality ratings by section, overall quality score and PDQI-9 scores were then averaged across physicians within each of the documentation methods. We also pulled data on note word count, and physician characteristics such as specialty, age, gender, and busyness, which was defined by estimated patients per hour over a one year period.

3.5 Statistical Analysis

A one factor analysis of variance (ANOVA) test was performed for PCPs and specialists separately to determine whether differences in mean quality scores and PDQI-9 scores among the groups (free form, template, dictate) were significant.

We ran a linear regression analysis on both the overall quality average and the total PDQI-9 score to determine predictors of perceived note quality, which included provider characteristics such as specialty (PCP or specialist), age, gender, and busyness, patient chronic condition (CAD, Diabetes or both), documentation method and note word count. We incorporated Generalized Estimating Equations (GEE) in the regression analysis to account for clustering by physician.

4. Results

The final sample meeting our criteria included a total of 240 notes written by 112 physicians (71 PCPs and 41 specialists) in 2010. Of the 71 PCPs, 30 predominantly used templates when documenting notes, 29 documented free-form notes, and 12 dictated their notes. Of the 41 specialists, 6 predominantly used templates when documenting, 26 type free-form notes, and 9 dictated their notes. The physicians who dictated were significantly older than those that used templates (mean age 55 and 49 respectively; p = 0.006). There were also significantly more males in the dictation group (84%) than in the template (51%) or free form groups (53%), (p<0.0001).

Average quality score and total average PDQI-9 score results are in the tables below for primary care and specialist physicians (►Table 1). For PCPs, overall quality score for free form notes was 0.25 and 0.26 points higher than templated notes and dictated notes, respectively. For specialists, templated notes had significantly higher overall quality (p<0.001) and PDQI-9 scores (p = 0.03), with a difference in overall quality score of. 48 and a difference in PDQI-9 total of 1.54 when compared to dictated notes.

Table 1.

Overall Note Quality Score and PDQI Total Score

Primary Care Provider Documentation Method (N = 110 notes)
	Template (N = 30 notes)	Free Form (N = 30 notes)	Dictated (N = 50 notes)	p-value
Overall Quality	3.72	3.97	3.71	0.20
PDQI Total	38.27	38.83	37.94	0.52
Specialists Documentation Method (N = 130 notes)
	Template (N = 50 notes)	Free Form (N = 30 notes)	Dictated (N = 50 notes)	p-value
Overall Quality	4.12	4.00	3.64	<0.001
PDQI Total	39.08	39.03	37.54	0.03

Quality ratings for each of the content areas and ratings on the individual PDQI-9 attributes are shown for both primary care providers (►Table 2) and specialists (►Table 3). For primary care physicians, the quality ratings of vitals, labs and lifestyle sections favored a higher documentation quality score for free form notes (0.35 points compared to templated notes, p = 0.01 for vitals; 0.35 points, p = 0.002 for labs; 0.06 points, p = 0.002 for lifestyle). Medications were the exception with significantly better quality ratings for templated notes versus the other note types (0.63 points compared to free form notes, p<0.0001). For specialists, the quality ratings of vitals, labs, lifestyle and medications sections were all significantly higher for templated notes compared to free form (0.30 points, p<0.001; 0.72 points, p<0.0001; 0.78 points, p<0.001; 0.75 points, p<0.001, respectively). Symptoms was the exception with significantly better quality ratings for free form notes compared to the other note types (0.23 points for template and 0.37 points for dictated notes, p = 0.20).

Table 2.

Primary Care Physicians Average Quality Ratings by Documentation Method

	Documentation Method
Collection Form Sections	Template	Free Form	Dictated	P-value
Vitals	4.30	4.65	3.92	0.01
Labs	3.45	3.80	3.09	0.002
Lifestyle	3.62	3.68	3.01	0.002
Medications	4.50	3.87	3.15	<0.001
Symptoms	3.60	3.80	3.60	0.49
Assessment & Plan	3.73	4.05	3.99	0.16
PDQI Attributes	Template	Free Form	Dictated	P-value
Up-to-date	3.70	3.95	3.30	0.002
Accurate	4.67	4.75	4.62	0.17
Thorough	3.45	3.78	3.29	0.04
Useful	4.23	4.32	4.22	0.77
Organized	4.33	4.20	4.15	0.52
Comprehensible	4.45	4.28	4.51	0.20
Succinct	4.35	4.28	4.52	0.14
Synthesized	4.22	4.43	4.42	0.31
Consistent	4.87	4.78	4.92	0.10

Table 3.

Specialists Average Quality Ratings by Documentation Method

	Documentation Method
Collection Form Sections	Template	Free Form	Dictated	P-value
Vitals	4.67	4.37	3.88	<0.001
Labs	3.99	3.27	3.01	<0.001
Lifestyle	3.65	2.87	2.72	<0.001
Medications	4.52	3.77	3.13	<0.001
Symptoms	3.64	3.87	3.50	0.02
Assessment & Plan	4.12	4.17	3.94	0.21
PDQI Attributes	Template	Free Form	Dictated	P-value
Up to date	4.14	3.68	3.19	<0.001
Accurate	4.78	4.77	4.60	0.01
Thorough	3.90	3.63	3.04	<0.001
Useful	4.37	4.47	4.15	0.03
Organized	4.38	4.25	4.16	0.14
Comprehensible	4.36	4.45	4.47	0.51
Succinct	4.09	4.22	4.56	<0.001
Synthesized	4.36	4.58	4.43	0.15
Consistent	4.75	4.93	4.86	0.05

The PDQI-9 ratings also differed between PCPs and specialists. For PCPs, Up-to-date and Thorough attributes of notes were found to be significantly higher for free form than for templated or dictated notes. For specialists, templated notes had significantly higher Up-to-date, Accurate and Thorough scores (p<0.001, p = 0.01, p<0.001 respectively), free form notes had significantly higher Useful (p = 0.03) and Consistent scores (p = 0.05). Dictated notes had the highest Succinct score (p<0.001).

Two additional subjective statements were rated by the reviewers (►Table 4). For specialists, templated notes ranked significantly higher (p = 0.02) than the other methods on “the note is formatted and laid out in a way that makes it easy to read and find relevant and critical information”, although the actual difference was small.

Table 4.

Subjective Statements and Ratings

Statement 1: The note reflects the writer’s understanding of the patient’s overall status and synthesizes the main problems and goals into a clear assessment and plan.
	Documentation Method
	Template	Free Form	Dictated	P-value
PCPs	3.90	4.18	4.03	0.20
Specialists	4.24	4.30	4.12	0.27
Statement 2: The note is formatted and laid out in a way that makes it easy to read and find relevant and critical information.
	Documentation Method
	Template	Free Form	Dictated	P-value
PCPs	4.20	4.08	3.90	0.18
Specialists	4.22	4.03	3.88	0.02

After adjusting for clustering by physician, the results of the linear regression indicate that clinicians were consistent in the quality of their documentation; quality scores and PDQI-9 ratings were highly correlated within a physician, 0.65 and 0.64, respectively. Overall quality of documentation and overall PDQI-9 score were significantly higher when the clinician was documenting for a patient with both CAD and diabetes than with just diabetes alone (p = 0.02 for PDQI-9 and overall quality). Also, notes for patients with both CAD and diabetes were associated with higher PDQI-9 ratings (p = 0.005) than if the patient had CAD only. There was no significant association between quality ratings and provider specialty, age, gender, busyness, note word count or documentation method.

5. Discussion

We evaluated key elements to assess the quality of the documentation, comparing three types of electronic note entry from both specialty and primary care physicians’ notes for patients with two chronic diseases. Overall quality and PDQI-9 values for the sample for both PCPs and specialists were above 3.7 on a 5 point scale and 37.5 on a 40 point scale, respectively. There were no statistically significant differences in overall quality and PDQI-9 scores of notes written by PCPs using the three methods. Notes documented using templates were rated as having significantly higher overall quality and total PDQI-9 score among specialists than other documentation methods. The regression found no significant associations between quality ratings and physician specialty, age, gender or busyness, note word count or documentation method.

Differences in the quality and content of templates available for specialist visits compared to primary care visits may partly explain the significant results for the specialists. All but one section (assessment and plan) had quality ratings that were significantly better for templated specialists’ notes. Templates received highest ratings on the subjective statement “The note is formatted and laid out in a way that makes it easy to read and find relevant and critical information.” Templates may be a better option for specialists who tend to focus on a more narrow set of problems to be documented. Thus, designing a template for a specialist visit with a predefined set of questions and responses may be easier than designing one or multiple templates for a PCP, who often must address myriad issues and types of patients. Future research should include a variety of note complexity and additional diseases.

Free form notes may offer more flexibility and readablity when constructing clinical narratives for other clinicians reading PCP notes. Templates were rated highest on many measures for specialists; however, free form notes were rated higher than templates on the Useful attribute. Some attributes of the notes received higher scores than others, supporting the idea that certain methods may have strengths in some areas whereas others do not. Thus the best choice of note documentation type may vary depending on the note’s purpose.

For both PCPs and specialists, notes documented using dictation were given the lowest scores for overall quality and total PDQI-9 score. Dictated notes scored slightly higher on some note attributes such as succintness, but lower on most other quality measures that were related to note completeness. Dictated notes may be easier for a reader to understand the overall picture; however, it may be more difficult to locate details regarding lab results or medications which may be better entered and viewed within a more structured format [14, 16, 17], though the extent to which these data should routinely be included in electronic visit notes is debatable since they are readily accessible elsewhere in the EHR.

One challenge in determining the quality of a note is that a patient’s data may be available in other parts of the medical record (e.g. the patient’s problem and medication lists etc.). A key question is to understand when data should be included in the note (to both document clinician awareness as well as include in for other readers of the note) vs. when does this become redundant “clutter” that distracts from key narratives in clinical notes? For example, does the list of medications need to be documented in the actual note, or is an updated structured medication list a better “source of truth” for documenting the medications a patient is taking? And, how will these decisions affect the medication reconciliation process? Future analysis to assess how the documentation of such patient data stored elsewhere in the EHR affects the note quality would be useful to better understand and design efficient workflow and more useful notes.

EHR use is growing rapidly, and at the same time, the EHRs themselves are being improved, with more effort being put into system design and development. The availability of clinical decision support, structured data, and ability to pull in data from other parts of the EHR are additional features of electronic documentation that may impact note quality, depending on their design, implementation and utilization. Recent research in this area has focused on the relationship between structured documentation and patient quality outcomes [13], but our study sought to take a step in understanding the quality of the note itself. Note quality may have implications for patient safety, provider satisfaction and efficiency, and provider communication. Continued research in the ways in which physicians interact with these systems and their expectations for the system is necessary in order to better design these tools as technology, and practice and physician workflow evolve.

While there are some differences in the quality of notes based on the type of documentation method, these results suggest that no single current approach is clearly superior across the board. Getting providers to adopt EHRs is a major challenge, so that building systems to support a variety of methods and optimize the potential of electronic documentation thus may be important for ensuring physician buy-in as well as their ability to produce the highest quality notes for the specific situation using their preferred method. Many organizations are struggling with finding the balance between free-text and structured documentation or choosing a vendor product that meets their needs. Alternatively, there may be better solutions that incorporate many of the positive aspects of several of these methods, such as combining voice recognition entered free text into structured templated notes. Schiff and Bates describe the ways in which the EHR and electronic documentation could reduce diagnostic errors by such re-engineered documentation [18]. It may be important to take a step back and understand the ways in which documentation can support the clinician’s thinking and revise the system as a whole rather than prescribe a method currently in place. In addition, many decisions about what to include about notes have been driven by billing, which is probably not in alignment with better care delivery—the billing rules regarding notes badly need reform.

5.1 Limitations

We evaluated providers from only one healthcare system with a locally developed EHR and in only two chronic diseases, so the results may not be generalizable to other organizations or to patients with other conditions. The instrument we used included both objective items and subjective rating scales. Note quality is inherently subjective and difficult to measure. We used a well validated scale and analyzed subjective items by averaging the ratings of the two reviewers to obtain an overall score for those items.

6. Conclusions

While some differences in note quality were present among the methods of documentation, there was no clear method that was superior in all aspects. Each method has relative advantages and disadvantages and proposing one method for all physician specialties in all settings may not be the best option. Designing a documentation system to accommodate the majority of physicians may require a complete shift in the way in which documentation systems are designed and used today to ensure the best combination of note quality, physician satisfaction, system usability, and patient care outcomes. Future policies – both locally and at the national level – which attempt to address these issues should consider these factors. More research on documentation approaches is needed, as documentation takes substantial time, but the best approaches are not yet clear.