The Use of Evidence-Based, Problem-Oriented Templates as a Clinical Decision Support in an Inpatient Electronic Health Record System

Summary

Background

The integration of clinical decision support (CDS) in documentation practices remains limited due to obstacles in provider workflows and design restrictions in electronic health records (EHRs). The use of electronic problem-oriented templates (POTs) as a CDS has been previously discussed but not widely studied.

Objective

We evaluated the voluntary use of evidence-based POTs as a CDS on documentation practices.

Methods

This was a randomized cohort (before and after) study of Hospitalist Attendings in an Academic Medical Center using EPIC EHRs. Primary Outcome measurement was note quality, assessed by the 9-item Physician Documentation Quality Instrument (PDQI-9). Secondary Outcome measurement was physician efficiency, assessed by the total charting time per note.

Results

Use of POTs increased the quality of note documentation [score 37.5 vs. 39.0, P = 0.0020]. The benefits of POTs scaled with use; the greatest improvement in note quality was found in notes using three or more POTs [score 40.2, P = 0.0262]. There was no significant difference in total charting time [30 minutes vs. 27 minutes, P = 0.42].

Conclusion

Use of evidence-based and problem-oriented templates is associated with improved note quality without significant change in total charting time. It can be used as an effective CDS during note documentation.

1. Background

Clinical documentation is an essential part of communication amongst physicians, nurses, and the healthcare team in the hospital setting [1,2]. Good documentation is often thought of as a surrogate for good Quality of Care (QOC) [3]. Yet providers face many challenges to achieve good documentation, including lack of time, redundant data, as well as inaccurate and incomplete notes [1, 4]. Each of the existing methods for note documentation (free-form text, structured templates, dictation, forms, and dynamic documentation) has disadvantages that limit integration of EHRs with accurate, efficient, data-structured, and high-quality notes [3]. Furthermore, most approaches do not take advantage of clinical decision support measures to guide physician practices.

Our novel approach to this challenge was the introduction of evidence-based problem-oriented templates (POTs). See ► Figure 1 for an example POT. Like problem-oriented medical records, POTs enable a highly organized approach to the complexity of patient care [2, 5, 6] with the most widely adopted method of note writing [6, 7]. Like standard templates and forms, POTs are structured to ensure thoroughness of care and increase documentation efficiency [8]. Like free text, they can be inserted and edited in the note as needed. As an evidence-based resource, they are a non-interruptive form of clinical decision support, and help guide better provider care by ensuring that key information is included for the specific problem for which the template is designed [9].

Fig. 1.

Problem Oriented Template

1.1 Free Text and Structured Templates

Among the main methods of documentation, free-form text and structured templates are the most ubiquitous approaches used in electronic health records. Free-form texts have greater expressivity and thoughtfulness, whereas structured templates tend to be more thorough and can support structured data [10]. Existing research in the outpatient setting suggests structured templates have greater adoption among physicians with single or focused patient problems, but are less used among providers with multiple patient problems [3]. This is likely because most traditional structured templates are full-length notes that tend to populate the entire patient document (history, physical exam, assessment, plan, etc.). A more focused structured template could populate a more narrow area (e.g., just the history or physical exam portion of the note) allow the flexibility to address multiple problems. Finally, most structured templates miss the opportunity to include information that can act as clinical decision-making aids. The potential value of combining a focused structured template with an evidence based clinical decision aid could offer a hybrid approach that is most beneficial in the inpatient setting, where most patients have multiple problems and many physicians seek informative clinical decision aids.

1.2 Clinical Decision Support

The difficulty of integrating clinical decision support (CDS) with documentation practices in EHRs has resulted in few previously published studies or successful designs, and even fewer studies in the inpatient setting [11–16]. There are two methods to integrate decision support in documentation practices. The first uses structured data elements in templated documentation to trigger downstream decision support process. For example, a physician who completes note with a diagnosis of pneumonia but forgets to order antibiotics could trigger an alert to suggest antibiotics for treatment. Another method of integrating decision support in documentation practices is to have clinically relevant aids presented during the note-writing process itself. For example, a focused and clinically organized template for the diagnosis of Pneumonia (► Figure 1) could help remind Physician to ask about TB exposure or consider Incentive Spirometry treatments, options the Physician might otherwise have overlooked. This approach can take into consideration clinical standards of care, by prioritizing the presentation of more clinically useful information that corresponds to recommendations from clinical or evidence based guidelines. For example, the treatment options listed in a Pneumonia template start by listing all the antibiotic options from IDSA (Infectious Disease Society of America) guidelines and separated into categories for CAP (Community Acquired Pneumonia), HCAP (Health Care Associated Pneumonia), or ASP (Aspiration Pneumonia). The two methods are not mutually exclusive, and it is possible to integrate the clinical organization of information with structured data elements to create a potentially multifaceted and versatile decision support aid.

1.3 Problem-Oriented Charting

Problem-oriented medical records aimed to better structure the documenting of information that guides diagnosis and plan for care. They enable a highly organized approach to the complexity of patient care [2, 5, 6]. As problem-oriented notes are the most widely adopted method of note writing, problem-oriented templates would more easily integrate into existing note-writing practices [6, 7].

For our study, we had to create a new group of focused structured templates organized as clinical decision aids, which we called problem-oriented templates (POT, ► Figure 1). The POT includes focused templates with easy-to-select list options for symptoms, conditions, risk factors, possible diagnoses, tests, and treatments from existing evidence-based resources and guidelines. List options are organized to provide the most clinically relevant suggestions, and can be mapped to structured data elements such as SNOMED and ICD10 terminologies. Physicians can delete a list rather than choosing any of the pre-defined options, and can also change, remove, or add text to the POT at any time to allow for fully editable notes (structured data elements are only stored from selected lists once the note is completed). The list options were connected with appropriate prepositions in a common lexicon pattern [1] consistent with existing medical language [17–21] to increase consistency, brevity, and accuracy.

POT can be designed to insert into notes automatically or voluntarily. Automatic insertion can populate into Physician notes based on pre-existing information such as problem lists, medications, or laboratory results. On the other hand, voluntary use requires Physician to recognize an opportunity to use POT during note-writing, recall the correct template to insert into the note, and then complete the use of POT. As POT aids in note completion while providing clinically relevant information, they passively integrate into note-writing practices and normal physicians workflows without active interruption.

Each POT was designed from a evidence based guideline and peer reviewed by 2 relevant Physicians Specialist (i.e. Cardiologist to review the Heart Failure template, Nephrology for the Renal Failure template, etc) to ensure clinical accuracy and quality. Finally, each POT has a built-in hyperlink to a webpage with references to all used evidence-based resources, as well as a hyperlink to a web-based feedback form for ratings, suggestions, and improvements. Minor improvements are updated quarterly, and major reviews are conducted annually to reflect any new guidelines or evidence. Physicians using POT could easily verify evidence-based source for information and quickly provide feedback should any templates require correction.

Unlike other problem-driven documentation tools, the focus of POT is to provide greater evidence-based recommendations in prompts and organization. Many other tools focus on enhancing EHR usability with engine or predicative databases of common clinical elements. POT are novel in their approach to prioritize the most evidence based options. As given by example earlier, the list of antibiotic treatment options for the Pneumonia POT (► Figure 1) are all organized per IDSA guidelines, instead of by antibiotic class or category. This allows the POT to store structured data from selected list choices and act as a clinical decision aid by prompting appropriate clinical options. POT also requires considerable clinical input and time to design and review to ensure information is up to date and uphold clinical standards quality of care. This means that POT can take much longer to create compared to other problem-based documentation tools.

2. Objectives

We aimed to evaluate the voluntary use, adoption, effects on note quality, and overall efficiency of POTs in a hospital setting by primary providers. The primary outcome of the study was note quality which was a surrogate for QOC, given the short time period of the study and the difficulty of evaluating QOC outcomes on multiple different problems. The secondary outcome was total charting time to directly assess physician efficiency on documentation practices.

3. Methods

3.1 Setting

This is a historically controlled (before and after) cohort study that followed documentation practices for a group of physicians before and after implementation of POTs in an academic hospital setting (► Figure 2 for study design). We created 20 POTs (► Table 1) from existing evidence-based resources and national guidelines, reviewed them with academic specialists to ensure quality, and included user options for feedback or corrections. We integrated the POTs as a voluntary, non-interruptive decision support tool in our EHR system.

Fig. 2.

Study Design

Table 1.

List of Twenty Problem-Oriented Templates

Problem	Abbreviations
Acute Coronary Syndrome	ACS
Anemia	ANEMIA
Asthma	ASTHMA
Atrial Fibrillation	AFIB
Cellulitis	CELLULITIS
Chest Pain	CHESTPAIN
Chronic Heart Failure	HF
Chronic Obstructive Pulmonary Disease	COPD
Diabetes Mellitus	DM
Diarrhea	DIARRHEA
Gastro Intestinal Bleed	GIB
Hypertension	HTN
Hypertensive Crisis	HTNCRISIS
Liver Cirrhosis	LF
Neutropenic Fever	NPFEVER
Pneumonia	PNA
Renal Failure	RF
Sickle Cell Crisis	SSC
Syncope	SYNCOPE
UTI or Pyelonephritis	UTI

3.2 Power Calculations

In our study, we used the 9-point Physician Documentation Quality Instrument (PDQI) to evaluate our primary outcome, note quality [22]. The PDQI is a validated tool to consistently and reliably assess note quality using a 5 point likert scale of 9 different criteria (Up-to-date, Accurate, Thorough, Useful, Organized, Comprehensible, Succinct, Synthesized, and Internally Consistent). Based on a goal of recruiting at least 8 physicians and aim to notice a difference of at least 10% change in note quality, we estimated that a total of 200 chart reviews would be required. 100 would be from the preintervention control group and 100 would be from the post-intervention group.

3.3 Sample

Our Inclusion criteria were Hospitalist Physicians who worked at least 5 continuous days in a calendar month. Exclusion criteria included (1) Hospitalist Physicians who previously used problem-oriented templates, and (2) Hospitalist Physicians who were new to the hospital (i.e. had worked for fewer than 3 months). We recruited 10 attending Hospitalist Physicians to participate in our study from 1 November 2013 to 1 May 2014. Each physician completed a questionnaire prior to the start of the study (► Table 2) to evaluate age, gender, and self-assessed proficiency with EHR systems to evaluate for any potential selection bias from over or under skilled EMR users.

Table 2.

Demographic Characteristics and Self-Assessment of Electronic Medical Record Proficiency in Study Participants

Demographics
Age	Average (years)	33 ± 5
	Range (years)	28 – 42
Gender	Male (total)	3
	Female (total)	7
Electronic Medical Record Proficiency Self-Assessment
Responses from 5-point rating scale: ’Very Poor, Poor, Fair, Good, Very Good’	What is your familiarity with using EMRs?	3.8 ± 0.9
How would you consider your…	Time efficiency with note-documenting?	3.2 ± 0.6
	Satisfaction with note-documenting content?	3.6 ± 0.5
	Ease of use with note-documenting?	3.8 ± 0.6
	Overall evaluation of note-documenting practices?	3.8 ± 0.6
How would you consider your…	Time efficiency with EMR?	3.2 ± 0.6
	Satisfaction with EMR content?	3.5 ± 0.5
	Ease of use with EMR?	3.4 ± 0.7
	Overall evaluation of EMR?	3.5 ± 0.5

Hospitalist Physicians in our practice generally work in continuous 5– to 7-day blocks with an average of 15 patients per day and 50 unique patient encounters per week. The randomization process used a random number generator to sample 10 unique notes from the available average of 50 encounters per Physician block. The 10 notes sampled per each 10 Hospitalist from both the before and after period of the study resulted in a total of 200 charts reviews. This study was approved by the University of Florida Institutional Review Board.

3.4 Intervention

The problem-oriented templates were designed prior to the start of the study. They were introduced to the Hospitalist Physicians during a 2 week ‘Training Period’ followed by a 2 week intervention period. After the Hospitalist Physicians completed their work block in the intervention period, we randomly sampled 10 notes per Hospitalist to create our post-intervention group. For our pre-intervention control, we randomly sampled 10 notes from a calendar month prior to the training period for each corresponding Hospitalist. All changes to the problem-oriented templates from feedback or review were done after the study period.

3.5 Data Collection

To assess note quality, we completed the PDQI on 200 randomly selected charts, 100 from the preintervention control group and 100 from the post-intervention group. The chart reviews were conducted by six independent, board certified physicians not associated with the study. Reviewers were all blinded to the identity of patients and the identity of the Hospitalists who wrote the notes.

To assess time and efficiency, we used our EHR auditing tools. The EHR system in our institution automatically stores the logs of every user. By auditing these logs, we collected data on the total daily time each physician spends on any patient chart. The total daily charting time includes all the time the chart was open, regardless of activity, idleness, or work interruptions. Data from time-audits were stored with corresponding data in the chart reviews conducted above.

3.6 Statistical Analysis

All statistical analysis was done in Microsoft Excel and R, version 3.0.3. Unpaired T-test was used to evaluate differences in pre- and post-intervention PDQI results [23]. ANOVA was used to analyze results for more than two categories, such as note quality and the number of POTs in use [24]. Finally, a Kaplan-Meier Estimate, with right censoring of total charting time greater than 60 minutes, was used to compare average charting time between pre- and post-intervention groups [25].

4. Results

4.1 Note Quality

The assessment for quality (using PDQI-9 form) found a statistically significant improvement with the use of POTs [score 37.5 vs. 39.0, P = 0.0020] (► Table 3). When excluding notes from the intervention period that did not use POTs (36% of sampled notes), the improvement was even greater [score 37.5 vs. 39.3, p=0.0075]. The benefits of POTs scaled with use (► Table 4, P = 0.0262); the greatest improvement in note quality from pre-intervention was found in post-intervention notes using three or more POTs [score 37.5 vs 40.2].

Table 3.

PDQI Scores between Pre-Intervention and Post-Intervention Groups

	Pre	Post	P-value	PDQI
Uptodate	4.2	4.3	0.2508	The note contains the most recent test results and recommendations
Accurate	4.3	4.5	0.0003	The note is true. It is free of incorrect information.
Thorough	4.2	4.3	0.4313	The note is complete and documents all of the issues of importance to the patient.
Useful	4.1	4.3	0.0257	The note is extremely relevant, providing valuable information and/or analysis.
Organized	4.0	4.3	0.0022	The note is well-formed and structured in a way that helps the reader understand the patient’s clinical course.
Comprehensible	4.1	4.2	0.1181	The note is clear, without ambiguity or sections that are difficult to understand.
Succinct	4.1	4.2	0.2109	The note is brief, to the point, and without redundancy.
Synthesized	4.2	4.4	0.0280	The note reflects the author’s understanding of the patient’s status and ability to develop a plan of care.
Consistent	4.3	4.5	0.0005	No part of the note ignores or contradicts any other part.
Total	37.5	39.0	0.0020

Table 4.

PDQI Scores by Number of Templates Used in Note

Category	Average PDQI-9
Pre-Intervention	37.5
One Template in Note	38.8
Two Templates in Note	39.6
Three or more Templates in Note	40.2
P-value	0.0262

4.2 Time

Time audits found no significant difference in total daily charting time [30 minutes vs. 27 minutes, P = 0.42]. There was a trend for greater time efficiency with the use of problem-oriented templates, with an average note time of 27 minutes versus an average time of 30 minutes in the control (► Figure 3).

Fig. 3.

Kaplan Meier Estimate of Total Time in Chart

4.3 Retention

Of the 100 sampled post-intervention notes, POTs were found in 64% of notes. Eight out of ten physicians wrote at least one note with a POT during the 2-week intervention period. At 3-month follow-up, four out of ten of the physicians (40%) were still using problem-specific templates.

5. Discussion

5.1 Summary

We found the use of POTs among Hospitalists at a large Academic Medical Center is associated with improved quality of notes and a non-significant trend toward decreased total charting time. Furthermore, we found that the benefits of note quality were cumulative with the number of POTs used.

Our study found that use of POTs led to significantly improved note quality in the following four categories: note accuracy, usefulness, organization, and consistency. There were no significant changes in the remaining five categories: up to date, thorough, comprehensible, succinct, and synthesized. In the inpatient setting, where note quality is an integral part of communication between teams in management of acute conditions,[26, 27] the use of POTs could help mitigate the problems of copy and paste, misleading data, note bloat, and inadvertent ambiguity [28]. Furthermore, our study shows that for physicians who use three or more POTs per note on a consistent basis, the cumulative benefits of improved note quality offer a higher standard of quality achievement. The high pre-intervention quality scores in ► Table 3 also suggests that there was limited room for potential improvement. In a different setting, or with a different group of physicians (i.e. residents), the potential for improvement may be even greater.

The voluntary study design was an important way to assess POT adoption. It required Physicians to recognize an opportunity during note-writing to use POT, recall the correct template to insert into the note, and then complete the use of POT. Busy physicians could therefore either forgot or miss opportunities to make use of POT, which likely contributed to finding in the intervention group sample where 2 of 10 physicians and 36% of notes overall did not include any POT. The follow up after 3 months found that the number of physicians using POT decreased from 8/10 to 4/10. As physicians all have different note writing processes, it is likely that some physicians found greater value from POT relative to others. Additionally, we failed to do any further promotion of POT in the 3 months after the study, which may have caused many providers to forget about their use. We have since changed some of our messaging approach by increasing demonstrations and email reminders, which seems to be generating more interest and feedback from Physicians. Ultimately, the goal rate of POT adoption will likely vary for different locations and clinical scenarios. Our study suggests that for goals of more than 40% POT adoption, voluntary use may be insufficient and automatic insertion should be considered as an alternative.

Finally, the absence of significant increase in total daily charting time suggests that POTs will not impair physician efficiency. If the trend towards reduced total charting time can be shown to be a statistically significant change in future studies, it is possible that POTs may even make physicians more efficient. This is likely due to the way POTs are structured. They have prompted lists as an aid to compensate for limits of memory or attention and ensure consistency and completeness without negatively impacting normal physician workflow. By creating POTs that can be inserted into notes as needed, we have avoided the problem with standard templates which are limited to a single problem with a heavy specialty focus. POTs also have limited pre-generated text, which while still editable, could help to improve the speed and efficiency of note writing [8].

Previous studies have already shown that notes generated from templates are more likely to be associated with greater quality of care [29–32]. This is consistent with initiatives in quality improvement, where use of structured lists and reminders lead to more consistent and better care [33–36]. As POT are structured templates, they can be mapped to standard medical terminologies such as SNOMED and ICD10. This has the potential to allow downstream decision support triggers and data monitoring. Additionally, the use of standard medical terminologies can make inter-institution sharing easier and allow for possible collaboration on peer review and revision responsibilities. Although not evaluated in our study, we also believe POTs may have important institutional value through improvement in utilization management, expected mortality, and coding. POTs may help support the necessity for admission by reminding the physician when appropriate to use the statement “patient failed outpatient treatment,” support the level of acuity by including the lactate result in severe sepsis, or help support patient complexity by including indicators of malnutrition. Finally, POTs have the potential to act as a direct Clinical Decision Support by providing patient specific recommendations at point of care while Physicians are writing notes.

5.2 Limitations

Limitations of the study include the before and after study design, which may have introduced bias toward ‘early adopter’ physicians who were more inclined to use novel electronic support. Historical Controls could also bias results if there were any large systemic influence on charting practices; however, we were not aware of any such influence. Additionally, we conducted the study at a single academic setting with a specific EHR system (EPIC Care); this may limit generalizability.

Our measurement of total daily charting time was inclusive of all time that a Hospitalist spent with a chart open, and thus unable to account for interruptions, idleness, or activities other than note writing. The total daily charting time is a useful surrogate for determining overall provider efficiency, but does not directly address total note-writing time itself. Unfortunately, the time logs in our EMR system did not allow for more specific data collection such as time spent only in the note writing section. This may be a possibility in the future, with updates, customization, or different EMR vendors.

There also is a distinction between note quality and quality of care. In our study, we use note quality as a surrogate for quality of care. Although there is evidence that in the outpatient setting note quality is not always a good surrogate for QOC in chronic illness [3], in the inpatient setting note quality is an important aspect of accurate and timely communication of care [26–28], which makes it a more reliable surrogate for QOC in acute illness. Directly measuring QOC outcomes was beyond the scope of this study due to limitations of time and focus; it should be noted, however, that even direct QOC measures are challenging to evaluate [37] and have large variability in reporting accuracy [38].

Written feedback from providers at the end of the study suggested additional qualitative factors which may have impeded local physician adoption of POT. The first was a lack of a problem-oriented template for the specific problem at hand, as only 20 templates were created and implemented for this study. The second was with concerns over the text formatting of POTs, with some Physicians finding the text overly algorithmic and preferring more narrative texts. Other Physicians had issues with some perceptible font size and style differences in the POT compared to default font values in the note (this has now been fixed). We did not receive any feedback for more information or options in the templates, but we did receive some feedback requesting make POT more concise and with less options. It is unclear how much of these factors may have influenced physician adoption as we did not include them in our study measures, but future studies should include a check on the need for more specific templates and better formatting as part of the outcomes assessment. Nevertheless, the feedback is helpful in how we improve future POTs, as we make quarterly changes and annual reviews to ensure the POT remain up to date.

One final limitation with the use of focused and editable templates is reducing the ability to fully structure and map medical terminologies as compared to single full-length structured templates. Allowing users to edit, delete, or possibly duplicate items from multiple POT makes it difficult to ensure full capture of structured data and limit their interaction with other available Decision Support triggers. Furthermore, the flexibility to use multiple POT could also lead to conflicting or incompatible processing of structured data elements. Ultimately, any downstream use of structured data elements would require testing for each specific scenario to ensure that the structured data elements are working as intended.

6. Conclusion

The use of problem-oriented templates as clinical decision support has great potential to revolutionize note-writing practices. It is associated with increased note quality, and voluntary physician adoption without significant increase in total charting time. It has the possibility to include evidence-based clinical decision support, structured EHR data recording, improved documentation accuracy for use by utilization, and determinations of case mix index and billing. We recommend future studies to further evaluate the use of POTs as a decision support and its effects on quality of care, physician workflows, and ultimately clinical outcomes.