Challenges optimizing the after visit summary

Abstract

Background:

The after visit summary (AVS) is a paper or electronic document given to patients after a medical appointment, which is intended to summarize patients’ health and guide future care, including self-management tasks.

Objective:

To describe experiences of health systems implementing a redesigned outpatient AVS in commercially available electronic health record (EHR) systems to inform future optimization.

Materials and Methods:

We conducted semi-structured interviews with information technology and clinical leaders at 12 hospital and community-based healthcare institutions across the continental United States focusing on the process of AVS redesign and implementation. We also report our experience implementing a redesigned AVS in the Epic EHR at the Mount Sinai Hospital in New York City, NY.

Results:

Health systems experienced many challenges implementing the redesigned AVS. While many IT leaders noted that the redesigned AVS is easier to understand and the document is better organized, they claim the effort is time-consuming, Epic system upgrades render AVS modifications non-functional, and primary care and specialty practices have different needs in regards to content and formatting. Our team was able to modify the document by changing the order of print groups, modifying the font size, bolding section headers, and inserting page breaks. Similar to other health systems, our team found that it is difficult to achieve some desired features due to limitations in the EHR platform.

Conclusion:

Health IT leaders view the AVS as a valuable source of information for patients. However, limitations to AVS modifications in EHR systems present challenges to optimizing the tool. EHR vendors should incorporate learning from healthcare systems innovation efforts and consider building more flexibility into their product development.

1. Introduction

The after visit summary (AVS) is given to patients after medical appointments to summarize their health and guide future care. If properly designed, the AVS can be an educational tool to facilitate patients’ understanding of their health, reduce recall problems, and encourage adherence to self-management tasks [1–9].

The AVS is nearly universal in the United States, resulting from incentives to promote the meaningful use of electronic health records (EHRs). Meaningful Use requirements mandated provision of an AVS and specified required elements [10]. However, patients infrequently reference, use, or even retain their AVS, suggesting currently designed documents do not meeting patients’ needs [11]. Since Meaningful Use dropped the requirement of providing an AVS in 2016, health care systems have been free to redesign their AVS as they choose to optimize its usefulness for patients.

The purpose of this paper is to provide information that may facilitate the work of health systems seeking to improve or modify their outpatient AVS. We do so by reporting the results of qualitative interviews with health information technology (IT) leaders from across the US who worked on AVS customization at their institutions, and by providing a narrative report of our experience implementing a redesigned, patient-centered AVS within the Epic EHR at the Mount Sinai Hospital.

2. Materials and methods

All procedures were approved by the institutional review boards of the Icahn School of Medicine at Mount Sinai and Feinberg School of Medicine at Northwestern University.

2.1. Semi-Structured interviews with health IT experts

To provide implementation insights, we conducted a qualitative study comprised of semi-structured interviews with 12 health IT experts from clinical settings to elicit their experiences with AVS improvement and implementation. Qualitative methods followed standard procedures of data collection and analysis as described by Patton [12]. We conducted convenience sampling by identifying key informants through an announcement on the American Medical Informatics Association Epic Users’ listserv, which asked individuals having experience with AVS modification to contact us to share their insights. Additionally, we contacted individuals known to the investigators to have been involved in such work. Thereafter, we identified other individuals by snowball sampling, and purposively contacted potential participants to achieve geographic and practice setting variation [12]. Once we contacted a potential interviewee, we stated the purpose of the interview and confirmed whether the individual played a central role in redesigning and implementing changes to the AVS. If the person was not, we asked her/him to identify the appropriate contact.

One investigator (AF) conducted all interviews, each lasting 20–30 minutes, using an interview guide (Appendix A). Interview topics included practice workflow, institutional AVS modifications, and facilitators and barriers to AVS implementation. Interview notes were taken and independently reviewed by two investigators for themes and sub-themes, using logical analysis [12]. The two investigators then compared and reconciled their coding. Subsequent interviews were conducted until no new themes emerged.

2.2. AVS optimization

Independent of our interviews with IT leaders, we used a four step process to redesign and implement the AVS for primary care practices at Mount Sinai Hospital: 1) identify patients’ and clinicians’ preferred content, formatting, and order; 2) draft an AVS “mock-up” in Microsoft Word (Appendix B); 3) refine the AVS mock-ups with patient feedback; and 4) modify the AVS in Mount Sinai’s Epic EHR (version 2014) to resemble the redesigned AVS mock-up as closely as possible (Appendix C). We report Step 1 results elsewhere [13]. The remaining steps follow.

2.2.1. Create AVS mock-ups

We applied results of our prior research to create AVS mock-ups by prioritizing specific content, organization/formatting, and understandability features [13,14]. We developed four mock-ups differing by format elements, including order of content, use of page breaks, differing styles of presenting medication data, and variations in font size. The mock-ups accounted for known technical challenges in modifying the Epic AVS and health literacy design principles to ensure clear and effective print communication, as specified in the Patient Education Materials Assessment Tool (PEMAT) and similar resources (Table 1) [15–17]. Additionally, we applied the Social Cognitive-Self-Efficacy theory to promote self-management behaviors by presenting only relevant, easy to follow information [18].

Table 1.

Theoretical and Empirical Underpinnings of AVS Optimal Design.

AVS Domain	Optimal Design Features	Sources and Resources
Content	• Limit and tailor content • Primary care physician contact information • Appointment dates/times • Medication list • Vital signs • Specific instructions	Sources Black et al. [19] Federman et al. [13] Jiggins [20] Salmon et al. [21] Sarzynski et al. [14] Wolf et al. [22]
Formatting/organisation	• 1-page summary • Large font • White space • Most important information first • Title case • Chunk into sections with informative titles • Highlight action items • Give clear instructions	Resources AHRQ Health Literacy Universal Precautions Toolkit [23] CDC Clear Communication Index (CCI) [24] Patient Education and Materials Assessment Tool (PEMAT) [25] Ten Attributes of Health Literate Health Care Organizations: Attribute 8 [26] Toolkit for Clear and Effective Written Materials [27] Health Literacy Online [28]
Language	• Plain, everyday language •Translation into non-English languages

2.2.2. Refine AVS mock-ups

We conducted cognitive interviews with patients to obtain feedback on mock-ups and refine their design. We recruited patients from the outpatient primary care practice of the Mount Sinai Hospital, which serves adults from predominantly low-income communities of upper Manhattan and the South Bronx. Investigators used a think-aloud procedure to identify patients’ perceptions of AVS documents and their understanding of content [29]. At the end of the interview, we asked patients to select a preferred document. We continued this process with iterative refinement of mock-ups using patient feedback (iterative user-centered design) [30]. The AVS was considered redesigned when the majority of patients single mock-up.

2.3. Implementation

To implement our redesigned AVS, we held extensive discussions with members of the Mount Sinai Epic EHR team to ensure maximal use of all possible EHR functionality to support customization. Additionally, the team spoke frequently with the vendor for additional technical support. Over the course of several conference calls, email exchanges, and one in-person meeting between Epic technical support and Mount Sinai AVS development staff, members of the research team (JK, JJ, PF) identified ways to implement the redesigned AVS in Epic (version 2014) as closely as possible.

3. Results

3.1. Semi-structured interviews with health IT experts

We interviewed health care IT leaders from 12 institutions in eight states. Five were academic medical centers, four were non-academic medical centers, two were outpatient clinical networks, and one was a federally qualified health center. These institutions used the Epic (n = 7), NextGen (n = 2), Vista (n = 2), and eClinical Works (n = 1) EHR platforms. Interviewees were seven chief health informatics officers, one electronic health record “champion,” one quality improvement director, one chief medical officer, one clinical investigator, and one primary care-focused division chief. These individuals participated in AVS improvement efforts directly or closely with key personnel. All participants reported the motivation of their respective AVS development committees was to improve the document because it was a sub-optimal patient education tool and represented their institution poorly. Most believed their patients did not use the AVS and felt physicians in their practices thought the AVS had little value for their patients. However, they said their redesigned AVSs were more organized and easier to understand than the standard AVS generated by their EHR.

Respondents’ comments identified three activity domains for AVS revision: preparation, implementation, and dissemination phases. Among these, we identified six principal activities to consider when implementing AVS changes (Table 2). We identified four activities in the preparation phase: 1) engage stakeholders, 2) organize the redesign effort, 3) identify changes, and 4) identify methods to enact the changes. Engagement involved a variety of stakeholders, including patients, clinicians (e.g., physicians, nurses, pharmacists), and health IT experts. One institution specifically sought out medical directors or practice leaders from multiple practices across the institution to ensure the revised AVS would largely meet the needs of its diverse practice settings. Several participants noted the importance of reaching out to colleagues at other institutions who previously optimized their AVS to identify best practices.

Table 2.

Perspectives on AVS Redesign and Implementation from Health IT Leaders.

Domains	Sub-Domains	Examples
Preparation	Engage varied stakeholders	• Patients, patient advisory boards
		• Physicians, including representation from multiple practices • Nursing leadership • Pharmacists • Informatics experts, EHR optimization committees • Administrative and clinical leaders
	Organize the process	• Hold regular meetings • Use an improvement model (e.g., plan-do-study-act)
	Identify targets for modification	• Patient and clinician focus groups
		• Use actual AVS
	Solution discovery	• Gain insights from colleagues at other institutions that have implemented AVS changes • Search print group libraries
Implementation	Anticipate customization challenges	• Time consuming
		• Effort intensive
		• Costly • Different formats for different settings* • Print group** constraints • Changes may not be compatible with system upgrades • Linking across multiple practice sites (important for branding)
	Support	• Work directly with vendor
Dissemination	New users	• Encourage use to facilitate communication with patients • Repeated presentations are necessary to support adoption
	Existing users	• Highlight new features and improvements in existing features • Repeated presentations are necessary to support adoption

^*Epic enables health systems to deploy different AVS in different departments and practices.

^**Print groups are predefined groups of formatted text that auto-populate data.

All sites described multiple challenges when implementing the AVS redesign. Most notably, the effort was highly time-consuming, requiring several meetings over the course of 12 months or more. In many cases, considerable resources were devoted to the project, including funds for programming and outside consultants. Several institutions indicated they would not optimize their AVS in the future because EHR system upgrades rendered some or all of their key modifications non-functional.

Another challenge was differences in content and formatting needs for primary care and specialty practices. Some clinical leaders found a one-size-fits-all approach to AVS design would not meet their varied institutional needs. By comparison, one respondent stated their team wanted an AVS that was recognizable for patients receiving care in multiple locations within their health system.

Respondents reported having difficulty achieving the design features they most desired. Specifically, they reported limitations in their ability to format text and images, to create tables for medications, and to incorporate patient-friendly text without creating additional work for clinicians during clinical encounters. Therefore, many respondents indicated the redesigned AVS at their institution was “not optimal.” For example, the documents did not have a “clean” appearance, contained excessive medical jargon, were lengthy, and not available in foreign languages.

Finally, several respondents highlighted the importance of alerting clinicians in their practices to their revised AVS. Practice leaders provided refresher presentations to ensure clinicians were fully apprised of the document’s features and were using it routinely.

3.2. AVS optimization

The majority of patients identified the same AVS mock-up as their favorite, which we then further refined through iterative user-centered design (Appendix B).

3.3. Implementation – Mount Sinai Outpatient Clinics

We applied several implementation lessons from results of our interviews with health IT leaders, including: acknowledge the limitations of the EHR vendor’s existing architecture for the AVS and work within those limitations; by extension, modify expectations of whether the prototype could be implemented with fidelity; avoid extensive investment in new programming because of the potential loss of functionality with system upgrades; focus initial implementation on one clinical practice to simplify the process; and widely disseminate information about AVS modifications to support awareness and promote its use. Herein we describe key lessons learned during implementation.

3.3.1. Limited success in implementing redesigned AVS

Our team succeeded in modifying the AVS in a few ways: 1) changing the order of “print groups” on the document, 2) modifying font size and bolding of section headers, and 3) inserting page breaks. Print groups are predefined groups of formatted text that auto-populate data. For example, the “medication” print group auto-populates information on current medications and allows for modification of the type of information included (e.g. name, dose, route, frequency, indication, etc.). It also allows limited manipulation of how information is displayed (e.g., table versus line format). Some text in print groups is customizable. For example, instructions to pick up prescribed medications are modifiable. However, the information about each medication (name, dose, etc.) and the pharmacy information are not modifiable. This constraint resulted from how medication records are maintained in many EHRs, wherein third parties standardize medication records for consistency across platforms.

The least complex and trouble-free alteration to the AVS was changing the order of print groups. Overall, we could closely follow patients’ and physicians’ preference for content order. However, like the health IT experts we interviewed, we discovered major challenges to AVS modification in Epic for key areas: system reliance on medical jargon to auto-populate patient-facing documents, limited flexibility to modify information, poorly updated medication and problem lists, and few mechanisms to create an aesthetically appealing document.

3.3.2. Inability to eliminate medical jargon

Both patients and clinicians wanted plain language in the AVS and to provide reasons for services (e.g., reasons for specialty referrals, purpose of medication). However, the process for linking easy-to-understand terms to content would require expanding clinicians’ workflows and was therefore considered untenable for busy practices. The default procedure for auto-populating involves ICD-CM 10 diagnoses and codes for which an order is placed and a billable code is generated. Such diagnostic terminology is highly technical and often very detailed, and may be difficult for patients to understand (e.g. cardiac-induced pulmonary edema due to heart failure with reduced ejection fraction versus heart failure). As with medication records, diagnosis records are maintained via a third party data that uses structured data fields. These data reflect descriptions maintained for the nearly 80,000 ICD-10 codes, and there are no algorithms to convert codes to plain language.

3.3.3. Barriers to adding or modifying information

We found the AVS architecture offers limited free-text options, which is a barrier to clinicians’ ability to add information to various locations in the document. For example, the “instructions” window of Epic AVS provides a space to enter free text, but information elsewhere cannot be modified to provide clarity. Another example is the inability to update incorrect contact information for referrals since provider address information is maintained via a local provider lookup database and must be modified at that source. Moreover, referral and consultation requests cannot be separated from orders, including blood tests and imaging. Each should have its own category to facilitate patients’ understanding of their care plan.

3.3.4. No simple solutions for problem lists and medication lists

We chose to put the problem list toward the end of the AVS based on patient feedback to summarize patients’ health. Unfortunately, including patients’ problem lists increases AVS length, highlighting the tension between comprehensive information and brevity. Similarly, medication lists frequently auto-populate information no longer relevant to the patient, such as short-term antibiotics. This lengthened medication lists and confused patients about the medications they were supposed to take. We were unable to identify a simple way to streamline medication information in the AVS, aside from encouraging clinicians to remove outdated entries.

3.3.5. Limited flexibility to create an aesthetically appealing document

The AVS has uneven margins, inconsistent spacing, and multiple font sizes, which are visually distracting and inconsistent with other educational materials provided to patients at our institution. These flaws diminish the communication potential of the AVS. In many places, we were unable to adjust margins, modify font, change headings, and we often found that adding new features resulted in a cluttered appearance. For example, we presented medication lists in table format and included indication for each medication by linking ICD-10 diagnosis codes to medication orders. This increased the length of the table, awkward headings, and generated word truncations.

The Geriatric Medicine practice identified another formatting issue upon using larger default fonts (14–16 point). The Epic system allows users to choose AVS font size, but larger fonts unexpectedly created multiple page breaks resulting in a considerably longer AVS. Even when we selectively positioned page breaks there was no automatic adjustment to avoid creating subsequent blank pages.

4. Discussion

This paper is the first to describe experiences implementing evidence-based best practices for print materials and AVS design in an EHR. We identified strategies used by other institutions to optimize and implement a revised AVS and used their experiences to guide our development and implementation of a new AVS. The lessons learned provide rich insights into future document improvements. We identified challenges to improving the quality of patient-facing print and electronic EHR-generated documents, which has broad implications.

Overall, we found that implementation trumps design in AVS improvement. We could not reproduce our patient-informed AVS redesign, which followed best practices for effective communication in print materials [31,32]. This disconnect arose from the inflexible architecture of the EHR, resulting in a more cluttered, lengthier, and less aesthetically pleasing AVS than we intended to create. We were unable to achieve consistent font style and sizes, justify margins, or create regular line spacing in our EHR-generated AVS. Moreover, we were unable to eliminate irrelevant text from print groups that distract from essential information. Another important constraint was our inability to replace medical jargon with plain language because the EHR relies on ICD-10 codes. While workarounds exist, they obligate health care providers to several additional steps, an impractical strategy for busy clinical practices [33,34].

An unexpected but critical finding was that several sites lost their investment in AVS redesign when upgrading to a newer EHR version, rendering their optimization non-functional. Ironically, the loss stemmed from Epic making improvements in their standard AVS template, overriding local customization. Health care systems should consider this risk before investing resources in customizing their AVS. Unfortunately, leaving design advances to vendors raises concerns. Vendor consolidation may stifle innovation and slow improvement efforts [35]. It also forces a one-size-fits-all approach that may be illsuited to health systems’ and patients’ specific needs. For example, an AVS designed around the experiences, expectations, and skills of an affluent, highly educated population may not suit the needs of patients from low-income communities. As health systems adopt patient-centered practices, providing clear EHR-generated information that matches the needs of its patients will gain importance and advanced health care systems may have little patience to wait for vendors to offer new innovations.

Health care systems can respond by becoming learning organizations, adapting based on their own and others’ experiences (Table 3) [36]. First, health care systems can use a redesigned AVS generated through rigorous processes as the basis of their own improvement efforts. Second, they should track implementation steps to reproduce improvements if subsequent vendor upgrades erase their customization. Third, health care systems can proactively petition vendors for ways to preserve customization during upgrades.

Table 3.

Recommendations for Implementation of Patient-Centered After Visit Summaries.

Health Care Systems

EHR Developers/Vendors

Work with vendor to determine if desired changes will be compatible with upcoming system upgrades Use evidenced-based redesigned AVS as a basis for improvement efforts Document implementation process to facilitate replication Petition EHR vendors for more flexible architecture to make and preserve customized changes

Formatting Spacing (include ample white space) Strategically placed page breaks Allow for change in font size for visually impaired populations without sacrificing clean appearance and organisation of the document Language Plain language Action-focused terms Convert diagnostic codes to plain language diagnoses Enable labeling of purpose of medications, other therapies, tests, and referrals Program Architecture Changes Architectural flexibility to enable customization by end users that is retained with system upgrades

Vendors, for their part, should consider building greater flexibility in their programming architecture to permit tailoring at the health care systems level. Such flexibility could generate more advances for the vendor’s EHR, as well as build customer loyalty and greater market value. Vendors can also lead enhancements to promote patient use and understanding of the AVS, such as creating plain language translations of ICD-10 codes for patient-facing materials. EHR vendors should align the design of their standard AVS templates with established best practices for communication in health care, principles of patient-centeredness, and maximize workflow integration to ensure clinicians take full advantage of this important patient education tool [14,15,37].

This study is limited by our focus on the printed AVS, although patient portal-accessible versions have similar design issues [28]. The printed AVS has value for many patients, particularly those with limited adoption of electronic communication and Internet use like many older adults and those with income-related barriers [38–43]. Some patients carry a copy of the AVS with them, in case of emergencies or as a source of information when visiting doctors [13].

We conducted qualitative interviews with only 12 health IT leaders. A larger sample might reveal additional insights in AVS development and implementation. Furthermore, the majority of health IT leaders we interviewed used Epic EHR at their institutions. While those who worked with other EHR platforms had similar experiences as the Epic users, their numbers were small; discussions with additional non-Epic users might reveal other insights.

In conclusion, the AVS could be a valuable source of information for many patients since it has the potential to effectively educate, remind, and guide patients through various aspects of their health [19]. However, our team was unable to introduce many patient-centered improvements to the AVS because our EHR allowed for very few modifications. If the rigid architecture remains, the responsibility for improving the AVS will increasingly lie with EHR vendors, possibly stifling innovation and advances in patient-centered care.

Summary points.

What is already known

• Meaningful Use mandated the provision of an After Visit Summary (AVS) at the end of clinical encounters

• Currently versions of electronic health record (EHR)-generated AVS documents may be difficult for patients to understand due to lack of key content, poor organization and formatting, and inclusion of medical jargon.

What this study added to our knowledge

• This study identifies specific changes that can be readily made to the AVS that make them more patient friendly.

• It also identifies previously unreported challenges to optimizing AVS documents in existing EHR platforms.