Adapting Comparative Effectiveness Research Summaries for Delivery to Patients and Providers through a Patient Portal

Abstract

Despite increases in the scientific evidence for a variety of medical treatments, a gap remains in the adoption of best medical practices. This manuscript describes a process for adapting published summary guides from comparative effectiveness research to render them concise, targeted to audience, and easily actionable; and a strategy for disseminating such evidence to patients and their physicians through a web-based portal and linked electronic health record. This project adapted summary guides about oral medications for adults with type 2 diabetes to a fifth-grade literacy level and modified the resulting materials based on evaluations with the Suitability Assessment of Materials instrument. Focus groups and individual interviews with patients, diabetes providers, and health literacy experts were employed to evaluate and enhance the adapted summary guide. We present the lessons learned as general guidelines for the creation of concise, targeted, and actionable evidence and its delivery to both patients and providers through increasingly prevalent health information technologies.

Introduction

The availability and application of evidence to support the effectiveness of treatment options are critical to the reform of healthcare in the United States. The American Recovery and Reinvestment Act of 2009 allocated $1.1 billion for the conduct of comparative effectiveness research. Comparative effectiveness research is defined by the Institute of Medicine as “the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care” with the purpose “to assist consumers, clinicians, purchasers, and policymakers to make informed decisions that will improve health care at both the individual and population levels.”¹ Although comparative effectiveness research has increased over the past decades, the translation of the evidence to clinical practice does not seem to be consistent.

The Agency for Healthcare Research and Quality (AHRQ) received substantial funding for comparative effectiveness research and developed the Innovative Adaptation and Dissemination of AHRQ Comparative Effectiveness Research Products (iADAPT) program to address the gap between the creation of evidence and its application in practice. The iAdapt program sought to increase the accessibility and application of summary guides of comparative effectiveness research for high-priority, high-impact diseases, such as type 2 diabetes.

This manuscript presents a process for adapting summary guides of comparative effectiveness research to render them concise, targeted to audience, and easily actionable; and also a strategy for disseminating such evidence to patients through a patient portal and to their physicians through a linked electronic health record (EHR). Patient portals are “healthcare-related online applications that allow patients to interact and communicate with their healthcare providers, such as physicians and hospitals”² and many are directly linked to electronic health records, providing an innovative platform to allow consumers and clinicians to share information and decision making. This manuscript presents the experience and lessons from developing and applying adaptation and delivery processes to the AHRQ Summary Guide, “Comparative Effectiveness and Safety of Oral Diabetes Medications for Adults With Type 2 Diabetes.”

Methods

Overview

The overall goals of this project were to adapt the contents of an evidence-based summary guide, “Comparative Effectiveness and Safety of Oral Diabetes Medications for Adults With Type 2 Diabetes,” to be concise, targeted, and easily actionable, while preserving the fidelity of the original guide's technical contents, and to disseminate this evidence to patients and providers through information technologies routinely used for healthcare delivery. Concise was defined as succinct and 5^th grade literacy level appropriate; targeted was defined as explicitly tied to relevant patient cases; and actionable was defined as being linked to a series of concrete steps patients and physicians could take to follow the summary guide's contents when making healthcare decisions.

An initial adaptation and web page for delivery through a patient portal were designed using a 5^th grade literacy level and sequential evaluations and modifications using the Suitability Assessment of Materials (SAM). The project applied qualitative methods including focus groups, individual interviews, and surveys to assess and revise the adaption and web page.

This research was conducted at the Vanderbilt University Medical Center (VUMC), a comprehensive health care facility in central Tennessee. The project leveraged the widely adopted VUMC patient portal, My Health At Vanderbilt (MHAV). The MHAV patient portal was launched in 2005 and provides standard patient portal features such as access to selected portions of the electronic health record, secure electronic messaging between patients and healthcare providers, and the delivery of customized health information. MHAV currently has over 215,208 registered users. On average, over 11,469 users access the site each day; and MHAV has been successfully used to deliver targeted educational materials to patients based on their health problems.³ MHAV has employed a systematic approach to creating procedures and policies in an effort to develop best practices for patient portal access and functionality.⁴ Among the features offered through MHAV, the most widely used is secure messaging between patients and healthcare providers (855,840 threads between providers and patients in 2012), and self-reported use of secure messaging has been associated with improved glycemic control in diabetic patients at our institution.⁵ Portal features such as messaging and links to the VUMC EHR were employed to make the information targeted and actionable (i.e., linked to tools that allowed communication with a provider) to relevant patients (i.e., individuals with type 2 diabetes who were taking oral medications).

Initial Adaption

The summary guide, known as the “Quick Type 2 Diabetes Summary Guide,” was adapted by reviewing electronically-available AHRQ materials for type 2 diabetes patients on their web site. Three pharmacy interns assisted the research team in condensing this information into a one-page guide to help patients understand and communicate with their healthcare providers about oral medications for diabetes.

Materials were designed for people with a 5^th grade health literacy. Literacy plays an important role in disease management interventions for patients with type 2 diabetes mellitus. Many researchers have demonstrated the effects of health literacy and numeracy on various clinical outcomes. Literacy skills have explained racial differences in medication adherence,⁶ and self-efficacy has been shown to mediate the impact of limited literacy skills on medication adherence.⁷ Low diabetes numeracy skills, not low general numeracy or limited literacy skills, have explained the association between African American race and poor glycemic control.⁸ With regard to health literacy, the summary guide was adapted to be succinct and to use a short, directive sentence structure.

The goal of 5^th grade literacy was verified by utilization of the Suitability Assessment of Materials (SAM).⁹ This instrument assesses the appropriateness of health education materials on a variety of factors described in Table 1, and it been used for both written materials and web-based materials.^{10, 11, 12} When the initial adaptations and design of the portal website were near complete, the materials were assessed using SAM and scored by a research team member (AM). Sequential modifications to the adaptation were made to ensure at least a superior rating (over 70%). Later, two other team members were asked to evaluate the materials using the SAM, and the Fleiss Kappa statistic was measured to determine consistency across the three raters.

Table 1.

Evaluation metrics from the Suitability Assessment of Materials, by Doak and Doak.⁹

Factor to be Rated	Factor Aspects to be Evaluated
Content	purpose of materials is evident, includes content about behaviors, scope is limited to key factors, and includes a summary or review at the end of the materials
Literacy Demand	reading grade level, writing style, vocabulary, context, and use of advance organizers (ex. headers or “road signs” to help guide the reader)
Graphics	cover graphic shows purpose, type of graphics, relevance, proper explanations and captions
Layout/Typography	layout factors such as white space, short sentences, contrast and use of shadowing, typography focus on large easy to read fonts, and use of subheadings to partition excessive information
Learning, Stimulation, Motivation	includes interactions such as worksheets or questions/answers, desired behavior patterns are modeled and shown in specific terms, and topics are broken down into smaller parts to provide for small successes that can promote motivation for behavior change
Cultural Appropriateness	materials are matched in logic, language and experience; uses culturally appropriate images and examples).

Focus Groups, Interviews, and Surveys

Focus groups and one-on-one interviews were conducted with patients, healthcare providers, and domain experts in diabetes and health literacy to design the adaptations of the summary guide to fit the needs of these target groups and to identify potential barriers to successful adoption of the evidence. Patients were recruited from the VUMC affiliated adult medicine clinics, which collectively care for over 50,000 adult patients, including over 3,700 with diabetes. The clinics serve a diverse patient population, including all racial and ethnic groups, and represent a broad range of literacy, educational levels, and socioeconomic statuses.^{8, 13, 14} The research team recruited patients with type 2 diabetes mellitus using several methods, including email messages to the VUMC research listserv, fliers posted in the VUMC Internal Medicine and Diabetes clinics, and a blind in-clinic approach in VUMC Internal Medicine. Eligible participants were English-speaking patients with a diagnosis of type 2 diabetes mellitus for at least 12 months, age over 18 years, current prescription for oral diabetes medications, usage of MHAV within the last 24 months, and a primary care provider at a VUMC clinic.

Healthcare providers were recruited through the VUMC research notifications listserv and email messages sent to internal medicine clinic providers. Eligible healthcare providers were licensed primary care practitioners in an internal medicine and /or pediatric practice who used MHAV and provided clinical care for adult patients with type 2 diabetes. A purposive selection process was used to recruit domain experts, requiring at least 5 years of experience either in health literacy, diabetes, or both. This study was approved by the VUMC Institutional Review Board, and all study subjects provided informed consent. Patients and healthcare providers were compensated for participation.

A total of 7 focus groups, comprised healthcare providers, patients, or subject-matter experts, were conducted by a pair of study researchers: one researcher (YXH) served as moderator to facilitate discussion and ensure full and equal participation, and the other researcher served as note-taker. The moderator conducted 4 one-on-one interviews with healthcare providers who were unavailable to attend a focus group. Participants completed a survey that included information about demographics and familiarity with technology. Participants were provided with links to the summary guide on the AHRQ web site and were also given printed copies of the summary guides.

Patient focus groups engaged in a discussion that followed a semi-structured guide including items about current diabetes treatments and use of MHAV, and they answered questions about the summary guides such as the following: How satisfied were you with the interface design of the Summary Guide? Would you recommend the Summary Guide to other patients with diabetes? Physicians and nurses were asked similar items tailored to their role as healthcare providers for patients with diabetes. Additional adaptations were made to the summary guide and portal web sites based on feedback from these focus groups.

Results

Initial Adaptation

The initial adaptation of the AHRQ materials produced a one-page guide designed to promote behavior change and reinforce evidence about oral medications for diabetes in an easy to read layout. It was made to be literacy-appropriate and culturally relevant for the study populations involved. Literacy considerations include brevity, white spaces, numbered steps divided into discrete actions, sentence construction, and word choice. The top of the guide lays out “4 Things to Know About Your Diabetes Pills,” and it provides easily actionable, brief steps to follow for better results from a patient’s diabetes management with medication. Important side effects are strategically highlighted underneath, where they are easy to see, but do not detract from the main steps to take in applying the evidence. More warning signs are underneath the steps, and a list of symptoms that should be acted upon by the patient calling a doctor. Below the warning is a brief chart of medications and associated side effects. The chart helps the patient find personally relevant information about his/her drugs. Layout establishes importance by order and placement, and the color in the document makes it friendlier and more attractive to the patients.

Focus Group Input

A total of 20 patients (12 females, 8 males; 15 white, 5 black, 1 other), 8 healthcare professionals (including 2 diabetes educators and 2 endocrinologists), and 6 domain experts participated in this study’s focus groups and interviews. Focus groups and interviews lasted approximately 60 to 75 minutes. Focus groups and interviews lasted approximately 60 to 75 minutes.

Patient focus group input resulted in a variety of changes and additions to the adaptations that are described below. First, personalized information was added at the top of the portal resource. Specifically, a list of oral diabetes medications from the patient’s medication list in the EMR system was provided. Another feature recommended by the patient and expert focus groups was a pronunciation key. Adjacent to each medication name, the web page includes pronunciation guides with an audio demonstration of the articulation (Figure 1).

Figure 1:

The iAdapt resource in the MHAV Patient Portal. This screen shows features described in the text including: personalized drug chart, pronunciation guide and audio file, Fast Facts, volunteer solicitation, and the top of the AHRQ management guides, included our adapted summary guide.

Patient focus group comments also guided the addition of educational links to the portal site that presented the summary guide evidence. Relevant comments included, “I have a huge problem with the fact that they’ve chosen to leave out exercise and diet in their perspective...I wish they could give us a limited amount of information on diet and exercise…” “It tells you how they work. This is the first time somebody actually told me HOW the medicines are supposed to work, rather than just take this---it will make you better...It was a good beginning. If I wanted more, I know I could go find some somewhere;” “It tells me what the symptoms [of lactic acidosis] are, but I still don't know what it is;” “Like nutrition information and other things that could be part of a package [of materials for a patient just diagnosed with diabetes].” Supplementary resources added to the site included links to expert-curated sites featuring information about nutrition, exercise, type 2 diabetes research volunteer opportunities, Fast Facts, a graph using pictures to explain type 2 diabetes, and a mechanism for users to provide feedback about the site. All links take the user directly to the relevant portion of each web site and are appropriate for patients with type 2 diabetes. Sources of these educational links are VUMC, Centers for Disease Control (CDC), Let’s Move!, the National Institute of Diabetes and Digestive and Kidney Diseases, and Diabetes for Dummies (Figure 2).

Figure 3:

Screenshot of iAdapt resource in the MHAV Patient Portal Note: Figure 3 shows the rest of the AHRQ materials, share tools, and the supplementary physical activity, nutrition, and information links.

A set of “Fast Facts” was created to appear in a box located in the upper right corner of the page. A relevant diabetes fact is highlighted, and the fact changes each time the patient visits the site. The user can scroll to another fact easily by pushing the arrow located at the bottom of the “Fast Facts” widget. Information for these facts was sourced from web sites such as the AHRQ, CDC, the National Diabetes Education Program (NDEP), diabetes.com, and National Institutes of Health (NIH) websites and then rewritten to be literacy appropriate. Some examples of fast facts chosen are “You have a higher risk of being diagnosed with diabetes if you are: Asian American, Hispanics/Latino, African American,” “In 2005, 2.4 percent of American adults ages 20–39 had type 2 diabetes (CDC),” and “The risk for death among people with diabetes is about double that of people without diabetes (ndep.nih.gov).” This feature was included to address patients’ expressed interest in quick information about their diabetes and diabetes medications. The focus groups indicated that such information would be interesting and informative: “I like the fast facts, you know, the bullet points” (patient); “I think this is the most valuable information of the whole thing” (expert).

Several other additions and changes to the site directly resulted from focus group suggestions. For example, a link to relevant research opportunities was included. Volunteers can sign up for studies related to their condition by emailing the person listed in that section of the page. The patient focus group also liked the idea of a graphic representation of how type 2 diabetes works, “that’s the thing that threw me because it was just all text. When reading some of this, I like to see, you know, like a slide of what’s going on.” The team addressed this request by incorporating the diagram displayed in Figure 3 into the MHAV patient portal. Finally, emphasis was placed on the portal resource being easily navigable. Specifically, the research team attempted to reduce the number of clicks for the user to get to desired information and limited page length to reduce scrolling.

Figure 3.

This figure demonstrates the process of type 2 diabetes graphically.

Final Patient Portal Interface and Evaluation

The final product of the adaptation process was a patient-portal based web site that included the adapted “Quick Summary Guide,” (Figure 4), other AHRQ materials relevant to the type 2 diabetes mellitus oral medications, and supplementary links to help patients understand how to improve their health. The site was designed to allow patients to learn and take action in a single location. Patients can see their medications, read relevant literacy-appropriate materials, and message their healthcare providers regarding what they have learned through MHAV. The research team is in the process of evaluating whether this information affected communication between patients and providers and clinical outcomes.

Figure 4:

Quick Type 2 Diabetes Summary Guide Note: Figure 1 shows the study’s one-page adaptation of the AHRQ summary guides. At the top are 4 things to know about a patient’s diabetes pills, and how to manage type 2 diabetes with pills. The middle is comprised of side effects, and what merits an emergency side effect. The chart at the bottom indicates which side effects are most closely associated with each oral medication for type 2 diabetes.

Three members of the research advisory team evaluated the adapted materials using the SAM instrument as described above. We found slight agreement among the three raters, who were chosen based on expertise related to health literacy and/or diabetes: Kappa = 0.05 with p value equal to 0.6, (95 percent confidence interval :- 0.201, 0.321). The maximum possible score for this project on the SAM was 40, due to the fact that the researchers discerned that items 3a (“cover graphic shows purpose”) and 5a (“interaction used”) were non-applicable to the scope of this project. All three reviewers rated the resource in the adequate to superior range (40 percent and above), scores being 35, 34, and 18. The SAM evaluators offered written and verbal feedback with their assessments to explain some of their rating decisions.

Discussion

This experience of adapting evidence from comparative effectiveness research for delivery to patients and providers through a technology patient portal yielded several generalizable lessons. Through a review of related research and direct patient comments, we learned that form was important. Patients described a desire to be able to easily access the information that was relevant to them and have links to related information about managing their health. Patients also wanted personalized information and multimedia such as graphics and audio to support the content.

This project offers several important lessons. First, the project defines a generalizable approach to adapting summary guides to be more readily and effectively used online by patients and healthcare providers. Collecting feedback from end-users (i.e., patients and providers) during the design phase of a project ensures that materials are adapted to be relevant and targeted. Adapted summary guides and supplementary information such as those described in this paper can be introduced into provider workflow through EHRs that can match relevant resources to patients with specific diagnoses. This information also can be provided on patient health portals, and providers and patients alike can be informed of this information through email, appropriate clinic advertisements, and other forms of dissemination as contextually appropriate. Second, the project utilizes state of the art health information technology tools to disseminate adapted comparative effectiveness research evidence, and attempts to reach providers both directly and through patients. The adapted evidence is made inherently actionable when presented in conjunction with patient portal and EHR tools that support patient and provider communication. In addition, this online presentation allows users to provide immediate feedback about the adapted material, which can be collected in a formative manner through messaging or simple feedback plug-ins. Third, the project expands on burgeoning research investigating whether providing scientific evidence directly to relevant patients is more effective than delivering evidence to healthcare providers, and whether such knowledge learned from patients is incorporated into general practice. Empowering patients with concise and targeted information along with a mechanism to communicate directly with their healthcare teams can facilitate the management of health conditions (e.g., medication change) as well as improve the quality of the summary guides (e.g., inclusion of new medications on the market).

The research team plans several enhancements to this adaptation after completion of its evaluation; many of these ideas arose from patient and provider feedback on the resources. We plan to incorporate graphics to represent concepts such as exercise, nutrition, and information about diabetes. Such graphics would make the site not only more literacy appropriate, but warmer and more user-friendly. Both healthcare providers and patients expressed interest in adding pictures of pills. As one provider from our focus group sessions indicated, “I have a lot of patients come in and say I’ve got this pillbox and I don’t know what I’m taking.” The dynamic nature of both the appearance of pills and the availability of generics pose resource and practically challenges for this enhancement. Insurance information and medication cost are also features that we would like incorporate into to this portal resource. Ideally, patients could receive personalized information about the cost of pills, cased on their insurance provider. Providers from our focus groups indicated that “the dollars makes a big difference in compliance. You know, if the patient knows that you're sort of considering that when you are prescribing….” We believe the current medical comparison chart would be more useful as a dynamic application, which allowed patients to selectively hide or show side effects and medications, and to record personal experiences with side effects, which would then be automatically reported to their healthcare providers. Finally, printing capabilities on the website would be useful for patients, along with other sharing tools. Many users still prefer paper to reading on a screen, as evidenced in both our patient and provider focus groups, “I’m kind of old school and I do prefer to sit down with a paper and read it (patient); “I like printouts and stuff I can give out” (provider). The print button on the screen, next to the materials to be read would allow less-computer-literate and computer-savvy users alike the capability to easily print out from the page.