Development of a Plain Language Decision Support Tool for Cancer Clinical Trials: Blending health literacy, academic research, and minority patient perspectives

Abstract

Background:

Despite the promise of clinical trials for improving cancer care, less than 5% of all cancer patients participate. Racial/ethnic minorities continue to be underrepresented in cancer clinical trials (CCTs). To address this gap, we developed a plain language, web-based decision support tool (CHOICES DST) in English and Spanish to support decision making about CCTs among Blacks and Hispanics.

Methods:

In Phase 1 (information collection), we conducted qualitative interviews with 45 cancer patients, completed a thorough literature review, and reviewed results from a telephone survey of 1100 cancer patients. In Phase 2 (content generation), we created the first iteration of the CHOICES DST. In Phase 3 (usability testing), we gathered user experience and acceptability data from a small sample of cancer survivors (n=9).

Results:

The Knowledge, Empowerment, and Values Clarification (KEV) Model of decision making was developed based on data from Phase 1. The KEV Model and other Phase 1 data allowed us to create the CHOICES DST platform. Usability testing of the CHOICES DST showed highly favorable responses from users, satisfaction with content, ease of navigation, and a desire to use the tool. Qualitative results identified addressable points that would benefit from content and navigation-related alterations.

Conclusions:

The final version of the CHOICES DST was well received and understood by Black and Hispanic participants, and adheres to the mandates for plain language communication. This research provides preliminary data that CHOICES DST holds promise for improving knowledge of CCTs and potentially improving informed decision making about participation in trials.

BACKGROUND

Cancer is the second leading cause of death in the United States [1]. For 2018, it is estimated that there will be approximately 1.7 million newly diagnosed cancers, and over 609,640 deaths from cancer in the United States [2]. Cancer clinical trials (CCTs) evaluate new methods of preventing, treating, and managing symptoms of cancer [3]. Despite the promise of clinical trials for improving cancer care, fewer than 5% of all adult cancer patients participate in clinical trials [4]. Of those who do participate, racial/ethnic minorities continue to be significantly underrepresented [5]. Crucially, however, studies have found that both Black and Hispanic cancer patients are just as willing as Whites to participate in clinical trials [6–9].

Documented barriers to minority enrollment in clinical trials include medical mistrust, fear of experimentation, disparities in patient-provider communication, logistical challenges, limited knowledge of clinical trials and how to find them, and a lack of culturally targeted interventions to educate racial/ethnic minorities about trials [10–15]. While barriers to CCT participation have been well documented in Blacks [16–18], less is known about Hispanic or Spanish speaking populations, although previous studies have documented lower levels of knowledge of CCTs in Hispanics than in other racial/ethnic groups [19–21]. In addition, enrollment in CCTs by minority cancer patients may also be hampered by a lack of engagement and outreach to appropriate communities, and a failure to address health literacy and plain language needs in the development of interventions to improve knowledge of trials. The 2003 National Assessment of Adult Literacy found that minorities had lower levels of general [22] and health [23] literacy relative to Whites. Thus, for all cancer patients, but in particular for minority cancer patients, decision making regarding participation in CCTs is not likely being done in a manner which fosters quality, informed decision making using literacy-level appropriate methods.

For situations involving a need to make complex and preference sensitive decisions, such as decisions about joining a CCT, decision aids and decision support tools (DST) have been shown to improve patient decision making and lead to better patient outcomes [24–27]. In addition, to address and overcome health literacy barriers, experts urge that materials be developed using plain language [28], which has been described as a strategy that seeks to employ “communication your audience can understand the first time they read or hear it” [29]. Clear communication is important for any decision aid or decision support tool, but it is perhaps even more imperative for decisions such a joining a clinical trial where the concepts and information that must be conveyed in the DST are complex and often unfamiliar to patients.

The goal of the CHOICES DST was to provide Black and Hispanic cancer patients with three components to facilitate informed decision making about participation in CCTs. In this manuscript, we describe the development and initial usability testing of the CHOICES DST. This development was grounded in: a) information from target users of this type of decision support tool; b) a conceptual model of informed decision making; c) plain language principles; and d) the International Patient Decision Aid Standards (IPDAS) guidelines for decision making support tools. We believe the process for development of CHOICES can be translated to other medical decisions where plain language DSTs are needed, particularly those designed for racial/ethnic minority populations.

METHODS

The development of the CHOICES DST included contributions of an interdisciplinary investigative team of research scientists and practitioners with expertise in health communications, medical decision making, decision aid development, clinical trials, and health literacy; as well as partnerships with three community-based organizations in Miami, Florida who collaborated on all aspects of the study. There were three main phases of development reported here: 1) Information collection, 2) DST development, and 3) Usability testing, which are described in more detail below.

Phase 1: Information Collection

In Phase 1 of CHOICES development, data to inform the design of the CHOICES DST was collected from 3 sources: 1) reviewing results from a telephone survey of 1100 cancer patients regarding participation in clinical trials, 2) performing a literature review, and 3) conducting 45 semi-structured interviews with cancer survivors.

The first source of CHOICES content included survey data regarding knowledge, attitudes and participation in CCTs conducted in 2009 with 1100 White, Hispanic, and Black cancer patients [11]. Study participants diagnosed with breast, lung, colorectal, or prostate cancer were identified through the Florida Cancer Data System and then surveyed by telephone to obtain demographic information, past participation and willingness to participate in clinical trials, and barriers and facilitators to clinical trial participation.

Second, an extensive and rigorous literature search for data related to participation in CCTs was also conducted. Specific search terms were based on Medical Subject Headings (MeSH®) in the National Library of Medicine’s vocabulary thesaurus and included CCTs, recruitment, race, ethnicity, minorities, patient selection, barriers, underrepresented populations, socioeconomic status, health disparities, and decisions. Initially, 676 unique articles were identified. The research team later deemed 208 as likely relevant, and ultimately 24 were found to be most useful for informing the development of CHOICES, and provided information regarding the specific educational content and attitudes that warranted attention within the DST.

Third, in-depth key informant interviews were conducted by trained interviewers with 15 English speaking Blacks, 15 English speaking Hispanics, and 15 Spanish speaking Hispanics to gather information regarding experiences, attitudes, participation, comfort, and barriers associated with CCTs. All participants were from Southeast Florida (specifically Miami-Dade and Broward counties). Spanish speaking Hispanics were interviewed by bi-cultural, bilingual Spanish members of the development team. Interviews lasted from 30 to 60 minutes and were guided by a semi-structured interview guide. Audio from the key informant interviews was recorded and transcribed by a professional transcriptionist.

Data collection/coding for both the literature review and the semi-structured key informant interviews was facilitated by the development and use of (separate) detailed data collection forms and coding dictionaries. For the interviews, multiple research personnel reviewed each interview transcript to develop a preliminary coding dictionary which was subsequently revised and refined using input from all study personnel. This coding dictionary was applied using Atlas.ti to all key informant interviews. All interviews were coded individually by at least 2 study team members, and inconsistent coding was resolved through discussions to reach consensus. If consensus could not be reached, the specific material in question was reviewed by the PI or qualitative research lead who determined the best resolution to coding the material. Rigorous training and supervision of interviewers involved discussions of reflexivity and exploration of interviewer reflections on the process and data collection.

A separate data collection sheet and coding guide was developed – in a similar collaborative and iterative fashion – for use in the literature review. Three members of the research team conducted the literature searches on PubMed and systematically coded articles that met the inclusion criteria. The final 24 most relevant articles were coded by at least 1 member of the research team. Example articles in the final list of 24 included those by Juraskova et al.,[30] Entwistle et al. [31], Wood et al. [32], McCaskill-Stevens et al.,[33] and Corbie-Smith et al.[34]

Phase 2: Content Development

Phase 2 involved the development of parallel CHOICES DST websites, one in English and the other in Spanish. Based on the tripartite information sources in Phase 1, the initial content development was led by the PI with input from the investigative team. Content was subsequently reviewed, reorganized, and modified by a plain language expert, so that it conformed to health literacy guidelines. CHOICES DST development style guidelines emphasized the following points: 1) using familiar, conversational words (e.g., cancer vs. oncology) and sentences; 2) explaining technical terms needed for understanding an issue (e.g., “placebo”); 3) using active voice and personal pronouns; 4) organizing complex information into short, logically sequenced sections; 5) emphasizing simple, clean graphic design and layout; 6) reflecting culture in text and images; and 7) field testing material to ensure that it was well received by the target audience. Finally, the content and presentation of the CHOICES DST was developed with an eye to the IPDAS guidelines for decision aids/support, to ensure that our tool was closely aligned with best practices in decision aid development.

Phase 3: Usability Testing

Lastly, we assessed the web-based CHOICES DST for usability, acceptability, and understandability among our target population. To do this, we conducted usability assessments with 6 Hispanic and 3 Black cancer survivors. Three related strategies were used to elicit usability information, including cognitive interviewing/talk aloud technique, quantitative usability assessment using standardized surveys, and a debriefing interviewing.

Cognitive Interviewing (CI).

Cognitive interviewing is a technique used to elicit the cognitive processes used by respondents when completing a DST, questionnaire, or other instrument. The technique allows researchers to observe and query respondents as they engage and complete the DST, identify any potential misinterpretation of information or responses, and detect any structural problems that could impair valid administration. We used a combination of a concurrent talk-aloud and probing strategy as the participants navigated through the web-based DST. While talking-aloud was the primary strategy, we also employed emergent probes to stimulate additional comments when indicated or to remind participants to talk-aloud as they respond to the DST.

Usability Survey.

The System Usability Scale (SUS) developed by Brooke was administered to each participant after completing the cognitive interview of CHOICES [35]. Broadly, the SUS elucidates the effectiveness of a tool (does it achieve its objectives), efficiency of a tool (how quickly and with how much effort are objectives met), and overall user satisfaction with the tool.

Debriefing Interview.

As an additional assessment of the overall user experience, debriefing interviews were conducted with each participant. The debriefing questions included the following: 1) In general, what were your overall impressions of CHOICES? 2) What difficulties did you run into and what would have made it easier for you to get through the CHOICES information? 3) Considering everything we have already discussed, is there anything else you’d like to mention about CHOICES or your experience with it?

RESULTS

Phase 1: Information Collection

Information from the 1100 telephone interviews revealed expected racial differences in participation rates (i.e., Hispanics were less likely to have participated in a clinical trial compared to White and Black patients). However, the data also demonstrated no racial differences in overall willingness to participate in clinical trials [11]. Therefore, our development of the content of the CHOICES DST was based on the idea that minority cancer patients do not necessarily need to be persuaded of the value of participation, but rather that the CHOICES DST should focus on overcoming knowledge deficits, structural barriers, and other challenges to participation.

Information from our literature review, which ultimately focused on 24 of the most relevant published articles, also supported this premise, concluding that decision support approaches would be a viable strategy for supporting informed decision making about clinical trials decisions. For example, Wells and Zebrack found that minority cancer patient decisions about treatment and participation in clinical trials are impacted by barriers and multiple level (e.g., intrapersonal, interpersonal, environmental, and sociocultural), but that greater trust in researchers lead to more favorable views on participation.[36] Wood et al. found that minority cancer patients were more likely to view the informed consent process as a legal protection for the physicians/researchers, and not the patients; but that this misconception and barrier to participation could be corrected through improved patient knowledge.[32] Juraskova et al. found that a decision aid booklet for a breast cancer prevention trial (IBIS-II DCIS) increased women’s knowledge about the purpose of the trial and helped them decide whether or not to participate. [30]

The semi-structured interviews with 45 diverse cancer survivors (15 Black, 15 English-speaking Hispanics, and 15 Spanish-speaking Hispanics) provided the most directly useful information for development of a conceptual model of decision making and the CHOICES DST content. Interview participants were mostly women (93.3%) with breast cancer (66.7%), and had an average age of 56.0 (SD 10.6) years old. The education level of participants was also diverse (31.1% high school or less; 35.6% some college; 22.2% 4 year college degree; and 11.1% post-graduate education).

From the interview responses, we found that almost 90% had never been asked to participate in a cancer clinical trial and less than 10% reported having a discussion of clinical trials with a health care provider. Nevertheless, over 70% indicated that they would be willing to participate in a trial. Almost all participants (97.8%) stated that it would be helpful to hear the experiences of someone who has participated in a trial, but only 16% thought that it would make a difference if the person relaying their experiences were the same or different cultural heritage than the participant.

After thematic coding of the interviews conducted by members of the investigative team, three overarching themes emerged from the interview data: knowledge (about clinical trials), empowerment (e.g., social support, sources of information), and values (e.g., thoughts about religion, culture, and altruism). For knowledge, there was a range in understanding and awareness about CCTs, including confusion about how CCTs differed from standard treatment and whether an individual had actually participated in a CCT as part of their previous treatment. Specific to empowerment, there was substantial variability regarding information seeking and perceptions about trusted sources of information. Some cancer survivors reported only following the advice of their doctor, while others conducted independent, in-depth health information searches using web-based resources or social networks. Many individuals stated that they were reluctant to ask “too many questions” of their cancer providers. Finally, with regard to values, various altruistic perspectives were noted, including the idea of “paying it forward,” a willingness to help others, and wanting to help advance science. Results from the qualitative interviews were used to generate a framework based on patients’ Knowledge, Empowerment, and Values Clarification (KEV Model; see Figure 1) to facilitate informed decision making about CCT participation [37].

Fig. 1.

KEV model for CCT participation

Phase 2: Content Development

Integrating the rich qualitative and quantitative data collected in Phase 1, the investigative team developed a web-based, plain language DST called CHOICES. The CHOICES DST was developed in both English and Spanish language versions.

Based on the KEV model, CHOICES DST covered three primary content domains: patients’ knowledge, empowerment, and values clarification regarding CCTs. The knowledge domain encompassed CCT options (e.g., benefits and risks, randomization concepts, who participates, etc.), and personal risk assessment including eligibility considerations. The empowerment domain incorporated communication skills training and deliberative skills development activities. Communication skills training emphasized that patients could make autonomous decisions about participating or not participating in a CCT, and that concordance with their physicians’ recommendations was not necessary. The values domain facilitated elucidation of personal and values clarification. CHOICES DST presented users with 12 attributes or feelings (pros and cons) that they might have about clinical trials using statements like: 1) It is important to me to help others with this disease in the future, 2) I would feel like a guinea pig if I were to participate, or 3) It is important to me that the research team has an excellent reputation. Participants were asked to move a slider to indicate how important each attribute was to them. After assessing all attributes, CHOICES DST generated a summary of response by domain (e.g., altruism, trust, inconvenience), which was presented to the participants, who in turn could use the summary to consider their values and make decisions that best reflected their individual perspectives.

CHOICES DST also integrated composite patient stories to illustrate choices that a patient might make in situations of a first diagnosis of cancer or a recurrence. As noted by Shaffer and Zikmund-Fisher [38], narratives can serve multiple purposes within a DST including: 1) providing information, 2) making material more engaging, 3) modeling certain behaviors, 4) persuading a target population to start or stop a behavior, and 5) providing comfort to patients and families experiencing a chronic health condition or acute medical event. By including stories, we aimed to provide easily understood information, make CHOICES DST more engaging, model an informed decision-making approach, and emphasize that participation in a CCT is a choice that is right for some people but not for others. To give a balanced perspective and to present alternative outcomes to hard choices, our stories included patients who said yes and patients who said no to participating in CCTs.

Finally, CHOICES DST was designed to prepare patients for the informed consent process. CHOICES DST emphasized that an adequate understanding of CCTs is necessary before participants make optimal personal decisions about participation and described the process and meaning of signing an informed consent document.

As the investigative team conducted iterative development and refinement cycles on CHOICES DST, an expert web designer worked with the team to develop an attractive, easy to use web design that facilitated easy and flexible navigation. The design goal sought to minimize complexity but allow for a high-quality experience for both novice and tech-savvy web users to optimize user engagement among individuals with varying levels of computer literacy. Research and community partner team members reviewed and evaluated evolving content and web design multiple times throughout the development and refinement process.

Phase 3: Usability Testing

Both quantitative (survey) and qualitative (cognitive interviews and debriefing interviews) were analyzed to form study results from Phase 3. Participants (n=9) in Phase 3 all spoke English: 4 Black, and 5 English-speaking Hispanics. Usability testing participants were all women with an average age of 57.8 (SD 6.3) years, and most 78% had breast cancer. The education level of participants was 44.4% high school or less; 44.4% with some college; and 11.1% with a 4 year college degree or more education.

Quantitative Data.

As shown in table 1, responses to 9 out of the 10 SUS items indicated highly favorable reviews of the website’s usability (Table 1), achieving mean ratings above 4 on the 1 to 5 rating scale. Most importantly for usability, all participants rated their confidence in using CHOICES as a 5, indicating excellent user experience in terms of navigation and comprehension. Additional high levels of usability were noted for other items indicating ease of navigation, learning to use the website quickly, and being user-friend or not requiring substantial technical expertise. The lowest rated item involved assessment of the intention to use the site frequently, which only had a mean of 3.8 on the 5-point scale. Overall, responses to the SUS provided favorable support for the usability and overall user experience associated with CHOICES.

Table 1:

SUS Items Results

Question	Item	Mean (sd)
Q1	I think that I would like to use this site frequently	3.78 (1.48)
Q2.R	I found the web site unnecessarily complex	4.17 (1.12)
Q3	I thought the web site was easy to use	4.28 (1.39)
Q4.R	I think that I would need the support of a technical person to be able to use this web site	4.28 (1.44)
Q5	I found the various functions in this web site were well integrated	4.44 (1.01)
Q6.R	I thought there was too much inconsistency in this web site	4.50 (0.93)
Q7	I would imagine that most people would learn to use this web site very quickly	4.56 (0.73)
Q8.R	I found the web site very cumbersome to use	4.17 (1.32)
Q9	I felt very confident using the web site	5.00 (0.0)
Q10.R	I needed to learn a lot of things before I could get going with this web site	4.13 (1.64)
	Average of all responses	4.33 (0/75)

^*Values range from 1=Strongly Disagree to 5=Strongly Agree. The values for even number items (QX.R) are reversed coded so that for all items high numbers indicate more positive response.

Qualitative Data.

Rich narrative data was drawn from participants as they completed the talk-aloud element of cognitive interviewing. Additional narrative data was drawn from the debriefing interview which posed specific questions about the CHOICES user experience.

Cognitive Interviewing Results.

Data from the cognitive interviews identified a small number of navigational challenges and a few areas where the visual layout was not clear for users. In particular, users posed comments regarding navigation tabs that could be changed and clarified and a few images that could be improved and relocated. However, the vast majority of users did not appear to experience barriers to navigating the website, reading the material, or using the integrated activities such as the values clarification activity.

Debriefing Interview Results.

Several user-driven suggestions were identified during the debriefing interviews. First, users posed questions about the focus of CHOICES, specifically pertaining to whether the tool was meant to be generally used for any CCT decision or if it was meant to be tailored to the decision about a particular CCT. Second, users appreciated information about the investigative team, but requested additional information about whether cancer survivors had contributed (i.e., participants felt that it would be encouraging to know that the study team also included cancer survivors). Third, several participants posed questions about potential harms associated with participation in a CCT along with a more detailed description of potential side effects. Fourth, participants noted that additional information about a thorough informed consent process and what to expect might help off-set some of the challenges of feeling like “guinea pigs” in the research process. Fifth, the description of placebos was viewed as limited and problematic with users requesting more clarification and detail. Finally, and perhaps most importantly, many participants reported that they wished they had had access to a tool like this immediately after their initial diagnosis when they were making their treatment decisions.

DISCUSSION

This research contributes to the broader literature base addressing how to engage patients during the process of developing health education materials. In particular, this effort is an example of how to develop a DST that is targeted to racial/ethnic populations, and which intentionally integrated health literacy guidelines throughout design phases. Our process of patient engagement can be summarized in the following steps: 1) Information collection to guide the development of the DST; 2) Content development of the DST based on data from diverse sources; and 3) Usability testing of the DST. As DSTs and other interventions become more widely used, particularly with minority populations, there is a need for careful tool development, including attention to plain language principles and attentive intervention design. These efforts are particularly timely for interventions designed to improve knowledge and decision making about enrolling in CCTs [25, 26, 31, 39]. The CHOICES DST goes beyond education and knowledge enhancement to incorporate values and empowerment components as recommended by IPDAS [40, 41].

Plain language and health literacy are often overlooked areas in the patient centered care and shared decision making movements, and warrant more attention moving forward. In developing the CHOICES DST, the investigative team was mindful of using plain language and usability guidelines for designing effective health-related websites [42, 43]. Ensuring that information about CCTs is relevant and understandable also has ethical implications. For example, the respect for persons principle in the Belmont report is founded upon the idea of autonomy (i.e., individuals have a right to decide for themselves whether to participate in a CCT). Without accurate clinical trial information and an informed consent process that patients can comprehend, full autonomy about the decision to participate in CCT is compromised.

Some limitations of this project should be noted along with its strengths. First, although the sample in the pilot test was diverse, the findings may not be applicable to all racial/ethnic groups, men, or all cancer types. Second, by deliberate decision, all of the participants were at least one-year post-treatment. This decision was made based on the fact that the investigative team believed that better initial information on content could be obtained from individuals who had experienced both the initial treatment decision and the treatment, so as to experience the impact of the decision. The investigative team also believed it was unethical to conduct initial development of the CHOICES DST with vulnerable newly diagnosed cancer patients. Finally, although the CHOICES DST was well received by participants, we have not yet assessed its impact on CCT decision making for specific trials, actual enrollment in a CCT, or the perspectives of oncology clinicians or clinical research assistants.

Conclusions

The CHOICES DST was created by following a specific developmental process that can be adapted for future DSTs. We believe the process for development of CHOICES can be translated to other medical decisions where plain language DSTs are needed, particularly those designed for racial/ethnic minority populations.