Abstract
Background:
Racial/ethnic minorities are underrepresented in clinical research in the United States for multifarious reasons, including barriers to effective communication between researchers and potential research participants. To address the communication barriers between researchers and potential participants, we developed a Research Literacy Support (RLS) tool. The focus of this report is to present findings from the second and third phases of development that refined and assessed usability of the RLS tool.
Methods:
We utilized a mixed methods approach that entailed iterative cognitive testing with participants (N=52) from diverse racial/ethnic backgrounds and interviews with clinical research recruiters (N=20) to modify and refine the design and content of the RLS tool (Phase 2). This was followed by assessment of the usability of the RLS tool by 100 participants (Phase 3).
Results:
During Phase 2, participants provided feedback about layout, word choice, and comprehension of the tool. In Phase 3, participants recognized that they had gained knowledge about clinical research from the RLS tool, although they still had a substantial learning gap after using the tool, indicating an opportunity for further refinement.
Conclusion:
The RLS tool may help advance health equity by addressing communication barriers that may impede minority participation in clinical research.
Keywords: cancer health disparities, health equity, research literacy, clinical research, biobanking
Introduction
Increasing the diversity of participants enrolled in clinical research is a national public health priority in the United States [10, 23]. Barriers to minority participation in clinical research include mistrust, costs, transportation, and study design inclusion and exclusion criteria such as comorbidities and socioeconomic status [3]. While many individuals face major challenges in making informed decisions about enrollment in clinical research, including clinical trials and biobanking research [8, 15], these challenges to participation affect minority and other underrepresented populations at higher rates than the general population [2]. Challenges with communication about clinical research are varied, and include critical information that is difficult for researchers to convey to potential participants in plain language and incorrect assumptions researchers can make about what is understandable language [8, 12].
Reaching a clinical trial’s or bio-banking study’s recruitment and retention goals remains critical for successful clinical research program implementation and data analysis, particularly with regard to minority and other underrepresented populations This underscores the need for innovative recruitment strategies [22, 24]. The disparity in research participation by minority populations, together with assumptions that potential participants understand clinical research/bio-banking terminology in the informed consent process, served as the impetus for our development of the Research Literacy Support (RLS) tool. Using plain language principles [7], we designed the RLS tool. Our goals were to increase awareness of clinical trials and biospecimen research among underrepresented populations by enhancing literacy regarding the purpose and process of medical research. The RLS tool is an interactive tool to further enhance communication, allowing recruiters to address areas of concern or misinformation.
We previously reported the first phase of development of the RLS tool in partnership with the Cancer Disparities Research Network (CDRN) and under the auspices of the NCI funded Geographic and Biobanking Management Program (G/BMaP) [21]. CDRN experts and other stakeholders first defined the objectives of the tool and the research concepts for inclusion, developed a prototype, and, following the principle of user-centered design, obtained feedback from end users (prospective research participants and recruiters) for further development of the prototype. The prospective research participants represented diverse, English speaking, underrepresented groups, and the recruiters were experienced in enrolling participants for different types of research studies, including cancer clinical trials [21]. We now report the second and third phases of development and assessment of the RLS tool, which utilized a mixed methods approach that entailed iterative cognitive testing to modify and refine the design and content (Phase 2) and a survey of end users to assess tool usability (Phase 3).
MATERIALS AND METHODS
Study setting.
The study setting included five academic medical centers with cancer centers and CDRN membership or their CDRN affiliates [20]. Northwestern University served as the Coordinating Center for RLS tool development. The other participating CDRN member organizations were Fox Chase Cancer Center, Mayo Clinic Cancer Center, Ohio State University, and Roswell Park Comprehensive Cancer Center. Settings were located in the greater Chicago area, the western New York region, rural Appalachian Ohio, the greater Philadelphia area, and also included a Native American urban sample.
Study design and intervention development process.
Developing the RLS tool involved four steps, delineated in three phases. Phase 1 was recently published; it entailed development of the prototype of the RLS tool by a panel of CDRN experts and other stakeholders followed by pilot testing of the prototype with five clinical trial recruiters and 10 prospective research participants [21]. Upon completion of Phase 1, the RLS tool consisted of a series of 34 cards introducing key informed consent concepts associated with research. The tool was organized into sections on basic research information, study-specific information, and research participants’ rights in addition to cards that addressed myths and truths. Multiple prompts served as conversation starters to facilitate discussion between study recruiters and research participants [21]. The tool was divided into two sets of cards, one focused on biobanking and the other on clinical trial research. Each set consisted of 28 cards that included 22 shared cards and either six clinical trial-specific cards or six biobanking-specific cards.
Phase 2 consisted of two iterations of semi-structured interviews with potential research participants (cognitive testing) (N=52) and experienced clinical research recruiters (N=20) followed by modifications to the tool based on the results. Phase 3 was a post-test survey of the tool modified in Phase 2 that was given to prospective research participants (N=100). Our mixed methods approach of cognitive testing, semi-structured interviews, and surveys produced new insights during the development process that guided the study design, including the number of iterative testing cycles and core constructs. This study was approved by the Institutional Review Board (IRB) of Northwestern University (which served as the central IRB), as well as Ohio State University, Fox Chase Cancer Center, Mayo Clinic, and Roswell Park Comprehensive Cancer Center IRBs.
Study Population.
The study population consisted of adults and clinical trial recruiters at multiple sites. Participants of diverse race/ethnicity were recruited from Ohio State University (Rural Whites), Mayo Clinic (Native American urban sample), Roswell Park (African Americans and Native Americans), and Fox Chase Cancer Center (Latinos/Hispanics). In both Phase 2 and Phase 3 of the study, eligibility criteria were: men and women, 18 years of age or older, and able to read and speak English. Researchers recruited a purposive sample of adults from diverse racial/ethnic backgrounds, research experiences (with/without prior research participation), and cancer status (with/without a cancer diagnosis). Research experience was broadly defined as any previous research participation in clinical, social/behavioral and/or community interventions requiring written informed consent. Potential participants were recruited via newsletters, referral from clinical research staff, and flyers posted in clinics and community organizations. Recruitment materials indicated participants would receive a gift card upon completing the interview.
In Phase 2, researchers recruited a purposive sample of clinical trial recruiters who were employed at the above CDRN member sites and had experience recruiting participants from diverse racial/ethnic backgrounds. Efforts to recruit clinical trial recruiters consisted of sharing study information and materials via e-mail listservs, newsletters, and presentations at staff meetings.
Cognitive testing of participants.
Phase 2 was iterative and consisted of two sets of cognitive interviews intended to modify and refine the design and content of the prototype to ensure the tool’s appropriateness and usability among diverse English-speaking populations. At each site, recruitment goals for Phase 2 consisted of 10 research participants and four clinical research recruiters. Researchers conducted the first round of semi-structured interviews between August and September 2013 at Northwestern University and a partnering suburban community clinic that serves low-income uninsured patients. Researchers made the first set of revisions to the RLS tool after this round. Researchers then conducted the second round of interviews with the modified RLS tool between September 2013 and June 2014 at four additional CDRN affiliates. Interview length was approximately 30–45 minutes. Researchers revised the RLS tool based on the feedback provided in the second round of interviews.
Researchers collected the following information: 1) participant demographics (age, sex, education level, race/ethnicity, bom in the US, country of origin, length in the US [if not US bom], primary language/s spoken, personal/family cancer history, and previous research/biobanking participation), 2) participant interviews (general preferences about various aspects of the tool, understanding of card-specific clinical research concepts, and tool usability) , and 3) clinical recruiter interviews (perspectives about tool language comprehensibility, challenges associated with recruiting participants, and suggestions to improve content, overall usability, and potential utility of the RLS tool for recruitment. Additional information collected included experience with different types of studies [screening, treatment, etc.], diseases/conditions, and years of recruitment experience.)
Post-test survey.
In Phase 3, researchers conducted a post-test survey of participants about the RLS tool after its revision in Phase 2, in order to assess usability. The participant survey included questions on content, format and readability, comprehension/understanding, willingness to participate in research, and preferred methodology for delivery/administration of this instrument. Each site had a recruitment goal of 20 participants, 10 with and 10 without prior research participation. Within each group of 10, participants were randomized, with half reviewing the clinical trial-specific and half reviewing the biobanking-specific set of cards. Interviews of approximately 30–45 minutes were conducted between May and September 2015. The recruitment sites remained the same as in Phase 2 except that Native Americans were recruited from urban (off-reservation) communities located throughout the western New York region.
Data Collection.
After reviewing the RLS tool while alone, participants and recruiters (Phase 2 only) were interviewed about the tool. To standardize data collection, the coordinating center developed interview guides for participants and recruiters and trained research staff conducting the interviews via telephone conference calls. For cognitive testing (Phase 2), investigators and researchers across the five academic centers jointly developed the qualitative data collection instruments. For the post-test survey (Phase 3), using the interview guide, research staff solicited quantitative feedback on the RLS tool. The post-test survey was a quantitative, multiple choice instrument conducted in the form of an interview, i.e, it was read to participants.
Data analysis.
Qualitative analysis.
During Phase 2, trained research staff took detailed notes during interviews and uploaded the notes to REDCap® software to extract common themes and responses as they relate to words, images, readability, and overall content of the communication tool. Using grounded theory and the constant comparative method as previously described [25], the research team examined and coded the transcripts using data analysis software Atlas.ti version 6.2, and then used a priori codes based upon the semi-structured interview guide to develop an initial codebook. In Phase 2, the research team used feedback from participants and recruiters to modify the RLS tool. Researchers across sites discussed revisions and reached consensus before implementing any changes for the next version of the tool.
Quantitative analysis.
We used a cross-sectional analysis approach using SPSS (version 23) by calculating descriptive statistics to categorize topics. Demographic data from each site were analyzed using descriptive statistics. This intervention development study was not powered for inferential statistics. Because the primary purpose of Phase 3 was assessing overall usability, no additional modifications were done after the post-test survey.
RESULTS
Demographics
A total of 172 participants and recruiters took part in this study, including 52 participants and 20 recruiters in Phase 2, and 100 participants in Phase 3. Participant demographics are shown separately and combined for Phase 2 and Phase 3 (Table 1). Briefly the aggregate demographics across Phase 2 and Phase 3 include the following. Participants were mostly female (68%). Participants’ self-identified race included: White (42%), African American (30%), and Native American (20%). In addition, 23% of participants self-identified ethnicity as Hispanic/Latino. Nearly one-third (30%) reported completing high school as their highest educational attainment. Although less than half of the participants indicated a personal cancer diagnosis, 88% of the participants reported having a family member who had cancer, which included breast, colon, ovarian, and prostate cancer. In addition, consistent with our intention of recruiting participants with a mix of research experiences, 43% of participants reported participating in research studies, e.g., colorectal cancer and diabetes behavior screening surveys, and 22% reported donating biospecimens such as breast tissue, prostate tissue, blood and urine for research.
Table 1:
Participant Demographics
| Phase 2 (N=52) | Phase 3 (N=100) | Total (N=152) | |
|---|---|---|---|
| Characteristic | N (%) | N (%) | N (%) |
| Site | |||
| NU | 12 (23.1) | 20 (20.0) | 32 (21.1) |
| FCCC | 10 (19.2) | 18 (18.0) | 28 (18.4) |
| Mayo | 10 (19.2) | 0 (0.0) | 10 (6.6) |
| OSU | 10 (19.2) | 21 (21.0) | 31 (20.4) |
| RPCI | 10 (19.2) | 41 (41.0) | 51 (33.6) |
| Age (yrs) | |||
| 21–30 | 3(5.8) | 19 (19.0) | 22 (14.5) |
| 31–40 | 9 (17.3) | 13 (13.0) | 22 (14.5) |
| 41–50 | 13 (25.0) | 12 (12.0) | 25 (16.4) |
| 51–60 | 9 (17.3) | 23 (23.0) | 32 (21.1) |
| 61+ | 18 (34.6) | 33 (33.0) | 51 (33.6) |
| Sex | |||
| Male | 23 (44.2) | 26 (26.0) | 49 (32.2) |
| Female | 29 (55.8) | 74 (74.0) | 103 (67.8) |
|
Ethnicity (Hispanic) |
|||
| Yes | 11 (21.2) | 24 (24.0) | 35 (23.0) |
| No | 40 (76.9) | 76 (76.0) | 116 (76.3) |
| NR | 1 (1.9) | 0 (0.0) | 1 (0.7) |
|
Region (Appalachia) | |||
| Yes | 5(9.6) | 16 (16.0) | 21 (13.8) |
| Race | |||
| White | 20 (38.5) | 44 (44.0) | 64 (42.1) |
| Asian | 0 (0.0) | 0 (0.0) | 0 (0.0) |
| African American | 16 (30.8) | 30 (30.0) | 46 (30.3) |
| Native American | 9 (17.3) | 21 (21.0) | 30 (19.7) |
| Other | 4(7.7) | 5 (5.0) | 9(5.9) |
| NR | 3(5.8) | 0 (0.0) | 3 (2.0) |
| Education | |||
| <HS | 12 (23.1) | 13 (13.0) | 25 (16.5) |
| HS | 27 (51.9) | 18 (18.0) | 45 (29.6) |
| Some college | 13 (25.0) | 39 (39.0) | 52 (34.2) |
| College graduate | 0 (0.0) | 18 (18.0) | 18 (11.8) |
| Graduate school | 0 (0.0) | 12 (12.0) | 12 (7.9) |
| Born in the US | |||
| Yes | 41 (78.8) | 88 (88.0) | 129 (84.9) |
| No | 10 (19.2) | 12 (12.0) | 22 (14.5) |
| Did not answer | 1 (1.9) | 0 (0.0) | 1 (0.7) |
| Personal cancer diagnosis | |||
| Yes | 25 (48.1) | 40 (40.0) | 65 (42.8) |
| No | 27 (51.9) | 60 (60.0) | 87 (57.2) |
| Family cancer diagnosis | |||
| Yes | 41 (78.9) | 92 (92.0) | 133 (87.5) |
| No | 9 (17.3) | 8 (8.0) | 17 (11.2) |
| Unsure | 1 (1.9) | 0 (0.0) | 1 (0.7) |
| NR | 1 (1.9) | 0 (0.0) | 1 (0.7) |
| Previous research participation | |||
| Yes | 16 (30.8) | 49 (49.0) | 65 (42.8) |
| No | 35 (67.3) | 51 (51.0) | 86 (56.6) |
| Unsure | 1 (1.9) | 0 (0.0) | 1 (0.7) |
| Previous biobanking research participation | |||
| Yes | 9 (17.3) | 24 (24.0) | 33 (21.7) |
| No | 41 (78.8) | 76 (76.0) | 117 (77.0) |
| NA/Refused | 2(3.8) | 0 (0.0) | 2(1.3) |
Study results are chronologically reported to document the iterative development process to the first standardized edition of the RLS tool. Phase 2 data were collected across the themes 1) overall feedback and layout, 2) comprehensibility and word use, and 3) usability of the tool.
Phase 2 Participant Feedback
Theme 1. Overall Feedback and Layout.
Participants valued the use of pictures to pace the presentation and thereby prevent information overload. Participants indicated that the language was clear and respectful, and that the content addressed clinical research questions logically.
Theme 2. Comprehensibility and Word Use.
Participants reported learning new information about clinical research, and valued this opportunity to learn at their own pace. Participants also valued that the cards encouraged active learning by prompting questions. Participants reported learning that healthy people could participate in research and that informed consent is wholly voluntary. Despite the use of plain language in the RLS tool, participants still required additional information to gain a fundamental understanding of terms and concepts associated with standard of care, informed consent, and bio-banking.
Theme 3. Usability of the tool.
Participants confirmed the cards were a helpful resource for individuals invited to participate in research, noting that the cards clarified the consent process, addressed myths, and supported informed decision-making. Participants also agreed that using the RLS tool would make individuals more likely to participate in research and saw their value in prompting important questions that a participant might not ask without this tool.
Phase 2 Recruiter Feedback
Recruiters generally viewed the format and the concise and simple language as appropriate for participants to encourage active learning through question prompts. Recruiters deemed the cards a valuable educational tool for diverse populations during recruitment. Regarding comprehensibility and word use, recruiters identified terms requiring further clarification. Some of these words directly related to information in the “Who has access to my information?” and “How is my identity protected?” cards. For example, recruiters suggested that the terms “personal identifiers” and “coded” be removed and cards revised to read, “information that can identify you will be removed and replaced with a special code”. They also recommended the term “privacy” replace “confidentiality”. They further recommended revising the “Standard of care” card to explain that “most doctors would agree patients should receive current clinical care (standard of care)” and to clarify that “some research studies focus on improving the current standard of care.” fn addition, recruiters viewed the cards as a recruiter training tool that could guide them in explaining research concepts in simple, comprehensible language to potential participants.
Summary of changes made after Phase 2
Based on participant and recruiter feedback, researchers made several changes to the RLS tool. New images were added to enhance diversity, and comprehension of research concepts presented was improved by addressing areas where clarity was insufficient, (e.g., the meaning of informed consent and participants’ right to withdraw from a study at any time). New cards were subsequently created to address those needs. One encouraged participants to inquire about the benefits and potential harms of participation in a study. Another indicated that if a potential participant decided to participate in a study, research staff would review study-specific details and outline immediate next steps. As an example of card revision, Figure 1 shows the text in the biobanking card “How long will my samples be stored?” before (Panel A) and after (Panel B) revisions to the card at the end of Phase 2.
Figure 1.
Example of the Text in a Biobanking Card Before (A) and After (B) Phase 2 Card Revision
Phase 3 Post-test Survey
Phase 3 data related to format and content, comprehension, attitudes towards research and delivery methods preferences.
Format and content.
In the post-test survey, nearly all (98%) of participants agreed the information on the cards helped them understand clinical research; however 11% of participants still reported the information on the card was hard to understand (Table 2). According to 82% of participants, pictures helped them understand the information, indicating the value of visual aids in the tool. A fifth of participants (20%) felt that it took them too long to read the cards (Table 2).
Table 2.
Format and Content (Phase 3) N(%)
| Total | Site A | Site B | Site C | Site D | Site E | |
|---|---|---|---|---|---|---|
| N | 100 | 20 | 21 | 18 | 21 | 20 |
| The cards looked professional. | ||||||
| Agree | 96 (96.0) | 20 (100.0) | 20 (95.2) | 18 (100.0) | 20 (95.2) | 18 (90.0) |
| Disagree | 0 (0.0) | 0 (0.0) | 0 (0.0) | 0 (0.0) | 0 (0.0) | 0 (0.0) |
| The pictures on the cards helped me understand the information presented. | ||||||
| Agree | 82 (82.0) | 16 (80.0) | 15 (71.4) | 17 (94.4) | 18 (85.7) | 16 (80.0) |
| Disagree | 8 (8.0) | 1 (5.0) | 5 (23.8) | 0 (0.0) | 0 (0.0) | 2 (10.0) |
| The information on the cards was important; it helped me understand clinical research. | ||||||
| Agree | 98 (98.0) | 20 (100.0) | 21 (100.0) | 17 (94.4) | 20 (95.2) | 20 (100.0) |
| Disagree | 0 (0.0) | 0 (0.0) | 0 (0.0) | 0 (0.0) | 0 (0.0) | 0 (0.0) |
| It look too long to read the cards. | ||||||
| Agree | 20 (20.0) | 2 (10.0) | 0 (0.0) | 7 (38.9) | 7 (33.3) | 4 (20.0) |
| Disagree | 75 (75.0) | 18 (90.0) | 20 (95.2) | 11 (61.1) | 12 (57.1) | 14 (70.0) |
| The information on the cards was hard for me to understand. | ||||||
| Agree | 11 (11.0) | 1 (5.0) | 0 (0.0) | 3 (16.7) | 5 (20.0) | 2 (10.0) |
| Disagree | 89 (89.0) | 19 (95.0) | 21 (100.0) | 15 (83.3) | 16 (76.2) | 18 (90.0) |
| It was easy for me to learn from reading this cards. | ||||||
| Agree | 98 (98.0) | 19 (95.0) | 21 (100.0) | 18 (100.0) | 20 (95.2) | 20 (100.0) |
| Disagree | 2 (2.0) | 1 (5.0) | 0 (0.0) | 0 (0.0) | 1 (4.8) | 0 (0.0) |
| When medical words were used in the cards, they were explained. | ||||||
| Agree | 93 (93.0) | 17 (85.0) | 21 (100.0) | 18 (100.0) | 18 (85.7) | 19 (95.0) |
| Disagree | 4 (4.0) | 1 (5.0) | 0 (0.0) | 0 (0.0) | 2 (9.5) | 1 (5.0) |
Note: The menu of choices was Agree, Disagree, and Not Sure. Shown here are values for the Agree and Disagree responses. Values for Not Sure and Did Not Answer are not shown.
Knowledge/comprehension.
Participants completed a series of knowledge questions after reviewing the cards, which illuminated some important learning gaps. Notably, a small percentage of participants believed that participating in research would cure their disease (6%), 20% did not know that a research participant can withdraw from a study at any time, and only 52% affirmed that in a randomized study there is a chance of not being randomized to the treatment group.
Attitudes towards research.
Following their review of the cards, 82% of participants reported that they would be likely to participate in research. Among the participants randomized to review the biobanking set of cards, 76% reported they would be likely to donate a specimen.
Delivery Preferences and Usability.
While 61% of participants indicated they preferred the cards be mailed to them in advance, 34% preferred to review the cards at their clinic appointment (Table 3). Comparable percentages of participants were interested in reviewing the cards in print (43%) versus video/digitally format (44%). Over three quarters (79%) of participants reported they anticipated it could take up to 30 minutes to review the cards while 20% of participants indicated it could take more than 30 minutes. Among the racial/cthnic groups, 46% of the Hispanic/Latino participants indicated that it would take them more than 30 minutes to review the cards.
Table 3.
. Preference & Usability N (%)
| Total | Site A | Site B | Site C | Site D | Site E | |
|---|---|---|---|---|---|---|
| N | 100 | 20 | 21 | 18 | 21 | 20 |
| I would like to review these by myself. | ||||||
| Agree | 65 (65.0) | 11 (55.0) | 12 (57.1) | 16 (88.9) | 13 (61.9) | 13 (65.0) |
| Disagree | 11 (11.0) | 3 (15.0) | 2 (9.5) | 1 (5.6) | 4 (19.0) | 1 (5.0) |
| I would like to review the cards with family or a friend. | ||||||
| Agree | 59 (59.0) | 13 (65.0) | 12 (57.1) | 17 (94.4) | 8 (38.1) | 9 (45.0) |
| Disagree | 14 (14.0) | 2 (10.0) | 6 (28.6) | 1 (5.6) | 5 (23.8) | 0 (0.0) |
| I would like to review these cards with research staff. | ||||||
| Agree | 72 (72.0) | 14 (70.0) | 13 (61.9) | 16 (88.9) | 16 (76.2) | 13 (65.0) |
| Disagree | 7 (7.0) | 1 (5.0) | 3 (14.3) | 2 (11.1) | 1 (4.8) | 0 (0.0) |
| I would like to review these cards with my doctor or nurse. | ||||||
| Agree | 64 (64.0) | 12 (60.0) | 17 (81.0) | 16 (88.9) | 12 (57.1) | 7 (35.0) |
| Disagree | 11 (11.0) | 4 (20.0) | 2 (9.5) | 2 (11.1) | 3 (14.3) | 0 (0.0) |
| I would like to have these read to me. | ||||||
| Agree | 17 (17.0) | 1 (5.0) | 1 (4.8) | 6 (33.4) | 4 (19.0) | 5 (25.0) |
| Disagree | 70 (70.0) | 17 (85.0) | 17 (81.0) | 11 (61.1) | 16 (76.2) | 9 (45.0) |
| I would like to have the cards mailed to me. | ||||||
| Agree | 44 (44.0) | 2 (10.0) | 13 (61.9) | 12 (66.7) | 12 (57.1) | 5 (25.0) |
| Disagree | 28 (28.0) | 11 (55.0) | 4 (19.0) | 5 (27.8) | 3 (14.3) | 5 (25.0) |
| I would like to review the cards on a print format (e.g. booklet). | ||||||
| Agree | 43 (43.0) | 9 (45.0) | 6 (28.6) | 10 (55.6) | 10 (47.6) | 8 (40.0) |
| Disagree | 26 (26.0) | 6 (30.0) | 6 (28.6) | 6 (33.3) | 3 (14.3) | 5 (25.0) |
| I would like to review the cards in a video format. | ||||||
| Agree | 44 (44.0) | 8 (40.0) | 6 (28.6) | 9 (50.0) | 11 (52.4) | 10 (50.0) |
| Disagree | 30 (30.0) | 8 (40.0) | 9 (42.9) | 7 (38.9) | 2 (9.5) | 4 (20.0) |
| The video format(s) that I would like to review: | ||||||
| Computer | 49 (49.5) | 8 (40.0) | 14 (66.7) | 3 (16.7) | 15 (75.0) | 9 (45.0) |
| TV | 38 (38.4) | 5 (25.0) | 10 (47.6) | 2 (11.1) | 15 (75.0) | 6 (30.0) |
| DVD | 37 (37.4) | 3 (15.0) | 8 (38.1) | 6 (33.3) | 14 (70.0) | 6 (30.0) |
| iPad or tablet | 36 (36.4) | 5 (25.0) | 8 (38.1) | 1 (5.6) | 12 (60.0) | 10 (50.0) |
| Smart phone | 32 (32.3) | 8 (40.0) | 6 (28.6) | 0 (0.0) | 7 (35.0) | 11 (55.0) |
| If you were asked to participate in a research study, when would you want the cards given to you review? | ||||||
| -Mailed prior to appointment | 61 (61.0) | 12 (60.0) | 15 (71.4) | 11 (61.1) | 11 (52.4) | 12 (60.0) |
| -At the office when I arrive for my | 34 (34.0) | 5 (25.0) | 6 (28.6) | 7 (38.9) | 8 (38.1) | 8 (40.0) |
| appointment -After appointment (take them home) | 4 (4.0) | 2 (10.0) | 0 (0.0) | 0 (0.0) | 2 (9.5) | 0 (0.0) |
| How much time do you think patients would need to review the cards? | ||||||
| −0–15 minutes | 37 (37.0) | 11 (55.0) | 6 (28.6) | 1 (5.6) | 4 (19.0) | 15 (75.0) |
| −16–30 minutes | 42 (42.0) | 8 (40.0) | 12 (57.1) | 6 (33.3) | 11 (52.4) | 5 (25.0) |
| >30 minutes | 20 (20.0) | 0 (0.0) | 3 (14.3) | 11 (61.1) | 6 (30.0) | 0 (0.0) |
| NR | 1 (1.0) | 1 (5.0) | 0 (0.0) | 0 (0.0) | 0 (0.0) | 0 (0.0) |
Note: The menu of options for agreement or disagreement was Strongly Agree, Somewhat Agree, Neutral, Somewhat Disagree, and Strongly Disagree. In this table, Agree is the sum of Strongly Agree and Somewhat Agree, Disagree is the sum of Strongly Disagree and Somewhat Disagree, and Neutral and Did Not Answer are not shown.
DISCUSSION
The multi-site development of the RLS tool reported here was based on user-centered design and entailed a step-by-step learning approach rather than a prospective study design. Specifically, feedback from end users (potential clinical research participants representative of diverse underrepresented populations and clinical research recruiters experienced in working with diverse underrepresented groups) provided social and cultural context that informed the iterative study design in-process. The approach applied here lends further support to the pragmatic value of making adaptations and changes to study design based on context over the course of a study in implementation science [11]. Our approach provides a framework for analogous initiatives in health equity research, e.g., creation of patient education tools, as well as continual improvement of the RLS through testing in the field and further adaptation to address learning gaps.
Participants and recmiters noted the value of the RLS tool in improving potential participants’ understanding of research concepts, thus fostering informed participation in clinical trials and bio-banking studies. Participants confirmed that the RLS tool supports decision making by answering general questions, addressing concerns, and debunking myths related to clinical research. Training recruiters in communication has been identified as an essential skill to improve study recruitment and retention [1], Recruiters indicated this RLS tool could serve as a training tool for recmiters as it could explain research concepts in simple language and serve as a reminder to ask potential participants if they had any questions. Participants appreciated the question-answer format, because it addressed clinical research in a logical way and encouraged them to ask questions of research staff. These findings are consistent with general guidelines aimed to increase health literacy [16],
Because the RLS tool has utility in making comprehensible specific terms and concepts in clinical research that are difficult for participants to understand [8] and that clinicians and recruitment staff often have difficulty explaining to potential participants [14], the RLS tool could provide the additional support needed for research participants to understand clinical research information, a gap in the informed consent process identified in a systematic review [8]. The depth of this gap is conveyed by the estimate that research participants only understand between 30% - 81% of the information presented to them, and these numbers vary with respect to education and literacy levels [4, 8]. In this study, participants recognized that they had gained knowledge about clinical research from the RLS tool, although they still had a substantial learning gap after reading the cards.
Although there was no pretest comparator because the primary focus was on gaining the trust of participants to foster engagement and active learning, the measurement of participants’ comprehension of clinical research terms and concepts post-test identified learning gaps where further development of the tool is warranted. For example, one such area of opportunity is to address the rather large percentage of users of the tool who did not fully understand their rights to withdraw from a study. This gap identification is illustrative of how the RLS is intended to evolve by addressing areas of improvement needed that are identified by participant feedback.
Recruiters indicated that the bio-banking component was a strength of this RLS tool. Participants needed reassurance that bio-specimens are stored in a secure private place, and that they could request that specimens be destroyed. The RLS tool addressed these and other concerns about bio-banking research [19], After reviewing the RLS tool, 76% of participants reported a willingness to donate specimens for bio-banking research. This number suggests room for improvement, underscoring the importance of further addressing known barriers to bio-specimen donation by minorities, such as medical mistrust and suspicion of exploitation [6], by reinforcing participants’ understanding of their rights in bio-banking research participation and the societal benefits of participation. Understanding the potentially multifactorial basis for participants’ hesitance to participate in bio-banking research may identity learning gaps that the RLS can be modified to address.
Based on feedback from participants, several key overall changes were made during Phase 2 to address concerns and further tailor this tool for diverse racial and ethnic groups. We included new pictures/images that were more representative of African American and Native American populations. In addition, we created a “Take home message” card that specified that many tribes have governing bodies that provide oversight of studies including bio- specimen collection.
Large-scale efforts to advance diversity in clinical research include the All of Us Research Program, a Precision Medicine Initiative of the National Institutes of Health, that aims to accelerate research and thereby improve health by enrolling and collecting data on a diverse cohort of at least one million volunteers (participants) representative of persons from all walks of life in the US, including racial/ ethnic minorities and other underrepresented groups. The All of Us Research Program represents a large scale effort underway to overcome the well-documented underrepresentation of racial/ethnic minority and other marginalized groups in clinical research in the US [3, 4], We anticipate that the RLS tool may be used by recruiters to improve communication with potential research participants for the All of Us Research program, and may find utility in community education and community-based participatory research.
In addition, the RLS tool may prove useful along the various recruitment points including community outreach efforts as well as within the healthcare and community settings to support continuity of messaging. For example, partnerships with patient resource and public libraries may broaden the reach of this RLS tool. Integration of the RLS tool within the work of community health workers and patient navigators, who often have personal or cultural ties to underserved communities, are also promising strategies for facilitating open communication about research participation. While lay patient navigators have already demonstrated efficacy in increasing minority patient recruitment to cancer clinical trials [9], additional research is warranted to examine the potential roles of patient navigators in participant recruitment for research and the potential role of the RLS tool in this process.
Recruiters acknowledged this RLS tool was a valuable training resource that can support clinical research recruitment efforts with diverse populations [5, 13]. This study complements qualitative recruitment studies with recruiters and diverse populations that showed the importance of effective verbal communication behaviors specifically translating and simplifying study information, adapting conversation style, using examples to illustrate concepts, incorporating risks and benefits, and encouraging participants [17, 18].
Participants showed a preference for a digital/video version of the RLS tool, and for reviewing the cards independently. However, the protracted time period required in some cases to review the cards (up to 45 minutes) suggest the benefit of having the participant review the cards with a recruiter or a trained lay patient navigator, whose cultural affinities [9] could potentially help facilitate comprehension and enhance motivation through interpersonal interactions. Development of a RLS user guide for recruiters could further enhance their role in making the tool fully accessible to participants. In addition, given the diverse modes of preferred delivery, wide use of the tool may best be effected by offering an array of delivery modes that are compatible with adult learning theories that people learn in different ways. Further development of the RLS tool through pre- and post-test comprehension assessments may also guide best practices regarding RLS delivery.
This study had several limitations. Foremost, the purposive sample of recruiters and participants limits the generalizability of the findings. In particular, the decision to solicit feedback from participants at academic medical centers may have resulted in recruitment of participants with higher education level than anticipated. Second, the limited sample size did not allow for disentangling feedback provided by those with prior research participation experience. Third, the tool is currently only available in English, which could have affected the study population. Future efforts to translate this tool and use it with non-English speaking populations are underway.
Conclusions
In tliis mixed-methods study to develop a Clinical Research Literacy Support (RLS) tool, researchers obtained invaluable feedback about the communication tool from potential end users and applied findings toward iterative development. The RLS tool was clearly useful to both participants and researchers at initiating, guiding, and fostering communication between researchers and participants, but key challenges remain - including optimizing the delivery mode to realize comprehensive learning in an acceptable timeframe. Overcoming communication barriers could improve future clinical research recruitment and retention efforts and help increase diverse participation in clinical trials and biobanking studies.
Acknowledgments
Financial Support. This work was supported by the National Cancer Institute’s Center to Reduce Cancer Health Disparities under Patient Navigation Research Grants U01CA116874, 340CA1168875–04-53, and 5U01 CA116875– 05S4, the National Institutes of Health under grants U54CA153605 (JSK), U54CA203000 (MAS), and P20CA165592 (NU), and the National Library of Medicine under Grant G08LM012688–01 (MAS). In addition, tins work used shared resources supported by the National Cancer Institute under Roswell Park Comprehensive Cancer Center Support Grant P30CA016056 and the National Institutes of Health under the National Center for Advancing Translational Sciences Grant UL1TR000150.
Footnotes
Disclosure Statement. The authors declare that they have no conflicts of interests. Melissa Simon is a member of the United States Preventive Services Task Force (USPSTF). This article does not necessarily represent the views and policies of the USPSTF.
Readers may obtain a copy of the Research Literacy Support Tool described in the report by contacting the corresponding author.
REFERENCES
- 1.Bower P, Brueton V, Gamble C, Treweek S, Smith CT, Young B, and Williamson P. 2014. Interventions to improve recruitment and retention in clinical trials: a survey and workshop to assess current practice and future priorities. Trials 15:399. doi: 10.1186/1745-6215-15-399. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Breese PE, Burman WJ, Goldberg S, and Weis SE. 2007. Education level, primary language, and comprehension of the informed consent process. J Emplr Res Hum Res Ethics 2 (4):69–79. doi: 10.1525/jer.2007.2.4.69. [DOI] [PubMed] [Google Scholar]
- 3.Chen MS Jr., Lara PN, Dang JH, Paterniti DA, and Kelly K. 2014. Twenty years post-NIH Revitalization Act: enhancing minority participation in clinical trials (EMPaCT): laying the groundwork for improving minority clinical trial accrual: renewing the case for enhancing minority participation in cancer clinical trials. Cancer 120 Suppl 7:1091–1096. doi: 10.1002/cncr.28575. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Cortes DE, Drainoni ML, Henault LE, and Paasche-Orlow ΜK. 2010. How to achieve informed consent for research from Spanish-speaking individuals with low literacy: a qualitative report. J Health Commun 15 Suppl 2:172–182. doi: 10.1080/10810730.2010.499990. [DOI] [PubMed] [Google Scholar]
- 5.Daley CM, James AS, Barnoskie RS, Segraves M, Schupbach R, and Choi WS. 2006. “Tobacco has a purpose, not just a past”: Feasibility of developing a culturally appropriate smoking cessation program for a pan-tribal native population. Med Anthropol Q 20 (4):421–440. [DOI] [PubMed] [Google Scholar]
- 6.Dang JΗ, Rodriguez EM, Luque JS, Erwin DO, Meade CD, and Chen MS Jr. 2014. Engaging diverse populations about biospecimen donation for cancer research. J Community Genet 5 (4):313–327. doi: 10.1007/sl2687-014-0186-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Federal Plain Language Guidelines. https://plainlanguage.gov/media/FederalPLGuidelines.pdf. Accessed April 16, 2018.
- 8.Flory J, and Emanuel E. 2004. Interventions to improve research participants’ understanding in informed consent for research: a systematic review. Jama 292 (13):1593–1601. doi: 10.1001/jama.292.13.1593. [DOI] [PubMed] [Google Scholar]
- 9.Fouad MN, Acemgil A, Bae S, Forero A, Lisovicz N, Martin MY, Oates GR, Partridge EE, and Vickers SM. 2016. Patient Navigation As a Model to Increase Participation of African Americans in Cancer Clinical Trials. J Oncol Pract 12 (6):556–563. doi: 10.1200/JOP.2015.008946. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Frieden TR, Control Centers for Disease, and Prevention. 2013. CDC Health Disparities and Inequalities Report - United States, 2013. Foreword. MMWR Suppl 62 (3):1–2. [PubMed] [Google Scholar]
- 11.Gaglio B, Phillips SM, Heurtin-Roberts S, Sanchez MA, and Glasgow RE. 2014. How pragmatic is it? Lessons learned using PRECIS and RE-AIM for determining pragmatic characteristics of research. Implement Sci 9:96. doi: 10.1186/s13012-014-0096-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Haring RC, Henry WA, Hudson M, Rodriguez EM, and Taualii M. 2018. Views on clinical trial recruitment, biospecimen collection, and cancer research: population science from landscapes of the Haudenosaunee (People of the Longhouse). J Cancer Educ 33 (1):44–51. doi: 10.1007/s13187-016-1067-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Kreuter MW, Lukwago SN, Bucholtz RD, Clark EM, and Sanders-Thompson V. 2003. Achieving cultural appropriateness in health promotion programs: targeted and tailored approaches. Health Educ Behav 30 (2):133–146. doi: 10.1177/1090198102251021. [DOI] [PubMed] [Google Scholar]
- 14.Loh WY, Butow PN, Brown RF, and Boyle F. 2002. Ethical communication in clinical trials. Issues faced by data managers in obtaining informed consent. Cancer 95 (11):2414–2421. doi: 10.1002/cncr.10994. [DOI] [PubMed] [Google Scholar]
- 15.Lorenzen B, Melby CE, and Earles B. 2008. Using principles of health literacy to enhance the informed consent process. Aorn j 88 (1):23–29. doi: 10.1016/j.aorn.2008.03.001. [DOI] [PubMed] [Google Scholar]
- 16.McCray AT 2005. Promoting health literacy. J Am Med Inform Assoc 12 (2):152–163. doi: 10.1197/jamia.M1687. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Morgan SE, Mouton A, Occa A, and Potter J. 2016. Clinical Trial and Research Study Recruiters’ Verbal Communication Behaviors. J Health Commun 21 (7):765–772. doi: 10.1080/10810730.2016.1157654. [DOI] [PubMed] [Google Scholar]
- 18.Morgan SE, Occa A, Potter J, Mouton A, and Peter ME. 2017. “You Need to Be a Good Listener”: Recruiters’ Use of Relational Communication Behaviors to Enhance Clinical Trial and Research Study Accrual. J Health Commun 22 (2):95–101. doi: 10.1080/10810730.2016.1256356. [DOI] [PubMed] [Google Scholar]
- 19.Page SA, Manhas KP, and Muruve DA. 2016. A survey of patient perspectives on the research use of health information and biospecimens. BMC Med Ethics 17 (1):48. doi: 10.1186/s12910-016-0130-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Simon MA, de la Riva EE, Bergan R, Norbeck C, McKoy JM, Kulesza P, Dong X, Schink J, and Fleisher L. 2014. Improving diversity in cancer research trials: the story of the Cancer Disparities Research Network. J Cancer Educ 29 (2):366–374. doi: 10.1007/s13187-014-0617-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Torres S, de la Riva EE, Tom LS, Clayman ML, Taylor C, Dong X, and Simon MA. 2015. The Development of a Communication Tool to Facilitate the Cancer Trial Recruitment Process and Increase Research Literacy among Underrepresented Populations. J Cancer Educ 30 (4):792–798. doi: 10.1007/s13187-015-0818-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Treweek S, Lockhart P, Pitkethly M, Cook JA, Kjeldstrom M, Johansen M, Taskila TK et al. . 2013. Methods to improve recruitment to randomised controlled trials: Cochrane systematic review and meta-analysis. BMJ Open 3 (2). doi: 10.1136/bmjopen-2012-002360. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Vickers SM, and Fouad MN. 2014. An overview of EMPaCT and fundamental issues affecting minority participation in cancer clinical trials: enhancing minority participation in clinical trials (EMPaCT): laying the groundwork for improving minority clinical trial accrual. Cancer 120 Suppl 7:1087–1090. doi: 10.1002/cncr.28569. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Wells KJ, Arevalo M, Meade CD, Gwede CK, Quinn GP, Luque JS, San Miguel G et al. . 2014. Development and validation of the biobanking attitudes and knowledge survey (BANKS). Cancer Epidemiol Biomarkers Prev 23 (3):374–382. doi: 10.1158/1055-9965.EPI-13-0746. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Yee LM, McGuire JM, Taylor SM, Niznik CM, and Simon MA. 2015. “I Was Tired of All the Sticking and Poking”: Identifying Barriers to Diabetes Self-Care Among Low-Income Pregnant Women. J Health Care Poor Underserved 26 (3):926–940. doi: 10.1353/hpu.2015.0073. [DOI] [PubMed] [Google Scholar]