Abstract
The purpose of this pilot study was to determine if a cancer research study website increased comprehension among patients and caregivers and if website evaluations differed across patient and caregiver groups. Participants (N = 200) were cancer patients and caregivers living in the USA. Comprehension was determined by the number of correct responses to a series of questions about key characteristics of cancer research studies that are frequently unknown or misinterpreted by patients and/or caregivers. Quantitative and qualitative analyses were conducted to determine participant evaluations across four domains: perceived website credibility, perceived website attractiveness, perceived information effectiveness, and perceived information clarity. Patients and caregivers perceived the website as highly credible and attractive and the information as both easy to understand and moderately effective in helping them make decisions about CCTs. Qualitative feedback underscores the importance of testimonials to website credibility. However, the range in the number of correct responses of certain items across participants coupled with discrepancies in comprehension between patients and caregivers suggests the need for stronger mechanisms evaluating knowledge outcomes.
Keywords: Cancer patients and caregivers, Online treatment decision-aids, Cancer clinical trial comprehension, Communication interventions
Introduction
Cancer clinical trials (CCT) represent the state of the science in cancer treatment and are essential for discovering new treatments [1]. Yet, limited patient awareness and understanding of CCTs (e.g., studies, trial procedures) result in many CCTs failing to recruit an adequate number of patients [2]. In order to overcome these barriers, many studies have examined the benefits of decision aids, evidence-based tools designed to educate patients and to help them make informed decisions about treatment [3]. Typically, decision aids for CCTs focus on a single treatment option (e.g., chemotherapy, surgery, or a single trial) and rarely include information for caregivers (i.e., non-clinical persons involved in decision-making, such as family members) [4, 5]. The potential role of web-based content in CCT decision-making is also not well understood. The Internet is widely used by individuals to identify second opinions or alternative perspectives about treatment [6] and is a preferred source of information among cancer patients [7] and cancer survivors [8]. Web-based decision-aid tools (e.g., digital, interactive), in particular, increase patient knowledge and positively influence their decision-making satisfaction [3].
Creating an online CCT decision aid requires the same considerations as traditional decisions aids (e.g., clarity, effectiveness), as well as factors unique to processing online health information (e.g., website structural features). For instance, developing health messages that are clear and easy to understand [9], as well as content that is perceived as credible (e.g., accurate) [10] and effective at inciting changes in attitudes, behaviors, or intentions among recipients [11], are important features of all decision aids. On the Internet, inclusion of certain message formats (e.g., testimonials) is positively associated with user perceptions of credibility [12], and individuals perceive health information presented on websites as more credible than health information presented on blogs [13]. Presentation of online health information, specifically, message clarity also influences participant comprehension and evaluations of online information about cancer. In one study, comprehension and accurately recalling web-based information about lung cancer was greatest among individuals who received an informal description of the topic, in language that was free of complex medical jargon [14]. Online users also respond favorably (e.g., are highly satisfied) to personalized health websites (i.e., websites with health information about a specific illness or treatment, such as cancer or CCTs) and perceive websites that incorporate text and visuals (e.g., pictures) to explain cancer as more attractive than websites with text only [12, 14]. Other characteristics, including age, Internet use, inclusion of external web links, and website headings influence evaluations of online health information [12, 14]; however, it remains unknown how these factors influence patient and caregiver’s evaluation and comprehension of a personalized CCT decision-aid website with information about CCTs as treatment.
Despite their potential to increase patient and caregiver knowledge about cancer treatment, cancer decision-making tools have focused primarily on patients and largely overlooked caregivers, despite their role in assisting patients with their treatment decisions [4, 5]. Ignoring caregivers as critical audiences involved in decision-making and failing to account for recipient preferences for receiving information about treatment (i.e., channel, format) places excess pressure on physicians by relying on them to present and discuss alternate treatment options not included in decision aids. Moreover, with few exceptions [8], evaluations of decision-aid tools have failed to account for extant factors, such as patient literacy or familiarity with the Internet, or adequately consider how those factors influence recipients’ evaluations in the cancer decision-making context. Thus, we developed a cancer clinical study decision-aid website tool for cancer patients and caregivers. Our goal of this pilot study was to determine if the cancer research study website increased comprehension of CCTs among patients and caregivers and if website evaluations differed between cancer patients and caregivers.
Methods
Participants and Recruitment
A cross-sectional survey was approved by The University of Florida Institutional Review Board, with participants recruited through Qualtrics Panels, a proprietary opt-in online panel company. Two hundred and ten participants who were over the age of 18, able to read and write in English, and self-reported as either having been personally diagnosed with cancer (n = 105) or having a loved one (i.e., family member or friend) who relied on them to help make decisions about their cancer (n = 105) were recruited to participate in the study. Participants who indicated they did not seek treatment for cancer (n = 4) and those with incomplete data (n = 6) were removed prior to analyses, resulting in a final sample of 200. See Table 1 for participant demographic information.
Table 1.
Sociodemographic characteristics of caregivers and patients
| Participant characteristics | Patient (n = 100) | Caregiver (n = 100) | ||
|---|---|---|---|---|
| N = 200 (%)a | n (%) | n (%) | χ2 | |
| Age, mean (SD) | 45.06 (14.46) | 48.29 (13.59) | 41.83 (14.66) | 10.45***c |
| Sexb | .11 | |||
| Male | 48 (24.0) | 25 (25.0) | 23 (23.0) | |
| Female | 152 (76.0) | 75 (75.0) | 77 (77.0) | |
| Race | .03 | |||
| White | 155 (77.5) | 78 (78.0) | 77 (77.0) | |
| Other | 45 (22.5) | 22 (22.0) | 23 (23.0) | |
| Education | 5.81* | |||
| No college education | 107 (53.5) | 45 (45.0) | 62 (62.0) | |
| College education | 93 (46.5) | 55 (55.0) | 38 (38.0) | |
| Income | .00 | |||
| Less than or equal to 60 k | 96 (48.0) | 48 (48.0) | 48 (48.0) | |
| Greater than $60 k | 104 (52.0) | 52 (52.0) | 52 (52.0) | |
| Treatment status | 4.00* | |||
| Completed treatment | 110 (55.0) | 62 (62.0) | 48 (48.0) | |
| Currently being treated | 90 (45.0) | 38 (38.0) | 52 (52.0) | |
| Cancer type | 19.60** | |||
| Breast | 69 (34.5) | 39 (39.0) | 30 (30.0) | |
| Colorectal | 10 (5.0) | 3 (3.0) | 7 (7.0) | |
| Lung or bronchus | 19 (9.5) | 7 (7.0) | 12 (12.0) | |
| Prostate | 16 (8.0) | 8 (8.0) | 8 (8.0) | |
| Gynecological | 21 (10.5) | 18 (18.0) | 3 (3.0) | |
| Melanoma | 18 (9.0) | 9 (9.0) | 9 (9.0) | |
| Other | 47 (23.5) | 16 (16.0) | 31 (31.0) | |
| Asked CCT | .34 | |||
| No | 163 (81.5) | 80 (80.0) | 83 (83.0) | |
| Yes | 37 (18.5) | 20 (20.0) | 17 (17.0) | |
| Participated CCT | .16 | |||
| No | 170 (85.0) | 84 (84.0) | 86 (86.0) | |
| Yes | 30 (15.0) | 16 (16.0) | 14 (14.0) | |
| eHEALS, mean (SD) | 3.84 (0.84) | 3.85 (0.80) | 3.83 (0.88) | .03c |
| Time on website (mins) | 3.20 (2.85) | 3.41 (2.63) | 2.99 (3.06) | 1.12c |
| Page clicks | 2.73 (2.21) | 2.85 (2.38) | 2.61 (2.03) | .59c |
Percentages totaled within column may exceed 100% due to rounding
Participants who failed to respond were removed (n = 1)
F statistic derived from a one-way analysis of variance
eHEALS, electronic health literacy scale; Asked CCT, whether the patient/patient of the caregiver participating in the study had ever been asked to partcipate in a cancer clinical trial; Participated CCT, whether the patient/patient of the caregiver participating in the study had ever participated in a cancer clinical trial;
p < 0.05;
p < 0.01;
p < .001
Website Development
Content for the cancer clinical study website was derived from two main sources: a clinical research educational booklet that was developed by the senior author and tested among cancer patients and caregivers, and information generated from a series of community panels about CCTs, including testimonials from patients, healthcare providers, physicians, and clinical trial coordinators and video excerpts from the question and answer sessions. Content was developed by the research team to ensure accuracy and readability (i.e., team members transcribed videos and excerpts included on the website) and optimized through formatting (e.g., headings), the inclusion of images, and through feedback from experts in visual design. We also linked to and referenced content from other clinical research websites, such as clinicaltrials.gov and other established institutions (e.g., umhealthresearch.org) to enhance credibility and to certify this website as a central resource about CCTs.
Procedures
After completing the informed consent, participants answered a demographic questionnaire and questions about their experiences as a cancer patient or caregiver. Next, participants were directed to an instruction page within the survey and asked to provide insight on a website created for cancer patients to help them make better decisions about treatment. The information about the website was presented to participants in slightly different ways. After reading the instructions, all participants were directed to view the same website with content about CCTs for patients and caregivers. After viewing the CCT website, participants completed the remainder of the survey, including measures assessing their perceptions of the website (e.g., credibility) and the information (e.g., clarity), their beliefs about CCTs, as well as an open-ended question designed to solicit qualitative feedback about the CCT website. Full description of procedures are available from the first author upon request. All participants received a $10 gift card as compensation for their participation.
Predictors
Participant Type
Participant type was a dichotomous variable, with caregivers coded as referent condition.
Covariates
Demographic measures were dummy coded and controlled for in the analyses, including participant sex (male coded as referent), race (White coded as referent), education (some college education coded as referent), income (less than $59,999 coded as the referent), treatment status of the patient participating our study or the patient of the caregiver participating in our study (completed treatment coded as the referent), whether the patient participating in our study or the patient of the caregiver participating in our study had ever been asked to participate in a CCT (‘no’ coded as the referent), whether the patient participating in our study or the patient of the caregiver participating in our study had ever participated in a CCT (‘no’ coded as the referent), as well as the six most commonly reported cancer types among participants in the study (Breast, Colorectal, Lung or bronchus, Prostate, Gynecologic, and Melanoma, and ‘Other’ as the referent). We also controlled for participant’s age and website engagement using the number of clicks on the website and the amount of time (in minutes) participants spent on the website. We used the time stamps from when participants first clicked on the website link embedded in the survey and when participants initiated (i.e., clicked on) the post-test survey to determine duration of the website visit [15]. See Table 1 for covariates by participant type.
eHealth Literacy
Participant’s experience with the Internet was measured using Norman and Skinner’s ehealth literacy (eHEALS) [16]. Items measured the extent to which participants could effectively engage information technology for health. Eight items (e.g., “I know how to use the Internet to answer my questions about health”) were rated on a five-point Likert scale, with responses ranging from “strongly disagree” to “strongly agree” (M = 3.84, SD = .84, α = .93).
Dependent Variables
Website Credibility
Perceptions of website credibility was measured using items from Appelman and Sundar [10] and assessed participants’ perceptions that the website was accurate, credible, and believable. Three items (e.g., “The information I received from the clinical study website is accurate”) were rated on a five-point Likert scale, with responses ranging from “strongly disagree” to “strongly agree” (M = 4.27, SD = .69, α = .82).
Website Attractiveness
Perceptions of website attractiveness was adapted from Bull and colleagues [17] and measured the extent to which participants found the website appealing. A single item (i.e., “How attractive is the website?”) was rated on a seven-point scale, with response options ranging “very much” and “not at all.” Responses were reverse-coded, with higher scores indicating greater perceived attractiveness (M = 6.06, SD = 1.02).
Information Effectiveness
Perceptions of information effectiveness was adapted from Fishbein et al. [11] and assessed the extent to which participants perceived the information would be useful in assisting them with their decisions about CCTs. Four items (e.g., “Would the website be helpful in convincing you to participate in a clinical study?”) were rated on a five-point Likert scale, with responses ranging from “strongly disagree” to “strongly agree” (M = 3.40, SD = .56, α = .88).
Information Clarity
Perceptions of information clarity was adapted from Cacioppo, Petty, and Morris [9] and measured the extent to which participants perceived the information about CCTs was clear and easy to understand. Two items (e.g., “The website content is clearly explained”) were rated on a five-point Likert scale, with responses ranging from “strongly disagree” to “strongly agree” (M = 4.51, SD = .63, α = .80).
CCT Comprehension
Participants’ knowledge and beliefs about cancer research studies were measured using six true or false questions [18]. Participants selected a response to each of the six questions (e.g., “In a cancer research study, doctors know beforehand which treatment is best”) based on whether they believed the question was “True” or “False.” We used four items (1, 2, 3, and 5) to create a composite score for each participant, with higher scores indicating a greater comprehension of the information presented on the website (M = 2.08, SD = 1.24, α = .60).
Open-Ended Response Coding
To gain further understanding of how patients and caregivers appraised the website decision-aid tool, we completed qualitative analyses on participants’ open-ended responses evaluating the website. We developed a codebook with four of the dependent evaluation variables as categories (i.e., website credibility, website attractiveness, information effectiveness, information clarity) (See Table 3 for descriptions) and used the codebook to identify the prevalence of these themes across the qualitative data. Our codebook was established using team-based qualitative analysis techniques [19] which included concise code definitions (See Table 3) as well as explicating when to use the code and when not to use the code, all of which were agreed upon during the codebook development phase. Participants who evaluated multiple aspects of the website in their responses (e.g., mentioned website attractiveness and information clarity) were assigned multiple evaluation codes. In addition, we incorporated qualitative analysis techniques and drew from research on narrative frame [20, 21] to conduct valence coding and to determine the overall tone of the participant’s qualitative feedback about the website (i.e., positive, negative, neutral). The cornerstone of this research is the potential for individuals to experience emotional transformation—from either a neutral or negative state to a positive affective state or from a positive or neutral state to a negative affective state—as they make sense of an experience (e.g., their feelings about the CCT decision-making website) [20]. Participant responses that were complementary, encouraging, or emphasized participant’s positive feelings about the website or that began as neutral or negative and progressed to positive were assigned a positive valence code, whereas as those that emphasized participant’s negative feelings about the website or that began as positive or neutral and progressed to negative were assigned a negative valence code [20]. Responses that were neither positive or negative or were ambivalent at the end of the response were assigned a neutral valence code [21]. In line with this framework, open-ended participant responses were assigned a single valence code (i.e., responses were coded as either positive or negative or neutral) regardless of how they fit within the four evaluation categories.
Table 3.
Coding categories, descriptions, and examples from participant responses about the website
| Category | Description | Example | Valence |
|---|---|---|---|
| Website credibility | References to the website, or content as (un)reliable, (in)accurate, or (un)believable. Assessments of credibility often reflected the extent to which the information presented on the website matched participant’s knowledge of cancer or experiences with the CCTs. | “I was diagnosed with neuroblastoma in 1980, with my tumor wrapped around my heart and spine. At the time, there was no cure and very poor odds of survival. However, with surgery, radiation, and 2 years of a highly experimental chemo protocol of 4 different drugs, I survived. I do have MANY late effects, but I am happy to be alive and I fight for all the children fighting cancer every day!!” (Patient, #2) | Positive |
| “Seems to understate the role of the treating doctors in the drug testing process” (Caregiver, #126) | Negative | ||
| “do not be afraid to put the scary facts as well as the good. Cancer is bad and has more bad outcomes than good, do not under play that” (Caregiver, #198) | Neutral | ||
| Website attractiveness | References to the appearance or overall presentation of the website, including the design, color scheme, and presentation of content (e.g., videos). | “The site is well designed and user-friendly” (Caregiver, #127) | Positive |
| “More pictures teaching mechanisms of cancer and treatment instead of just random people posing” (Patient, #75) | Negative | ||
| “The website was overall good, but a little generic” (Patient, #50) | Neutral | ||
| Information effectiveness | References to the website or information as being informative and useful at helping individuals make informed decisions CCTs or increasing confidence and communication with loved ones about CCTs. Information participants would like to see on the website to assist with decision-making was also included in this category. | “I especially like the section for family and friends. They often have questions and that is a great resource. I also like the section with previous patient testimonies and comments. I often look for resale patient opinions about treatments and/or products” (Patient, #67) | Positive |
| “I think the website would help with a lot of questions people may have about cancer treatments like how family and friends can help support their loved one, how much it will cost and the videos are another great way to get information” (Caregiver, #187) | Positive | ||
| “I would like to see more information about how to sign up for current studies in my area” (Patient, #61) | Negative | ||
| Information clarity | References to the website or information as (un)clear or as easy or difficult to understand. | “I thought that the website was laid out in a simple way & was not overwhelming. It clearly listed all the available information” (Caregiver, #200) | Positive |
| “Clear and informative” (Patient, #70) | Positive | ||
| “Easy to digest” (Patient, #104) | Positive |
Examples of negative and neutral responses are included for categories in which they emerged across the data
Three coders independently coded 20% of the data and were highly reliable (κ = .80 for thematic coding, κ = .91 for valence coding). After discussing discrepancies, reaching consensus, and revising the codebook as needed, one of the coders independently coded the remainder of the data. All three coders reviewed the final dataset to determine fidelity with the codebook, after which participant quotes that most accurately reflected the four coding categories were selected as example quotes (see Table 3).
Results
Participant Characteristics
A series of descriptive analyses were conducted to compare the sociodemographic characteristics of caregivers and patients (Table 1). Caregivers were significantly younger, F(1, 198) = 10.45, p = .001, and had less college education, X2 (1) = 5.81, p < .05. Patients and caregivers reported differences in treatment status, X2 (1) = 4.00, p < .05, and differences in the type of cancer diagnosed, X2 (6) = 19.60, p < .01.
Participant Comprehension Across Items
To determine whether there were differences in participant comprehension of key concepts frequently misunderstood by patients and caregivers about CCTs, chi-square analyses were conducted on individual comprehension items. Table 2 reports the number and percentage of correct responses to each item. There were no significant differences in comprehension between patients and caregivers at the item level. Variability in the rate of correct responses differed greatly across the six items, with overall response accuracy ranging from 28 to 90% among participants. Differences between patients and caregivers were also examined within the four item-composite model of comprehension and were nonsignificant, F(19, 180) = .93, p = .55.
Table 2.
Patient and caregiver evaluations of the CCT website
| Website evaluation and outcomes | Total (N = 200) |
Patients (n = 100) |
Caregivers (n = 100) |
|
|---|---|---|---|---|
| M (SE)adja | M (SE)adja | M (SE)adja | F | |
| Website credibility | 4.27 (0.46) | 4.22 (0.70) | 4.33 (0.70) | 1.24 |
| Website attractiveness | 6.06 (0.71) | 5.91 (0.11) | 6.20 (0.11) | 3.22† |
| Information effectiveness | 3.40 (0.38) | 3.44 (0.59) | 3.36 (0.70) | 0.90 |
| Information clarity | 4.51 (0.44) | 4.46 (0.66) | 4.57 (0.66) | 1.11 |
| Composite comprehension | 2.08 (0.10) | 2.21 (0.13) | 1.94 (0.13) | 1.98 |
| Individual comprehension statements | ||||
| 1. Doctors who place their patients in cancer research studies often choose the treatment that their patients should receive (F).b | 73 (36.3) | 43 (43.0) | 30 (29.7) | 3.84†c |
| 2. In a cancer research study, doctors know beforehand which treatment is best (F).b | 136 (67.7) | 63 (63.0) | 73 (72.3) | 1.98c |
| 3. The decision to participate in a cancer research study is made by a patient’s doctor (F).b | 152 (75.6) | 80 (80.0) | 72 (71.3) | 2.07c |
| 4. A patient enrolled in a cancer research study can choose to stop his/her treatment at any time (T).b | 179 (89.1) | 88 (88.0) | 91 (90.1) | 0.23c |
| 5. The goal of a cancer research study is to match people to the best treatment for them (F).b | 56 (27.9) | 33 (33.0) | 23 (22.8) | 2.62c |
| 6. It may take many years for the results of a cancer research study to help people with cancer (T).b | 181 (90.0) | 90 (90.0) | 91 (90.1) | 0.00c |
Adjusted for age, sex, race, education, treatment status, cancer type, ask CCT, participate CCT, eHEALS, message condition, time spent on website, and clicks on website
Counts and within-column percentages of correct answers
Unadjusted Pearson chi-square statistic
A principal components analysis with a varimax rotation identified two factor loadings within the comprehension items. Factor 1, which consisted of Statements 1, 2, 3 and 5, had a rotated eigenvalue of 1.81 and helped explain 30.09% of the variance, but had a weak Cronbach’s alpha (α = .60). Factor 2 consisted of Statements 4 and 6, had a rotated eigenvalue of 1.17 and helped explain a further 19.50% of the variance, but had inadequate Cronbach’s alpha for further investigation (α = .18). Mean and SE for composite comprehension reflect the 4-item scale used in the GLM analyses;
p ≤ 0.10
Participant Evaluations of the CCT Website
A series of general linear models (GLM) were conducted to determine if website evaluations differed among patients and caregivers on perceived website credibility, website attractiveness, information effectiveness, and information clarity (see Table 2 for adjusted means). Within the models, website evaluation differed based on participant age, eHEALS, and the length of time the participant spent on the website. The GLM for website credibility was statistically significant, F(19, 180) = 2.63, p < .001, R2 = .22, with both patients and caregivers perceiving the website as highly credible, F(1, 180) = 1.23, p =.27, b = − 0.11, t = − 1.11, adjMpatient = 4.22 vs. adjMcaregiver = 4.33. Within the model, participants who were younger (b = 1.67, t = − 2.01, p < .05) reported a higher eHEALS (b = 0.29, t = 4.89, p < .001), and those who spent more time on the website (b = 0.04, t = 2.60 p = .01) were statistically more likely to perceive the website as credible. The GLM for information effectiveness was also statistically significant, F(19, 180) = 1.82, p < .05, R2 = .16, with both patients and caregivers perceiving the website as moderately effective at helping them make decisions about treatment, F(1, 180) = .90, p = .35, b = 0.08, t = 0.95, adjMpatient = 3.45 vs. adjMcaregiver = 3.36. Within the model, participants who reported a higher eHEALS (b = 0.19, t = 3.89, p < .001) were statistically more likely to perceive the information as highly effective compared to those with lower eHEALS. The GLMs for website attractiveness, F(19, 180) = 1.33, p = .17, R2 = .12, and information clarity, F(19, 180) = 1.49, p = .09, R2 = .14, were not statistically significant and no further subgroup analyses were explored.
Participant Characteristics, Website Evaluations, and Comprehension
To explore whether there was an association with any participant characteristics, participant evaluations of the CCT website, and comprehension at the item level, we conducted a series of adjusted binary logistic regressions. Statement 4, “A patient enrolled in a cancer research study can choose to stop his/her treatment at any time (T),” was the only model that significantly predicted comprehension, X2 (21) = 52.49, p < .001, Cox and Snell R2 = .23. Within the model, there were no significant differences between the two participant groups (OR = 2.19, 95% CI = 0.59–8.16, p = .25) nor was comprehension influenced by perceived website credibility (OR = .67, 95% CI = 0.19–2.38, p = .54) or perceived information effectiveness (OR = 4.02, 95% CI = .77–20.96, p = .10). However, older participants (OR = 1.09, 95% CI = 1.03–1.17, p <.01), female participants (OR = 12.39, 95% CI = 2.17–70.86, p < .01), participants with a greater eHEALS score (OR = 2.49, 95% CI = 1.10–5.64, p < .05), and participants who spent more time on the website (OR = 2.13, 95% CI = 1.00–4.54, p < .05) were more likely to answer this statement correctly, whereas participants (i.e., patients participating this study or patients of the caregiver participating in this study) who were currently in treatment were less likely to answer this statement correctly than those who already completed treatment (OR = .20, 95% CI = .04–0.90, p < .05). In other words, comprehension of key concepts surrounding CCTs was unrelated to the participant type (i.e., role as a patient or caregiver) and was not influenced by perceptions of website credibility or perceptions of information effectiveness, but was related to participant age, eHealth literacy, gender, the amount of time spent on the website, and treatment status.
Open-Ended Responses
We coded a total of 89 open-ended responses from participants, including 39 responses from patients (43.8%) and 50 responses from caregivers (56.2%).1 Descriptive analyses were used to calculate the average word count of participant responses. The overall average word count among participants was 13.82 words (SD= 14.87), with similarities in patient word count (M = 15.31, SD= 18.63) and caregiver word count (M = 12.66, SD= 11.17). An ANOVA revealed non significant differences between the means for patient and caregiver word count, F(1, 87) = .69, p = .41.
In total, 82% of participants (n = 73) evaluated (i.e., commented on) a single aspect of the website (e.g., website attractiveness; “it actually needed more color, it was a little bland” Patient, #97) whereas 18% participants (n = 16) commented on multiple aspects of website (e.g., website attractiveness, information clarity, information effectiveness; “Beautiful, straight forward, and very helpful testimonials” Caregiver, #203). Over half (n = 53, 59.6%) of the participants referenced information effectiveness and its potential to facilitate decision-making about CCTs (e.g., “[this website] made me feel more confident in my decision to talk to my mom about joining a research trial. Thank you” Caregiver, #183), 36% of participants mentioned website attractiveness (n = 32), 13.5% (n = 12) mentioned information clarity (e.g., “[the website] is very clear and concise” Caregiver, #201), and only 11.2% of participants (n =10) referenced website credibility (e.g., “[the website] does not seem all that professional and could use some outside resources and validation” Caregiver, 185), (see Table 3 for descriptions and additional examples). Caregivers were more likely to reference website attractiveness (n =18, 20.2% vs. n =14, 15.7%, X2 (1) = .00, p =.59), as well as the information effectiveness than caregivers (n = 26, 29.2% vs. n = 27, 30.3%, X2 (1) = 1.60, p = .16), but neither differed significantly. Patients and caregivers referenced the website’s credibility with a similar frequency (n = 4, 4.5% vs. n = 6, 6.7%, X2 (1) = .07, p = .54). However, caregivers were significantly more likely than patients to comment on the clarity of the information (n = 2, 2.2% vs. n = 10, 11.2%, X2 (1) = 4.15, p < .05).
Responses were overwhelmingly positive (n = 72, 80.9%; e.g., “I think it’s a useful tool if considering becoming part of a study” Patient, #74), with a smaller proportion of participants providing negative (n = 10, 11.2% negative; e.g., “Interesting information, but video voiceovers in several cases included individuals who SEEMED to be “talking down” to the potential study participants” Patient, #99), and neutral (n = 7, 7.9%; e.g., “I was not sure about the cost/monetary gain from participating in clinical research studies” Patient, #77) feedback about the website. Findings were similar for both patients (40.3% of responses were positive, 17.9% negative, 7.7% neutral) and caregivers (59.7% of responses were positive, 6% negative, 8% neutral). A chi-square analysis revealed non significant differences between patients and caregivers in their sentiment toward the website, X2 (2) = 3.15, p = .21.
Discussion
Cancer patients and caregivers perceived the CCT website as highly credible and attractive and indicated that its content was easy to understand and moderately effective in terms of helping them make decisions about cancer treatment. Qualitative findings further substantiate participant evaluations and illuminate ways that website features shaped their acceptability. Despite overwhelming acceptance, variability in correct responses and discrepancies in comprehension of certain items between patients and caregivers suggests a disconnect between patient’s and caregiver’s actual and perceived information needs. This gap elucidates the need for increased precision when evaluating knowledge outcomes. Results have important practical implications for designing and implementing CCT decision-aid websites for patients and caregivers across the cancer continuum.
Patients were more likely to report a remission status, whereas caregivers reported currently supporting a patient through the treatment process. Despite their different placement on the cancer care continuum, patients and caregivers reported a comparably high degree of CCT website acceptability. Few participants were presented with a CCT as an option for treatment, which corresponds with trends reporting low rates of CCT accrual in the general population [22]. While there is evidence that Internet use is associated with clinical trial awareness [23], a greater degree of eHealth literacy (i.e., familiarity with and use of the Internet as a health information tool) was positively associated with each element of website acceptability and emerged as a positive predictor of comprehension. Therefore, being a proficient, critical consumer of online health information as a patient or caregiver positively contributes to the acceptability of a CCT website tool and certain aspects of comprehension, regardless of sociodemographic, treatment status, or previous opportunities to participate in CCTs.
Another key finding from this study is the variability in comprehension of certain CCT concepts regardless of participant type or whether participants were instructed to view a personalized website or simply a website. The majority of patients and caregivers misinterpreted CCT content about treatment. Participants overwhelmingly believed that doctors who place patients in CCTs choose the treatment that their patients should receive and that patients in CCTs are matched to the best treatment option for them. Indeed, poor comprehension of CCTs negatively influences enrollment [2]. However, given the significant role of caregivers and their involvement in patient care and treatment decision-making [4, 5], the discrepancies identified in our study offer additional insight as to why CCT participation and retention suffer. Poor comprehension of CCTs (i.e., treatment choice and the goals of CCTs), coupled with reports from participants that the website was highly clear, easy to understand, and would be a helpful tool to facilitate conversations about treatment, underscore the need for greater precision and accuracy in measuring comprehension in order to increase overall knowledge of CCTs among patients and caregivers.
The length of time spent on the CCT website was positively associated with the perceived credibility of the website and with comprehension of certain CCT concepts. Online users spend limited time on web pages (e.g., some estimates are in seconds) [24], and initial assessment of credibility is often based on the website’s content [13]. Perceptions of website credibility are also positively influenced by website structural features (e.g., navigation menu and external links) [9]. The implied credibility of the CCT website due to its use of structural features coupled with a high eHealth literate audience may have contributed to the above average amount of time spent on the website and likely contributed to increased comprehension of certain CCTs concepts. This finding also has positive implications for promoting retention in CCT websites (i.e., use, visits), as credibility and knowledge of a health topic increases the probability to revisit a website [9]. More research is needed to examine what online behaviors participants were engaged in during the extended time spent on the CCT website.
Age, eHealth literacy, and gender were also positively associated with perceptions of website credibility. Specifically, female participants, participants who were younger, and those with a higher eHealth literacy were more likely to perceive the website as credible. Age and use of the Internet to search for health information are linked to positive user perceptions of online health information [6, 12]. Thus, individuals who are younger and have a higher eHealth literacy may be more likely to evaluate the website as credible because they have a greater familiarity and trust in the Internet as a source of health information. In terms of gender, men and women perceive the Internet as a credible source of health information, although women are more likely obtain health information from online sources whereas men rely on interpersonal sources (e.g., spouses/partners) for health information and evaluate these sources more favorably in terms of credibility [25]. Differences in men’s and women’s preferences for seeking and receiving online health information likely contributed to differences in evaluations of website credibility. These findings demonstrate the need to incorporate audience segmentation strategies into the development and dissemination of CCT decision-making interventions. For example, it may be beneficial to develop online CCT decision-aid tools targeting younger adults, females, and individuals with higher eHealth literacy.
Females and older participants were more likely to answer the statement about CCTs, “A patient enrolled in a cancer research study can choose to stop his/her treatment at any time” correctly. Similarly, participants (i.e., patients participating this study or patients of the caregiver participating in this study) who were currently in treatment were also less likely to answer this statement correctly than those who already completed treatment. Women play a prominent role in caregiving [5] and are more likely than men to obtain health information from the Internet [6], which could explain their increased comprehension of certain CCT concepts. Older adults are at a greater risk for being diagnosed with cancer [1], and thus, may have more knowledge of CCTs and treatment than younger age cohorts due to personal (i.e., own, family, friend) experience. Given the low proportion of participants who were presented or participated in a CCT for treatment, participants currently involved in treatment may be more concerned with locating information pertaining to their specific treatment situation (e.g., chemotherapy, medication side effects) and less concerned with information pertaining to CCT decision-making (i.e., treatment termination). These findings reinforce the importance of incorporating audience segmentation strategies into the dissemination of CCT decision-making interventions and underscore the timing of intervention delivery as a critical to CCT comprehension and decision-making (e.g., study enrolment).
Qualitative feedback from patients and caregivers about the website was overwhelmingly positive. Consistent with empirical evidence exploring the positive effect of narratives on health attitudes [26], qualitative data revealed that inclusion of personal narratives (e.g., from cancer survivors, CCT participants) enhanced perceptions of credibility and positively influenced perceptions of information effectiveness among participants. Testimonials enhanced the salience of messages about CCTs among participants and increased their attention to the website. Indeed, esthetic, structural features also facilitate heuristic processing [27] to enhance the perceived credibility of website or message. Structural features increase perceptions of credibility and positive attitudes toward behavior change, whereas use of testimonials enhances positive perceptions of a health behavior or health topic [9]. In this study, however, narratives, alone, served as a heuristic cue for website credibility. In other words, our findings demonstrate that integrating narratives into CCT websites will enhance the perceived credibility of CCTs among patients and caregivers and may increase time spent reviewing CCT content.
Results of this study provide important implications for the development of CCT recruitment websites. A recent content analysis [28] found that clinical research websites rarely include participant-oriented portals with information targeted to this population, and less than half of the portals include testimonials or thoroughly describe the clinical trial process. In attempts to advance the quality and accessibility of CCT websites for patients and caregivers, an equal emphasis should be placed on perceived attractiveness, credibility, clarity, and effectiveness of CCT websites, with narrative techniques incorporated across the website to sustain perceptions of credibility and to retain user engagement. These elements should be salient to patients and caregivers at varying degrees of eHealth literacy, not just to those who are proficient and critical eHealth users.
This study is not free from limitations. First, this study was conducted through a web-based survey. Despite comprising adults over the age of 40, this survey modality introduces sampling bias because participants are likely regular Internet users with a high degree of confidence in their online health information seeking and evaluation skills [29]. Another potential limitation has to do with our coding procedures. Specifically, we used different units of analysis for our evaluation and valence coding procedures. Participants who commented on multiple aspects of the website (e.g., website attractiveness and credibility) in their responses were assigned multiple evaluation codes whereas open-ended responses were assigned a single valence code. Although the vast majority of participants whose responses were included in the qualitative analyses (n = 73) commented on a single aspect of the website and, therefore, were assigned a single evaluation (e.g., website credibility), future studies should consider using the same unit of analysis across qualitative coding procedures. Finally, patients and caregiver survey responses were not linked; however, this remains a challenging population to recruit into health research. Therefore, enrolling an equal number of patients and caregivers for comparison purposes is also a significant strength.
Footnotes
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
A total of 136 participants completed the open-ended question on the survey. Responses that reflected one of the valence codes (e.g., “I like [the website]” Caregiver, #190, Positive; “I really have nothing to add” Patient, #90, Neutral), but did not fit within any of the four dependent coding categories were removed from the analyses.
References
- 1.Cancer Statistics. National Cancer Institute website https://www.cancer.gov/about-cancer/understanding/statistics. Updated April 27, 2018. Accessed June 29, 2018
- 2.Ford JG, Howerton MW, Lai GY, et al. (2008) Barriers to recruiting underrepresented populations to cancer clinical trials: a systematic review. Cancer. 112(2):228–242. 10.1002/cncr.23157 [DOI] [PubMed] [Google Scholar]
- 3.Stacey D, Légaré F, Col NF, et al. (2017) Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev 4:CD001431 10.1002/14651858.CD001431.pub5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Hobbs GSLM, Arora NK, Ganz PA, et al. (2015) The role of families in decisions regarding cancer treatments. Cancer. 2015(121): 1079–1087 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Krieger JL (2014) Family communication about cancer treatment decision making. A description of the DECIDE typology. Ann Int Commun Assoc 38(1):279–305 [Google Scholar]
- 6.Rains SA (2007) Perceptions of traditional information sources and use of the world wide web to seek health information: findings from the Health Information National Trends Survey. J Health Commun 12:667–680. 10.1080/10810730701619992 [DOI] [PubMed] [Google Scholar]
- 7.Tustin N (2010) The role of patient satisfaction in online health information seeking. J Health Commun 15(1):3–17 [DOI] [PubMed] [Google Scholar]
- 8.Mayer DK, Terrin NC, Kreps GL, et al. (2007) Cancer survivors information seeking behaviors: a comparison of survivors who do and do not seek information about cancer. Patient Edu Cous 65(3): 342–350 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Cacioppo JT, Petty RE, Morris KJ (1983) Effects of need for cognition on message evaluation, recall, and persuasion. J Pers Social Psy 45(4):805–818. 10.1037/0022-3514.45.4.805 [DOI] [Google Scholar]
- 10.Appelman A, Sundar SS (2016) Measuring message credibility: construction and validation of an exclusive scale. J Mass Commun Q 93(1):59–79 [Google Scholar]
- 11.Fishbein M, Hall-Jamieson K, Zimmer E, Von Haeften I, Nabi R (2002) Avoiding the boomerang: testing the relative effectiveness of antidrug public service announcements before a national campaign. Am J Public Health 92(2):238–245 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Rains SA, Karmikel CD (2009) Health information-seeking and perceptions of website credibility: examining web-use orientation, message characteristics, and structural features of websites. Comput Human Beh 25:544–553 [Google Scholar]
- 13.Hu Y, Sundar S (2010) Effects of online health sources on credibility and behavioral intentions. Commun Res 37(1):105–132 [Google Scholar]
- 14.van Weert JC, van Noort G, Bol N, van Dijk L, Tates K, Jansen J (2011) Tailored information for cancer patients on the Internet: effects of visual cues and language complexity on information recall and satisfaction. Patient Edu Cous 84(3):368–378 [DOI] [PubMed] [Google Scholar]
- 15.Couper MP, Alexander GL, Zhang N et al. (2010) Engagement and retention: measuring breadth and depth of participant use of an online intervention. J Med Internet Res 12(4):e52 10.2196/jmir.1430 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Norman CD, Skinner HA (2006) eHealth literacy: essential skills for consumer health in a networked world. J Med Internet Res 8(2): e9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Bull FC, Holt CL, Kreuter MW, Clark EM, Scharff D (2001) Understanding the effects of printed health education materials: which features lead to which outcomes? J Health Commun 6: 265–279. 10.1037/0278-6133.26.5.598 [DOI] [PubMed] [Google Scholar]
- 18.Curbow B, Fogarty LA, McDonnell KA, Chill J, Scott LB (2004) Can a brief video intervention improve breast cancer clinical trial knowledge and beliefs? Soc Sci Med 58:193–205 [DOI] [PubMed] [Google Scholar]
- 19.MacQueen KM, McLellan E, Kay K, Milstein B (1998) Codebook development for team-based qualitative analysis. CAM J 10(2):31–36 [Google Scholar]
- 20.McAdams DP, Reynolds J, Lewis M, Patten AH, Bowman PJ (2001) When bad things turn good and good things turn bad: sequences of redemption and contamination in life narrative and their relation to psychosocial adaptation in midlife adults and in students. Personal Soc Psychol Bull 27(4):474–485 [Google Scholar]
- 21.Koenig Kellas J, Baxter LA, LeClair-Underberg C, et al. (2014) Narratively (re) framing stepfamily beginnings: the relationship between adult stepchildren’s stepfamily origin stories and their perceptions of the family. J Fam Commun 14:149–166 [Google Scholar]
- 22.Rimel BJ (2016) Clinical trial accrual: obstacles and opportunities. Front Oncol 6:103 10.3389/fonc.2016.00103 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Leiter A, Diefenbach MA, Doucette J, Oh WK, Galsky MD (2015) Clinical trial awareness: changes over time and sociodemographic disparities. Clin Trials 12(3):215–223. 10.1177/1740774515571917 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Nah FF (2004) A study on tolerable waiting time: how long are web users willing to wait. Beh Information Technology 23(3):153–163 [Google Scholar]
- 25.Carpenter DM, DeVellis RF, Hogan SL, Fisher EB, DeVellis BM, Jordan JM (2011) Use and perceived credibility of medication information sources for patients with a rare illness: differences by gender. J Health Commun 16(6):629–642 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Neil JM, Gough A, Kee F, George TJ, Pufahl J, Krieger JL. The influence of patient identification and narrative transportation on intentions to participate in cancer research. J Cancer Edu 2018. 1–10 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Sillencea E, Briggsa P, Harrisb PR, Fishwicka L (2007) How do patients evaluate and make use of online health information? Soc Sci Med 64:1853–1862 [DOI] [PubMed] [Google Scholar]
- 28.Flood-Grady E, Paige SR, Karimipour N, Harris PA, Cottler LB, Krieger JL (2017) A content analysis of clinical and translational science award (CTSA) strategies for communicating about clinical research participation online. J Clin Transl Sci 1(6):340–351. 10.1017/cts.2018 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Neter E, Brainin E (2012) eHealth literacy: extending the digital divide to the realm of health information. J Med Internet Res 14(1):e19 10.2196/jmir.1619 [DOI] [PMC free article] [PubMed] [Google Scholar]