Decisional outcomes following use of an interactive web-based decision aid for prostate cancer screening

Abstract

Informed decision-making tools are recommended for men considering prostate cancer screening. We evaluated the extent to which use of an interactive, web-based decision aid was associated with decisional and screening outcomes. Participants (N = 253) were 57 (7.0) years old and completed telephone interviews at baseline, 1 month, and 13 months post-baseline. Tracking software captured minutes spent on the website (median = 33.9), sections viewed (median = 4.0/5.0), testimonials viewed (median = 4.0/6.0), and values clarification tool (VCT) use (77.3 %). In multivariable analyses, all four website use variables were positively associated with increased knowledge (p’s < 0.05). Complete VCT use and number of informational sections were positively associated with greater decisional satisfaction (p’s < 0.05). Decisional conflict and screening behavior were not associated with measures of website use. Increased use of informational content and interactive elements were related to improved knowledge and satisfaction. Methods to increase utilization of interactive website components may improve informed decision-making outcomes.

INTRODUCTION

Recent clinical trials have produced mixed results about the benefits of prostate cancer (PCa) screening for reducing disease-related mortality [1–4]. While screening recommendations vary somewhat by professional organization [5–7], the U.S. Preventive Services Task Force now recommends against routine screening for all ages [8]. In the midst of this debate, most men undergoing PCa screening are aware of the benefits, but fewer are aware of the limitations [9, 10].

Decision aids can help to educate patients and to encourage shared decision making, a process whereby patients discuss treatment options and decisions with their doctors [11, 12]. A growing number of decision aids are being provided online as adults are using the Internet in greater numbers to access health information [13, 14]. Additionally, studies of men with PCa found that the Internet and electronically based information were important sources of information regarding both screening and diagnosis [15–18]. In general, online health information seekers say they feel more confident in starting a discussion with their doctor than those who do not use the Internet to find health information [19]. Delivering accurate and current medical information is important for open communication between patients and doctors [20].

With men increasingly relying on the Internet for PCa screening information, it is important to examine the effectiveness of web-based decision aids [14]. In a prior study, we conducted a three-arm randomized controlled trial to compare the impact of a print-based decision aid, an interactive web-based decision aid, and usual care on PCa screening decision-making outcomes [21]. Among 1879 primary care patients, we found that, compared to usual care, the print- and web-based decision aids resulted in greater PCa knowledge, lower decisional conflict, and increased satisfaction with the screening decision, at the 1-month and 13-month follow-up assessments. There was no impact of the decision aids on screening behavior.

In other randomized controlled trials that included a web-based decision aid for PCa screening, participant knowledge consistently improved [21–28], while decisional conflict results were more variable, with some studies reporting a decrease compared to controls [21–23, 25, 28] and others finding no change [26, 27]. Similarly, there have been variable results on screening outcomes, possibly depending on the timeframe of the follow-up assessment. Studies that have assessed shorter-term screening outcomes have reported that men were less likely to get screened after using a web-based decision aid [23–25, 27], while in our trial of longer-term screening outcomes, we found no change in screening behavior [21].

In a prior analysis of our PCa screening website [29], we presented the number of unique logins, total and average time spent on particular sections of the website, use of video testimonials, and the characteristics of website users vs. non-users. Men responded favorably to the website, spending a median of 34 min on the website, and over 88 % of users responded to all 10 values clarification items [29]. In the current study, we assessed the degree to which our web-based decision aid impacted important decision-making outcomes. Using the Ottawa Decision Support Framework [30], which identifies barriers to informed decision making such as poor knowledge and decisional conflict, we focused on the association of different facets website use with the outcomes of PCa knowledge, decisional conflict, satisfaction, and screening behavior. Tracking software captured website use, allowing for an investigation of whether use of informational sections, testimonial videos, and the values clarification tool were associated with decisional outcomes. Based on prior studies in several domains [14, 28, 31–34] and on a model of Internet interventions [14], we hypothesized that greater interaction with the website would be associated with improved knowledge and satisfaction, decreased decisional conflict, and decreased screening rates.

METHODS

We have briefly described the prior randomized trial below [21].

Subjects

Eligibility criteria for the randomized trial included (1) men between the age of 45 and 70, (2) no prior history of PCa, (3) ability to provide informed consent, (4) English speaking, (5) living independently (e.g., nursing home occupants were excluded), and (6) having had an outpatient visit in the last 24 months.

Procedure

We mailed invitation letters to eligible primary care patients between October 2007 and January 2010 and followed up with a phone call to describe the study and confirm eligibility. Among those interested in participation, we obtained verbal consent and conducted the 20–25-min baseline interview. After the baseline assessment, participants were randomized to one of the three study arms: print (n = 630), usual care (n = 632), or web (n = 631; Fig. 1).

Fig. 1.

Flow of participants

Written consent forms were mailed after the baseline assessment, with web arm participants receiving a letter with information about how to log on to the website, a unique user ID and password for access, and instructions for the website’s navigation. Participants were encouraged to review the website prior to the 1-month interview and were asked to not share their user ID or password with anyone else to ensure only study participants accessed the site. Men assigned to the web arm were given access to the website for the full 13-month study period (although very few men used it following the 1-month assessment). Men completed the second telephone interview 1 month post-baseline and completed the final telephone interview 13 months post-baseline. Men received a gift card after the T1 interview and were entered into a gift card lottery after completing the T2 interview, but did not receive additional incentives to review the decision aids. The remainder of this paper concerns only the participants from the web arm.

Description of the website

A detailed description of the development and content of the website has been described previously [35]. Briefly, the website was created in accordance with the International Patient Decision Aid Standard criteria [36] by providing balanced information about the risks and benefits of screening, showing a range of potential experiences with PCa screening, providing a means for patients to express their values, and using the most up-to-date literature. Further, the website allows for easy step-by-step navigation and allows for multiple visits, necessary criteria for web-based decision aids. We have recently updated the website to include the AUA and the USPSTF recommendations (prostatedecision.georgetown.edu).

Once men logged on to the website, they completed a single-item assessment of their initial PCa screening preference. Responses were dichotomized into those favorably inclined toward screening vs. those uncertain about screening. Participants were then immediately shown a testimonial that presented the opposite view of their stated screening preference (i.e., those who were favorably inclined viewed a testimonial that emphasized the limitations of screening, and those who were uncertain about screening viewed a testimonial that emphasized the benefits of screening). Users were then able to proceed by clicking on the sections or features they wished to see. There were pop-up definitions of medical terms, voiceovers for a majority of the text, and information was written at an 8th grade reading level [37]. The site contained six informational sections, six remaining video testimonials, and a values clarification tool, described below.

Informational Sections

The website included six informational sections, each divided into three to four subsections. The Introduction section demonstrated navigation of the website and provided basic information about the prostate and PCa screening. The Screening section gave information about different types of PCa screening exams and what results may indicate. The Treatment Issues section provided facts about different PCa treatment options, including risks and treatment side effects. The Steps You Can Take section discussed PCa risk factors and encouraged men to discuss screening with their doctors. The Decision Aid section described users’ screening preference based on their answers to questions from the values clarification tool (described below). Finally, the More Information section included a glossary of medical terms, references, and links to cancer-related organizations with further information on PCa screening.

Testimonials

Eight testimonial videos are included on the website, ranging in length from 25 to 60 seconds and showing either the benefits or limitations of: screening, undergoing a biopsy, different treatment modalities, and the side effects of treatment. As described above, the first two testimonials insured that men were aware of the opposing viewpoint associated with PCa screening. The remaining six testimonials, accessible to all users, appeared in pairs throughout the site and presented a favorable and an uncertain view of screening.

Values Clarification Tool

The values clarification tool is a 10-item tool to help men better understand their preferences regarding screening [38]. The 10 items appear throughout the website in the relevant content sections. One half of the items concern the benefits of screening (“Getting screened will help me feel like I’m doing everything I can for my health.”) and one half concern the limitations (“Screening may result in an unnecessary biopsy. I will wait until screening is more accurate before getting tested.”). After each statement, participants responded to “Does this statement sound like you?” with “yes”, “no”, or “not sure yet.” On the Decision Aid summary page, responses to the 10 statements were displayed on a balance scale graphic, with a statement indicating whether the participant was “leaning toward” or “leaning away” from screening. Answers could be modified on the summary page and the summary page could be printed to facilitate the doctor/patient discussion about patient preferences.

Measures

Demographic and Clinical Data

At the baseline assessment, we assessed age, marital status, education level, race/ethnicity, income, and employment status. Clinical information included having a regular doctor, health insurance status, comorbidities, personal and family histories of PCa and other cancers, whether men had discussed screening with their doctor, and their PCa screening history (both ever in their lifetime and within the last 12 months).

PCa Knowledge

The PCa Knowledge questionnaire includes 18 items assessing men’s understanding of the PCa screening controversy, PCa risk factors (such as race and age), PCa treatments and their side effects, and the natural history of PCa. Response choices were “true,” “false,” or “don’t know,” with “don’t know” responses scored as incorrect. PCa knowledge was assessed via telephone interviews at all three time points. Possible scores ranged from 0 to 18 and alpha reliability was 0.76.

Satisfaction with Decision Scale

[39]. The Satisfaction with Decision Scale includes six items with response choices on a five-point scale, from “strongly disagree” to “strongly agree.” Satisfaction with decision was assessed via telephone interviews at 1 month and 13 months. Alpha reliability was 0.87. Satisfaction with decision was highly positively skewed and therefore was dichotomized at the median at both time points (median = 4.7 at 1 month; median = 4.8 at 13 months).

Decisional Conflict Scale

[40]. The Decisional Conflict Scale measured participants’ certainty about their PCa screening decision. We used the 10-item version, with response choices of “yes,” “no,” or “unsure.” Decisional conflict was measured via telephone interviews at all three time points. Possible scores ranged from 0 to 100, with higher scores indicating greater conflict. Alpha reliability was 0.81. Due to a skewed distribution (over half of the sample reported no conflict at baseline, and nearly three fourths reported no conflict at both follow-up assessments), we dichotomized decisional conflict into those with no conflict vs. those with any conflict (median = 10.0 at baseline; median = 0.0 at 1 month and at 13 months).

PCa Screening

At the 13-month interview, men reported whether they had received a prostate-specific antigen (PSA) test and/or a digital rectal exam (DRE) since randomization.

Time Spent on Website

Customized tracking software captured the time spent and responses provided on the website. Users of the site were those who logged on for at least 1 min prior to the 1-month assessment and who also self-reported that they logged on. Due to the positively skewed distributions of the website use variables, we categorized each variable as described in the “Results” section.

Number of Website Sections

All participants visited the Introduction section, resulting in a total of five sections of the website available for analysis.

Testimonial Videos

Testimonial use was defined as viewing at least half of a particular video (e.g., viewing at least 30 s of a 60-s video). We limited analyses to the six testimonial videos that were available to all users.

Values Clarification Tool

We defined a complete user of the values clarification tool as those who answered all 10 items and visited the summary page of the Decision Aid section.

Data analysis

We first conducted descriptive analyses of the demographic and clinical variables (Table 1) and the website use variables (Table 2). Next, we examined the association of website use with 1-month and 13-month outcomes using linear and logistic regression models with generalized estimating equations (GEE). GEE methods account for correlated outcomes, such as repeated measurements. In linear regression models for the continuous outcome of PCa-related knowledge, we adjusted for significant covariates (race, education, site, comorbidities, employment, and if men had ever been screened for PCa) and baseline knowledge scores. For linear regression analyses, estimated beta coefficients (B) are presented, which represent adjusted mean differences between groups (e.g., low vs. high website users) at both the 1-month and 13-month assessments. We conducted separate logistic regression models for the categorical outcomes of decisional conflict, satisfaction with decision, and screening, and adjusted for the significant covariates of age, race, education, marital status, screening history, and the baseline measure of each outcome (except for satisfaction, as it was not measured at baseline). For logistic regression analyses, we present adjusted odds ratios (OR), the ratio of the odds that an outcome will occur in one group vs. another (e.g., low vs. high website users). There were no statistically significant interactions between any of the website use variables and time of follow-up assessment (1 vs. 13 months). As such, we removed the interactions from all analyses and therefore the estimated beta coefficients and odds ratios apply to both the 1-month and 13-month assessments. We used SPSS version 20.0 for all analyses.

Table 1.

Demographic and clinical characteristics of website users

	All users (N = 253)
Age (range 45–70) mean (SD)	57.5 (7.0)
	N (%)
Education
Some college/less	80 (31.6)
College degree+	173 (68.4)
Married/marriage-like relationship
No	64 (25.3)
Yes	189 (74.7)
Race
White	188 (74.3)
African American	54 (21.3)
Other	11 (4.3)
Site
GUH	117 (46.2)
WHC	14 (5.5)
MPP	122 (48.2)
Employment
Unemployed	68 (26.9)
Full/part time	185 (73.1)
Annual income
<100 K	101 (42.9)
>100 K	136 (57.1)
Missing	16
Comorbidities
0	93 (36.8)
1	79 (31.2)
2+	81 (32.0)
Regular doctor (yes)	245 (96.8)
Insurance (yes)	251 (99.2)
Personal history of cancer (yes)	48 (19.0)
Family history of PCa (yes)	62 (25.2)
Ever discussed PCS with doctor (yes)	189 (74.7)
Ever screened for PCa (yes)	238 (94.1)
Screened in the past 12 months (yes)	183 (72.3)
Missing	0

Table 2.

Descriptives of website usage (N = 253)

Website use	N (%) used	Median	Interquartile range	Mean (SD)
Total time on site (min)		33.7	21.9–45.7	34.3 (17.6)
≤10 min	24 (9.5)
10.01–20.00	34 (13.4)
20.01–30.00	43 (17.0)
30.01–40.00	63 (24.9)
40.01–50.00	44 (17.4)
50.01–60.00	27 (10.7)
60.01–70.00	11 (4.3)
70.01–112.00	7 (2.8)
Sections viewed		4.0	4.0–5.0	3.8 (1.3)
0–1	29 (11.5)
2–3	14 (5.5)
4	132 (52.2)
5	78 (30.8)
Testimonial use		4.0	1.0–6.0	3.2 (2.4)
0	61 (24.1)
1–2	44 (17.4)
3–4	43 (17.0)
5	29 (11.5)
6	76 (30.0)
Values clarification tool use
Values clarification tool questions answereda		10.0	10.0–10.0	9.5 (1.6)
1–3	7 (2.8)
4–6	5 (2.0)
7–9	25 (9.9)
10	214 (84.6)
Visited values clarification tool summary page		N/A	N/A	N/A
Yes	214 (84.6)
Complete values clarification tool use		N/A	N/A	N/A
Incomplete: answered ≤10 questions OR visited summary page	56 (22.1)
Complete: answered 10 questions AND visited summary page	197 (77.9)

^a N = 2 (0.8 %) data missing

RESULTS

Participants

The tracking software indicated at least one unique login for 260 of the 631 participants assigned to the Web arm. However, as only 256 of the 260 users also self-reported logging on, we excluded the four participants with discrepancies between the tracking software and self-report. We also excluded the three men who were diagnosed with PCa during the study period, resulting in 253 participants in the current analyses.

Table 1 presents the demographic and clinical characteristics of the users of the website. Users were primarily white (74.3 %), college educated (68.3 %), and had been screened for PCa in the past 12 months (72.3 %). The significant demographic differences between website users vs. non-users have been described in detail elsewhere [29]. Briefly, compared to non-users, users were more likely to have at least a college degree, to be white, and to have a higher income.

Characteristics of Website Use

Table 2 presents the distribution of minutes spent on the website, number of sections visited, number of testimonials viewed, and number of values clarification questions answered. Due to the skewed distribution of time spent on the website, we categorized each of the website use variables. We found virtually identical results when the website use variables were continuous.

Total Time on Website

We created a two-level categorical variable of low users [<30 min (N = 101, 39.9 %)] and high users [≥30 min (N = 152, 60.1 %)]. The mean time spent was 34.3 min (SD = 17.6) and the median was 33.7 min. Results were comparable when categorized as a median split (33.7 min) and also in three levels: <30, 30–45, or >45 min. Thus, we used the most straightforward categories of </≥30 min.

Number of Website Sections

Based on the distribution of the number of sections visited, we classified this variable into low users [1–3 sections visited (n = 43; 17.0 %)] and high users [4–5 sections visited (n = 210; 83.0 %)]. When we categorized the number of sections into three levels (1–3 sections, 4 sections, and 5 sections visited), we found similar results to the two-level variable and thus elected to use the simpler two level variable.

Testimonial Videos

Based on the distribution of the number of videos viewed, we classified participants as low users [0 videos (n = 61, 24.1 %)], medium users [1–5 videos (n = 116, 45.8 %)], and high users [6 videos (n = 76, 30.0 %)]. When we classified the number of testimonial videos viewed as two levels (0 videos vs. 1–6 videos), we found similar results but elected to use the three-level variable in order to maintain the separation of users at each extreme.

Values Clarification Tool Use

Complete users of the values clarification tool answered all 10 values clarification tool questions and visited the values clarification tool summary page. Incomplete users answered less than 10 questions or did not visit the summary page. Over 84 % visited the values clarification tool summary page. We classified values clarification tool use into complete users (n = 197, 77.9 %) and incomplete users (N = 54, 21.1 %).

Descriptive information on the outcome variables

We present descriptive information for the four outcomes at each assessment (Table 3).

Table 3.

Descriptive information for outcomes measures for all web users

	N	Website users
PCa knowledge mean (SD)
Baseline	253	11.2 (2.8)
1 month	251	15.4 (2.5)
13 months	236	14.2 (2.8)
Decisional conflict
Baseline (% any conflict)	252	45.5 %
1 month (% any conflict)	252	26.9 %
13 months (% any conflict)	237	26.1 %
Decisional satisfactiona
1 month (% high satisfaction)	231	51.8 %
13 months (% high satisfaction)	215	47.8 %
Self-reported screening (PSA + DRE) at 13 months	234
No		37.68 %
Yes		62.4 %
Don’t know		3
Missing		16

^aDecisional satisfaction was not assessed at baseline

Multivariable outcome models

PCa Knowledge (N = 246)

Linear regression analyses with GEE revealed that, averaged across both follow-up assessments, greater website use was associated with significantly higher knowledge scores: (1) ≥30 vs. <30 min on the website [B = 0.91 (95 % CI = 0.40, 1.42), p < 0.001]; (2) visiting 4–5 sections of the website vs. 1–3 sections [B = 0.92 (95 % CI = 0.21, 1.63), p < 0.05]; (3) watching 6 testimonial videos vs. 0 videos [B = 1.38 (95 % CI = 0.71, 2.06), p < 0.001] and watching 1–5 videos vs. 0 videos [B = 0.92 (95 % CI = 0.33, 1.51), p < 0.01]. There was no significant difference between watching 6 videos vs. 1–5 videos [B = 0.42 (95 % CI = −0.12, 0.97), p = 0.13] and (4) complete use of the values clarification tool vs. incomplete use [B = 0.74 (95 % CI = 0.15, 1.33), p < 0.05].

Satisfaction with Decision (N = 238)

Multivariable logistic regression analyses with GEE revealed that, averaged across both follow-up assessments, greater website use was variable in its association with satisfaction: (1) high satisfaction with the screening decision was not significantly associated with high vs. low website viewing time [OR = 0.87 (95 % CI = 0.54, 1.39), p = 0.55]. (2) However, visiting 4–5 vs. 1–3 sections of the website was associated with being twice as likely to report high satisfaction [OR = 2.05 (95 % CI = 1.09, 3.87), p < 0.05]. (3) Watching 1–5 vs. 0 testimonial videos [OR = 0.73 (95 % CI = 0.40, 1.32), p = 0.30], 6 vs. 0 videos [OR = 0.73 (95 % CI = 0.39, 1.37), p = 0.32], or 6 vs. 1–5 videos [OR = 0.97 (95 % CI = 0.58, 1.63), p = 0.92] was not associated with higher satisfaction. (4) However, complete vs. incomplete users of the values clarification tool were twice as likely to report high satisfaction [OR = 2.02 (95 % CI = 1.13, 3.60), p < 0.05].

Decisional Conflict (N = 252)

Logistic regression analyses with GEE revealed that lack of decisional conflict was not significantly associated with (1) high vs. low viewing time [OR = 1.19 (95 % CI = 0.72, 1.96), p = 0.50] or (2) visiting 4–5 vs. 1–3 website sections [OR = 1.61 (95 % CI = 0.85, 3.07), p = 0.15]. (3) Lack of decisional conflict was also not associated with watching 1–5 vs. 0 testimonial videos [OR = 1.00 (95 % CI = 0.53, 1.90), p = 0.99], 6 vs. 0 videos [OR = 0.78 (95 % CI = 0.39, 1.54), p = 0.47], or viewing all 6 videos vs. viewing 1–5 [OR = 0.76 (95 % CI = 0.42, 1.35), p = 0.35]. (4) Complete use of the values clarification tool did not differ from incomplete use [OR = 1.57 (95 % CI = 0.90, 2.79), p = 0.11] with respect to their odds of having decisional conflict.

PCa Screening (N = 234)

At the 13-month assessment, 62 % of participants reported having been screened (defined as either a PSA or DRE) since randomization. Logistic regression analyses revealed that a greater likelihood of PCa screening was not significantly associated with any of the predictor variables: (1) high viewing time vs. low [OR = 1.70 (95 % CI = 0.94–3.09), p = 0.08], (2) 4–5 sections vs. 1–3 sections [OR = 1.65 (95 % CI = 0.79–3.46), p = 0.19], (3) testimonial viewing [1–5 vs. 0 videos—OR = 1.23 (95 % CI = 0.59, 2.55), p = 0.58; 6 vs. 0 videos—OR = 1.58 (95 % CI = 0.71, 3.50), p = 0.26; 6 vs. 1–5 videos—OR = 1.30 (95 % CI = 0.65, 2.57), p = 0.46], or (4) complete values clarification tool use vs. incomplete use [OR = 1.29 (95 % CI = 0.65, 2.54), p = 0.47].

DISCUSSION

This prospective analysis of the association between men’s website use and informed decision-making outcomes for PCa screening provides important information regarding the influential aspects of web-based DAs. In adjusted analyses, all measures of website use were positively associated with greater knowledge. Further, we found that men who viewed more website sections or who used the values clarification tool reported greater satisfaction with their screening decision. These results confirm the feasibility of using web-based decision aids to convey important health information and the association of interactive website features with long-term decisional outcomes.

Greater engagement with specific website features was associated with increased knowledge about PCa screening, compared to participants who used the same features sparingly or not at all. Earlier studies have shown similar results by comparing users vs. non-users of web-based decision aids [22–28]. However, the current analyses add to existing literature by using tracking software to determine the particular aspects of the website that contributed to increased knowledge. Tracking participants’ website use revealed that greater use of key features of the website (e.g., video testimonials, values clarification tool) was associated with greater PCa knowledge. Additionally, our results showed that spending more time overall on the website was positively associated with greater knowledge, a feature that was not available in prior studies that categorized participants only as users or non-users of the decision aid [22–28].

This study also examined men’s satisfaction with the screening decision after using a PCa screening web-based decision aid, finding that more interaction with the website was associated with a greater likelihood of having high satisfaction with the screening decision. In particular, complete use of the values clarification tool and viewing more informational sections of the website were associated with high satisfaction with the screening decision. Total time spent on the website was not a significant predictor of satisfaction with decision, suggesting that specific content in the informational sections may have been important in men’s satisfaction with the screening decision, more than just the amount of time on the website. Future research should focus on determining the facets presented in decision aids that are most relevant to the outcomes.

Although men’s website use was positively associated with improved knowledge and satisfaction, decisional conflict and screening behavior were not significantly associated with website use. The lack of variability in decisional conflict may have limited the ability to find significant associations with website use. Further, the long follow-up period between website use and assessment of screening outcomes may have attenuated this association [21].

These results should be considered in light of study limitations. The differences in PCa knowledge scores due to website use, though significant, were modest. Second, we cannot conclude that greater website use caused improved knowledge and satisfaction. Although we adjusted for the baseline assessment of knowledge, decisional conflict, and screening (satisfaction was not measured at baseline), and website use preceded the outcome assessments by several weeks to 13 months, without a randomized trial that varied website exposure variables, we cannot definitively conclude that greater website use was causal. Although it is unlikely that the causal pathway could have operated in the opposite direction, we have been careful not to imply causation. Finally, of the participants in the web arm, 47.9 % logged onto the website. Reasons for not logging on included illness or lack of time before the 1-month assessment [21]. Although this is a usual finding for web-based studies [24, 28], it does indicate that web-based decision aids continue to reach only a portion of eligible participants. It is also important to note that website users were more likely to be white, college-educated, have a higher income, and to have been screened for PCa in the 12 months prior to enrollment. While we have controlled for these factors in our current analyses, these significant demographic differences speak to the larger issue of the need to expand the availability and usability of web-based decision aids for groups who may not have access to this information. Previous studies have shown evidence that incorporating web-based decision aids into primary care appointments can improve rates of decision aid use [25, 27, 41].

The primary strengths of this study include the detailed assessment of the associations between website use and decision-related outcomes, as well as the long-term follow-up of a large sample of web users. This is one of the first studies to capture participants’ use of a web-based decision aid, including the extent of use of the website components. Additionally, our use of the interactive values clarification tool may have facilitated more thoughtful engagement with the materials, as participants were asked to consider their own values about screening. Since greater use of the values clarification tool was associated with increased knowledge and more satisfaction, developing methods to increase engagement may further improve these outcomes. The Science Panel on Interactive Communication and Health noted the benefits of interactive elements in health communication by helping individuals better understand their health-related values and decisions in an engaging way [42]. Though a recent paper has found no direct benefit of using a values clarification tool for heart disease-related decision making [43], several recent studies in cancer-related decision making have found that both breast cancer and PCa patients have better outcomes, such as less decisional conflict, when using media with interactive elements versus informational materials with no interactivity [28, 31–33]. Additionally, a recent paper assessing participants’ use of a web-based smoking cessation tool found that increased use of interactive website features and more frequent use of the website contributed to sustained smoking cessation [34]. Increased interactivity may play a key role in using the Internet to improve decision-making outcomes [20], such as expanding the use of a values clarification tool or expanding opportunities for patients to provide feedback throughout the website [44], such as brief quizzes about PCa facts.

While the website reflected the most up-to-date information during the study period, there have been many changes in the PCa screening literature, including the publication of the PLCO and ERSPC trial results and the USPSTF recommendation against screening [8]. To reflect these changes, we have recently updated the website in 2014. There have also been changes in the ways that people access the Internet, such as through tablets and smartphones, that makes web use easier and more convenient in some cases. While our website preceded the ubiquity of tablets or smartphones, current research is assessing the impact of these alternatives for using decision aids.