Evaluation of a Personalized, Web-Based Decision Aid for Lung Cancer Screening

Abstract

Introduction

Informed decision making has been highlighted as an important aspect of lung cancer screening programs. This study seeks to assess the efficacy of a web-based patient decision aid for lung cancer screening, www.shouldiscreen.com.

Methods

A before-and-after study (August through December 2014) was conducted where participants navigated a web-based decision aid that provided information about low-dose computed tomography lung cancer screening. Using an established prediction model, the decision aid computed baseline lung cancer risk and an individual’s chances of benefiting from, and risk of being harmed by, screening. Outcome measures included knowledge of lung cancer risk factors and lung cancer screening, decisional conflict, concordance, and acceptability of the decision aid. Data were collected from 60 participants who were current or former smokers, had no history of lung cancer, and had not received a chest computed tomographic scan in the previous year. Analysis took place in 2015.

Results

Knowledge increased after seeing the decision aid compared with before (p<0.001), whereas the score on the Decisional Conflict Scale decreased (p<0.001). Concordance between a participant’s preference to screen and the U.S. Preventive Services Task Force recommendation improved after seeing the decision aid (p<0.001). Risk perceptions among the screen-ineligible group changed (n=49), contrary to those who were eligible (n=11). Ninety-seven percent of the participants reported that the decision aid was likely useful for lung cancer screening decision making.

Conclusions

The web-based decision aid should be a helpful resource for individuals considering lung cancer screening, as well as for practitioners and health systems with lung cancer screening programs.

Introduction

Lung cancer screening (LCS) programs are being implemented across the U.S. following evidence that low-dose computed tomographic screening can significantly reduce lung cancer mortality.¹ Population-wide screening is most efficient if high-risk individuals, who are most likely to benefit, are identified and encouraged to screen, whereas those who are less likely to develop lung cancer are discouraged.

Decision aids (DAs) improve decision quality by helping users understand the pros and cons of available options, decrease decisional conflicts, and potentially prevent underuse or overuse of screening services.^2–5 The Centers for Medicare and Medicaid Services made shared decision making a requirement for LCS reimbursement, recommending the use of one or more DAs to facilitate the shared decision-making process.⁶ The authors’ DA complies with Centers for Medicare and Medicaid Services requirements in terms of content: benefits and harms of screening, follow-up diagnostic testing, overdiagnosis, false positive rate, and total radiation exposure.⁶ However, with LCS being a relatively new screening procedure, there is a paucity of DAs available. To the authors’ knowledge, only one DA has been peer-reviewed to date.⁷ Moreover, current DAs^7–11 do not consider individual characteristics and only provide average risks and benefits of LCS. Precise risk prediction should be a critical part of LCS because clinically important differences in benefit exist even among screen-eligible individuals.¹²

The authors developed a web-based DA that provides individual estimates of lung cancer risk, and screening benefits and harms. They then tested its efficacy with current and former smokers aged 45–80 years.

Methods

Study Sample

An uncontrolled, before-and-after study was conducted with 60 participants to assess the efficacy of the DA (August through December 2014). A sample size of ≥52 was calculated to detect a 20% improvement in knowledge assuming an initial mean of 7.8 (score of 60%), with power of 0.8. Participants were a convenience sample of volunteers who were current/former smokers, aged 45–80 years, with no previous history of lung cancer and no chest computed tomographic scan in the previous year at the time of recruitment. Of all eligible subjects who initially expressed interest in the study, 60% enrolled. The study was approved by the University of Michigan IRB.

Data Collection

Eligible participants were invited to answer a survey administered via Qualtrics (www.qualtrics.com) at the University of Michigan. Participants were asked to complete a “Before” survey and were then redirected to the study website. Subsequently, participants were directed to the “After” survey. Surveys used along with additional details on the development of the DA are described elsewhere.¹³ The International Patient Decision Aid instrument version 4.0 checklist¹⁴ and current risk communication best practices were used in the development.^15–17 The current version of the DA is available at www.shouldiscreen.com.

Measures

Study outcome measures were adapted from the Ottawa Decision Support Framework¹⁸: knowledge of LCS benefits and harms, decisional conflict,¹⁹ and acceptability.²⁰ The ten-item Decisional Conflict Scale is composed of four subscales: Uncertainty, Informed, Values Clarity, and Support. A total score of 0 indicates no decisional conflict, to 100 showing extremely high conflict. Concordance was also measured between the U.S. Preventive Services Task Force (USPSTF) recommendation²¹ and an individual’s preference as assessed by the question Which option do you prefer now in terms of lung cancer screening? Participants who answered I prefer to screen and were also eligible for screening based on USPSTF criteria were deemed “concordant,” as were participants not eligible for screening who preferred not to get screened. Lastly, participants’ risk perception was measured.

Statistical Analysis

Wilcoxon signed-rank test and McNemar’s exact test were used to compare the before/after continuous and binary outcomes, respectively. All analyses were performed in 2015 using Stata, version 13.

Results

The average participant was aged 60.6 years, half were male, 27% were current smokers, and 18% fulfilled the USPSTF’s eligibility criteria for screening (Table 1). The average 6-year lung cancer risk was 0.012 (PLCOm2012 model²²). Average time spent on the study website/DA was 10 minutes.

Table 1.

Characteristics of Participants [Number of Participants (%)]

	Total (N=60)
Agea	60.6 (7.3)
Male	30 (50%)
Race
Black	7 (12%)
White	53 (88%)
Education
Less than high school	1 (2%)
High school graduate	4 (7%)
Some training after high school	4 (7%)
Some college	13 (22%)
College graduate	17 (28%)
Postgraduate or professional degree	21 (35%)
Eligible for screening by USPSTF criteria	11 (18%)
Current smoker	16 (27%)
Quit >15 years ago	30 (50%)
Pack-yearsa	24.08 (23.85)
Have heard about lung cancer CT screening (other than from study)	28 (47%)

^aThe mean and SD were expressed for age and pack-years.

CT, computed tomography; USPSTF, U.S. Preventive Services Task Force.

Table 2 provides before/after changes for knowledge, decisional conflict, and concordance. Knowledge for all questions improved significantly after viewing the DA (p<0.001). Most people were not aware that the majority of lung nodules detected by computed tomography are not cancer. Appendix Table 1 (available online) presents specific items that were asked.

Table 2.

Before and After Results for Knowledge, Decisional Conflict Scale, and Concordance (n=60)

	Before	After
	Mean (SD)
Knowledge (overall)a	7.52 (1.89)	10.93 (2.19)
1. Factors that increase the chances of getting lung cancer [6]	3.77 (0.95)	4.67 (1.05)
2. Possible benefits of lung cancer screening [3]	1.95 (0.77)	2.70 (0.56)
3. Possible harms of lung cancer screening [3]	1.20 (0.99)	2.23 (1.00)
4. Which of the individuals would be eligible for screening based on the age, given that they all meet the smoking status and pack-year criteria? [1]	0.52 (0.21)	0.75 (0.25)
5. What percentage of lumps found on your lung by the CT screening is NOT going to be cancer?b [1]	0.08 (0.28)	0.58 (0.50)
Decisional Conflict Scale (overall)c	46.33 (29.69)	15.08 (25.78)
Uncertainty subscale	55 (40.07)	18.33 (34.71)
Informed subscale	62.22 (39.28)	16.94 (30.91)
Values Clarity subscale	48.33 (41.65)	16.25 (34.08)
Support subscale	23.33 (21.74)	10.28 (21.50)
Concordanced	14 (23.73%)	35 (59.32%)

Note: Boldface indicates statistical significance (p<0.001).

^aThe overall maximum score for the knowledge section is 14. Each specific question’s maximum score is specified in square parentheses.

^bThe figures presented for question 5 are the proportion of participants that answered correctly as there was only one correct answer.

^cLower scores in the Decisional Conflict Scale signify less decisional conflict. A score of lower than 25 is associated with implementing a decision. The overall score is the average of the subscales’ scores.

^dParticipants who preferred to get screened and were also eligible for screening based on U.S. Preventive Services Task Force criteria were deemed “concordant,” as were participants not eligible for screening who preferred not to get screened. The figures reported are based on 59 responses, and represent the frequency and proportion of those who were concordant.

CT, computed tomography.

The mean overall Decisional Conflict Scale score was 46.33 (SD=29.69) prior to viewing the tool, and 15.08 (SD=25.78) after (p<0.001, before/after difference). All subscales showed significant decreases (p<0.001) and were below the threshold score of 25, which suggests that individuals were ready to implement an LCS decision.²³ Among those ineligible for screening (n=49, Table 3), perceptions about lung cancer screening benefits decreased significantly, whereas perceived harms from screening increased. In fact, those who would get screened for lung cancer if it were free decreased by 26% from 46 to 34 individuals (p<0.001, Table 3). On the other hand, risk perceptions among screen-eligible individuals did not change significantly. Note that knowledge improved significantly in both screen-ineligible and screen-eligible groups (data not shown).

Table 3.

Risk Perception, Before and After (n=60)

	Questions	Eligible for screeninga		Not eligible for screeninga
		(n=11)		(n=49)
		Before	After	Before	After
		Mean (SD)		Mean (SD)
1.	In your opinion, how likely are you to die from cancer without CT screening?	5.54 (3.14)	5.36 (3.74)	4.22 (2.69)	3.18 (2.77)**
2.	How worried would you be about dying from cancer without CT screening?	5.27 (3.47)	5.27 (3.04)	3.91 (3.11)	2.90 (2.79)**
3.	In your opinion, how effective is CT screening at reducing your chance of dying from cancer?	7.73 (2.10)	8.27 (1.68)	6.78 (2.24)	5.96 (3.13)
4.	If you were to get CT screening, how likely are you to avoid a cancer death?	6.91 (2.59)	7.36 (2.54)	6.41 (1.94)	5.53 (2.62)
5.	If you were to get CT screening, how likely are you to experience a harm from screening?	2 (1.05)	1.82 (1.17)	1.92 (1.06)	2.78 (1.94)**
6.	If you were to get CT screening, how worried would you be about experiencing a harm from screening?	1.73 (1.19)	1.64 (1.21)	1.69 (0.98)	2.33 (1.63)**
7	Among people getting CT screening, how common do you think harms due to screening are?	2.18 (1.99)	1.82 (0.98)	1.84 (1.21)	2.10 (1.19)*
8	Considering both the risks and benefits of CT screening, how good a choice is CT screening as a way to reduce your risk of dying from cancer?	8.55 (1.81)	8.09 (2.07)	7.76 (2.07)	6.32 (3.20)**
9	If getting CT screening was free for you, would you get screened? (Yes)b	11 (100%)	11 (100%)	46 (93.88%)	34 (69.39%)***
10	If getting CT screening would cost you $50, would you get screened? (Yes)b	8 (72.73%)	9 (81.82%)	32 (65.31%)	26 (53.06%)
11	If getting CT screening would cost you $200, would you get screened? (Yes)b	3 (27.27%)	3 (27.27%)	15 (30.61%)	11 (22.45%)

Note: Participants were asked to evaluate questions 1 to 8 on a 10-point scale where “1” is not likely/worried/effective to “10” being extremely likely/worried/effective. Boldface indicates statistical significance

^*p<0.05,

^**p<0.01,

^***p<0.001).

^aThe average 6-year lung cancer risks (calculated by the PLCOm2012 model) for screen-eligible and screen-ineligible participants were 0.042 (SD=0.035) and 0.006 (SD=0.008), respectively.

^bThe number and proportion of participants who responded “yes” to questions 9–11 are reported.

CT, computed tomography.

Concordance between participants’ preferences and USPSTF recommendations increased significantly (24% to 59%, p<0.001). In particular, among those ineligible for screening, concordance increased from 14% to 53% (p<0.001), as those who preferred to screen decreased by 35% from 31 to 20 individuals (p<0.001; Appendix Table 2 (available online) shows all before/after preferred options). Acceptability of the DA was high. Seventy-seven percent (n=46) found the presentation of information to be balanced; 82% (n=49) thought the DA included enough information to help people make the decision whether or not to screen; and 97% (n=58) found it useful in helping them come to an LCS decision.

Discussion

This web-based DA was highly accepted, improved LCS knowledge, decreased decisional conflict, and raised concordance between USPSTF recommendations and the screening option preferred by the participants.

After receiving personalized estimates of lung cancer risk, screening benefit–harm comparisons, and current guideline recommendations, there were significant reductions in the number of individuals ineligible for screening who stated their preferred option was to get screened, and in those who responded that they would get screened if it were for free. However, there remained 40% (n=20) who preferred to get screened, and 69% (n=34) who answered they would get screened if it were for free even after using the DA. Thus, there may be some trade-offs between catering to individual health decisions, which take into account personal values and risk attitudes, and optimizing LCS value at the population level. Results also highlight the need for shared decision-making conversations between informed individuals and their physicians.

Limitations

History and maturation unlikely played a significant role in the findings given the short duration between pre- and post-test. However, pre-testing could have sensitized participants about various aspects of LCS, artificially enhancing the efficacy of the DA. About 63% of study participants were college graduates or had postgraduate/professional degrees. Therefore, generalizations cannot be made about the acceptability or usefulness of the tool in populations with lower education. Similarly, only 18% (n=11) of study participants were eligible for screening, limiting the generalizability to this group. The study did not ascertain whether the participants who preferred to be screened ended up getting screened.

Because the DA is web-based, it requires patients to have access to a computer or similar technology. Future studies should investigate how viable web-based aids are for populations with low resources, given the known disparities in Internet use to access cancer information.²⁴ Further validation of the efficacy and acceptability of this and other DAs in older, minority, lower-literacy, and at-risk groups is paramount.

Appendix Supplementary data

Supplementary data associated with this article can be found at http://dx.doi.org/10.1016/j.amepre.2015.07.027.