Randomized Trial of a Patient-Centered Decision Aid for Promoting Informed Decisions about Lung Cancer Screening: Implementation of a PCORI Study Protocol and Lessons Learned

Abstract

Purpose:

We describe the methods, stakeholder engagement, and lessons learned from a study comparing the a video decision aid to standard educational materials on lung cancer screening decisions.

Methods:

The study followed rigorous methodology standards from the Patient-Centered Outcomes Research Institute. The importance of patient-centeredness and patient/stakeholder engagement are reflected across the study’s conceptualization, execution, interpretation, and dissemination efforts. Advisory groups of current and former smokers, quitline service providers, clinicians, and patient advocates were formed for the project. The study used both retrospective and prospective recruitment strategies. Randomization of patients occurred within state-based quitline, with aggressive tracking of participants. We collected data at baseline and 1-week, 3-month and 6-months after receiving the intervention. The patient-centered outcomes included whether patients’ receiving the decision aid a) felt better prepared to make a decision, b) felt more informed about the screening decision, c) had more clarity on their values regarding the benefits and harms of lung cancer screening, and d) were more knowledgeable about lung cancer screening than patients receiving the standard education materials. Exploratory outcomes included making an appointment with a health care provider to discuss screening, scheduling and completing lung cancer screening.

Results:

We have enrolled and randomized 516 quitline patients and learned many lessons. about executing the trial based on significant patient and stakeholder engagement.

Conclusions:

Conducting patient-centered outcomes research requires new ways of thinking and continuously checking-in with patients/stakeholders. The engagement of quitline service providers and patient advisors has been key to successful recruitment and dissemination planning.

ClinicalTrials.gov NCT ID: NCT02286713

1. INTRODUCTION AND BACKGROUND

Lung cancer is the second most common cancer and the leading cause of cancer deaths in the U.S. (1). Five-year survival rates are only about 16.6%, in part because many patients have advanced disease at the time of diagnosis (2). Smoking is the most important risk factor for developing and dying from lung cancer and is thought to cause about 90% of all lung cancers in the U.S (1, 3).

In June 2011, the National Lung Screening Trial (NLST) published its primary finding where 20% fewer lung cancer deaths were observed among current and former heavy smokers screened with low-dose computed tomography (LDCT) compared to those screened with standard chest x-rays (4). Current estimates suggest that over 12,000 lung cancer deaths would be prevented each year if heavy smokers were screened annually (5). Yet, LDCT for lung cancer screening is not without risks, most notably, radiation exposure and a high false-positive rate leading to subsequent follow-up and testing with its own associated harms. For the individual, the benefit of being screened for lung cancer must be weighed against the potential harms: false positives, radiation exposure, and overdiagnosis (4, 6). A recent systematic review on the benefits and harms of LDCT screening for lung cancer confirmed the results of the NLST while raising concerns about the potential harms of screening (7).

The use of patient decision aids to support informed decision making for lung cancer screening with LDCT is required by the Centers for Medicare & Medicaid Services (CMS) (8). Evidence-based guidelines have been released endorsing LDCT scans for high-risk smokers, while emphasizing the importance of making an informed decision about screening within the context of receiving smoking cessation services for people who continue to smoke (7, 9). The U.S. Preventive Services Task Force (USPSTF) recommended lung cancer screening with LDCT annually and emphasized the need for patients making informed screening decisions (10). In the CMS national coverage determination memo, lung cancer screening with LDCT is covered as a preventive service, thereby increasing the pool of high-risk individuals able to receive scans without copays (8). There is an acute need for quality, evidenced-based patient decision support tools to help patients make informed screening decisions.

Here we describe a patient-centered outcomes research (PCOR) study of a video-based patient decision aid for lung cancer screening used with high-risk smokers recruited from state-based tobacco cessation quitlines. The study is funded by the Patient-Centered Outcomes Research Institute (PCORI) and reflects the importance of patient-centeredness and stakeholder involvement through its conceptualization, execution, interpretation, and dissemination efforts. We describe how we address PCORI’s mandate for PCOR.

2. METHODS

2.1. Study Aim and Hypotheses

The aim of this study is to compare a video-based decision aid to standard educational materials on quitline patients’ preparation to make a decision about lung cancer screening. The goal of the video-based decision aid is to prepare patients to have a conversation with their primary care provider and not to sway patients to be for or against lung cancer screening. We hypothesized that high-risk smokers eligible for lung cancer screening who receive the decision aid would: a) be more prepared to make a decision about lung cancer screening; b) feel more informed about the screening decision, and c) have more clarity on their values regarding the benefits and harms of lung cancer screening with LDCT compared to patients receiving standard educational materials. We further hypothesized that eligible, high-risk smokers who viewed the decision aid will be more knowledgeable about lung cancer screening than patients receiving the standard educational materials. We will also collect data about screening intentions and completion of screening for exploratory purposes, but offer no a priori hypotheses about these latter outcomes because the literature on the impact of decision aids on cancer screening uptake are mixed (11).

2.2. The Patient-Centered Outcomes Research Institute (PCORI)

This project is funded by a contract from the PCORI (www.pcori.org). PCORI is an independent, non-profit, non-governmental organization authorized by the U.S. Congress in 2010 through the Patient Protection and Affordable Care Act. PCORI’s mission is to help people make informed healthcare decisions, by producing high-quality comparative effectiveness research guided by patients, caregivers, and the broader healthcare community. Described as “research done differently,” hallmark qualities of PCOR supported by PCORI include significant patient and stakeholder engagement throughout all phases of the research process and careful attention to methodology standards (12). Our approach to addressing these two aspects of PCOR in our study are described here.

2.2.1. PCORI Methodology Standards

PCORI has developed a series of methodology standards that serve as recommendations for researchers to meet requirements for PCOR best practices (12). The standards address inconsistencies and deficiencies in how methods are applied in PCOR, and promote transparency in reporting study protocols and findings. The 45 standards fall into 11 categories, with five cross-cutting standards that apply to most PCOR studies. The cross-cutting areas include standards for 1) formulating research questions, 2) patient-centeredness, 3) data integrity and rigorous analyses, 4) preventing and handling missing data, and 5) addressing heterogeneity of treatment effects.

Appendix Table 1 lists the methodology standards for the five cross-cutting areas, along with the strategies used in this study to address the standard. These strategies are described in the following sections.

2.2.2. Patient and Stakeholder Engagement: Advisory groups

Our engagement strategy addressed the Methodology Standards area about patient-centeredness (Appendix Table 1). In order to ensure that the study aims, methods, and intervention (i.e., the patient decision aid) would benefit and reflect the needs of all end users, we sought guidance from stakeholders representing patients, primary care clinicians, advocacy groups, and quitlines (including the state agencies that fund quitlines and the service providers that operate them). The Patient Advisory Group was composed of five patients and a patient advocate who served as members of the research team. We used the term patient investigator to clarify and reinforce their role as members of the research team as opposed to research subjects. The patient investigators were selected from patients who had participated in the Tobacco Treatment Program at MD Anderson and provided feedback on the prototype decision aid and from research participants who participated in prior community-based smoking cessation studies. The patient advocate, also part of the Stakeholder Advisory Group, was an asset to the group by providing advisement on literacy considerations and issues surrounding equity in access.

The Patient Advisory Group (PAG) met one to two times per year in-person, supplemented by quarterly email/postal mail updates and phone correspondence as needed. We found this frequency and varied modalities to be ideal for gathering initial feedback on the decision aid and other patient educational materials as well as keeping the PAG updated on the study and address emerging concerns as the study progresses. The balance of face-to-face meetings and email/phone interaction was also practical and respectful to our PAG group, which consists of busy community members who work full time across a sprawling metropolitan area.

The group has already contributed to the refinement of the prototype decision aid, providing guidance on content and its use in the clinic visit and offering suggestions for additions to the aid (e.g., course of screening, lung cancer symptoms, insurance coverage for screening). These suggestions have been valuable to its final design. They were also involved in the trial; reviewing and providing feedback on study documents and supplemental educational materials in addition to monitoring recruitment and study progress. We shared and discussed early preliminary data with the group, including demographics and baseline lung cancer screening knowledge questions. As data collection was completed, they worked with the Stakeholder Advisory Group in dissemination planning of the study results and the decision aid for continued use.

The Stakeholder Advisory Group represented non-patient stakeholders, including primary care clinicians, state quitlines, and the American Cancer Society. Stakeholder Advisory Group members were also involved in the refinement of the decision aid and planning for the dissemination of the study results and the decision aid. Meetings with the Stakeholder Advisory Group was via web conferencing (2 in years 1 and 2) and email contacts. In the final year of the project, we met more frequently (up to 4 web conference calls) with the Stakeholder Advisory Group to discuss how best to make the products of the research available on national levels and engage other stakeholder groups in dissemination. We also met with also plan the Stakeholder Advisory Group in person during the North American Quitline Consortium (NAQC) Annual Conference.

2.3. Study Setting: State Quitlines

We used a multi-prong strategy to identify state quitlines to participate in the study. We contacted both states departments of health and quitline service providers because quitline service providers contract with states to run their state quitlines. We developed multiple partnerships with quitline service providers and states through partnering with the NAQC and attended their annual conference to meet and recruit new partners in person. States and quitline service providers were directly involved with participant outreach and recruitment. Each state operates differently with their quitline service providers and have different requirements. Therefore, funding, IRB approvals, and recruitment procedures had to be customized for the requirements for each state and quitline service provider.

2.4. Study Design

The study is a randomized controlled trial to assess the effectiveness of a video-based patient decision aid compared to standard education materials for lung cancer screening with LDCT. Figure 1 depicts the study design and assessment schedule. After the baseline assessment, participants were randomized to receive the patient decision aid or standard educational materials using a computer-generated randomization schedule with sealed envelopes containing each assignment. Randomization was done within each state organization (i.e., pre-stratification by state quitline) to control for any systematic differences in services provided to quitline patients. The design included an assessment at 1-week to collect data on the outcomes for the study and 3-month and 6-month follow-up assessments to determine how well patients retained the information and track any use of screening services.

Figure 1. Schema for the Randomized Trial.

^aAt the baseline assessment, participants completed a 4-item knowledge measure. At the follow-up assessments, participants completed the full knowledge measure.

2.5. Intervention decision aid: “Lung Cancer Screening: Is it right for me?”

A previous version of a video-based patient decision aid about lung cancer screening (13) was updated for the study to reflect the changes in guidelines from ACS, American College of Chest Physicians, American Society for Clinical Oncology, and National Comprehensive Cancer Network (9, 14–16). The video-based decision aid was designed to be neutral, neither encouraging nor discouraging lung cancer screening. The video-based decision aid was refined and updated with significant input from patient, primary care clinicians, and tobacco cessation experts. The aid content was iteratively refined following cognitive testing and usability testing (Figure 2). These changes included shortening the lengh to six minutes from ten minutes, allowing the video to be accessible on the web and on DVD, clarifying the screening eligibility criteria, providing instructions on how to calculate pack-years, and contextulizing radiation exposure in comparison to other sources of radiation.

Figure 2. Steps Taken to Update the Prototype of the Patient Decision Aid “Lung Cancer Screening: Is it right for me?”.

The process of updating the prototype of the patient decision aid involved input from stakeholders and iterative refinement following cognitive testing. Abbreviations: UT, University of Texas.

For cognitive testing, ten smokers from MD Anderson’s Tobacco Treatment Program were shown the updated content in PowerPoint slides and were asked to participate in “think aloud” exercises. They viewed the content and described what the content was trying to convey. Because the aid included new features (e.g., how to calculate pack-years) and updated content, we conducted usability testing with 30 patients from the Tobacco Treatment Program to assess acceptability of its length, clarity of information, and balance of the information provided. We also completed external peer review of the updated aid with three experts in decision aids and/or lung cancer screening.

2.6. Participant Eligibility and Sampling

Eligible participants included quitline patients aged 55–77 years, with a 30 pack-year smoking history, and who speak English. Eligibility criteria for the proposed study follows the high-risk categorization from the NLST that was endorsed by the American Cancer Society and the multi-society collaborative guideline published in 2012 (7, 9). State-based quitlines identified potential participants from new referrals for smoking cessation services, including self-referrals and patients referred by primary care physicians and other health providers.

For patients 55 to 77 years of age, the intake staff members assessed their interest in learning about lung cancer screening. Interested patients were sent recruitment materials, which included a toll-free number, an email address, and a postcard that patients could use to contact the research team about participating in the study. In addition to the recruitment materials, patients were sent a QuitKit that includes information about different kinds of services provided (e.g., phone counseling, web-based information), how to access the services, and other resources available to patients to help them quit smoking. Additionally, some of the state-based quitlines sent out recruitment materials to those who had called the quitline over the past year and met age eligibility criterion.

Once potential study participants contacted the study team, the research staff provided more information about the study, assessed eligibility, obtained verbal consent using a verbal consent script, and collected the study participants’ demographic information. Research staff instructed the study participants to review the study materials once they received them in the mail. Intervention study participants were given the option to view the decision aid either on a DVD or the internet. If these options were not accessible, the research staff assisted participants in finding a location where they could have access locally (e.g., public libraries).

2.8. Measures and Data Sources

Documenting the use of validation scales is a PCORI methodology standard in the area of data integrity and rigorous analyses (Appendix Table 1).

2.8.1. Preparation for Decision Making

Preparation for decision making was assessed with the Preparation for Decision Making© Scale (PrepDM) (17). This scale assesses a patient’s perception of how useful a decision aid or educational intervention are in preparing the patient to communicate with their health care provider about making a screening decision (17). The scale discriminates well between patients who do and do not find a decision aid helpful in making an informed choice. Internal consistency reliability of the scale is excellent, with alpha coefficients ranging from 0.92 to 0.96.

2.8.2. Informed and Values Clarity Subscales of the Decisional Conflict Scale© (DCS)

The Informed and Values Clarity subscales of the DCS (18) was used to assess decision quality. The DCS Informed subscale assesses smokers’ awareness, advantages, and disadvantages of options, while the DCS Values Clarity subscale assesses the importance of the advantages and disadvantages of the options. The traditional DCS has been validated with Canadian and U.S. patients for a variety of health care decision contexts and has excellent internal consistency reliability, discriminant validity, and construct validity (19).

2.8.3. Knowledge

Nine items from the LCS-12 (20), a knowledge measure developed by the research team, was administered at 1-week, 3-month, and 6-month follow-up assessments. Additional items were added to the follow-up data collection times. Thus, the nine items were available across all data collection times. The scoring involved computing the percentage of questions answered correctly at the 1-week, 3-month, and 6-month follow-up assessments.

2.8.4. Intentions to be Screened and Screening Behaviors

At the 1-week follow-up assessment, participants were asked “How likely is it you will be tested for lung cancer with a CT scan this coming year?” Screening behaviors were assessed at the 3-month and 6-month follow-up. Participants were asked if they scheduled or had a visit with their health care provider to discuss lung cancer screening, and if they scheduled or had a LDCT scan since they participated in the study. Additionally, we asked about the date of the screening, the location or facility where the scan was completed, and who referred the patient for screening (e.g., doctor referral or self-referral).

2.8.5. Acceptability

Acceptability of the decision aid was assessed by the Ottawa Acceptability Measure, including ratings of the aid regarding length, clarity, balance of information (21).

2.9. Data Management and Statistical Analysis

2.9.1. Sample Size/Power Justification

The primary analysis will compare the three primary outcomes (Preparation for Decision Making© Scale, DCS Informed subscale and DCS Values subscale) between the intervention and control arms. Since the three primary outcomes are of equal significance, we will compare each of the three primary outcomes at a significance level of 0.017 (0.05/3) using the Bonferroni multiple comparison adjustment.

From previous research with decision aids using DCS as a primary outcome, we estimate that the intervention arm would have a mean DCS Informed subscale or Values subscale score of 25 while the control arm would have a mean DCS Informed subscale or DCS Values subscale of 30 (22). A sample size of 190 in each arm had 80% power to detect a difference in means of 5 using a two group t-test with a 0.017 two-sided significance level assuming a common standard deviation (SD) of 15 on Informed subscale or Values subscale (22). We calculated the effect size the study would be able to detect with the given sample size for preparation for decision making because we had limited preliminary data for this measure. A sample size of 190 in each arm had 80% power for the study to detect an effect size of 0.332 using a two group t-test with a 0.017 two-sided significance level. We expected attrition of up to 20% by the 6-month follow-up, and therefore used a target sample size of 500.

2.9.2. Analysis Plan

Analyses will be done based on intention-to-treat. Participants’ demographic and clinical characteristics at baseline will be summarized using descriptive statistics such as mean, standard deviation, median, interquartile range (IQR), and frequency where appropriate. We will apply Student t-test/Wilcoxon test and Kruskal-Wallis test/ANOVA to compare continuous variables between arms, and the chi-square test or the Fisher’s exact test to assess the differences of categorical variables between arms.(23)

As primary analysis, we will perform two-sided two group t-test to compare the differences of the three primary endpoints between the two arms. Similar analyses will be conducted for the knowledge scores. For these endpoints, intention-to-treat analysis will be applied to the participants. PCORI methodology standards address the importance of considering heterogeneity of treatment effects in planning the analyses (Appendix Table 1). We will look at the impact of race/ethnicity (African American vs. Caucasian) on the relationship between the outcomes and lung cancer screening materials using general linear regression. We will also assess the interaction between intervention and race/ethnicity using linear regression model to examine whether or not the decision aid has differential effects between African Americans and Caucasians.

Since the knowledge outcome was assessed at 1-week, 3-months, and 6-months, linear mixed effect models for longitudinal measures (24) will be employed to assess the change in the magnitude of the outcomes over time adjusting for multiple covariates including intervention indicator, age, gender, race, education level, insurance status, and other factors. We may also look at the interaction between intervention and time. Appropriate transformation of the measures will be considered in the analyses to satisfy the normality assumption of linear mixed effect model. Mode of administration and completion of intervention/control materials (i.e., dose/exposure) will be tracked to account for their effects.

Finally, logistic regression analyses will be used to assess the relationship between screening intentions and behaviors and the intervention group. All analyses will be adjusted for multiple covariates, including patient age, gender, education level, insurance status, quitline service provider, and method of recruitment.

2.9.3. Handling of Missing Data

Strategies for preventing and handling missing data are another methodology standard area for PCORI (Appendix Table 1). The primary data collection method was telephone interviews. Research staff made up to three calls. If the research staff could not reach the participant by the second call during a follow-up assement time window, she/he mailed questionnaires with a postatge paid return envelope. We used a structured tracking system that monitors completion of assessments and triggers scheduling of additional patient contacts when assessments were due.

If there are substantial missing data, analyses will be performed with multiple imputation techniques (25, 26) to generate individual values for missing data fields based upon generalized linear models. The results will be compared with the results of analyses obtained by omitting cases with missing data to serve as a check on the accuracy of the imputation methods.

2.9.4. Avoidance of Bias

Several assumptions were made in the study that will be tested using sensitivity analyses (Appendix Table 1). Exploratory analyses will be conducted to compare the three primary outcomes and knowledge scores between the survey administration routes. We will also conduct the analyses with and without the following covariates to determine their impact on the intervention effects: age, gender, race, insurance status, education, quitline service provider, and recruitment method.

3.0. RESULTS

3.1. State Quitlines

Four quitline service providers covering 13 states participated in the study. Our initial quitline partnerships were with the Alabama Tobacco Quitline and Tennessee Tobacco Quitline operated by Information & Quality Healthcare, which prospectively and retrospectively recruited participants. Then we expanded to the New York State Smokers’ Quitline run by Roswell Park Cancer Institute. They had additional resources, which enabled us to send the recruitment materials directly to the state tobacco quitline and they mailed the materials to those meeting inclusion criteria. We also worked with National Jewish Health and Alere Wellbeing (now Optum) for state tobacco quitlines in Virginia, South Carolina, Washington, Kentucky, Michigan, Pennsylvania, Ohio, Vermont, and Wyoming. A current list of states and quitline service providers is available at http://map.naquitline.org.

3.2. Participant Recruitment

Our initial plan was for Information & Quality Healthcare, the service provider for two states, to estimate the number of people calling the quitline who met the 30+ pack-year history threshold and refer them to the study. This initial step was not feasible for the service providers because it required additional time for callers to complete the assessment, and therefore patients were initially screened for eligibility at the time of intake into the quitline program based solely on age. Research staff assessed pack-year smoking history at the time patients contacted the research team about their interest in the study. To date, 516 quitline patients have been recruited and randomized into the trial.

4. DISCUSSION

4.1. Lessons Learned

There were many lessons learned regarding working with state tobacco quitlines and high-risk smokers (Table 1). By working closely with our states and quitline service providers to get their input on realistic roles and expectations for the project, we were able to meet the needs of states and quitline service providers. As a result, we tailored study procedures for each state or quitline service provider. It also made it easier for us to expand our efforts to multiple states when the quitline service provider operated in multiple states. We were sensitive to vendors competing for contracts in the same states, waiting to work with vendors when their contracts had been awarded. It was essential to work with the states because if they were interested, they would direct the quitline service provider to participate, supporting our partnerships with the quitline service provider.

Table 1.

Lessons Learned During Execution of the Study.

Lesson		Strategy
Recruiting and Working with Quitlines
Engaging with states (as well as service providers) is important		Worked through state departments of health, because they contract with individual providers and can advocate for the project.
Be prepared to work with different quitline service providers because contracts end		Ensured support of state departments of health and involved NAQC.
Allow for multiple participant recruitment methods because each quitline service provider was different		Coordinated quitline services providers to use prospective and/or retrospective recruitment strategies. Prospective recruitment involved quitline service providers giving patients study information when they sought services. Retrospective recruitment involved quitlines mailing study information to patients who recently received services.
Study Design and PCOR Best Practices
Randomization within states important		Randomization was stratified by state.
All participants should have access to the intervention materials		Will send participants who were randomized to the standard education materials, the video-based decision aid at the end of the study.
Many barriers to paying patient advisors as consultants		Provided patient advisors gift cards when meetings were held.
Data Collection and Tracking
Need to track delivery of intervention materials		Used U.S. Mail with delivery tracking to ensure materials have been received.
Plan for call volumes varying across time		Ensured sufficient staffing when recruitment mailings were sent out.
Multiple contact telephone numbers are needed		Maintained tracking database with contact telephone numbers and addresses.
Participants may complete some assessments but not all		Didn’t drop subjects from study if missed an assessment. Tried to recruit at later assessment periods.
Customer Service Approach to Participant Recruitment and Retention
Personal contact with participant matters, increasing follow-up rates	Used telephone interviews as primary data collection method, with mail as back-up. Mailed survey questionnaires to non-responders only.
Return telephone calls quickly	Used a toll-free number, alerted participants about caller ID when we return a call, and returned calls within 1–2 business days. Voice mail was essential to facilitate returning phone calls.
Participants need to be compensated consistent with their time	Adjusted incentive schedule so that $50 was given for the 1-week assessment because it required more time. A $25 incentive was used for other assessments.
Availability for redemption of incentives needs to be considered	Used Walmart gift cards first, but not all participants had easy access. Switched to a generic gift card with a 4–5 year expiration date.
Participants threw away the mailed incentives that came in a blank envelope	Notified and reminded participants that their incentives will be delivered in a white, plain envelope.
Participants sometimes thought the research staff were from the quitlines	Clarified that quitline service providers were not part of the research team.

Abbreviations: NAQC, North American Quitline Consortium; PCOR, Patient-Centered Outcome Research.

By partnering and attending the North American Quitline Consortium (NAQC) national conference, we were able to recruit additional states and quitline service providers which accelerated participant recruitment. During these conferences we were also able to provide technical assistance about the study methods. For example, one quitline service provider expressed concern over the calls they were receiving from quitline clients who were participating in our study reporting missing gift cards. We worked with the quitline service provider to establish our relationship as distinct from the quitline to help reduce calls about the study directed to the quitline. Engagement of quitline service providers has been key to successful recruitment and dissemination planning.

Lessons were learned regarding conducting a large PCOR study working with state quitlines. For one, it was essential to stratify the randomization by state. We also felt that an important aspect of PCOR is equity, meaning all participants should have access to the intervention materials. Depending on institutional policies, it is not always feasible to have patient advisors as consultants. One way to compensate patient advisors is to pay them an honorarium in the form of gift cards, which were delivered in conjunction with a meeting.

Regarding data collection and tracking of subjects, it was essential to use a mail tracking system to ensure that participants received the intervention. We learned that we needed to plan for call volumes to vary based upon when mailings were sent. Participants did not always complete every assessment. As a result, we would try and re-contact the participant at the next assessment.

There were additional lessons pertaining to the recruitment and retention of study participants. A customer service approach was key to enhancing participant recruitment and retention. One strategy was to have research staff available for telephone assessments at times that were convenient for participants, especially given the different time zones across the U.S. Research staff also returned participants’ calls and established a rapport with participants. Study participants could leave voicemails, allowing research staff to respond to any questions or concerns in a prompt fashion. Finally, research staff had to differentiate themselves from the quitline service provider because participants past experience showed that study participants would often mistakenly contact the quitline regarding incentives for participating in the study.

Appendix Table 1. PCORI Methodology Standards Addressed by the Project.

Methodology Standard Categories	Strategy to address standard in the project
Standards for Formulating Research Questions (RQ)
RQ-1 Identify gaps in evidence.	Identified the lack of any patient decision aids about lung cancer screening at the time of the study.
RQ-2 Develop a formal study protocol.	A study protocol was prepared and submitted to ClinicalTrials.gov.
RQ-3 Identify specific populations and health decision(s) affected by the research.	The population includes high risk current and former smokers meeting eligibility criteria from the Centers for Medicare & Medicaid for lung cancer screening. The health decision is whether or not to be screened for lung cancer.
RQ-4 Identify and assess participant subgroups.	High risk smokers are the target population. Subgroups include African American and non-Hispanic white smokers.
RQ-5 Select appropriate interventions and comparators.	The intervention is a video-based patient decision aid. The comparator is an information sheet describing lung cancer screening without presenting it as a decision involving tradeoffs.
RQ-6 Measure outcomes that people representing the population of interest notice and care about.	Smokers need information about lung cancer, screening, their options, and the harms and benefits of screening. Outcomes include feeling prepared to make a decision and being clear about personal values related to the choice.
Standards Associated with Patient-Centeredness (PC)
PC-1 Engage people representing the population of interest and other relevant stakeholders in ways that are appropriate and necessary in a given research context.	Patient Advisory Group: includes five current/former smokers, one patient advocate. Stakeholder Advisory Group: primary care and cancer prevention clinicians, leadership from quitline organizations/service providers, expert in health equity.
PC-2 Identify, select, recruit, and retain study participants representative of the spectrum of the population of interest and ensure that data are collected thoroughly and systematically from all study participants.	Subjects are recruited through state quitlines. Includes multiple states and quitline service providers. Research staff use a systematic approach to retain participants, including a detailed tracking program. Flexibility in mode of data collection is used (mail, telephone).
PC-3 Use patient-reported outcomes when patients or people at risk for a condition are the best source of information.	All data collected in this study is patient-reported.
PC-4 Support dissemination and implementation of study results.	A dissemination involves ongoing input for quitlines through the Stakeholder Advisory Group.
Standards for Data Integrity and Rigorous Analyses (IR)
IR-1 Assess data source adequacy.	Not applicable to this study.
IR-2 Describe data linkage plans, if applicable.	Not applicable to this study.
IR-3 A priori, specify plans for data analysis that correspond to major aims.	Primary endpoints will be assessed using two-group t-tests. Linear mixed effect models for longitudinal measures will be used to assess change in outcomes over time. Analyses will be performed based on intension-to-treat.
IR-4 Document validated scales and tests.	Outcome measures include the Decisional Conflict Scale (18), Preparation for Decision Making Scale (17), Lung Cancer Screening Knowledge (LCS-12) (20).
IR-5 Use sensitivity analyses to determine the impact of key assumptions.	Four primary assumptions are being made: 1) the outcome data are normally distributed; 2) self-report will not differ by mode of administration (mailed, telephone); 3) outcomes will not differ by quitline service provider; and 4) we do not know the impact of exposure to the intervention on the outcomes (patients will be asked how much of the materials they reviewed; it is not feasible to collect objective data on exposure). Sensitivity analyses will be performed to test these assumptions.
IR-6 Provide sufficient information in reports to allow for assessments of the study’s internal and external validity.	We will follow the CONSORT reporting guidelines for reporting on the study’s internal and external validity.
Standards for Preventing and Handling Missing Data (MD)
MD-1 Describe methods to prevent and monitor missing data.	We will systematically track subjects. Subjects will be contacted for follow-up assessments will use multiple methods (mail, telephone). The overall length of the assessments will be addressed to minimize subject burden. Loss to attrition will be monitored following CONSORT guidelines.
MD-2 Describe statistical methods to handle missing data.	We will check validity of accuracy of imputation by comparing with a data set which omits subjects with missingness.
MD-3 Use validated methods to deal with missing data that properly account for statistical uncertainty due to missingness.	We will first record reasons for missing data and check missing-data mechanism. If the missing data are missing at random (MAR), we will use multiple imputation techniques implemented in SAS PROC MI procedure to handle missing data according to type of missing data pattern assuming that the model parameters of the data model and the parameters of the missing data indicators are distinct. If the missing data are missing not at random (MNAR), we will apply methods that add an explicit model for the missing mechanism to the data model.
MD-4 Record and report all reasons for dropout and missing data, and account for all patients in report.	We will maintain a tracking log for all patients assessed for eligibility, enrolled in the study, and lost to follow-up including reasons for dropout.
MD-5 Examine sensitivity of inferences to missing data methods and assumptions, and incorporate into interpretation.	We will conduct sensitivity analysis fo different data-missing mechanisms, data-missing patterns as well as different imputation methods and provide appropriate interpretation of the results.r
Standards for Heterogeneity of Treatment Effects (HTE)
HT-1 State goals of HTE analyses.	The goals of HTE analyses in this study are to examine whether or not the decision aid has differential effects by age, gender or race.
HT-2 For all HTE analyses, pre-specify the analysis plan; for hypothesis-driven HTE analysis, pre-specify hypotheses and supporting evidence base.	We will assess the HTE between different subject groups by testing the interaction between the decision aid and covariate of interest such as race with the main effects of the decision aid and covariate in the models.
HT-3 All HTE claims must be based on appropriate statistical contrasts among groups being compared, such as interaction tests or estimates of differences in treatment effect.	We will use CONTRAST statement in SAS PROC GENMOD procedure to estimate treatment differences within each subgroup if the interaction between the decision aid and covariate of interest is statistically significant.
HT-4 For any HTE analyses, report all pre-specified analyses and, at minimum, the number of post-hoc analysis, including all subgroups and outcomes analyzed.	We will explore the differential effects of the decision aid by age, gender or race by testing the interaction first and then perform subgroup analyses using appropriate statistical contrasts if the interaction is statistically significant.

Note. The relevant PCORI methodology standard is denoted by the abbreviation for each standard and its number. For example, RQ-1 is the first standard for reporting research questions. The full methodology standards can be found here www.pcori.org.