Cognitive Testing of Tobacco Use Items for Administration to Cancer Patients and Survivors in Clinical Research

Abstract

Background

There are currently no standardized measures of tobacco use and secondhand smoke exposure in patients diagnosed with cancer, and this gap hinders the conduct of studies examining the impact of tobacco on cancer treatment outcomes. Our objective was to evaluate and refine questionnaire items proposed by an expert task force to assess tobacco use.

Methods

Trained interviewers conducted cognitive testing with cancer patients age 21 or older with a history of tobacco use and cancer diagnosis of any stage and organ site, recruited at the National Institutes of Health Clinical Center (Bethesda, MD). Iterative rounds of testing and item modification were conducted to identify and resolve cognitive issues (comprehension, memory retrieval, decision/judgment, response mapping) and instrument navigation issues until no items warranted further significant modification.

Results

Thirty participants (6 current cigarette smokers, 1 current cigar smoker, 23 former cigarette smokers) were enrolled from September 2014 to February 2015. Most items functioned well. However, qualitative testing identified wording ambiguities related to cancer diagnosis and treatment trajectory, such as “treatment” and “surgery”; difficulties with lifetime recall; errors in estimating quantities; and difficulties with instrument navigation. Revisions to item wording, format, order, response options, and instructions resulted in a questionnaire that demonstrated navigational ease as well as good question comprehension and response accuracy.

Conclusions

The NCI-AACR Cancer Patient Tobacco Use Questionnaire (C-TUQ) can be utilized as a standardized item set to accelerate investigation of tobacco use in the cancer setting.

BACKGROUND

Assessment of tobacco use and treatment of tobacco dependence among cancer patients is critically important for both patient care and research. Cigarette smoking rates remain high among patients who have been diagnosed with cancer.^1–6 Smoking increases all-cause and cancer-specific mortality and the risk of tobacco-related second primary cancers.^7–10 Some studies have found that smoking may interact with cancer therapy efficacy, that it increases toxicity,^11–14 and that it is associated with poor health-related quality of life.^{15, 16} Other studies have demonstrated that cigarette smoking cessation improves the prognosis of cancer patients.¹⁰

Notwithstanding this existing literature, much remains to be learned about the effects of smoking on cancer treatment and outcomes. For example, it is unclear whether the interaction between tobacco use and cancer therapy is specific to particular treatment modalities or diagnoses. In addition, relatively little research has addressed the extent to which smoking cessation at diagnosis can improve oncology treatment outcomes. A major challenge associated with assessment of tobacco use is its dynamic nature. Smokers can quit smoking before diagnosis, at diagnosis, during cancer treatment or after treatment, but they may also experience tobacco use relapse at any time.^2–6 Longitudinal tobacco use assessment must distinguish patients who smoke during therapy from those who quit smoking upon diagnosis. The increasing diversity of available tobacco products is another challenge for tobacco use assessment. There are currently no standardized measures of tobacco use in patients diagnosed with cancer, and this gap hinders the conduct of studies examining the interaction between use and cancer treatment outcomes.¹⁷ Improved assessment requires valid, standardized measures that capture tobacco use throughout the trajectory of cancer diagnosis, treatment, and survivorship.¹⁷

The National Cancer Institute (NCI) and the American Association for Cancer Research (AACR) formed a task force (see Supporting Information) to develop recommendations for assessing tobacco use by cancer patients. The NCI-AACR Cancer Patient Tobacco Use Assessment Task Force developed patient-reported tobacco use assessment items by selecting and then adapting items from other surveys. This initiative addresses a unique measurement gap because these tobacco items were tailored specifically for use with cancer patients and survivors, and were designed to provide necessary research data on smoking history within the context of cancer care that are not captured by existing tobacco use measures. However, although these items were designed by experts in tobacco and oncology, it cannot be assumed that the items will function well with cancer patients.

Strongly recommended for developing high-quality patient-reported measurement tools,¹⁸ cognitive testing assesses item comprehension, memory retrieval, response difficulty, and accuracy of response among representatives of the target respondent population.¹⁹ Tourangeau developed a cognitive model of the four stages an individual goes through when responding to a survey item: (1) comprehending the intent of the item, (2) retrieving relevant information from memory, (3) judging or estimating the answer retrieved from memory, and (4) mapping one’s answer to the response options given in the item.²⁰ Cognitive testing is now an established step in evaluating the content validity of self-report instruments^{19, 21} for use in patient populations.^22–25 This report presents the formative work and cognitive testing conducted to create, evaluate, and refine the NCI-AACR Cancer Patient Tobacco Use Questionnaire (C-TUQ).

METHODS

Preliminary item development

The Task Force selected and adapted English-language items from tobacco surveys used in the general population (e.g., National Health Interview Study) and in the oncology care and research settings. Tobacco constructs prioritized by the Task Force for inclusion were: current tobacco use/history, tobacco use in relation to cancer diagnosis and treatment, smoking cessation (quitting) history (use of cessation products and assistance), and current and past secondhand smoke exposure. In addition, although the Task Force elected to primarily focus on cigarettes because they have long been the most commonly used tobacco product, constructs such as use of a variety of combustible, smokeless, and aerosol products (e.g., e-cigarettes) were included. Table 2 provides the full list of constructs.

Table 2.

Constructs in C-TUQ

1	Ever smoker status
2	Age first smoked
3	Age began smoking regularly
4	Total duration smoked
5	Average number of cigarettes per day
6	Time since last cigarette
7	Smoking frequency during time periods related to cancer diagnosis and treatment
8	Regular use of other products (combustible, smokeless, and aerosol products, e.g., e-cigarettes) since diagnosis
9	Longest time stayed off cigarettes since cancer diagnosis
10a	Smoking at all in past 30 days
10b	Number of days smoked in past 30 days
11	Use of tobacco cessation products since diagnosis
12	Quit assistance methods since diagnosis
13	Cancer doctors advised to quit
14	Trying to quit in past 30 days
15	Smoking cessation products in past 30 days
16	Quit assistance methods in past 30 days
17	Regular use of other (combustible, smokeless, and aerosol products, e.g., e-cigarettes) products (ever)
18	Other tobacco products in past 30 days
19	Currently living with a smoker
20	Secondhand cigarette smoke exposure in home and work environments in past 30 days
21	Secondhand cigarette exposure in home (ever) and total years
22	Secondhand cigarette exposure in workplace (ever) and total years

The Task Force agreed upon four “Core” constructs essential for baseline assessment in all cancer research settings and two constructs considered essential for follow-up assessments. The remaining “Extension” items provide options for use when more detailed assessment is needed. Before cognitive testing, two survey methodologists revised the items through an expert review process, applying item design best practices^26–28 and results from other cognitive testing studies of similar tobacco-related items in other contexts. The principal investigator (SRL), two survey methodologists, and Task Force members reviewed and finalized the set of constructs to be measured, the item wording and order, response options, and response formats. These items were evaluated and refined through iterative cognitive interviewing rounds.

Participants, setting, and procedures

Participants were cancer patients recruited from the NCI Center for Cancer Research (Bethesda, MD). Eligible patients had received a cancer diagnosis (any stage or organ site, and any stage of oncology treatment and survivorship), were at least 21 years old, and were able to read, write, and speak English. From the convenience sample of 315 individuals scheduled for clinic visits on recruitment days, purposive sampling was used to promote the inclusion of individuals who were current or recent cigarette smokers, and had a broad range of cancer diagnoses, races/ethnicities, genders, and ages. (Details of recruitment are provided in Supporting Information.) Patients provided written informed consent. The protocol and consent forms were approved by the Institutional Review Boards of the NCI and Westat.

Three iterative rounds of testing of 10 patients each were planned in the protocol, to allow for revision and re-testing. The protocol permitted additional rounds if problematic items remained after Round 3. Each interview was conducted by one of four interviewers (see qualifications in Supporting Information) in the clinical setting (e.g., waiting area or examination room). Each interview lasted up to an hour. For each interviewing round of 10 patients, two versions of questionnaire items were created to test alternative phrasing or response formats, and to allow for between-subject comparisons. Items were mapped for applicability to never, former, and current cigarette smokers. Each version was completed by 5 participants.

Cognitive interviewing guides contained the research objectives and a set of scripted verbal probes for assessing item comprehension, memory retrieval, judgment, and response mapping for each item (see Box S1 in Supporting Information). Interviewers were encouraged to selectively present or paraphrase probes to address item comprehension and response accuracy (based on participants’ explanations of their answers). For selected items, probes were included to elicit patients’ understanding of specific terms, including the temporal reference period, ease or difficulty of selecting a response, and understanding of the response format. Interviewers also used emergent probing¹⁹ when a participant described an unanticipated circumstance that related to the explicit research objectives for each item. For Round 1, interviewers administered probes concurrently by asking patients to respond to probes after providing their written response to each item. For Rounds 2 and 3, the probes were administered retrospectively, to better replicate conditions of questionnaire administration in practice, and to assess patient ability to navigate through the questionnaire independently. Each cognitive interview was audio-recorded.

Analytic approach

Based on the open-ended Text Summary Procedure,²¹ an analyst listened to the audio recordings of interviews and extracted notes and quotes into a structured template. To facilitate analysis of any relationship between smoking status and participants’ responses, patient smoking status was also captured in each template. After each round, the interviewers reviewed detailed notes and documented the issues related to comprehension, memory retrieval, judgment, and response mapping. They also noted how many patients experienced each issue identified as a threat to questionnaire validity. Variations in question comprehension or response accuracy were noted. The interviewers recommended revisions (e.g., wording change, format change, item order change). A Task Force subgroup (GW, BT, SL, DH, JO, JC) made final decisions about changes for the subsequent round. After the third round, only very minor revisions were recommended, and testing was concluded.

RESULTS

From September 2014 through February 2015, 35 eligible patients were recruited. Five patients were unavailable due to: time constraints (n=3), hesitant to participate (n=1), or refused participation (n=1). Thirty participants were enrolled and interviewed (50% age ≥60, 67% male, 40% college educated, 83% white; see Table 1). Patients had been diagnosed with cancers of the prostate (n=9), bladder (n=3), lung (n=8), pancreas (n=1), testis (n=1), thyroid (n=1), thymus (n=4), and pleural mesothelioma (n=3). All but 7 patients were being seen for advanced cancer (third-line treatment and/or metastatic disease). The majority of patients were enrolled in a tumor-directed therapy clinical trial (n=21) or an observational study (n=6); the remaining 3 patients were being screened for trial eligibility. The time from cancer diagnosis to the date of the interview ranged from one month to 24 years (median 3 years). Six patients were current smokers at the time of the interview; 23 were former smokers (quit 4–54 years ago); and 1 was a cigar smoker who had never smoked cigarettes. The 6 current smokers and 6 of the former smokers had smoked cigarettes at the time of their cancer diagnosis.

Table 1.

Cognitive interview study participants

Demographic Characteristics		Number (%)
All participants		30 (100%)
Age	20 – 29	1 (3.3)
	30 – 39	0 (0.0)
	40 – 49	1 (3.3)
	50 – 59	13 (43.3)
	60 – 69	8 (26.7)
	70 – 79	7 (23.3)
Gender †	Female	10 (33.3)
	Male	20 (67.7)
Race	White	25 (83.3)
	Black	4 (13.3)
	Asian/PI	1 (3.3)
Ethnicity	Hispanic	1 (3.3)
	Non-Hispanic	29 (96.7)
Education (highest level completed)	High school/GED	7 (23.3)
	Some college	11 (36.7)
	College degree	4 (13.3)
	Post-graduate degree	8 (26.7)
Cigarette Smoking Status	Former (quit at least one year prior to diagnosis)	17 (56.7)
	Former at enrollment; current at diagnosis	6 (20.0)
	Current	6 (20.0)
	Never (current cigar smoker)	1 ( 3.3)
Diagnosis	Prostate cancer	9 (30.0%)
	Bladder cancer	3 (10.0%)
	Lung cancer	8 (26.7%)
	Pancreatic cancer	1 (3.3%)
	Testicular cancer	1 (3.3%)
	Thyroid cancer	1 (3.3%)
	Pleural mesothelioma	3 (10%)
	Thymus cancer	4 (13.3%)

^†The greater proportion of males resulted from recruitment in a prostate cancer clinic.

The content of the questionnaire item set is presented in Table 2. Constructs 1, 4, 5, and 6 yielded the Core items recommended for baseline assessment in all cancer research studies. Constructs 1 and 6 yielded the Core items recommended for follow-up assessments in all cancer research studies. The remaining constructs produced Extension items that are available for inclusion as needed in baseline and follow-up assessments. The final questionnaire is available at the NCI Grid-Enabled Measures Database.²⁹

Cognitive and structural issues identified

The major finding of the study is that most items performed well and generated few difficulties for respondents. However, cognitive testing identified some difficulties with comprehension, memory retrieval, decision/judgment, and/or response mapping for several items. One common area of difficulty arose with items that required an assessment over the patient’s lifetime (e.g., “On average, when you have smoked, about how many cigarettes do you (or did you) smoke a day?” or duration of time living with a smoker). These items were challenging in two respects. First, tobacco use and exposure varied over the lives of the respondents. Several patients recalled one particular interval (e.g., early in life or recently) and responded with respect to that interval only. Items were reworded to provide explicit text about considering the entire lifetime. Second, some patients had difficulty providing numerical estimates of daily consumption or duration of use. Some participants disregarded or misunderstood instructions that required a summation or average, and instead reported the maximum or most recent value. In several cases, patients disregarded or misunderstood an instruction to subtract (e.g., “do not count any time you may have stayed off cigarettes”). Some gave responses in an improper format, such as a date rather than a duration. Item formats were revised to address these issues (e.g., through better labeling of response options). Another revision was to replace the term “smoking” with “smoking cigarettes.” Patients differed in their interpretation of whether “smoking” and “smoker” meant that both tobacco and non-tobacco products (e.g., marijuana) should be considered. Additional recommendations are discussed in the Discussion.

The three other items that posed the greatest difficulties for respondents are presented in Table 3 (Supporting Information). Oncology-specific measurement items primarily relate to the timing of tobacco use and exposure relative to cancer diagnosis and treatment (e.g., Table 3, constructs 7, smoking frequency during time periods related to cancer diagnosis and treatment, and 9, longest time period stayed off cigarettes since cancer diagnosis). Patients were able to easily recall these events and their associated time periods. However, testing revealed that patients had differing interpretations of the term “treatment” in items such as “after treatment ended.” For some patients, treatment began at diagnosis and continued into survivorship, while other patients viewed “treatment” more narrowly, as referring to chemotherapy and/or radiation therapy. “Surgery” was interpreted by some respondents as including biopsy, whereas other respondents thought that surgery referred only to tumor resection that occurred after biopsy. Cognitive testing revealed that patients who have experienced recurrences or second primary cancers may be unsure which experience to consider when responding. One patient had two extended courses of treatment over 11 years, with 3 intervening years in remission when she improved enough to return to work. Items could have applied to her initial diagnosis and treatment or to the more recent episode. For items that referenced diagnosis, we tested variations that provided more specificity, including “Since you were first told you had cancer” and “Since receiving your most recent diagnosis.” Patients understood all of these variations except “most recent,” which confused patients who had received only one diagnosis. Other approaches to mitigate these ambiguities are discussed below. An oncology-specific item that asked about “cancer doctors” (Table 2, construct 13, cancer doctors advised to quit) was considered easy to comprehend, and probing did not reveal inconsistencies.

In addition to cognitive issues, several former smokers overlooked a key skip instruction (to skip items if they had quit smoking more than one year before cancer diagnosis). The skip instructions were made more visually apparent in the revised questionnaire. Items were also re-ordered to minimize the number of skip instructions needed, thereby improving the questionnaire’s ease of use.

DISCUSSION

The Task Force’s instrument development and subsequent cognitive testing produced an item set (C-TUQ) that is comprehensible and has content validity for evaluating tobacco use in cancer clinical research. Patient-reported and patient-centered measures have played an increasingly important role in medical research. Patient input is essential to the development of patient-reported measures.^{30, 31} Our cognitive interviewing findings provide confirmation that the target population of cancer patients are able to complete items as scientifically intended.

A few issues that arose during cognitive testing could not be addressed with item revision in our study. When ambiguity regarding time periods relative to cancer treatment is likely to arise, terms such as “treatment” or “surgery” should be replaced with specific terms to suit the research. For example, for a patient receiving radiation therapy, the term “radiation therapy” should be used instead of “treatment,” and in a research protocol testing a particular chemotherapy agent, the name of the agent may be used. If the research goal is to capture data relative to a particular cancer episode (e.g., the initial or most recent cancer diagnosis), the researcher can modify the wording to specify that episode. Such modifications should be archived and shared with the research community to ensure standardization across studies. The NCI Grid-Enabled Measures Database²⁹ provides a platform for this information. Some of the difficulties we observed with numerical estimation were not addressable within the framework of a short questionnaire, but items that address different periods of life separately might provide more accurate results and free the patient from estimation.³² For example, childhood might be addressed separately from adulthood. However, for our objective of creating a measure that is feasible and scalable across settings, high priority was given to developing a concise instrument that could be readily incorporated into most research studies.

General population measures of tobacco use history employ reference periods such as the past 7 or 30 days. Such items will generally not disambiguate tobacco use before or after diagnosis, surgery, and other events in cancer treatment and survivorship. For example, if a patient in his third month of chemotherapy responds that he has not smoked cigarettes within the past 30 days, it may not be possible for researchers to discern whether he smoked during chemotherapy. “How long has it been since you last smoked a cigarette?” is captured in the proposed new questionnaire as a specific number of days, weeks, or years, which will permit researchers to compare that with dates of events in the patient’s cancer history. In some cases, however, that item alone will not reveal important distinctions. If that patient in his third month of chemotherapy responds that he smoked a cigarette today, he might have smoked regularly during therapy, or abstained until today. In this case, construct 7 (Table 2), which measures smoking frequency during time periods related to cancer diagnosis and treatment, provides the needed information. Similarly, use of other products (combustible, smokeless, and aerosol products, e.g., e-cigarettes) or of cessation assistance during time frames relative to cancer diagnosis is captured in the newly developed instrument. As such, the C-TUQ collects the specific information needed to address under-studied research questions regarding the effect of tobacco use on cancer treatment and other outcomes. On the other hand, conventional 7-day or 30-day abstinence reference periods (as in constructs 10a, smoking at all within past 30 days, or 10b, number of days smoked in past 30 days) may be needed for reporting tobacco cessation in clinical trials³³ and identifying current smokers or tobacco users for referral to tobacco use treatment.

Several caveats should be considered in interpreting these findings. First, this study was conducted at a tertiary care research center. As is typical with cognitive interview studies, the sample size was not large, and may not be representative of all cancer patient or cancer survivor subpopulations. The instrument was completed by a higher proportion of white, male participants with advanced disease (resulting from recruitment in a prostate medical oncology clinic, not patient self-selection). Few current smokers were available for recruitment, although we did include 13 participants who were current smokers at the time of their initial cancer diagnosis (12 who smoked cigarettes; 1 who smoked cigars). Our difficulty identifying current smokers to recruit is consistent with other studies in this setting.³⁴ On the other hand, we achieved saturation (satisfactory measurement performance without need of further revision) in a sample of 30 respondents,³⁵ a majority of whom had educational attainment less than college. This work developed a U.S. English-language paper questionnaire; language translations or an electronic mode of administration will warrant additional testing. The C-TUQ constructs were considered highest priority for research regarding the effect of tobacco use on clinical outcomes of cancer patients. Questionnaires designed for the treatment of tobacco dependence might require additional or different items (e.g., a tobacco dependence measure). Finally, research regarding poly-tobacco use among cancer patients may warrant the development of additional items beyond those included in the C-TUQ.

The current study provides evidence for the content validity of the C-TUQ, which captures tobacco history and use items measured in the context of cancer diagnosis and treatment, from the perspective of current and former smokers. Dissemination and implementation of this questionnaire will greatly facilitate integration of tobacco use assessment, with standardized assessment and common data elements, across cancer clinical research. Use of these items can advance our understanding of how tobacco influences health outcomes across the continuum of cancer treatment.