Mixed-Methods for Comparing Tobacco Cessation Interventions

Abstract

Introduction

The National Comprehensive Cancer Control Program (NCCCP) and National Tobacco Control Program (NTCP) are both well-positioned to promote the use of population-based tobacco cessation interventions, such as state quitlines and Web-based interventions.

Aims

This paper outlines the methodology used to conduct a comparative effectiveness research study of traditional and Web-based tobacco cessation and quitline promotion approaches.

Methods

A mixed-methods study with three components was designed to address the effect of promotional activities on service usage and the comparative effectiveness of population-based smoking cessation activities across multiple states.

Results/Findings

The cessation intervention component followed 7,902 smokers (4,307 quitline users and 3,595 Web intervention users) to ascertain prevalence of 30-day abstinence rates 7 months after registering for smoking cessation services. User characteristics and quit success was compared across the two modalities. In the promotions component, reach and use of traditional and innovative promotion strategies were assessed for 24 states, including online advertising, state Web sites, social media, mobile applications, and their effects on quitline call volume. The partnership intervention component studied the extent of collaboration among six selected NCCCPs and NTCPs.

Conclusions

This study will guide program staff and clinicians with evidence-based recommendations and best practices for implementation of tobacco cessation within their patient and community populations and establish an evidence base that can be used for decision making.

Introduction

Although research comparing the effectiveness of health care interventions and strategies has been conducted for more than a century, the term comparative effectiveness research has taken on new meaning.¹ According to the Agency for HealthCare Research and Quality (AHRQ), comparative effectiveness research (CER) is designed to guide health-care decisions by providing evidence about the effectiveness, benefits, and harms of different treatment options. The evidence is generated from research studies that compare drugs, medical devices, tests, surgeries, or ways to deliver health care.² Although CER has traditionally been used to compare treatment options, clinical trials, tests, and procedures, the opportunities for chronic disease prevention and control programs are only now being recognized. This paper provides the methodology used to conduct comparative effectiveness research in tobacco cessation interventions.

Tobacco use is the leading cause of preventable disease and death in the United States, causing approximately 443,000 deaths annually.³ Tobacco use contributes to pulmonary disease, infertility, birth defects, and cardiovascular disease.⁴ Furthermore, tobacco use is a risk factor for cancers of the bladder, cervix, esophagus, kidney, larynx, lung, oral cavity, pancreas, stomach, and acute myelogenous leukemia.⁵

Two public health programs funded by the Centers for Disease Control and Prevention (CDC) have activities and interventions aimed at reducing tobacco use. The National Comprehensive Cancer Control Program (NCCCP) funds 65 state, tribal, and territorial cancer control programs throughout the country. These programs focus on six priority areas: emphasis of primary prevention of cancer; support of early detection and treatment activities; addressing public health needs of cancer survivors; implementing policy, systems, and environmental changes to guide cancer control; promoting health equity; and demonstrating outcomes through evaluation (www.cdc.gov/cancer/ncccp). CDC established the National Tobacco Control Program (NTCP) to reduce disease, disability, and death related to tobacco use. The goals for comprehensive tobacco control include the promotion of quitting among adults and youths, prevention of initiation among youth and young adults, elimination of exposure to secondhand smoke, and elimination of tobacco-related disparities among different populations (www.cdc.gov/tobacco/tobacco_control_programs/ntcp/index.htm).⁶ This comprehensive approach combines regulatory, economic, educational, clinical, and social strategies.⁷ The NTCP provides funding for state quitlines, which provide telephone-based smoking cessation services, including individualized telephone counseling and self-help material to help smokers quit. Quitlines are cost-effective, population-based interventions that increase successful smoking cessation,⁸ and have the potential of reaching underserved populations.^9,10 Quitline services in all states can be accessed through a toll-free number, 1-800-QUIT-NOW.

This project was developed to compare existing and traditional tobacco cessation interventions with innovative Web-based interventions. This study includes three components (i.e., cessation, promotion, partnership) and was designed with unique study questions and methods for each component of the study.

Methods

Cessation Component 1: Quantitative Study

Study Questions

The cessation component assessed the effectiveness of telephone quitlines versus Web-based tobacco cessation programs by comparing the prevalence of 30-day abstinence that was measured 7 months after enrollment. To date, four studies have specifically compared the two intervention modalities in either insured populations or a single state.^{11,12, 13, 14} This study’s large sample size and multistate collaboration will provide much needed data to compare state-funded telephone quitlines and Web-based tobacco cessation programs. The study questions were as follows: (a) What are the demographic profiles of participants who enroll in a specific intervention (e.g., telephone-based counseling or Web-based intervention)? (b) Do baseline smoking and quitting behaviors of participants differ between those who enroll in telephone-based counseling versus a Web-based cessation program? (c) What are the predictors of successful quitting (e.g., prevalance of 30-day abstinence) for each intervention? and (d) Does quitting success differ significantly among participants who enroll in telephone-based counseling versus those who enroll in a Web-based cessation program?

Study Sample Size

The study aimed to recruit 8,000 participants aged 18 years or older (4,000 quitline users and 4,000 Web-based intervention users) from Alabama, Arizona, Florida, and Vermont. Two studies helped guide the power calculation.^11,12 Dr. Lawrence C. An and colleagues reported a 16 percentage point difference between the 30-day point prevalence abstinence (29% for telephone users versus 13% for Web-based users) at the 6-month follow-up among a sample of Minnesota’s QUITPLAN program (n = 7,049; 4,968 Web site users and 2,351 telephone users). Similarly, Zbikowski and colleagues reported an 11 percentage point difference between 30-day quit rates at the 6-month follow-up among 11,143 quitline users who were using an employee or health-care plan cessation program that included integrated phone and Web-based services. The quit rates for each intervention were 18% for telephone and 7% for Web. We used power and sample size calculations based on methods published by Rosner.¹⁵ Calculations assumed an alpha of 0.05, power of 0.80, equal sample sizes, and an oversample of 50% (anticipating an approximate participation rate of 50%) to determine a 3.0% difference between interventions. This approach was conservative because loss to follow-up rates varied greatly among previous phone-based cessation studies, from 4% to 55%.¹⁶

State programs varied for the contracted program vendor, tobacco control interventions, and quitline call volume to meet sample size requirements. All states that had separate telephone and Web-based interventions, that conformed to National Quitline Data Warehouse standards for intake and follow-up data collection, were eligible to participate.

Data Collection Methods

The study was reviewed by CDC’s Human Subjects Research Protection Office, and Institutional Review Board [IRB] approval was obtained. Selected states received IRB approval before data collection. In compliance with the Paperwork Reduction Act, the data collection was approved by the Office of Management and Budget (Control #0920-0917). We obtained permission for follow-up contact when the participant first registered for tobacco cessation services.

Participants who completed the follow-up survey were offered a $40 incentive as compensation for their time. Intake and follow-up data were provided by state-contracted cessation service companies that collected information for users of both telephone and Web-based interventions as part of their normal monitoring and evaluation activities, or the data were collected by the CDC contractor for this study.

Intake data conformed to the North American Quitline Consortium’s (NAQC’s) Minimum Data Set (MDS) requirements to ensure uniform information collection. ¹⁷ Intake measures for quitline users include demographics, current smoking status, smoking history, current smoking behaviors, previous quit history, self-reported source of referral to the program, and services requested from the program.¹⁷ Similar Web-based intake questionnaires use comparable measures formatted for Web administration.¹⁸

This study component used the standard measure of cessation program effectiveness by recontacting participants 7 months after intake to ask about tobacco use during the past 30 days.^11,19 Follow-up was conducted that entailed contact attempts by e-mail, standard mail, and phone.²⁰ To increase the response of Web-based participants, investigators delivered recruitment follow-up letters by express mail service at the midpoint of the study.

This survey component was based upon the NAQC requirements, and specific questions were added to address study questions. This approach collected information about participants’ satisfaction with the cessation intervention, current smoking status, quit attempts made since their enrollment, duration of quit (if applicable), use of nicotine replacement therapies or medication to help quit, use of various behavioral smoking cessation strategies, and use of technology and various forms of media. The main outcome of interest was prevalence of 30-day abstinence, defined as not having smoked any cigarettes, even a puff, during the past 30 days. Service providers were asked to provide information about the number and length of interactions (e.g., telephone calls or signing into the Web site) completed by telephone and Web-based intervention users. Variables collected are listed in Table 1.

Table 1.

Cessation Component 1 Variables

Variable	Responses
Date of interview
State	Alabama/Arizona/Florida/Vermont
Tobacco use in last 30 days	Yes/No/Don’t know
Type of tobacco use	Cigarettes/Cigars/Pipe/Chewing/Other
Frequency of use	Everyday/some days/not at all/don’t know (quantified for each type of tobacco by uses per day and any-use days in the last 30 days)
Cigarette smoking upon waking	Ordinal, range from “within 5 minutes” to “more than 60 minutes”
Intention to quit within next 30 days	Yes/No/Don’t know (for each type of tobacco use)
Since registering has participant stopped use for ≥24 hours	Yes/No/Don’t know
Cessation product/medication use since registering	Yes/No/Don’t know If yes → type
Other cessation assistance	Advice/Website/Telephone/Counseling/Self-help/Something else
	With write-in field to specify
Time spent on phone/web with quitline staff	Ordinal, range from < 1 min. to > 30 min
Number of counseling sessions or times visited website	Ordinal, range from 1 to > 5
Satisfaction with services	Ordinal, range from “Very Satisfied” to “Not At All Satisfied”
Extent quitline met participant’s needs	Ordinal, range from “Almost all…” to “None…”
If seeking help again, will participant contact quitline?	Ordinal, range from “Yes, Definitely” to “No Definitely Not” (write in for “Why Not”)
Would recommend quitline to a friend	Ordinal, range from “Yes, Definitely” to “No Definitely Not” (write in for “Why Not”)
Does participant go online?	Yes/No/Don’t know
Select where participant uses internet	Home/Work/School/Public Library/Community Center / Someone Else’s House / Some Other Place
Type of internet access	Telephone/Cable/DSL
Frequency of accessing internet	Ordinal, range from “Several Time A Day” to “Less Than Once Per Week”
Use of internet-based cessation activities	Social Networking Site/Read Blog/Posted Comments on a Blog/Used Search Engine To Find Information About Cessation
Type of telephone services used	Landline/Mobile/Mobile With Internet/Mobile Internet Search Engine Used To Find Information About Cessation
Frequency of communication or socialization with friends on a landline/send text messages/talk on mobile phone/instant messages/social networking site messages	Ordinal, range from “Everyday” to “Never”
Others who smoke in household	Yes/No/Don’t know
Marital Status	Standard Responses
Occupational Status	Standard Responses
Highest level of education	Standard Responses
Race	Standard Responses

Analysis and Outcomes of Interest

Descriptive statistics were used to summarize sample characteristics of users who enrolled in each type of intervention (telephone vs. Web-based program), their levels of program satisfaction, and use rates (e.g., number of calls, Web entries, use of program features). Bivariate and multivariate regression modeling was used to assess the association between mode of intervention and successful quitting. The main outcome measured was prevalence of 30-day abstinence (i.e., quitting success) with select secondary outcomes as noted in Table 1. These models adjusted for demographic and baseline smoking characteristics. In addition, we included state indicators in each regression model to capture any latent differences across states that were not explicitly accounted for by the covariates. We used a model-driven approach to identify whether variables significantly affected the outcome while controlling for other factors. Confounding, effect modification, and collinearity were considered where relevant.

Promotional Strategies Component 2: Quantitative Study

Study Questions

The primary objective of the promotional component was to compare the effectiveness of traditional versus innovative promotional strategies to increase telephone quitline call volume. Given the level of funding available for media promotions, as well as the changing media landscape, it is important to understand how states’ usage of innovative promotion efforts, such as digital advertising and social media, affect telephone quitline call volume. This study component addresses the following research questions: (a) What innovative promotional and educational activities are states implementing for cessation in general and for the quitline? (b) What is the reach and use of states’ cessation Web sites and social media platforms? (c) How do audiences reached via innovative media platforms compare with audiences reached via traditional media platforms? (d) What is the comparative effectiveness of traditional media versus online advertising in driving calls to the quitline?

Study Sample Size

We recruited TCPs from the 50 states and U.S. territories to participate. Twenty-four states agreed to participate in the study component. These states were as follows: Alabama, Arizona, Arkansas, California, Delaware, Florida, Indiana, Iowa, Louisiana, Massachusetts, Michigan, Missouri, Nebraska, Nevada, New Hampshire, New York, North Carolina, Oregon, Rhode Island, Texas, Utah, Vermont, Wisconsin, and Wyoming.

Data Collection Methods

Media purchase and placement information for smoking cessation promotional strategies and quarterly quitline call volume data were obtained from each participating state TCP for up to 21 months during October 2011–June 2012. Investigators analyzed this data submitted by the 24 states to identify and describe both traditional and innovative promotional efforts, and to comparatively assess the impact of these efforts on quitline call volume and caller characteristics. There were no eligibility requirements for participation, and states were allowed to select the data items that they submitted to CDC.

Analysis and Outcomes of Interest

Traditional Media Purchase and Placement Data

Each promotional strategy was classified as either traditional (i.e., television, radio, print, or out-of-home) or innovative (i.e., digital, paid search, or social media). Appropriate metrics were identified to measure the level of exposure or reach of each type of advertising. For instance, Gross Rating Points (GRPs), the product of reach times frequency, was used to measure level of exposure to television and radio advertising. The number of impressions, (the estimated number of times a print advertisement or billboard was viewed), was used to measure level of exposure for print advertising. Finally, Click through Rates (CTRs), (the proportion of viewers who clicked on an advertisement out of the total impressions delivered), were used to measure exposure to online banner ads.

Innovative Quitline Call Volume Data and Online Activity

Investigators worked with participating state TCP staff and telephone and Web-based vendors to gather media activities data, Web analytics, and social media metrics. De-identified data on quitline services were obtained from the National Quitline Data Warehouse. Similar to the cessation intervention component, the promotional intervention component used the quitline intake MDS variables to describe caller characteristics and call volume trends associated with specific promotional efforts. For TCPs that promoted their quitline with a state Web site, the reach and use were measured by using the state’s preferred Web analytics platform. If the state did not monitor Web traffic, assistance was provided by Google Analytics^™ (a free Web analytic tool) on their site.

Google Analytics^™ relies on a string of JavaScript code that is inserted into each Web page to track unique visitors on a Web site.²¹ Aggregate traffic metrics for designated periods were then obtained from the platform. Metrics included the number of visits, number of unique visitors, average time spent on the site, number of pages viewed, and geographic region of visitors. For states that use social media to promote the quitline, activity on media platforms (e.g., YouTube^™, Facebook^™, Twitter^™) was tracked by using free public programs, such as Facebook Insight^™.22 These social media tracking programs allow groups or organizations to create a user-friendly dashboard of metrics, including the number of posts on a wall, number of comments, and number of likes, subscribers, fans, and friends for a particular time.

Other contextual factors, such as smoking prevalence for the state, level of smoke-free ordinance coverage (in terms of both local vs. state coverage, as well as types of public spaces covered), and cigarette taxes, were obtained by investigators in aggregate form from the appropriate government sources. Descriptive statistics were used to summarize the overall reach and use of each promotional activity. Bivariate statistics were used to describe callers to the quitline by type of self-reported referral source. In addition, call volume trends were described by promotional activity. A model-drive approach was used to identify significant promotional activity contributions to call volume by using multivariable models. Confounding, effect modification, and collinearity were considered where relevant.

Partnership Component 3: Qualitative Study

Study Questions

The Partnership Component was an effort to document the extent of collaboration among six NCCCP and NTCP states: Alabama, Arkansas, Delaware, Florida, Nebraska, and Vermont. This study questions included the following: (a) What is the level of integration between CCC and TCP programs? (b) In what ways do the CCC and TCP programs collaborate? (c) What are the key factors that facilitate (and hinder) collaboration? and (d) What additional opportunities for collaboration could be used?

Study Sample Size

Six states were selected on the basis of the following inclusion criteria: (a) ability to effectively carry out activities under the CCC Program Cooperative agreement; (b) existing relationships with the TCP program in their states; (c) ability to effectively carry out activities under the TCP cooperative agreement; (d) a history of conducting research; (e) capacity to designate an epidemiologist who can participate in these study activities; (f) existing innovative tobacco cessation activities; and (g) experience collecting data for the National Tobacco Clearinghouse or states with statewide quitline registries. The states selected represented all geographic regions in the continental United States with the exception of the Pacific.

Data Collection Methods

To develop a thorough description of the NCCCP and NTCP programs and their collaborative efforts, data was incorporated into NVivo (Version 10.0) from a variety of sources. These include a document review, which included a systematic review of available organizational charts, Web sites, and state cancer plans. Finally, key stakeholder interviews (via site visits) were conducted by using a semistructured protocol with key health department, CCC, and TCP leadership staff in each of the six states to better understand aspects of each program’s organizational structure, activities, and collaborative efforts around cancer and tobacco control.

Qualitative methods provide flexible in-depth exploration of the participants’ perceptions and experience, and the interviews yield descriptions in the participants’ own words. They allowed the interviewer flexibility to pursue relevant and important issues as they arose during the discussion. By using a grounded theory approach,²³ the researcher becomes immersed in the data, thus allowing for openness to nonforced and nonpreconceived discovery of emergent themes²⁴ and generation of theories based on interpretive procedures.²⁵ The discussion guide included probes to ensure that input was obtained on specific items of interest, and open-ended questions ensured that participants’ responses and perceptions were fully captured. Site visits were conducted from March to July 2012. With the permission of the respondent, interviews were digitally recorded to supplement any information missed by the interviewer’s notes. Individual responses were not linked to participants.

Analysis and Outcomes of Interest

Qualitative data collected from the key stakeholder interviews were organized and analyzed by using NVivo (Version 10.0) software to facilitate the cross-referencing of qualitative data from multiple sources, coding by multiple researchers, and the development of findings for reports. A code list was developed on the basis of the prioritized research domains (i.e., infrastructure, priority of cancer and tobacco control activities, collaborative efforts, cross promotion) and applied to the qualitative data collected. Once codes were developed and all coders were in agreement on what each meant, additional steps were taken to ensure consistent coding and to enhance reliability, including pilot-testing of codes, double-coding, and training of project staff to reliably collect, enter, and analyze the data. To support triangulation, information obtained from each data source (interviews and observations, document review, secondary data sources) was used to verify findings and provide a more accurate description of each program.

Results and Discussion

Quitlines have gained prominence because they have provided evidence of their clinical efficacy, their effectiveness in real-world settings, and because of their potential to make cessation services more universally available. ²⁶ However, research is still needed on the effect of the promotion and use of quitlines on the prevalence of tobacco use in states that have them. Quitlines currently reach only 1% to 2% of the tobacco users in their states per year. ²⁷ An increase in utilization rates may have a substantial population impact on decreasing smoking prevalence. The large spikes in call volume commonly experienced during promotional campaigns also indicate a large untapped demand for services. ²⁶ Consequently, as tobacco control programs try to increase their population impact, new ways to increase public awareness, the use of evidence-based services, and quit rates may all be needed. This study will provide evidence-based information needed by State, Tribal, and Territorial health departments regarding effective cessation modalities and promotional strategies. The findings will help public health agencies further develop and tailor their specific cancer and tobacco programs.

The overall goal for the investment in comparative effectiveness research is to promote high-quality healthcare through broad availability of information that helps clinicians and clients match the best science to individual needs and preferences. ²⁸ However, one of the major challenges in research is ensuring that evidence-based interventions are disseminated to various key audiences, including, but not limited to decision-makers, practitioners, or the public. Therefore, methods for this study include the use of diffusion principles to guide dissemination of the key findings. A dissemination plan based on these principles will be developed to target key audiences such as staff from both the NCCCPs and the NTCP’s, key decision-makers, clinicians, as well as underserved populations.

The strengths of this study include the development of a mixed-methods design utilizing both quantitative and qualitative techniques. The study examines multiple factors that are associated with cessation interventions, and builds upon existing data sources. However, some limitations exist. Comparative effectiveness research is limited by the intrinsic methodologies it utilizes.¹ And as with many studies, self-reported bias or confounders may limit the reliability of data obtained.

Although there is an evidence base for the use of both traditional and innovative cessation interventions, evidence is lacking about which may be more effective. Results from this multi-component study may help to accelerate adoption of findings to all NCCCP and NTCPs. One of the NCCCP-funded program mandates is the development, implementation, and dissemination of a comprehensive cancer control (CCC) plan for each program. Findings from this study may be used to help update existing plans to reflect the adoption of study findings and usage of quitlines versus previous used methods as standard practice. The results of this study may also provide valuable information for clinicians and health care providers who counsel their patients on tobacco cessation. Since data was received from four state health departments from a diverse population across the U.S. region, information on cessation and promotional strategy interventions that is targeted to specific populations (patient and larger community) has potential in increasing tobacco cessation rates in the United States.