Exposure to positive peer sentiment about nicotine replacement therapy in an online smoking cessation community is associated with NRT use

Abstract

Background:

Little is known about the influence of online peer interactions on health behavior change. This study examined the relationship between exposure to peer sentiment about nicotine replacement therapy (NRT) in an online social network for smoking cessation and NRT use.

Methods:

Participants were 3297 current smokers who enrolled in an Internet smoking cessation program, participated in a randomized trial, and completed a 3-month follow-up. Half received free NRT as part of the trial. Automated text classification identified 27,038 posts about NRT that one or more participants were exposed to in the social network. Sentiment towards NRT was rated on Amazon Mechanical Turk. Participants’ exposure to peer sentiment about NRT was determined by analysis of clickstream data. Modified Poisson regression examined self-reported use of NRT at 3-months as a function of exposure to NRT sentiment, controlling for study arm and post exposure.

Results:

One in five participants (19.3%, n = 639) were exposed to any NRT-related posts (mean exposure = 6.5 ± 14.7, mean sentiment = 5.4 ± 0.8). The association between sentiment exposure and NRT use varied by receipt of free NRT. Greater exposure to positive NRT sentiment was associated with an increased likelihood of NRT use among participants who did not receive free NRT (adjusted rate ratio 1.22, 95% CI 1.01, 1.47; p = .043), whereas no such relationship was observed among participants who did receive free NRT (p = .48).

Conclusions:

Exposure to positive sentiment about NRT was associated with increased NRT use when smokers obtained it on their own. Highlighting user-generated content containing positive NRT sentiment may increase NRT use among treatment-seeking smokers.

1. Introduction

With nearly universal Internet adoption (Pew Research Center, 2017) and the growing use of dedicated social networks for health (Centola, 2013), online social networks have become increasingly common sources of “peer-to-peer healthcare” (Fox, 2011). Through online social networks, people learn from the personal experiences of others with similar conditions (Graham, Cobb, & Cobb, 2016). Health information obtained through social networks can influence the decisions people make about coping with chronic conditions, such as decisions about taking medication (Cobb, Mays, & Graham, 2013; Health Research Institute, 2012).

Over twelve million smokers search online for information about quitting each year, (Graham & Amato, 2018) and hundreds of thousands participate in online social networks for cessation (McCausland et al., 2011; van Mierlo, Voci, Lee, Fournier, & Selby, 2012; Wangberg, Nilsen, Antypas, & Gram, 2011; Zhao et al., 2016). Advances in computing methods that allow coding of large volumes of user-generated content have enabled several studies of common topics of discussion in online networks for cessation (Brandt, Dalum, Skov-Ettrup, & Tolstrup, 2013; Burri, Baujard, & Etter, 2006; S Myneni, Cobb, & Cohen, 2016; S. Myneni, Cobb, & Cohen, 2013; Selby, van Mierlo, Voci, Parent, & Cunningham, 2010). Less is known about the sentiment expressed or its influence on cessation-related behavior. To date, most tobacco-related sentiment analysis studies have involved Twitter data, using tweets to describe sentiment towards conventional and emerging tobacco products (Cole-Lewis et al., 2015; Myslin, Zhu, Chapman, & Conway, 2013; Rose, Binns, Buenger, Emery, & Ribisl, 2017) and to survey smoking status and sentiment about smoking (Sofean & Smith, 2013). We are aware of only one sentiment analysis study in an online social network for cessation that examined exposure to sentiment about varenicline and subsequent change in cessation medication preferences (Cobb et al., 2013); however, this analysis did not include behavioral outcome data.

Building upon prior work, this study focuses on whether exposure to peer sentiment concerning nicotine replacement therapy (NRT) influences smokers’ use of NRT. NRT can double the chance of successful cessation and is a central component of tobacco dependence treatment (Fiore, Jaén, Baker, & Tobacco Use and Dependence Guideline Panel, 2008; Stead et al., 2012; Zhang, Cohen, Bondy, & Selby, 2015). However, most smokers do not use NRT (Fu et al., 2008; Hung, Dunlop, Perez, & Cotter, 2011; Soulakova & Crockett, 2017, 2018) and many perceive it to be ineffective and as harmful as cigarettes (Shiffman, Ferguson, Rohay, & Gitchell, 2008). The pros and cons of NRT are often a contentious point of discussion among smokers. Scientific debate about the population-level effectiveness of NRT (Alpert, Connolly, & Biener, 2013; Smith & Chapman, 2014) and related media coverage (Carey, 2012; Kaplan, 2012) periodically fuel such discussions. This study leveraged a unique dataset that included NRT use from smokers participating in a randomized trial and a complete mapping of their exposure to user-generated content and sentiment about NRT in an online social network for smoking cessation. The primary aims of this research were to: 1) characterize the sentiment of user-generated content about NRT; 2) determine if there were differences by participant characteristics in the extent of NRT post exposure or mean sentiment of post exposure; and, 3) examine the relationship between exposure to NRT sentiment and actual NRT use. Previous findings from the parent study (Graham et al., 2017) demonstrated that provision of free NRT increased the use of NRT. As a secondary aim, we took advantage of the study design to examine whether provision of free NRT moderated the relationship between NRT sentiment exposure and actual NRT use.

2. Methods

2.1. Human subjects

Participants were current smokers enrolled in a randomized smoking cessation treatment trial conducted on BecomeAnEX, a free, publicly available Internet cessation program. The trial was conducted from March 2012–January 2015 (ClinicalTrials.gov). All participants provided informed consent. The trial protocol was approved by Western Institutional Review Board (#20110877). These analyses link data on trial participants with the full longitudinal BecomeAnEX dataset that spans 2008–2015 conducted under a study protocol approved by Chesapeake IRB (#00010302).

2.2. Setting

Launched in 2008, BecomeAnEX was developed in collaboration with Mayo Clinic (McCausland et al., 2011) in accordance with national treatment guidelines (Fiore et al., 2008). BecomeAnEX teaches problem-solving and coping skills to quit smoking, educates users about cessation medications, and facilitates social support through a large social network of current and former smokers (Zhao et al., 2016). A national mass media campaign (Graham, Cha, Cobb, et al., 2013; McCausland et al., 2011; Vallone, Duke, Cullen, McCausland, and Allen, 2011) and online advertising have resulted in over 800,000 registered users since the launch of the site. To register on the site, individuals must agree to the Privacy Policy which states that 1) BecomeAnEX collects information about users and their use of the site; and 2) information is used for research and quality improvement purposes only. All user actions are date- and time-stamped. Thus, data from all registered users was available for analysis. Data were stripped of all personally identifiable information, such as email or phone number, prior to analysis.

2.3. Participants & procedures

The trial protocol (Graham, Cha, Papandonatos, et al., 2013), characteristics of the trial sample (Cha, Erar, Niaura, & Graham, 2016), and impact of the intervention arms in increasing treatment utilization (Graham et al., 2017) and abstinence (Graham et al., 2018) have been published elsewhere. Briefly, new registrants on the BecomeAnEX website (WEB) were recruited to test the individual and combined effects of two strategies to improve treatment adherence and cessation outcomes: 1) a social network approach (SN) to integrate study participants into the BecomeAnEX social network via direct outreach from longstanding members who were recruited to the study team; and 2) a 4-week supply of free nicotine replacement therapy (NRT) mailed to participants. Eligibility criteria were US residence, current smoking (every day, some days), age 18 or older, and no contraindications for NRT use per labeling instructions. In a 2 × 2 factorial design, N = 5290 participants were randomized to WEB, WEB+SN, WEB+NRT, or WEB +SN + NRT. Participants had access to BecomeAnEX through the final follow-up at 9 months; however, the SN and NRT interventions occurred prior to the 3-month assessment. Participants were compensated $20 for surveys completed via web, and $15 for surveys completed via telephone.

2.4. Sources of data and measures

Baseline measures include demographic characteristics (age, gender, race, ethnicity, education), cigarettes smoked per day, time to first cigarette in the morning as a measure of cigarette dependence (Heatherton, Kozlowski, Frecker, & Fagerstrom, 1991), history of smoking-related illness, desire and confidence to quit (1 = not at all, 5 = very much), and intention to use NRT (1 = no, definitely will not, 4 = yes, definitely will). Attitudes/beliefs about NRT were assessed with 11 items from existing instruments (Bansal, Cummings, Hyland, & Giovino, 2004; Ferguson et al., 2011). On a 4-point Likert scale (1 = strongly disagree; 4 = strongly agree), participants rated agreement with statements about NRT products such as “NRT products taste bad” and “NRT products are too expensive.”

The 3-month follow-up survey was administered online, with telephone follow-up for online non-responders and incentives to maximize response rates. The main dependent variable in these analyses was any self-reported use of any NRT (study provided or self-purchased) in the past 3 months. Automated tracking data on number of website visits were extracted at 3 months. We determined which posts participants were exposed to through website clickstream data; details of this process have been previously published (Zhao et al., 2016). Briefly, the clickstream data consisted of 3 types of posts: message boards, group discussions, and blogs. We assumed that participants were exposed to NRT-related content if a participant visited the page and the NRT-related content was visible as either the main post or one of the most recent comments on the page associated with the main post.

2.5. Identification of NRT posts using machine learning

To identify NRT-related content viewable by trial participants, we first used a list of NRT-related keywords (e.g., patch, gum, lozenge, NRT, nicotine) in an automated search. This search yielded 37,271 posts. A random sample (n = 5000) was manually annotated for NRT relevance by BecomeAnEX community members who served as domain experts for this study. Two coders reviewed each post and a third served as tiebreaker for discrepancies. Inter-rater agreement was high (Cohen’s kappa = 0.93).

Machine-learning-based automated text classification was used to determine NRT relevance of the remaining 32,271 posts. Each post was represented as a vector of word frequencies (unigrams). Classification also drew on meta-features of a post, such as its length and type (i.e., initial post or comment). We used 10-fold cross validation to train and evaluate binary classifiers on NRT relevance: 90% of manually-annotated posts were used to train different classification algorithms and the remaining 10% were used to evaluate classifier performance. Validation and training sets were rotated in 10 different trials; classifier performance was evaluated using accuracy, F1 score, and area under the curve (AUC) metrics. If no algorithm dominated on all three metrics, the algorithm with the highest AUC was used since it is more robust against skewed prior distributions (Gao, Fan, Han, & Philip, 2007). The best performing classifier algorithm was applied to classify each post as NRT-related or not.

2.6. Sentiment of user-generated content

The view or attitude towards NRT in each post (“sentiment”) was rated by Amazon Mechanical Turk (MTurk) workers. MTurk is an online labor market that allows researchers access to thousands of people who can accomplish small tasks for payment. Sentiment ratings were collected from MTurk workers, rather than domain experts, to resemble the sentiment of naïve readers (i.e., new social network members). Nine MTurk workers rated each post on two dimensions: 1) whether the post discussed NRT (yes/no), and 2) overall sentiment towards NRT (1 = extremely negative, 5 = neutral, 9 = extremely positive). A total of 77 workers provided sentiment ratings for a total cost of $1985 (number of posts rated per worker: median = 71, mean = 312). Five workers were excluded because > 10% of their ratings were inconsistent with instructions. Excluded workers rated 63 posts, representing 0.3% of all data. Posts flagged by 3 or more workers as “not about NRT” were determined to be falsely machine-classified (n = 267) and were excluded. For all remaining posts, a post-level mean sentiment rating was calculated and used in analyses.

2.7. Statistical analyses

Descriptive statistics were used to characterize the full sample (n = 3297) and the subsample of participants exposed to at least one NRT post (n = 639). Next, we evaluated the number of NRT posts the subsample was exposed to, and the mean sentiment rating of those posts. We then assessed differences in the number and sentiment of NRT post exposure by participant characteristics. Wilcoxon rank sum tests (for binary characteristics) and Kruskal-Wallis rank sum tests (for multi-category characteristics) assessed group differences in NRT post exposure, given its highly skewed distribution. Mean sentiment was symmetrically distributed, and group differences were evaluated using Student’s t-tests for binary characteristics and one-way ANOVA for multi-category characteristics.

Finally, we estimated the relationship between cumulative exposure to NRT sentiment (coded as number of NRT posts viewed × mean NRT sentiment per post) and NRT use at the 3-month follow-up. We selected the 3-month follow-up since this is when participants are most likely to be active in the network (Eysenbach, 2005; Schwarzer & Satow, 2012). We used modified Poisson regression (Zou, 2004) with a logarithmic link, controlling for treatment assignment. In addition, we examined whether free NRT provision moderated the association between cumulative exposure to NRT sentiment and NRT use. Treatment assignment to the four study arms was coded using indicators for SN integration and free NRT provision. Their product described SNxNRT interaction effects. Backwards elimination was used to simplify the model, starting with the 2-way interaction terms. All analyses were conducted in R, Version 3.4, and robust standard errors were obtained using the GEE package (V. Carey, Lumley, & Ripley, 2015; R Core Team, 2016; G. Zou, 2004; G. Y. Zou & Donner, 2013).

3. Results

3.1. Automated text classification

Of the 5000 posts manually annotated by domain experts, 72.9% were related to NRT. Table 1 summarizes the performance of seven different classification algorithms. We tested seven different classification algorithms: Naïve Bayes, J48 Decision Tree, SVM with poly kernel, KNN, Random Forest, AdaBoost (an ensemble learner) with Naïve Bayes, and J48 as weak learners. Adaboost (Freund & Schapire, 1997) with J48 as the weak learner dominated on all three metrics with an accuracy of 0.91, F1 score of 0.91 and an AUC of 0.96. Note that all the three performance measures have values ranging from 0 to 1, with 1 being perfect. After applying this classification model to the un-annotated posts, a total of 27,038 posts (73.3% of posts with one or more NRT keywords) were determined to be related to NRT (including annotated posts). Study participants were exposed to a total of 2680 NRT-relevant posts; MTurk sentiment ratings were collected for each of those posts.

Table 1.

Comparing the performance of different algorithms for NRT relevance classification.

Algorithm	Accuracy	F1 score	AUC
Naïve Bayes	0.77	0.78	0.81
Decision Tree (J48)	0.90	0.89	0.89
SVM (poly kernel)	0.86	0.86	0.80
KNN	0.70	0.68	0.56
Random Forest	0.76	0.68	0.91
Adaboost (Naïve Bayes)	0.81	0.81	0.84
Adaboost (J48)	0.91	0.91	0.96

3.2. Participant characteristics

Within the analytic sample, 19.3% (n = 639) were exposed to any NRT-related posts as determined by analysis of community clickstream data. Characteristics of all participants and of the subsample of participants who were exposed to at least one NRT post are presented in Table 2. Roughly 40% of both groups smoked within the first 5 min after waking, nearly 95% reported a strong desire to quit, and about 20% reported little or no confidence in their ability to quit. About half of participants in both samples reported they would probably/definitely use NRT in their quit attempt.

Table 2.

Demographic and smoking characteristics of the full analytic sample (n = 3297) and subsample of users who were exposed to at least one NRT post by the 3-month follow-up (n = 639).

Characteristic	Full sample (N = 3297)	Sub-Sample (N = 639)	NRT post exposure (subsample)a		Mean sentiment rating of NRT posts (subsample)a
	Percent	Percent	Median (IQR)	p-valueb	Mean (SD)	p-valuec
	100.0	100.0	3 (1.0, 5.0)		5.4 (0.8)
Demographics
Age group, years
18–24	10.6	7.4	3 (1.0, 4.0)	.345	5.3 (0.8)	.324
25–44	45.2	44.3	3 (1.0, 5.5)		5.3 (0.8)
45–64	40.0	44.8	3 (1, 5)		5.4 (0.8)
65 +	4.2	3.6	3 (2, 7)		5.5 (0.8)
Gender
Male	37.8	31.8	3 (1, 4)	.088	5.4 (0.9)	.553
Female	62.2	68.2	3 (1, 6)		5.4 (0.7)
Race
White	80.1	82.9	3 (1, 6)	.167	5.4 (0.8)	.805
Black or African American	15.7	12.4	3 (1, 4)		5.3 (0.8)
Asian	1.3	1.1	3 (2, 3.5)		5.6 (1.0)
Otherd	2.9	3.6	2 (1, 4)		5.3 (1.0)
Ethnicity
Hispanic or Latino	5.1	4.2	2 (1, 3)	.007	5.4 (0.9)	.671
Non-Hispanic	94.9	95.8	3 (1, 5)		5.4 (0.8)
Education
< High school degree	3.9	4.1	3 (1, 4.75)	.234	5.6 (1.0)	.395
High school degree or GED	20.2	16.7	2 (1, 5)		5.3 (0.7)
Some college (1–3 years)	50.1	48.4	3 (2, 6)		5.4 (0.8)
College graduate (≥ 4 years)	25.8	30.8	3 (1, 5)		5.4 (0.8)
Smoking variables
Cigarettes per day
≤ 10	32.7	30.7	3 (1.8, 5.3)	.774	5.4 (0.7)	.173
11–20	49.2	51.0	3 (1, 5)		5.4 (0.8)
21 +	18.2	18.3	3 (1, 5)		5.3 (0.8)
Time to first cigarette
Within 5min	37.8	40.5	3 (1, 5)		5.3 (0.8)
6–30 min	39.6	38.7	3 (1.5, 6.5)	.280	5.4 (0.8)	.445
31–60min	12.9	12.5	2 (2, 5)		5.5 (0.8)
After 60 min	9.8	8.3	3 (2, 5)		5.3 (0.6)
Illness from smoking
Yes	65.4	70.4	3 (1.3, 6)	.016	5.4 (0.8)	.765
No	34.6	29.6	2 (1, 4)		5.4 (0.9)
Desire to quit
Not at all, a little, somewhat	6.5	5.0	3 (1, 4.25)	.526	5.3 (0.8)	.927
A lot	32.2	30.8	3 (2, 5)		5.4 (0.7)
Very much	61.2	64.2	3 (1, 5)		5.4 (0.8)
Confidence to quit
Not at all, a little	20.2	21.3	3 (1.8, 4)	.768	5.4 (0.8)	.615
Somewhat	42.9	44.1	3 (1, 5)		5.3 (0.7)
A lot, very much	36.9	34.6	3 (1, 6)		5.4 (0.9)
Intention to use NRT
No, definitely will not	16.7	14.2	3 (1.5, 5.5)	.027	5.4 (0.9)	.158
I probably will not	30.3	30.7	2 (1, 5)		5.5 (0.8)
I probably will	34.5	36.6	3 (1, 5)		5.3 (0.8)
Yes, I definitely will	18.5	18.5	4 (2, 6)		5.3 (0.8)

^aOut of 2413 posts that were viewed by at least 1 user and confirmed as relevant by raters.

^bObtained using Wilcoxon rank sum tests for binary variables and Kruskal-Wallis rank sum tests for multi-category variables.

^cObtained using t-tests for binary variables and one-way ANOVA for multi-category variables.

^dAmerican Indian, Alaska Native, Native Hawaiian or other Pacific Islander.

Attitudes and beliefs about NRT were similar in the full sample and among participants who were exposed to at least one NRT post (Table 3). The majority of participants in both samples questioned the efficacy of NRT and were skeptical of NRT safety.

Table 3.

Baseline behavior, attitudes and beliefs about NRT among full analytic sample (n = 3297) and the subsample of users who were exposed to at least one NRT post by the 3-month follow-up (n = 639).

	Overall (N = 3297)	Sub-Sample (N = 639)	Number of NRT posts read (subsample)a		Mean sentiment of NRT posts read (subsample)a
	Percent	Percent	Median (IQR)	p-valueb	Mean (SD)	p-valuec
NRT behavior
Past year NRT use
Yes	29.0	31.9	3 (1, 6)	.694	5.4 (0.8)	.920
No	71.0	68.1	3 (1, 5)		5.4 (0.8)
Attitudes/beliefs about NRT products
They double the chance of quitting compared to cold turkey
Agree	23.7	23.3	3 (1, 6)	.736	5.6 (0.8)	< .001
Disagree	76.3	76.7	3 (1, 5)		5.3 (0.8)
They do not work
Agree	65.1	67.3	3 (1, 5.75)	.037	5.3 (0.8)	.178
Disagree	34.9	32.7	2 (1, 4)		5.4 (0.8)
They help smokers to quit short-term, but not long-term
Agree	51.6	50.7	3 (1, 5)	.779	5.4 (0.8)	.566
Disagree	48.4	49.3	3 (1, 5)		5.4 (0.8)
The only way to quit is to go cold turkey
Agree	63.5	66.8	3 (1, 5)	.750	5.3 (0.8)	.222
Disagree	36.5	33.2	3 (1, 6)		5.4 (0.8)
They are too expensive
Agree	10.4	11.9	3 (2, 5)	.893	5.4 (0.8)	.881
Disagree	89.6	88.1	3 (1, 5)		5.4 (0.8)
They taste bad
Agree	35.3	36.5	3 (1, 6)	.193	5.5 (0.7)	.032
Disagree	64.7	63.5	3 (1, 5)		5.3 (0.8)
They just trade one addiction for another
Agree	45.4	45.4	3 (1, 5.75)	.258	5.3 (0.8)	.085
Disagree	54.6	54.6	3 (1, 5)		5.4 (0.8)
The nicotine in nicotine stop smoking products is more dangerous than the nicotine in cigarettes
Agree	83.3	85.3	3 (1, 5)	.069	5.4 (0.8)	.803
Disagree	16.7	14.7	2 (1, 4)		5.4 (0.8)
They are dangerous
Agree	73.7	75.0	3 (1, 5)	.036	5.3 (0.8)	.008
Disagree	26.3	25.0	2 (1, 4.25)		5.5 (0.9)
They are addictive
Agree	57.6	58.1	3 (1, 5)	.861	5.4 (0.8)	.373
Disagree	42.4	41.9	3 (1, 6)		5.4 (0.8)
They are designed to make you feel sick if you slip and have a cigarette
Agree	67.5	70.4	3 (1.25, 6)	< .001	5.4 (0.8)	.366
Disagree	32.5	29.6	2 (1, 4)		5.3 (0.9)

^aOut of 2413 posts that were viewed by at least 1 user and confirmed as relevant by raters.

^bObtained using Wilcoxon rank sum tests for binary variables and Kruskal-Wallis rank sum tests for multi-category.

^cObtained using t-tests for binary variables and one-way ANOVA for multi-category variables.

3.3. Association of participant characteristics and exposure to NRT posts/sentiment

Participants with any exposure to NRT posts were exposed to a mean of 6.5 NRT posts (SD = 14.7; M = 3, IQR = 1–5) between study enrollment and 3-month follow-up. The mean sentiment of NRT posts was symmetrically distributed around a moderately positive rating of 5.4 (SD = 0.8; M = 5.3, IQR = 5.0–5.7). Neither study arm assignment nor past year use of NRT was associated with the number or mean sentiment of NRT post exposure.

Examination of the relationship between participant characteristics and extent of NRT post exposure yielded several findings (Table 2). Exposure to NRT-related posts (median 3 vs. 2) was significantly higher among non-Hispanic vs. Hispanic participants (p = .007), those with prior illness from smoking (p = .016), as well as those expressing greater intention to use NRT at baseline (p = .027). Similar increases in NRT post exposure (median 3 vs. 2) were observed among participants endorsing the following beliefs about NRT (Table 3): “NRT does not work” (p = .037); “NRT is dangerous” (p = .036); and “NRT is designed to make you feel sick if you slip and have a cigarette” (p < .001).

Participants’ mean sentiment exposure did not differ by demographic or smoking behaviors (Table 2). However, there were small differences in mean sentiment exposure by baseline beliefs about NRT (Table 3). Participants who agreed at baseline that NRT products “double the chance of quitting compared to cold turkey” were exposed to posts with slightly more positive mean NRT sentiment than participants who disagreed (5.6 vs. 5.3, p < .001), as did those who agreed that they “taste bad” (5.5 vs. 5.3, p = .032). In contrast, participants who agreed with the statement “NRT is dangerous” at baseline were exposed to posts with slightly more negative mean NRT sentiment (5.3 vs 5.5, p = .008).

3.4. Exposure to peer sentiment about NRT and actual NRT use

Among the 639 users who were exposed to at least one NRT post by the 3-month follow-up, 415 (65%) reported use of NRT at the 3-month follow-up. Modified Poisson regression showed that neither the SN intervention effect (p = .996) nor the effect of number of NRT posts viewed (p = .189) differed by free NRT provision (results not shown). Failing to detect a main effect of the number of NRT posts viewed on NRT use (p = .887), we dropped this variable from the model (results not shown). As seen in Table 4, participants exposed to NRT posts of average sentiment had a 35% likelihood of using NRT if assigned to the WEB arm (95% CI 0.29, 0.42). For such participants, NRT assignment increased the likelihood of NRT use to over 92% (WEB+NRT vs. WEB adjusted rate ratio [aRR] 2.69, 95% CI: 2.25, 3.23; p < .001), while SN assignment reduced the likelihood of NRT use to 31% (WEB+SN vs. SN aRR 0.90, 95% CI: 0.82, 0.98; p = .022). The effect of joint SN + NRT assignment was multiplicative, resulting in 82% NRT utilization rates among participants exposed to NRT posts of average sentiment in the combined intervention arm.

Table 4.

Relative risk of using NRT at the 3-month follow up among users who were exposed to at least one NRT post by the 3-month follow-up (n = 639).

	aRR	95% CI		p-value
		LL	UL
Intercepta	0.35	0.29	0.42	< 0.001
Assigned to SN condition	0.90	0.82	0.98	0.022
Assigned to NRT condition	2.69	2.25	3.23	< 0.001
NRT post sentimentb	1.22	1.01	1.47	0.043
NRT post sentiment × Assigned to NRT condition	0.81	0.67	0.98	0.033

aRR = Adjusted Rate Ratio. LL = 95% lower confidence limit; UL = 95% Upper Confidence Limit.

^aIntercept represents rate of NRT use among WEB participants exposed to NRT posts of average sentiment.

^bNRT post sentiment was centered at the sample average of 5.37.

A significant interaction was detected between mean NRT sentiment and assignment to an NRT condition (aRR 0.81, 95% CI .67, .98, p = .033). In particular, no relationship between NRT sentiment and NRT use emerged among participants who received free NRT (aRR 0.98, 95% CI 0.94, 1.03; p = .48; not shown in Table 4). However, among participants who did not receive free NRT, each 1-point increase in mean sentiment about NRT (observed range 2.6–8.1) increased the likelihood of NRT use at the 3-month follow up by 22% (aRR 1.22, 95% CI 1.01, 1.47; p = .043). Conversely stated, among participants that did not receive free NRT, each 1-point decrease in mean sentiment about NRT decreased the likelihood of NRT use at the 3-month follow up by 18% (aRR 0.82, 95% CI .68, .99; p = .043; not shown in Table 4).

4. Discussion

This study examined the relationship between exposure to peer sentiment towards NRT in an online social network for smoking cessation and actual NRT use. Analyses revealed an important – and unexpected – finding. There was no association of exposure to peer sentiment and NRT use among participants who did receive free NRT as part of the trial. In contrast, exposure to peer sentiment was associated with NRT use among those who did not receive free NRT: for every 1-point increase in positive sentiment about NRT, a participant’s relative odds of NRT use increased by 22%. These findings suggest that others’ opinions about NRT may be influential when smokers decide whether to purchase NRT but may not matter when smokers are given free NRT. To our knowledge, this is the first study to demonstrate links between exposure to online peer sentiment and “offline” cessation behavior (Cobb et al., 2013).

More broadly, this study adds to the literature exploring the mechanisms by which participation in online social networks for health can improve health outcomes. Using Berkman et al.’s framework of social support (Berkman, Glass, Brissette, & Seeman, 2000), these results can be understood in terms of the relative importance of informational support and instrumental support. Participants who did not receive free NRT as part of the study may have relied on informational support from their peers to decide whether to procure and use NRT. In contrast, participants who received instrumental support from the study team in the form of free NRT appear to have been uninfluenced by peer informational support. Future research should further investigate how the presence of instrumental support moderates the importance of other forms of social support for specific health behaviors in online smoking cessation programs.

These results have important implications for the management of online social networks for smoking cessation. Many smokers seeking online cessation support may not have access to free cessation medication. Curating and showcasing user-generated content that reflects positive sentiment towards NRT may encourage more individuals to consider purchasing and using NRT. To the extent that user-generated content counters negative beliefs about NRT, featuring such posts could increase the number of smokers who use NRT during quit attempts and increase quit rates. Additionally, these findings reinforce the power of providing free NRT on smokers’ decision to use NRT. These empirical questions have potentially significant public health impact that are worthy of future exploration.

4.1. Strengths & Limitations

Strengths of the study include a unique dataset with behavioral outcome data on NRT use. Previous sentiment analysis studies have been largely descriptive and have lacked behavioral outcomes (Cole-Lewis et al., 2015; Myneni et al., 2016; Myneni et al., 2013; Rose et al., 2017; Sofean & Smith, 2013). Our examination of baseline NRT perceptions showed small and likely clinically insignificant differences in exposure to peer sentiment about NRT between the full sample and the sample of participants who were exposed to at least one NRT post, suggesting that exposure to NRT content was not simply a function of users searching to confirm their existing perceptions about NRT. A second strength was that the rich expertise of longtime social network members informed development of the machine classifier, which performed at > 0.90 across all metrics. Finally, while our research focused on NRT sentiment, this work provides a model for the exploration of other topics.

Four limitations should be noted. First, we cannot make causal statements about the link between sentiment exposure and use of NRT, given that NRT use was only measured at follow-up. While study participants may have sought information prior to NRT use, it is also possible that participants sought information after using NRT. Future research should consider more fine-grained measurement intervals to establish a direct link between peer sentiment and behavioral outcomes. Relatedly, though we assume that participants who visited a page containing an NRT-related post actually read that post, we cannot verify that assumption, though our findings linking exposure to NRT sentiment and actual NRT use seems to support our assumption. Third, trial exclusion criteria (i.e., contraindications for NRT use) may influence the relevance of our findings to smokers without such contraindications who participate in an online social network. Finally, we were not able to account for exposure to NRT sentiment outside of the BecomeAnEX website, which could be a source of confounding for these analyses.

5. Conclusion

Online social networks facilitate the exchange of personal testimonies among users and may be a powerful means of motivating behavior change. Understanding the influence of peer sentiment on NRT use may inform community management strategies to more prominently feature such content. Results from this study warrant further research into the effect of peer sentiment on behavior change across online health behavior change interventions.

HIGHLIGHTS.

• Concerns about the safety and efficacy of NRT are common among treatment-seeking smokers.

• Positive peer sentiment about NRT in an online social network increases the likelihood of use when smokers have to acquire NRT on their own.

• When NRT is available for free, peer sentiment does not appear to influence NRT use.