Content-Driven Analysis of an Online Community for Smoking Cessation: Integration of Qualitative Techniques, Automated Text Analysis, and Affiliation Networks

Abstract

Objectives. We identified content-specific patterns of network diffusion underlying smoking cessation in the context of online platforms, with the aim of generating targeted intervention strategies.

Methods. QuitNet is an online social network for smoking cessation. We analyzed 16 492 de-identified peer-to-peer messages from 1423 members, posted between March 1 and April 30, 2007. Our mixed-methods approach comprised qualitative coding, automated text analysis, and affiliation network analysis to identify, visualize, and analyze content-specific communication patterns underlying smoking behavior.

Results. Themes we identified in QuitNet messages included relapse, QuitNet-specific traditions, and cravings. QuitNet members who were exposed to other abstinent members by exchanging content related to interpersonal themes (e.g., social support, traditions, progress) tended to abstain. Themes found in other types of content did not show significant correlation with abstinence.

Conclusions. Modeling health-related affiliation networks through content-driven methods can enable the identification of specific content related to higher abstinence rates, which facilitates targeted health promotion.

Epidemiological evidence indicates that modifiable risky health behaviors place a substantial socioeconomic burden on human health and wellness.¹ Understanding human behavior in real-time settings is essential to improving health outcomes related to these behaviors.^2,3 Technological advances in connectivity offer the means to obtain potentially valuable data sets in the form of electronic traces of the activities of online social communities. These data may help us to understand the intra- and interindividual intricacies of health-related behaviors. Studies of online and offline social networks provide valuable insight into social influence, information spread, and behavioral diffusion.^4–6 Most of these analyses have paid more attention to the frequency of communication between members than to its content. The content, however, is relevant to behavior change theories, which address the use of specialized content to stimulate and support individuals to achieve a desired change.^7,8 Contemporary work on social media data rarely addresses this fundamental concern of behavior change theorists.

Outside the context of online networks, several theories have been formulated to explain behavior change. Some, such as the Transtheoretical Model,⁹ belong to the intrapersonal category; others, such as Social Cognitive Theory¹⁰ and social network and support models,¹¹ are classified as interpersonal. (Appendix A, available as a supplement to the online version of this article at http://www.ajph.org, provides an overview of the theoretical constructs.) Empirical research on the applicability of these models to behavior change of health consumers in the digital era is minimal.¹² Recent research showed that participation in health issue–specific social networking sites significantly influenced social factors such as identification, perceived subjective norms, and social support, which in turn resulted in greater smoking cessation self-efficacy.¹³ Content inclusion in analytical models of social networks can enable us to examine the content-specific patterns of social factors underlying behavior change. Through mapping of the specific content to theories, such content inclusion can facilitate the development of network interventions for health behavior changes by harnessing the power of social relationships.

Studies of QuitNet, an online social network for smoking cessation, have examined the structure of peer-to-peer communication patterns and provided insights into social integrators and opinion leaders.^5,14 Previous work showed the applicability of affiliation networks to real-world diffusion networks, enriching our understanding of the affiliation-based sources of influence on individuals’ behavior. Examples include the diffusion of (1) ratification of the World Health Organization Framework Convention on Tobacco Control by comembership with an online forum among countries¹⁵; (2) gunshot victimization by co-offending with victims among Chicago, Illinois, gangsters¹⁶; (3) substance use by coparticipating in school-sponsored sports or co-identifying with the same crowd types^17,18; and (4) sexual behavior by coaffiliating with venues among male sex workers.¹⁹

We used affiliation networks to analyze messages for content-specific patterns of network diffusion. We took an interdisciplinary approach, integrating methods from sociobehavioral sciences, social network analytics, and biomedical informatics. We employed qualitative techniques derived from grounded theory, automated text analysis, and affiliation network analysis to investigate the communication patterns underlying human behavior in online environments. Our study had 3 major components: (1) a qualitative study of human communication within user-generated data in QuitNet, (2) computational text analysis to further extend this analysis, and (3) identification of communication patterns pertinent to behavior change in affiliation networks. We anticipate that the insights gained from this research will enhance our understanding of behavior change and will have implications for the design of sociobehavioral interventions that draw upon social influence.

METHODS

QuitNet is one of the first online social networks whose purpose is health behavior change. It is widely used, with more than 100 000 new registrants per year.²⁰ Previous studies of QuitNet found that participation in the online community was strongly correlated with abstinence.²¹ Our data set came from a previously studied quality improvement database of de-identified messages in public threaded forums, in which participants post messages and reply directly to each other. This database comprised 16 492 public messages (original posts and their replies) exchanged between March 1 and April 30, 2007. All messages were stripped of identifiers but recoded for sender ID, receiver ID, self-reported smoking status, date, and position within the thread.

Analysis of voluminous and context-rich social network data requires scalable methods. Qualitative coding provides useful insights into text data exchanged between QuitNet members. For larger data sets, the qualitative method needs to be complemented with an automated technique to optimize resource utilization. Such an extension enhances the validity and generalizability of the manual coding in addition to facilitating the application of network analysis methods to an entire data set. In our case, affiliation network analysis allowed us to understand the content-specific attributes of network diffusion on smoking abstinence. Figure 1 provides detailed information about the research methods and their rationale.

FIGURE 1—

Overview of research strategy in analysis of QuitNet user messages and smoking cessation.

Qualitative and Automated Text Analysis

We first analyzed the message content of QuitNet with the qualitative techniques of grounded theory.^22,23 We employed open coding, axial coding, and constant comparison to arrive at concepts and themes that captured the behavioral, interpersonal, and individualistic concepts expressed in QuitNet communication. We analyzed 585 randomly selected messages to derive the themes embedded in the messages. We analyzed an additional 210 messages to ensure thematic saturation.

Two researchers coded a separate data set of 100 messages according to the 12 themes that emerged from the grounded theory analysis. The codes we assigned had a Cohen’s κ measure of 82%. We resolved disagreement in coding through follow-up discussions. We then mapped the derived themes to the theoretical constructs of behavior change. A full list of the constructs used in the mapping process can be found in Appendix A. The goal of the mapping process was to understand the interplay between the behavior change constructs and QuitNet communication content.

Next, we exploited recent developments in automated text analysis to measure the extent to which key concepts of interest were expressed within messages between QuitNet users, regardless of the specific terms used to express these concepts at the surface level. We applied latent semantic analysis, a method of distributional semantics,²⁴ in conjunction with a machine-learning algorithm to derive a measure of relatedness between a given message and the previously identified QuitNet themes to estimate the distribution of different types of content across QuitNet. This use of automated techniques facilitates scaling qualitative analysis to large-scale social media data sets. This, in turn, mediates the development of a novel approach to structuring a network in accordance with the content of the communication within it, so it can be subjected to adaptations of existing network analysis methods.

Affiliation Network Analysis

Finally, we employed 2-mode affiliation network analysis, which uses 2 distinct sets of nodes (actors [n] and events [V]).²⁵ The first distinct set of nodes in our affiliation network represented the set of QuitNet members (n = 1423). The second distinct set represented the communication themes (V = 12). We considered a QuitNet member to be affiliated with a particular theme if a message exchanged by that member was classified under that theme. We divided the themes into 4 groups according to the behavior change constructs to which they related: (1) group-centric interpersonal (social support, traditions, motivation), (2) individual-centric interpersonal (progress, virtual rewards, family and friends), (3) intrapersonal with sociobehavioral beliefs (benefits, obstacles, conflict), and (4) intrapersonal themes with personal experience stories (cravings, relapse, nicotine replacement therapy). Then, we created a user-by-theme 2-mode matrix (1423 users by 12 themes) that represented affiliation ties between users and themes, which we used for subsequent analyses.

Our smoking behavior measure was the reported abstinence quotient (RAQ), an aggregate measure of an individual’s reported smoking behavior. We determined the RAQ by averaging the abstinence status self-reported by QuitNet members every time they logged in to the system over a period of 60 days, which gave us an estimate of a member’s self-reported abstinence status during this period. The RAQ could range from 0 to 1; 0 indicated that a member did not report a period of abstinence during the study period, and 1 indicated that a member stayed abstinent throughout. The higher the RAQ, the higher the proportion of self-reported status updates that reported abstinence from smoking.

We generated a visual representation of the affiliation networks between members and communication themes derived from actual user communication in the QuitNet community. We used these networks to identify the major content types connecting the QuitNet members by estimating the degree centrality metrics. We constructed graphs with the NetDraw plugin of UCINET 6.²⁶

We used the Affiliation Exposure Model to measure the extent to which individual users were exposed to other abstinent members through affiliating with at least 1 common communication theme.¹⁷ The resulting exposure values ranged from 0 to 1, which we used as the main explanatory variable in our regression analysis. In addition, we used a 1-mode Network Exposure Model to measure the extent to which users were exposed to abstinent members, according to their peer-to-peer communication network.^27–29 A communication network was represented by an adjacency matrix that recorded the number of direct message exchanges among QuitNet users during the study period. Technical details on the 1-mode and 2-mode network exposure models are provided in Appendix A.

We estimated a Network Autocorrelation Model by including an affiliation exposure term³⁰ and a 1-mode communication network exposure term. We treated the former as our explanatory variable and the latter as one of our controlled variables. Other controlled variables in our model were gender, age, and number of themes each user affiliated with.

The network autocorrelation model handles potential violations of correlated errors commonly observed in network data in the parameter estimation process.^31,32 We employed the sna package in R, an open-source statistical analysis software for this purpose.³³ The results indicated that 43% of our affiliation exposure values were unity (full exposure to abstinent users). To address this issue, we included dummy variables that indicated nonunity exposure values versus unity exposure values.³⁴

RESULTS

More than three fourths of users in our data set were female, and the mean age was 44.3 years. Around 70% of the users had a self-reported RAQ of 1, and 22% had a 0 RAQ. On average, a user posted 22.4 messages. The difference between the RAQ = 0 and RAQ = 1 groups was statistically significant for gender (2-sided χ² = 335.07; P < .05), but not significant for age and posting frequency.

Qualitative Analysis

We identified 43 different concepts, grouped under 12 themes (e.g., traditions, obstacles, progress). A complete list of the themes, their definitions, and sample messages can be found in Appendix A. The identified themes related to the constructs of existing behavior change theories. Detailed description of the theoretical mapping process is beyond the scope of this article, but is described in detail elsewhere,³⁵ and we give an example here.

Stimulus control, a construct from the Stages of Change Model,⁹ involves the use of reminders that encourage healthy behavior as a substitute for unhealthy behavior. For early morning smokers, QuitNet has a tradition whereby members post messages describing early morning weather and reaffirm their commitment to stay abstinent. Examples of other traditions are (1) bonfire, a virtual event hosted regularly in which members bring their unsmoked cigarettes and throw them into a fire, which facilitates observational learning, and (2) pledge, a ritual in which members extend a hand to the next member, indicating their commitment to staying abstinent and providing helping relationships. Community-driven activities such as these traditions organized in QuitNet are manifestations of behavior-related theoretical constructs in online settings.

Similarly, we mapped other themes to the theoretical constructs. The alignment was highest for traditions, with 13 constructs, followed by relapse, with 9 constructs. This result indicated that the member-generated group-centric strategies, such as bonfires and pledges, that formed traditions were associated with more theoretical constructs than were other themes. It is important to note that no single behavior change theory provides a basis for all of the themes that emerged from the QuitNet messages.

Automated Analysis

We processed our entire QuitNet database, consisting of 16 492 messages, with an automated classification system, the Semantic Vectors package,³⁶ developed through methods of distributional semantics.¹⁴ For each unannotated message, the system provided a ranked list of themes derived from the nearest neighbors from the annotated set of messages. The program produced a score for a particular theme by adding the relatedness measures of the nearest neighbors that corresponded to that theme.

The reliability measures calculated by using Cohen’s κ indicated that the average agreement of the system with human raters was 0.71. (Appendix A shows reliability measures for each theme.) Figure 2 illustrates the distribution of the QuitNet communication themes derived from automated analysis. Having established user–message–theme connections, we were able to analyze the QuitNet data set for discernible relationships between smoking status and communication themes.

FIGURE 2—

Distribution of themes in QuitNet user messages, derived from automated analysis.

In addition to determining the distribution of the QuitNet communication themes, the large-scale qualitative analysis facilitated by the automated methods allowed us to quantify the theoretical constructs embedded in unannotated QuitNet messages. The results indicated that the QuitNet messages facilitated the transmission of behavior change constructs, such as emotional coping responses, helping relationships, and self-efficacy.

Affiliation Network Analysis

Figure 3 illustrates the affiliation networks formed by the 1423 users and the 12 themes representing QuitNet communication. According to their degree centrality, social support and cravings were the most prevalent themes embedded in the communication among QuitNet users who were self-reported smokers during the study period. Progress was the theme identified most often in the messages communicated among QuitNet users who were successful quitters (i.e., they changed their self-reported status from smokers to ex-smokers during the study period). Social support was the most communicated theme among QuitNet users who relapsed 1 or more times, and traditions was the leading theme among messages communicated among QuitNet users who self-reported as ex-smokers during the study period.

FIGURE 3—

Affiliation networks formed by QuitNet users and themes in their communication.

Note. Green circles represent QuitNet users who have maintained successful quitting with abstinence status “1.” Lime green circles represent QuitNet users who have changed their status from smokers to ex-smokers. Light red circles represent QuitNet users who have relapsed (changed their status from ex-smokers to smokers). Yellow circles represent users with multiple relapses (multiple changes in self-reported abstinence status). Blue squares represent the QuitNet communication themes.

The effect of affiliation exposure to abstinent coaffiliates on abstinence was statistically significant for interindividual themes, according to a 2-tailed test at the α = 0.05 level (Table 1). The autocorrelation parameter estimate showed that as QuitNet users were more exposed to other users who reported abstinence through sharing the group-centric interpersonal themes, they were more likely to stay abstinent themselves (b = 0.038; P < .01), after adjustment for age, gender, and communication network exposure to abstinent members. Similarly, as QuitNet users were more exposed to other users who reported abstinence by engaging in communication related to individual-centric interpersonal themes, they were more likely to be abstinent themselves (b = 0.078; P < .05).

TABLE 1—

Affiliation Exposure Among QuitNet Members and Network Exposure Computations for Reported Abstinence Quotient Derived From Network Autocorrelation Model, 2007

Type of Theme	b (SE)
Group-centric interpersonal
Affiliation exposure	0.038** (0.014)
Network exposure	0.021** (0.012)
Age	0.008** (0.0007)
Gender	0.037 (0.027)
Themes affiliated, No.	0.008 (0.018)
Individual-centric interpersonal
Affiliation exposure	0.078* (0.032)
Network exposure	0.092** (0.041)
Age	0.009** (0.0008)
Gender	0.176 (0.032)
Themes affiliated, No.	0.03 (0.038)
Intrapersonal (personal experience stories)
Affiliation exposure	0.04 (0.023)
Network exposure	0.017 (0.021)
Age	0.003** (0.001)
Gender	0.08 (0.03)
Themes affiliated, No.	−0.03 (0.03)
Intrapersonal (sociobehavioral beliefs)
Affiliation exposure	−0.02 (0.022)
Network exposure	0.034 (0.016)
Age	0.008** (0.002)
Gender	0.08 (0.06)
Themes affiliated, No.	0.18** (0.049)

Note. The sample size was n = 1423.

*P < .05; **P < .01.

Among intrapersonal themes, the parameter estimate indicated that members who were more exposed to other abstinent QuitNet members through sharing of personal experiences were more likely to abstain themselves. The number of themes embedded in the messages a QuitNet member exchanged in the intrapersonal belief category was significant and indicated that the more themes a member exchanged through messages in this category, the more likely it was that the member abstained from smoking. It is important to note that the significance of the number of themes was not a network construct. The number of themes, per se, was not significant for the rest of the thematic groups. However, their affiliation exposures were significant. Results of 1-mode network exposure effect were significant for interpersonal themes (social support, progress, traditions) and intrapersonal themes encompassing sociobehavioral beliefs such as benefit and obstacles.

DISCUSSION

To our knowledge, ours was among the few studies that have attempted to explain the underexplored area of network diffusion according to affiliation exposure in the context of online communities for smoking cessation. Our work provides new techniques to better personalize interventions by our use of content-inclusive network analysis of online peer-to-peer exchanges.

Our study provides public health researchers and interventionists with new ways to tailor sociobehavioral intervention programs. We found that engaging in peer-to-peer communication involving specific kinds of content is related to smoking cessation. This finding can inform the design of targeted interventions that encourage people to sustain positive behavior change, which sets the stage for a new generation of empirically grounded interventions in public health. Our interdisciplinary method integrated qualitative coding, automated text analysis, and affiliation network analysis to provide further insight into the structural features underlying health behavior in social networks. Inclusion of content attributes in network models of QuitNet resulted in valuable insights into communication patterns that may enable us to better tailor network-related interventions. Examples of these approaches include identifying content-specific key players and engineering mentor–mentee matches according to user needs.

We examined the structural aspects of QuitNet members’ content-based affiliation with different kinds of communication content as a means to investigate content-based network diffusion. Unlike the majority of studies on social networks, where the ties between actors were deductive in nature,³⁷ our study took an empirical, bottom-up approach to explaining the communication patterns within QuitNet. We derived themes from grounded theory–based qualitative analysis of QuitNet messages. Our findings suggest new directions for developing network interventions³⁸ for public health by focusing on content-based, targeted behavior change strategies. In addition, our research shifts intervention tailoring from an exclusive reliance on theory^39,40 to an empirical approach driven by the characteristics and needs of end users, as revealed by their communication content.

Limitations

Our QuitNet data set was recorded in 2007 and was limited in size. Future studies should obtain more recent data. The accuracy of the automated system may be further improved by sophisticated choice of machine-learning algorithms.

Our 2-mode network analysis also had a few limitations. First, although themes were empirically derived from the QuitNet data set, we might not have captured all of the communication themes. Network diffusion through content-based affiliation may not have been limited to these themes. Second, our analysis was cross-sectional and not longitudinal, which might have limited the understanding of causality of content affiliation to abstinence behavior as well as to potentially dynamic patterns of content affiliation in a network. Third, our analysis considered only online communication that occurred on the QuitNet platform. Some of these QuitNet members might have had offline communication,⁴¹ and previous research indicates that both are important in influencing behaviors.⁴² Future research should consider both in-person and online communication that occurs among individuals.

Finally, the autoregression models we used did not account for the dependency of ties in the data. Modeling the networks with exponential random graph models is 1 way to consider network dependency.^43–45 However, complexity of estimation and simulation of these models cause some practical difficulties in achieving model convergence, especially for large networks like ours. Nonetheless, future work would benefit from the use of exponential random graph models to model 2-mode^46,47 and multilevel⁴⁸ networks to address the interdependencies associated with the network data.

Conclusions

Our findings offer insights into the utility of (1) qualitative analysis of QuitNet communication data to understand the manifestation of behavior change constructs in the context of online platforms, (2) automated text analysis methods to facilitate large-scale analysis of online network data, and (3) affiliation networks to understand the nature of the specialized content underlying communication events in the context of smoking cessation efforts in an online community.

We found that QuitNet members who were exposed to abstinent members who exchanged content related to group-centric interpersonal themes (e.g., social support, traditions) tended to remain abstinent. This result can be used to build better online social interventions, with enhanced persuasive and engagement features. For instance, incorporation of an explicit display of member profiles that contribute such content may enhance affiliations to these people and, thereby, facilitate meaningful connections that harness network diffusion and affiliation exposure.

Online social networks have been gaining in popularity and present health researchers with a unique opportunity to understand human behavior change and to deliver scalable and sustainable interventions. The development of better tools to analyze social network content of this nature allows us a greater understanding how such social networks mediate behavior change and thereby provides us the opportunity to implement empirically grounded interventions to further help these communities attain their goals. Content-based network analysis of QuitNet, made feasible by large-scale qualitative analysis with automated methods, has been shown to yield content-specific patterns that can be used for targeted health promotion and behavior change.