Highlights
-
•
Social media analytic tools offer ways to evaluate online sex health interventions.
-
•
Most tools measure reach, aggregate activity, and page- and post-level engagement.
-
•
User-level activity and engagement metrics are rarely provided.
-
•
Various metrics comprising composite engagement scores are difficult to discern.
-
•
Most tools have limited or no capabilities in text and social network analytics.
Keywords: Interventions, Social media, Social media analytics, Engagement, Intervention evaluation, Sexual health
Abstract
Unprecedented public engagement with social media has provided viable and culturally relevant platforms for application in sexual health interventions, yet there are concerns that methods for evaluating engagement in these interventions have not kept pace with their implementation. More recently, the rise of social media analytics (SMA) and online marketing has spawned the development of analytic tools that boast promise for such a task. In this paper, we review a sample of the most popular of these tools, paying particular attention to: (1) the social media platforms that can be analyzed; (2) analytic capabilities; and (3) measures of engagement. We follow this with a review of sexual health intervention studies that apply these tools in evaluation efforts. Our findings suggest that these tools have numerous analytic capabilities that would be useful for evaluating interventions more efficiently. However, in nearly all cases, the tools we reviewed alone would not be sufficient to fully grasp engagement dynamics, as they need to be complemented with additional tools for textual analysis and social network analysis. Therefore, we consider this fertile ground for future collaborations between software developers and behavioral health scientists to develop more comprehensive analytic platforms with applications for public health research.
1. Introduction
Social media — the Web 2.0 technologies that enable connection, communication, and content sharing among users (Holloway, 2014) — have rapidly become mainstream features of the American social milieu and, consequently, have altered how individuals interact, communicate, learn, and make decisions. Technologies classified as “social media” are numerous and include online social networking platforms like Facebook, Instagram, and Twitter, wikis for collaborative content development, blogs, message boards, and two-way mobile messaging platforms that connect users through cell phones and personal digital assistants (Korda and Itani, 2013). Today, about seven-in-ten Americans from across income and age spectrums use social media of some kind, including men and women, and Whites, African Americans, and Latinos in nearly equal parts (Pew Research Center, 2019). And, for many of these users, social media is part of their daily routine, with majorities of Facebook, Instagram, and Snapchat users visiting these sites at least once daily (Pew Research Center, 2019).
Additionally, social media, like the Internet more broadly, has quickly become a source of health information. About 80% of adult internet users report seeking some form of health information online (Fox, 2012). Although a vast majority of these searches (80%) begin with search engines, there is evidence that growing numbers are turning to social media as well (Korda and Itani, 2013, Fox, 2011), for example by following friends’ personal health experiences or seeking health information and advice from their peers. Taken together, unprecedented public engagement with social media for social and health-related purposes has provided viable and culturally appropriate platforms for conducting public health interventions (Bennett and Glasgow, 2009, Rietmeijer et al., 2009, Kamel Boulos and Wheeler, 2007, Young, 2013).
This is particularly true for interventions that address highly stigmatized health topics like HIV prevention, for which fears of discrimination and desires for anonymity create barriers to traditional in-person modalities of healthcare engagement (Mahajan, 2008, Taggart, 2015, Eaton, 2015, Fisher, 2017, Rossman et al., 2017). In addition to its ubiquity and reach, social media offer affordances that are critical for engaging people around stigmatized health issues like HIV (Korda and Itani, 2013, Webb, 2010). For example, the networked architecture of social media creates an incubator for the provision of social support and a sense of connectedness among users of stigmatized populations (Meng et al., 2016, Strand and Blogging, 2012), and can also inform perceptions of health behavior norms (Latkin, 2009, Latkin, 2003, Leahey, 2015, Barrington, 2009). Social media also prioritize peer-driven messaging and content creation, which are thought to have greater influence on the attitudes and decisions of younger adults (Kamel Boulos and Wheeler, 2007, Wong et al., 2011) and people who experience stigma and discrimination from mainstream institutions (Young, 2011, Young, 2017, Kelly, 1991). As a result of these affordances, social media HIV prevention interventions are on the rise and are showing promise for engaging traditionally hard to reach populations like adolescents, men who have sex with men, and people who use injection drugs (Young, 2013, Bull, 2012, Ko, 2013, Lelutiu-Weinberger, 2015, Gaysynsky et al., 2015, Patel, 2018, Horvath, 2013, Bauermeister, 2015, Rhodes, 2016).
That said, there are concerns that methods for evaluating social media-based interventions, particularly with respect to how participants engage with them, have not kept pace with their implementation (Lim, 2017, Perski, 2017, Short, 2018). In general, as individuals use social media, they produce original content in the forms of posts and photos, interact with the content of other users via comments, reactions, or shares, and make network connections. Together, these data create time-stamped digital archives of user activity that grant researchers the ability to track real-time responses to the intervention. For example, in group-based social media interventions, the number of posts, comments, and reactions made by each participant can be used as measures of their personal study engagement (Cheung, 2015, Haines-Saah, 2015, Valle and Tate, 2017). At the post-level, the degree to which particular types of posts elicit comments can be used to show which types of content are more engaging (Thrul et al., 2015, Young and Jaganath, 2013). Further, in studies with multiple treatment groups, metrics like number of unique posts and number of photo or video views can be used to measure engagement at the group-level (Bull, 2012, Pedrana, 2013). Metrics like these not only help quantify and describe the nature of engagement, but they can also be used to evaluate relationships between intervention engagement and study outcomes or participant satisfaction and retention (Bauermeister, 2017).
Perhaps due to the vast amount, dynamics, and complexity of these data, early efforts to evaluate engagement in group-based social media interventions have typically relied on either high-level summaries of “usage” (e.g., number of page hits, time spent on a page) (Korda and Itani, 2013, Lim, 2017), or relied on manually tallied metrics (e.g., counting the number of posts each participant interacted with and/or initiated) (Hales et al., 2014, Kim, 2017, Merchant, 2014). While the former is not well suited for assessing the granular nuances of study engagement, the later leaves the door open for human error and compromised data integrity.
More recently, however, the rise of social media analytics (SMA) has spawned the development of tools that boast promise for application in the evaluation of social media interventions (Lim, 2017, Baghaei, 2011). Applied across domains including business, marketing, and politics, SMA enables continuous collection, monitoring, analysis, and summary of user-generated content and social interactions to provide in-depth analysis of real-time user preferences, choices, and sentiments (Stieglitz, 2014). Despite the availability of these tools, little is known about whether and how they have been applied to evaluate engagement in HIV prevention and other sexual health interventions. The purpose of this review is to: (1) describe the analytic capabilities of some of the more popular SMA tools on the market, with an emphasis on how they measure user engagement, (2) illustrate how these SMA tools have been applied to HIV/sexual health interventions through a review of the literature, (3) discuss limitations of these tools as made apparent by our assessment of the literature and from our experiential knowledge of the needs of public health interventionists, and (4) propose directions for future research and development.
2. Methods
2.1. Search strategy
2.1.1. Social media analytic (SMA) tools
To illustrate the essential analytic capabilities of SMA tools, we identified a small but heterogeneous sample of exemplars. To this end, we conducted a basic search engine query using the terms: (“Social media analytics” OR “web analytics” OR “social media software” OR “social media management”). Search results included two types of hits — websites for individual software companies and industry reviews and blogs. Although both types of hits represent algorithmic or industry biases, we opted to rely on industry reviews and blogs, as our goal was to identify popular exemplars as opposed to creating an exhaustive or even comprehensive directory. Industry reviews and blogs were, therefore, a more efficient approach to identifying these models.
We included analytic tools if: (1) their websites described at least one measure of engagement; (2) they were reviewed on at least two industry blogs; and (3) they were compatible with Facebook. Further, we excluded tools that were almost singularly designed for brand management and competitive benchmarking as we deemed these SMA tools to be the least applicable for evaluating group-based health interventions. Our choice to limit our analysis of SMA tools to include only those that were compatible with Facebook (along with a variety of other social media platforms) was a pragmatic one. Although social media platforms like Instagram and Twitter are used widely in diverse communities and offer technological facilities for intervention delivery, no platform is as ubiquitous or as familiar to target audiences as Facebook. For this reason, researchers tend to use Facebook for public heath interventions, as it presents few barriers to acceptance. We opted to conduct this review of SMA tools, then, in a way that reflected these trends and would be most useful to researchers in the field.
The first and second authors screened the results of the search engine inquiry using the inclusion and exclusion criteria, after which they reviewed the websites for each analytic tool in detail for key descriptors. Data from this review were compiled into a table of evidence (see Table 1). For each included tool, we extracted the platforms (e.g., Facebook, Instagram, Twitter, etc.) it analyzes, key metrics for Facebook data, and types of engagement metrics.
Table 1.
Characteristics of social media analytic Tools.
| Tool Name and URL | Access | Social Media Platforms | Analytics for Facebook Data | Engagement Metrics |
|---|---|---|---|---|
|
Paid |
|
|
|
|
Paid |
|
|
|
|
Paid |
|
|
|
|
Free |
|
|
|
|
Free |
|
|
|
|
Paid |
|
|
|
|
Paid |
|
|
|
|
Paid |
|
|
|
|
Paid |
|
|
|
|
Paid |
|
|
|
|
Paid |
|
|
|
|
Free |
|
|
|
|
Paid |
|
|
|
|
Paid |
|
|
|
|
Paid |
|
|
|
|
Paid |
|
|
|
2.1.2. Literature search
The literature review used a systematic approach to retrieve relevant studies. In March 2020, we searched PubMed Central for published studies from 1/1/2010 to 2/29/2020. The search was developed with the assistance of a librarian based on key terms, medical subject headings (MeSH) terms, synonyms, and subject headings related to three topics: (1) HIV, (2) behavior interventions, (3) social media, and (4) the SMA tools featured in this paper (see Table S1).
The retrieved articles were screened for relevance and selection criteria. The inclusion criteria were: (1) focus on HIV/AIDS or sexual health behaviors; (2) included social media as a program/intervention component; (3) measured program/intervention engagement; and (4) used a SMA tool to capture that engagement. Exclusion criteria included: (1) discussion of a behavioral intervention was limited to study implications; (2) discussion of sexual health was limited to study implications; (3) not in English; and (4) systematic reviews and commentaries. We completed title, abstract, and full-text review to identify all studies meeting inclusion and exclusion criteria.
3. Results
3.1. Overview of SMA tools
A total of 78 tools were identified from product reviews on industry blogs. After deduplication, 36 remained for further consideration. Of these, five were excluded because they were not designed to handle Facebook data. An additional nine were excluded for being named in only one industry product review, and six more were excluded for their orientation toward competitive benchmarking and brand management. The remaining 16 software tools met criteria and were included in this review. A summary of the data extracted for this review is shown in Table 1.
3.1.1. Social media platforms
Given our inclusion criteria, all 16 analytic tools are compatible with Facebook. Of these, all but one (Grytics) were designed for Facebook Pages, while three (Facebook Insights, Grytics, Sociograph.io) analyzed Facebook Group data. Collectively, the tools included in this review also analyzed Instagram, Twitter, LinkedIn, Pinterest, YouTube, Snapchat, and Tumblr data, with Twitter and Instagram being the most common platforms next to Facebook. With the exceptions of Google Analytics and Shared Count, which can be used on any url including social media pages, Quintly is compatible with the most social media platforms (7 in total), followed by Agorapulse, Amplifr, and Sprout Social with applications for five platforms each. In contrast, tools like Grytics and Sociograph.io are designed strictly for Facebook.
3.1.2. Analytics for Facebook data
Our review of the analytic capabilities of each SMA tool was informed by two concepts — activity and engagement — that we consider germane to the task of evaluating participation in a social media-based intervention. Activity in a social media-based intervention captures the degrees to which participants are paying attention to intervention content (e.g., time spent logged into the intervention profile page or group, page or content views) and/or are contributing to intervention content (e.g., posting). We distinguish activity from engagement, which we define as participants’ interactions with intervention content (e.g., comments, reactions, shares).
A review of the websites for each SMA tool yielded six types of analytics for Facebook data that we felt could contribute to a more robust understanding of intervention activity and engagement. These include analytics of Reach & Impressions, Audience Analysis, Aggregate Activity, Post Types, Engagement, and Text Analysis. Reach & Impression metrics capture the scope of a Profile’s audience, either in the form of how many people saw any content from the Profile or how many times any content from the Profile entered a person's screen. Audience Analysis metrics provide insights on the demographics of a Profile’s member, follower, or fan base as well as the growth of that base over time. Aggregate Activity metrics provide tallies, often over time, of profile-level activities like total time spent on the site or total number of posts. Post Type metrics capture at the profile-level the information content of posts, for example whether they contain photos, videos, or links. Engagement metrics encapsulate at profile-, post-, and member-levels of analysis the degree to which members/followers/fans interact with original posts, for example through comments, likes, reactions, and shares. Finally, Text Analysis capabilities like tagging and sentiment characterization were also present.
Based on what we could confirm from our survey of websites, we identified 12 tools that provided aggregated activity metrics, nine that provided metrics on profile reach and impressions, ten that performed audience analysis, seven that provided breakdowns of post types, and three that performed some form of text-based analysis. And, given the inclusion criteria for this review, all 16 analytic tools provided metrics on aspects of engagement. All of the analytic tools in this review performed analysis in at least 2 of these areas, with Agorapulse and Buffer each offering analytics in 5 data areas.
3.1.3. Engagement metrics
Given that the primary motivation of this study was to evaluate how SMA tools can be leveraged to measure participant engagement in social media-based interventions, we prioritized engagement metrics in our review. Specifically, we investigated how each SMA tool captured engagement with intervention content through comments, likes, shares, and reactions. In general, we were able to identify four types of engagement metrics, including page-level, post-level, member-level metrics, and engagement-based recommendations. Page-level metrics of engagement are summary statistics of how frequently members/followers/fans interact with posts on a social media profile page. Quite often these page-level statistics are in the form of totals or rates of interactions like comments, likes, shares, and reactions. Another common page-level engagement metric is a composite engagement score for the page. Unfortunately, the lack of information provided on a tool’s website about how composite engagement scores are calculated means that metrics like these can be challenging to substantively interpret.
Post-level engagement was captured as either summary statistics representing the average number of interactions per post on a given page, or as more granular statistics, which detail the total number of interactions (e.g., comments, likes, shares, reactions) for each post, for “top posts”, and/or for types of posts (e.g. photos, video, links). Two tools (Facebook Insights and sociohgraph.io) also provided member-level engagement metrics, which provided insights about particular members/followers/fans, typically for those who were considered “top contributors” or “influencers.” That said, due to changes in Facebook’s privacy policies, Facebook group members must now opt-in to allow access to their personal information by third-party SMA tools. Thus, this metric may no longer be as useful. Finally, engagement-based recommendations drew on past content interaction trends and machine-learning algorithms to make recommendations for which post types, post times, and post lengths will yield the greatest return in engagement.
Of the 16 analytic tools in this review, 11 provide page-level engagement metrics, 12 provide post-level metrics, two provide member-level metrics, and two provide engagement-based recommendations. Unsurprisingly, Facebook Insights is the only tool that provides analytics in all four engagement categories.
3.2. SMA tools in the literature
In the second stage of this review, we conducted a literature search to assess whether and how the SMA tools identified in the first stage of the review have been applied in HIV/sexual health interventions. We do this not only as a way to illustrate the practical utility of these particular tools, but to also demonstrate limitations in the tools themselves and the scope to which they have been applied in the field.
3.2.1. Study characteristics
Of the 77 articles extracted, 19 were deemed relevant based on an initial review of the title and abstract. Of those 19, six were ultimately excluded after a more thorough full-text review revealed that the discussion of HIV/sexual health or of a health behavior intervention was limited to study implications, per the exclusion criteria previously outlined. This left 13 studies that met criteria for final inclusion (Bull, 2012, Patel, 2018, Pedrana, 2013, Andrade, 2018, Dehlin, 2019, Dowshen et al., 2015, Dulli, 2018, Jones et al., 2015, Lambert, 2014, Ragsdale, 2015, Solorio, 2016, Syred, 2014, Wang, et al., 2019) (Fig. 1). Study characteristics are shown in Table 2. Of the 13 studies, 10 were observational (Pedrana, 2013, Andrade, 2018, Dehlin, 2019, Dowshen et al., 2015, Jones et al., 2015, Lambert, 2014, Ragsdale, 2015, Solorio, 2016, Syred, 2014, Wang, et al., 2019), two were randomized-control trials (RCTs) (Bull, 2012, Patel, 2018), and one employed a pre-post design (Dulli, 2018). In total, 11 studies were conducted in the United States (Bull, 2012, Patel, 2018, Andrade, 2018, Dehlin, 2019, Dowshen et al., 2015, Jones et al., 2015, Lambert, 2014, Ragsdale, 2015, Solorio, 2016, Syred, 2014, Wang, et al., 2019), with only two studies coming from other countries (Pedrana, 2013, Dulli, 2018).
Fig. 1.
Study Selection Process.
Table 2.
Characteristics of the Studies Included in the Review (n = 13).
| Study | Study Design | Study Location | Target Population | Health Issue | Social Media Platforms | Role of Social Media | Analytic Tools | Metrics |
|---|---|---|---|---|---|---|---|---|
| Andrade (2018) | Observational | USA (Washington, DC) | Latino Immigrant Youth | Sexual health; Substance use; Interpersonal violence | Facebook pages | Disseminate information; Engage participants; Build community | Facebook Insights |
|
| Bull (2012) | RCT | USA (multiple settings) | Young adults | STI prevention | Facebook pages | Disseminate information; Engage participants | Google Analytics |
|
| Dehlin (2019) | Observational | USA (Chicago, IL) | Gender and sexually marginalized communities | Pre-exposure Prophylaxis (PrEP) for HIV prevention | Facebook AdsInstagram Ads Campaign website | Disseminate information | Google Analytics |
|
| Dowshen et al. (2015) | Observational | USA (Philadelphia, PA) | Youth | HIV and STI testing | Facebook page; Instagram page, Twitter page | Disseminate information | Facebook Insights |
|
| Dulli (2018) | Pre-post Study | Nigeria | Youth and Young Adults | HIV care | Facebook groups | Disseminate information; Engage participants; Build community | Grytics |
|
| Jones et al. (2015)) | Observational | USA (multiple settings) | Black women | HIV prevention | Facebook pages | Recruit participants | Google Analytics |
|
| Lambert (2014) | Observational | USA (Atlanta, GA) | Youth and Young Adults | Dating abuse/partner violence | Facebook pages; Twitter, YouTube, Flickr, Tumblr, and Pinterest | Disseminate information | Facebook Insights |
|
| Patel (2018) | RCT | USA (New York City, NY) | Young Black and Latino Gay and Bisexual Men (YBLGBM) | Pre-exposure Prophylaxis (PrEP) for HIV prevention | Facebook groups | Disseminate information; Engage participants | Grytics |
|
| Pedrana (2013) | Observational | Australia | Gay men | Sexual health-general | Facebook pages | Disseminate information; Engage participants | Facebook Insights |
|
| Ragsdale (2015) | Observational | USA (Mississippi)) | Youth | Sexual health-general | Facebook pages | Disseminate information | Facebook Insights |
|
| Solorio (2016) | Observational | USA (Seattle, WA) | Young Latino Immigrant Men who have sex with men (MSM) | HIV testing | Facebook pages | Disseminate information | Sprout Social |
|
| Syred (2014) | Observational | USA (nation-wide) | Youth and Young Adults | STI prevention | Facebook pages | Engage participants | Google Analytics |
|
| Wang et al. (2019) | Observational | USA (nation-wide) | Young Latinos | Sexual health-general | Facebook pages | Disseminate information; Engage participants | Facebook Insights |
|
3.2.2. Key populations
Young people were the primary population for the featured studies, with 10 of 13 studies tailored to youth (aged 12–17) (Andrade, 2018, Dowshen et al., 2015, Ragsdale, 2015), young adults (aged 18–34) (Bull, 2012, Patel, 2018, Solorio, 2016, Wang, et al., 2019), or both (Dulli, 2018, Lambert, 2014, Syred, 2014). Of these, four focused on young people of color (Patel, 2018, Andrade, 2018, Solorio, 2016, Wang, et al., 2019). Irrespective of age and race, four of the 13 studies were specifically tailored to gender and sexual minority populations (Patel, 2018, Pedrana, 2013, Dehlin, 2019, Solorio, 2016) and one study focused singularly on Black women (Jones et al., 2015).
3.2.3. Sexual health context
Although the inclusion criteria for this review stipulated that all studies focus on sexual health, the sexual health issues featured were diverse. In total, seven studies focused on HIV or STI prevention modalities. Of these, two underscored HIV and/or STI testing (Dowshen et al., 2015, Solorio, 2016), two more focused on pre-exposure prophylaxis (PrEP) (Patel, 2018, Dehlin, 2019), and three studies addressed HIV and/or STI prevention more broadly (Bull, 2012, Jones et al., 2015, Syred, 2014). In addition to prevention themes, one study concentrated on HIV care engagement (Dulli, 2018), one focused on dating abuse/partner violence (Lambert, 2014), and one more addressed the confluence of sexual health, substance use, and interpersonal violence (Andrade, 2018). Three studies cast a wide net on sexual health more broadly (Pedrana, 2013, Ragsdale, 2015, Wang, et al., 2019).
3.2.4. Social media platforms
The primary social media platform used in all 13 studies was Facebook. Specifically, 10 studies used Facebook Pages (Bull, 2012, Pedrana, 2013, Andrade, 2018, Dowshen et al., 2015, Jones et al., 2015, Lambert, 2014, Ragsdale, 2015, Solorio, 2016, Syred, 2014, Wang, et al., 2019), two studies used Facebook Groups (Patel, 2018, Dulli, 2018), and one study used Facebook Advertisements (Dehlin, 2019). The use of Facebook as a component of the sexual health intervention or program was exclusive in 10 studies (Bull, 2012, Patel, 2018, Pedrana, 2013, Andrade, 2018, Dulli, 2018, Jones et al., 2015, Ragsdale, 2015, Solorio, 2016, Syred, 2014, Wang, et al., 2019), while three studies leveraged additional platforms (Dehlin, 2019, Dowshen et al., 2015, Lambert, 2014), including Instagram, Twitter, YouTube, Tumblr, Pinterest, and Flickr.
3.2.5. Role of social media
From our qualitative assessment of the studies included in this review, we identified four functions that social media played in the context of the featured intervention or program. Across all studies, disseminating information was the most common function observed, with 11 studies using social media in this capacity (Bull, 2012, Patel, 2018, Pedrana, 2013, Wang, et al., 2019, Andrade, 2018, Dehlin, 2019, Dowshen et al., 2015, Dulli, 2018, Lambert, 2014, Ragsdale, 2015, Solorio, 2016). Using social media in this way represents a top-down style of communication, where researchers provide the information they want participants to see and engage with, a common approach when there is an educational objective to the intervention. Social media also served as a means to engage participants in the intervention. Interventions that encourage participants to post their own content on the platform, to respond to others’ posts (i.e., commenting, liking, reacting), or to share content fall into this category. In total, eight studies leveraged social media to this end (Bull, 2012, Patel, 2018, Pedrana, 2013, Andrade, 2018, Dulli, 2018, Syred, 2014, Wang, et al., 2019). Two more studies used social media to build community among participants (Andrade, 2018, Dulli, 2018), for example by strengthening social support or co-producing shared knowledge and understanding. Finally, one study used social media purely as a recruitment tool for an online video intervention (Jones et al., 2015). In six studies, social media played more than one role (Bull, 2012, Patel, 2018, Pedrana, 2013, Andrade, 2018, Dulli, 2018, Wang, et al., 2019), most often pairing dissemination and engagement functions.
3.2.6. SMA tools
Across the studies, a total of four SMA tools were utilized to evaluate aspects of each sexual health intervention or program. Six studies used Facebook Insights (Pedrana, 2013, Andrade, 2018, Dowshen et al., 2015, Lambert, 2014, Ragsdale, 2015, Wang, et al., 2019), four studies used Google Analytics (Bull, 2012, Dehlin, 2019, Jones et al., 2015, Syred, 2014), two studies used Grytics (Patel, 2018, Dulli, 2018), and one used Sprout Social (Solorio, 2016).
3.2.7. Analytic metrics
Across the studies in our sample, we found that the featured SMA tools were used by the authoring teams to provide four types of metrics relevant to intervention evaluation, namely Reach, Audience Analysis, Aggregate Activity, and Engagement metrics. In the studies included in this review, 11 studies reported measures of intervention or campaign reach (Bull, 2012, Pedrana, 2013, Andrade, 2018, Dehlin, 2019, Dowshen et al., 2015, Jones et al., 2015, Lambert, 2014, Ragsdale, 2015, Solorio, 2016, Syred, 2014, Wang, et al., 2019), which speaks to the degree of exposure the study received. Reach was typically operationalized as either the number of views a recruitment advertisement received (Dehlin, 2019, Jones et al., 2015), the number of unique visitors to the study’s social media page (Bull, 2012, Pedrana, 2013, Andrade, 2018, Dowshen et al., 2015, Lambert, 2014, Ragsdale, 2015, Solorio, 2016, Syred, 2014), or the number of times a post was viewed (Wang, et al., 2019). Five of these 11 studies also featured Audience Analysis metrics (Pedrana, 2013, Dehlin, 2019, Lambert, 2014, Ragsdale, 2015, Syred, 2014) to understand the demographic characteristics of those who were touched by components of the intervention or program.
SMA tools were also used to capture activity and engagement metrics. Specifically, nine studies reported aggregate measures of page activity (Bull, 2012, Patel, 2018, Pedrana, 2013, Dehlin, 2019, Dulli, 2018, Jones et al., 2015, Ragsdale, 2015, Solorio, 2016, Syred, 2014), typically in the form of average time spent on a page and the number of original posts. In total, 11 studies also captured the degree to participants engaged with posts (Bull, 2012, Patel, 2018, Pedrana, 2013, Andrade, 2018, Dulli, 2018, Jones et al., 2015, Lambert, 2014, Ragsdale, 2015, Solorio, 2016, Syred, 2014, Wang, et al., 2019). Nine studies featured page-level metrics of engagement (e.g., total number of comments, likes, reactions) (Bull, 2012, Patel, 2018, Pedrana, 2013, Dulli, 2018, Jones et al., 2015, Lambert, 2014, Ragsdale, 2015, Solorio, 2016, Syred, 2014) and five studies featured post-level metrics of engagement (e.g., comments, likes, reactions per post) (Patel, 2018, Andrade, 2018, Dulli, 2018, Ragsdale, 2015, Wang, et al., 2019). One study captured engagement at the user level (e.g., number of comments, like, reactions per user) (Dulli, 2018). No studies included analysis of post types or text.
4. Discussion
The mainstreaming of social media has had unprecedented effects on how individuals interact and communicate and, consequently, has opened the door to innovative ways to engage at-risk populations in HIV and sexual health care. That said, more rigorous methods of evaluating participation and engagement in social media-based interventions are needed to push these programmatic applications forward. SMA tools designed initially for marketing and audience research may be well positioned to satisfy this need.
Our exploration of this hypothesis through a review of SMA tools and the HIV/sexual health studies that apply those tools yielded several insights that have implications for intervention design and evaluation. Regarding platform compatibility, all 16 tools included in the review provided analytics for Facebook data (which was guaranteed given the inclusion criteria we adopted). Still the most ubiquitously used social media platform by Internet using adults, Facebook offers two types of forums that are conducive to online health programs – Facebook Groups and Facebook Pages, which, together, were almost ubiquitously used as the intervention platform of choice in the studies featured in our review. That said, Facebook Pages become public and visible once they are published, which may deter participation in online health programs, especially those that focus on stigmatized behaviors and populations. Facebook Groups, on the other hand, can be made private and, therefore, invisible to third-parties, making them ideal for online behavioral health interventions that require high degrees of administrative control and discretion. That only three of the SMA analytic tools were compatible with Facebook Group data is a noteworthy limitation in this regard.
Facebook aside, we found that most of the SMA tools were also compatible with other social media platforms like Instagram, Twitter, and YouTube, which have immense potential for reaching and engaging larger and more diverse populations in sexual health prevention and treatment. As health behavior interventions expand beyond the world of Facebook and adopt a repertoire of platforms to foster interactions with participants, having access to tools capable of analyzing user activity and engagement on multiple social media platforms, while also making cross-platform comparisons on those metrics, will be critical.
Our review also identified in each tool an array of analytic capabilities including reach and impressions analysis, audience analysis, aggregate activity analysis, post analysis, text analysis and engagement analysis. At first glance, the utility of these metrics for evaluating activity and engagement in a social media intervention might appear obvious. For example, a public health campaign that uses an open community like a Facebook Page as its online face would find Reach & Impressions metrics might seem useful for evaluating the overall reach of the campaign. In the same vein, the ability to analyze audience demographics could help researchers determine whether the intended audience is being reached. Metrics summarizing aggregate activity could help researchers understand the degree to which their intervention is eliciting appropriate levels of participation, while analyses of post content could provide a more granular view of what that activity looks and sounds like. And finally, engagement metrics potentially offer insights on the degree of interactivity among participants and the content that attracts the most attention.
That being said, a more careful examination of these tools and their applications to sexual health interventions reveals that the utility of these metrics is more limited than one would initially assume. First, some of the metrics discussed are rarer and, therefore, more under-developed than others. This is particularly the case for text-based analytics. One of the advantages to hosting group interventions online is that what participants discuss in those settings is digitally archived. This discourse, in turn, can be analyzed to provide critical insights about topics and themes related to the behavioral outcome of interest. However, only three analytic tools in the review provided any form of built-in textual analysis. Until this trend is reversed, efforts to analyze topics of conversation in social-media based interventions will have to rely on additional means to analyze social media text, for example human coding, whereby researchers use grounded or deductive coding schemes to categorize posts into various topics or themes (Young and Jaganath, 2013, Rus and Cameron, 2016), or machine learning approaches for automated textual analysis (Aramaki et al., 2011, Cole-Lewis, 2015). A single cohesive platform that has the ability to analyze both post engagement and post content would enable near real-time analysis of which Facebook intervention topics are most engaging.
Second, we also note features that would likely be useful for public health interventionists but are entirely absent from the SMA tools we identified. In a social media intervention, the structure of relationships among group members or followers/fans of a profile page are fundamental mechanisms in both study and behavioral engagement (Valente, 2012, Ghosh, 2017). However, to perform social network analyses with social media data, one must rely on a combination of data extraction and analytic techniques beyond the proprietary tools featured here. For example, in one study, researchers assessed changes in the social network structure of individuals participating in an online Facebook group, and whether these changes were associated with HIV prevention behaviors (Young, 2014). To capture newly formed Facebook friendships among study group members, manual screenshots were taken of each participants’ “friend lists” to identify new network ties that emerged during the study. The development of SMA tools that can perform this type of data extraction and analysis, therefore, requires attention.
Also absent are adequate descriptions of how many of the metrics we underscored are calculated, particularly for composite engagement scores. One can surmise that these “black boxes” may be a result of business decisions to protect proprietary information. Although it is possible that this information may be available to paid subscribers, the lack of transparency around these scores is problematic, particularly for academic researchers. The expectation that researchers be able to adequately describe how they derived their measures is central to their academic integrity. As such, “black box” measures such as these should be avoided or used with caution. Furthermore, when a general lack of transparency is combined with competitive pressures to produce distinctive products, it becomes difficult to imagine that one standardized measure of engagement could emerge from the marketing research sector.
Finally, our review also reveals a striking limitation in how engagement metrics are applied in intervention evaluations. Although all of the SMA tools we reviewed provided some measure of engagement, particularly at page- and post-levels, we learned that studies applying these tools are largely doing so to capture high-level summaries as opposed to more granular measures at the post- or user-levels. The limited reporting of user-level engagement metrics is almost certainly derivative of changes Facebook and other platforms have made to their data sharing and privacy policies. Regardless, this is suggestive of a missed opportunity to link a participant’s study engagement to desired health or health behavior outcomes, which is particularly relevant for HIV prevention interventions where achieving adequate levels of interactivity to stimulate desired behavior change can be challenging (Swanton et al., 2015). Individual-level metrics of content engagement could be used to determine how much interactivity is needed to achieve intervention effectiveness and, subsequently, to provide a benchmark for desired engagement for future interventions.
The fact that social media interventionists seem to apply SMA metrics in a limited fashion may be indicative of a misalignment between the measurement demands being met by SMA software developers (e.g., product/brand engagement) and the measurement needs of public health interventionists (e.g., intervention engagement). When developing an evaluation plan for an intervention, it is incumbent that outcome measures reflect the definitions of those outcomes as prescribed by the research team. Thus, for measuring engagement, it is imperative that SMA metrics that capture engagement be tailored to the specific aims and engagement outcomes of the intervention. For this reason, it seems more prudent for social media interventionists to drive the development of their own computational tools that suit their measurement needs than to draw on available SMA tools that are not designed to be congruent with the demands of intervention evaluation.
There are limitations to our review worth highlighting. Regarding our search strategy, because we lacked an exhaustive sampling frame of all SMA tools available, we relied on industry blogs to guide us toward the more popular tools. As such, our sample was biased toward industry favorites and may not be representative of the full range of analytic services available. That said, our literature search revealed high reliance on an even smaller subset of these industry favorites, namely the few free ones, putting into question the relevance of an even more expansive search. Relatedly, we also biased our search toward SMA tools that were compatible with Facebook, thereby ignoring any tools designed without Facebook analytic capabilities. However, our review showed that Facebook was the ubiquitous platform of choice in the studies included in our review, leaving us with the impression that any tool that could be used to measure intervention engagement would almost certainly have to be compatible with Facebook to be relevant. Given our special interest in Facebook interventions, we also oriented our review of analytic capabilities to only those for Facebook data. Whether or not the SMA tools provided comparable metrics for each additional platform they serviced is an open question.
5. Conclusion
Motivated by the need for computational methods for measuring activity and engagement in social media-based sexual health interventions, we sought to review the analytic capabilities of a sample of SMA tools. Our findings suggest that these tools have numerous analytic capabilities that could be useful for evaluating interventions, but rarely at the granular levels needed to link intervention engagement to individual health outcomes. In almost all cases, the tools we reviewed would not be sufficient on their own to fully grasp engagement dynamics, as they lack critical capacities for evaluating its discursive and relational underpinnings. Based on our knowledge and exploration of tools, no software is able to achieve this goal as of yet. Therefore, we consider this fertile ground for collaborations between computer scientists, technology developers, and behavioral health scientists to develop more comprehensive analytic platforms with applications designed to be more responsive to the situational needs and goals of public health interventionists.
Ethical approval
NA.
Informed consent
NA.
Funding
Support of this work was provided by the National Institute of Mental Health (Grant Number: 5R01 MH106415) and the National Institute of Child Health and Human Development (Grant Number: K99 HD094648).
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper
Acknowledgements
We would like to thank Anamika Barman-Adhikari and Claire Houlihan for their comments on earlier drafts of the paper.
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.pmedr.2020.101158.
Appendix A. Supplementary data
The following are the Supplementary data to this article:
References
- Andrade E.L. Strategies to increase latino immigrant youth engagement in health promotion using social media: mixed-methods study. JMIR Publ. Health Surveillance. 2018;4(4) doi: 10.2196/publichealth.9332. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Aramaki E., Maskawa S., Morita M. Proceedings of the Conference on Empirical Methods in Natural Language. Association for Computational Linguistics; 2011. Twitter catches the flu: detecting influenza epidemics using Twitter; pp. 1568–1576. [Google Scholar]
- Baghaei N. Engaging families in lifestyle changes through social networking. Int. J. Human-Comput. Interact. 2011;27(10):971–990. [Google Scholar]
- Barrington C. Talking the talk, walking the walk: social network norms, communication patterns, and condom use among the male partners of female sex workers in La Romana, Dominican Republic. Soc. Sci. Med. 2009;68(11):2037–2044. doi: 10.1016/j.socscimed.2009.03.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bauermeister J.A. Acceptability and preliminary efficacy of a tailored online HIV/STI testing intervention for young men who have sex with men: the Get Connected! program. AIDS Behav. 2015;19(10):1860–1874. doi: 10.1007/s10461-015-1009-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bauermeister J.A. Addressing engagement in technology-based behavioural HIV interventions through paradata metrics. Curr. Opin. HIV AIDS. 2017;12(5):442–446. doi: 10.1097/COH.0000000000000396. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bennett G.G., Glasgow R.E. The delivery of public health interventions via the Internet: actualizing their potential. Annu. Rev. Public Health. 2009;30:273–292. doi: 10.1146/annurev.publhealth.031308.100235. [DOI] [PubMed] [Google Scholar]
- Bull S.S. Social media–delivered sexual health intervention: a cluster randomized controlled trial. Am. J. Prev. Med. 2012;43(5):467–474. doi: 10.1016/j.amepre.2012.07.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cheung Y.T. Using WhatsApp and Facebook online social groups for smoking relapse prevention for recent quitters: a pilot pragmatic cluster randomized controlled trial. J. Med. Internet Res. 2015;17(10) doi: 10.2196/jmir.4829. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cole-Lewis H. Social listening: a content analysis of e-cigarette discussions on Twitter. J. Med. Internet Res. 2015;17(10) doi: 10.2196/jmir.4969. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dehlin J.M. # PrEP4Love: An evaluation of a sex-positive HIV prevention campaign. JMIR Publ. Health Surveillance. 2019;5(2) doi: 10.2196/12822. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dowshen N. IknowUshould2: Feasibility of a youth-driven social media campaign to promote STI and HIV testing among adolescents in Philadelphia. AIDS and behavior. 2015;19(Suppl. 2(02)):106–111. doi: 10.1007/s10461-014-0991-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dulli L. An online support group intervention for adolescents living with HIV in Nigeria: a pre-post test study. JMIR Publ. Health Surveillance. 2018;4(4) doi: 10.2196/12397. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Eaton L.A. The role of stigma and medical mistrust in the routine health care engagement of black men who have sex with men. Am. J. Publ. Health. 2015;105(2):e75–e82. doi: 10.2105/AJPH.2014.302322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fisher C.B. Facilitators and barriers to participation in PrEP HIV prevention trials involving transgender male and female adolescents and emerging adults. AIDS Educ. Prev. 2017;29(3):205–217. doi: 10.1521/aeap.2017.29.3.205. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fox S. Pew Internet & American Life Project: Health Fact Sheet. Pew Research Center; Washington, D.C: 2012. [Google Scholar]
- Fox S. The Social Life of Health Information, Pew Internet & American Life Project. Pew Research Center; Washington, DC: 2011. [Google Scholar]
- Gaysynsky A., Romansky-Poulin K., Arpadi S. “My YAP Family”: analysis of a Facebook group for young adults living with HIV. AIDS Behav. 2015;19(6):947–962. doi: 10.1007/s10461-014-0887-8. [DOI] [PubMed] [Google Scholar]
- Ghosh D. Social network strategies to address HIV prevention and treatment continuum of care among at-risk and HIV-infected substance users: a systematic scoping review. AIDS Behav. 2017;21(4):1183–1207. doi: 10.1007/s10461-016-1413-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Haines-Saah R.J. Picture Me Smokefree: a qualitative study using social media and digital photography to engage young adults in tobacco reduction and cessation. J. Med. Internet Res. 2015;17(1) doi: 10.2196/jmir.4061. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hales S.B., Davidson C., Turner-McGrievy G.M. Varying social media post types differentially impacts engagement in a behavioral weight loss intervention. Transl. Behav. Med. 2014;4(4):355–362. doi: 10.1007/s13142-014-0274-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Holloway I.W. Online social networking, sexual risk and protective behaviors: considerations for clinicians and researchers. Curr. Addict. Rep. 2014;1(3):220–228. doi: 10.1007/s40429-014-0029-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Horvath K.J. Feasibility, acceptability and preliminary efficacy of an online peer-to-peer social support ART adherence intervention. AIDS Behav. 2013;17(6):2031–2044. doi: 10.1007/s10461-013-0469-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jones R., Lacroix L.J., Nolte K. “Is Your Man Stepping Out?” An online pilot study to evaluate acceptability of a guide-enhanced HIV prevention soap opera video series and feasibility of recruitment by Facebook advertising. J. Assoc. Nurses AIDS Care: JANAC. 2015;26(4):368–386. doi: 10.1016/j.jana.2015.01.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kamel Boulos M.N., Wheeler S. The emerging Web 2.0 social software: an enabling suite of sociable technologies in health and health care education 1. Health Inf. Libr. J. 2007;24(1):2–23. doi: 10.1111/j.1471-1842.2007.00701.x. [DOI] [PubMed] [Google Scholar]
- Kamel Boulos M.N., Wheeler S. The emerging Web 2.0 social software: an enabling suite of sociable technologies in health and health care education. Health Info. Libr. J. 2007;24(1):2–23. doi: 10.1111/j.1471-1842.2007.00701.x. [DOI] [PubMed] [Google Scholar]
- Kelly J.A. HIV risk behavior reduction following intervention with key opinion leaders of population: an experimental analysis. Am. J. Public Health. 1991;81(2):168–171. doi: 10.2105/ajph.81.2.168. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kim S.J. Harnessing Facebook for smoking reduction and cessation interventions: Facebook user engagement and social support predict smoking reduction. J. Med. Internet Res. 2017;19(5) doi: 10.2196/jmir.6681. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ko N.-Y. Effects of Internet popular opinion leaders (iPOL) among Internet-using men who have sex with men. J. Med. Internet Res. 2013;15(2) doi: 10.2196/jmir.2264. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Korda H., Itani Z. Harnessing social media for health promotion and behavior change. Health Promotion Pract. 2013;14(1):15–23. doi: 10.1177/1524839911405850. [DOI] [PubMed] [Google Scholar]
- Lambert D.N. A formative evaluation of social media campaign to reduce adolescent dating violence. JMIR Res. Protoc. 2014;3(4) doi: 10.2196/resprot.3546. e64-e64. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Latkin C.A. Norms, social networks, and HIV-related risk behaviors among urban disadvantaged drug users. Soc. Sci. Med. 2003;56(3):465–476. doi: 10.1016/s0277-9536(02)00047-3. [DOI] [PubMed] [Google Scholar]
- Latkin C.A. Relationships between social norms, social network characteristics, and HIV risk behaviors in Thailand and the United States. Health Psychol. 2009;28(3):323. doi: 10.1037/a0014707. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Leahey T.M. Social networks and social norms are associated with obesity treatment outcomes. Obesity. 2015;23(8):1550–1554. doi: 10.1002/oby.21074. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lelutiu-Weinberger C. Feasibility, acceptability, and preliminary efficacy of a live-chat social media intervention to reduce HIV Risk among young men who have sex with men. AIDS Behav. 2015;19(7):1214–1227. doi: 10.1007/s10461-014-0911-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lim M.S.C. Reach, engagement, and effectiveness: a systematic review of evaluation methodologies used in health promotion via social networking sites. Health Promotion J. Aust. 2017;27(3):187–197. doi: 10.1071/HE16057. [DOI] [PubMed] [Google Scholar]
- Mahajan A.P. Stigma in the HIV/AIDS epidemic: a review of the literature and recommendations for the way forward. AIDS (London, England) 2008;22(Suppl. 2):S67–S79. doi: 10.1097/01.aids.0000327438.13291.62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Meng J., Chung M., Cox J. Linking network structure to support messages: effects of brokerage and closure on received social support. J. Commun. 2016;66(6):982–1006. [Google Scholar]
- Merchant G. Click “like” to change your behavior: a mixed methods study of college students’ exposure to and engagement with Facebook content designed for weight loss. J. Med. Internet Res. 2014;16(6) doi: 10.2196/jmir.3267. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Patel V.V. Empowering with PrEP (E-PrEP), a peer-led social media–based intervention to facilitate HIV preexposure prophylaxis adoption among young Black and Latinx gay and bisexual men: protocol for a cluster randomized controlled trial. JMIR Res. Protoc. 2018;7(8) doi: 10.2196/11375. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pedrana A. Queer as F** k: reaching and engaging gay men in sexual health promotion through social networking sites. J. Med. Internet Res. 2013;15(2) doi: 10.2196/jmir.2334. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Perski O. Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Transl. Behav. Med. 2017;7(2):254–267. doi: 10.1007/s13142-016-0453-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pew Research Center, Social Media Fact Sheet. 2019. p. https://www.pewinternet.org/fact-sheet/social-media/.
- Ragsdale K. Social media to enhance sexual health education for youth: FactNotFiction’s (Re) design and launch. Case Stud. Strateg. Commun. 2015;4:88–112. [Google Scholar]
- Rhodes S.D. Using social media to increase HIV testing among gay and bisexual men, other men who have sex with men, and transgender persons: outcomes from a randomized community trial. Clin. Infect. Dis. 2016;62(11):1450–1453. doi: 10.1093/cid/ciw127. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rietmeijer C.A., McFarlane M. Web 2.0 and beyond: risks for sexually transmitted infections and opportunities for prevention. Curr. Opin. Infect Dis, 2009;22(1):67–71. doi: 10.1097/QCO.0b013e328320a871. [DOI] [PubMed] [Google Scholar]
- Rossman K., Salamanca P., Macapagal K. A qualitative study examining young adults’ experiences of disclosure and nondisclosure of LGBTQ identity to health care providers. J. Homosex. 2017;64(10):1390–1410. doi: 10.1080/00918369.2017.1321379. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rus H.M., Cameron L.D. Health communication in social media: message features predicting user engagement on diabetes-related Facebook pages. Ann. Behav. Med. 2016;50(5):678–689. doi: 10.1007/s12160-016-9793-9. [DOI] [PubMed] [Google Scholar]
- Short C.E. Measuring engagement in eHealth and mHealth behavior change interventions: viewpoint of methodologies. J. Med. Internet Res. 2018;20(11) doi: 10.2196/jmir.9397. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Solorio R. Tu Amigo Pepe: evaluation of a multi-media marketing campaign that targets young Latino immigrant MSM with HIV testing messages. AIDS Behav. 2016;20(9):1973–1988. doi: 10.1007/s10461-015-1277-6. [DOI] [PubMed] [Google Scholar]
- Stieglitz S. Social media analytics. Bus. Inf. Syst. Eng. 2014;6(2):89–96. [Google Scholar]
- Strand C. E-Health communities and online self-help groups: Applications and usage. IGI Global; 2012. Blogging: a new tool for coping and accessing psycho-social support for people living with HIV? pp. 106–120. [Google Scholar]
- Swanton R., Allom V., Mullan B. A meta-analysis of the effect of new-media interventions on sexual-health behaviours. Sex. Transm. Infect. 2015;91(1):14–20. doi: 10.1136/sextrans-2014-051743. [DOI] [PubMed] [Google Scholar]
- Syred J. Would you tell everyone this? Facebook conversations as health promotion interventions. J. Med. Internet Res. 2014;16(4) doi: 10.2196/jmir.3231. e108-e108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Taggart T. Social media and HIV: a systematic review of uses of social media in HIV communication. J. Med. Internet Res. 2015:17(11). doi: 10.2196/jmir.4387. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Thrul J., Klein A.B., Ramo D.E. Smoking cessation intervention on Facebook: which content generates the best engagement? J. Med. Internet Res. 2015;17(11) doi: 10.2196/jmir.4575. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Valente T.W. Network interventions. Science. 2012;337(6090):49–53. doi: 10.1126/science.1217330. [DOI] [PubMed] [Google Scholar]
- Valle C.G., Tate D.F. Engagement of young adult cancer survivors within a Facebook-based physical activity intervention. Transl. Behav. Med. 2017;7(4):667–679. doi: 10.1007/s13142-017-0483-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wang, H., et al., Social media fandom for health promotion? Insights from East Los High, a transmedia edutainment initiative, 2019.
- Webb T. Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J. Med. Internet Res. 2010;12(1) doi: 10.2196/jmir.1376. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wong C.A., Merchant R.M., Moreno M.A. Elsevier; 2011. Using social media to engage adolescents and young adults with their health. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Young S.D. Effect of a community popular opinion leader HIV/STI intervention on stigma in urban, coastal Peru. AIDS Behav. 2011;15(5):930–937. doi: 10.1007/s10461-010-9826-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Young S.D. Social networking technologies as an emerging tool for HIV prevention: a cluster randomized trial. Ann. Intern. Med. 2013;159(5):318–324. doi: 10.7326/0003-4819-159-5-201309030-00005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Young S.D. Project HOPE: online social network changes in an HIV prevention randomized controlled trial for African American and Latino men who have sex with men. Am. J. Public Health. 2014;104(9):1707–1712. doi: 10.2105/AJPH.2014.301992. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Young L.E. PrEP Chicago: a randomized controlled peer change agent intervention to promote the adoption of pre-exposure prophylaxis for HIV prevention among young Black men who have sex with men. Clin. Trials. 2017;15(1):44–52. doi: 10.1177/1740774517730012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Young S.D., Jaganath D. Online social networking for HIV education and prevention: a mixed methods analysis. Sex. Transm. Dis. 2013;40(2):p. doi: 10.1097/OLQ.0b013e318278bd12. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.