HPV Vaccine Searches on Pinterest: Before and After Pinterest’s Actions to Moderate Content

Abstract

Objectives. To compare how human papillomavirus (HPV) vaccination was portrayed on Pinterest before and after the platform acted to moderate vaccine-related search results to understand (1) what the information environment looked like previously and (2) whether Pinterest’s policy decisions improved this environment in terms of sources and content.

Methods. In this quantitative content analysis, we compared 2 samples of 500 HPV vaccine–focused Pinterest posts (“pins”) collected before and after Pinterest’s actions to provide more reliable vaccine-related information. Pins were based on search results and were analyzed using the Health Belief Model.

Results. The majority of preaction search results leaned toward vaccine skepticism, specifically focused on perceived vaccine barriers. Few pins were published by public health–related Pinterest accounts. Postaction search results showed a significant shift to HPV vaccination benefits, and the number of pins by government or medical accounts increased. However, the proportion of pins in search results containing HPV content of any type was significantly lower.

Conclusions. Pinterest’s efforts to moderate vaccination discussions were largely successful. However, the ban also appeared to limit HPV vaccination search results overall, which may contribute to confusion or an information vacuum.

Human papillomaviruses (HPVs) represent the most common sexually transmitted diseases. They are the primary cause of warts and cancers of the genital areas, as well as several other cancers.^1,2 Since 2006, a vaccine against most HPV infections that can lead to cancer has been introduced in more than 80 countries.³ Recent estimates indicate that 70% to 90% of HPV-related cancers may be prevented through universal HPV vaccination.⁴ Despite the potential benefits of HPV vaccination, there are several challenges that hinder vaccine uptake.³ Many individuals have negative attitudes toward the vaccine⁵ or are concerned about vaccine safety.⁶ Others believe it is for females only or that being in a monogamous relationship prevents individuals from getting HPV.^7,8

While individuals likely learn about the HPV vaccine from a variety of sources, searching for health information online is increasingly common. Although the specific influence of social media on medical decision-making remains understudied, research supports an association between online health information searches and medical decision changes.⁹ Visiting antivaccination Web sites for 5 to 10 minutes can increase perceptions of vaccination risks and decrease intentions to vaccinate.¹⁰ In addition, HPV vaccine uptake may be lower in states where vaccine misinformation and conspiracies made up higher proportions of Twitter exposure,¹¹ suggesting that negative representations of vaccines may influence vaccine acceptance. Recent studies have demonstrated that Pinterest is home to a large volume of health information,^12–14 including significant amounts of antivaccine content.^12,15 These studies prompted Pinterest to take action in 2019 to improve the available information about vaccines on its platform.¹⁶ In this study, we examined the content of pins resulting from searches for HPV vaccine information on Pinterest and how their efforts to mitigate vaccine-related misinformation have affected the information environment.

HEALTH MISINFORMATION ON SOCIAL MEDIA

Despite widespread scientific agreement about the value of vaccines,¹⁷ misinformation about vaccines is common. Campaigns that question the safety and efficacy of vaccination often outweigh official narratives on Twitter.^18,19 Likewise, analyses of videos about the HPV vaccine on YouTube²⁰ and portrayals of vaccination on Pinterest¹² both found that antivaccination posts elicited more engagement, echoing research suggesting that mothers who oppose childhood vaccination communicate more about the issue on social media.²¹ These antivaccination posts often pair logical and seemingly factual language with fewer anxiety-related words than provaccination posts, heightening their appeal.²² While concerns about health misinformation on social media are widespread,^22,23 vaccine misinformation deserves special attention, given its potential for compromising herd immunity and putting vulnerable populations not eligible to receive vaccinations at risk.

Human Papillomavirus Vaccine on Pinterest

In 2018, 29% of adults in the United States reported using Pinterest, with usage particularly high among women (41%) and adults aged 18 to 49 years (34%).²⁴ In addition, 80% of mothers and 38% of fathers in the United States use the platform.²⁵ As mothers continue to be the primary care coordinators for children²⁶ and make most health care decisions for their families,²⁷ Pinterest’s demographics make it particularly well suited for examining HPV vaccination messages.

In response to calls for social media platforms to take proactive approaches to manage misinformation, Pinterest instituted a series of policy changes. In early 2019, Pinterest first disabled searches for pins that contained certain vaccine-related information, regardless of the results’ reputability.²⁸ In late summer of 2019, Pinterest announced that it would only show information about vaccines from public health organizations.¹⁶ As a consequence, the search results for HPV vaccine content have deviated from the broader body of posts on this topic, making it critical to adopt a “search as research” approach to understand the information that the public receives when they seek HPV vaccine–related information in Pinterest.²⁹

To date, to our knowledge, no other studies have examined the content that Pinterest displays when people search for the HPV vaccine or evaluated Pinterest’s effort to reduce vaccine misinformation in that content. Therefore, this study offers 2 advancements to our understanding of health information on Pinterest. By comparing how HPV vaccination was portrayed in search results before the platform took action to moderate vaccine-related content (the “preaction” time period) versus after these actions (the “postaction” time period) this study shows (1) what the information environment looked like previously and (2) whether Pinterest’s policy decisions improved the search environment in terms of the sources and the content of pins.

Health Belief Model

The Health Belief Model (HBM) provides a useful theoretical framework for analyzing media messages about vaccination.^30,31 The HBM posits that motivation for adopting healthy behaviors is determined by perceptions of susceptibility, severity, benefits, barriers, self-efficacy, and cues to action. In the context of HPV vaccination, perceived susceptibility refers to beliefs about the likelihood of contracting HPV vaccine–preventable diseases. Perceived severity is the assessment of the seriousness of contracting HPV vaccine–preventable diseases. Perceived benefits are assessments of the effectiveness of the HPV vaccine and associated positive consequences of avoiding HPV-related diseases. Perceived barriers about the HPV vaccine include safety concerns and any perceptions that inhibit someone from getting the vaccine (e.g., fear of needles, cost). Perceived self-efficacy is individuals’ assessments of their ability to get the HPV vaccine. Lastly, cues to action are defined as external triggers prompting people to get the vaccine for themselves or their children.³¹ Broadly, we consider “pro–HPV vaccine” pins those that highlight the susceptibility and severity of contracting HPV-related diseases, promote the benefits of the HPV vaccine and self-efficacy, and offer cues to action to get the vaccine. Conversely, those highlighting barriers—including visuals likely to elicit fear (e.g., include a large needle, fearful expression, perceived vaccine adverse effects)—are considered to be “anti–HPV vaccine” content.

A large body of research demonstrates the utility of the HBM in predicting vaccination and other preventive behaviors such as health screenings,^32,33 making it an appropriate framework for examining whether Pinterest’s policy changes had the effect of improving vaccine-related content in alignment with the best available medical evidence. Given that Pinterest took actions designed specifically to boost the prominence of high-quality official sources of vaccine information, we hypothesized the following:

1. A greater proportion of official sources regarding HPV vaccine–related posts will appear in the postaction period as compared with the preaction period.

2. The postaction period will have (1) a greater proportion of HBM constructs that support HPV vaccination and (2) a smaller proportion of HBM constructs that oppose HPV vaccination (compared with the preaction period).

Notably, Pinterest removed engagement indicators in the postaction period. However, we hypothesized the following:

3. In the preaction period, posts with (1) HBM constructs that oppose HPV vaccination will receive more engagement, and (2) HBM constructs that support HPV vaccination will receive less engagement than posts without these constructs.

METHODS

We conducted a quantitative content analysis of 2 samples of 500 HPV vaccine–related Pinterest posts. The first sample was collected October 2, 2018, before Pinterest’s actions to moderate vaccine-related search results; the second was collected December 2, 2019, following Pinterest’s actions to moderate vaccine-related search results. For both searches, we cleared the limited browsing history of the computer used (a brand-new computer that had not been used except to install basic programs for future use), used procedures to access Pinterest without being logged in to the platform, and used the same computer and procedure for both searches to ensure that algorithms for an existing account or user would not influence the search.

We collected both samples via a form of manual systematic random sampling, in which every third pin from the search results was selected, and this process was carried out for each time period until 250 posts were reached for each of 2 search terms, “HPV vaccine” and “HPV vaccination,” for a total of 500 posts per time period. Consistent with the “search as research” approach, the sample is representative of the search results yielded by Pinterest, although not of the overall body of posts that people have created on the platform.

We manually coded all posts (n = 1000) for engagement (i.e., the total number of saves and comments each were counted and included), account characteristics (e.g., individual, commercial, health-related, and government), and whether the post linked to an external Web site (yes or no; if yes, the type of Web site). We also recorded visual characteristics (e.g., primarily image, primarily text, mix of image and text, infographic, drawing, or video), and the presence of fear visuals (e.g., large needle, mask, or threat sign, such as a skull and crossbones). Furthermore, we coded pins for the presence of HBM constructs (e.g., perceived benefits of and barriers to the HPV vaccine, perceived severity of and susceptibility to HPV, as well as perceived self-efficacy and cues to action). Examples for the HBM constructs present in search results are included in Appendix Figures A through F (available as supplements to the online version of this article at http://www.ajph.org).

Intercoder Reliability

After a period of training, 3 coders (J. P. D. G., H. M. M., and Y. Q.) independently coded 20% of posts (n = 200). After pretesting and subsequent coding protocol changes, intercoder reliability using Krippendorff’s α ranged from 0.70 to 1.00, with an average of 0.77, meeting the standard for reliability. One coder coded 600 additional posts, and each of the remaining coders coded 150 more.

Statistical Analyses

We carried out descriptive analyses for all variables. We assessed differences between time periods with the χ² test. We used the Mann–Whitney U test to check for differences in Pinterest engagement between posts with (vs without) a range of dichotomous variables. We conducted all analyses with SPSS version 26 (IBM, Somers, NY).

RESULTS

The purpose of this study was to analyze HPV vaccine–related search results on Pinterest, comparing results before and after Pinterest’s 2019 actions on vaccine-related searches. All of the preaction pins mentioned HPV or the HPV vaccine in some way, but only 40.6% of the postaction sample did, despite our explicitly using the search terms “HPV vaccine” or “HPV vaccination.” Some pins in the postaction sample (n = 500) focused on a non-HPV vaccine (e.g., influenza; measles, mumps, and rubella [MMR]; 18.6%), while others mentioned vaccines in general (29.2%) or made no mention of vaccines (11.6%). Within the postaction sample, 74.0% of pins from the “HPV vaccination” search (n = 250) mentioned the HPV vaccine or the virus itself, compared with only 7.2% of the “HPV vaccine” pins (n = 250).

In the preaction period, most visuals (64.4%) primarily consisted of an image. The remaining pins were 9.4% primarily text-based, 16.4% a mix of image and text, and 3.6% an infographic. In the postaction period, the proportion of visuals consisting primarily of an image was lower (36.0%), while the proportions of primarily text-based (15.6%), a mix of image and text (29.2%), and infographic (10.0%) pins were higher. The χ² test showed that all of these differences were significant (Table 1).

TABLE 1—

Characteristics of Human Papillomavirus Vaccine Posts on Pinterest by Time Period

Variable/Subvariable	Before Actions to Moderate Contenta (n = 500), % (no.)	After Actions to Moderate Contentb (n = 203), % (no.)	Pc
Pinner source
Individual	79.2 (396)	25.8 (129)	< .001
Individual small business	4.0 (20)	2.6 (13)	.22
Commercial	2.8 (14)	4.0 (20)	.30
Government	0 (0)	7.6 (38)	< .001
Official medical	3.2 (16)	46.2 (231)	< .001
Other health-focused	5.6 (28)	5.2 (26)	.78
Medical professional	1.6 (8)	0.6 (3)	.13
Antivax organization	1.2 (6)	0 (0)	.014
Other	1.8 (9)	6.0 (30)	< .001
Visual type
Primarily image	64.4 (322)	36.0 (180)	< .001
Primarily text	9.4 (47)	15.6 (78)	.003
Mix of image and text	16.4 (82)	29.2 (146)	< .001
Infographic	3.6 (18)	10.0 (50)	< .001
Drawing	0 (0)	1.6 (8)	.005
Video	6.0 (30)	5.4 (27)	.68
Other	0.2 (1)	2.2 (11)	.004
Fear visual	32.2 (161)	14.2 (71)	< .001
Health Belief Model
Perceived benefits vaccine	13.0 (65)	40.4 (82)	< .001
Perceived barriers vaccine	65.6 (328)	11.3 (23)	< .001
Perceived severity HPV	3.6 (18)	23.2 (47)	< .001
Perceived susceptibility HPV	5.0 (25)	21.7 (45)	< .001
Cue to action—get vaccine	8.2 (41)	14.8 (30)	< .001
Self-efficacy—get vaccine	2.6 (13)	5.9 (12)	< .001
Barriers
Conspiracy	13.8 (69)	1.2 (6)	< .001
Vaccine adverse reactions	55.0 (275)	1.6 (8)	< .001

Note. HPV = human papillomavirus. Pinner source and visual type categories were mutually exclusive, Health Belief Model constructs were not.

^aCollected October 2, 2018.

^bCollected December 2, 2019. A portion of the total sample of n = 500 was not relevant; only the relevant search results (n = 203) are included here.

^cP values determined by χ² test, unless cell size was < 5, in which case the Fisher exact test was used.

Regarding information sources, most pins (79.2%) in the preaction period originated with individuals, while only 3.2% were published by official medical entities (e.g., hospitals), and no pins originated with government entities (e.g., public health departments, the Centers for Disease Control and Prevention; Table 1). By contrast, in the postaction period, only a quarter of pins (25.8%) originated with individuals, with 46.2% published by official medical entities and 7.6% by government entities. The χ² test revealed that all of these differences were significant (Table 1), supporting our first hypothesis.

Next, we examined the HBM constructs present in HPV-vaccine related search results. In the preaction sample, the most frequently mentioned HBM construct was perceived barriers (65.6%), with perceived benefits a distant second (13.0%; Table 1). Barriers included adverse effects of the HPV vaccine (55.0%), fear visuals (32.2%), and conspiracy theories (13.8%; Table 1).

Among pins in the postaction sample that focused on HPV or the HPV vaccine, the HBM constructs present were categorically different (supporting our second hypothesis). Perceived benefits were mentioned most frequently (40.2%), followed by perceived severity of HPV infection and its consequences and perceived susceptibility to HPV infection (22.1%; Table 1). By contrast, perceived barriers to vaccination were present in only 11.3% of posts, vaccine adverse reactions in 1.6%, conspiracy theories in 1.2%, and fear appeals in 14.2% of posts. A large needle remained the most common fear visual. The χ² test again showed that all of these differences were significant (Table 1) and broadly provided support for our second hypothesis, as provaccine content increased and antivaccine content decreased in postaction pins.

Finally, we assessed the relationship between message features and engagement in the preaction period. As shown in Table 2, pins mentioning barriers to getting the HPV vaccine were significantly more likely to produce Pinterest engagement than pins that did not mention barriers. Mentioning benefits of the HPV vaccine, self-efficacy, and cues to action were each associated with lower median save frequencies (Table 2), supporting our third hypothesis.

TABLE 2—

Dichotomous Health Belief Model Independent Variables and Median Engagement on Pinterest in Preaction Period

Engagement Variable	Variable	Median No. Engagement Present	Median No. Engagement Absent	P
Increased engagement
Saves	Barriers to getting HPV vaccine	9.50	2.00	< .001
Decreased engagement
Saves	HPV vaccine benefits	2.00	8.00	< .001
Saves	Self-efficacy to get HPV vaccine	2.00	7.00	.009
Saves	Cues to action to get HPV vaccine	2.00	7.00	< .001
No change in engagement
Comments	Benefits HPV vaccine	0	0	.032a
Comments	Barriers to getting HPV vaccine	0	0	.032a
Saves	Susceptibility to HPV	5.00	7.00	.13
Comments	Susceptibility to HPV	0	0	.98
Saves	Severity of HPV	5.50	6.00	.98
Comments	Severity of HPV	0	0	.20
Comments	Self-efficacy to get HPV vaccine	0	0	.27
Comments	Cues to action to get HPV vaccine	0	0	.043a

Note. HPV = human papillomavirus. Data were collected October 2, 2018.

^aStatistically significant at P < .05, but practically not significant (no difference in median frequency).

DISCUSSION

While Pinterest still tends to be seen as a place to find and share recipes and decorating ideas, several studies demonstrate not only that vaccine posts are present but also that these posts tend to be dominated by vaccine skeptics.^10,12 The results of the preaction time period of this study confirm these trends: 65.4% of pins mention specific barriers to getting the HPV vaccine, such as perceived adverse effects of the vaccine and conspiracy theories. Not only do HPV vaccine–skeptic posts dominate this part of the sample, but when they appeared, they also were often associated with higher levels of engagement in the form of saves (Pinterest’s main engagement metric). Apart from perceived barriers, HBM constructs were almost entirely absent from the preaction sample. When we considered the high percentage of listed barriers to getting the HPV vaccine, together with very little mention of the severity of and susceptibility to HPV, it was clear that vaccine skepticism was prevalent in these search results, and more strategic efforts to promote the HPV vaccine in this space were needed.

Another finding of interest for the preaction sample was that the majority of the pinners in the search results were individuals, while only 3% of all pins were confirmed to originate with official medical accounts (e.g., hospitals, medically focused nonprofits), and there was no presence of any pins published by government accounts such as the Centers for Disease Control and Prevention. This is concerning, as it appeared to suggest that the public health experts’ voice—often likely to provide trustworthy and reliable health information—was again largely absent on this platform, which was a concern shared by earlier health-focused Pinterest studies.^12–15

Pinterest engaged in several strategic efforts over the course of 2019—first disabling searches for certain vaccine-focused terms, and then repopulating these searches with posts originating with reliable public health entities. The postaction sample in this study yielded several encouraging results: more of the posts originated with either public health– or health-related entities, and of the HBM constructs, perceived benefits of the HPV vaccine, as well as severity of and susceptibility to HPV infection, were mentioned in 23.5% and 22.1%, respectively, of the postaction sample (as compared with 3.6% and 5.0% of the preaction sample).

However, a few serious concerns also surfaced. First, in the postaction sample, 60% of all pins did not address the HPV vaccine or the virus in any way, instead focusing on other, often younger childhood vaccines about which information may not be relevant for people searching for HPV vaccine information. We performed additional searches on December 21, 2019, to probe these results. At the time this article was written, keyword searches for “HPV vaccine,” “flu vaccine,” “MMR vaccine,” and “vaccine” yielded identical results (i.e., the same mix of messages focused on diverse vaccine-related issues such as the MMR vaccine, influenza vaccine, HPV vaccine, herd immunity, history of vaccines, and general advice on how to make vaccines an easier experience for young children (Appendix Figures G–J, available as supplements to the online version of this article at http://www.ajph.org). Although we are not privy to Pinterest’s search algorithm, we suspect that the results for the search for keyword “vaccine” were replicated and provided for the search for any type of “vaccine”-related search, whether that be “flu vaccine,” “HPV vaccine,” or “MMR vaccine.”

In addition, 12.2% of the postaction sample did not discuss vaccines of any kind in their posts. Interestingly, the distribution of HPV- versus non–HPV-related pins appeared to be clearly skewed in 1 direction: pins from the “HPV vaccine” keyword search only yielded 7.2% of HPV vaccine- and virus-related posts, while pins from the “HPV vaccination” keyword search yielded 74% of HPV vaccine- and virus-related posts. As postaction results were identical for “HPV vaccine” and non–HPV vaccine searches, the low rate of HPV content from the “HPV vaccine” search may be an artifact of how Pinterest has implemented its strategy for reducing misinformation around vaccines more broadly. Of further interest is that a search for keyword “HPV vaccine” produced a warning from Pinterest, but “HPV vaccination” did not (Figure 1). Finally, even when a pin mentioned the HPV vaccine or virus, visuals were often of questionable quality and utility, frequently because the quality of the image was lacking (Appendix Figure K, available as a supplement to the online version of this article at http://www.ajph.org) or because the pin was not designed for Web use (Appendix Figure L) or it was not conveying any kind of clear message or information (Appendix Figure M).

FIGURE 1—

Example of Pinterest Warning Message After “HPV Vaccine” Keyword Search

Limitations and Future Directions

There is no current application programming interface available for Pinterest data collection by keyword, so the sampling method, while approximating randomization as closely as possible, could be improved.^12,14,34 Second, while network analysis on Pinterest is in its infancy, future studies should consider using both current (posts that are saved from another user, similar to Twitter’s “retweeted” posts) and original (posts created by the pinner as original Pinterest content) pinner data to better understand which messages spread fastest and how they spread. Third, a qualitative analysis should be considered to extend this research, as should message-testing studies that focus on correcting misinformation and spreading accurate information on Pinterest. Finally, we did not seek to capture the overall body of Pinterest posts about HPV vaccination, but rather focused on Pinterest as a source of information via its search mechanism. Future work should consider what individual people post to Pinterest and if their behaviors have changed in light of Pinterest revising its algorithms to reduce antivaccine content in search results.

Public Health Implications

Overall, Pinterest’s efforts to mitigate HPV vaccination misinformation in its search results were partially successful. The information landscape before Pinterest’s systematic effort was largely anti–HPV vaccination, with individuals sharing information that highlighted barriers, promoted conspiracy theories, and focused on rare or inaccurate negative side effects of the vaccine rather than the severity of and susceptibility to HPV. Although these trends were not eradicated with Pinterest’s efforts, the postaction information available skewed more positively, focusing on the benefits of vaccination and the dangers of HPV infections. The postaction pins originated more often with reputable accounts, without removing individuals’ ability to communicate by creating their own pins.

However, the way in which the content restrictions were implemented also appeared to limit HPV vaccination content. Many of the pins that emerged when we searched for “HPV vaccine” were not related to vaccination at all or focused on other, younger childhood vaccinations. This could lead to another type of misinformation if audiences fail to realize when a pin is referring to a different vaccine type, or an information vacuum. Although Pinterest took an important first step with its 2019 actions, it can and should improve the curational aspect of vaccine-related searches on its platform.

Health communication and public health professionals as well as advocacy organizations should also do more to fill the information gap related to HPV vaccination on social media. More health professionals should consider expanding their social media activities to include Pinterest, particularly considering Pinterest’s largely female user base and the dominant role of women in making health care decisions for their families.²⁷ Second, while more messages in the postaction sample discussed benefits of vaccination and dangers of HPV infection, content could further encourage self-efficacy for or actions to get the vaccine, as self-efficacy is important in achieving behavior change to overcoming perceived health threats.³⁵ Finally, more strategic partnerships between social media platforms like Pinterest and health communication professionals would help create a space where valuable—and accurate—health information flourishes.

Conclusions

To our knowledge, this research is the first to evaluate Pinterest’s actions to moderate content to combat the spread of vaccine-related misinformation. Although Pinterest’s efforts represent initial progress toward thwarting messages with potentially harmful information, improved approaches are necessary moving forward. Social media platforms should do more than block harmful content and redirect search results. From a public health perspective, it is imperative that Pinterest users seeking vaccine information receive search results that are not only medically accurate but also relevant to the specific vaccine of interest. To achieve more meaningful moderation and promote the dissemination of accurate information on Pinterest, a 2-pronged approach is needed: improved strategies for filtering in relevant and vaccine type–specific pins, as well as generating a greater volume of high-quality, reliable content to fill the void left by blocking misleading pins. Public health content creators should provide more well-designed, informative, and effective Pinterest messages and visuals.