Deep learning analysis of trajectories and spatial variations in HPV vaccine discussions on Chinese Weibo

Abstract

Background

Since 2020 China has piloted free human papillomavirus (HPV) vaccinations to address low coverage. We aim to assess the public perceptions, perceived barriers and facilitators towards HPV vaccination in real time, utilizing deep learning-driven social media listening.

Methods

We collected all HPV vaccination discussions on Weibo, a popular Chinese social media platform, made from 2018 to 2023. We annotated 6600 randomly sampled posts manually against behavior change theories, and iteratively fine-tuned and trained deep learning models to auto-annotate all collected posts. Temporal and geographic analyses were conducted regarding public attitudes towards HPV vaccination and their determinants.

Results

Among 1,972,495 posts identified as relevant to HPV vaccines, deep learning models achieve a predictive accuracy of 0.78 to 0.96. 66.6% and 6.1% of posts contain positive and negative attitudes, respectively. The prevalence of positive attitudes increases from 15.8% to 79.1% (P = 3.02×10⁻¹¹), negative attitudes decline from a peak of 20.3% to 5.5% (P = 1.28×10⁻⁵), and misinformation declines from 36.6% to 10.7% (P = 1.33×10⁻⁶). Central regions of China exhibit a higher prevalence of positive attitudes, whereas Beijing, Shanghai and northeastern regions show higher prevalence of negative attitudes and misinformation. Positive attitudes are significantly lower for 2-valent vaccines (65.7%) than 4-valent (79.6%; P = 0.0005) or 9-valent vaccines (74.1%; P = 0.0005).

Conclusions

Social media listening represents a promising and economically feasible surveillance approach for timely monitoring of public perceptions of vaccination and a potential tool for policymakers to understand public response to policy adaptation.

Plain Language Summary

The HPV (Human Papillomavirus) vaccination can reduce the chances of getting cervical and some other cancers by reducing the likelihood of being infected with HPV. HPV vaccination rates in China remain low despite government efforts to expand access. Understanding public perceptions of HPV vaccines can help guide strategies for improving vaccination coverage. We analyzed nearly two million posts about HPV vaccines on Weibo, a popular Chinese social media platform, from 2018 to 2023. We used a type of AI, called deep learning algorithms, to classify posts by attitudes towards vaccination, health beliefs, exposures to misinformation, and practical barriers to getting vaccinated. We found that positive attitudes increased substantially over time, while negative attitudes and misinformation declined. However, concerns about vaccine accessibility remained common, and some regions and vaccine types showed higher levels of negative attitudes. These findings suggest that social media monitoring using artificial intelligence can help policymakers track public opinions in real time and inform development of targeted strategies to improve vaccination uptake.

Wang, Yang, Ding et al. apply deep learning to analyze nearly two million posts of HPV vaccine discussions on Weibo from 2018 to 2023. Positive attitudes towards HPV vaccination increase over time while negative attitudes and misinformation decline, with geographic and vaccine-type variations highlighting populations for targeted intervention.

Introduction

As a significant public health challenge, human papillomavirus (HPV) infections contribute to approximately 108,000 cervical cancer cases in China annually, and lead to an increased incidence of anogenital and head and neck cancers^1,2. HPV vaccination has been proven to effectively prevent HPV infections, and is included in the national immunization program of 136 countries³. However, it has not been included in China’s national immunization program, and cumulative 2.24% of the vaccine-eligible female population in China are estimated vaccinated against HPV in 2018–2020⁴. Until 2022, 4% and 0.31% of females aged 9 to 14 years have received the first dose and the third dose of HPV vaccines, respectively, while 10.15% and 6.21% among overall 9–45 year-old population⁵.

In alignment with recommendations by the World Health Organization which tentatively seeks to achieve 90% immunization coverage of girls by age 15 before 2030⁶, the Chinese government has been encouraging greater vaccine supply and adapting HPV vaccination policies^7–9. In addition to imported quadrivalent and nine-valent vaccines, domestic bivalent vaccines have also been approved for marketing¹⁰. Through the end of 2022, 4% and 4.66% of 9–14 and 15–19 girls have received first dose of HPV vaccination⁵. A small but increasing number of cities or provinces have started introducing HPV vaccines into local immunization programs since 2022, providing free or subsidized vaccination for school-aged girls^8,11.

The national drive to promote HPV vaccination requires both public support and a favorable policy environment. However, in China, and east Asia generally, stigma against HPV, rooted in conservative sexual norms, impedes open discussions of sexual health^12,13. This cultural backdrop leads to limited awareness and increased hesitancy towards HPV vaccination, often due to misconceptions about its necessity that are influenced by traditional views on sexual activity and health education^14,15. Such vaccine hesitancy is not unique to HPV vaccination; it has also been observed for other important vaccines in China^16–18. Vaccine hesitancy can fluctuate over time, sometimes driven by critical events such as adverse events, vaccine scandal, or other vaccine-related crises, and these public reactions are reflected in traditional media and social media^16–19. Similarly, HPV vaccine hesitancy could also be influenced by key events and recent health policy adaptations, necessitating ongoing and prolonged monitoring of public perceptions, which social media can potentially facilitate. As social media becomes increasingly popular, massive discussions about HPV vaccination sprung up on social media in China²⁰, especially among the young generation, which may drive changes in sexual norms and public perceptions^21,22. Therefore, understanding the dynamics of public perceptions is essential for informing vaccine promotion policies.

More important, social media data constructs a nuanced narrative that reveals public perceptions and actions in real time^23,24. With the incorporation of advanced machine learning techniques, social media listening has been emerging as a supplement to traditional surveillance methods such as surveys, as its extensive data sourcing, real-time surveillance, and lowered cost^25–27. Social media listening can scrutinize how public perceptions evolve and are affected by external events or policies, and therefore can be used as a tool for informing policy-makers to adapt health communication strategies^28–31. Public health agencies can leverage social media listening for real-time monitoring of vaccine confidence and swiftly identifying and responding to rising vaccine hesitancy due to adverse events or policy changes. For example, AI-driven social media listening tracking global mRNA vaccine discussions and tools like the Vaccine Sentimeter, a publicly accessible natural language processing-based platform, have enabled public health professionals to quickly detect shifts in online sentiment and inform vaccination strategies accordingly^32,33. During the COVID-19 pandemic, it has been widely applied to track public sentiments and concerns regarding the pandemic itself and COVID-19 vaccines, and further investigate the impacts of pandemic containment policies on public sentiments^34–38. Despite massive online discussions about HPV vaccination in China, over 60 thousand posts in 2019 and about thousands of posts a week in 2020 on Weibo^21,39, few studies have investigated it on a massive scale through machine learning-driven social media listening.

Our study aimed to investigate HPV vaccine-related discussions posted on Weibo, the Chinese counterpart of X with more than 480 million monthly active users from 2017 in the social media landscape, and its role in HPV vaccination in the Chinese context. We fine-tuned and deployed deep learning (DL) techniques to annotate millions of Weibo posts regarding HPV vaccines from 2018 to 2023, according to behavior change theories^40–43. Based on the DL annotation results, we assessed the temporal and spatial trends in public perceptions, perceived barriers, and facilitators towards HPV vaccination. The study aligns with the growing field of infodemiology, the study of information distribution and its impact on public health, which has been further advanced by incorporating machine learning and natural language processing⁴⁴. Our work contributes to the field by leveraging DL techniques to analyze large-scale social media data with improved model performance. Our analysis reveals that positive attitudes towards HPV vaccination increase substantially from 2018 to 2023, while negative attitudes and misinformation decline. Geographic and vaccine-type variations highlight regions and populations where negative attitudes remain more prevalent. These findings demonstrate the feasibility of deep learning-driven social media listening as a cost-effective surveillance tool and can assist policymakers to adapt HPV vaccination policy and develop tailored education strategies in China.

Methods

An overview of the study design is presented in Fig. 1.

Fig. 1.

Overview of study design.

Data collection

Weibo was leveraged as our data source due to its substantial user base in China. We conducted a data screening process using a predefined set of keywords to capture posts through the Weibo application programming interface (API) via Weibo Open Platform (https://open.weibo.com/). We registered and obtained API access by signing the official contract with Weibo company, which is publicly available to all researchers. Using the Python library sinaweibopy3 (https://github.com/olwolf/sinaweibopy3), we programmatically retrieved posts based on a predefined set of keywords relevant to the HPV vaccine. The keywords we used included “HPV vaccine” (“宫颈癌疫苗”, “HPV疫苗”), “vaccinate against HPV” (“打HPV”), “9-valent” (“9价”), “nine-valent” (“九价”), “4-valent” (“4价”), “four-valent” (“四价”), “2-valent” (“2价”), and “two-valent” (“二价”). Since the Weibo API restricts queries to 30-day increments, we segmented the full data collection time range (January 1, 2018, to June 30, 2023) into monthly intervals for comprehensive data capture. The resulting dataset comprised a total of 4,154,274 posts. We also collected self-reported gender and province-level location data for each posting account. Some posts may have not contained location or gender data, as Weibo users have the option not to disclose their locations or gender profile.

Annotation framework

A conceptual framework is needed to guide our annotation of Weibo posts. Behavior change theories such as the Health Belief Model (HBM) and Theory of Planned Behavior (TPB), provide the fundamental frameworks to understand public attitudes and behaviors in health fields^40,42. According to HBM and TPB, the Increasing Vaccination Model is further developed for vaccination and adopted by the World Health Organization^41,43. Therefore, we developed our annotation framework and defined constructs in the framework on the basis of HBM, TPB, and Increasing Vaccination Model.

In this conceptual framework (Fig. 2), health beliefs and information environment influence attitudes towards vaccination, and practical barriers to vaccination moderate the process between attitudes (i.e., positive, neutral, negative) and behaviors. Health beliefs, as related to the internal factors of HPV vaccination, include three constructs (i.e., moderators in the behavior change model) – perceived disease risks, perceived benefits of vaccines, and perceived barriers to accepting vaccines. The information environment, as related to the external factors of HPV vaccination, includes two constructs—misinformation and social norms. Among these framework constructs, perceived disease risks, perceived benefits of vaccines, and social norms can be seen as facilitators towards HPV vaccination, whereas perceived barriers to accepting vaccines, misinformation, and practical barriers to vaccination can be seen as barriers to HPV vaccination. See Table S1 for definitions of framework constructs and corresponding raw examples of posts.

Fig. 2.

Conceptual framework of HPV vaccination.

Manual annotation

We implemented a stratified random sampling approach where each calendar month during the observation period served as a stratum, with 100 posts randomly selected from each stratum, resulting in a sample of 6600 posts for manual annotation. Exclusion criteria are (1) posts by non-human users such as official accounts, media accounts, news reports, and propaganda identified by annotators; (2) comments; (3) forwards without indicating the opinions or sentiments of the forwarder; (4) not posted in mainland China. Two annotators independently annotated each post according to its relevance to the constructs in the conceptual framework. Firstly, annotators coded each post’s overall attitude toward the HPV vaccine as positive, negative, or neutral. Afterwards, each post was annotated according to its relevance to the remaining six constructs as factors of HPV vaccination. A post could be relevant to one or more constructs or to none. Cohen’s κ interrater agreement between coders was 0.933 for attitude annotation, and ranged from 0.846 to 0.980 for each non-attitude construct (Table S2). This meticulously curated gold-standard corpus of 6600 manually annotated posts served as the foundation for fine-tuning and evaluating DL algorithms.

Fine-tuning deep learning models for classification

We applied the SetFit (Sentence Transformer Fine-tuning) approach to auto-annotate the total sample of Weibo posts, which is a few-shot classification approach based on Sentence Transformer models⁴⁵. Sentence Transformers commonly add a (mean) pooling layer to pre-trained encoder-only models like vanilla BERT (Bidirectional Embedding Representation Transformer) and are trained using datasets regarding sentence relationship to generate sentence-level embeddings (i.e., high-dimensional vectors)⁴⁶. This can be coupled with a classification head downstream to accomplish classification tasks, enhancing training efficiency⁴⁶. The SetFit model comprises two training phases: the first involves fine-tuning Sentence Transformer, and the second involves training the classification head. This approach can achieve classification efficacy through few-shot training without prompts and can yield high few-shot performance.

We adopted a two-stage hierarchical classification strategy to fine-tune multiple DL classification models using 6600 manually annotated posts as the gold-standard corpus. The first stage is a fine-tuned DL classification model to distinguish whether or not a post is related to the HPV vaccine, and the second stage includes seven DL models for labeling HPV vaccine-relevant posts with each of the seven constructs in the annotation framework. For all DL models, we used DMetaSoul/sbert-chinese-general-v2, a Chinese-optimized Sentence Transformer model pre-trained on large-scale Chinese corpora, as the foundation model. For each DL model, we randomly selected 80% of the manually annotated Chinese data as the training set, which was used for both fine-tuning the pre-trained model and training the classification head, and reserved 20% as the test set, and all the DL models achieved an accuracy of over 0.78 (see Table 1 and Table S3).

Table 1.

Framework constructs and predictive performance of SetFit models in Weibo posts

Framework construct	Predictive performance of SetFit models
	Accuracy	Precision	Recall	F-1 score
Irrelevant	0.9572 (0.9426–0.9718)	0.9925 (0.9863–0.9986)	0.9572 (0.9426–0.9718)	0.9747 (0.9665–0.9829)
Attitude	0.7829 (0.7132–0.8527)	0.7600 (0.6774–0.8427)	0.7490 (0.6677–0.8303)	0.7508 (0.6700–0.8316)
Health Beliefs
Perceived disease risk	0.8827 (0.8148–0.9506)	0.6310 (0.4286–0.8333)	0.9300 (0.7778–1.)	0.7325 (0.5714–0.8936)
Perceived benefits	0.8567 (0.8000–0.9133)	0.7830 (0.6591–0.9070)	0.7353 (0.6078–0.8627)	0.7545 (0.6575–0.8515)
Perceived barriers to accepting vaccines	0.8645 (0.8037–0.9252)	0.7630 (0.6286–0.8974)	0.8492 (0.7317–0.9667)	0.7987 (0.700–0.8974)
Environments
Social norms	0.8343 (0.7952–0.8735)	0.7750 (0.6970–0.8529)	0.7425 (0.6636–0.8214)	0.7568 (0.6946–0.8190)
Misinformation	0.8862 (0.8293–0.9431)	0.8152 (0.6875–0.9429)	0.7698 (0.6333–0.9063)	0.7867 (0.6792–0.8941)
Practical barriers to vaccination	0.8592 (0.8338–0.8846)	0.7893 (0.7368–0.8417)	0.7760 (0.7236–0.8283)	0.7819 (0.7414–0.8224)

Mean (95% CI) are shown.

Statistics and reproducibility

We performed statistical analyses on the DL-generated prevalence for all framework constructs using R (Version 4.3.1), including both temporal and geographic analyses. All tests were two-tailed and the level of statistical significance was established at P < 0.05.

Firstly, we performed the temporal analysis by calculating the monthly prevalence of posts relating to each framework construct. We employed locally estimated scatterplot smoothing (LOESS) to reveal the temporal trends of prevalence and remove random noise components⁴⁷. Time-series analyses were used to identify whether and when these constructs exhibited statistically significant increases or decreases in trends.

Secondly, we performed geographic analysis at the provincial level, focusing on spatial distribution of the prevalence of each framework construct. We used self-reported location from user profiles to identify the geographic location of each post.

We also compared the prevalence of framework constructs by different valent types of HPV vaccines and users’ gender. We utilized keyword-based retrieval to classify relevant posts into the three valent types, each retaining only posts exclusively mentioning 2 v, 4 v, or 9 v vaccines. We also distinguished relevant posts by gender, which was self-reported in users’ profiles.

The total sample comprised 4,154,274 posts, of which 1,972,495 were identified as relevant to HPV vaccines. For deep learning model training and evaluation, we used a gold-standard corpus of 6600 manually annotated posts, with an 80/20 training-test split applied to each model. Two independent annotators annotated all posts, achieving a Cohen’s κ of 0.933 for attitude annotation and 0.846 to 0.980 for non-attitude constructs. Model performance was additionally evaluated on previously unseen data from April 2024 (Table S4).

Public involvement

Patients or members of the general public were not involved in the design, or conduct, or reporting, or dissemination plans of this research, because all the raw data are openly accessible online. The purpose of this study is exactly to better detect and understand the public perceptions of HPV vaccination, a health policy and service approach.

Ethical statement

This study was approved by the Institutional Review Board (IRB) of the School of Public Health, Fudan University (IRB#2022-01-0938). The requirement for informed consent was waived by the IRB because the study exclusively analyzed publicly available data from a social media platform and did not involve individual-level surveys or personally identifiable information. According to the Weibo Terms of Service, the Weibo platform operator holds the right to use user-generated Weibo data. All users must agree to these terms prior to using the platform, which includes consenting to the potential use of their publicly posted data for research purposes. For this research, authorization to utilize Weibo data was obtained from the Weibo platform operator. Furthermore, the data collection and usage processes for this study were subject to oversight by Weibo. No personally identifiable information is reported in this study.

Results

Among all 4,154,274 Weibo posts identified from keyword-based screening, 1,972,495 posts from 1,263,324 unique users were identified as relevant to HPV vaccines and based in mainland China, which were then used for our statistical analyses. Among these, 1,217,389 posts (61.7%) from 803,238 users (63.6%) were identified with provincial locations. The fine-tuned SetFit models demonstrated robust predictive performance and consistent accuracy (see Table 1 and Table S3), ranging from 0.7829 (95% CI: 0.7132–0.8527) to 0.9572 (95% CI: 0.9426–0.9718), and achieved mean F-1 scores ranging from 0.7325 (95% CI: 0.5714–0.8936) to 0.9747 (95% CI: 0.9665–0.9829). We additionally evaluated the performance of the models in the previously unseen data, i.e., emerging Weibo posts about HPV vaccination in April 2024 (see detailed performance metrics in Table S4).

Overall status of the framework constructs

During the study period of January 2018 to June 2023, 1,314,510 (66.6%), 120,717 (6.1%), and 537,268 (27.2%) posts were classified as containing positive, negative, and neutral attitudes towards HPV vaccines, respectively. Regarding internal health beliefs, 233,130 posts (11.8%) were classified as regarding perceived disease risks, 245,622 posts (12.5%) regarded perceived benefits of vaccines, and 328,442 posts (16.7%) regarded perceived barriers to accepting vaccines. Regarding exposure to external information environments and practical issues, 432,754 posts (21.9%) were classified as regarding social norms, 224,130 posts (11.4%) regarded misinformation, and 580,590 posts (29.4%) regarded practical barriers to vaccination.

Temporal trends of the framework constructs

The temporal trends with smoothing techniques for the number of relevant posts and the prevalence of each framework construct are illustrated in Fig. 3, with raw data available in the supplementary material (Fig. S1). Over the study period, we observed a noteworthy increase in the number of HPV vaccine-related posts after 2020, peaking in the first half of 2022. For attitudes towards the HPV vaccine, the prevalence of positive attitudes notably increased after March 2018, growing from 15.8% to 79.1% (P = 3.02 × 10⁻¹¹) by the end of the observation period. Concurrently, the prevalence of negative attitudes displayed a declining trend (P = 0.0011), especially after its peak in August 2019 (20.3%) until the end of our observation period in June 2023 (5.5%; P = 1.28 × 10⁻⁵). The prevalence of positive attitudes far outweighed the prevalence of negative attitudes.

Fig. 3. Temporal trends of the number of Weibo posts and prevalence of posts related to each framework construct of HPV vaccination.

A Monthly number of Weibo posts related to HPV vaccination and prevalence of posts with each attitude towards HPV vaccination. B Temporal trends of prevalence of Weibo posts related to each health belief construct. C Temporal trends of prevalence of Weibo posts related to each environment construct. NOTE: Health beliefs indicate constructs of individual’s internal beliefs about HPV and HPV vaccine, including perceived disease risk of HPV, perceived benefits of vaccines and perceived barriers to accepting vaccines; environments indicate constructs of individual’s exposure to environmental barriers or facilitators regarding HPV vaccination, including social norms facilitating HPV vaccination, misinformation about HPV or HPV vaccines, and practical barriers to vaccination. Locally estimated scatterplot smoothing (LOESS) is employed to reveal the temporal trends of prevalence and remove random noise components on monthly raw data.

Among the constructs of internal health beliefs, the prevalence of perceived disease risks declined, from its peak in April 2018 (65.4%) to June 2023 (when our observation ended; 4.5%; P = 1.05 × 10^-11). Similarly, the prevalence of perceived benefits of vaccines declined significantly from July 2018 (30.4%) to June 2023 (9.9%; P = 1.36 × 10^-10). The prevalence of perceived barriers to accepting vaccines stayed high and stable from July 2018 (34.6%) until February 2020 (36.9%; P = 0.84), and then decreased from this period until June 2023 (13.1%; P = 3.01 × 10^-7). While constantly decreasing, the prevalence of perceived benefits was significantly lower than that of perceived barriers (mean difference, -5%; P = 3.51 × 10^-9).

Among the constructs of external environments, the overall prevalence of social norms stabilized (P = 0.87), with a few outliers (the highest point in September 2022, 48.9%). It only decreased slightly from early 2018 to its lowest point in February 2021 (12.7%; P = 0.006), and thereafter remained consistent (P = 0.94). Misinformation decreased following the middle of 2018, peaking in July 2018 (36.6%), and progressing downward until the middle of 2023 (10.7%; P = 1.33 × 10⁻⁶). It is noteworthy that the highest prevalence of misinformation occurred during 2018 and 2019 (June 2018, 30.3%; July 2018, 36.6%; April 2019, 32.4%; July 2019, 30.6%; August 2019, 28.1%). The prevalence of practical barriers to vaccination increased from early 2018 to its peak in August 2020 (46.8%; P = 0.0044), and thereafter decreased (June 2023, 17.2%; P = 9.02 × 10^-7). Over the observation period, the prevalence of practical barriers was sustained at a significantly higher rate than social norms and misinformation. While the prevalence of social norms was generally higher than misinformation, their relationship varied temporally: from January 2018 to March 2020, no significant difference was observed; however, from March 2020 to June 2023, the prevalence of social norms was significantly greater (mean difference 9.5%; P = 1.05 × 10^-8).

Geographical variations of the framework constructs

The inter-province variations for the number of relevant posts and the prevalence of attitudes are depicted in Fig. 4, and the variations for constructs of health beliefs and environment are available in the supplementary materials (Fig. S2). Monthly numbers of relevant posts by province are depicted in the supplementary material (Fig. S3). The number of relevant posts revealed a significant regional disparity in online discussions, from 3651 (0.3%) in Qinghai province to 134,318 (11.0%) in Guangdong province. For attitudes, a predominance of positive attitudes was noted across all provinces, ranging from 60.0% to 74.1%, while negative attitudes were less common, ranging from 4.4% to 6.7%. Provinces in the central region exhibited a higher prevalence of positive attitudes (72.1% to 74.1%) and a lower prevalence of negative attitudes (4.5% to 5.2%), and also showed a relatively higher prevalence of social norms. Conversely, Shanghai, Beijing and three provinces in the northeastern region showed the highest proportion of negative attitudes (5.3% to 6.7%), which also contained more misinformation. Northern regions exhibited a higher prevalence across all three health belief constructs (i.e., perceived disease risk, perceived benefits, perceived barriers to accepting vaccines) than did southern regions. Central and western provinces, particularly Shaanxi (47.8% of 59,868 posts), reported a significantly higher prevalence of practical barriers associated with HPV vaccine uptake.

Fig. 4. Geographic variations in the number of Weibo posts and the prevalence of posts with positive or negative attitude towards HPV vaccination.

A Number of Weibo posts related to HPV vaccination by provinces. B Prevalence of Weibo posts with positive attitude towards HPV vaccination. C Prevalence of Weibo posts with negative attitude towards HPV vaccination. NOTE: Insufficient data indicates regions excluded from geographic analyses: (1) Hong Kong, Macau, and Taiwan because of different HPV vaccination policy from mainland China; (2) Qinghai, Tibet, and Ningxia (3651, 3709, and 3749 posts) because of too small sample size.

Comparison by types of HPV vaccines and gender

Weibo posts related to the 9 v vaccine (1,474,618) far outnumbered those on the 2 v (26,264) and 4 v vaccines (51,538) (Fig. S4). The temporal trends for the prevalence of attitudes towards each type of HPV vaccine with smoothing techniques are depicted in Fig. 5, and temporal trends for constructs of health beliefs and environment are available in the supplementary material (Fig. S5). In total, Weibo users exhibited the most positive attitudes to the 4 v and 9 v vaccines (79.6% and 74.1%), which was higher by about 10% than attitudes towards 2 v vaccines (65.7%). We also conducted chi-square tests on vaccine type, which suggest significant differences among the three vaccine types in all framework constructs (P = 0.0005). Larger differences by vaccine types were observed for barriers to vaccination than for facilitators. Sentiments regarding perceived barriers to accepting vaccines (33.5%) and misinformation (16.9%) were significantly more prevalent towards 2 v vaccines than towards 4 v (12.1% and 9.4%) and 9 v (13.1% and 8.6%) vaccines, but sentiments regarding practical barriers to 2 v vaccination were less prevalent (Fig. S5).

Fig. 5. Temporal trends of prevalence of attitudes towards different HPV vaccine types on Weibo.

A Temporal trends of prevalence of positive attitude by different HPV vaccine types. B Temporal trends of prevalence of negative attitude by different HPV vaccine types. NOTE: Locally estimated scatterplot smoothing (LOESS) is employed to reveal the temporal trends of prevalence and remove random noise components on monthly raw data.

The number of relevant posts from female users (1,533,300) were far greater than those from male users (204,938) (Fig. S6). Positive attitudes towards HPV vaccination were significantly more prevalent among female users than males (71.5% vs 52.2%).

Discussion

DL-driven social media listening is emerging as a novel and promising method to monitor public perceptions on health issues in a timely manner, and therefore a useful tool for policy-makers to understand public response to policies⁴⁸. By leveraging DL techniques, we analyzed social media posts to capture the evolvement of public perceptions, perceived barriers, and facilitators towards HPV vaccination. Our study can help to inform tailored education strategies and develop a more favorable environment for current policy adaptation to address the low rates of HPV vaccination in China.

Analyzing public attitudes via social media listening can serve as a pivotal tool for identifying prevailing public sentiments, conducting surveillance, and addressing prompt responses to anti-vaccine activity. We found that the prevalence of positive attitudes towards HPV vaccination among Weibo users increased substantially from 2019 onwards and thereafter remained consistently high. Interestingly, this finding contrasts with the situation among English social media users, where negative attitudes towards HPV vaccination were either very similar to or even more prevalent than positive attitudes^49–51.

Differing from most high-income countries (e.g., US) that only supply the 9 v vaccine, three types (2 v, 4 v, 9 v) of HPV vaccines are available in China. However, a sustained higher prevalence of negative attitudes, perceived barriers to accepting vaccines, and misinformation was observed regarding the 2 v vaccines in our study. As the domestic 2 v vaccine demonstrated cost-effectiveness¹⁰, health communications should be promoted that aim to reduce rumors and misinformation, especially regarding 2 v vaccines. Based on observed gender disparities in discussions and attitudes, we recommend the promotion of HPV-related health communications targeting both males and females, facilitating a more gender-inclusive environment for vaccine promotion.

On social media, public discussions of HPV disease risk and vaccination benefits displayed a decreasing trend nationwide from early 2018 to mid-2023, which resembles international trends⁵⁰. After the introduction of HPV vaccination in China, public attention shifted from discussions on disease risks and vaccination benefits to practical considerations regarding vaccination decisions, such as vaccine supply, prices, and scheduling processes. Given the pivotal role of positive health beliefs on attitudes towards disease risks and vaccines, health education should continue to emphasize HPV infection risks and the benefits of vaccines before the introduction of HPV vaccination campaigns at the city- or national-level in China. Furthermore, our study found that practical barriers to vaccination, particularly accessibility issues (e.g., insufficient supply and high price of vaccines), were the most prominent public concerns, which aligns with the findings of a previous study⁹. In contrast with what is needed in high-income countries, improving accessibility is the most paramount factor to enhancing HPV vaccination coverage in China, and addressing public attitudes and social norms towards HPV vaccination should be the first step in policy change. It is imperative for the Chinese government to prioritize addressing the barriers that have been identified in social media listening and survey studies.

On Weibo, the prevalence of misinformation and perceived barriers to accepting vaccines decreased throughout the study period, while social norms became more prevalently discussed than did misinformation following March 2020. Social norms not only exert a positive influence on individuals’ intention to receive the HPV vaccine²², but can also raise perceived benefits of vaccination and counter misinformation and perceived barriers^52,53. Although HPV vaccine-related perceived barriers and misinformation declined in China and were lower than what has been observed on platforms like Instagram and X^54,55, the importance of managing misinformation should still not be ignored. Given the isolated nature of information on social media, misinformation tends to gather and spread rapidly, making it challenging to address⁵⁶. Health systems can leverage social media listening to identify and track misinformation in real time, facilitate timely interventions, maintain vigilance regarding the nature of misinformation, and mitigate its impact on vaccine perceptions.

Additionally, more attention should be paid to regions with a higher prevalence of negative attitudes. Through geographical analysis, we observed that Beijing, Shanghai, and the north-eastern provinces exhibited higher levels of negative attitudes and misinformation. Higher internet penetration rates in megacities like Beijing and Shanghai may contribute to the accelerated spread of misinformation⁵⁷. Therefore, it is imperative to establish fact-checking mechanisms and promptly prevent the spread of misinformation. North-eastern regions faced challenges regarding low HPV-related health literacy, which hinders the public’s capacity to obtain health knowledge and combat misinformation.

This study adopted a state-of-the-art DL-based approach to conduct social media listening. This methodology not only enables public health surveillance in real time, but also offers an economically efficient and expeditious means of surveillance, facilitating temporal and geographical analyses on extensive datasets^30,31. One advantage of this method is its direct extraction of public discussions from social media platforms, eliminating potential biases stemming from researcher-participant interactions in surveys. Although there is a possible age bias between social media users and the general population, the primary target groups for HPV vaccination are young adults and college students⁵⁸, which match the age profiles of social media users. Our study also contributes to addressing a methodological issue reported in previous social media listening research, namely that the performance of DL algorithms could be hurt by the training dataset having a highly imbalanced label distribution^44,59. As some framework constructs had imbalanced labeling in our social media data, we alleviated this issue to improve the DL performance by integrating an advanced few-shot DL approach (SetFit in our study) and down-sampling technique. The few-shot approach reduces the requirement for manually labeled data and allows the designing of training datasets using diverse strategies. The down-sampling technique can convert an unbalanced training dataset into a more balanced training dataset. The few-shot approach with down-sampling technique is helpful to process social media data with imbalanced label distributions.

This deep learning-driven social media monitoring study can be leveraged to direct strategic efforts by informing health communication strategies and tailoring them to specific populations and areas. Efforts should address both persistent and emerging misinformation trends and negative discussions, with tailored interventions to specific regions exhibiting high levels of negative sentiment or misinformation. These interventions may include misinformation-busting messages, clarifications of scientific evidence, and promotion of vaccine benefits, possibly adapted to regional concerns and cultural context. For current HPV vaccination efforts in China, policymakers and researchers should apply social media monitoring techniques to track HPV vaccine discussions online, rapidly detect misinformation trends, and swiftly respond to emerging concerns related to adverse events or misinformation. Health care providers can use our findings to guide patients to credible sources and dispel falsehoods propagated on social media. Moreover, AI-powered tools like chatbots may offer scalable and responsive channels to disseminate accurate HPV vaccine information and engage positively with social media users and the public.

This study has several limitations. Firstly, a population bias may be present, as social media users may not represent the broader demographic characteristics of the general population. However, social media users tend to be younger, which aligns with the target population for HPV vaccination. Secondly, our analyses are based on individual posts instead of on social media users. Future research may develop innovative computational algorithms to analyze the individual user level by incorporating users’ profiles and historical posts. Thirdly, the gold-standard corpus of 6600 posts may not fully represent the diversity of the larger unlabeled dataset or unseen data, which may affect the robustness of the DL prediction. Fourthly, although we leveraged an advanced DL approach to mitigate bias that could have been possibly induced by label imbalance, there is still a need to improve prediction accuracy. Nevertheless, DL classifiers in our study demonstrated high accuracy across all tasks, ensuring reliability in analyses.

Conclusion

We conducted a deep learning analysis of HPV vaccine-related discussions posted on the Weibo platform according to behavior change theories. Our study highlighted the potential of using social media listening with DL techniques in the field of public health. Time-series analysis revealed dynamic shifts in public perceptions and exposures to information environments and practical barriers to HPV vaccination. Geographical analyses uncovered regions with higher prevalence of negative attitudes and misinformation. The findings highlight the need for sustained investment in HPV vaccine confidence/hesitancy monitoring and targeted health communication efforts. Our study can assist policymakers to inform tailored education strategies and adapt vaccination policy to address the low rates of HPV vaccination in China.

Supplementary Data 1. Source data underlying the figures. The file contains monthly prevalence data for all framework constructs (Fig. 3), provincial-level post counts and attitude prevalence (Fig. 4), and monthly attitude prevalence by HPV vaccine type (Fig. 5).

Author contributions

Z.H. conceived the study and led the analysis. Y.W., Z.D., Y.Z. and L.M. completed the manual annotation. H.Y. implemented data collection, data pre-processing, deep learning analysis, and data visualization. X.Z. participated in deep learning analysis. Y.W. implemented statistical analyses. Y.W. and Z.H. verified the data. Y.W., Z.D., L.L. and Z.H. wrote the manuscript. All authors contributed to reviewing and editing of the manuscript. Z.H. had full access to all the data in the study and made the final decision to submit for publication.

Peer review

Peer review information

Communications Medicine thanks Dian Hu, Monique Luisi and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Funding

This work is supported in part by a research grant from Investigator-Initiated Studies Program (No. 61185) of MSD. L.L. is supported by the AIR@InnoHK administered by the Innovation and Technology Commission of the Government of the Hong Kong Special Administrative Region. The funders had no role in the study design, data collection, data analysis, data interpretation, or writing of the report. The opinions expressed in this paper are those of the authors and do not necessarily represent those of MSD.

Data availability

The source data for Figs. 3–5 is in Supplementary Data 1. Original Weibo posts are not shared in accordance with Weibo’s data use policy. Other data are available from the corresponding author (Z.H., zyhou@fudan.edu.cn) upon reasonable request, with a response expected within 30 days.

Code availability

The corresponding Python and R codes used for data sampling, deep learning model training, implementation, and statistical analyses are available on GitHub (https://github.com/youwang13/HPVVaccineWeiboDL) and archived on Zenodo⁶⁰.

Competing interests

Zhiyuan Hou received funding from Merck Investigator Initiated Studies (61185). Leesa Lin is part of the research group from the Vaccine Confidence Project^TM at LSHTM, which, during the same period of the study, received research grants from GlaxoSmithKline, Merck, and Meta. The other authors declare no competing interests.

Consent to publish

Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

Supplementary information

The online version contains supplementary material available at https://doi.org/10.1038/s43856-026-01720-5.