A cross sectional content analysis evaluates chemotherapy health information quality and reliability on TikTok and Bilibili

Abstract

Short-video platforms, specifically TikTok and Bilibili, are widely utilized for health information seeking in China. However, the quality of chemotherapy-related content on these channels remains unverified. This study compares the reliability and quality of chemotherapy videos across these two platforms, evaluating the impact of uploader identity, video characteristics, and user engagement metrics. A total of 188 chemotherapy-related videos were retrieved from TikTok (n = 89) and Bilibili (n = 99). Content quality and reliability were independently assessed using the Global Quality Scale (GQS) and modified DISCERN (mDISCERN). Intra-platform interactions, video length, and uploader profiles were cross-sectionally analyzed using localized stratified Spearman’s rank correlation and fully adjusted multivariate Poisson regression models. Overall information quality was suboptimal. Significant baseline disparities were observed between platforms: while TikTok dominated in interaction metrics (P < .001), Bilibili achieved significantly higher reliability (mDISCERN, P < .001) and overall quality scores (GQS, P = .034). Certified oncologists were the primary contributors (46.28%), producing significantly higher-quality content than patients (P < .001). Localized stratified analysis exposed a platform-specific popularity paradox: within TikTok, GQS scores were significantly negatively correlated with likes (r = − .211, P = .047) and comments (r = − .269, P = .011), whereas this quality-engagement trade-off completely vanished within Bilibili, where mDISCERN positively aligned with collections (r = .221, P = .028) and shares (r = .249, P = .013). Fully adjusted Poisson regression confirmed that micro-level engagement counts and video length possessed no independent predictive capacity (P > .20), whereas the macro-platform identity of Bilibili emerged as a robust, standalone independent positive predictor of information reliability (RR = 1.388, 95% CI 1.069–1.801, P = .0138). A critical socio-technical divide exists within the digital health landscape, manifested as a localized popularity paradox heavily plaguing engagement-first networks like TikTok. Fully adjusted estimations reveal that the overarching macro-platform architecture itself, rather than micro-level clip features or standalone video length, serves as the independent determinant of clinical reliability. Platforms must transition from traffic-oriented metrics toward professional, quality-weighted recommendation system interventions to effectively bridge the gap between evidence-based clinical rigor and public digital accessibility.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1038/s41598-026-56880-0.

Introduction

As the cornerstone of systemic treatment for malignant tumors, chemotherapy plays a pivotal role throughout the entire continuum of care—ranging from neoadjuvant therapy aimed at tumor reduction to palliative care for advanced-stage disease¹. Although modern medicine has significantly improved patient tolerance through refined interventions, the process of eradicating malignant cells inevitably entails collateral damage to the hematopoietic and digestive systems^2–4; this inherent toxicity has long cast a shadow of “chemotherapy phobia” over the public consciousness. The limited scope of communication during outpatient visits struggles to bridge this cognitive gap⁵, and the resulting “information vacuum” compels patients and their families to turn to digital platforms in search of supplementary health information.

Digital platforms have become deeply embedded in the decision-making pathways of oncology patients^6,7. According to research by the Pew Research Center⁸, 90% of adults now have access to the internet, and the penetration rate of social media among cancer patients ranges from 40% to 70%. Studies indicate that the vast majority of cancer patients report that online communities play a critical role in their health-related decision-making^9,10. Distinct demographic groups exhibit differentiated usage patterns: young adults aged 18 to 29 demonstrate the highest usage rates, while middle-aged and elderly populations are showing a rapidly increasing reliance on intuitive forms of information, such as short-form videos. This high degree of digital dependency means that the quality of information patients acquire is, to a significant extent, contingent upon the specific media characteristics of the platforms they utilize¹¹.Against this backdrop, TikTok¹² and Bilibili¹³ have emerged as two representative yet distinct communication channels within China’s digital health landscape, each operating on a fundamentally different dissemination logic. TikTok’s core strength lies in its ability to achieve exceptionally high information reach, driven by powerful interest-based algorithms¹⁴. Its video format predominantly features concise, vertical-screen clips that encourage highly fragmented, immersive consumption. This “traffic-oriented” nature makes it an ideal window for fostering initial health awareness, excelling particularly at leveraging emotional resonance and visual impact to rapidly capture public attention regarding specific medical topics. However, because its algorithms prioritize user retention and engagement rates, in-depth professional knowledge is often distilled into easily shareable “soundbites” or narrative fragments¹⁵.In contrast, Bilibili demonstrates a markedly different logic regarding the dissemination of knowledge¹⁶.As a specialized content hub, Bilibili attracts a user base characterized by active, search-driven learning—often referred to as a “video-based encyclopedia.” Typical engagement on Bilibili involves long-form. Furthermore, the platform’s unique “Danmu” (real-time commentary) culture facilitates cohesive community interactions, providing a dedicated space for in-depth experience sharing between medical professionals and patients.

However, this digital transformation also presents formidable challenges. Although social media has emerged as a primary arena for the dissemination of chemotherapy-related knowledge, platform algorithms often prioritize content that is highly emotionally provocative or visually striking over scientifically rigorous medical evidence. This dissemination logic can lead to the viral spread of pseudoscientific information, thereby interfering with patients’ clinical decision-making^17,18.Currently, while cross-sectional evaluations on Chinese short-video platforms have emerged for specific malignancies like liver cancer¹⁹, benign conditions like uterine fibroids²⁰, or distinct modalities like radiotherapy²¹, there remains a clear dearth of systematic comparative evidence evaluating health communication patterns specifically focused on systemic chemotherapy across these two mainstream networks.This study aims to systematically evaluate the quality and reliability of chemotherapy-related short-form videos on TikTok and Bilibili, thereby illuminating the asymmetrical relationship between content reach (traffic volume) and professional rigor. By comparing the differential performance of information within these two distinct ecosystems, we hope to provide actionable insights that assist oncologists in enhancing their digital communication capabilities, guide platforms in optimizing their quality-weighted algorithms, and empower patients to identify reliable medical advice amidst an information vacuum.

Methods

Study design

This study employed a cross-sectional content analysis of publicly available short videos on major Chinese platforms, without involving human participants or identifiable personal information.

Platform selection and video capture

TikTok and Bilibili were designated as the primary data sources for this study. The selection was predicated on their extensive public reach and their status as the representative paradigms of the Chinese short-video ecosystem^22,23. Specifically, these two platforms were selected to encompass the full spectrum of digital health information dissemination: from algorithm-driven, decentralized content distribution (TikTok) to search-oriented, community-based knowledge sharing (Bilibili). By utilizing these distinct platforms, the study ensures that the sample is not only reflective of maximum user exposure but also captures the structural diversity of digital health communication in China. To maintain linguistic and contextual consistency, the analytical scope was strictly confined to Chinese-language content.

The data collection and video retrieval process was conducted between March 13, 2025, and July 4, 2025. The specific keyword “chemotherapy” (in Chinese: “化疗”) was utilized for the search query across both platforms. To simulate the search habits of the general public and minimize bias from personalized recommendation algorithms, videos were retrieved using newly registered accounts on devices with no prior browsing history. The search results were sorted by the platforms’ default modes of “comprehensive ranking” or “relevance.” All video metadata and engagement metrics were captured and archived during this specified period to ensure the consistency of the cross-sectional data.

Inclusion and exclusion criteria

The inclusion criteria were defined as follows: (1) content explicitly addressing chemotherapy mechanisms, management, or patient experiences; and (2) sufficient visual and audio quality to ensure information comprehensibility. The exclusion criteria comprised: (1) duplicate uploads; (2) content unrelated to medical topics; and (3) strictly commercial advertisements or product promotions. Following the screening process, a total of 188 videos were included in the final study sample, consisting of 89 videos from TikTok and 99 from Bilibili (Fig. 1).

Fig. 1.

Search strategy and videos screening process for chemotherapy.

Data extraction and categorization

For the included videos, key metadata were systematically extracted, including the video URL, publication date, duration, and user interaction metrics (likes, favorites, comments, shares, and follower counts). Based on the primary focus of the content, videos were categorized into five thematic groups: disease mechanisms, management of side effects, patient experiences, treatment regimen selection, and clinical case sharing. Furthermore, video uploaders were classified into four distinct categories based on their profile verification information: Oncologists, Other Specialists, Institutions (comprising official hospital accounts and media outlets), and Patients (including patients and their family members).

Quality assessment

To ensure objectivity and professional rigor, video content was systematically evaluated using two internationally validated instruments (Table 1):

Table 1.

The global quality score and the modified DISCERN are described to assess the quality and reliability of videos containing chemotherapy information.

Cumulative scale	Level	Interpretative descriptor
Global quality score
1	Poor quality	Specifically, the content is illogical, the narrative flow is poor, and most of the information is missing, making it useless for patients.
2	Generally poor quality	The content logic is poor, although some information is listed, more critical information is still missing, and the use of patients is minimal.
3	Moderate quality	Some vital information is adequately discussed.
4	Good quality and flow	Specifically, the video logic is clear and smooth, covering most of the relevant information, which is helpful for patients.
5	Excellent quality and flow	Specifically, the video logic is clear, and the content is very smooth, which is very useful for patients.
Modified DISCERN
1	Unreliable	Is the video clear, concise, and easy to understand?
2	Less reliable	Are valid sources cited?
3	Fairly reliable	Was the content presented balanced and unbiased?
4	Relatively reliable	Are additional sources of content listed for patient reference?
5	Reliable	Are areas of uncertainty mentioned?

The Modified DISCERN Tool (mDISCERN)²⁴: Employed to assess content reliability, focusing on structural clarity, source citation transparency, and the explicit disclosure of clinical uncertainty. To strictly adhere to the validated scoring architecture of the original instrument, the scale was operationalized as a cumulative index summing five independent binary items (assigned 1 point for a ‘Yes’ and 0 points for a ‘No’ or ‘Not applicable’), ultimately yielding an aggregate interval score ranging from 0 to 5 points. For clarity, the hierarchical arrangement presented in Table 1 reflects the progressive framework of informational facets evaluated (e.g., source reliability, bias control, and comprehensive treatment coverage) rather than an alteration to the scoring mathematics. A higher summated value explicitly signifies a superior compliance rate with rigorous evidence-based clinical communication guidelines.The Global Quality Scale (GQS): Used to evaluate the overall flow, pedagogical quality, and practical utility of the video for patients. This instrument utilizes a standard 5-point ordinal scale, where a score of 5 represents excellent educational quality.

All videos were independently evaluated by two raters with clinical medical backgrounds in a blinded manner. Prior to formal evaluation, the raters underwent standardized training to calibrate scoring criteria. Any significant discrepancies in scoring between the two raters were resolved through arbitration by a third senior researcher, reaching consensus through discussion to ensure data accuracy. To statistically demonstrate the reproducibility of the independent evaluations, inter-rater reliability for the initial scoring prior to arbitration was quantified using the two-way random-effects, absolute agreement, single-rater Intraclass Correlation Coefficient (ICC). The initial baseline ICC for the GQS was 0.772 (95%confidence interval [CI]: 0.707–0.825, P < .001), and the baseline ICC for the modified DISCERN was 0.861 (95% CI 0.817–0.895, P < .001). These parameters indicate substantial to excellent consistency between the two primary evaluators, ensuring the robust reproducibility of the content quality analytics.

Statistical analysis

Data normality was assessed using the Shapiro-Wilk test, revealing a non-normal distribution. Consequently, continuous variables are reported as medians with interquartile ranges (IQR), and categorical variables are presented as frequencies and percentages. Differences between groups were evaluated using the Mann–Whitney U test (for two groups) or the Kruskal–Wallis H test (for multiple groups). Post hoc pairwise comparisons were performed using Dunn’s test with Bonferroni correction. Correlations between variables were assessed using Spearman’s rank correlation coefficient.

To identify independent predictors of video quality, a Poisson regression model was constructed to calculate relative risks (RR). This model was selected due to the nature of quality scores as discrete count data and because RRs offer superior clinical interpretability compared to odds ratios (OR) derived from logistic regression. All statistical analyses were conducted using R software (version 4.5.0), with a two-sided P < .05 considered statistically significant.

Results

Based on keyword searches, a total of 188 videos were included for analysis (Table 2). The Mann-Whitney U test revealed significant divergence between the two platforms regarding interaction characteristics and content quality.In terms of user engagement, TikTok demonstrated a predominant advantage. Its median counts for likes (11,804.00 vs. 176.00), comments (744.00 vs. 2.00), favorites (4097.00 vs. 51.00), and shares (3426.00 vs. 17.00) were all significantly higher than those of Bilibili ( P<.001). Moreover, TikTok creators possessed a substantially larger follower base (275,000.00 vs. 7706.00, P<.001).However, this advantage in traffic volume did not translate into superior content quality. Although there was no statistical difference in video duration between the two platforms (124.00 vs. 139.00, P=.12), Bilibili exhibited a significant advantage in information quality and reliability (Fig. 2). Specifically, the median mDISCERN score for Bilibili (3, IQR: 2–4) was significantly higher than that of TikTok (2, IQR 2–3, P<.001). Similarly, the distribution of GQS scores for Bilibili was statistically superior to that of TikTok (P=.034).

Table 2.

Characteristics of videos on TikTok and Bilibili.

Variables	Total (n = 188)	Bilibili (n = 99)	Tiktok (n = 89)	Statistic	P
Video length, M (Q₁, Q₃)	130.00 (86.75, 216.50)	139.00 (89.50, 243.00)	124.00 (81.00, 179.00)	Z=− 1.55	0.120
Likes, M (Q₁, Q₃)	1853.00 (155.75, 10655.00)	176.00 (11.00, 490.50)	11804.00 (6533.00, 18006.00)	Z=− 11.59	< 0.001
Collections, M (Q₁, Q₃)	536.50 (42.50, 3843.00)	51.00 (7.00, 147.50)	4097.00 (1703.00, 8216.00)	Z=− 11.35	< 0.001
Comments, M (Q₁, Q₃)	51.50 (1.00, 643.25)	2.00 (0.00, 13.50)	744.00 (295.00, 1293.00)	Z=− 11.61	< 0.001
Shares, M (Q₁, Q₃)	247.50 (15.75, 2827.00)	17.00 (3.00, 80.50)	3426.00 (1169.00, 6474.00)	Z=− 11.42	< 0.001
Fans, M (Q₁, Q₃)	52500.00 (6755.00, 248000.00)	7706.00 (1213.50, 34000.00)	275000.00 (87000.00, 609000.00)	Z=− 10.04	< 0.001
mDiscern, M (Q₁, Q₃)	3.00 (2.00, 3.00)	3.00 (2.00, 4.00)	2.00 (2.00, 3.00)	Z=− 4.93	< 0.001
GQS, M (Q₁, Q₃)	3.00 (2.00, 3.00)	3.00 (2.00, 3.00)	3.00 (2.00, 3.00)	Z=− 2.12	0.034

Fig. 2.

Comparison of quality and reliability scores between TikTok and Bilibili videos. (A) Global quality score; (B) Modified DISCERN score.

To further characterize the foundations of this digital ecosystem, we conducted an in-depth analysis of video authorship and thematic content. The results delineate a dissemination system heavily reliant on professional medical expertise (Fig. 3). Contrary to the common perception that short-video content is dominated by “lay narratives,” medical professionals constituted the absolute majority of content creators. Oncologists alone contributed nearly half of the videos (46.28%, 87/188); combined with other specialists (21.28%, 40/188), the proportion of medical professionals far exceeded that of patients (17.02%) and medical institutions (15.43%).

Fig. 3.

Distribution of video sources and content themes. (A) Composition of video uploaders classified by identity. (B) Classification of video content themes.

Thematically, this professional dominance translated into a focus on technical medical knowledge. Disease mechanisms (37.77%) and side effect management (25.53%) were the most prevalent topics, whereas specific treatment regimen selection remained relatively niche (6.91%) (Fig. 3B). However, a potential tension emerges between the high prevalence of professional credentials (Fig. 3A) and the previously noted variability in quality scores (Fig. 2).

To ascertain whether the prevalence of professional content creators translates into consistently high-quality information, we further analyzed the specific distribution of quality tiers across both platforms. Despite the dominance of medical professionals in content production, the distribution of Global Quality Scale (GQS) scores revealed disparities in content depth (Fig. 4A; Table 3). On Bilibili, a substantial proportion of videos achieved a GQS score of 4 (“Good”), accounting for 21.2% of the sample (21/99). In contrast, content on TikTok struggled to meet this standard, with only 7.9% (7/89) of videos rated as “Good.” While the proportions of “moderate quality” videos were comparable across both platforms (44.4% and 46.1%, respectively), the prevalence of content rated as “poor” or “generally poor” was marginally higher on TikTok than on Bilibili.

Fig. 4.

Distribution of video quality scores by platform. Bar charts illustrating the frequency and percentage of videos; (A) Global quality score; (B) Modified DISCERN scores.

Table 3.

Global quality score and modified DISCERN scores for TikTok and Bilibili videos related to chemotherapy.

Scale, score	Bilibili (n = 99)	TikTok (n = 89)	Comparative trend
Global quality score
1 (Poor quality)	7	10	Close
2 (Generally poor quality)	27	30	Stable
3 (Moderate quality)	44	41	Close
4 (Good quality and flow)	21	7	Bilibili
5 (Excellent quality and flow)	0	1	Tiktok
Modified DISCERN
1 (Unreliable)	10	17	TikTok
2 (Less reliable)	19	42	TikTok
3 (Fairly reliable)	44	23	Bilibili
4 (Relatively reliable)	22	7	Bilibili
5 (Reliable)	4	0	Bilibili

This polarization became more pronounced when assessing information reliability via the modified DISCERN instrument (Fig. 4B; Table 3). The analysis highlighted a concerning accumulation of low-reliability content on TikTok, where 66.3% (59/89) of videos were classified as “unreliable” (score 1) or “poorly reliable” (score 2). Conversely, Bilibili demonstrated superior performance regarding high-reliability content. Notably, 26.3% (26/99) of Bilibili videos were rated as “relatively reliable” or “reliable” (scores of 4 or 5), whereas only 7.9% of TikTok videos met these criteria. This distribution suggests that while users on both platforms have access to content produced by professionals, the information presented on Bilibili is significantly superior in terms of depth, balance, and reliability compared to the fragmented narratives prevalent on TikTok.

We further investigated whether content creator identity influences video characteristics and information quality. As presented in Table 4, the Kruskal-Wallis test revealed significant variations in video duration (χ² = 29.18, P < .001). Videos produced by medical institutions (median 257.00 s) and patients (median 218.00 s) were significantly longer than those created by oncologists (115.00 s) and other specialists (126.00 s).

Table 4.

Characteristics of video uploaders on TikTok and Bilibili about chemotherapy.

Variables	institution (n = 29)	Oncologist (n = 87)	Other specialist (n = 40)	Patient (n = 32)	Statistic	P
Video length, M (Q₁, Q₃)	257.00 (125.00,454.00)	115.00 (76.50,157.00)	126.00 (80.25,182.00)	218.00 (109.00,293.25)	χ²=29.18	< 0.001
Likes, M (Q₁, Q₃)	1032.00 (67.00,8766.00)	2973.00 (29.00,9710.50)	3610.50 (304.50,12839.25)	589.00 (250.50,12220.00)	χ²=1.79	0.618
Collections, M (Q₁, Q₃)	892.00 (77.00,4788.00)	535.00 (18.50,4276.50)	834.50 (59.50,3770.50)	170.50 (45.00,1289.75)	χ²=1.75	0.626
Comments, M (Q₁, Q₃)	45.00 (3.00,376.00)	45.00 (0.00,515.50)	276.00 (1.00,858.50)	22.00 (11.75,1433.00)	χ²=4.97	0.174
Shares, M (Q₁, Q₃)	331.00 (48.00,5559.00)	243.00 (4.00,3233.00)	406.50 (26.25,2201.75)	94.00 (15.00,902.00)	χ²=3.12	0.373
Mdiscern, M (Q₁, Q₃)	3.00 (2.00,4.00)	3.00 (2.00,3.00)	3.00 (2.00,3.25)	2.00 (1.00,2.00)	χ²=25.90	< 0.001
GQS, M (Q₁, Q₃)	3.00 (2.00,4.00)	3.00 (2.00,3.00)	3.00 (2.00,3.00)	2.00 (1.00,2.00)	χ²=31.54	< 0.001

However, contrary to expectations that professional content might command higher authority or that patient narratives might offer greater appeal, no statistically significant differences in interaction metrics were observed among these four groups. Regardless of whether the uploader was a medical expert or a layperson, the counts for likes (P = .618), favorites (P = .626), comments (P = .174), and shares (P = .373) remained at comparable levels.

Despite comparable user engagement, significant disparities emerged regarding information quality and reliability (Table 4; Fig. 5). Statistical analysis indicated highly significant differences in mDISCERN scores (χ² = 25.90, P < .001) and GQS scores (χ² = 31.54, P < .001) across groups. As illustrated by the violin plots in Fig. 5, the quality of videos uploaded by patients was generally lower, with a median of 2.00 for both metrics. Pairwise comparisons demonstrated that patient-generated content was of significantly lower quality than content produced by oncologists (P < .001), other specialists (P < .001), and medical institutions (P < .001). Conversely, no significant differences were observed among the professional groups (oncologists, other specialists, and institutions).

Fig. 5.

Quality and reliability scores of chemotherapy-related videos from different sources. (A) Global quality score; (B) Modified DISCERN score. NS: not significant at P<.05.

We further dissected specific dimensions of information quality across different uploader categories to identify distinct strengths and systemic deficits. As illustrated in the radar chart (Fig. 6A), performance across the five mDISCERN criteria exhibited significant variation. Encouragingly, all uploader groups—regardless of professional background—demonstrated exceptional performance in the dimension of “clarity,” achieving high scores. This indicates that content on these platforms is generally clear, concise, and accessible to the lay public.

Fig. 6.

(A) Relative strength and balance of modified DISCERN scores across different dimensions; (B) Overall distribution of Global Quality Score from different sources; (C) Overall distribution of modified DISCERN scores from different sources.

However, distinct disparities emerged in the dimensions of “reliability of sources” and “balanced and unbiased information.” Oncologists, other specialists, and medical institutions outperformed the patient group, who scored notably lower in these domains.

Despite these professional advantages, the radar chart also exposed pervasive deficiencies. Scores across all groups were universally low—approaching zero—in the dimensions addressing “areas of uncertainty” and “provision of additional information sources.” This suggests that even professional content creators rarely elucidate medical uncertainties or provide supplementary references for the audience.

The ridgeline plots for GQS (Fig. 6B) and mDISCERN (Fig. 6C) further visualized these distributional trends. The density curves for oncologists and medical institutions were distinctly skewed towards higher values, peaking around scores of 3 and 4. In contrast, the patient group exhibited a peak at the lower end of the spectrum (approximately 2). These findings visually confirm that while professional content is generally more reliable, the broader ecosystem lacks the necessary rigor regarding source citation and the discussion of clinical uncertainty.

To intuitively elucidate the complex interplay between user engagement and information quality, we constructed a Spearman correlation coefficient heatmap (Fig. 7) and conducted an in-depth analysis combining specific statistical tests (Table 5). The heatmap clearly revealed two internally consistent yet mutually opposing clusters of variables. On one hand, the quality assessment tools exhibited a significant positive correlation (GQS and mDISCERN, r = .76). On the other hand, user interaction metrics (likes, comments, favorites, shares) displayed extremely strong multicollinearity (r>.89), forming a tight “engagement cluster.”

Fig. 7.

Spearman correlation heatmap between video quality scores, user engagement metrics, and video characteristics. The “×” symbol denotes correlations that are not statistically significant (P > .05).

Table 5.

Spearman correlation analysis between video interaction metrics/characteristics and the Global Quality Scores and modified DISCERN scores.

Variable	Global quality score		modified DISCERN
	r	P value	r	P value
Likes	− 0.213*	0.003	− 0.345*	< 0.001
Comments	− 0.247*	< 0.001	− 0.38*	< 0.001
Collections	− 0.046	0.534	− 0.211*	0.004
Shares	− 0.029	0.696	− 0.209*	0.004
Video length	0.167*	0.022	0.189*	0.009
Fans	− 0.148*	0.043	− 0.304*	< 0.001

*Indicates statistical significance (P < .05).

However, when examining the interaction between these two clusters, the heatmap presented a dominance of “cool tones,” revealing a widespread negative correlation between popularity and professional standards. Specifically, as detailed in Table 5, mDISCERN scores demonstrated highly significant negative correlations with comments (r = − .38, P < .001) and likes (r = − .345, P < .001), as well as significant negative correlations with favorites (r = − .211, P = .004) and shares (r = − .209, P = .004). A parallel trend was observed regarding GQS scores, which exhibited significant negative correlations with comments (r = − .247, P < .001) and likes (r = − .213, P = .003). In sharp contrast, video duration emerged as the sole variable positively correlated with quality, showing significant positive associations with both GQS (r = .167, P = .022) and mDISCERN (r = .189, P = .009).

To further dissect the independent intra-platform algorithmic dynamics and eliminate potential aggregate data artifacts (such as Simpson’s paradox), a stratified Spearman correlation analysis was cross-sectionally executed within each individual platform network (Tables 6 and Fig. 8). Within the TikTok ecosystem (n = 89), the ‘popularity paradox’ was independently validated as an inherent reality; video quality (GQS) remained significantly negatively correlated with Likes (r = − .211, P = .047) and Comments (r = − .269, P = .011), while information reliability (mDISCERN) was similarly inversely coupled with Comments (r = − .215, P = .043). Video length showed no significant correlation with either GQS (r = .096, P = .370) or mDISCERN (r = .157, P = .143) within the TikTok ecosystem. Conversely, within the Bilibili knowledge-sharing network (n = 99), all localized negative correlations completely vanished (P > .05). Crucially, video quality (GQS) was found to be significantly positively associated with Video length (r = .225, P = .025). Furthermore, information reliability (mDISCERN) was significantly positively aligned with deep audience utility behaviors, specifically Collections (r = .221, P = .028) and Shares (r = .249, P = .013), although its localized correlation with Video length did not reach statistical significance (r = .188, P = .062).

Table 6.

Stratified Spearman correlation analysis between video quality metrics and user engagement indicators within individual platforms.

Platform and variable	Global quality score		Modified DISCERN
	R	P value	r	P value
TikTok (n = 89)
Video length	0.096	0.370	0.157	0.143
Likes	− 0.211	0.047*	− 0.173	0.105
Comments	− 0.269	0.011*	− 0.215	0.043*
Collections	0.22	0.038*	0.165	0.122
Shares	0.173	0.105	0.115	0.285
Fans	0.089	0.407	0.173	0.105
Bilibili (n = 99)
Video length	0.225	0.025*	0.188	0.062
Likes	− 0.105	0.302	0.033	0.745
Comments	− 0.195	0.053	− 0.085	0.404
Collections	0.109	0.283	0.236	0.019*
Shares	0.186	0.065	0.276	0.006*
Fans	− 0.102	0.316	− 0.158	0.119

*Indicates statistical significance (P < .05).

Fig. 8.

Spearman correlation heatmaps between video quality scores, user engagement metrics, and video characteristics within TikTok and Bilibili ecosystems. (A) Localized stratified analysis within TikTok (n = 89); (B) Localized stratified analysis within Bilibili (n = 99). Note: The “×” symbol denotes correlations that are not statistically significant (P > .05). Color gradients and numbers represent the strength and direction of Spearman rho coefficients (r).

Based on the aforementioned correlations, we constructed a generalized linear model (Poisson regression, Table 7) to investigate whether interaction metrics and video characteristics could serve as independent predictors of high-quality content. The regression analysis revealed that, despite the presence of univariate correlations, no single video variable—including likes, comments, duration, or follower count—could significantly predict GQS or mDISCERN scores in the multivariate model.

Table 7.

Association between video variables and Global Quality Score and modified DISCERN score.

Scale	Variable	RR (95% CI)	P value
Global quality score	Likes	0.905 (0.694–1.180)	0.4605
	Comments	0.000 (0.851–1.175)	1.0000
	Collections	0.963 (0.769–1.206)	0.7422
	Shares	0.948 (0.696–1.293)	0.7363
	Video length	1.000 (1.000–1.000)	0.0501
	Fans	1.000 (1.000–1.000)	0.4396
Modified DISCERN	Likes	0.969 (0.813–1.156)	0.7288
	Comments	0.955 (0.743–1.228)	0.7214
	Collections	1.071 (0.884–1.298)	0.4806
	Shares	0.992 (0.705–1.397)	0.9645
	Video length	1.000 (1.000-1.001)	0.1318
	Fans	1.000 (1.000–1.000)	0.2622

Specifically, P-values for all variables exceeded 0.05. Even video duration, which showed a trend towards significance, yielded a P-value of only 0.0501 for GQS prediction, failing to meet the threshold for statistical significance. Furthermore, the relative risks for all variables hovered around 1.000 (e.g., for likes in the GQS model: RR = 0.905, 95% CI 0.694 − 1.180), and all 95% confidence intervals spanned the null value (1.00).

Furthermore, to statistically control for cross-platform baseline confounding environments and account for the structural attributes of individual clips, a fully adjusted multivariate Poisson regression model was reconstructed by simultaneously incorporating both platform identity (Bilibili vs. TikTok) and Video length as explanatory covariates (Table 8). In the fully adjusted GQS model, the standalone predictive effects of all micro-engagement features and video length remained deeply non-significant (P > .20). Strikingly, in the fully adjusted mDISCERN model, while isolated user interaction counts—including Likes (P = .7849), Comments (P = .9239), Collections (P = .6688), Shares (P = .9157), and Fans (P = .3006)—and Video length (RR = 1.000, P = .2479) exhibited no independent predictive capacity, the macro-platform identity of Bilibili emerged as a robust, highly significant independent positive predictor of medical information reliability (RR = 1.388, 95% CI 1.069–1.801, P = .0138). This mathematical adjustment isolates that even when micro-level virality and video length are perfectly equalized across the pooled sample, Bilibili platform identity independently carries a significantly higher baseline premium for clinical reliability standards compared to TikTok.

Table 8.

Fully adjusted multivariate Poisson regression analysis evaluating independent predictors of video quality and reliability (n = 188).

Scale	Variable	RR (95% CI)	P value
Global quality score	Video length	1.000 (1.000–1.000)	0.2331
	Likes	1.000 (1.000–1.000)	0.6091
	Comments	1.000 (1.000–1.000)	0.4358
	Collections	1.000 (1.000–1.000)	0.8317
	Shares	1.000 (1.000–1.000)	0.8920
	Fans	1.000 (1.000–1.000)	0.8746
	Platform (Bilibili vs. TikTok)a	1.106 (0.864–1.417)	0.4234
Modified DISCERN	Video length	1.000 (1.000–1.000)	0.2479
	Likes	1.000 (1.000–1.000)	0.7849
	Comments	1.000 (1.000–1.000)	0.9239
	Collections	1.000 (1.000–1.000)	0.6688
	Shares	1.000 (1.000–1.000)	0.9157
	Fans	1.000 (1.000–1.000)	0.3006
	Platform (Bilibili vs. TikTok)a	1.388 (1.069–1.801)	0.0138

^aTikTok was set as the reference group.

Discussion

Principal findings

This study systematically evaluated a total of 188 chemotherapy-related videos across the TikTok and Bilibili platforms. Overall, the informational quality of these videos was suboptimal—a finding consistent with previous domestic Chinese studies focusing on hepatocellular carcinoma and uterine fibroids^19,20. Disparities between platforms were highly prominent: while TikTok dominated in terms of raw user engagement, Bilibili significantly outperformed TikTok in absolute reliability metrics (P < .001). Crucially, this reliability premium for Bilibili remained robustly significant even under fully adjusted multivariate conditions (RR = 1.388, P = .0138).

Methodologically, this study employed GQS and mDISCERN instruments and differentiated “oncologists” from “other specialists”. This stratification reflects the sub-specialty barriers inherent in chemotherapy and provides a more accurate representation of expert content distribution^25,26. Oncologists were the primary creators (46.28%), producing significantly higher-quality content than patients (P < .001).

Furthermore, our analysis unveiled that the ‘popularity paradox’ is a highly platform-specific phenomenon rather than a uniform blanket baseline across all digital domains. Strikingly, when aggregate data artifacts were mathematically decoupled via localized stratified profiling (Table 6; Fig. 8), the inverse relationship between virality and professional quality held firmly and exclusively within the TikTok ecosystem (n = 89). In TikTok, GQS scores exhibited significant negative correlations with likes (r = -.211, P = .047) and comments (r = -.269, P = .011), proving that its vertical, immersive short-feed algorithms natively prioritize prompt emotional triggers over evidence-based scientific depth. Conversely, this quality-engagement trade-off completely vanished within the Bilibili network (n = 99), where higher information reliability (mDISCERN) was positively coupled with deep audience utility markers like collections (r = .221, P = .028) and shares (r = .249, P = .013). This clear divergent alignment highlights that the localized popularity paradox is heavily governed by independent socio-technical platform mechanics rather than a generalized public cognitive bias toward oncological therapeutics.

Quality limitations of short videos and the “popularity paradox”

However, the intrinsic characteristics of the short-video format limit the breadth and depth of chemotherapy-related content^27,28. The brevity and fragmented presentation style often impede a comprehensive understanding of treatment regimens; specifically, complex topics such as pharmacologic mechanisms and side effect management are difficult to adequately elucidate within seconds. Our findings support this observation: Bilibili, characterized by longer video durations and a culture of in-depth engagement²⁹, exhibited significantly superior information quality compared to TikTok, which is dominated by “snackable” consumption.

Interaction metrics—such as likes, comments, and shares—are pivotal for enhancing visibility on social media^30–32. Higher engagement rates trigger algorithmic recommendations, thereby amplifying content reach^33,34. However, our stratified empirical evidence exposes a polarized platform ecosystem. The localized popularity paradox was uniquely verified within TikTok (e.g., GQS vs. comments: r = -.269, P = .011), heavily driven by the feed’s immediate behavioral feedback loop mechanism. In the context of oncology, chemotherapy is inherently coupled with profound public health anxiety, which recommendation engines often exploit by boosting sensationalized or emotionally charged content³⁵.

To mathematically isolate this platform effect from micro-level attributes, our fully adjusted multivariate Poisson regression model (Table 8) yielded a critical socio-technical insight: after strictly neutralizing all variances in user interactions and standalone video length, individual engagement features and clip durations cross-sectionally exhibited relative risks precisely centered at 1.000 (RR = 1.000, P > .20), whereas the macro-platform identity of Bilibili emerged as a powerful, standalone independent positive predictor of clinical reliability (RR = 1.388, 95%CI 1.069–1.801, P = .0138). This premium confirms that the foundational architecture of the network environment itself—Bilibili’s search-oriented infrastructure and community-based retention habits—acts as an institutional filter that structurally safeguards evidence-based rigor, successfully neutralizing the algorithmic popularity paradox observed in traffic-first models.

The implications of this structural misalignment are profound. When high-quality, evidence-based content is marginalized, a critical information imbalance occurs, leaving patients primarily exposed to incomplete or biased narratives³⁶. If oncologists, who are the primary content contributors, find their high-quality medical content suppressed due to a lack of sensationalism, or if they feel compelled to oversimplify information to “chase traffic,” the integrity of information is compromised. Such informational bias risks precipitating unwarranted fear of chemotherapy or misunderstandings regarding treatment, potentially culminating in treatment delays and posing serious risks to patient prognosis³⁷.

To resolve this “popularity paradox” and support the evolution towards precision medicine, modern digital tools must transition from passive information sources to functional decision-support platforms³⁸. The successful implementation of precision medicine relies heavily on the accurate delivery of individualized and complex medical information—such as targeted therapies for specific genetic mutations³⁹. First, digital tools should utilize big data profiling to establish precision dissemination mechanisms, directing sophisticated knowledge (e.g., the clinical significance of biomarker testing) specifically to matching pathological patient populations, thereby overcoming the information gaps caused by the popularity paradox⁴⁰. Second, platforms should leverage multimedia advantages for the visualization of complex logic, using animations to intuitively demonstrate molecular-level targeting mechanisms, which reduces the cognitive barriers for patients to understand precision therapeutic regimens⁴¹. Finally, by incorporating “quality-weighted” recommendation mechanisms based on professional authority, digital tools can ensure that information in digital spaces is highly synchronized with clinical precision treatment pathways. Only when communication mechanisms possess a level of precision matching that of the medical technology itself can digital tools truly serve as an efficient engine for the universal implementation of precision oncology.

Professionalization pathways and improvement strategies

Given the stringent requirements for scientific accuracy in the field of chemotherapy, we view the dominance of oncologists in content creation (46.28%) as a positive trend. In our sample, certified medical professionals constituted the primary force of information dissemination, thereby reinforcing the foundational reliability of the content. However, professional credentials alone do not guarantee high-quality communication. Our radar chart analysis indicates that even medical professionals exhibit systemic deficiencies in citing reliable sources and disclosing clinical uncertainty.

Consequently, oncology professionals must transcend their clinical roles to enhance their digital communication competencies. We advocate for the use of clear visual metaphors to deconstruct complex pharmacological mechanisms into accessible steps, mitigating the inherent technicality of chemotherapy content. ^42,43. Furthermore, given the substantial inter-individual variability in chemotherapy responses, professionals should actively address clinical uncertainty and utilize evidence-based visual aids. This approach not only enhances public comprehension and retention of medical knowledge but also fosters robust doctor-patient trust within a limited timeframe, ensuring that information dissemination is both scientifically rigorous and humanistically oriented^44,45.

Practical implications and policy recommendations

This study provides critical empirical evidence regarding the ecology of chemotherapy-related knowledge on short-video platforms. By comparing TikTok and Bilibili, we exposed a structural misalignment between traffic volume and information quality. Our identification of the “popularity paradox”—where high interaction correlates with low quality—serves as a wake-up call for medical professionals and platform regulators, indicating that current algorithmic mechanisms are misaligned with public health goals^46–48.

To address these challenges, we advocate a shift from “traffic-oriented” to “quality-oriented” algorithmic interventions. Specifically, platforms should integrate professional verification and evidence-based criteria into traffic distribution weights to prevent high-quality content from being suppressed. Given the lack of source citation among professionals, we call for standardized publishing guidelines requiring data sourcing and uncertainty disclosures. Furthermore, mandatory content labeling (distinguishing “personal experience” from “medical advice”) is essential to empower user critical thinking, mitigating the risks posed by lower-quality patient-generated content. Future research should assess the clinical impact of video consumption on treatment adherence and develop automated quality evaluation tools to bridge the “digital divide” in chemotherapy education.

Strengths and limitations

This study demonstrates distinct methodological robustness²⁰. First, the dual-platform analysis moves beyond single-source perspectives, elucidating how different algorithmic ecologies differentially impact health communication. Second, the analytical rigor extends beyond descriptive statistics; the application of Spearman heatmaps and Poisson regression models provided strong empirical validation for the “popularity paradox.” Third, our granular classification of uploaders quantified the dominance of oncologists, offering precise data on the professionalization trends in digital health.

However, limitations exist. First, the GQS and mDISCERN instruments were originally designed for text, potentially failing to capture the nuances of audiovisual pedagogy. Second, while we quantified interaction metrics, the lack of qualitative sentiment analysis limits our understanding of the nature of user feedback (e.g., genuine learning vs. controversy). Third, although chronological baseline information regarding publication dates was recorded, the localized longitudinal temporal accumulation of user engagement metrics was not dynamically controlled as a regression covariate. In digital health communication, metrics such as likes and comments naturally aggregate over time, meaning older videos may inherently possess inflated absolute interaction volumes irrespective of clinical precision. Nevertheless, in our fully adjusted multivariate Poisson regression models (Table 8), after strictly neutralizing platform environments and video length, the Relative Risks for all standalone micro-engagement features were clamped precisely at 1.000 with deeply non-significant values (P > .20), indicating that such potential temporal inflation noise was not structurally robust enough to distort the cross-sectional predictive capacity of virality markers. Fourth, the cross-sectional design within the Chinese context restricts global generalizability and causal inference⁴⁹, and small sample sizes in specific sub-themes warrant cautious interpretation. Future studies should explore video-specific evaluation frameworks, integrate standardized engagement-velocity indexes (e.g., likes accumulated per day post-publication), and conduct cross-cultural qualitative audience analyses.

Conclusion

The evaluation of 188 chemotherapy-related videos revealed critical structural divides in the digital health landscape: Bilibili significantly outperformed TikTok regarding information reliability and professional quality. Despite a progressive shift toward professionalization—with certified oncologists contributing nearly half of the content—a localized ‘popularity paradox’ heavily plagues TikTok’s engagement-first network, wherein high interaction volumes are inversely associated with medical accuracy, while Bilibili successfully aligns clinical reliability with deep audience utility habits. Furthermore, our fully adjusted regression analysis confirms that the macro-platform architecture itself, rather than micro-level clip attributes or standalone video length, serves as the independent determinant of information reliability. Rectifying current systemic imbalances necessitates the implementation of professional ‘quality-weighted’ recommendation weights by social media corporations alongside a concerted effort by oncology experts to harmonize accessibility with scientific integrity in digital environments.

Ethical considerations

This study utilized exclusively publicly available data and involved no direct interaction with human subjects or animals; therefore, Institutional Review Board (IRB) approval was not required. All data were de-identified to protect user privacy. The datasets generated and analyzed during the current study are available from the corresponding author upon reasonable request.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary Material 1 ^{(21.3KB, xlsx)}

Abbreviations

GQS

Global quality scale

mDISCERN

Modified DISCERN

IQR

Interquartile range

RR

Relative risk

OR

Odds ratio

CI

Confidence interval

Author contributions

CCX: Responsible for research design, data analysis, results interpretation, and drafting the initial manuscript；TSR: Responsible for data processing, statistical analysis, and chart creation；BQW: Responsible for project conception, research guidance, and review and approval of the final manuscript; serves as the corresponding author.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Data availability

The raw data used in this study came from Douyin (https://www.douyin.com/) and Bilibili (https://www.bilibili.com/), and these data are publicly available on the platforms. Analysis datasets generated during the research process can be obtained from the corresponding author upon reasonable request.

Declarations

Competing interests

The authors declare no competing interests.