Scrolling for Answers About Breast Implant Illness: Application of Validated Tools to Assess the Quality of Content on YouTube and TikTok

Abstract

Introduction: Popular video-sharing platforms YouTube and TikTok offer a plethora of information on the topic of breast implant illness (BII). As a largely patient-driven phenomenon, it is important to understand the influence of social media on patient knowledge regarding BII. This study sought to evaluate the quality, reliability, visibility, and popularity of YouTube and TikTok videos about BII. Methods: Two validated tools for health information, DISCERN and the Patient Education Materials Assessment Tool (PEMAT), were utilized to evaluate the quality of information regarding the topic of BII on YouTube and TikTok. High DISCERN score indicates content of superior quality and reliability, while elevated PEMAT scores signify content that is easily understandable and actionable for viewers. The search phrase “breast implant illness” was used to screen videos, which were sorted based on relevance and view count. The first 100 videos that fulfilled inclusion criteria were independently graded by three reviewers. Results: TikTok videos of longer duration, a higher number of shares, and in the patient education category were all significantly associated with a higher total DISCERN score (P < 0.05). YouTube videos that included a provider or a patient were significantly more likely to have a higher total DISCERN and PEMAT understandability score (all P < 0.05). Discussion of physician education, operation details, and patient experience was significantly associated with higher total DISCERN and PEMAT understandability scores (all P < 0.05). Conclusions: Total DISCERN and PEMAT scores for videos regarding BII on two popular social media platforms are low. Video length, patient experience categorization, and the presence of a provider are worth considering when developing high-quality online content for breast reconstruction and augmentation patients.

Introduction

Breast implant illness (BII) is a patient-driven phenomenon that has yet to be added to the international classification of diseases list of acknowledged diseases. It describes a constellation of systemic symptoms that patients report to be associated with their breast implants. ¹ The concept of BII is not a new phenomenon; a link between breast implants and systemic symptoms has been reported since the 1960s.^1,2 While BII is still largely debated in medical literature, the rapid growth and popularity of BII on social media has contributed to its awareness and interest worldwide. ¹ The rise of patient advocacy and communication through social media has led to an increasing number of presenting cases for plastic surgeons. ¹ To gain insight into the process by which patients begin to seek medical and/or surgical intervention, it is critical to examine how BII is discussed on the social media platforms that are widely used by patients.

With over 248 million unique monthly viewers in the United States, YouTube (Alphabet Inc., San Bruno, California) is the most popular video-sharing site in the world. ³ Similarly, TikTok (Culver City, CA, USA) is a widely growing digital media platform that has amassed 1.2 billion monthly active users in the third quarter of 2021. ⁴ These platforms are increasingly being utilized as a medical resource for cosmetic surgery, where patients look for insight on treatment options. ⁵

These hubs of digital information, while successful in amassing many viewers, currently lack a designated, validated, and reliable tool to assess the quality of their content. As a medical resource, the internet has the tendency to sensationalize and rapidly popularize certain opinions, regardless of it originating from a reputable source, in part due to the ease at which information can spread rapidly online. ⁶ As such, it can be difficult for patients to discern which information comes from reputable and trusted sources. ⁷ Patients are at risk of making dangerous decisions with potentially detrimental outcomes if they source information from poor quality internet resources.^2,8 This can further contribute to patient distrust and ultimately a rift in the physician-patient relationship. ⁹ As BII is a largely patient-driven phenomenon, the quality of online information is of utmost importance. The objective of this study was to evaluate the quality of online BII video content using validated evaluation tools.

Methods

DISCERN and PEMAT Tools

DISCERN and Patient Education Materials Assessment Tool (PEMAT)^10,11 are tools that are frequently used in medical literature to assess the quality and reliability of videos.^12–15 DISCERN is a brief questionnaire designed to assist users in assessing the quality of health information. ¹² DISCERN examines the reliability (8 items), quality of information on treatment options (7 items), and the rater's overall impression of the publication (1 item). The DISCERN grading tool assesses 16 items on a scale of 1 to 5, where a rating of 5 indicates full agreement, 3 represents partial agreement, and 1 signifies a lack of agreement. ¹³ An average score was calculated for each question. The last question regarding the rater's overall impression was omitted due to irrelevance, in accordance with prior studies.^16–18 The total sum ranged from 15 to 75. The total DISCERN score was subdivided into five categories: very poor (15-26), poor (27-38), fair (39-50), good (51-62), and excellent (63-75), depending on overall final DISCERN score. The PEMAT tool consists of 17 items and two subscales. Thirteen items are related to comprehension and understandability, and four are related to actionability. Each item is assigned one of three ratings: agree (1 point), disagree (0 points), or not applicable (no point and noted as not applicable). Final scores were calculated using an average of raters' scores for each question and then calculating the PEMAT understandability and PEMAT actionability sub scores. PEMAT understandability was calculated as “(Total Points / Total Possible Points x 100)” using items 1, 3-5, 8-14 and 18-19, and PEMAT actionability was calculated using items 20-22 and 25. Scores of 70% or higher indicate that the information is easily understandable or actionable. ¹²

Data Collection

TikTok and YouTube were queried with the search phrase “breast implant illness” in November of 2022. TikTok and YouTube's proprietary search algorithms returned videos that were eligible for the study. The results were sorted based on relevance and view count. Non-English videos, videos that were not directly related to BII, videos that were deleted from the platform at the time of analysis, and YouTube videos under 60 s were excluded. The first 100 TikTok and 100 YouTube videos that fit these criteria were included. Video characteristics, DISCERN, and PEMAT grading were evaluated by three independent reviewers for TikTok (NR, EK, MA), and YouTube (ADVI, MA, and JJ). Video characteristics, including video link, account name, number of account subscribers, account type, date of upload, length of video (seconds), number of likes, number of comments, and the presence of overlying audio were collected from each video. The number of likes + comments + saves + shares for TikTok and number of likes + comments + views for YouTube were tallied as “digital currency” for the purposes of this study. Video topics were categorized as: physician education, physician experience, operation, patient education (videos uploaded with the intention of educating viewers on BII), patient experience, and/or advocacy.

Statistical Analysis

Data analyses were performed using the SPSS software (IBM SPSS 25.0; Armonk, NY, USA: IBM Corp). To investigate differences in video characteristics and total DISCERN and subscores of PEMAT scores between platforms, Mann-Whitney and Chi-square tests were conducted. Kruskal-Wallis Test, Mann-Whitney, and Spearman's Correlations were utilized to identify characteristics associated with DISCERN, PEMAT understandability, and PEMAT actionability scores. Statistically significant relationships were identified when the P-value < 0.05.

Results

Comparison of TikTok and YouTube Results

A total of 100 TikTok and 100 YouTube videos were retrieved that met inclusion criteria, and were subsequently analyzed (Table 1). All videos were uploaded between 2015 and 2022. The mean video duration was 36 s for TikTok and 890 s for YouTube. TikTok videos on BII had an average of 2087 likes, compared to an average of 441 likes on YouTube (P < 0.001). TikTok videos on BII had an average of 93 200 views, compared to an average of 22 969 views on YouTube (P < 0.001). Videos uploaded to YouTube had significantly more comments on average than those uploaded to TikTok (P < 0.05). Thirty-nine percent of YouTube videos featured health providers, compared to 22% on TikTok (P = 0.009). TikTok featured more videos with overlying audio than YouTube (P = 0.005). The majority of videos uploaded to both platforms were done so by non-healthcare providers (Figure 1). Plastic surgeons uploaded more videos to TikTok than to YouTube (Figure 1). No videos were uploaded by an academic institution on TikTok.

Table 1.

Video Characteristics.

Social media characteristics (median, IQR)
	TikTok	YouTube	P-value
Followers/Subscribers	42 900 (10 350-106 600)	82 600 (3935-446 500)	0.203
Duration (Seconds)	36 (12-64)	890 (417-1817)	<0.001*
Likes	2087 (405-26 150)	441 (196-1400)	<0.001*
Comments	66 (16-207)	106 (51-283)	0.041*
Views	93 200 (21 100-540 600)	22 969 (11 179-65 825)	<0.001*
Saves	81 (22-380)	–	–
Shares	43 (12-231)	–	–
Stitches	2 (2%)	–	–
Duets	6 (6%)	–	–
Video categories
Includes provider	22 (22%)	39 (39%)	0.009*
Includes patient	75 (75%)	84 (84%)	0.115
Physician education	0 (0%)	29 (29%)	<0.001*
Physician experience	0 (0%)	13 (13%)	<0.001*
Operation	0 (0%)	86 (86%)	<0.001*
Patient education	26 (26%)	63 (63%)	<0.001*
Patient experience	57 (57%)	82 (82%)	<0.001*
Advocacy	4 (4%)	81 (81%)	<0.001*
Self-promotion	13 (13%)	10 (10%)	0.506
Overlaying audio	57 (57%)	37 (37%)	0.005*
Questionnaires
Total DISCERN	24 (21-27.5)	36.8 (29.8-42.8)	<0.001*
PEMAT understandability	0.8 (0.7-0.8)	0.8 (0.7-0.9)	0.046
PEMAT actionability	0 (0-0.1)	0.5 (0.2-0.7)	<0.001

*P < 0.05.

PEMAT, Patient Education Materials Assessment Tool.

Figure 1.

Video upload by Account Type. Distribution of BII-related video uploads on TikTok and YouTube categorized by account type, highlighting a greater frequency by plastic surgeons on TikTok versus YouTube.

Total DISCERN score for videos uploaded to YouTube was significantly higher than that of videos uploaded to TikTok (P < 0.001) (Supplementary Table 1). Ninety-eight percent of videos uploaded to TikTok and 58% of videos uploaded to YouTube were characterized as being of poor or very poor quality by DISCERN criteria (Figure 2). Two (2%) of YouTube videos were characterized as being of excellent (DISCERN score of 63 or greater) quality. Zero TikTok videos were characterized as being of excellent quality. Median PEMAT understandability scores for YouTube and TikTok were 0.76 and 0.74, respectively. Median PEMAT actionability scores for YouTube and TikTok were 0.49 and 0, respectively. (Figure 3a and b).

Figure 2.

DISCERN Categorical Scores. Comparative quality assessment of BII videos using the DISCERN tool across TikTok and YouTube, indicating a predominant “poor” to “very poor” quality categorization for TikTok videos and limited “excellent” ratings on YouTube.

Figure 3.

(a) Range of PEMAT Understandability Scores—TikTok. PEMAT understandability scores for BII-related TikTok videos illustrate variability in content clarity. (b) Range of PEMAT accountability scores—YouTube. PEMAT actionability scores for BII videos on YouTube demonstrate variability in content effectiveness in guiding actionable insights for viewers. PEMAT, Patient Education Materials Assessment Tool.

TikTok

TikTok video duration, number of shares, and videos that were categorized as patient education were all significantly associated with a higher total DISCERN score (all P < 0.05) (Table 2). Increased video duration was also significantly associated with a higher PEMAT actionability score (P < 0.001). Videos in the patient experience category scored higher on PEMAT understandability (P = 0.004). A higher total DISCERN score was significantly associated with higher PEMAT actionability (P < 0.001). Videos in the self-promotion category had lower DISCERN and PEMAT understandability scores (P = 0.018).

Table 2.

Univariate Analysis of Video Characteristics—TikTok.

	Total DISCERN		Total DISCERN: categorized		PEMAT understandability		PEMAT actionability
	Correlation	P	Correlation	P	Correlation	P	Correlation	P
Followers/Subscribers	0.067	0.509	0.165	0.100	−0.044	0.663	0.010	0.921
Duration (Seconds)	0.650	<0.001*	0.538	<0.001*	0.031	0.760	.432	<0.001*
Likes	−0.109	0.281	−0.118	0.244	0.096	0.344	−0.105	0.301
Comments	0.081	0.433	−0.010	0.923	0.113	0.269	0.031	0.764
Views	−0.175	0.084	−0.126	0.218	−0.047	0.644	−0.155	0.127
Saves	0.074	0.471	0.057	0.578	0.132	0.199	0.094	0.363
Shares	0.249	0.015*	0.089	0.388	0.147	0.153	0.198	0.054
Digital currency (Likes + Comments + Saves + Shares)	−0.145	0.168	−0.156	0.138	0.067	0.528	−0.075	0.475
Account type	–	0.190	–	0.303	–	0.133	–	0.557
Includes provider	–	0.041	–	0.086	–	0.389	–	0.233
Includes patient	–	0.101	–	0.225	–	0.476	–	0.246
Physician education	–	–	–	–	–	–	–	–
Physician experience	–	–	–	–	–	–	–	–
Operation	–	–	–	–	–	–	–	–
Categorized as patient education	–	0.025*	–	0.121	–	0.136	–	0.043*
Categorized as patient experience	–	0.367	–	0.901	–	0.004*	–	0.082
Categorized as advocacy	–	0.641	–	0.875	–	0.888	–	0.173
Lacks self-promotion categorization	–	<0.001*	–	0.081	–	0.018*	–	0.386
Overlaying audio present	–	0.122	–	0.318	–	0.585	–	0.092
Total DISCERN	–	–	–	–	0.350	<0.001*	0.369	<0.001*
Total DISCERN: categorized	–	–	–	–	0.132	0.191	0.287	0.004*
PEMAT understandability	0.350	<0.001*	0.132	0.191	–	–	0.062	0.538
PEMAT actionability	0.369	<0.001*	0.287**	0.004*	0.062	0.538	–	–

*P < 0.05; Correlation = B.

PEMAT, Patient Education Materials Assessment Tool.

YouTube

YouTube videos that included a provider or a patient had higher total DISCERN and PEMAT understandability scores (all P < 0.05) (Table 3). Higher total DISCERN and PEMAT understandability scores were also seen in videos that discussed physician education, operation details, and patient experience (all P < 0.05). Videos that included patient education had higher total DISCERN, PEMAT understandability, and PEMAT actionability scores (all P < 0.05). Videos that were not categorized as self-promotion, of either a physician's services or a product that the video uploader is advertising, were also associated with a higher DISCERN score (P = 0.041). Total DISCERN score was positively associated with both PEMAT understandability and PEMAT actionability scores (P < 0.001).

Table 3.

Univariate Analysis of Video Characteristics—YouTube.

	Total DISCERN		Total DISCERN: categorized		PEMAT understandability		PEMAT actionability
	Correlation	P	Correlation	P	Correlation	P	Correlation	P
Followers/Subscribers	0.221	0.027	0.267	0.007*	0.031	0.760	−0.008	0.934
Duration (Seconds)	0.132468	0.189	0.167	0.096	0.182	0.070	0.124	0.218
Likes	0.077406	0.444	0.146	0.146	0.022	0.829	0.043	0.672
Comments	0.018122	0.859	0.065	0.526	−0.038	0.710	0.020	0.846
Views	−0.04797	0.636	0.031	0.763	−0.098	0.330	−0.027	0.791
Digital currency (Likes + Comments + Views)	−0.05029	0.621	0.029	0.778	−0.099	0.331	−0.021	0.833
Saves	–	–	–	–	–	–	–	–
Shares	–	–	–	–	–	–	–	–
Account type	–	0.009*	–	0.016*	–	0.072	–	0.557
Includes provider	–	<0.001*	–	<0.001*	–	0.002*	–	0.327
Includes patient	–	0.015*	–	0.019*	–	0.007*	–	0.108
Physician education	–	<0.001*	–	<0.001*	–	0.018*	–	0.433
Physician experience	–	0.001*	–	<0.001*	–	0.442	–	0.388
Operation	–	0.001*	–	0.004*	–	0.009*	–	0.094
Categorized as patient education	–	<0.001*	–	<0.001*	–	0.026*	–	0.003*
Categorized as patient experience	–	0.006*	–	0.012*	–	0.043*	–	0.395
Categorized as advocacy	–	0.065	–	0.073	–	0.259	–	0.124
Lacks self-promotion categorization	–	0.041*	–	0.027*	–	0.073*	–	0.539
Overlaying audio present	–	0.032*	–	0.045*	–	0.622	–	0.960
Total DISCERN	–	–	–	–	0.616	<0.001*	0.447	<0.001*
Total DISCERN: categorized	–	–	–	–	0.600	<0.001*	0.452	<0.001*
PEMAT understandability	0.616	<0.001*	0.600	<0.001*	1.000	<0.001*	0.464	<0.001*
PEMAT actionability	0.447	<0.001*	0.452	<0.001*	0.464	<0.001*	1.000	<0.001*

*P < 0.05.

PEMAT, Patient Education Materials Assessment Tool.

Discussion

Platform Discrepancy in BII Content

The present study found that more plastic surgeons uploaded BII-related videos to TikTok than to YouTube (Figure 1). This is in contrast to a prior study that showed that plastic surgeons uploaded breast augmentation content to YouTube more often than other available social media sites. ¹⁹ These findings underscore a significant shift in the digital content-sharing practices of plastic surgeons, which reflects the dynamic nature of social media engagement within the medical community and highlights the importance of adapting to emerging platforms and audience preferences. Importantly, the present study showed that the overall quality of online videos on BII was low, with TikTok scoring lower on both rating scales. Additionally, longer video duration was associated with higher quality scores for TikTok, consistent with previous findings. ¹² This may suggest that the longer form structure is more conducive to producing content that is high quality. Interestingly, while TikTok is a relatively newer social media platform, it had higher views on BII-related videos than YouTube. However, a significantly larger number of comments were seen on YouTube. This suggests that viewers may be exposed to videos on BII more readily on TikTok, but are more inclined to actively share their thoughts and reactions to BII-related content on YouTube. One study used commentary on Facebook to evaluate perceptions of BII online, showing that individuals are using comments to share experience, seek support, and express frustration on social media. ⁹ Providers may therefore seek to create educational content that merely raises awareness on BII to viewers on TikTok, with more in-depth details about BII reserved for YouTube.

Enhancing Understandability Through Patient Experience

Videos on TikTok that featured a patient's experience were significantly associated with a higher PEMAT understandability score. Prior research has found that COVID-19-related TikTok videos featuring a human character were more likely to be shared within the platform. ²⁰ These findings suggest that the human experience of disease resonates with viewers and may lend itself to a better understanding of the experience of BII as well.^21,22 Interestingly, videos that were characterized as “patient education” on YouTube had higher PEMAT actionability scores. This suggests that educational content on BII may influence viewers to take action. Given that these variables are associated with higher scores on a tool that is designed to evaluate the quality of health information, providers may consider developing content that prioritizes the patient's experience and knowledge about BII, rather than self-promotion of their services or videos that go into great surgical detail.

Opportunity for Major Healthcare Institutions

The present study revealed that a higher number of shares on TikTok was significantly associated with a higher DISCERN score (P < 0.05). This finding suggests that higher quality videos are shared more often. Higher quality content may therefore have an increased likelihood of reaching a larger audience. Physicians should understand the key components of a quality educational video on social media, to better tailor content to their audiences on these platforms. The authors of the present study propose a list of “Dos and “Don’ts” when addressing the topic of BII on social media (Table 4). More research is required to understand the exact components of an educational video on social media that contribute to increased visibility.

Table 4.

Dos and Don’ts When Addressing BII in Social Media.

Do

Include a medical provider

Focus on patient education, specifically referencing patient experiences

Utilize YouTube over TikTok

Don’t

Self-promote (advertise one's own services or products)

Include a “call to action” for the viewer

BII, Breast implant illness.

Importance of Transparent Healthcare Communication

A critical finding in the present study is that no videos on BII on TikTok were uploaded by a major healthcare institution. Providers and health advocates from major health institutions are thus faced with a unique opportunity to bolster their presence online. There are studies that investigate health institution transparency in their communication with vaccine skepticism, showing that high transparency increases patient trust long term, and less informative communication reinforces vaccine non-acceptance.^23,24 Therefore, health institutions have a major responsibility to accurately educate the general public in a transparent and accessible way. They should seize this opportunity to create content that informs and educates viewers and potential patients about the concept of BII, its risk factors, treatment options, and prognosis. Thus, as the importance of BII awareness among the plastic surgeon community grows, it is equally important to generate accurate and informative material about BII for the patient community online, specifically by health institutions.

Limitations

The results of the present study should be interpreted with respect to its limitations. Since both TikTok and YouTube have individualized algorithms for search terms and users, no other viewer will see the same list of videos when searching for BII on either platform. To mitigate this bias, new TikTok and YouTube accounts were created for the sole purpose of this study. Videos on these sites are also dynamic and may be deleted by the user after being chosen for analysis. This study is a cross-sectional analysis that only reflects video quality at a single point in time, and the results may change over time for the reasons stated previously. Additionally, observer bias is unavoidable due to the subjective evaluation instruments used in our study, which was mitigated by utilizing three independent graders.

Conclusion

To the authors’ knowledge, this is the first study that examines the current quality of online video content on BII. While TikTok and YouTube videos on BII have a low overall quality, a number of factors, including length, a focus on patient and physician experience and education, and the presence of a provider in the video contribute to a higher quality of videos on these platforms. A cohesive framework for designing high-quality educational content on the risks, diagnosis, and management of BII is critically needed in order to most accurately inform patients who have undergone implant-based breast reconstruction and/or augmentation.