Quality of healthcare information on YouTube: psoriatic arthritis

Abstract

Background

YouTube is an increasingly used platform for medical information. However, the validity of health-related information on psoriatic arthritis (PsA) on YouTube has not been determined.

Objective

The purpose of this study was to evaluate the accuracy and quality of YouTube videos concerning PsA.

Materials and methods

A YouTube search was performed on April 18, 2021, using the keyword “psoriatic arthritis.” Two independent raters accessed the content, source, and detailed characteristics of the included videos. The reliability and quality of the videos were analyzed using the modified DISCERN score, Journal of the American Medical Association benchmark criteria score, and global quality scale score.

Results

Of the 200 videos screened, 155 were included in the study after applying the exclusion criteria. A total of 132 (85.2%) videos revealed useful information about PsA, whereas the remaining 23 (14.8%) were misleading. Video interaction parameters including the median number of views, views per day, likes, dislikes, and comments demonstrated no significant difference between the two groups. The videos posted by universities and professional organizations displayed the highest reliability and quality scores.

Conclusion

The majority of YouTube videos on PsA contained useful information. However, physicians should alert patients to the possibility of misinformation and non-validated sources. Professional organizations in the field of rheumatology, such as the American College of Rheumatology and European League Against Rheumatism, should consider collaborating with YouTube to deliver high-quality content.

Introduction

PsA is a common systemic inflammatory disease included in the spondyloarthropathy spectrum, which involves both the peripheral joints and the axial skeleton [1, 2]. PsA usually develops in those with psoriasis, but in about 20% of patients, it precedes the onset of psoriasis [3]. Up to 30% of patients with psoriasis develop PsA at some time in their lives [4, 5]. Musculoskeletal manifestations vary from enthesitis to severe involvement of peripheral joints, called arthritis mutilans [3]. Going beyond joint and skin involvement and extra-articular manifestations, such as uveitis and inflammatory bowel disease, PsA is also associated with several comorbidities, including metabolic syndrome, obesity, and cardiovascular disease [2, 5]. Although PsA was initially considered as a relatively benign disorder, registry data suggest that the progressive and devastating character of the disease cause functional disability and impair quality of life [1, 4].

Seeking online health information has dramatically increased in recent years [6], with nearly 80% of internet users searching for medical information using online platforms [7]. Today, social media is widely used to share health-related information [8]. Especially patients with chronic diseases trust internet-based sources, and 75% are reported to base their disease-related decisions on information they have obtained from the internet [7]. YouTube is the most popular video-sharing website worldwide, receiving more than 1 billion hours of views every day, including 30 million medical videos [8, 9]. However, there have been concerns about the accuracy and quality of health-related information in YouTube videos due to the minimal regulation of the enormous volume of information [7, 10, 11].

Physicians should be aware of the quality of content presented on YouTube, given that patients are increasingly using this source and it affects their decision-making processes [12]. Previous studies have found varying accuracy and quality of YouTube content on rheumatic diseases such as rheumatoid arthritis [13], spondyloarthritis [14], systemic lupus erythematosus (SLE) [15], Sjögren’s syndrome [16], and fibromyalgia [17]. In addition, several studies have been performed to investigate the quality of information about psoriasis on YouTube [7, 18]. However, to our knowledge, there is no study that has analyzed the quality of YouTube videos pertaining to PsA. Therefore, the purpose of the current study was to analyze the quality of YouTube videos on PsA.

Materials and methods

Search procedure

A YouTube search was performed on April 18, 2021, using the keyword “psoriatic arthritis.” The “incognito” mode of Google Chrome (Google Inc., CA, USA) was used for browsing YouTube. Browsing history was deleted prior to the search to prevent results from being influenced by previous online activity. YouTube’s default relevance mode was used to mimic most consumers’ search pattern. It is known that most viewers do not go beyond the first few pages of search results; therefore, the first 200 videos regarding PsA were listed, similar to previous studies [13, 17, 19]. The selected videos were saved in a playlist in YouTube Library for further analysis. Duplicates, videos that were not related to PsA, videos presented in a language other than English, those lasting more than 1 h, and music and advertisement videos were excluded from the study. After applying the exclusion criteria, the remaining 155 videos were included in the study (Fig. 1).

Fig. 1.

Flowchart demonstrating video selection

Data review

Viewer engagement parameters, including the number of views, likes, dislikes, and comments were documented by the authors. The remaining characteristics of the videos, namely upload date, video length, target audience, and video source were also noted. Since the upload date and online duration of the videos varied, the number of views per day was calculated for a strong analysis. The content of the videos (epidemiology, pathogenesis, clinical characteristics, diagnosis, and treatment) was also recorded. Two rheumatologists (MEO and OZ) blindly analyzed each video. Any discrepancy between the raters was resolved by reevaluation and consensus.

Video source and targeted audience

The video sources were grouped into five categories as universities and professional organizations, physicians, health information websites, TV programs, and independent users. The primary target audience of the videos was classified as healthcare professionals and patients.

Usefulness

The videos were deemed useful if the information they contained was completely scientifically accurate, and misleading if they contained any inaccurate information regarding any aspect of PsA, in accordance with the guidelines of professional organizations in the field of rheumatology such as the American College of Rheumatology (ACR) and European League Against Rheumatism (EULAR). Previous studies have followed the same path [13, 20].

Video popularity

To evaluate video popularity, the video power index (VPI) was calculated using a formula including the like ratio [(likes × 100) / (likes + dislikes)] and view ratio (views per day): VPI = [(like ratio * view ratio) / 100] [21–23].

Quality and reliability evaluation

The Global Quality Scale (GQS), which was first developed by Bernard et al. [24], was used to evaluate the quality of the videos. This is a five-point scale in which higher scores indicate greater quality. The Journal of the American Medical Association (JAMA) benchmark criteria to determine authorship, attribution, disclosure, and currency were also used to assess quality. Each of these criteria was rated with 1 point, with the maximum score being 4 points. Lastly, a five-point modified DISCERN score was obtained to evaluate reliability. Each question in this instrument is scored 1 point, giving a maximum score of 5 points, and higher scores indicate better reliability. Several studies on YouTube videos have used GQS [13, 22], JAMA [17, 25], and modified DISCERN [14, 15] for similar purposes. Questions of the GQS [22], JAMA [25], and modified DISCERN [15] tools are shown in Table 1.

Table 1.

Assessment tools for reliability and quality

Global Quality Scale (Select the appropriate one)

1. Poor quality, poor flow of the video, most information missing, not useful for patients. 2. Generally poor quality, poor flow, some information given but many important topics missing, of very limited use to patients. 3. Moderate quality, suboptimal flow, some information is adequately discussed but other information inadequately discussed, somewhat useful for patients. 4. Good quality, good flow, most of the relevant information is listed, but some topics not covered, useful for patients. 5. Excellent quality and flow, very useful for patients

The Journal of the American Medical Association benchmark criteria (Each of the criteria was rated as 1 point)

1. Authorship: Author and contributor credentials and their affiliations should be provided. 2. Attribution: References and resources for all content should be listed clearly, and all relevant copyright information noted. 3. Disclosure: Website “ownership” should be prominently and fully disclosed, as should any sponsorship, advertising, underwriting, commercial funding arrangements or support, or potential conflicts of interest. 4. Currency: Dates of uploaded content and subsequent updates should be provided

Modified DISCERN tool (1 point for every “yes,” 0 points for “no”)

1. Is the aim of the video clear and understandable? 2. Are reliable sources of information used? (i.e., publication cited, speaker is rheumatologist) 3. Is the information presented balanced and unbiased? 4. Are additional sources of information listed for patient reference? 5. Are areas of controversy/uncertainty mentioned?

Statistical analyses

The normality of data was examined using the Shapiro–Wilk test. Continuous variables were stated as median and interquartile range, while categorical variables were presented as counts and percentages. The Mann–Whitney U-test was carried out for two groups and the Kruskal–Wallis test for more than two groups to compare non-normally distributed data. The Mann–Whitney U-test was also used for pairwise comparisons following the Kruskal–Wallis test to determine the groups that led to significant differences. Correction for the p-value was done automatically by SPSS (IBM SPSS Statistics for Windows; IBM Corp, Armonk, NY, USA). Fischer’s exact test was used for categorical variables to assess the differences between the two groups. Inter-rater agreement was evaluated using Cohen’s kappa coefficient. A p-value of < 0.05 was set as significant. SPSS version 22 was used for all statistical analyses.

Results

Of the total 200 videos screened, 155 met the inclusion criteria and were included in the study. For the 155 videos analyzed, the total number of days on YouTube was 163,517, the total video duration was 23.16 h, and the total view count was 794,999. The first video was uploaded on March 30, 2011, and the last on March 26, 2021. The number of uploaded videos generally increased over the years. More than half of the videos (52%) were uploaded within the last 3 years. While 2019 represented the year with the most video uploads, there was a partial decrease in 2020 (Fig. 2). Cohen’s kappa coefficient for inter-rater agreement was 0.783 for the DISCERN score, indicating a substantial agreement, and 0.862 and 0.824 for the JAMA and GQS scores, respectively, indicating near-perfect agreement for both.

Fig. 2.

Distribution of the analyzed videos over the years. Note: samples for 2021 only gathered until March 2021

Target audience and video content

Of the total 155 videos, 74.8% targeted patients and 25.2% targeted healthcare professionals. Since more than one topic was mentioned in 53 videos, each topic was recorded separately. Over 60% of the videos contained information about the treatment of PsA, followed by clinical features (n = 89, 56.8%), diagnosis (n = 37, 23.9%), pathogenesis (n = 24, 15.5%), and epidemiology (n = 24, 15.5%).

General characteristics of the videos

For the 155 videos analyzed, the median view count was 1047 (range 271–4213), the number of views per day was 1.80 (range 0.49–5.67), and the number of likes was 10 (range 2–49). The median duration of the videos was 5.00 (range 2.22–9.12) minutes. The median VPI was 1.66 (range 0.49–5.44). The median reliability score measured with the modified DISCERN was 3 (range 2–4). Concerning quality, the median score was 3 (range 2–3) for JAMA and 4 (range 2–4) for GQS. Baseline characteristics of the analyzed videos were demonstrated in Table 2.

Table 2.

Baseline features of the analyzed videos on PsA

Video data	(n = 155)
Number of days on YouTube	776 [(6–3655) / (444–1491)]
Number of views	1047 [(16–97,957) / (271–4213)]
View per day	1.80 [(0.04–188.38) / (0.49–5.67)]
Number of likes	10 [(1–2600) / (2–49)]
Number of dislikes	0 [(0–52) / (0–2)]
Number of comments	0 [(0–175) / (0–5)]
Duration (minute)	5.00 [(0.23–54.53) / (2.22–9.12)]
VPI	1.66 [(0.04–184.69) / (0.49–5.44)]
Modified DISCERN score	3 [(1–5) / (2–4)]
JAMA score	3 [(1–4) / (2–3)]
GQS score	4 [(1–5) / (3–4)]

Data presented as median [(minimum–maximum) / (interquartile range)] values

PsA psoriatic arthritis, JAMA Journal of the American Medical Association, benchmark criteria, GQS Global Quality Scale, VPI video power index

Assessment of usefulness

Of the total 155 videos evaluated, 132 (85.2%) were categorized as useful, while 23 (14.8%) contained misleading information. There was no significant difference detected between the two groups in terms of the median number of days on YouTube. Video metrics, including the median number of views, views per day, likes, dislikes, and comments did not significantly differ between the two groups (p = 0.926, p = 0.594, p = 0.367, p = 0.373, and p = 0.189, respectively). The median length was significantly higher in the useful videos than in the misleading ones (p = 0.028). The VPI score, reflecting the popularity of the videos, showed no significant difference between the useful and misleading video groups (p = 0.587). The modified DISCERN, JAMA, and GQS scores were significantly higher in the useful videos, indicating better reliability and quality. Independent users constituted the source that posted the highest percentage of misleading videos (6/10, 60%). In contrast, all the videos uploaded by universities and professional organizations were determined to be useful. Independent users were the source that contributed only 3% of useful videos, whereas they uploaded 26% of misleading videos. The majority of videos were related to treatment in the useful group (65.2%) and clinical signs in the misleading group (56.5%). In terms of the target audience, all of the misleading videos were aimed at patients. Of the total 19 videos regarding patients’ personal experience, five (26.3%) contained misleading information (Table 3).

Table 3.

Detailed characteristics of YouTube videos on PsA based on usefulness

	Useful	Misleading	P-value
	132 (85.2%)	23 (14.8%)
Video metrics
Days on YouTube	773.50 (451.75–1491)	849 (414–1512)	0.641
Number of views	1041 (274.25–4426.25)	1367 (183–4213)	0.926
View per day	1.73 (0.59–5.91)	2.84 (0.26–5.67)	0.594
Number of likes	10 (2–45.25)	13 (3–94)	0.367
Number of dislikes	0 (0–2)	1 (0–4)	0.373
Number of comments	0 (0–3)	2 (0–11)	0.189
Duration (minutes)	5.02 (2.23–9.98)	3.19 (1.37–5.22)	0.028*
VPI	1.62 (0.55–5.49)	2.41 (0.20–544)	0.587
Reliability and quality scores
mDISCERN	4 (3–4)	2 (1–3)	< 0.001*
JAMA	3 (2–3)	1 (1–2)	< 0.001*
GQS	4 (3–5)	2 (1–3)	< 0.001*
Source, n (%)
Universities and professional organizations	20 (15.2%)	0 (0%)	N/A
Physicians	39 (29.6%)	3 (13%)	N/A
Health information websites	60 (45.5%)	11 (47.8%)	N/A
TV program	9 (6.8%)	3 (13%)	N/A
Independent users	4 (3%)	6 (26%)	N/A
Video content
Epidemiology	22 (16.7%)	2 (8.7%)	N/A
Pathogenesis	22 (16.7%)	2 (8.7%)	N/A
Clinic	76 (57.6%)	13 (56.5%)	N/A
Diagnose	35 (26.5%)	2 (8.7%)	N/A
Treatment	86 (65.2%)	10 (43.5%)	N/A
Target audience
Patients	93 (70.5%)	23 (100%)	N/A
Healthcare professionals	39 (29.5%)	0 (0%)	N/A
Patient personal experience	14 (73.7%)	5 (26.3%)	N/A

Data presented as n (%) or median (interquartile range)

Pairwise comparisons were performed between the “useful” and “misleading” videos using the Man–Whitney U-test

*P < 0.05 was accepted as significant

PsA psoriatic arthritis, VPI video power index, mDISCERN modified DISCERN score, JAMA Journal of the American Medical Association benchmark criteria score, GQS Global Quality Scale score, N/A not applicable

We also conducted an extra analysis to assess how the top 10 high-performer videos (in regard to the number of views) were separated into useful and misleading, and detected one misleading video (10%). In addition, we compared the top 10 high-performer videos with the remaining 145 videos and detected no significant difference concerning the number of useful and misleading videos between the two groups (p = 0.546), suggesting that the videos with a high view count may not serve as a guide for viewers to distinguish between useful and misleading videos.

Evaluation of reliability, quality, and popularity according to video source

In view of reliability and quality, the median DISCERN, JAMA, and GQS scores revealed a statistically significant difference between the video sources (p < 0.001 for all). The videos posted by universities and professional organizations had the highest median scores concerning the modified DISCERN (5, range 4–5), JAMA (4, range 3–4), and GQS (5, range 4.25–5), followed by physicians (4, range 3–4; 3, range 2–3; and 4, range 3–5, respectively). As expected, the videos uploaded by universities and professional organizations had significantly higher reliability and quality scores compared to all the remaining sources. The median DISCERN score of the videos uploaded by universities and professional organizations was significantly higher than those of the videos posted by physicians, health information websites, TV programs, and independent users (p = 0.024, p < 0.001, p < 0.001, and p < 0.001, respectively). The median JAMA and GQS score of the videos posted by universities and professional organizations were also higher compared to those uploaded by physicians (p = 0.010 and 0.034, respectively) and the remaining sources (p < 0.001 for all). The videos provided by independent users displayed the lowest reliability and quality scores. The VPI scores, assessing popularity, did not significantly differ between the upload sources (Table 4).

Table 4.

Comparison of the reliability, quality, and popularity scores according to video sources

Reliability, quality and VPI scores	Universities and professional organizations	Physicians	Health-related websites	TV program	Independent users	P-value
	20 (12.9%)	42 (27.1%)	71 (45.8%)	12 (7.7%)	10 (6.4%)
mDISCERN	5 (4–5)	4 (3–4)	3 (2–4)	3 (2–3)	1.5 (1–2)	< 0.001*
JAMA	4 (3–4)	3 (2–3)	2 (1–3)	2 (1–2.75)	1 (1–2)	< 0.001*
GQS	5 (4.25–5)	4 (3–5)	4 (2–4)	3 (2–3.75)	2 (1–3)	< 0.001*
VPI	1.37 (0.55–7.25)	2.72 (0.69–5.98)	0.95 (0.31–4.61)	2.11 (0.36–4.82)	5.15 (2.79–5.15)	0.114

Data presented as n (%) or median (interquartile range)

Kruskal–Wallis test: *p < 0.05 significant difference between the groups

mDISCERN modified DISCERN score, JAMA Journal of the American Medical Association benchmark criteria score, GQS Global Quality Scale score, VPI video power index score

Discussion

In recent years, social media has become a trendy source of medical information [26]. Especially patients with chronic conditions increasingly trust online healthcare information sources [10], and most patients diagnosed with a rheumatic disease seek online medical information to manage their diseases [27].

The database obtained in the current study included 155 videos with approximately 1 million total views and a total duration of 23.16 h, suggesting that YouTube is commonly utilized when seeking medical information regarding PsA. In view of video metrics, a cumulative number of 9901 likes, 448 dislikes, and 1185 comments was recorded. Compared to the number of views, the rate of video interaction parameters (likes + dislikes + comments) was quite low (1.15%) in the current study. This ratio was 1.24% in a study performed by Elangovan et al. [14] on SLE, and 0.98% in another study performed by Kumar et al. [28] on hypertension, similar to our study. These results suggest that viewers mostly prefer just to watch passively and hardly interact with the platform. The videos covered a wide variety of issues related to PsA, mainly focusing on the clinical signs and treatment of the disease.

While information on online platforms is sometimes useful, it is not always appropriate [28]. In our study, 132 (85.2%) contained scientifically accurate information and were categorized as useful, whereas 23 (14.8%) contained misleading information. Previous studies investigating YouTube as a source of health-related information on several diseases have reported various results regarding misleading information. Mueller et al. [7] reported misinformation 63% on psoriasis, Kumar et al. [29] 33% on hypertension, and Pathak et al. [30] 26% on Ebola virus disease. Considering rheumatic diseases, Singh et al. [13] determined that 30% of videos contained misleading information on rheumatoid arthritis, while Ng et al. [15] reported it to be 16% for SLE and Elongovan et al. [14] 14% for spondyloarthritis, similar to our study. Spread of misinformation may result in patients following inappropriate medical advice or experiencing excessive anxiety [28]. On January 26, 2021, a letter from YouTube CEO, Susan Wojcicki, stated that more than 500,000 videos containing misinformation associated with COVID-19 were removed from the platform according to their new health policies [31]. YouTube should also consider expanding this misinformation policy to other health topics, including rheumatology.

In this study, as expected, the reliability and quality of the useful videos were much better than those of the misleading videos, which is in agreement with previous studies [15, 32]. The duration of the useful videos was significantly longer, as also determined by Moon et al. for COVID-19 videos on YouTube [33]. This may be because useful videos are presented from a more comprehensive and detailed perspective, resulting in a longer duration. Video interaction parameters, including likes, dislikes, and comments showed no significant difference between the useful and misleading videos. In addition, the useful and misleading videos did not significantly differ in popularity, similar to the results of previous research [13, 33]. The literature contains conflicting results concerning whether interaction parameters are similar between useful and misleading videos [13, 16]. While some authors reported higher interaction scores for useful videos [33], others indicated higher values for misleading videos [19]. As video interaction parameters are poor determinants of accuracy and usefulness, they should not be used as a guide to assess usefulness. During consultations, physicians should consider directing patients to reliable sources of online medical information rather than relying on video metrics.

In the current study, it was observed that the majority of the videos had been uploaded by health information websites (45.8%), followed by physicians (27.1%) and universities and professional organizations (12.9%). In view of reliability and quality, universities and professional organizations had higher modified DISCERN and JAMA scores, which significantly differed as compared to the other upload sources, except physicians. However, the relatively lower number of videos posted by universities and professional organizations was noteworthy. This analysis suggests that universities and professional organizations should increase the number of videos they produce. In addition, the videos with the lowest reliability and quality scores had been mostly posted by independent users, which is in agreement with the studies of Li et al. [32] and Khatri et al. [20]. We suggest that YouTube content creators use validating scoring systems to make better-quality and unbiased videos.

It is essential for viewers to access credible, evidence-based accurate information provided by trusted sources. Recently, YouTube has announced that they have established a team under the leadership of Garth Graham (who has worked as a cardiologist and as a researcher educating people about public health over the past 20 years, and as Deputy Assistant Secretary at the U.S. Department of Health and Human Services prior to joining YouTube) to make high-quality health content available for viewers in collaboration with some of the most respected organizations and clinicians. They have also initiated novel collaborations with organizations such as the American Public Health Association, the Mayo Clinic, and the Cleveland Clinic, and expressed their willingness to establish new health partnerships to help offer trustworthy medical information on YouTube [34]. Professional organizations, especially ACR and EULAR, should consider participating in this collaboration in order to deliver better-quality videos in the field of rheumatology to the target audience.

Conclusion

The results of our study demonstrated that the majority of the YouTube videos contained useful information regarding PsA. Given patients’ limited consultation time with their physicians, YouTube can be recommended as a complementary source to receive additional information on PsA. However, rheumatologists should warn their patients about the possibility of misinformation and refer them to trusted sources available on YouTube. We recommend universities and professional organizations to share more videos on rheumatic diseases on this platform. Considering that YouTube has begun to make partnerships with reputable healthcare organizations to create high-quality health content, professional rheumatology organizations, such as ACR and EULAR, should also consider collaborating with YouTube for this purpose.

Declarations

Conflict of interest

M. E. Onder and O. Zengin declare that they have no competing interests.

For this article no studies with human participants or animals were performed by any of the authors. All studies performed were in accordance with the ethical standards indicated in each case.