YouTube as a Source of Information About the Posterior Cruciate Ligament: A Content-Quality and Reliability Analysis

Abstract

Purpose

The purpose of this study was to evaluate the reliability and educational content of YouTube videos concerning injuries to the posterior cruciate ligament (PCL) of the knee.

Methods

The first 50 videos specific to the PCL identified through the YouTube query posterior cruciate ligament were evaluated by a method of video selection demonstrated to be feasible in prior YouTube studies. Videos were classified by content and upload source. Video reliability was assessed using the Journal of the American Medical Association (JAMA) benchmark criteria (score range 0-5). Video educational content was assessed using the Global Quality Score (GQS) (range 0-4) and the PCL Score (PCLS) (score range 0-18). Analysis of variance was used to determine differences in video reliability and educational content quality based on video content and upload source. Multivariate linear regressions were used to identify predictors of video reliability and educational content quality.

Results

The mean number of views per video was 50,477.9 ± 15,036. Collectively, the 50 videos were viewed 14,141,285 times. Video content was classified primarily as information about disease (62.0%). The most common upload sources were physicians (24.0%) and nonphysician health care providers (26.0%). Significant between-group interactions were found between video source and the JAMA score, with physicians and medical sources having significantly higher mean JAMA scores (P = 0.037). Videos uploaded by physicians were an independent positive predictor of greater JAMA scores (β:1.27; P = 0.008). Videos uploaded by a medical source (β:2.06; P = 0.038) were an independent positive predictor of a greater GQS. There were no independent associations between video content category or upload source and the PCLS.

Conclusions

Videos concerning the PCL were frequently viewed on YouTube, but the educational quality and reliability of these videos were low.

Clinical Relevance

Physicians and health care providers treating PCL pathology should take the initiative to counsel patients about which outside resources are reliable to better inform patients about their treatment decisions. With regard to YouTube videos specifically, providers should caution their patients that this source of information may be unreliable.

Introduction

The internet is a rapidly growing source of health care information. An estimated 84% of adults used the internet on a regular basis in 2015, and 60% of adults reported having accessed health-related information online in the previous month.1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 Video-sharing sites such as YouTube have become popular resources for health information. More than 1.9 billion users visit YouTube each month, and it contains a growing library of health-education videos.²

Despite the common usage of the internet and YouTube to gain an understanding of medical issues, 82% of patients either never or only sometimes discuss this online information with their physicians.³ Having educated themselves based on online information prior to their office visits, patients may present with a presumptive diagnosis in mind and a preferred treatment based on this information. Indeed, 75% of patients with chronic conditions report that their last online search of the illness affected a decision about their treatment.¹ Considering that there are no quality-control measures or peer-review processes to ensure the accuracy of health-education videos on YouTube, patients may be subjected to inaccurate or misleading information.^4,5 The accuracy of patient education videos on YouTube has been investigated in the context of other orthopedic conditions and treatments, including hip and knee arthritis, femoroacetabular impingement, articular cartilage defects, and others.6, 7, 8, 9, 10, 11, 12, 13 Results suggest that these videos have, overall, poor educational content. (The quality of online educational content for other common orthopedic injuries is largely unknown.)

One such condition is posterior cruciate ligament (PCL) injuries. The incidence of PCL injuries has been reported to be approximately 3% in the general population, most commonly resulting from athletic injuries or motor vehicle accidents.14, 15, 16, 17 Considering that internet and social media use is more common in younger patients and in patients seeking treatment by sports orthopaedic surgeons, understanding the quality of online video content regarding PCL injuries is particularly important because the average age of patients with PCL injuries is 27.5 years.^17,18 To evaluate the reliability and educational content of YouTube videos concerning injuries to the PCL of the knee, the authors hypothesized that the quality and reliability of educational content on the PCL would be low.

Methods

YouTube Search

The YouTube online library (https://www.youtube.com/) was queried by using the key words posterior cruciate ligament on May 14, 2019. The first 50 videos based on this key term were recorded for evaluation; this has been reported to be a feasible method of video selection in the literature because this method of evaluation has been accepted in other peer-reviewed literature concerning orthopedic surgery.¹¹ Exclusion criteria included: (1) videos in non-English languages and audio-only soundtracks. In these cases, the next consecutive video that did not violate exclusion criteria was used.

Extracted Video Characteristics

For each video, the following variables were recorded for the final analysis: (1) title; (2) video duration; (3) number of views; (4) video source/uploader; (5) type of content; (6) days since upload; (7) view ratio (views/day); (8) number of likes; (9) number of dislikes; (10) like ratio (like*100/like + dislike); and (11) video power index (VPI). The VPI is a calculation derived from the following formula: like ratio*view ratio/100. This measurement is an index of video popularity based on the number of views and likes; it has been used in previous literature.¹¹

Video Upload Sources

Video sources/uploaders were categorized by the following: (1) academic (pertaining to authors/uploaders affiliated with research groups or universities/colleges); (2) physician (independent physicians or physician groups without research or university/college affiliations); (3) nonphysicians (health professionals other than licensed medical doctors); (4) athletic trainers; (5) medical sources (content or animations from health websites); (6) patients; and (7) commercial.

Video Content Categories

Content was categorized by the following: (1) exercise training (videos on rehabilitation and therapy for the PCL); (2) disease-specific information; (3) patient experience (personal accounts of PCL injuries); (4) surgical techniques or approaches; (5) nonsurgical management; and (6) advertisements.

Assessment of video reliability and educational content

The Journal of the American Medical Association (JAMA) benchmark criteria were used to assess video accuracy and reliability.¹⁹ The JAMA benchmark criteria (Table 1) are a nonspecific and objective tool consisting of 4 individual criteria that are identifiable in online videos and resources. To use this tool, an observer assigns 1 point for each criterion present in a video. A score of 4 indicates higher source accuracy and reliability, whereas a score of 0 indicates poor source accuracy and reliability. Although not validated, these criteria have been used extensively in previously literature to assess the reliability of online resources.^7,11

Table 1.

The Journal of the American Medical Association Benchmark Criteria¹⁹

Criteria	Description
Authorship	Author and contributor credentials and their affiliations should be provided.
Attribution	Clearly lists all copyright information and states references and sources for content.
Currency	Initial date of posted content and subsequent updates to content should be provided.
Disclosure	Conflicts of interest, funding, sponsorship, advertising, support, and video ownership should be fully disclosed.

Nonspecific educational content quality was assessed using the Global Quality Score (GQS), a nonvalidated but commonly used score that assesses the content quality of online resources. The GQS^11,20 evaluates the educational value of online content based on 5 criteria (Table 2). One point is assigned for each of the 5 identifiable criteria present in a video. The GQS has a maximum score of 5, which indicates high educational quality.

Table 2.

The Global Quality Score Criteria^11,20

Grading	Description of Quality
1	Poor quality; is unlikely of be to use for patient education.
2	Poor quality; is of limited use to patients because only some information is present.
3	Suboptimal quality and flow; is somewhat useful to patients; important topics are missing, some information is present.
4	Good quality and flow; useful to patients because most important topics are covered.
5	Excellent quality and flow; is highly useful to patients.

To evaluate the quality of educational content relating to information about the PCL specifically , we created the Posterior Cruciate Ligament-Specific Score (PCLS), which is composed of 20 items based on guidelines published by the American Academy of Orthopaedic Surgeons.²¹ The use of novel, nonvalidated orthopaedic topic-based instruments to assess online video educational quality has been demonstrated in previous literature.⁹ The PCLS specifically evaluates information about (1) common patient presentations and symptoms; (2) anatomy of the PCL; (3) diagnosis and evaluation of PCL pathologies; (4) treatment options; and (5) postoperative course and expectations (Table 3). One point is assigned for each item present, conferring a maximum possible score of 22, with higher scores indicating better PCL-specific educational quality.

Table 3.

Posterior Cruciate Ligament-Specific Score and Specific Educational Content

Patient presentation

Describes symptoms

Describes patient population

Information about PCL

Describes anatomy/function of PCL

Discusses regenerative potential of PCL

Discusses acute vs chronic tears

Diagnosis and evaluation

Mentions physical examination and findings

Discusses ability for stress x-rays to evaluate PCL tears

Discusses use of MRI as gold standard

Describes surgical candidates (young adults or those with symptoms impacting function/quality of life, excludes patients with significant osteoarthritis)

Describes surgical noncandidates (older adults with few symptoms)

Treatment

Mentions conservative treatment

Mentions diagnostic arthroscopy and other pathologies that may be addressed concomitantly

Describes single-bundle reconstruction

Describes double-bundle reconstruction

Postoperative course

Describes complications and outcomes

Mentions physical/weight-bearing restrictions

Mentions physical therapy

Outlines return-to-function timeline

PCL, posterior cruciate ligament.

Statistical Analysis

All statistical tests were performed using Stata version 15.1 (StataCorp, College Station, Texas). Descriptive statistics were used to quantify video characteristics as well as video reliability and quality scores. Continuous variables are presented as means with standard deviations and ranges. Categorical variables are presented as relative frequencies with percentages. One-way analysis of variance (ANOVA) tests (for normally distributed data) and Kruskal-Wallis tests (for non-normally distributed data) were used to determine whether the video reliability and quality differed based on video source and video content. Multivariate linear regression analyses were used to determine the influence of specific video characteristics on video reliability (JAMA score) and educational quality (GQS and PCLS). A 2-tailed P value of less than 0.05 was considered to indicate statistical significance.

Results

Finally, 58 studies were screened because the search elicited 8 studies regarding the anterior cruciate ligament. Of the 50 studies regarding the PCL that were included in the final analysis, the mean number of views per video was 50,477.9 ± 15,036. Collectively, the 50 videos were viewed 14,141,285 times. The maximum number of views was 1,026,666, and the minimum number of views was 121. Other video characteristics are described in Table 4.

Table 4.

Video Characteristics of the YouTube Videos Included

Characteristic	Mean	Standard Deviation	Minimum	Maximum
Video duration	281.5	258.4	41.0	1115.4
Views	50,477.9	150369.8	121.0	1,026,666
Days since upload	1385.3	1057.0	134.0	4,164.0
View ratio	27.8	42.8	0.14	246.6
Comments	14.2	34.9	0.0	201.0
Likes	135.6	233.8	0.0	1,300.0
Dislikes	7.9	11.8	0.0	54.0
Like ratio	91.0	10.1	50.0	100.0
Video power index	1,240.4	948.7	93.8	3708.0

Video content was classified primarily as information about disease (62.0%). Advertisements accounted for the lowest proportion of video content, at 2.0% (Fig. 1). The most common video sources were classified as physicians (24.0%) and nonphysician health care providers (26.0%) (Fig. 2). The mean JAMA score was 2.02, the GQS was 2.3, and the PCLS was 2.9. Intraobserver reliability for the JAMA score was 0.94 (0.91-0.96); for the GQS it was 0.91 (0.87-0.94), and for the PCLS it was 0.92 (0.89-0.95). Analysis of variance (Table 5) did not reveal significant between-group interactions between video content classification and the VPI, JAMA, GQS, or PCLS (P > 0.09). Significant between-group interactions were found between the video source and the JAMA score, with physicians and medical sources having higher mean JAMA scores. There were no significant associations between video source and the VPI, GQS or PCLS (P > 0.05).

Fig 1.

Donut chart depicting the relative frequency of video content for PCL-related YouTube videos. Each pattern represents a unique video content category as labeled adjacent to the chart. Percentages are derived from the first 50 identified videos in the current study.

Fig 2.

Donut chart depicting the relative frequency of video upload sources for PCL-related YouTube videos. Each pattern represents a unique video upload source category as labeled adjacent to the chart. Percentages are derived from the first 50 identified videos in the current study.

Table 5.

Mean Quality and Reliability Scores per Video Content and Video Source Variables

Grouping Variables	JAMA	GQS	PCLS
	Mean (SD)	Mean (SD)	Mean (SD)
Video content
Exercise training	1.3 (0.5)	2.5 (0.6)	2.8 (1.7)
Disease-specific	2.1 (1.2)	2.6 (1.1)	3.3 (2.9)
Patient experience	0.0 (0.0)	1.5 (0.7)	1.0 (0.4)
Surgical technique	2.6 (1.3)	1.7 (1.3)	2.4 (2.1)
Nonsurgical	1.6 (0.2)	1.5 (0.6)	3.0 (1.1)
Advertisement	1.0 (0.7)	2.0 (0.3)	2.1 (0.6)
Video source
Academic	1.0 (0.4)	1.0 (0.1)	0.7 (0.6)
Physician	2.6 (0.9)	2.1 (1.2)	3.0 (2.8)
Nonphysician	1.8 (1.4)	2.3 (0.9)	3.1 (2.6)
Trainer	1.9 (1.6)	3.0 (1.2)	2.4 (1.9)
Medical	2.8 (1.0)	2.9 (1.2)	3.7 (2.6)
Patient	0.0 (0.0)	2.0 (0.4)	2.0 (0.3)
Commercial	1.2 (0.4)	1.8 (0.8)	2.4 (1.7)

GQS, Global Quality Score; JAMA, Journal of the American Medical Association; PCLS, Posterior Cruciate Ligament Score; SD, standard deviation of the mean.

P value for video content between-group effects: JAMA = 0.098, GQS = 0.22, PCLS = 0.82.

P value for video source between-group effects: JAMA = 0.037, GQS = 0.11, PCLS = 0.75.

Multivariate linear regression analyses were performed to determine whether independent associations existed between video characteristics, video content category, video upload source, and video reliability and educational quality by using the JAMA score, the GQS and the PCLS. This analysis revealed that videos uploaded by physicians were a significant independent predictor of greater JAMA scores (β = 1.27; P = 0.008). Videos uploaded by medical sources (β = 2.06; P = 0.038) were the only independent predictor of a greater GQS. There were no independent associations among video characteristics, video content categories or video upload sources and the PCLS.

Discussion

The main findings of this study were that the first 50 YouTube videos regarding the PCL had accrued a large viewership on the date when the search was conducted. This study also found that both the video reliability and the quality of educational content in these videos was low. Furthermore, this study identified that the video upload source was an important determinant of video reliability and educational content because videos uploaded by physicians were independently associated with greater JAMA scores, whereas videos uploaded by medical sources were independently associated with greater GQSs.

The current study quantified that the mean number of views was 50,477.9 and that, collectively, all videos at the time of the analysis had been watched a total of 14,141,285 times. This is in accordance with other studies that have sought to analyze the quality and popularity of orthopaedic-related YouTube videos. Erdem et al.¹¹ found that the mean number of views of the first 50 YouTube videos concerning kyphosis was 131,644 and that the total viewing number was 6,582,221. Staunton et al.²² found that the mean number of views of the first 50 videos about scoliosis was 71,152. Other orthopaedic-based YouTube analyses have identified lower mean video-viewing rates as low as 2,651.5¹³ and 34,037 views per video,⁶ further supporting the idea that the PCL is a topic that reaches a large viewership.

Collectively, the videos analyzed in the current study provided unreliable and low-quality information regarding the PCL. Strikingly, the mean PCLS, a measure of PCL-specific educational quality for patients, was 2.9 out of a maximum possible score of 18. This finding is consistent with YouTube quality assessments of other orthopaedic topics. Indeed, Cassidy et al.⁷ evaluated 39 videos about the anterior cruciate ligament and determined that the mean JAMA score was 2.4 and that the mean anterior cruciate ligament-specific score was 5.5 (of a maximum possible score of 25). Orthopaedic-specific YouTube studies concerning the hip and spine have also concluded that YouTube videos provide low-quality information.^6,11,12 YouTube videos are currently unregulated, providing a potential explanation for the consistently low reliability and educational quality of video content in these studies. It would probably take significant cost and effort to instate an editorial process for medical content so that all videos are both reliable and provide high-quality educational content for viewers. Therefore, patients should be cautioned with regard to the current quality of information published on YouTube.

It is interesting that no significant associations were identified between the category of video content or particular video characteristics and the quality and reliability of YouTube videos about the PCL. This finding suggests that these characteristics were not valuable in creating higher quality YouTube videos for patient education. However, videos uploaded by physicians were independently associated with higher JAMA scores, and videos uploaded by medical sources were independently associated with greater GQSs, suggesting that upload source may have a more important role in providing higher quality videos for patients. For viewers to be educated accurately through YouTube, we believe that these videos should reflect the points published by the American Academy of Orthopaedic Surgeons, which we adapted into the PCLS because these data are peer reviewed and reliable. Encouraging the use of such materials with known high-quality content would allow patients to make better-informed decisions regarding their care and management options. To this end, we recommend that patients refer to reliable sources of online information regarding orthopaedic care such as the American Academy of Orthopaedic Surgeons and for physicians and health care providers to take more prominent roles in resource counseling for patients with PCL pathology because these sources were found to be statistically independent predictors of higher quality educational content and reliability.

The dissemination of accurate and reliable information regarding the PCL may also play a significant role in educating patients properly and optimizing outcomes. More accurate information about the PCL can have a positive impact on patient outcomes in a multifactorial manner. If patients are properly informed, they will be more likely to seek appropriate and timely treatment, and better outcomes may ensue. Additionally, because associated pathologies are common in PCL injuries,²³ a 6-fold increased risk of developing osteoarthritis can be expected if the injuries are not properly addressed.²⁴ This highlights the importance of ensuring that patients are properly informed so they can avoid the potential of additional morbidity.

Limitations

The results of the current study should be interpreted in the context of a few limitations. The videos analyzed were limited to the first 50 generated by the search query. Although this limits the generalizability of the findings to all PCL videos currently available on YouTube, viewers rarely explore more than the first few pages of a search for information,²⁵ making the methodology in this study strongly applicable to actual video searching patterns. Furthermore, the consistently low quality and reliability of the first 50 videos are likely to be good approximations of the remainder of the PCL videos, and this is consistent with other literature. Another limitation is the lack of validated instruments for reviewing online information. However, these tools continue to be the most commonly used in the literature evaluating online resources and YouTube quality and reliability in particular, and the interobserver reliability was found to be good to excellent for all 3 tools.

Conclusions

Videos concerning the PCL were frequently viewed on YouTube, but the educational quality and reliability of these videos were low.