Radionuclide Therapy Videos on YouTube as An Educational Material: Has the COVID-19 Pandemic Changed the Quality, Usefulness, and Interaction Features

Abstract

Introduction

Current treatment approach aims to achieve greater efficacy with fewer side effects, by targeted cancer therapy as much as possible. Radionuclide therapy is a modality that uses cancer theranostics and is increasingly applied for various cancers as a targeted therapy. YouTube is a preferred tool for obtaining medical information from the internet. This study aims to determine the content quality, level of interaction and usefulness as education material of radionuclide therapy YouTube videos and to reveal the impact of the COVID-19 process on these parameters.

Materials and Methods

The keywords were searched on YouTube on August 25, 2018, and May 10, 2021. After removing duplicate and excluded videos, all remaining videos were scored and coded.

Results

Majority of the videos were useful educational material. Most of them were high quality. Popularity markers were unrelated to quality level. After COVID, the power index of videos with high JAMA scores increased. The COVID-19 pandemic did not have a negative effect on video features; the quality of the content increased even more after the pandemic.

Conclusion

Radionuclide therapy YouTube videos have high-quality content and provide useful education material. The popularity is independent of the content quality. During the pandemic, video quality and usefulness characteristics did not change, while the visibility is increased. We consider YouTube to be an appropriate educational material for patients and healthcare professionals to gain basic knowledge of radionuclide therapy. The Covıd-19 pandemic highlighted the power of radionuclide therapy YouTube videos as an educational material.

Introduction

Even a lot of treatment strategies has been developed, cancer is still fatal and continues to adversely affect people’s life. The current treatment approach aims to achieve greater efficacy with fewer side effects, by personalizing the treatment with targeted cancer therapies as much as possible. Radionuclide therapy (RNT), which is an effective method of targeted cancer therapy, is increasingly used in the personalized treatment of various cancers such as common cancer types such as neuroendocrine tumors, prostate cancer, and thyroid cancer. Ease of access and increase in digital content have made the internet an easily accessible and low-cost resource for accessing information. Access to medical information is no exception. Many patients consult internet sources about their illness and treatment options [1, 2]. YouTube is a digital sharing platform and the largest online video streaming source [3, 4]. It is also the second most visited website with 5 billion daily views worldwide [5]. YouTube provides visual data along with information. It is a preferred tool when obtaining medical information from the internet during the pandemic, as it was before COVID-19 [6, 8]. Physicians and patients seeking education material for medical issues have started to turn to YouTube for this purpose [5–11]. Health anxiety caused by COVID-19 has reduced live interaction in all areas of life and increased people’s search for digital information [10]. This digitalization trend has increased the importance of the content quality of sources. With the onset of the COVID-19 pandemic, scientific congresses and meetings related to RNT have become online, as in all other fields. The digital records created in this process are shared in real time or later in the form of video. Although a growing body of research has been published on the quality and metric characterization of medical content on YouTube, studies on RNT videos are limited. Several studies have been published recently on radioactive iodine therapy videos. As far as we know, there are no studies on the content evaluation of the videos explaining the RNT concept in general and their value as an educational material. In addition, there is no study investigating the change in medical video features during the COVID process. The aim of this study is to determine the overall content quality, level of interaction with viewers, and usefulness as an education material of YouTube videos about RNT, and then to reveal the impact of the COVID-19 process on these parameters.

Materials and Methods

Video Selection

YouTube was searched on August 25, 2018, using the following terms “radionuclide therapy” and “targeted therapy).” The 100 top-viewed YouTube videos for each terms (totally 200 videos) watched by two experienced nuclear medicine physicians and one irrelevant clinician to avoid vocational biases. In order to detect the change in the COVID-19 pandemic, the same process was repeated by the same people on May 10, 2021. Only videos in English (with expression language or subtitles) were included to study. Videos that were non-English, unrelated to searching terms, and unrelated “radionuclide therapy” were excluded. Videos with intro only are also excluded. Duplicate videos were counted as one video. Each video rank in the YouTube list of results was recorded by viewers. The videos appeared under both search terms, has been used by rank they appeared closest to the top of the ranking list, and deleted from the other search results. After removing the duplicate and excluding videos and using the inclusion criteria, 55 videos in the first evaluation and 48 in the second evaluation remained for analysis. All videos were scored and coded bye viewers.

Scoring Process

For each video, the following information was determined: title, authors, number of views (NOV), number of comments (NOC), number of days on YouTube (NOD), like ratio [LR = like × 100/(like + dislike)], and view ratio (VR = NOV/NOD). Viewers coded videos independently as excellent, inadequate but useful or misleading based the information in the content. The groups “excellent” and “inadequate but useful” were considered “useful.” The videos were divided as personal (P) and educational (E), based to their definitions made on their content and source. The popularity of the videos were evaluated with a video power index (VPI) defined by Erdem and Karaca [12].

The visual quality, educational quality, and accuracy of the source of information were measured using the JAMA score [13] and Global Quality Score (GQS) [14]. The videos were divided into three groups according to their JAMA and GQS scores as low quality (JAMA 1–2 and GQS 1–2), medium quality (JAMA 3 and GQS 3), and high quality (JAMA 4 and GQS 4–5). When a video was classified differently by viewers, it was revisited and discussed to reach a consensus. The scoring period ended when the three viewers reached agreement for all scores, codes, and code definitions.

Statistical Analysis

Data were analyzed by IBM SPSS Statistics version 22. Groups were compared in terms of usefulness, length, JAMA score, GQS score, and VPI. Spearman’s rho correlation analysis was used in assessing the correlation between the parameters. Pearson chi-square tests were used for selected paired comparisons. The conformity of the data to the normal distribution was checked with the Shapiro–Wilk test. The t-test was used to compare the means of data with normal distribution, and the Mann–Whitney u test was used to compare data that did not fit. Homogeneity of variances was determined according to Levene’s test. The significance level was set at p < 0.05.

Results

Aim and Source

Most of the videos [pre-COVID n = 49/55 (89.1%); post-COVID n = 45/48 (93.8%)] were uploaded by non-commercial sources (Fig. 1). All of the commercial videos were educational.

Fig. 1.

Distribution of video sources at pre-COVID and post-COVID situation

In both the pre-COVID and post-COVID era, educational videos were about 7 times more than personal videos (educational content in all videos was 87.3% before COVID and 91.7% after COVID; personal content was 12.7% before COVID and after COVID 8.3%).

Useful and Misleading Content Distribution

The majority of RNT videos were identified as useful in both the pre-COVID and post-COVID era (87.3% and 97.9%, respectively). The usefulness rate of educational videos was significantly higher than personal videos in both period (pre-COVID 97.9% vs 85.7% and post-COVID 100% vs 71.4%; CI 95%, p: 0.001) (Fig. 2).

Fig. 2.

Distribution of educational and personal videos by useful and misleading content

Content Quality

High-quality videos (JAMA 4 and GQS 4–5) dominated the group with both educational and personal content (Fig. 3).

Fig. 3.

Distribution of videos with educational and personal content according to JAMA (upper row) and GQS (lower row) scores before and after COVID

Educational videos had higher JAMA and GQS scores than personal videos (p: 0.005 for JAMA; p: 0.008 for GQS; CI 95%). Likewise, useful videos had higher GQS and JAMA scores than those with misleading content (CI 95%, p: 0.02). Educational videos were longer in length than personal ones in both period (p = 0.001; mean length 18.6 vs. 21.1 min).

Popularity and Interaction Metrics

Number of Views and Comments

Mean NOV value was 8032.2 (± 24,216.5), and NOC value was 2.24 (± 6.5) before COVID-19. In this period, the rate of videos with more than 10,000 views was only 12.7%. The videos (74.6%) were watched by less than 1000 people and 58.2% by less than 500 people. During the pandemic, 27% of E videos (13 of 48 videos) were closed to comments. Among those allowing comments, the mean NOV value was 5913.6 (± 28,387.9) and the mean NOC value was 1.88 (± 6.06). The rate of videos watched less than 1000 after COVID was 68.8%, and the rate of those watched less than 500 was 48%. Only 6.3% of the videos had more than 10,000 views. There was no difference between video viewing rates before and after COVID (CI 95%, p: 0.3). However, the rate of videos with 1000–10,000 views increased after the pandemic (12.7% vs. 25.1%) (Table 1).

Table 1.

Distribution of the number of views of all videos (upper) and the number of comments on videos open to comments (bottom), before and after COVID

Number of views	Frequency		Percent
	Pre-COVID	Post-COVID	Pre-COVID	Post-COVID
0–199	23	15	41.8	31.3
200–499	9	8	16.4	16.7
500–999	9	10	16.4	20.8
1000–1499	2	5	3.6	10.4
1500–1999	1	3	1.8	6.3
2000–4999	3	2	5.5	4.2
5000–9999	1	2	108	4.2
More than 10,000	7	3	12.7	6.3
Total	55	35	100	100
Number of comments	Frequency		Percent
	Pre-COVID	Post-COVID	Pre-COVID	Post-COVID
0–4	49	29	89.1	82.9
5–9	2	4	3.6	11.4
10–19	2	1	3.6	2.9
20–49	2	1	3.6	2.9
Total	55	48	100	100

Video Power Index

The mean VPI values for all videos were 4.6 (± 11.1) before and 3.3 (± 8.9) after pandemic. There is no difference between VPI values before and after COVID (CI 95%, p: 0.5). When we examined the relationship between VPI and quality scores, we found no relationship between VPI of pre-COVID videos and theirs JAMA and GQS scores. After COVID, there was only a negative correlation with the JAMA score (p: 0.001, r: − 0.370, α: 0.01). VPI values of low-medium–high quality videos were similar in both periods according to both JAMA and GQS scores (CI 95%, p > 0.5).

Discussion

Due to the rapid development of cancer diagnosis and treatment procedures, both of the personnel working in this field and patients need to update their knowledge on this subject. The increasing use of the internet and sharing technologies has made YouTube a popular source of digital information for RNT as well as for other medical topics. Parallel to this trend, academic curiosity about the effectiveness as an education material and content quality of medical YouTube videos has increased. However, studies exploring the value of cancer-related videos as an education material are still rare. Studies on RNT videos are even rarer.

Topics of cancer-related videos that are studied are colorectal, bladder, prostate, pancreatic, oral, skin, cervical, larynx, thyroid, anal, breast, gynecological, and testicular cancer [15–34]. Besides these, glioblastoma treatment, cancer-related pain management, pediatric cancer clinical trials, optimizing rural patients’ cancer education, cancer recovery, racial disparity in cancer videos, and even videos about the compatibility of DISCERN and JAMA criteria were studied [35–42].

The results of our study showed that the information provided to viewers in RNT videos is useful in both the pre-COVID and post-COVID periods (Fig. 1). This is not surprising, as the vast majority of RNT videos are uploaded by professional institutions and individuals for educational purposes (Fig. 1) and are of high quality (Fig. 3). The further improvement in content quality post-COVID-19 can be attributed to the fact that scientific meetings are held almost entirely on the web and content is made available on YouTube during the pandemic. As far as we know, there is no other study in the literature focusing the change in videos during the COVID process. However, considering the general trend regarding meetings during the pandemic, it can be thought that the same change will apply to other videos with medical content.

The high content quality and usefulness rate of RNT-related videos differ from other medical YouTube videos. In related studies, we found that, depending on the topic, researchers often found videos to be “inadequate” and only a small percentage of videos being “adequate” quality uploaded by medical professionals and government agencies [16–22, 24–31, 33–35, 40]. This is an indication that the accuracy of YouTube as a source of information varies according to the subject and the dominant source.

The potency of the video is a separate issue from the content quality. When considered as an education material, the rate of reaching the audience is as important as the accuracy of the information in YouTube videos, sometimes even more. To measure the effectiveness of the video, parameters generated from view and comment rates, which are also a measure of popularity, are often used (e.g., video power index). In our own study, we found that VPI was unrelated to pre-COVID GQS and JAMA scores. After COVID, it was negatively correlated with the JAMA score. While better quality content is expected to be more “popular” and more “impactful,” proven data says the opposite. The general trend in literature data is less viewing of high-scoring videos, similar to our results [16, 17, 19–22, 24, 29, 30, 33, 40]. We did not find any study directly comparing the popularity of educational and personal videos. Also, we could not find a significant difference between educational and personal videos in terms of average duration, number of views, and number of comments. However, in the available literature, it has been reported that educational videos are generally of higher quality and almost always lower quality videos are more popular [16, 19, 20, 22, 24, 30, 36, 40]. The only exception we could find is Duran et al.‘s study on prostate cancer videos, in which no significant relationship was found between the quality markers (JAMA and GQS scores) and likes, comments, views, and VPI [33].

The average number of views of the videos included in our study is 8032, while the number of comments is only 2.24. Interestingly, while the number of views, comments, and likes were correlated, the number of likes is much higher than the number of comments on the same videos. For example, on the three videos with the highest number of views, the number of comments are 34, 1, and 24, respectively, while the number of likes are 398, 62, and 435. This rate of comments, which is very low compared to the viewing rate, and the high rate of appreciation according to the number of comments, is an indication that the videos about RNT meet the target audience, but the viewers do not need to contribute. In this regard, RNT videos differ from the general distribution in terms of their content quality, as well as their rate of comment, in other words, “audience contribution” [16–22, 24–31, 33–35, 40]. For RNT videos, the comments are mostly from other professionals or students. In fact, in a subject like RNT with a relatively small target audience and a heavy theoretical load, it is not surprising that non-professional listeners do not consider themselves competent enough to produce an opinion and are content to listen without comment.

Informing about diagnosis and treatment algorithms is a sufficient challenge in itself. Informing about RNT is even more complicated because of the radiation effect, the fact that the concept of targeted therapy is relatively new and it accompanies theranostic applications. We are fortunate that the current YouTube videos on RNT are of high quality and have been protected from misleading viewer comments.

Conclusion

The content quality and power of the medical-themed YouTube videos vary from subject to subject. In terms of RNT, the videos in circulation offer substantially high quality and useful content to the viewer.

Of course, in order to test the definitive effectiveness of an educational material, a measurement test should be administered to the target audience before and after its use. Since we only focus on the material, our study is limited in this respect. There is a need for studies that will measure the effectiveness of the material, with mini-tests to be made before and after watching the videos in the future. However, evaluating the current content about targeted radionuclide therapy with objective criteria and adding the effect of the COVID-19 process to the research highlights our study.

We suggest that while not fully replacing face-to-face communication, YouTube videos about RNT are useful education material to convey basic information to the cancer patients, physicians, and other professionals who want to learn about the RNT. The COVID-19 pandemic has not had a negative impact on these features of the videos.

Professional uploaders should continue to produce content so that the rate of high-quality RNT video in circulation does not decrease in the future. In addition, in order to keep the watching rate of the produced content high, the trends of watching should be closely followed and methods that increase interaction should be taken into account while preparing the video. Also, as an educational resource, the quality of circulating video content should be constantly evaluated with new studies and information should be kept up to date.

This study focuses on whether there is a change in video features during the COVID-19 process. Possible non-COVID causes of these changes have not been examined: video recording and sharing technologies, internet access, interactive technologies, changes in social behavior patterns, etc. Further detailed studies are needed to investigate the impact of changes in these areas.

Author Contribution

All authors contributed to the study conception and design. Material preparation and data collection were performed by UK, SS, and EA. Data analysis were performed UK and SS. The manuscript was written by UK, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Data Availability

Contact the corresponding author for data requests.

Declarations

Conflict of Interest

Authors Ulku Korkmaz, Selin Soyluoglu, and Ersan Arda declare that they have no competing interests.

Ethics Approval and Patient Consent

This study does not require ethical approval and patient consent.

Consent for Publication

Not applicable.

Authors’ Information

All authors have been working as physician and also as associate professors at Trakya University Faculty of Medicine for more than 4 years. Ulku Korkmaz and Selin Soyluoğlu are working as Nuclear Medicine physicians, and Ersan Arda is an urologist.