YouTube as a Source of Information on Echocardiography: Content and Quality Analysis

Abstract

Background

YouTube (YouTube, LLC, San Bruno, CA, USA) is an easily accessible and increasingly popular source of information on health-related issues. This study evaluated the scientific content and quality of English-language echocardiography videos posted to this Internet platform.

Methods

For this study, 583 videos were identified and evaluated following a YouTube search using the keyword ‘echocardiography’ in July 2020. A total of 92 videos were included in this study following the application of the study exclusion criteria. Both video quality and scientific content were evaluated based on the DISCERN score, global quality score, content score, and total quality score (TQS), which was calculated by adding the scores of the three aforementioned scoring systems.

Results

The median duration of the included videos was 315 seconds (interquartile range: 173-852 seconds). The median number of views was 33046 (interquartile range: 7445-89012). Twenty-four (26.08%), 43 (46.73%) and 25 (27.17%) videos, respectively, were stratified into the following three groups based on TQS values: group 1 (5-7 points), group 2 (8-11 points) and group 3 (12-14 points). The values of views (median: 45072), video duration (median: 474), power index (median: 80250) and popularity (median: 34) were significantly higher in group 3 than in the other groups. The duration that a video was online was statistically significantly higher in group 1.

Conclusions

The results suggest that YouTube is an important tool for easily sharing health-related information and disseminating it to large audiences. It can be concluded from the growing interest in videos with scientific content and the high quality scores recorded that YouTube offers accurate and quality information about echocardiography.

INTRODUCTION

YouTube (YouTube, LLC, San Bruno, CA, USA) is a rapidly developing online video platform that attracts 100 million viewers and nearly two billion views every day.^1,2 Studies have revealed a significant increase in the use of the Internet to access health-related information, and eight of 10 Internet users have accessed health-related information online.^3,4 Based on the 2018 results of a national health survey, more than 33% of patients visit YouTube to watch health-related videos.⁵ Moreover, this source of information conveyed using visual and auditory means is becoming an interesting platform not only for patients but also for physicians.⁶ However, the viewers and uploaders who use the platform are nonhomogeneous, and the contents of online videos are not subject to approval in terms of quality and content, which can facilitate the spread of false or misleading information, including that pertaining to health.^2,5,7 Echocardiography is an important, noninvasive diagnostic tool used in the diagnosis and follow-up of cardiovascular diseases. On YouTube, videos are shared that cover many diagnostic and treatment methods both to inform society and for clinician training purposes. In recent years, researchers have evaluated the quality and reliability of YouTube videos, which have become a popular source of information for individuals and public health workers. However, to our knowledge, no study has yet evaluated the quality of YouTube videos on echocardiography. The purpose of this study was therefore to evaluate the scientific content, reliability and quality of English-language videos on echocardiography uploaded to YouTube.

METHODS

Using the Google Trends application (http://www.google.com/trends/; Google, LLC, Mountain View, CA, USA),⁸ ‘echocardiography’ was determined to be the English keyword most preferred by users to search for content concerning the field of echocardiography on YouTube (http://www.youtube.com). Using this keyword, relevant videos were identified and accessed on YouTube in July 2020. To reduce any bias of the YouTube search engine for videos due to the location of the study computer and search history, all searches were performed in a single day using incognito mode of Google Chrome browser (Google, LLC), and the results were recorded for subsequent evaluation.⁹ Search results were evaluated by two independent cardiologists (E. A. and E. Y.). A total of 583 videos were identified through the search, and 381 videos related to the study topic were subjected to a preliminary evaluation.

Ninety percent of Internet search engine users click on the first three pages of search results, although 79% of users who do not see the results they were looking for on the first three pages go on to evaluate the contents of the next several pages.^10-12 Based on experience reported by video authors who previously uploaded their creations to YouTube, we evaluated the first 10 pages (n = 20 videos per page).^8,12-14 After the preliminary evaluation, the first 200 videos were analysed according to the inclusion and exclusion criteria based on prior examples in the literature to obtain the sample for this study.^13,14 Videos eligible for inclusion were those related to echocardiography with a run-time of longer than 20 seconds and with audio in English. Videos in which the image and sound quality were good, the narrator used clear and understandable language, appropriate information was offered to calculate the content score (CS), and the echocardiography imaging technique and images could be clearly understood were preferred for inclusion in this study. The exclusion criteria were: videos with low-quality sound or no sound, videos on other topics, online webinar presentations, advanced/technical echocardiography training videos and videos with slide presentations. In cases where all or parts of the same video were published by different users, the main video was included in the evaluation, while parts or copies of the same video were excluded. The videos included in the study were categorized according to their content and subject: (i) valvular diseases (n = 23), (ii) heart cavities and ejection fraction measurements (n = 42), (iii) structural heart diseases (n = 7), (iv) pericardial diseases (n = 8), (v) cardiac masses (n = 6), and (vi) aortic syndromes (n = 6). Consequently, a total of 92 videos published between December 2008 and June 2020 were selected for analysis (Figure 1).

Figure 1.

Study flow chart.

The following descriptive data were collected from the included videos: video duration (seconds), time from the day the video was posted to the search day (‘upload days’), the popularity score, and the number of views, likes, dislikes and comments. The video popularity score was defined as the ratio of the number of views of the video noted on the search date that had accumulated throughout the duration of its broadcast up until that date.^2,10,11 Video sources included independent user channels (IUCs), hospital-university channels (HUCs), medical dot-com channels (MDCs) (i.e., institutional channels shared by more than one healthcare professional user with an external Internet address where additional information could be accessed), and news agency channels (NACs). The video power index was calculated as (the number of views × the number of likes)/100.¹⁵ The reliability of the information provided in the selected videos was rated using a scoring-based scale adapted from the DISCERN tool, and was labelled as the DISCERN score (DS). This scale includes five questions answered in a ‘yes’ or ‘no’ format. For scoring purposes, each ‘yes’ answer was scored 1 point (indicating a good level of reliability) and each ‘no’ answer was scored 0 points (indicating a low level of reliability) (Table 1).^10-12,16

Table 1. Discern score, content score and global quality score.

Discern score (DS)

1. Are the educational goals clearly stated and achieved?

2. Are reliable sources of information used?

3. Is the information presented balanced and unbiased?

4. Are additional sources of information listed for users to refer to?

5. Are areas of uncertainty, gaps or differences of opinion mentioned?

Content score (CS)

1. What are the preparations before echocardiography imaging?

2. What are the indications for echocardiography?

3. What is the diagnostic power of echocardiography?

4. What are the imaging details of echocardiography?

Global quality score (GQS)

1. Not useful at all for viewers.

2. Poor quality in general and poor video streaming; very limited use recommended for viewers.

3. Medium quality and insufficient flow; some important information is provided, but other information is missing. A little useful for the audience.

4. Good quality and generally good flow; most of the relevant information is listed, but some topics have not been covered; useful for viewers.

5. Excellent quality and perfect flow; very useful for viewers.

The accuracy and quality of the information about echocardiography presented in the videos were evaluated based on the echocardiography training, competence and quality criteria of the European Association of Cardiovascular Imaging 2017 expert consensus document and the European Association of Cardiovascular Imaging 2020 recommendations.¹⁷

Quality and accuracy evaluations of the videos were conducted using a CS developed according to the European Association of Cardiovascular Imaging 2020 update of recommendations about echocardiography, training, competence and quality criteria.¹⁸ The CS was evaluated using four parameters based on similar examples in the literature. This scale includes four questions which are answered in a ‘yes’ or ‘no’ format; each ‘yes’ answer was scored 1 point and indicated the quality of the video content, while each ‘no’ was scored 0 points (Table 1).¹⁹ The total CS was expressed based on the four items, with each item evaluated using the factors listed below. Items that received positive points for the absolute majority of the relevant factors were scored 1 point. The preparations made before echocardiography, patient’s lying position, operator’s imaging technique, probe placement and probe manipulation were evaluated under item 1. Echocardiography indications, the expression of imaging details according to the indications and the evaluation of findings within the indications were evaluated and scored under item 2. The diagnostic power of echocardiography, clinical situations in which it is superior, clinical situations where its diagnostic power is insufficient, and guidance on when to consider alternative imaging methods were evaluated and scored within the scope of item 3. Finally, image quality, standard imaging and measurements of cardiac chambers were evaluated under item 4.

Furthermore, the videos were evaluated in terms of the general ease of interpretation and information flow using the global quality score (GQS) (Table 1),^20,21 which was used to rate the overall quality of each video (scored from 1-5 points, where 1 = low quality and 5 = excellent quality).

The total quality score (TQS) was calculated by adding together the DS, CS and GQS scores of the videos. The 25th and 75th percentiles of TQS values were used as cut-off points, and the videos were divided into three groups based on these values, where group 1 (low quality) included videos scored with 5 to 7 points, group 2 (medium quality) included videos scored from 8 to 11 points, and group 3 (high quality) included videos scored from 12 to 14 points.

Statistical analysis

Descriptive statistics (mean, standard deviation, median, frequency and percentage values) were used to evaluate the study data. The Kolmogorov-Smirnov test was used to approximate the normal distribution of quantitative data. The chi-squared test was used for categorical variables, while the Mann-Whitney U and Kruskal-Wallis tests were used for paired comparisons of continuous variables without normal distribution. The Spearman test was used for correlation analysis. The kappa statistic was used to measure intraobserver agreement (one week apart) and interobserver agreement. A p-value of less than 0.05 was considered to be statistically significant for all analyses. All statistical analyses were performed using the Statistical Package for the Social Sciences version 22 program (IBM Corporation, Armonk, NY, USA).

RESULTS

Overall, 92 English videos were included in this study, including 16 (17.4%) HUCs, 48 (52.2%) MDCs, 25 (27.2%) IUCs and 3 (3.3%) NACs. The median video duration was 315 seconds [interquartile range (IQR): 173-852]. The median number of views was 33046 (IQR: 7445-89012), the median number of upload days was 1304 (IQR: 725-2808), the median number of likes was 148 (IQR: 41-426), and the median number of dislikes was 6 (IQR: 1.25-21). While the median power index value of the videos was 45143 (IQR: 3304-190549), the popularity score was 24 points (IQR: 8-53). Finally, the median DS was 3 points (IQR: 2.7-4), the median GQS was 4 points (IQR: 3-4.75), the median CS was 2 points (IQR: 1-3) and the median TQS was 9 points (IQR: 7-12) (Table 2).

Table 2. Demographic data of study videos.

Variable	Median	IQR
Views	33046	(7445-89012)
Upload day (day)	1304	(725-2808)
Likes	148	(41-426)
Dislikes	6	(1.25-21)
Popularity score	24	(8-53)
Duration (seconds)	315	(173-852)
Power index	45143	(3304-190549)
HUC (%)	16	-0.174
IUC (%)	25	-0.272
MDC (%)	48	-0.522
NAC (%)	3	-0.033
Discern score (DS)	3	(2.7-4)
Global quality score (GQS)	4	(3-4.75)
Content score (CS)	2	(1-3)
Total quality score (TQS)	9	(7-12)

HUC, hospital-university channel; IUC, independent user channel; MDC, medical dot com; NAC, news agency channel; Popularity score, View/upload day; Power index, (view * likes)/100.

The study videos were stratified into three groups based on TQS data, where group 1 (low quality) included 24 videos (26.1%), group 2 (medium quality) included 43 videos (46.7%), and group 3 (high quality) included 25 videos (27.2%). Data on the number of views, video duration, power index and popularity were significantly higher in group 3, while counts of likes and dislikes were not significantly different among the groups. The number of upload days was statistically significantly higher in group 1. HUC-sourced videos were frequently included in groups 2 and 3, while IUC-sourced videos were mostly included in groups 1 and 2 (Table 3).

Table 3. Comparison of video groups data.

Variables	Group 1 (n: 24)		Group 2 (n: 43)		Group 3 (n: 25)		p value
	Median	IQR	Median	IQR	Median	IQR
Views	34369	(18286-88652)	29619	(5911-83600)	45072*	(10163-114275)	0.033
Upload day	2707*	(1382-3612)	1277	(617-2553)	1395	(630-2548)	0.021
Likes	132	(86-535)	155	(24-515)	169	(55-865)	0.053
Dislikes	3.5	(2-18)	5	(0-20)	10	(2-42)	0.205
Duration	195	(116-325)	251	(173-653)	474*	(238-921)	0.002
Power index	51890	(14857-200379)	26642	(15633-54599)	80250*	(9654-254136)	0.032
Popularity score	16	(7-42)	19	(8-46)	34*	(11-112)	0.003
HUC	0		7	16.2%	9	36%	0.015
MDC	14	58.3%	24	55.8%	10	40%
IUC	10	41.7%	11	25.7%	4	16%
NAC	0		1	2.3%	2	8%

HUC, hospital-university channel; IUC, independent user channel; MDC, medical dot com; NAC, news agency channel; Popularity score, view/upload day; Power index, (view * likes)/100.

In the correlation analysis, there was a strong positive correlation between the number of views and the numbers of likes and upload days. There was a strong positive correlation between the number of likes and popularity score, and a moderate positive correlation between the numbers of upload days and likes and the power index. Notably, a negative correlation between the number of upload days and quality evaluation criteria was observed. The GQS had a weak positive correlation with the numbers of views and likes, the power index and the popularity score (Table 4).

Table 4. Correlation analysis of study data.

Parameters	Correlation	r	p value
View	Upload day	0.658	< 0.001
View	Likes	0.858	< 0.001
View	GQS	0.231	0.027
Upload day	Likes	0.354	0.001
Upload day	Power index	0.493	< 0.001
Upload day	DS	-0.289	0.005
Upload day	CS	-0.261	0.012
Upload day	TQS	-0.265	0.010
Likes	Popularity score	0.863	< 0.001
Likes	DS	0.251	0.016
Likes	GQS	0.211	0.044
Power index	GQS	0.227	0.030
Popularity score	GQS	0.281	0.007
Popularity score	DS	0.286	0.006
Popularity score	TQS	0.298	0.004
Duration	DS	0.380	< 0.001

CS, content score; DS, discern score; GQS, global quality score; Popularity, view/upload day; Power index, (view * likes)/100; TQS, total quality score.

The kappa statistic for TQS for interobserver agreement was 0.90, while the values for intraobserver agreement for observations made one week apart were 0.92 and 0.87, respectively, for Ercan AYDIN and Emre YILMAZ The kappa analysis results for other scoring systems are presented in Table 5.

Table 5. The kappa analysis of inter-observer and intra-observer agreement.

Variable	Inter-observer agreement	Intra-observer agreement*
	Kappa statics value (p value)	Kappa statics value (p value)
		Ercan AYDIN	Emre YILMAZ
Discern score	0.84 (0.006)	0.82 (0.005)	0.85 (0.005)
Global quality score	0.78 (0.007)	0.80 (0.007)	0.77 (0.007)
Content score	0.92 (< 0.001)	0.90 (0.001)	0.88 (0.005)
Total quality score	0.90 (0.001)	0.92 (< 0.001)	0.87 (0.005)

DISCUSSION

In this study, we evaluated the content and quality of videos identified through a search using the keyword ‘echocardiography’ on the YouTube video hosting platform. Our results showed that 73.9% of the selected videos had an above-average level of quality. In previous YouTube studies on videos covering dialysis, hypertension and rhinosinusitis, only 45% to 65% of the videos provided positive information to the viewers about the disease process, while 20% to 55% provided misleading information.^12,13,22 Our findings thus suggest a trend of increased scientific content and improved quality of more recent videos, which may explain the higher quality rating in our study relative to that in previous studies. This result is an indication that videos on YouTube have an increasing tendency to provide accurate and good-quality information. However, it is also a reminder that the scientific evaluations of this dynamic platform, which attracts the attention of people from all walks of life every day, require periodic renewal.

The selected videos were first evaluated using the DS, GQS and CS, which were designed by the authors based on prior examples in the literature.^12,16,19-21 The TQS was calculated based on the sum of the scores of these three scoring systems to conduct a more objective evaluation and to allow for a comprehensive evaluation to be performed by widening the score range. The study videos were stratified into three groups using the TQS, where 26.08% of the videos were of low quality (group 1), 46.73% were of medium quality (group 2), and 27.17% were of high quality (group 3). Useful and valueless or misleading groupings have previously been established in the literature; however, these groupings were created using relatively subjective criteria based on the topic of interest.^12,23,24 Akgün et al. performed more objective grouping using their own ‘usefulness checklist’, whereas Khalil et al. used a similar grouping methodology to our study based on the total score calculated as the sum of the DS and CS. In our study, we aimed to conduct more comprehensive grouping using GQS data, which is an additional factor compared to the methods of the previous studies. Furthermore, we aimed to achieve a more appropriate distribution pattern by using 25th and 75th percentile data in groupings.

In this study, 17.4% of the videos were shared via HUCs and 52.18% were shared via MDCs. In the study by Akgün et al. on electrocardiography in 2014, the share rates were 8.4% for HUC-sourced videos, 58.8% for independent users’ posts and 32.7% for MDC-sourced videos.⁸ Relative to Akgün et al.’s study, the increase in the number of HUC- and MDC-sourced videos in our study is an indication that professional users in the field of health are sharing videos more frequently. Furthermore, based on our study results, HUC-sourced videos were significantly included in groups 2 and 3, while IUC-sourced videos were mostly found in groups 1 and 2. This result supports Akgün et al.’s conclusion that HUC-sourced videos are of a higher quality at a rate of 90%. Although there were more HUC-sourced videos in our study, we think that they should be enhanced further because some are still of dubious quality.

The number of views, video duration, power index and popularity score were significantly greater in group 3 in this study. While this is consistent with other studies in the literature, the number of views of useful or high-quality videos was higher in our study.^8,12,23 This result shows that there is more interest in and interaction with scientifically high-quality videos, which is a positive development. The fact that providing a minimum amount of information for a quality result will require a certain amount of time is supported by the finding that the video duration was longer in group 3.

Upload day values were statistically significantly greater in group 1 (low quality), which may indicate that the quality of scientific content is lower in older videos. Furthermore, this result is an indicator that the quality of more recent videos is increasing. Kumar et al. reported no significant difference in the number of upload days between groups; however, the data obtained in the study by Chen et al. are consistent with our results.^12,24 Moreover, the negative correlation between the number of upload days and quality evaluation criteria was a positive result which may indicate that the interest of video sharers in scientific content and quality has increased over time.

There was a strong positive correlation between the numbers of views and likes and the number of upload days. This result supports the fact that a video that has been posted for a long time and which reaches more users will thus be involved in more interactions.

There was a strong positive correlation between the number of likes and popularity score and a moderate positive correlation between the number of upload days and likes and the power index. These correlations support the fact that interactions will increase as the popularity and duration a video has spent online increases, as is expected on social platforms. Furthermore, the GQS had a weak positive correlation with the numbers of views and likes, the power index and the popularity score. Although weak, this correlation is an indicator that interest in quality sharing has started to increase.

Limitations

The primary limitations include focusing only on English-language videos and adopting a cross-sectional study design. Based on YouTube press statistics, 48 hours of video are uploaded to the system every minute, resulting in about eight years of content being uploaded each day. Therefore, the information published on this website varies significantly over time.²⁵ The popularity and accessibility of YouTube are also increasing day by day, so conducting scientific research on such a platform would require frequent updates. Moreover, because of the dynamic content and variable algorithm that YouTube employs, video optimization and analysis may lead to a degree of variability in the search results for the same search item between two users. We attempted to mitigate this limitation by conducting a high number of video evaluations. Viewer interactions with videos were directly evaluated on the YouTube website, with YouTube videos shared on other websites or social media platforms excluded from consideration. The qualities of the videos may be different from 2008 to 2020 as there are many updates and advances in echo devices, probes, and image storing/transfer programs. The wide range of videos evaluated in our study (2008-2020) may limit our results due to changes in guideline recommendations (American and European echocardiography/cardiovascular imaging associations). In addition, we did not include webinar presentations and lectures/congress descriptions containing advanced echo information as they were not suitable for the design and evaluation criteria of our study.

CONCLUSIONS

Our results support that the YouTube video sharing site is an important tool for easily sharing health-related information and disseminating it to large audiences. The growing interest in videos with scientific content and the high-quality scores recorded suggest that YouTube provides accurate and good-quality information regarding echocardiography. Although the number of quality shares tends to increase day by day, professional users in the field of health should be encouraged to share videos of an academic publication level of quality more frequently.

CONFLICTS OF INTEREST

There are no conflicts of interest for the authors in terms that are not clear and may affect their decisions on the content of their work (such as financial or personal interests).