Doctor YouTube’s opinion on seasonal influenza: A critical appraisal of the information available to patients

Short abstract

Background

Seasonal influenza is a respiratory illness caused by the influenza virus. During the 2017–2018 flu season, the Centers for Disease Control and Prevention noted approximately 959,000 hospitalizations and 79,400 deaths from influenza. We sought to evaluate the educational quality of informational videos pertaining to seasonal influenza on the popular social media forum, YouTube.

Methods

Using the keywords “seasonal influenza,” all videos from 28 January to 5 February 2017 were included and analyzed for characteristics, source, and content. The source was further classified as healthcare provider, alternative-medicine provider, the patient and/or their parents, company, media, or professional society. Videos about other categories of influenza (e.g. swine or Spanish) or in foreign languages were excluded. A total of 10 blinded reviewers scored each video independently.

Results

Overall, 300 videos were analyzed, with a median of 341.50 views, 1.00 likes, 0 dislikes, and 0 comments. Based on the average scores of videos by source, there was statistically significant difference in the average score among videos by video source (p < 0.01). Healthcare provider videos had the highest mean scores whereas alternative medicine provider videos had the lowest.

Conclusions

Although the aforementioned video sources scored higher than others, these videos did not fulfill our criteria as far as educating patients thoroughly. Our data also suggest alternative medicine and patient source videos were misleading for patients.

Clinical implications: Although videos by healthcare providers were a better source of information, videos on seasonal influenza were shown to be poor sources of valid healthcare information. This study reiterates the need for higher-quality educational videos on seasonal influenza by the medical community.

Introduction

Seasonal influenza has placed a substantial burden on the United States (US) resulting in approximately 9.2 million to 35.6 million cases of illness, 140,000 to 710,000 hospitalizations, and 12,000 to 56,000 deaths annually since 2010.¹ The seasonal influenza disease burden was highlighted further based on the 2003 US population, which estimated the influenza epidemic resulted in 3.1 million hospitalized days and 31.4 million outpatient visits.² In this active digital era, more and more patients utilize social media for medical advice. YouTube is a popular social media forum that facilitates for over a billion users to share media content.³ YouTube is available in more than 88 countries in 76 different languages.³

YouTube was previously evaluated for the quality of healthcare information available for chronic obstructive pulmonary disease patient education⁴ and asthma.⁵ We sought to determine the educational quality of freely accessible seasonal influenza YouTube videos.

Methods

We performed a YouTube search using the keywords “seasonal influenza” from 28 January to 5 February 2017. The inclusion criteria included videos in English, which lasted <20 minutes and good visual quality (pixels ≥240). Videos in languages other than English, not relevant (i.e. swine flu, Spanish flu, H1N1 influenza), greater than 20 minutes long, or were of poor visual quality were excluded from our study. The 300 most viewed relevant videos were included and analyzed for video characteristics, source, and content. Video characteristics included sex and race depicted, duration of video, number of views, likes, and dislikes. The videos were then divided into six categories based on their source: healthcare provider (e.g. physician), alternative medicine provider, the patient and/or their parents, company (e.g. pharmaceutical company), media (e.g. news channel), and professional society (e.g. hospital, organizations, healthcare society). Video content was subdivided into medical professional education, advertisement, personal experience, patient education, alternative treatment, or for increasing awareness.

A seasonal influenza data quality score was created utilizing a previously published method.^5–9 It was based on the Centers for Disease Control and Prevention (CDC) online information packet “Key Facts about Influenza (flu).”¹⁰ The goal of this scoring system is to evaluate the quality of YouTube web content about seasonal influenza. This scoring was revised by two content experts: a board-certified pulmonologist and a board-certified infectious disease physician. Videos were rated from a starting score of zero, with a total score of negative 10 to positive 40 points. Points were subtracted for misleading or inaccurate information that placed the patients at risk (e.g. “the influenza vaccine is a government conspiracy” or “catching influenza will prevent you from getting the influenza again”). In addition to the scoring system, a global quality scale (GQS), a five-point Likert scale, was used to subjectively assess the overall quality of each video. Videos were rated from a score of one to five, with one representing the lowest quality and five representing the highest. The GQS has been previously published and used to score websites pertaining to information about inflammatory bowel disease.^6–8

In total, 10 blinded reviewers evaluated all the videos: three pre-medical students, two medical students, four junior internal medicine (IM) residents, and one senior IM resident. Before scoring the videos, all reviewers were required to read the seasonal influenza information page on the CDC online website¹⁰ before participating in the study. This ensured all reviewers had a uniform platform of information regarding our topic. Reviewers scored each video independently and were blinded to each other’s comments.

REDCap™ software, hosted by East Tennessee State University¹¹ was used to conduct the study in addition to collecting and managing the data. This study was exempted from the Institutional Review Board. Upon conclusion of data collection, video characteristics were described using medians and quartiles for all continuous variables. All analyses were two-tailed and performed at a significance level of 0.05. Analysis of variance was used to compare score by video source. Intra-class correlation (ICC) was used to assess the reliability in scoring by reviewers. SAS 9.3 software (SAS Institute, Cary, NC) was used for all analyses.

Results

A YouTube search for “seasonal influenza” between 28 January and 5 February 2017 generated approximately 9450 results. The filter tool on YouTube was used to narrow the videos to the top 300 most viewed videos. In total, 588 videos were excluded based on the following criteria: duration >20 minutes (577), irrelevance of content to the study, such as Spanish influenza, swine flu, Ukraine-mutated swine flu (four), malfunction of the hyperlink, such as an inability to play audio/video (two), non-English language such as Somali, Chinese (two), repetition in videos (one), copyright infringement (one), and irrelevance such as parliamentary discussion briefly announcing about flu vaccine availability (one). The most depicted sex was male (39.3%) and race was Caucasian (71.2%). Videos had a median of 2240.0 days on YouTube, 341.50 views, 1.0 likes, and 0.0 dislikes. The most common video source was professional society (38.3%) with media and news channels the second most popular (24.3%) (Figure 1).

Figure 1.

YouTube video source by percentage.

Least square mean scores (DQS) by video source is summarized in Table 1. Based on the average score of videos by source, there was a statistically significant difference in the average score among videos by video source (p < 0.01). Healthcare provider videos had the highest mean score (4.500) whereas alternative medicine provider videos had the lowest (0.475). Multiple comparisons showed the average score of health care providers was far superior to the average scores of all other sources (p < 0.001). GQS scores by video source are summarized in Table 1. As with DQS scores, healthcare provider videos had the highest mean score (2.439) whereas alternative medicine provider videos had the lowest (1.425).

Table 1.

Total scores and global quality scale descriptive statistics.

Scores by video source	Total scores (95% CI)	Median	Standard deviation	GQS (95% CI)	Median	Standard deviation
Healthcare provider (e.g. physician)	4.500 (2.952–6.048)	3.0	4.906	2.439 (2.161–2.717)	2.0	0.881
Alternative medicine provider	0.475 (0.322–0.628)	0.0	0.826	1.426 (0.987–1.863)	1.0	0.275
Patient and/or their parents	0.784 (0.516–1.052)	1.0	4.054	1.562 (1.364–1.760)	1.0	0.698
Company (e.g. pharmaceutical company)	1.935 (0.724–3.147)	1.0	2.356	1.641 (1.302–1.980)	1.0	0.660
Media (e.g. news channel)	2.471 (1.640–3.303)	2.0	3.563	1.874 (1.712–2.036)	2.0	0.693
Professional society (e.g. hospital, organization, healthcare society)	4.178 (3.394–4.962)	2.0	4.244	2.314 (2.172–2.456)	2.0	0.770

CI: confidence interval; GQS: global quality score.

Finally, an ICC was carried out to assess interrater reliability. It was chosen to see the average values of the ICC as that indicates a sense of reliability as opposed to a single rate analysis. All values proved statistically significant (p < 0.001), which indicated all samples were truly random. Based on calculated values (Table 2), there was a high sense of reliability between the raters regardless of video source apart from alternative medicine provider. This source only contained 1.33% of all videos, which in turn resulted in a low ICC of 0.639.

Table 2.

Intraclass correlation for total scores and global quality scale.

Scores by video source	Total scores (95% CI)	p value	GQS (95% CI)	p value
Healthcare provider (e.g. physician)	0.990 (0.984–0.994)	<0.001	0.928 (0.878–0.959)	<0.001
Alternative medicine provider	0.847 (0.470–0.989)	<0.001	0.639 (0.340–0.974)	<0.001
Patient and/or their parents	0.981 (0.972–0.988)	<0.001	0.927 (0.881–0.956)	<0.001
Company (e.g. pharmaceutical company)	0.973 (0.949–0.989)	<0.001	0.913 (0.827–0.964)	<0.001
Media (e.g. news channel)	0.975 (0.964–0.983)	<0.001	0.894 (0.821–0.936)	<0.001
Professional society (e.g. hospital, organization, healthcare society)	0.972 (0.964–0.979)	<0.001	0.892 (0.831–0.930)	<0.001

CI: confidence interval; GQS: global quality score.

Discussion

Social media plays an increasingly prominent role in generations that are becoming progressively more technological. YouTube, along with many other social media networks, has become a vital resource in day-to-day ‘how-to’ searches.¹² YouTube was found to be the second most used social media network by parents of children with food allergies, and the top network for teenagers.¹³ Although YouTube houses a large amount of information, it also divulges large amounts of misinformation. For instance, it was demonstrated that videos about asthma were a poor source of accurate healthcare information⁵ and only 15% of videos correctly discussed and demonstrated a task as simple as using an asthma inhaler.¹⁴

We critically appraised the 300 most viewed videos available on YouTube about seasonal influenza. Although it is a reprieve that alternative medicine provider and the patient and/or their parents were proven to not be statistically significant, the remaining videos continued to be subpar in their educational quality. Least mean scores based on GQS for videos from healthcare providers were statistically significantly different, with highest average score of 2.439 (p < 0.001), which was closely followed by professional society (2.314; p < 0.001).

The scarcity of high-quality education regarding a common ailment such as seasonal influenza was highlighted repeatedly during our study. Over 50% of all videos, regardless of source, were rated a two or below on the Likert scale (Table 1) of one to five (with one indicating poor quality, explanation, and least use, and five indicating excellent quality, explanation, and extreme usefulness for patients). Several cross-sectional studies have been previously published on assessing the quality of information of YouTube spanning over a large subject base.^5,8,15–26 Repeated conclusions for these studies included the lack of emphasis of data and misrepresentation of data available.^15,16

Figure 2 compares DQS versus GQS on influenza YouTube videos. Videos by healthcare provider videos had the highest DQS (4.5) and GQS (2.439), whereas alternative medicine videos had the lowest DQS (0.475) and GQS (1.425). It should be noted that although healthcare provider videos had the highest DQS and GQS scores relative to the other categories, the mean GQS was still not above three, signifying that in general, influenza videos are still below satisfactory. It is also worth mentioning that videos by professional societies were the second highest rated videos for DQS (4.178) and GQS (2.314). With an R² value of 0.978 (Figure 2), it is apparent there is a strong correlation between DQS and GQS scores.

Figure 2.

Average global quality scale versus average data quality scale by video source.

$\mathit{Avg}. \mathit{GQS}=0.546463\times \mathit{Avg}. GQ{S}^3+-4.51745\times \mathit{Avg}. GQ{S}^2+15.2497\times \mathit{Avg}. \mathit{GQS}+-13.7299$ $R^2=0.978423,p=0.032191$.

The limitations of our study included that only the top 300 videos were evaluated after the application of our filters. Non-English videos and videos >20 minutes long were also excluded from evaluation. It is important to note that the source of the videos varied and not all the content was able to be reviewed by experts. The varying degree of medical knowledge among the reviewers might have led to a difference in the understanding of the microbiological aspects of the influenza virus and its transmission, hence leading to potential variations in information evaluation. It should be noted that despite this, interrater reliability was high for all video sources for FA-DQS, apart from Alternative Medicine Provider (ICC = 0.639; p < 0.001) possibly due to small sample size (only 1.33% of all videos came from alternative medicine providers).

In addition to the generally poorly rated videos, inefficacious alternatives to influenza immunity were proposed that included the acquisition of immunity through active infection versus vaccination.^27–29 Other unproven claims were made about vaccinations, such as government conspiracy theories of the government tracking people via a chip in the influenza vaccine, and the harmful effects of thimerosal, a preservative used in storing the vaccine. Thimerosal is used to store the vaccine but is not associated with adverse neurological outcomes.³⁰

Due to its dominant presence and accessibility for information, the Internet remains one of the main resources for patient education. Approximately eight in 10 Internet users search for health information online.³¹ Although there have been positive outcomes for Internet users enquiring about weight loss, tobacco cessation, and many other healthcare issues, there are still many opportunities for improving communication with patients.³² Although individual patient education can be utilized successfully by primary care physicians, it is not feasible with large patient populations.

There is a need for high-quality, evidence-based, educational videos on seasonal influenza, with an additional emphasis on dispelling common misconceptions associated with influenza and influenza vaccination. Practicing primary care physicians can use such videos or create their own for not only a standardized approach in patient education, but also for further outreach in patient populations.

Authors' Note

This work was partially presented at the CHEST annual meeting of 2017.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Peer review

This manuscript was reviewed by reviewers who have chosen to remain anonymous.

ORCID iDs

Akhil Kallur https://orcid.org/0000-0003-3164-344X

Alexei Gonzalez-Estrada https://orcid.org/0000-0002-5754-6130