Abstract
This study evaluates the quality and reliability of YouTube videos pertaining to rhinoplasty as a source of patient information.
Patients contemplating undergoing rhinoplasty may use the internet to search for information. YouTube (http://www.YouTube.com) is the most widely used video platform on the internet and is increasingly used to disseminate health information by educational institutions, for-profit companies, and others. Individual users posting videos of their experience of health conditions or surgical procedures on video blogs, or vlogs, have become a popular and influential presence on the website. This study aims to objectively assess the quality and reliability of information on YouTube pertaining to rhinoplasty.
Methods
Ethical approval was not required for this study according to the UK National Health Service Health Research Authority decision tool.1 The authors performed a YouTube search in private browsing mode with all cookies deleted, using the search terms rhinoplasty and nose job, a common colloquial term for rhinoplasty. Search parameters were set to worldwide. Non–English-language videos were excluded.
The first 100 videos retrieved for each search term were analyzed by 2 independent reviewers (nose job videos: B.O. and D.B; rhinoplasty videos: O.O. and R.A.) and categorized by source (independent users, government/news agencies, university channels/professional organizations; health information websites, or medical advertisements/profit companies),2 and upload location and then as useful, misleading, personal experience, or irrelevant.3 Useful videos were assessed using the Global Quality Scale (GQS) as a validated measure of quality. The GQS is a subjective measure of overall quality of information, graded on a 5-point Likert scale, with 1 representing poor quality (most information missing, not all useful for patients) and 5 representing excellent quality (very useful for patients). The modified DISCERN tool was used as a measure of reliability.3 The tool has a potential total score of 5 points, with higher scores indicating higher reliability.4,5 When there was discordance, a third independent reviewer (nose job videos: O.O.; rhinoplasty videos: B.O.) assessed the video and was the decider. Video characteristics including duration of video, video popularity (defined as the ratio of total views by number of days since upload), and number of likes and dislikes were noted for all relevant videos.
Effect sizes with 95% CIs were calculated using Microsoft Excel for Mac, version 16.10 (Microsoft Corp) and ESCI statistical software version 3 (Free Software Foundation, Inc).6
Results
Of the 100 videos retrieved using the search term rhinoplasty, 37 were categorized as useful, 61 as personal experience, and 2 as irrelevant. Of the 100 videos retrieved using the search term nose job, 16 were categorized as useful, 33 as personal experience, and 51 as irrelevant. The United States was the most popular upload location overall, accounting for 74 of the 100 videos evaluated (74%), with patient personal experiences constituting the majority of relevant videos retrieved (rhinoplasty, 61 of 98 [62%]; nose job, 33 of 49 [67%]). Independent users (vloggers) uploaded 70 videos (70%) under the search for nose job and 34 (34%) of rhinoplasty videos. Nose job searches were more likely to retrieve irrelevant videos (51 [51%] vs 2 [2%] for rhinoplasty searches; difference between proportions, 0.49; 95% CI, 0.38-0.59). Neither search retrieved any misleading videos.
The Table highlights the video characteristics and the effect sizes when comparing search terms. Global Quality Scale scores for useful rhinoplasty and nose job videos were similar (mean difference, –0.3; 95% CI, –1.16 to 0.55), a pattern repeated with personal experience videos (mean difference, 0.12; 95% CI, –0.14 to 0.39) and their reliability scores (mean difference for DISCERN score for useful videos, –0.35; 95% CI, –0.81 to 0.11).
Table. Characteristics of 143 Useful and Personal Experience YouTube Videos Retrieved Using Rhinoplasty and Nose Job as Search Terms.
| Category | Rhinoplasty, Mean (SD) or Median (Range) | Nose Job, Mean (SD) or Median (Range) | Mean Difference (95% CI) | Cohen d (95% CI) |
|---|---|---|---|---|
| Useful | ||||
| No. | 37 | 16 | NA | NA |
| Days available on YouTube, mean (SD) | 907 (675) | 802 (610) | −105 (−499 to 289) | −0.16 (−0.75 to 0.43) |
| Duration, min, mean (SD)a | 11 (16) | 6 (5) | −5 (−13 to 3) | −0.38 (−0.97 to 0.20) |
| Views, median (range) | 24 369 (468 to 1 021 108) | 47 231 (633 to 7 139 701) | −603 477.5 (−14 533.27 to 1 192 422) | 0.61 (0.01 to 1.21) |
| Popularity, median (range)b | 38 (1 to 1303) | 570 (18 to 3973) | 455 (101 to 808) | 0.77 (0.16 to 1.36) |
| Likes, median (range) | 51 (0 to 2115) | 92 (0 to 26 365) | 1905 (227 to 4038) | 0.54 (−0.06 to 1.13) |
| Dislikes, median (range) | 2 (0 to 165) | 13 (0 - 6584) | 471 (−62 to 1005) | 0.53 (−0.07 to 1.12) |
| Comments, median (range) | 12 (0 to 662) | 33 (0 to 15 053) | 1074 (−143 to 2291) | 0.53 (−0.07 to 1.11) |
| GQS score, mean (SD)c | 3.3 (1.54) | 2.09 (0.72) | −0.3 (−1.16 to 0.55) | −0.21 (−0.79 to 0.38) |
| DISCERN score, mean (SD)d | 3.16 (0.44) | 2.81 (1.22) | −0.35 (−0.81 to 0.11) | −0.46 (−1.05 to 0.14) |
| Personal experience | ||||
| No. | 60 | 33 | NA | NA |
| Days available on YouTube, mean (SD) | 740 (688) | 333 (323) | −407 (−660 to 155) | −0.70 (−1.13 to −0.26) |
| Duration, mean (SD)a | 9 (6) | 11 (7) | 1.96 (−0.80 to 4.73) | 0.31 (−0.12 to 0.73) |
| Views, median (range) | 27 329 (613 to 367 421) | 50 509 (5771 to 2 667 620) | 272 838 (109 639 to 436 038) | 0.72 (0.280 to 1.16) |
| Popularity, median (range)b | 72 (2 to 938) | 197 (16 to 8064) | 1124 (528 to 1719) | 0.81 (0.37 to 1.24) |
| Likes, median (range) | 131 (0 to 2023) | 383 (44 to 50 131) | 6849 (3275 to 10 424) | 0.83 (0.38 to 1.27) |
| Dislikes, median (range) | 9 (0 to 140) | 23 (0 to 9317) | 507 (78 to 936) | 0.51 (0.08 to 0.94) |
| Comments, median (range) | 42 (0 to 693) | 155 (3 to 10 263) | 1123 (434 to 1813) | 0.71 (0.27 to 1.14) |
| GQS score, mean (SD)c | 1.97 (0.55) | 2.09 (0.72) | 0.12 (−0.14 to 0.39) | 0.20 (−0.23 to 0.63) |
Abbreviations: GQS, Global Quality Scale; NA, not applicable.
Duration is rounded to nearest whole minute.
Popularity is defined as the ratio of total views for video per number of days on YouTube since upload. A greater ratio equals greater popularity.
The GQS score is calculated by rating the quality of the video content on a 5-point Likert scale from poor to excellent.4
The DISCERN score is calculated by completing the 5-question, modified DISCERN tool.3 It is presented only for videos categorized as useful because irrelevant videos or those containing patient experiences were deemed not to be categorized as health information, which the tool was validated to measure.
Discussion
To our knowledge, this was the first in-depth study to objectively assess the quality and reliability of information pertaining to rhinoplasty on YouTube. There were approximately 1.2 million views across the 200 videos included in this study, highlighting the importance of the internet and YouTube as sources of health information for patients. This study has found that information on YouTube pertaining to rhinoplasty is of average quality and reliability.
A key finding is that personal experience videos accessed using the term nose job had more views and engagements (likes, dislikes, and comments) and were more popular than videos accessed using rhinoplasty despite having been uploaded for less time and being significantly more likely to retrieve results categorized as irrelevant by reviewers. However, searching rhinoplasty rather than nose job did not alter the quality or reliability of information retrieved. The main limitations of this study are the lack of a standardized tool to assess health information and limitation of search results to English language videos, which will affect the generalizability of our findings.
To aid patients in benefiting from YouTube as a health information resource, specialists should consider referring their patients to validated online sources of information and uploading their own educational and promotional material to improve on the paucity of reliable, good-quality information on YouTube.
References
- 1.Health Research Authority. http://www.hra-decisiontools.org.uk/research/ Accessed August 21, 2018.
- 2.Singh AG, Singh S, Singh PP. YouTube for information on rheumatoid arthritis–a wakeup call? J Rheumatol. 2012;39(5):899-903. doi: 10.3899/jrheum.111114 [DOI] [PubMed] [Google Scholar]
- 3.Delli K, Livas C, Vissink A, Spijkervet FK. Is YouTube useful as a source of information for Sjögren’s syndrome? Oral Dis. 2016;22(3):196-201. doi: 10.1111/odi.12404 [DOI] [PubMed] [Google Scholar]
- 4.Bernard A, Langille M, Hughes S, Rose C, Leddin D, Veldhuyzen van Zanten S. A systematic review of patient inflammatory bowel disease information resources on the World Wide Web. Am J Gastroenterol. 2007;102(9):2070-2077. doi: 10.1111/j.1572-0241.2007.01325.x [DOI] [PubMed] [Google Scholar]
- 5.Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999;53(2):105-111. doi: 10.1136/jech.53.2.105 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Cumming GD. ESCI, exploratory software for confidence intervals. 2001. https://thenewstatistics.com/itns/esci/ Accessed October 13, 2018.