Abstract
Objective
In addition to being an entertainment channel, YouTube is also one of the most popular visual information sources today. People search YouTube to consult also on orthodontics, as well as on many other topics. The objective of the present study was to analyze the quality and reliability of information of the videos on YouTube about orthodontics.
Methods
YouTube was searched systematically by two researchers on orthodontics by using the keywords “Orthodontics,” “Orthodontist,” and “Orthodontic Treatment.” Videos on the first three pages (60 videos) for each keyword were assessed. Researchers evaluated the reliability of the videos by using the Reliability Score (adapted from DISCERN) and the quality of the videos by using the Global Quality Score (GQS).
Results
The mean GQS results were 2.6±1.3 for videos in the “Orthodontist” group, 3.2±1.3 for videos in the “Orthodontics” group, and 2.3±1.2 for videos in the “Orthodontic Treatment” group on a 5-point scale. The Reliability Score results were 2, 2, and 1.5 for videos in the “Orthodontist,” “Orthodontics,” and “Orthodontic Treatment” groups, respectively, on a 5-point scale. The Intraclass Correlation Coefficient results presented a positive relationship between the researchers.
Conclusion
Owing to the lack of peer-review process and pre-upload scientific evaluation process, videos on YouTube can lead the public to misinformation.
Keywords: YouTube, internet, orthodontics, orthodontist, orthodontic treatment
INTRODUCTION
Easy and fast accessibility, patients’ wish for accessing more information, and being a cost-effective way of reaching professional healthcare consultation are the parameters that triggered the use of the internet on seeking medical information in recent years (1). It has been found that 8 out of 10 internet users searched the internet for accessing healthcare data (2).
Even though many of its videos’ scientific reliability and credibility are open to question (3), YouTube, as a free access video-sharing site, is one of the most visited platforms by professionals and lay people (4). A total of 100 million videos are viewed, and >65.000 videos are uploaded on YouTube everyday (1). Google, Facebook, and YouTube are the most frequently visited websites, respectively (2).
The public’s comprehension on medicine, illness, and death is strongly shaped by media images since the 1990’s. Since then, there has always been incorrect and deceptive information on media (5). After many years, the reliability and the quality of the information on the internet are also open to question in this digital age that we are living in (2).
The videos on YouTube could be both educational and aimed at entertainment; however, they do not have a scientific peer-review process or standardized methodology for acception (4). Owing to consumer-generated strategy, the lack of peer review and detailed evaluation of the uploaded data on YouTube can result in spreading of misinformation (6).
Various studies analyzed the quality and the content of YouTube videos on various healthcare issues in the literature (1, 7–11). The aim of the present study was to analyze the reliability and the quality of the most-viewed videos on YouTube “related directly with orthodontics,” by using the keywords “Orthodontics,” “Orthodontist,” and “Orthodontic Treatment.” These keywords were found to be searched frequently on Google about orthodontics by lay people, by using Google Trends.
METHODS
As the paper does not deal with humans or any material previously collected from humans, no ethical approval was taken.
Video Selection
YouTube (www.youtube.com) was searched for three keywords: “Orthodontics,” “Orthodontic Treatment,” and “Orthodontist” on July 8, 2018. These keywords were found to be searched frequently on Google about orthodontics by lay people, by using Google Trends application. Google Trends is an online search tool that analyzes a given search term that is entered into Google’s search engine relative to the total search volume. The search results were 267.000 videos in total for “Orthodontics,” 61.700 videos in total for “Orthodontic Treatment,” and 141.000 videos in total for “Orthodontist.”
Exclusion criteria were videos in languages other than English, videos >10 min, videos with no sounds or visuals, duplicate videos, and not-related videos.
The remaining videos excluded from exclusion criteria were considered as suitable videos (Figure 1).
Figure 1.
The distribution of the included (suitable) and excluded (unsuitable) videos according to the groups
The YouTube account of one of the researchers was used for the present study. All related video links were sorted by “sort by view-count” with no additional filters.
In a recent study, it was mentioned that 95% of people were viewing only the first 60 videos of an online search (4). We preferred to use the research method by Desai et al. (4), and the top 60 videos (first 3 pages) were assessed according to this criteria.
Two independent researchers, GS and DDK, both orthodontists with 10 years of experience viewed and assessed videos for reliability of the content and quality of the videos.
Assessment of Videos
After the exclusion of unsuitable videos from the results according to the exclusion criteria, all of the remaining suitable videos were analyzed and scored from 1 to 5 (Reliability Score) for content, accuracy, and reliability by using a 5-point scale (Figure 2). This scale was based on five questions that were derived from DISCERN tool, which is a tool used for assessment of written health information (12).
Figure 2.
Reliability Score (adapted from DISCERN) and Global Quality Score (GQS) on a 5-point scale
To assess the quality of the videos, Global Quality Scale (GQS) was used to rate the general quality of the videos (13). This rating was also made by using a 5-point scale (Figure 2). The scoring system was based on the usefulness and general concern of the video to the patient who would watch the video.
Statistical Analysis
The Statistical Package for Social Sciences version 23 (IBM Corp., Armonk, NY, USA) was used for data entry and analysis. Kolmogorov-Smirnov test was used in the assessment of the conformity of the data for normal distribution. Kruskal-Wallis test was used for assessment of non-normally distributed variables. Intraclass Correlation Coefficient (ICC) analysis and Kappa test were used for assessment of inter-examiner concordance. Qualitative data were presented as median (min–max), and quantitative data were presented as frequency (%). A p-value <0.05 was considered as significant.
RESULTS
Two researchers scored the videos from 1 to 5 for GQS (Table 1). The correlation between the researchers was evaluated by ICC analysis. According to the results, there was a strong positive correlation between the researchers in all of the groups.
Table 1.
The frequency distribution of the GQS values according to the groups
| Score | Orthodontist | Orthodontics | Orthodontic treatment | |||
|---|---|---|---|---|---|---|
|
|
|
|
||||
| GS (Researcher 1) | DDK (Researcher 2) | GS | DDK | GS | DDK | |
| 1 | 9 (29) | 11 (35.5) | 3 (7.1) | 6 (14.3) | 9 (25.7) | 13 (37.1) |
| 2 | 5 (16.1) | 3 (9.7) | 10 (23.8) | 9 (21.4) | 13 (37.1) | 9 (25.7) |
| 3 | 8 (25.8) | 7 (22.6) | 11 (26.2) | 9 (21.4) | 7 (20) | 8 (22.9) |
| 4 | 5 (16.1) | 9 (29) | 7 (16.7) | 10 (23.8) | 2 (5.7) | 3 (8.6) |
| 5 | 4 (12.9) | 1 (3.2) | 11 (26.2) | 8 (19) | 4 (11.4) | 2 (5.7) |
| ICC | 0.905 (0.803–0.954) | 0.923 (0.857–0.959) | 0.941 (0.883–0.970) | |||
GQS: global quality score; ICC; intraclass correlation coefficient
The mean values for the GQS score, which was obtained by measuring the mean values of two researchers, were 2.6 in the “Orthodontist” group, 3.2 in the “Orthodontics” group, and 2.3 in the “Orthodontic Treatment” group, respectively (Table 2). The mean value was 2.8 without any group exception for all of the videos.
Table 2.
The descriptive statistics of the GQS values according to the groups
| Mean±SD | Median (min-max) | |
|---|---|---|
| Orthodontist | 2.6±1.3 | 2.5 (1–5) |
| Orthodontics | 3.2±1.3 | 3.3 (1–5) |
| Orthodontic treatment | 2.3±1.2 | 2.0 (1–5) |
| Total | 2.8±1.3 | 2.5 (1–5) |
GQS: global quality score
Five questions were scored in the reliability scale (Table 3). The best concordance between the researchers was in the “Orthodontist” group while evaluated for “Are the aims clear and achieved?” question. The median values of GQS differed according to the groups (p=0.007) (Table 4). There was no difference between the medians of reliability values according to the groups for GS (p=0.386) (Table 5). There was no difference between the medians of reliability values according to the groups for DDK (p=0.187). There was no difference between the median of the mean values of both researchers’ scores according to the groups (p=0.303).
Table 3.
The frequency distribution of the reliability values according to the groups
| Reliability questions | Researcher | Score | Orthodontist | Orthodontics | Orthodontic treatment |
|---|---|---|---|---|---|
| Are the aims clear and achieved? | GS | 0 | 9 (29) | 3 (7.1) | 4 (11.4) |
| 1 | 22 (71) | 39 (92.9) | 31 (88.6) | ||
| DDK | 0 | 10 (32.3) | 6 (14.3) | 8 (22.9) | |
| 1 | 21 (67.7) | 36 (85.7) | 27 (77.1) | ||
| K value | 0.773* | 0.632* | 0.607* | ||
| Reliable sources info? | GS | 0 | 10 (32.3) | 15 (35.7) | 15 (42.9) |
| 1 | 21 (67.7) | 27 (64.3) | 20 (57.1) | ||
| DDK | 0 | 20 (64.5) | 20 (47.6) | 24 (68.6) | |
| 1 | 11 (35.5) | 22 (52.4) | 11 (31.4) | ||
| K value | 0.415** | 0.566** | 0.512** | ||
| Balanced and unbiased? | GS | 0 | 11 (35.5) | 23 (54.8) | 24 (68.6) |
| 1 | 20 (64.5) | 19 (45.2) | 11 (31.4) | ||
| DDK | 0 | 20 (64.5) | 27 (64.3) | 26 (74.3) | |
| 1 | 11 (35.5) | 15 (35.7) | 9 (25.7) | ||
| K value | 0.464** | 0.510** | 0.721* | ||
| Additional sources info? | GS | 0 | 19 (61.3) | 28 (66.7) | 28 (80) |
| 1 | 12 (38.7) | 14 (33.3) | 7 (20) | ||
| DDK | 0 | 22 (73.3) | 28 (66.7) | 29 (82.9) | |
| 1 | 8 (26.7) | 14 (33.3) | 6 (17.1) | ||
| K value | 0.619* | 0.571** | 0.717* | ||
| Are areas of uncertainty mentioned? | GS | 0 | 22 (71) | 30 (71.4) | 29 (82.9) |
| 1 | 9 (29) | 12 (28.6) | 6 (17.1) | ||
| DDK | 0 | 24 (77.4) | 33 (78.6) | 32 (91.4) | |
| 1 | 7 (22.6) | 9 (21.4) | 3 (8.6) | ||
| K value | 0.665* | 0.559** | 0.624* |
K: kappa coefficent
There is good concordance between the researchers.
There is moderate concordance between the researchers.
Table 4.
Comparison of GQS scores according to the groups
| Median (min–max) | p | |
|---|---|---|
| Orthodontist | 2.5 (1–5) ab | |
| Orthodontics | 3.3 (1–5) a | 0.007 |
| Orthodontic treatment | 2 (1–5) b |
GQS, Global Quality Score.
Kruskal Wallis,
There is no difference between the same lettered groups.
Table 5.
Comparison of Reliability Scores according to the groups.
| Reliability (GS) Median (min–max) |
Reliability (DDK) Median (min–max) |
Overall rating Median (min–max) |
|
|---|---|---|---|
| Orthodontist | 3 (0–5) | 1 (0–5) | 2 (0–5) |
| Orthodontics | 2 (0–5) | 2 (0–5) | 2 (0–5) |
| Orthodontic treatment | 2 (0–5) | 1 (0–5) | 1.5 (0–5) |
| Total | 2 (0–5) | 1 (0–5) | 2 (0–5) |
| p | 0.386 | 0.187 | 0.303 |
Kruskal-Wallis
There was no difference between the median total views according to the groups (p=0.050) (Table 6). There was a difference between the median video durations according to the groups (p=0.016). There was a difference between the “Orthodontist” group and the “Orthodontic Treatment” group.
Table 6.
Comparison of total views, video duration, and likes and dislikes according to the groups
| Total views Median (min–max) |
Video durations (mins) Median (min–max) |
Likes Median (min–max) |
Dislikes Median (min–max) |
|
|---|---|---|---|---|
| Orthodontist | 25,049 (606–1,044,537) | 3.4 (1–9.5) a | 85 (8–4002) a | 9 (0–295) |
| Orthodontics | 25,548 (256–1,622,070) | 2.4 (0.3–9.2) ab | 71.5 (0–5397) ab | 5.5 (0–417) |
| Orthodontic treatment | 6689 (13–40,510,079) | 1.5 (0.5–9.6) b | 33 (0–77,908) b | 3 (0–15,272) |
| Total | 22,976 (13–40,510,079) | 2.3 (0.3–9.6) | 56.5 (0–77,908) | 6 (0–15,272) |
| p-Value | 0.050 | 0.016 | 0.016 | 0.065 |
There is no difference between the same lettered groups.
There was a difference between the median likes according to the groups (p=0.016). There was no difference between the median dislikes according to the groups (p=0.065).
DISCUSSION
As being a free access video-sharing site, YouTube is one of the most popular social media platforms comprising videos on the diagnosis, treatment, and prevention of illnesses (6). However, the quality of the information included by YouTube is still questionable (2). Knösel and Jung (14) reported that there is a large range of data on orthodontics on YouTube, and that the biggest part of these videos is uploaded by orthodontic patients. Some researchers stated that because of self-anecdotal reporting and personal opinions, the quality and reliability of videos on YouTube are suspicious (6, 15, 16). Additionally, the authors uploading the videos on YouTube are not directed for a scientific peer-review process, not asked for source of their videos, not required to report the currency of their videos, and not asked to update their videos in time (4, 6, 8).
Previous investigations proved the contrast between the quality of the videos and user interest (8, 9, 11). In their study, Singh et al. (6) used the DISCERN assessment tool to assess the content of videos, whereas Singh et al. (6) and Bernard et al. (13) used GQS to assess the quality of the videos. In the present study, DISCERN questionnaire was used to evaluate the reliability of the videos, and GQS was used to assess the overall quality of the videos.
In the present study, the search results were 267.000 videos in total for “Orthodontics,” 61,700 videos in total for “Orthodontic Treatment,” and 141.000 videos in total for “Orthodontist.” The term “Orthodontics” is found to be more searched because it is thought to include both the terms “Orthodontist” and “Orthodontic Treatment.” In their study, Murigiah et al. (11) categorized the videos on YouTube into three groups as “useful,” “misleading,” and “patient views.” In our study, videos were not categorized into these groups for an objective evaluation and to avoid bias. Videos with no access, with no sound or visual, >10 min, and in other languages than English were assessed as “unsuitable” videos for the present study. Overall, 29 (48.3%) of the 60 videos in the “Orthodontist” group, 18 (30%) of the 60 videos in the “Orthodontics” group, and 25 (41.7%) of the 60 videos in the “Orthodontic Treatment” group were not evaluated because of being unsuitable. The remaining videos in each group were evaluated as “suitable” for the study and were assessed for quality and reliability.
In their study, Desai et al. (4) selected the videos according to the sources and YouTube channels by which the videos were uploaded. However, Fox (17) stated that 75% of people using to reach medical data on the internet never inspect the information source. In the present study, all of the videos on the first three pages of all groups were evaluated to assess the overall data on orthodontics on a single YouTube search for a point in time, to achieve a more objective assessment. As a result of this, data were not evaluated according to the uploading channels or sources.
In previous studies, which assessed the content, reliability, and quality of the videos on YouTube on different medical topics, only 48% of immunization videos, 61% of H1N1 videos, and 58% of kidney stone videos were found to be useful (6).
Contrary to this, it was found that 32% of immunization videos, 23% of H1N1 videos, and 18% of kidney stone videos were spreading misinformation (6). In a very recent study, in 2018, Olkun and Ari Demirkaya (18) examined websites about lingual orthodontics and found the quality of the information on the websites to be low. Canigur Bavbek and Tuncer Balos (19) evaluated the Turkish websites about orthognathic surgery in a similar way to the method that we used in our study and found the overall quality of the scientific content of the websites at low or medium level. In their study, Lena and Dindaroğlu (20) found the content of the YouTube™ videos on lingual orthodontics to be incomplete. They mentioned that orthodontists should be aware of the information on YouTube™. In accordance with all of these results, the overall quality of the videos assessed in the present study was found to be average, and the reliability of the videos was low-grade. It was found that the reliability and the quality of the videos on YouTube about orthodontics are mostly of poor quality and unreliable.
Despite the recent tendency of academic institutions and journals to constitute their own educative YouTube channels, in a previous study, the researchers stated that the healthcare authorities and organizations issued a small number of highly educational and/or suitable medical videos (21). Only 27% of videos were found to be highly educational among medical videos (4, 8). However, it was shown that lay people are less interested in highly educational videos (4).
Hegarty et al. (2) stated that healthcare professionals should provide more information to social media resources, such as Google and YouTube, thus preventing misinformation of the community. Canigur Bavbek and Tuncer Balos mentioned that professional institutions, such as universities and educational institutions, which provide information to the community without the expectation of earnings, can overcome the lack of reliable information in this field (19).
The limitations of our study were: it was constructed only on YouTube videos not sorting any other healthcare sites; it was based on English language videos, but there were also local language videos uploaded on orthodontics; it was made in a single sort of the site in a particular time, but the content of the websites changes every second; and the assessment was made by professionals so the opinion of the public may have been dismissed.
CONCLUSION
The reliability of the videos assessed in the present study was found to be low-grade. The overall quality of the videos assessed in the present study was found to be average. Despite the fact that all of the information about orthodontics on YouTube is not accurate, YouTube can help to increase awareness about orthodontic treatments. Therefore, orthodontists should pay attention to inadequate data guiding patients on YouTube.
Footnotes
Ethics Committee Approval: As the paper does not deal with humans or any material previously collected from humans, no ethical approval was taken.
Peer-review: Externally peer-reviewed.
Author Contributions: Concept - D.D.K.; Design - D.D.K., G.S.; Supervision - D.D.K., G.S.; Data Collection and/or Processing - D.D.K., G.S.; Analysis and/or Interpretation - D.D.K., G.S.; Literature Search - D.D.K., G.S.; Writing Manuscript - D.D.K., G.S.; Critical Review - D.D.K., G.S.
Conflict of Interest: The authors have no conflict of interest to declare.
Financial Disclosure: The authors declared that this study has received no financial support.
REFERENCES
- 1.Hassona Y, Taimeh D, Marahleh A, Scully C. YouTube as a source of information on mouth (oral) cancer. Oral Dis. 2016;22:202–8. doi: 10.1111/odi.12434. [DOI] [PubMed] [Google Scholar]
- 2.Hegarty E, Campbell C, Grammatopoulos E, DiBiase AT, Sherriff M, Cobourne MT. YouTube™ as an information resource for orthognathic surgery. J Orthod. 2017;44:90–6. doi: 10.1080/14653125.2017.1319010. [DOI] [PubMed] [Google Scholar]
- 3.ElKarmi R, Hassona Y, Taimeh D, Scully C. YouTube as a source for parents’ education on early childhood caries. Int J Paediatr Dent. 2016;27:437–43. doi: 10.1111/ipd.12277. [DOI] [PubMed] [Google Scholar]
- 4.Desai T, Shariff A, Dhingra V, Minhas D, Eure M, Kats M. Is content really king? An objective analysis of the public’s response to medical videos on YouTube. PLoS One. 2013;8 doi: 10.1371/journal.pone.0082469. doi: 10.1371/journal.pone.0082469. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Diem SJ, Lantos JD, Tulsky JA. Cardiopulmonary resuscitation on television. Miracles and misinformation. N Engl J Med. 1996;334:1578–82. doi: 10.1056/NEJM199606133342406. [DOI] [PubMed] [Google Scholar]
- 6.Singh AG, Singh S, Singh PP. YouTube for information on rheumatoid arthritis-a wakeup call? J Rheumatol. 2012;39:899–903. doi: 10.3899/jrheum.111114. [DOI] [PubMed] [Google Scholar]
- 7.Keelan J, Pavri-Garcia V, Tomlinson G, Wilson K. YouTube as a source of information on immunization: a content analysis. JAMA. 2007;298:2482–4. doi: 10.1001/jama.298.21.2482. [DOI] [PubMed] [Google Scholar]
- 8.Steinberg PL, Wason S, Stern JM, Deters L, Kowal B, Seigne J. YouTube as source of prostate cancer information. Urology. 2010;75:619–22. doi: 10.1016/j.urology.2008.07.059. [DOI] [PubMed] [Google Scholar]
- 9.Pandey A, Patni N, Singh M, Sood A, Singh G. YouTube as a source of information on the H1N1 influenza pandemic. Am J Prev Med. 2010;38:1–3. doi: 10.1016/j.amepre.2009.11.007. [DOI] [PubMed] [Google Scholar]
- 10.Sood A, Sarangi S, Pandey A, Murugiah K. YouTube as a source of information on kidney stone disease. Urology. 2011;77:558–62. doi: 10.1016/j.urology.2010.07.536. [DOI] [PubMed] [Google Scholar]
- 11.Murigiah K, Vallakati A, Rajput K, Sood A, Challa NR. YouTube as a source of information on cardiopulmonary resuscitation. Resuscitation. 2011;82:332–4. doi: 10.1016/j.resuscitation.2010.11.015. [DOI] [PubMed] [Google Scholar]
- 12.Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999;53:105–11. doi: 10.1136/jech.53.2.105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Bernard A, Langille M, Hughes S, Rose C, Leddin D, Veldhuyzen van Zanten S. A systematic review of patient inflammatory bowel disease information resources on the World Wide Web. Am J Gastroenterol. 2007;102:2070–7. doi: 10.1111/j.1572-0241.2007.01325.x. [DOI] [PubMed] [Google Scholar]
- 14.Knösel M, Jung K. Informational value and bias of videos related to orthodontics screened on a video-sharing Web site. Angle Orthod. 2011;81:532–9. doi: 10.2319/091710-541.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Briones R, Nan X, Madden K, Waks L. When vaccines go viral: an analysis of HPV vaccine coverage on YouTube. Health Commun. 2012;27:478–85. doi: 10.1080/10410236.2011.610258. [DOI] [PubMed] [Google Scholar]
- 16.Lewis SP, Heath NL, Sornberger MJ, Arbuthnott AE. Helpful or harmful? An examination of viewers’ responses to nonsuicidal self-injury videos on YouTube. J Adolesc Health. 2012;51:380–5. doi: 10.1016/j.jadohealth.2012.01.013. [DOI] [PubMed] [Google Scholar]
- 17.Fox S. Online health search, Pew Internet & American Life Project. 2006. p. iii. [Google Scholar]
- 18.Olkun HK, Demirkaya AA. Evaluation of Internet Information about Lingual Orthodontics Using DISCERN and JAMA Tools. Turk J Orthod. 2018;31:50–4. doi: 10.5152/TurkJOrthod.2018.17042. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Bavbek NC, Tuncer BB. Information on the Internet Regarding Orthognathic Surgery in Turkey: Is It an Adequate Guide for Potential Patients? Turk J Orthod. 2017;30:78–83. doi: 10.5152/TurkJOrthod.2017.17027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Lena Y, Dindaroğlu F. Lingual orthodontic treatment: A YouTube™ video analysis. Angle Orthod. 2017;88:208–214. doi: 10.2319/090717-602.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Knösel M, Jung K, Bleckmann A. YouTube, dentistry, and dental education. J Dent Educ. 2011;75:1558–68. doi: 10.4103/0974-7761.103680. [DOI] [PubMed] [Google Scholar]