Abstract
Introduction:
Many patients conduct internet searches to manage their own health problems, to decide if they need professional help, and to corroborate information given in a clinical encounter. Good information can improve patients’ understanding of their condition and their self-efficacy. Patients with spinal cord injury (SCI) featuring neurogenic bladder (NB) require knowledge and skills related to their condition and need for intermittent catheterization (IC).
Methods:
Information quality was evaluated in videos accessed via YouTube relating to NB and IC using search terms “neurogenic bladder intermittent catheter” and “spinal cord injury intermittent catheter.” Video content was independently rated by 3 investigators using criteria based on European Urological Association (EAU) guidelines and established clinical practice.
Results:
In total, 71 videos met the inclusion criteria. Of these, 12 (17%) addressed IC and 50 (70%) contained information on NB. The remaining videos met inclusion criteria, but did not contain information relevant to either IC or NB. Analysis indicated poor overall quality of information, with some videos with information contradictory to EAU guidelines for IC. High-quality videos were randomly distributed by YouTube. IC videos featuring a healthcare narrator scored significantly higher than patient-narrated videos, but not higher than videos with a merchant narrator. About half of the videos contained commercial content.
Conclusions:
Some good-quality educational videos about NB and IC are available on YouTube, but most are poor. The videos deemed good quality were not prominently ranked by the YouTube search algorithm, consequently user access is less likely. Study limitations include the limit of 50 videos per category and the use of a de novo rating tool. Information quality in videos with healthcare narrators was not higher than in those featuring merchant narrators. Better material is required to improve patients’ understanding of their condition.
Introduction
The Internet is ubiquitous, most particularly social media. Given the popularity and use of the Internet as a source of information, it follows that users frequently consult familiar sources when looking for health information.1 According to a 2011 survey, 80% of Internet users look online for information on health topics.2 A high proportion of patients state that Internet-derived information empowers them to communicate with their physicians and helps them make health decisions1 and nearly half will consult Internet sources before their physicians on health-related questions.3 Consequently, physicians should be aware of the quality and veracity of the information their patients are acquiring online.
Patients conduct Internet searches to manage their own health concerns, to decide if they need professional help, and to corroborate information obtained in clinical encounters. Good information can improve patients’ understanding of their condition and their self-efficacy.1 We explored the information available via YouTube videos on intermittent catheterization (IC) in the context of spinal cord injury (SCI). The prevalence of patients with SCI in Canada was 85 556 in 2010, with 4259 new cases per year.4 Of these, 70% to 84% will develop a neurogenic bladder (NB).5 According to the European Association of Urology (EAU), the gold standard in the management of NB patients who cannot empty their bladders is intermittent catheterization (IC).6 Catheterization and self-catheterization can be difficult to master, and patients could benefit greatly from related quality online video resources.7
YouTube, a video-sharing Internet website started in 2005, receives the most traffic of all video websites on the Internet and the third-most of all websites;8 therefore, this platform was the one used in our study. The purpose was to conduct an integrative review the quality of information currently available on YouTube videos on IC in the context of NB after SCI.
Methods
Search strategy
On June 29, 2014, a YouTube search was conducted using the following 2 search terms: “neurogenic bladder intermittent catheter” and “spinal cord injury intermittent catheter.” After setting the YouTube search tool to sort results from most to least relevant, the first 50 results for each search term were collected. Videos were excluded if they were longer than 10 minutes or if they lacked narration in English or English subtitles. If videos appeared under both search terms, they were assigned to the search term under which they appeared closest to the top of the ranking list, and deleted from the other search term’s results. After removal of duplicate and excluded videos, 71 videos remained. We recorded each video’s rank or numeric position in the YouTube-generated list of results.
Content evaluation
Each video was viewed and the content independently rated by 3 investigators who each completed a written questionnaire (Appendix 1). The questionnaire assessed the following features:
The background of the video’s main narrator
The declared or apparent purpose of the video
The relevance of content to the topic of urethral IC
The quality of the description of the process for clean IC
The comprehensiveness/accuracy of the IC procedure depicted
The quality/accuracy of information on NB
The presence and extent of catheter marketing
Topic D was scored on a 10-point scale, with points awarded for inclusion of statements derived from the EAU guidelines on the clean technique for IC and management of NB.7 Topic F was also scored on a 10-point scale rating the inclusion of statements about the consequences and management of neurogenic bladder based on established best clinical practices.
Evaluation process and analysis
The three investigators were briefed on the questionnaire. Each of them independently viewed the videos in randomized order and rated the videos. They later reviewed the evaluation data jointly. If there was disagreement on a particular question, the response entered for analysis reflected the greatest degree of consensus; if there was no consensus, a rating of “indeterminate” was assigned. Inter-rater reliability of the questionnaire responses was analyzed via Rust and Cooil’s Proportional Reduction in Loss method,9 with the reliability of each question averaged across all 71 videos rated. Data were then summarized and significance was tested via t-test.
Results
In total, 71 videos met the eligibility criteria for this study. Twelve videos (17%) addressed the process of IC (Table 1). We tallied the ratings for the quality of information of the videos; the mean score was 5.4/10 (Table 2). The videos ranked by the YouTube search tool for relevance from 1 to 25 had a mean score that was not significantly different to those that appeared in from 26 to 50 (5.3 vs. 5.5, p = 0.91) (Table 3). Videos featuring IC instruction by a healthcare professional scored significantly higher than those narrated by a patient (7.1 vs. 2.2, p = 0.02), but did not score higher than videos with instruction from a merchant (7.1 vs. 7.0, p = 0.89). There was no difference between the scores of videos that depicted male versus female IC (5.4 vs. 6.2, p = 0.65). The mean inter-rater reliability was 0.9 to 1.0, except for questions 15, 23, 25, 28 and 29, which ranged between 0.74 and 0.89.
Table 1.
Summary of general video information
| Category | No. videos (%) |
|---|---|
| Relevance | |
| Described process of IC | 12 (17) |
| Total relevance to IC | 29 (41) |
| Relevant to other form of catheterization | 8 (11) |
| Relevant to urology | 17 (24) |
| Relevant to SCI | 13 (18) |
| Not relevant to any of above | 1 (1) |
| Indeterminate* | 4 (6) |
| Narrator | |
| Healthcare | 29 (41) |
| Patient | 13 (18) |
| Merchant | 17 (24) |
| News agency | 3 (4) |
| Uncertain | 5 (7) |
| Indeterminate* | 4 (6) |
IC: intermittent catheterization; SCI: spinal cord injury.
A video was classified as indeterminate if there was no consensus among reviewers.
Table 2.
Scoring characteristics of videos on IC information
| Video category | No. videos | Mean IC score/10 (95% CI) |
|---|---|---|
| All videos with IC information | 12 | 5.4 (3.9, 7.0) |
| Ranking | ||
| Ranked 1–25 | 5 | 5.3 (3.6, 7.1) |
| Ranked 26–50 | 7 | 5.5 (3.1, 8.0) |
| Type of narrator | ||
| Healthcare | 5 | 7.1 (6.1, 8.1) |
| Patient | 2 | 2.2 (0.0, 6.4) |
| Merchant | 2 | 7.0 (7.0, 7.0) |
| Unknown | 2 | 2.0 (1.3, 2.7) |
| Indeterminate | 1 | 7.33 |
| Male-female split | ||
| Videos depicting male IC | 6 | 5.4 (3.1, 7.8) |
| Videos depicting female IC | 5 | 6.2 (4.1, 8.3) |
| Videos depicting IC with no specific gender | 1 | 4.7 (4.7, 4.7) |
IC: intermittent catheterization; CI: confidence interval.
Table 3.
Significance testing for IC information
| Comparison 1 | Comparison 2 | p value |
|---|---|---|
| Rank 1–25 | Rank 26–50 | 0.91 |
| Healthcare source | Patient source | 0.02 |
| Healthcare source | Mercantile source | 0.89 |
| Patient source | Mercantile source | 0.16 |
| Male IC | Female IC | 0.65 |
IC: intermittent catheterization.
The criterion that videos most commonly neglected to address was “note volume of urine collected;” 5 of the 12 (42%) contained information that directly contradicted the EAU guidelines for IC. The most common contradictory information was reuse of a catheter, which appeared in 3 of these 5 videos. The second most common was improper cleansing technique of the urethra and surrounding area, which appeared in 2 of 5 videos.
Of the 71 videos, 50 (70%) contained information on NB. The mean score for quality of NB information was 1.5/10. Videos ranked 1 to 25 by YouTube did not score higher than those ranked 26 to 50 (1.7 vs. 1.2, p = 0.30) (Table 4). Videos primarily narrated by healthcare professionals did not score higher than those narrated by patients (2.1 vs. 1.1, p = 0.10), but did score higher than videos narrated by merchants (2.1 vs. 0.6, p = 0.04). There was no significant difference between videos with patient and merchant narrators (1.1 vs. 0.6, p = 0.19) (Table 5).
Table 4.
Scoring characteristics of videos on NB information
| Video category | No. videos | Mean NB score/10 (95% CI) |
|---|---|---|
| All videos with NB information | 50 | 1.5 (1.1, 1.9) |
| Ranking | ||
| Ranked 1–25 | 32 | 1.7 (1.1, 2.3) |
| Ranked 26–50 | 18 | 1.2 (0.6, 1.7) |
| Type of narrator | ||
| Healthcare | 25 | 2.1 (1.4, 2.8) |
| Patient | 10 | 1.1 (0.5, 1.6) |
| Merchant | 8 | 0.6 (0.4, 0.8) |
| News agency | 1 | 3.0 |
| Unknown | 4 | 0.4 (0.3, 0.5) |
| Indeterminate | 2 | 1.0 (0.5, 1.5) |
NB: neurogenic bladder; CI: confidence interval.
Table 5.
Significance testing for NB information
| Comparison 1 | Comparison 2 | p value |
|---|---|---|
| Rank 1–25 | Rank 26–50 | 0.30 |
| Healthcare narrator | Patient narrator | 0.10 |
| Healthcare narrator | Mercantile narrator | 0.04 |
| Patient narrator | Mercantile narrator | 0.19 |
NB: neurogenic bladder.
Of the 71 videos, 43 (61%) mentioned or depicted catheters; 20 (46%) referred to a particular catheter product or company; and 11 (15%) advertised a specified feature claimed to make the catheter superior to that of their competitors. No videos mentioned the cost of purchasing the advertised catheters (Table 6).
Table 6.
Video marketing characteristics
| Category | No. videos (%) |
|---|---|
| Video mentioned or depicted catheters | 43 (61) |
| No. of the videos mentioning or depicting catheters | |
| Advertised at least 1 product or category | 20 (46) |
| Claimed superiority to other products based on a particular catheter feature | 11 (15) |
| Advertised the specific cost of the catheter | 0 (0) |
Discussion
We evaluated YouTube videos containing information relating to NB and IC. Our analysis indicated that overall the quality of information is poor, misleading or irrelevant, with a few videos containing high-quality information. This phenomenon of overall poor quality is echoed in other studies of health information on social networks, particularly YouTube, with regards to urological10 and non-urological11–14 topics. This highlights a challenge for healthcare providers caring for patients and families who could benefit from accessing NB and IC information on the Internet.
Success can be achieved with online health education where good quality information is used.15 Internet-based applications are proving to be novel, convenient and affordable mechanisms for health education. Even in the developing world, positive experiences are widely reported. Traditionally problematic topics for health promotion have been addressed successfully, with knowledge about sexual health and access to sexual health services improved among adolescents in many countries, better management of diabetics achieved, and continuing education for midwives delivered effectively.15–17
Patients with NB and those requiring IC represent a significant population and yet the available online resources require improvement. Only 17% of videos contained a description of the IC process and only half scored 7 or above when graded for content based the EUA recommendations for IC. More videos contained NB information (70%), but the overall quality of the information was poor (mean score 1.5/10) and only 1 video scored over 7/10.
In this study, the quality of videos was compared to their rank based on the YouTube search-ranking tool, from most to least relevant. How high a video is ranked is relevant to potential educational impact, as users are more likely to select and view videos that appear earlier in the list. This is partly out of convenience and partly out of perceived significance. However, for both IC and NB, the videos that appeared in the top half of the YouTube results list did not contain significantly higher-quality information than those that appeared in the bottom half. For example, the video that contained the best IC information was ranked 33 out of 50, with 3 of the 4 next-highest scoring ranked 27 or lower. This system of ranking can compromise effective sourcing of accurate, higher-quality resources on IC and NB by patients and practitioners.18 Both patients and practitioners need to be made aware that rank does not equal quality. Practitioners should be prepared to direct patients who have gleaned incomplete or incorrect information from the Internet to material that does have good educational quality.
The video presenter is relevant. Although videos narrated by healthcare providers had the highest mean quality scores for both IC and NB, the information contained was not always of significantly higher quality than those narrated by patients, or merchants. Some healthcare-narrated videos achieved alarmingly low scores. This is potentially problematic because a viewer with limited knowledge of the topic could be more inclined to trust a video that featured a healthcare narrator.
A secondary objective of this study was to evaluate whether marketing practices existed and their impact on the overall message of the videos. Almost half the videos that mentioned catheters named a specific product or company, and 15% cited a particular feature as an advantage of a certain product (e.g., a hydrophilic catheter designed for use by people with limited hand function). The effect of such commercialization on IC is uncertain, but from an educational standpoint it is suboptimal that nearly half featured an advertisement.
Our study has its limitations. The cut-off for 50 videos in each category was selected based on a realistic number that an individual would logically search, and not on a formal power calculation, thus statistical inferences from the data are limited. The metrics and criteria by which we evaluated video quality were self-selected as no validated tool was available. Some reservations exist regarding the results as this is a de novo tool, although we based the evaluation criteria for IC on EUA recommendations and the NB criteria were generated based on what the authors agreed an informative video should include. We also only included two search terms, choosing those we thought patients most likely to use; however, it became clear during the study that even small variations in the search term would induce some change in the content and order of the results YouTube generated. Hence, the list of videos generated for our sample, although representative, may not have captured the full diversity of videos with IC and NB content that can be sourced on YouTube.
Importantly, it is evident that both the quality of information available in YouTube videos on IC and NB and the way in which search engines prioritize this material when patients are seeking to learn via the Internet warrants improvement.
Conclusion
Some good-quality and educational videos about NB and IC are available on YouTube, but most are incomplete, uninformative, or misleading. Videos we evaluated to have good or high quality content were not prominently positioned in the rank order displayed by the YouTube search algorithm, consequently user access to them is less likely. The quality of information in videos with healthcare narrators was not consistently higher than in those featuring patient or merchant narrators. About half of videos featuring catheter use also contained some kind of advertisement.
Appendix 1. Video evaluation questionnaire
| Basic information |
|---|
Select one answer for the following questions:
|
Footnotes
Competing interests: The authors declare no competing financial or personal interests.
This paper has been peer-reviewed.
References
- 1.McMullan M. Patients using the Internet to obtain health information: How this affects the patient–health professional relationship. Patient Educ Couns. 2006;63:24–8. doi: 10.1016/j.pec.2005.10.006. [DOI] [PubMed] [Google Scholar]
- 2.Fox S. The Social Life of Health Information, 2011. Washington, D.C.: Pew Research Centre; 2011. http://www.chcact.org/resources/PIP_Social_Life_of_Health_Info.pdf. Accessed September 14, 2015. [Google Scholar]
- 3.Hesse BW, Nelson DE, Kreps GL, et al. Trust and sources of health information: The impact of the internet and its implications for health care providers: Findings from the First Health Information National Trends Survey. Arch Intern Med. 2005;165:2618–24. doi: 10.1001/archinte.165.22.2618. [DOI] [PubMed] [Google Scholar]
- 4.Farry A, Baxter D. The Incidence and Prevalence of Spinal Cord Injury in Canada: Overview and estimates based on current evidence. Vancouver, BC: Rick Hansen Institute; 2010. http://fecst.inesss.qc.ca/fileadmin/documents/photos/LincidenceetlaprevalencedestraumamedullaireauCanada.pdf. Accessed September 14, 2015. [Google Scholar]
- 5.Ginsberg D. The epidemiology and pathophysiology of neurogenic bladder. Am J Manag Care. 2013;19:S191–6. [PubMed] [Google Scholar]
- 6.Pannek J, Blok B, Castro-Diaz D, et al. Guidelines on Neuro-Urology. Arnhem, Netherlands: European Association of Urology; 2014. http://uroweb.org/guideline/neuro-urology/. Accessed September 14, 2015. [Google Scholar]
- 7.Vahr S, Cobussen-Boekhorst H, Eikenboom J, et al. Evidence-based Guidelines for Best Practice in Urological Health Care: Catheterisation: Urethral intermittent in adults, Dilation, urethral intermittent in adults. Arnhem, Netherlands: European Association of Urology Nurses; 2013. http://nurses.uroweb.org/guideline/catheterisation-urethral-intermittent-in-adults/. Accessed September 14, 2015. [Google Scholar]
- 8. Alexa.com Alexa Internet, Inc.; c1996–2015. http://www.alexa.com/topsites. Accessed September 14, 2015.
- 9.Rust RT, Cooil B. Reliability Measures for qualitative data: Theory and implications. J Mark Res. 1994;31:1–14. doi: 10.2307/3151942. [DOI] [Google Scholar]
- 10.Sajadi KP, Goldman HB. Social networks lack useful content for incontinence. Urology. 2011;78:764–7. doi: 10.1016/j.urology.2011.04.074. [DOI] [PubMed] [Google Scholar]
- 11.Yaylaci S, Serinken M, Eken C, et al. Are YouTube videos accurate and reliable on basic life support and cardiopulmonary resuscitation? Emerg Med Australas. 2014;264:474–7. doi: 10.1111/1742-6723.12274. [DOI] [PubMed] [Google Scholar]
- 12.MacLeod MG, Hoppe DJ, Simunovic N, et al. YouTube as an information source for femoroacetabular impingement: A systematic review of video content. Arthroscopy. 2015;31:136–42. doi: 10.1016/j.arthro.2014.06.009. [DOI] [PubMed] [Google Scholar]
- 13.Knight E, Intzandt B, MacDougall A, et al. Information seeking in social media: A review of YouTube for sedentary behavior content. Interact J Med Res. 2015;4:e3. doi: 10.2196/ijmr.3835. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Butler DP, Perry F, Shah Z, et al. The quality of video information on burn first aid available on YouTube. Burns. 2013;39:856–9. doi: 10.1016/j.burns.2012.10.017. [DOI] [PubMed] [Google Scholar]
- 15.Dietrich JJ, Coetzee J, Otwombe K, et al. Adolescent-friendly technologies as potential adjuncts for health promotion. Health Educ. 2014;114:304–18. doi: 10.1108/HE-08-2013-0039. [DOI] [Google Scholar]
- 16.Gold J, Pedrana AE, Stoove MA, et al. Developing health promotion interventions on social networking sites: Recommendations from the FaceSpace project. J Med Internet Res. 2012;14:1–19. doi: 10.2196/jmir.1875. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Ybarra MB, Bull SS. Current trends in Internet and cell phone-based HIV prevention and intervention programs. Curr HIV/AIDS Rep. 2007;4:201–7. doi: 10.1007/s11904-007-0029-2. [DOI] [PubMed] [Google Scholar]
- 18.Keelan J, Pavri-Garcia V, Tomlinson G, et al. YouTube as a source of information on immunization: A content analysis. JAMA. 2007;298:2482–4. doi: 10.1001/jama.298.21.2482. [DOI] [PubMed] [Google Scholar]