ABSTRACT
Patient education is important as it gives patients a better understanding of the risks and benefits of medical and surgical interventions. Developing communication technologies have completely changed and enhanced patient access to medical information. The aim of this study was to evaluate available patient education materials (PEMs) regarding hip surgery on the websites of major hip societies and centers. The PEM from 11 selected leading hip centers and societies were evaluated with the following assessment tools: Flesch–Kincaid (FK) readability test, Flesch Reading Ease formula, Literature-Intelligence-Data-Analysis (LIDA) instrument and Discernibility Interpretability Sources Comprehensive Evidence Relevance Noticeable (DISCERN) tool. Videos were assessed using Patient Educational Video Assessment Tool (PEVAT). A total of 69 educational items, including 52 text articles (75.4%) and 17 videos (24.6%) were retrieved and evaluated. The median Interquartile Range (IQR) FK level of 52 text articles was 10.8 (2.2). The median (IQR) LIDA score of text articles by center was 45. According to the LIDA score, 60% of all website articles demonstrated high accessibility (LIDA score > 44). The median DISCERN score of text articles by center was 69. Overall, 52 (100%) of the text articles were deemed to be at ‘good’ quality rating or higher, and 23.2% (16 out of 69) of the articles had excellent quality. The mean PEVAT score for the 17 videos was 25 ± 1.9. Analysis of text and video articles from the 11 leading orthopedic surgery centers and societies demonstrated that by selecting a reliable source of information from main scientific societies and major centers in hip surgery, patients can find more accurate information regarding their hip conditions.
INTRODUCTION
Patient education is an important part of patient care. It gives patients a better understanding of the risks, benefits and sequence of medical and surgical interventions.
The Internet has become the most widely used source of data for patients seeking additional information about their conditions over the past decade. Developing communication technologies have completely changed and enhanced patient access to medical information.
While this provides a tremendous opportunity for equity in patient care, the quality of the available patient education materials (PEMs) is unclear [1].
Furthermore, health literacy is defined as ‘the degree to which individuals can obtain, process, and understand basic health information and services needed to make appropriate health decisions’. High-quality PEM increases patient adherence to treatment plans and promotes healthy lifestyle behaviors [2].
The approximate level of education needed to understand a PEM is an important factor called readability. The American Medical Association (AMA) suggests that PEMs are better understood by US audiences if they are written at a sixth grade reading level [3]. Evidence demonstrates that when searching health topics patients rarely look beyond the first 25 pages. Participants typically opted for one of the initial search results presented by the search engine (97.2% were on links positioned within the first 10 search results) [4].
Despite this widespread access to the Internet, it has been shown that the quality of orthopedic-related data available for patients is highly variable [5]. Additionally, varying website designs offer different levels of accessibility for patients, making it a significant variable in patient engagement.
The aim of this study was to evaluate available PEMs regarding hip surgery on websites of the major hip societies and centers.
METHODS
In February 2023, the 11 main hip and orthopedic institutes were searched for PEMs on hip issues from their websites (Table I). The rationale for the selection of top institutions and societies was that they would provide a reliable source of PEMs.
Table I.
Distribution of assessment scores from the included orthopaedic academic centers
| Academic center rank | Text articles | Videos | FK level [Median (IQR)] | FRE score (MEAN ± SD) | LIDA score [Median (IQR)] | DISCERN score [Median (IQR)] | PEVAT score (MEAN ± SD) |
|---|---|---|---|---|---|---|---|
| 1 AAOS | 10 | 4 | 9.9 (1.3) | 51.3 ± 4.8 | 45 | 70.5 (4) | 22.6 ± 0.75 |
| 2 AAHKS | 10 | 4 | 11.5 (2.2) | 46.1 ± 7.5 | 43 | 67 (4.8) | 27 ± 0.4 |
| 3 AHI | 9 | 5 | 11.4 (2.5) | 38 ± 10 | 49 | 68 (5.5) | 24.2 ± 0.45 |
| 4 HSS | 9 | 1 | 10.7 (1.8) | 48.2 ± 5.3 | 45 | 70 (4) | 26.5 ± NA |
| 5 BCH | 6 | 0 | 9.2 (0.4) | 58.3 ± 2.2 | 42 | 71 (3.3) | – |
| 6 DUKE | 4 | 1 | 11.7 (1.3) | 41.1 ± 4.5 | 46 | 69.5 (2.5) | 24.5 ± NA |
| 7 AANA | 3 | 1 | 11.2 (2.9) | 45.6 ± 11 | 47 | 68 (7) | 26 ± NA |
| 8 AAOSM | 0 | 1 | – | – | 43 | – | 26.5 ± NA |
| 9 ISHA | 1 | 0 | – | 27.6 ± NA | 42 | – | – |
| 10 EHS | 0 | 0 | – | – | 45 | – | – |
| 11 HPS | 0 | 0 | – | – | X | – | – |
| Total | 52 | 17 | 10.8 (2.2) | 46.5± 9.1 | 45 (3.75) | 69 (4) | 25 ± 1.9 |
These centers’ websites were searched by two independent orthopedic surgeons (A.P. and a.c.) for hip surgery-related PEMs including text, articles, slides and videos. There were two ways to find the hip-related data. First, some websites specialized in hip surgery or had a subsection on their website for hip surgery, and secondly, the entire website was reviewed and searched for all the site’s contents for PEMs.
Evaluation of the readability
The Flesch–Kincaid (FK) grade and Flesch Reading Ease (FRE) formula, which have been widely used to determine the objective and numerical reading level of the text documents, were used in our study to evaluate readability level [6–8]. This formula is commonly used for determining grades and level of readability and has been incorporated into various softwares including Microsoft Word [9, 10]. Every piece of informational text was copied into a Microsoft Office Word 365 document (Microsoft, Redmond, WA, USA). The text was then edited to eliminate HTML tags, unrelated text and superfluous punctuation. Finally, Microsoft Word was used to check the articles for spelling and grammar mistakes. After preparation of the document, Microsoft Word can automatically calculate and generate both FK score and FRE grade. The FK grade-level approximately indicates the US grade-level equivalent for a person to read and understand the content of an article.
Quality assessment of the educational materials
To evaluate the completeness, correctness, accessibility, and transparency of information provided to patients on the websites of selected hip centers, the DISCERN instrument was used. This tool was initially described by Charnock et al. [11].
The DISCERN instrument was selected because it is the first standardized quality assessment tool of consumer health knowledge that can be used as a critical appraisal index to assess health knowledge among patients, healthcare professionals and the public. The DISCERN questionnaire process consists of three sections with 15 questions and an overall quality rating. Questions 1 through 8 in the first section address reliability; questions 9 through 15 concentrates on the website’s treatment information; and question 16 asks about the website’s overall quality score. A score of >70 is categorized as ‘excellent’, while a score of >50 is regarded as ‘good’. The maximum score is 80.
Accessibility of the educational contents
The accessibility of the examined websites included hip content was assessed using the LIDA instrument (Minervation Ltd, Oxford, UK) [12]. LIDA is a standard tool that consists of three main parts of accessibility, usability and reliability. We have used the accessibility section of LIDA to evaluate how easy access PEMs are on selected websites.
Evaluation of videos
We used the PEVAT (Patient Educational Video Assessment Tool) to evaluate the hip-related videos were extracted from websites in this study. PEVAT was introduced by the first author previously [1]. It consists of three subscale sets of binary and ternary questions with maximum scores of 30. A higher score indicated higher reliability, higher accessibility and higher overall quality of the provided educational video. The maximum achievable score for the videos is 30.
RESULTS
Study characteristics
A total of 69 educational items, including 52 text articles (75.3%) and 17 videos (24.6%) were retrieved and evaluated by two authors (XX, YY). There were no significant observer differences noted in the number of articles selected or the scores calculated from text or video evaluation (intraclass correlation coefficient = 0.814). The median (IQR) quantity of educational items (text articles and videos) per center was 5.5. The quantity of items per center spanned a range of 0 (EHS & The Hip society) to 14 (AAOS, AAHKS, AHI). Figure 1 reflects the quantity of items per centers and shows distribution of content.
Fig. 1.
PEMs per centers.
Distribution of text articles based on their topics, explained in Fig. 2.
Fig. 2.
Distribution of hip preservation-related PEMs based on topics and content.
EVALUATION OF TEXT MATERIALS
Readability FK and FRE assessments
The median (IQR) FK level of 52 text articles was 10.8 (2.2). The range consisted of articles from the 5th ranked academic center, which was the easiest to read at an FK level = 9.2 and the article of the 6th ranked academic center was the most difficult to read at an FK level = 11.7 (Figs 3 and 4). No articles were at or below an eight-grade reading level, which is the average reading level in the United States. The mean FRE score of text articles by center was 46.5 ± 9.1. The range consisted of articles from the 5th ranked center being the easiest to read with an FRE score of 58.3 and the 3rd ranked center’s article, which was the most difficult to read with an FRE score of 38.
Fig. 3.
Mean FK level of text materials by centers.
Fig. 4.
Topic-oriented readability scores for PEMs (text articles).
Accessibility LIDA score
The median (IQR) LIDA score of text articles by the center`s was 45 (3.75). The score range varied between the 3rd ranked center’s website articles and those of the 5th. The 3rd ranked center displayed the greatest usability with a LIDA score of 49, while the 5th ranked center showed the lowest score of 42.
Quality DISCERN assessment
The median (IQR) DISCERN score of text articles by center was 69 [4] or 69%. The scores ranged between those of the 5th ranked center’s articles which displayed the highest quality DISCERN score with a DISCERN score of 71 or higher percent and the 9th ranked center’s articles displayed the lowest quality with a DISCERN score of 60 or 60%. Overall, 52 (100%) of the text articles were deemed to be at ‘good’ quality rating or higher (DISCERN score of >50) and 23.2% (16 out of 69) of the articles had excellent quality based on DISCERN.
EVALUATION OF VIDEO MATERIALS
PEVAT
The mean PEVAT score for the 17 videos was 25 ± 1.9 with a range of 22.6–27. Seven (70%) of the top 10 academic centers had video content available regarding hip surgery patient education. For this assessment, the preliminarily defined threshold of a high-quality video was >20. Among the educational videos, 17 (100%) were deemed to be high quality (Table II).
Table II.
Distribution of educational materials and text assessment scores by topic
| Topic | Text articles | Videos | FK level (mean) | FRE score (mean) |
|---|---|---|---|---|
| THR (Total Hip Replacement) | 6 | 6 | 11.1 | 49.5 |
| Hip resurfacing | 2 | 1 | 10.9 | 47.2 |
| Hip arthroscopy | 4 | 2 | 11.3 | 43.3 |
| Osteonecrosis | 3 | 0 | 10.5 | 47.9 |
| FAI (FemoroacetabularImpingment) | 4 | 0 | 10.9 | 45.7 |
| Labral tear | 3 | 1 | 10.2 | 53.3 |
| Dysplasia | 5 | 1 | 10.8 | 47.5 |
| Hip dislocation | 1 | 1 | 9.7 | 52.1 |
| Snapping syndrome | 2 | 0 | 10.5 | 50.3 |
| Orthobiologics | 1 | 0 | 10.8 | 46.7 |
| Trochanteric bursitis | 2 | 0 | 10.7 | 48.3 |
| Osteoarthritis | 4 | 1 | 10.8 | 46.8 |
| Miscellaneous | 15 | 4 |
DISCUSSION
The most significant findings of this study were that PEMs regarding hip surgeries (both text articles and videos) provided by the major specialized hip centers and societies are highly accessible and reliable with a high quality. All PEMs including articles and videos scored well with high scores according to the DISCERN and PEVAT tools. The overwhelming majority of websites scored relatively poor in terms of readability, which means that the provided information is harder to read and understand by median reading level of US population.
Poor readability level is a common issue in most available PEMs in almost all specialties in medicine. It has been shown that considering several tricks while preparing PEMs could be effective in improving readability. These include avoiding technical jargon, complex medical terminology and acronyms. Instead, using clear and concise language that patients can easily understand could help. Second, breaking down complex concepts into simple terms and providing explanations or definitions where necessary could alleviate confusion. Third, adopting a conversational writing style to make the content more engaging and relatable could improve readability. Fourth, using an active voice and addressing the reader directly helps to create a connection and fosters better comprehension. Fifth, short sentences and paragraphs are preferred because long sentences can be difficult for the audience to follow. Sixth, each paragraph should be limited to one main idea or concept [13–15].
In a recent study, Hanish et al. [16] showed that reducing the use of complex words and sentence length to less than 15 words have better readability of patient education materials for knee and sports injuries. The mean reading level in their study was the mean reading level for original PEMs was 9.77, which is very similar to our study (10.2). One concern for readability evaluation is the fact that computerized tools such as FK and FRE has limitations compared to hand-scoring tools. Computer programs utilize different methods for counting words, sentences and syllables, which is a source of possible discrepancies [17].
In order to help patients make informed decisions, reduce anxiety and actively participate in their recovery process following hip surgery, high-quality patient education is crucial. There are many resources for patient education available on the Internet and social media platforms. However, due to a lack of regulation and oversight, these materials’ dependability and accuracy are in doubt [18]. It has been shown in a recent study on online patient education for hip arthroplasty that <30% of websites were classified as ‘good’ sources of information when using the DISCERN score [19], which is in contrast with the quality of PEMs from leading centers and societies in this study.
Of the centers included in this study, 63.6% (7 out of 11) provided video contents on their website regarding hip surgeries for patients. This is encouraging because videos can be widely used and may be preferred to text materials regardless of literacy or reading ability [20, 21].
CONCLUSION
Analysis of text and video articles from the 11 leading orthopedic surgery centers and societies in this study demonstrated that by selecting a reliable source of information from main scientific societies and major centers in hip surgery, patients can find more accurate information regarding their hip conditions.
ACKNOWLEDGEMENTS
The authors would like to express their sincere gratitude to the dedicated and talented team at the American Hip Institute for their invaluable contributions to this article.
Contributor Information
Ali Parsa, American Hip Institute Research Foundation, Chicago, IL 60018, USA; Orthopedic Research Center, Department of Orthopedic Surgery, Mashhad University of Medical sciences, Mashhad, Iran.
Omkar N Prabhavalkar, American Hip Institute Research Foundation, Chicago, IL 60018, USA.
Sheema Saeed, American Hip Institute Research Foundation, Chicago, IL 60018, USA.
Julio Nerys-Figueroa, American Hip Institute Research Foundation, Chicago, IL 60018, USA.
Andrew Carbone, American Hip Institute Research Foundation, Chicago, IL 60018, USA.
Benjamin G Domb, American Hip Institute Research Foundation, Chicago, IL 60018, USA; American Hip Institute, Chicago, IL 60018, USA.
DATA AVAILABILITY
The data analyzed during the current study will be shared by the corresponding author upon reasonable request.
CONFLICT OF INTEREST STATEMENT
B.G.D. has had ownership interests in the American Hip Institute, Hinsdale Orthopaedics, SCD#3, North Shore Surgical Suites, Munster Specialty Surgery Center; research support from Arthrex, the Kauffman Foundation, Stryker; consulting fees from Adventist Hinsdale Hospital, Arthrex, Medacta, Stryker; has received educational support from Arthrex, Breg, Medacta, Medwest Associates; speaking fees from Arthrex; honoraria from Medacta, travel and lodging from Arthrex, Medacta, Stryker, food and beverage from Arthrex, Medacta, Stryker, DJO Global, Zimmer Biomet; royalties from Amplitude, Arthrex, DJO Global, Medacta, Stryker, Orthomerica. Dr. Domb is Medical Director of Hip Preservation at St. Alexius Medical Center, board member for the American Hip Institute Research Foundation, AANA Learning Center Committee, the Journal of Hip Preservation Surgery, Journal of Arthroscopy. The American Hip Institute Research Foundation funds research and is where our study was performed. A.P. reports educational support from Smith & Nephew Inc. A.C. reports educational support from Smith & Nephew Inc and Arthrex; grants and personal fees from Arthrex and Smith & Nephew, outside the submitted work.
FUNDING
No funding was received for this work.
REFERENCES
- 1. Parsa A, Nazal M, Molenaars RJ et al. Evaluation of hip preservation-related patient education materials from leading orthopaedic academic centers in the United States and description of a novel video assessment tool. J Am Acad Orthop Surg Glob Res Rev 2020; 4: 1–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Peterson PN, Shetterly SM, Clarke CL et al. Health literacy and outcomes among patients with heart failure. JAMA 2011; 305: 1695–701. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Kripalani S, and Weiss BD. Teaching about health literacy and clear communication. J Gen Intern Med 2006; 21: 888–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Eysenbach G, Köhler C. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ 2002; 324: 573–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. McCormick JR, Kerzner B, Tuthill TA et al. Patients with femoroacetabular impingement obtain information from low-quality sources online and are most interested in conservative treatment and expected recovery. Arthrosc Sports Med Rehabil 2023; 5: e21–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Akinleye SD, Krochak R, Richardson N et al. Readability of the most commonly accessed arthroscopy-related online patient education materials. Arthroscopy 2018; 34: 1272–9. [DOI] [PubMed] [Google Scholar]
- 7. Feghhi DP, Komlos D, Agarwal N et al. Quality of online pediatric orthopedic materials. J Bone Joint Surg Am 2014; 96: e194. [DOI] [PubMed] [Google Scholar]
- 8. Badarudeen S, Sabharwal S. Readability of patient education materials from the American Academy of Orthopaedic Surgeons and Pediatric Orthopaedic Society of North America Web sites. J Bone Joint Surg Am 2008; 90: 199–204. [DOI] [PubMed] [Google Scholar]
- 9. Cooley ME, Moriarty H, Berger MS et al. Patient literacy and the readability of written cancer educational materials. Oncol Nurs Forum 1995; 22: 1345–51. [PubMed] [Google Scholar]
- 10. Albright J, De Guzman C, Acebo P et al. Readability of patient education materials: implications for clinical practice. Appl Nurs Res 1996; 9: 139–43. [DOI] [PubMed] [Google Scholar]
- 11. Charnock D, Shepperd S, Needham G et al. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 1999; 53: 105–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Minervation . The LIDA Instrument. 2018. Available at: http://www.minervation.com/wp-content/uploads/2011/04/Minervation-LIDAinstrument-v1-2.pdf. Accessed: 10 May 2018.
- 13. Heppard ED, Hyde Z, Florence MN et al. Improving the readability of online foot and ankle patient education materials. Foot Ankle Int 2014; 35: 1282–6. [DOI] [PubMed] [Google Scholar]
- 14. Badarudeen S, Sabharwal S. Assessing readability of patient education materials: current role in orthopaedics. Clin Orthop Relat Res 2010; 468: 2572–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Meghan MY, Paul HY, Hussein KI et al. Readability of patient education materials from the web sites of orthopedic implant manufacturers. J Arthroplasty 2017; 32: 3568–72. [DOI] [PubMed] [Google Scholar]
- 16. Hanish SJ, Cherian N, Baumann J et al. Reducing the use of complex words and reducing sentence length to <15 words improves readability of patient education materials regarding sports medicine knee injuries. Arthrosc Sports Med Rehabil 2023; 5: e1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Grabeel KL, Russomanno J, Oelschlegel S et al. Computerized versus hand-scored health literacy tools: a comparison of Simple Measure of Gobbledygook (SMOG) and Flesch-Kincaid in printed patient education materials. J Med Libr Assoc 2018; 106: 38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Suarez-Lledo V, Alvarez-Galvez J. Prevalence of health misinformation on social media: systematic review. J Med Internet Res 2021; 23: e17187. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Sheridan GA, O’Brien C, Masri BA et al. Revision total hip arthroplasty: an analysis of the quality and readability of information on the internet. World J Orthop 2020; 11: 82. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Park JS, Kim MS, Kim H et al. A randomized controlled trial of an educational video to improve quality of bowel preparation for colonoscopy. BMC Gastroenterol 2016; 16: 64. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Abu Abed M, Himmel W, Vormfelde S et al. Video-assisted patient education to modify behavior: a systematic review. Patient Educ Couns 2014; 97: 16–22. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data analyzed during the current study will be shared by the corresponding author upon reasonable request.