Abstract
Background
High-quality information is essential if clients who request an abortion are to reach informed decisions and feel prepared for the procedure, but little is known concerning the readability of web-based sources containing such material. The aim was to investigate the readability of web-based information about induced abortion.
Methods
The search engine Google was used to identify web pages about induced abortion, written in the English language. A total of 240 hits were screened and 236 web pages fulfilled the inclusion criteria. After correcting for duplicate hits, 185 web pages were included. The readability of the text-based content of each web page was determined with Flesch Kincaid Grade Level, Gunning Fog Index, Coleman-Liau Index, Simple Measure of Gobbledygook, and Flesch Reading Ease. Data were analyzed with descriptive statistics, Pearson’s correlation coefficient and Kruskal-Wallis with Dunn’s test as post hoc analysis.
Results
Across all grade level measures, a small minority of the web pages had a readability corresponding to elementary school (n < 3, 1%), while the majority had readability corresponding to senior high school or above (n > 153, 65%). The means of the grade level measures ranged between 10.5 and 13.1, and the mean Flesch Reading Ease score was 45.3 (SD 13.6). Only weak correlations (rho < 0.2) were found between the readability measures and search rank in the hit lists. Consistently, web pages affiliated with health care had the least difficult readability and those affiliated with scientific sources had the most difficult readability.
Conclusions
Overall, web-based information about induced abortions has difficult readability. Incentives are needed to improve the readability of these texts and ensure that clients encounter understandable information so that they may reach informed decisions and feel adequately prepared when requesting an abortion.
Keywords: Consumer health information, Induced abortion, Readability, World wide web, Quality
Background
Worldwide, it is estimated that 56 million induced abortions occur annually, representing approximately 35 abortions per 1000 women [1]. Literature reviews and clinical guidelines emphasize the importance of pre-abortion counseling to offer preparatory information and support clients so that they may reach informed decisions regarding whether or not they want to undergo the abortion and which of the available abortion methods that they prefer [2–4]. Indeed, promoting informed decisions through sufficient and understandable information is essential in reproductive health services [5], including care before an abortion [3]. Those who request an abortion emphasize the importance of feeling sufficiently prepared by receiving sufficient information beforehand. Studies show that clients experience significant fears [6, 7] and uncertainties [6, 8] before an abortion. In order to decrease these fears and promote informed decisions, sufficient preparatory information about abortion is essential [2–4, 6]. It is known that the public now frequently turn towards the Web to access information about induced abortions [7, 9, 10]. If utilized appropriately, the Web has the potential to be a source for accessible information of high quality that may empower the public to reach informed decisions concerning their health [11, 12]. However, the size and structure of the Web implies a risk of contact with information of low quality [11], which may hinder informed decisions and result in ill-prepared patients.
Use of web-based services containing health-related information continues to increase and is today widespread, particularly among younger individuals of reproductive age [13]. The Web is available through handheld devices via mobile networks, with a considerable growth of mobile broadband subscriptions even in the least developed countries [14]. The fact that a high proportion of the public readily have access to the Web has shifted the focus from inequities in physical access to the Internet towards the skills needed to identify and interpret information found on the Web [15]. If web-based information is to reach its full potential, the public need to be able to understand the information that is written, meaning that they need to have sufficient literacy skills [16]. Given the fact that dissemination of health-related information is widespread today, health literacy is an important concept that can have a substantial effect on the lives of those seeking care [17]. Health literacy, defined as the capacity to obtain, process, and understand health-related information well enough to reach decisions [18], has repeatedly been acknowledged as a significant contributor to worsened clinical outcomes and poor use of health care services [19]. However, it is estimated that more than half of the population have intermediate or lower health literacy, and research indicates that as low as 12% have proficient literacy levels [20]. Low health literacy is associated with more hospitalizations, higher mortality and lesser ability to interpret health-related messages [19]. The ability to identify, understand and appraise health-related information from electronic sources is generally poor, even among younger college-level audiences [21].
Considering the aforementioned impacts that low health literacy may have, readability is an important aspect when evaluating the quality of written information developed for the public. Readability is a multidimensional concept that comprises aspects such as typography, the reader’s interest, and the style of writing in the text. The concept is broadly defined by Dale and Chall as ‘the sum total (including the interactions) of all those elements within a given piece of printed material that affects the success that a group of readers have with it’ [22]. In light of the raising use of the Web as a source for health-related information, an increasing number of studies have investigated the readability of web-based information developed for the public. Mainly, these studies focus on assessing the vocabulary and sentence structure with various calculations that illustrate readability levels. Many of these studies report difficult readability levels, which may hinder information uptake among those seeking information about a health-related topic. While studies indicate that web-based information about induced abortions have low quality in regard to various aspects, including information about treatment options, accuracy and comprehensiveness [23–27], little is still known about the readability of such sources.
Methods
Aim
The aim of this study was to investigate the readability of web-based information about induced abortion.
Design
This was a cross-sectional study investigating the readability of web-based information written in the English language, using quantitative readability measures assessing the text-based content. This manuscript follows the STROBE checklist (Additional file 1).
Identification of web pages
Google, which is the most used online search engine [28], was used to identify web pages about induced abortion. Preliminary search terms were designed by the researchers, based on their understanding of patient’s wordings related to induced abortion. Google Trends was then used to further explore worldwide use of search terms related to abortion, which confirmed the chosen terms. The final search terms used included “Abortion”, “Induced abortion”, “Termination of pregnancy”, “Terminate a pregnancy”, “Abortion pill”, “RU-486”, “Abortion facts”, and “Medical abortion”. According to previous research, members of the public search for health-related information by screening the first hits in search engines [29–32]. Thus, the first 30 hits were screened for inclusion, resulting in 240 screened hits in total. The searches were performed in September and October 2018. Two links did not work and two links led to web pages written in other languages than English, resulting in 236 hits leading to web pages about induced abortion. Of these, 51 hits led to duplicate web pages, resulting in 185 unique web pages that were included in the sample.
Data collection
Each included web page was accessed and the text-based content was copied to the web-based readability tool Readable.com, a tool recommended by U.S. National Library of Medicine [33]. The complete set of text from each included web page was imported in the tool and the imported texts were all checked for consistency in comparison with the original text found in each web page. Automated calculations were performed in the tool in regard to the following widely established readability measures: Flesch Kincaid Grade Level (FKGL), Gunning Fog Index (GFI), Coleman-Liau Index (CLI), Simple Measure of Gobbledygook (SMOG), and Flesch Reading Ease (FRE). Four of the measurements (FKGL, GFI, CLI and SMOG) generate a score corresponding to grade levels, which were interpreted as elementary school (1–5), junior high school (6–9) and senior high school or above (≥10). FRE generate a score ranging from 0 to 100, interpreted as easy (80–100), average (60–79) or difficult (0–59). The measurements base the score on different aspects of the number of words, sentences, syllables and letters.
Data analysis
The data were analyzed with RStudio version 1.0.143 (RStudio, Inc.). Descriptive statistics were calculated to describe the sample, and associations between the readability variables and search rank (i.e. rank in the hit list in the search engine) were calculated with Pearson’s Correlation Coefficient. For comparisons of readability scores between the different affiliations of web pages, the Kruskal-Wallis test was used. Dunn’s test with the bonferroni correction was used for post hoc analysis. In these comparisons, affiliations with < 20 included web pages were excluded due to small sample sizes. P < .05 was considered statistically significant.
Results
The most common affiliations among the included web pages were charities or private organizations (n = 50, 27%), while the least common were online shops (n = 1, 1%) and churches (n = 1, 1%) (Table 1). Across all grade level measures, a small minority of the included web pages had a readability corresponding to elementary school (n ≤ 2, 1%) and the majority had readability corresponding to senior high school or above (n ≥ 125, 68%). In total, 157 web pages (85%) had difficult readability according to FRE, i.e. a score between 0 and 59 (Table 2).
Table 1.
Affiliations among the included web pages (n = 185)
| Affiliation | n (%) |
|---|---|
| Charity or private/ patient organization | 50 (27.0) |
| Health care system | 35 (18.9) |
| News/magazine | 28 (15.1) |
| Independent information website | 20 (10.8) |
| Scientific | 20 (10.8) |
| Government | 14 (7.6) |
| Wiki | 6 (3.2) |
| Dictionary | 5 (2.7) |
| University/College | 3 (1.6) |
| Pharmaceutical company | 2 (1.1) |
| Online shop | 1 (0.5) |
| Church | 1 (0.5) |
Table 2.
Difficulty levels of the included web pages (n = 185)
| Measure | Difficulty level | n (%) |
|---|---|---|
| Flesch Kincaid Grade Level | Elementary school (1–5) | 2 (1.1) |
| Junior high school (6–9) | 58 (31.3) | |
| Senior high school and above (> 10) | 125 (67.6) | |
| Gunning Fog Index | Elementary school (1–5) | 1 (0.5) |
| Junior high school (6–9) | 11 (6.0) | |
| Senior high school and above (> 10) | 173 (93.5) | |
| Coleman-Liau Index | Elementary school (1–5) | 0 (0.0) |
| Junior high school (6–9) | 27 (14.6) | |
| Senior high school and above (> 10) | 158 (85.4) | |
| Simple Measure of Gobbledygook | Elementary school (1–5) | 0 (0.0) |
| Junior high school (6–9) | 2 (1.1) | |
| Senior high school and above (> 10) | 183 (98.9) | |
| Flesch Reading Ease | Easy (80–100) | 0 (0.0) |
| Average (60–79) | 28 (15.1) | |
| Difficult (0–59) | 157 (84.9) |
For the complete sample, the means of the grade level measures ranged between 10.7 and 13.2 (SD 1.8–2.7), indicating that the included web pages had difficult readability levels. Irrespective of grade level measure, the affiliation with the highest mean was scientific sources, and the affiliation with the lowest mean was health care. The mean FRE score was 44.4 (SD 13.5), with lowest mean for scientific sources and highest mean for sources developed and managed by health care. According to the Kruskal-Wallis test, there were significant differences in readability levels between web page affiliations across all reading measures. When comparing web page affiliations with the post hoc test, web pages affiliated with the health care consistently had scores indicating significantly less difficult readability, while web pages affiliated with scientific sources consistently had scores indicating more difficult readability (Table 3). Only weak correlations (r < 0.2) were found between the readability measures and search rank, indicating that readability level is not associated with the list generated by the search engine (Table 4). Although the correlations were weak, all measures increased in grade level as the search rank increased (Fig. 1).
Table 3.
Readability of the included web pages (n = 185)
| Readability measure | Affiliation | Mean (SD) | Range |
|---|---|---|---|
| Flesch Kincaid Grade Level | Charity/private organization1,2 | 10.5 (2.3) | 5–15 |
| Health care2 | 9.1 (2.0) | 4–14 | |
| News/magazine1 | 11.2 (2.4) | 8–16 | |
| Independent information website2 | 10.6 (2.2) | 7–16 | |
| Scientific1 | 12.7 (1.8) | 9–16 | |
| Other | 11.1 (2.0) | 7–15 | |
| All affiliations (complete sample) | 10.7 (2.4) | 4–16 | |
| Gunning Fog Index | Charity/private organization | 13.0 (2.7) | 5–18 |
| Health care2 | 11.9 (2.2) | 6–16 | |
| News/magazine | 13.3 (2.3) | 10–18 | |
| Independent information website2 | 13.2 (2.9) | 9–20 | |
| Scientific1 | 15.0 (2.4) | 10–18 | |
| Other | 13.1 (2.7) | 9–20 | |
| All affiliations (complete sample) | 13.1 (2.7) | 5–20 | |
| Coleman-Liau Index | Charity/private organization1,2 | 11.6 (2.0) | 6–14 |
| Health care2 | 10.6 (1.9) | 7–16 | |
| News/magazine2 | 11.6 (2.0) | 7–15 | |
| Independent information website2 | 12.1 (2.3) | 8–16 | |
| Scientific1 | 14.4 (1.9) | 11–18 | |
| Other | 12.5 (2.1) | 8–19 | |
| All affiliations (complete sample) | 11.9 (2.3) | 6–19 | |
| Simple Measure of Gobbledygook | Charity/private organization1 | 13.3 (1.8) | 9–17 |
| Health care2 | 12.2 (1.5) | 9–16 | |
| News/magazine1 | 13.7 (1.8) | 11–18 | |
| Independent information website | 12.9 (1.6) | 11–17 | |
| Scientific1 | 14.3 (1.8) | 11–17 | |
| Other | 13.2 (1.8) | 10–17 | |
| All affiliations (complete sample) | 13.2 (1.8) | 9–18 | |
| Flesch Reading Ease | Charity/private organization1,2 | 46.3 (10.8) | 30–78 |
| Health care2 | 53.5 (12.3) | 22–77 | |
| News/magazine2 | 47.0 (11.8) | 23–69 | |
| Independent information website1,2 | 42.0 (15.5) | 12–68 | |
| Scientific1 | 30.9 (8.7) | 15–51 | |
| Other | 39.0 (12.7) | 10–64 | |
| All affiliations (complete sample) | 44.4 (13.5) | 10–78 |
1P < .05 compared with web pages affiliated with health care (Dunn’s test); 2P < .05 compared with web pages affiliated with scientific sources (Dunn’s test)
Table 4.
Correlation between search rank and readability measures
| Readability measure | r | P-value |
|---|---|---|
| Flesch Kincaid Grade Level | 0.15 | 0.02 |
| Gunning Fog Index | 0.08 | 0.20 |
| Coleman-Liau Index | 0.11 | 0.10 |
| Simple Measure of Gobbledygook | 0.15 | 0.02 |
| Flesch Reading Ease | −0.09 | 0.14 |
Fig. 1.
Associations between readability measures and search rank
Discussion
Principal results
The aim of this study was to investigate the readability of web-based information about induced abortion. Five widely established measures for determination of readability were used, which consistently showed that the readability was difficult, corresponding to a grade level of senior high school or above. Readability was most difficult for web pages affiliated with scientific sources, while it was least difficult for web pages affiliated with health care. Weak correlation was observed between search rank and readability, indicating that the readability was difficult regardless of search rank in the hit lists.
The findings indicate that text-based information on the Web does not accommodate the needs of persons with literacy levels corresponding to elementary school or junior high school. The literature recommends readability levels below the sixth grade level for patient education materials [34]. The fact that the mean readability levels were higher than 10 across all investigated measures call attention to an imbalance between the literacy of the intended audience for abortion-related information and the readability of the material available they find on the Web. Studies in other health-related fields report similar findings, illustrating that difficult readability levels are widely dispersed across the Web [35–37]. Problematic readability levels are also present in other websites within the field of obstetrics and gynecology [38, 39], further strengthening the results. We argue that there is a considerable need to improve the overall readability levels of text-based information about induced abortions. Website developers need to carefully consider readability when producing content for the public. Improved readability may be achieved through various methods, such as using frequent words, short words [40], illustrations, and a narrative style [41]. This study investigated readability calculated with equations based on words and sentence structure. While it is possible that the included web pages had better readability in regard to other aspects within the concept of readability, including typology and use of illustrations, the results nevertheless illustrate quality deficits in regard to the text-based content.
Clinically, health professionals who provide abortion services need to promote enhanced information uptake among clients seeking an induced abortion. Low health literacy levels are prevalent in the general population [20], meaning that many clients may experience difficulties reading written information. This could result in a lack of psychological preparedness and an increased risk of misunderstandings when seeking an induced abortion, which has been highlighted in previous research exploring the perspectives among women with experience of an abortion [6, 7, 42]. Health professionals who consult clients seeking an induced abortion need to acknowledge the risk of encountering materials of difficult readability when searching for web-based information, perhaps even providing clients readable written information and offering suggestions how to identify high-quality information corresponding to their health literacy levels. Our results emphasize the importance of developing clinical guidelines how to appropriately discuss and inform about web-based information when working in abortion services. Health professionals, researchers and stakeholders should consider initiating multidisciplinary collaborations that aim to improve the readability of high-quality web-based information about induced abortions. When comparing web page affiliation, scientific sources showed the most difficult readability levels and web pages affiliated with health care showed the least difficult readability levels. While this indicate what type of resources clients could benefit the most from reading, these results need to be interpreted with caution. Health professionals who want to refer clients towards certain web-based sources should always aim to assess readability as well as other quality aspects before deciding to recommend a website for clients.
This study contributes with new knowledge that complements previous studies investigating quality of web-based information about induced abortions. Quality of web-based information is multidimensional and concerns many aspects besides readability [43, 44], many of which have been investigated in previous studies. According to previous assessments performed by health professionals and researchers, websites about abortion have considerable quality deficits in regard to comprehensiveness [23, 27], accuracy [25, 45], and transparency [27]. Laypersons also report issues with web-based information about abortions related to quality of information about treatment options, reliability, language, understandability, tone, design, layout, and logic [26]. When our findings are taken together with the results of previous studies, it can be concluded that web-based information about induced abortion has serious quality deficits in regard to many different aspects. Clients who request an abortion are at risk of encountering information that is difficult to comprehend. This can lead to misunderstandings and poor information uptake, calling attention to the need to guide those who plan to use the Web for supplemental information towards understandable sources of high quality.
Methodological considerations
We performed searches designed to mimic the search patterns of the public, using the most used search engine online and exploring Google Trends to validate our search terms. Reports indicate that over 80% of those who search for information via the Web use Google, a number which continues to rise in recent years and reaching as high as 97% of the population in some areas [46, 47]. In light of these statistics, we argue that only using Google could have produced more generalizable results in line with what most Internet users encounter. Nevertheless, it is possible that a proportion of the population decide to use other search engines and the results need to be interpreted with this in mind. We included the first 30 hits retrieved in the list from the search engine, which by far represent the limited number of hits usually accessed by the public when searching for health-related information [29–32].
Readability was determined with an online tool recommended by U.S. National Library of Medicine [33] and which has been used in previous reports [48, 49]. The readability was determined through a series of widely established automated calculations. These measures are efficient in determining readability in regard to quantitative variables based on calculations of texts, involving the number of and relationship between words, sentences, letters, and syllables. However, the tests do not take into consideration complex aspects related to the readability, such as use of words that few may recognize including medical terminology. Readability formulas have been criticized as problematic due to the potentially simplistic approach, as they are based on counting formal properties in texts [50]. For a comprehensive understanding of readability that complements the results of this study, more studies that explores how the intended audience experience abortion-related information is needed.
Conclusion
The readability of web-based information about induced abortions is difficult, corresponding to senior high school or above. The difficult readability is found irrespective of search rank in the hit list retrieved in the search engine. Members of the public who search for supplemental information about abortions are at risk of encountering information that is difficult to understand, possibly leading to misunderstandings, impaired decision-making, and insufficient preparatory information. Incentives that aim to improve the readability of web-based sources about abortions are needed. Health professionals who consult those who request an abortion should address the identified quality deficits and guide clients towards high-quality sources that contain readable and understandable information.
Supplementary information
Additional file 1. STROBE checklist for cross-sectional studies.
Acknowledgements
Not applicable.
Abbreviations
- FKGL
Flesch Kincaid Grade Level
- GFI
Gunning Fog Index
- CLI
Coleman-Liau Index
- SMOG
Simple Measure of Gobbledygook
- FRE
Flesch Reading Ease
Authors’ contributions
SG conceived and designed the study, and critically reviewed the manuscript. TC conceived and designed the study, collected the data, analyzed the data and wrote the manuscript. The authors have read and approved the final version of this manuscript.
Funding
Open access funding provided by Uppsala University.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
This research did not involve any human research subjects and only included written texts publicly available on the Internet. In Sweden, where the study was conducted, no ethical approval or consent to participate is required for such studies (Sveriges Riksdag. Lag (2003:460) om etikprövning av. forskning som avser människor [Law (2003:460) about ethical vetting for research that involve humans] [Internet]. 2003. Available from: https://www.riksdagen.se/sv/dokument-lagar/dokument/svensk-forfattningssamling/lag-2003460-om-etikprovning-av-forskning-som_sfs-2003-460).
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Supplementary information accompanies this paper at 10.1186/s12911-020-01132-y.
References
- 1.Sedgh G, Bearak J, Singh S, Bankole A, Popinchalk A, Ganatra B, et al. Abortion incidence between 1990 and 2014: global, regional, and subregional levels and trends. Lancet. 2016;388:258–267. doi: 10.1016/S0140-6736(16)30380-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Dennis A, Blanchard K, Bessenaar T. Identifying indicators for quality abortion care: a systematic literature review. J Fam Plann Reprod Health Care. 2017;43:7–15. doi: 10.1136/jfprhc-2015-101427. [DOI] [PubMed] [Google Scholar]
- 3.Royal College of Obstetricians and Gynaecologists . The care of women requesting induced abortion: evidence-based clinical guideline number 7. London: RCOG Press; 2011. [Google Scholar]
- 4.The American College of Obstetricians and Gynecologists . Legislative interderence with patient care, medical decisions, and the patient-physician relationship. 2019. [Google Scholar]
- 5.Tucker EB. Shared decision-making and decision support: their role in obstetrics and gynecology. Curr Opin Obstet Gynecol. 2014;26:523–530. doi: 10.1097/GCO.0000000000000120. [DOI] [PubMed] [Google Scholar]
- 6.Georgsson S, Krautmeyer S, Sundqvist E, Carlsson T. Abortion-related worries, fears and preparedness: a Swedish web-based exploratory and retrospective qualitative study. Eur J Contracept Reprod Health Care. 2019;24:380–389. doi: 10.1080/13625187.2019.1647334. [DOI] [PubMed] [Google Scholar]
- 7.Andersson I-M, Christensson K, Gemzell-Danielsson K. Experiences, feelings and thoughts of women undergoing second trimester medical termination of pregnancy. PLoS One. 2014;9:e115957. doi: 10.1371/journal.pone.0115957. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Jones K, Baird K, Fenwick J. Women’s experiences of labour and birth when having a termination of pregnancy for fetal abnormality in the second trimester of pregnancy: a qualitative meta-synthesis. Midwifery. 2017;50:42–54. doi: 10.1016/j.midw.2017.03.014. [DOI] [PubMed] [Google Scholar]
- 9.Foster AM, Wynn LL, Trussell J. Evidence of global demand for medication abortion information: an analysis of www.Medicationabortion.Com. Contraception. 2014;89:174–180. doi: 10.1016/j.contraception.2013.05.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Carlsson T, Bergman G, Wadensten B, Mattsson E. Experiences of informational needs and received information following a prenatal diagnosis of congenital heart defect. Prenat Diagn. 2016;36:515–522. doi: 10.1002/pd.4815. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Cline RJ, Haynes KM. Consumer health information seeking on the internet: the state of the art. Health Educ Res. 2001;16:671–692. doi: 10.1093/her/16.6.671. [DOI] [PubMed] [Google Scholar]
- 12.Eysenbach G, Jadad AR. Evidence-based patient choice and consumer health informatics in the internet age. J Med Internet Res. 2001;3:E19. doi: 10.2196/jmir.3.2.e19. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Kummervold PE, Chronaki CE, Lausen B, Prokosch H-U, Rasmussen J, Santana S, et al. eHealth trends in Europe 2005-2007: a population-based survey. J Med Internet Res. 2008;10:e42. doi: 10.2196/jmir.1023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.International Telecommunication Union . ICT facts and figures 2017. 2017. [Google Scholar]
- 15.van Dijk JAGM. Digital divide research, achievements and shortcomings. Poetics. 2006;34:221–235. [Google Scholar]
- 16.Norman CD, Skinner HA. eHealth literacy: essential skills for consumer health in a networked world. J Med Internet Res. 2006;8:e9. doi: 10.2196/jmir.8.2.e9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Witte PG. Health literacy: can we live without it? Adult Basic Educ Lit J. 2010;4:3–12. [Google Scholar]
- 18.Institute of Medicine (US) Committee on Health Literacy . In: Health literacy: a prescription to end confusion. Nielsen-Bohlman L, Panzer AM, Kindig DA, editors. Washington (DC): National Academies Press; 2004. [PubMed] [Google Scholar]
- 19.Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med. 2011;155:97–107. doi: 10.7326/0003-4819-155-2-201107190-00005. [DOI] [PubMed] [Google Scholar]
- 20.Kutner M, Greenberg E, Paulsen C, White S. The health literacy of America’s adults: results from the 2003 national assessment of adult literacy. 2006. [Google Scholar]
- 21.Stellefson M, Hanik B, Chaney B, Chaney D, Tennant B, Chavarria EA. eHealth literacy among college students: a systematic review with implications for eHealth education. J Med Internet Res. 2011;13:e102. doi: 10.2196/jmir.1703. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Dale E, Chall JS. The concept of readability. Elem Engl. 1949;26:19–26. [Google Scholar]
- 23.Foster AM, Jackson CB, Martin SB. Reproductive health and cyber (mis)representations: a content analysis of obstetrics and gynecology residency program websites. Contraception. 2008;78:99–105. doi: 10.1016/j.contraception.2008.03.012. [DOI] [PubMed] [Google Scholar]
- 24.Mashiach R, Seidman GI, Seidman DS. Use of mifepristone as an example of conflicting and misleading medical information on the internet. BJOG. 2002;109:437–442. doi: 10.1111/j.1471-0528.2002.t01-1-01124.x. [DOI] [PubMed] [Google Scholar]
- 25.Rowlands S. Misinformation on abortion. Eur J Contracept Reprod Health Care. 2011;16:233–240. doi: 10.3109/13625187.2011.570883. [DOI] [PubMed] [Google Scholar]
- 26.Carlsson T, Axelsson O. Patient information websites about medically induced second-trimester abortions: a descriptive study of quality, suitability, and issues. J Med Internet Res. 2017;19:e8. doi: 10.2196/jmir.6380. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Georgsson S, van der Spoel L, Ferm J, Carlsson T. Quality of web pages about second-trimester medical abortion: a cross-sectional study of readability, comprehensiveness, and transparency. J Adv Nurs. 2019;75:2683–2691. doi: 10.1111/jan.14080. [DOI] [PubMed] [Google Scholar]
- 28.eBizMBA . Top 15 Most popular search engines January 2019. 2019. [Google Scholar]
- 29.Eysenbach G, Köhler C. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ. 2002;324:573–577. doi: 10.1136/bmj.324.7337.573. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Peterson G, Aslani P, Williams KA. How do consumers search for and appraise information on medicines on the internet? A qualitative study using focus groups. J Med Internet Res. 2003;5:e33. doi: 10.2196/jmir.5.4.e33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Fiksdal AS, Kumbamu A, Jadhav AS, Cocos C, Nelsen LA, Pathak J, et al. Evaluating the process of online health information searching: a qualitative approach to exploring consumer perspectives. J Med Internet Res. 2014;16:e224. doi: 10.2196/jmir.3341. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Feufel MA, Stahl SF. What do web-use skill differences imply for online health information searches? J Med Internet Res. 2012;14:e87. doi: 10.2196/jmir.2051. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.U.S. National Library of Medicine . How to write easy-to-read health materials: MedlinePlus. 2019. [Google Scholar]
- 34.The Joint Commission . Advancing effective communication, cultural competence, and patient- and family-centered care: a roadmap for hospitals. Oakbrook Terrace: The Joint Commission; 2010. [Google Scholar]
- 35.Scott BB, Johnson AR, Doval AF, Tran BN, Lee BT. Readability and understandability analysis of online materials related to abdominal aortic aneurysm repair. Vasc Endovasc Surg. 2019;54(2):111–7. [DOI] [PubMed]
- 36.Murray KE, Murray TE, O’Rourke AC, Low C, Veale DJ. Readability and quality of online information on osteoarthritis: an objective analysis with historic comparison. Interact J Med Res. 2019;8:e12855. doi: 10.2196/12855. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Kloosterboer A, Yannuzzi NA, Patel NA, Kuriyan AE, Sridhar J. Assessment of the quality, content, and readability of freely available online information for patients regarding diabetic retinopathy. JAMA Ophthalmol. 2019;137:1240–1245. doi: 10.1001/jamaophthalmol.2019.3116. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Murphy J, Vaughn J, Gelber K, Geller A, Zakowski M. Readability, content, quality and accuracy assessment of internet-based patient education materials relating to labor analgesia. Int J Obstet Anesth. 2019;39:82–87. doi: 10.1016/j.ijoa.2019.01.003. [DOI] [PubMed] [Google Scholar]
- 39.Lange EMS, Shah AM, Braithwaite BA, You WB, Wong CA, Grobman WA, et al. Readability, content, and quality of online patient education materials on preeclampsia. Hypertens Pregnancy. 2015;34:383–390. doi: 10.3109/10641955.2015.1053607. [DOI] [PubMed] [Google Scholar]
- 40.Rello L, Baeza-Yates R, Dempere-Marco L, Saggion H. Frequent words improve readability and short words improve understandability for people with dyslexia. Hum-Comput Interact. 2013:203–19.
- 41.Michielutte R, Bahnson J, Dignan MB, Schroeder EM. The use of illustrations and narrative text style to improve readability of a health education brochure. J Cancer Educ. 1992;7:251–260. doi: 10.1080/08858199209528176. [DOI] [PubMed] [Google Scholar]
- 42.Mukkavaara I, Öhrling K, Lindberg I. Women’s experiences after an induced second trimester abortion. Midwifery. 2012;28:e720–e725. doi: 10.1016/j.midw.2011.07.011. [DOI] [PubMed] [Google Scholar]
- 43.Eysenbach G, Powell J, Kuss O, Sa E-R. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA. 2002;287:2691–2700. doi: 10.1001/jama.287.20.2691. [DOI] [PubMed] [Google Scholar]
- 44.Burkell J. Health information seals of approval: what do they signify? Inf Commun Soc. 2004;7:491–509. [Google Scholar]
- 45.Bryant AG, Narasimhan S, Bryant-Comstock K, Levi EE. Crisis pregnancy center websites: information, misinformation and disinformation. Contraception. 2014;90:601–605. doi: 10.1016/j.contraception.2014.07.003. [DOI] [PubMed] [Google Scholar]
- 46.The Internet Foundation In Sweden . Svenskarna och internet 2018 [swedes and the internet 2018] 2018. [Google Scholar]
- 47.Purcell K, Brenner J, Rainie L. Search engine use 2012. Pew research center. 2012. [Google Scholar]
- 48.Basch CH, Ethan D, Cadorett V, Kollia B, Clark A. An assessment of the readability of online material related to fluoride. J Prev Interv Community. 2019;47:5–13. doi: 10.1080/10852352.2018.1547304. [DOI] [PubMed] [Google Scholar]
- 49.Doruk C, Enver N, Çaytemel B, Azezli E, Başaran B. Readibility, understandability, and quality of online education materials for vocal fold nodules. J Voice. 2020;34:302.e15–302.e20. doi: 10.1016/j.jvoice.2018.08.015. [DOI] [PubMed] [Google Scholar]
- 50.Bailin A, Grafstein A. Readability: text and context. London: Palgrave Macmillan; 2016.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Additional file 1. STROBE checklist for cross-sectional studies.
Data Availability Statement
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.