Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Jun 1.
Published in final edited form as: J Genet Couns. 2017 Sep 26;27(3):608–615. doi: 10.1007/s10897-017-0155-y

Evaluation of the informational content, readability and comprehensibility of online health information on monogenic diabetes

Yue Guan 1, Kristin A Maloney 1, Debra L Roter 2, Toni I Pollin 1,3
PMCID: PMC5869070  NIHMSID: NIHMS908979  PMID: 28951986

Abstract

The purpose of this study was to assess the informational content, readability, suitability and comprehensibility of websites offering educational information about monogenic diabetes available to patients. The top 20 results from 15 queries in four search engines were screened. Content analysis was performed by two independent coders. Readability was determined using Flesch-Kincaid grade level (FKGL) and Simplified Measure of Goobledygook (SMOG). The Comprehensibility Assessment of Materials (SAM+CAM) scale was utilized to evaluate website suitability and comprehensibility. Only 2% (N=29) of 1200 screened websites met inclusion criteria. Content analysis showed that 16 websites presented information on at least the most common forms of MODY (1, 2 and 3), four addressed the utility of genetic counseling, and none included support resources for patients. All websites exceeded the consensus readability level (6th grade) as assessed by FKGL (10.1 grade) and SMOG (12.8±1.5 grades). Although the majority (N=20) of websites had an overall “adequate” to “superior” quality score (SAM+CAM score >= 40%), more than one-third scored “not suitable” in categories of content, literacy demand, graphics, and learning motivation. The online educational resources for monogenic diabetes have a high readability level and require improvement in ease of use and comprehensibility for patients with diabetes.

Keywords: Monogenic diabetes, patient education, website analysis, readability, comprehensibility

INTRODUCTION

Diabetes mellitus affects approximately 382 million individuals worldwide, including 29.1 million in the U.S., making it a leading cause of morbidity and mortality. Approximately 1–2% of all cases of diabetes in the U.S. are monogenic in origin, with most of these falling into one of the subtypes known as maturity onset diabetes of the young (MODY) (Hattersley, Bruining, Shield, Njolstad, & Donaghue, 2009; Shields et al., 2010). Genetic testing allows differentiation of monogenic diabetes from type 1 and type 2 diabetes and can identify the exact type of monogenic diabetes present, which is necessary for selection of appropriate treatment and recommendations for screening at-risk family members (“2. Classification and Diagnosis of Diabetes,” 2016; M Shepherd, 2003; M. Shepherd & Hattersley, 2004). For example, some types of monogenic diabetes are best treated with sulfonylurea pills instead of insulin, while monogenic diabetes caused by mutations in the GCK gene usually does not require any treatment except sometimes during pregnancy. Monogenic diabetes remains a challenging diagnosis, and most health care providers know little about these forms of diabetes. Despite the very significant treatment implications of accurate diagnosis, a recent study found that 95% monogenic diabetes is misdiagnosed as other forms of diabetes and 76% of patients with MODY receive less than optimal treatment (Shields et al., 2010).

Patients often use and trust online health information and may utilize websites to guide their decision-making about treatment, whether to visit a doctor, to ask their doctor about new medications or treatment approaches or to get a second opinion on diagnosis or treatment (Center, 2013). Online health resources, however, have been shown to be of limited utility when the potential users have literacy deficits (Agarwal, Hansberry, Sabourin, Tomei, & Prestigiacomo, 2013; Eysenbach, Powell, Kuss, & Sa, 2002). Limited health literacy is especially problematic for patients with diabetes; a number of studies have found that literacy deficits are associated with lower levels of diabetes knowledge, worse glycemic control and higher rate of diabetes-related complications (“Health literacy: report of the Council on Scientific Affairs. Ad Hoc Committee on Health Literacy for the Council on Scientific Affairs, American Medical Association,” 1999; Kim, Love, Quistberg, & Shea, 2004; Schillinger et al., 2002). Patients with monogenic diabetes may find it even more difficult than other individuals with diabetes to understand the complex and often unfamiliar concepts associated with the genetic nature of their condition.

Most studies of Web-based diabetes patient education materials have been limited to the common type 1 and type 2 diabetes (Chumber, Huber, & Ghezzi, 2015; Gimenez-Perez, Caixas, Gimenez-Palop, Gonzalez-Clemente, & Mauricio, 2005; Thakurdesai, Kole, & Pareek, 2004; van Esch, Cornel, & Snoek, 2006; Weymann, Harter, & Dirmaier, 2015) and have relied on readability formulas to assess the literacy demand for individuals’ capacity to obtain, process, and understand basic health information (Boulos, 2005; Hutchinson, Baird, & Garg, 2016; Kusec, Brborovic, & Schillinger, 2003), largely ignoring important additional factors such as presentation format, graphics and illustrations, and approaches to enhance reader stimulation and motivation (Doak, Doak, & Root, 1985; Finnie, Felder, Linder, & Mullen, 2010; Friedman & Hoffman-Goetz, 2006; Ley & Florio, 1996; “U.S. Department of Health and Human Services: Web Standards and Usability Guidelines,” 2016; “Web Accessibility Initiative (WAI): Web accessibility standards and guidelines ”, 2016).

Purpose of the study

This study was designed to systematically assess the informational content, readability, suitability and comprehensibility of websites offering educational information about monogenic diabetes available to patients to gauge the likelihood that a typical patient would be able to read, understand and apply the health information they gain from these online resources. Results are expected help health care providers to evaluate and recommend education materials for their patients and to identify needs that can be addressed with future web content.

MATERIALS AND METHODS

Search strategy

A list of 15 key words commonly used by patients and published literature to describe monogenic diabetes were used as search terms. These terms included “monogenic diabetes”, “MODY”, “maturity onset diabetes of the young”, “genetic diabetes”, “inherited diabetes”, “familial diabetes”, “diabetes gene”, “diabetes DNA”, “baby/infant/neonatal diabetes”, “thin/slim diabetes”, “T1DM genetics” and “T2DM genetics”. Each of these key terms were entered into four different search engines (Google, Bing, Yahoo, and Ask) because of their popularity (Alexa, 2016). The search was conducted under the “private browsing” setting so that the searches would not be influenced by previous browsing history. The first 20 links reported by each search engine per keyword were visited and screened for eligibility. Different web pages listed from the same website were analyzed as one site.

Inclusion/exclusion criteria

For the purpose of this study, sites were included if they contained information that would allow users to answer the question “what is monogenic diabetes?”. “For profit” websites were included if they attempted to educate users about monogenic diabetes. Sites were excluded if they were academic research articles, labelled specifically for health care providers only, nonfunctioning links, were not in English, required a password for access, or did not provide any information on monogenic diabetes in humans.

Website content

Websites were coded as non-profit or commercial and for inclusion of the following topics: subtypes of monogenic diabetes (all mentioned were noted in code table), inheritance pattern (yes or no), involvement of genetic counseling (yes or no), genetic testing in general (yes or no), genetic testing for specific monogenic diabetes subtypes (yes or no), treatment options (yes or no), and social networking links (yes or no).

Readability

Readability was determined using standard Flesch-Kincaid grade level based on average syllables per word and sentence length (Flesch, 1948) and Simplified Measure of Goobledygook (SMOG) based on the average number of polysyllabic words per sentence (Mc Laughlin, 1969). To perform the readability assessment, the first author pasted full webpage content into the text field of a free online readability score calculator tool (https://readability-score.com/) that gives scores for both of these measures. Titles, subtitles, body text, and bullet point text were included, while references, web links and advertising text were excluded from the analysis. The recommended readability standard for Flesch-Kincaid is grade level of 6–8 or below. The standards for SMOG reading level are: superior (<=6th grade level), adequate (6th–8th grade level), or not suitable (>=8th grade level) (Hernandez, 2009).

Suitability Assessment of Materials (SAM) and Comprehensibility (CAM) scale

SAM+CAM is an objective standard rating scale widely used to evaluate the suitability and comprehensibility of a website (Helitzer, Hollis, Cotner, & Oestreicher, 2009). It evaluates 22 individual variables relevant to six SAM+CAM categories: (1) content (purpose, summary/review, desired reader behavior, credibility), (2) literacy demand (writing style, vocabulary helpers, confusion reduction, context, scope/length), (3) numeracy (numeric presentation, calculation), (4) graphics (table/graph/illustration clarity), (5) layout/typography (organization, typography and font, subheadings/organizers), and (6) learning stimulation/motivation (motivators to attend to text, inclusion, reader interaction factors, theoretical application, tone, and persuasion techniques). Each variable was scored as superior (=2), adequate (=1), not suitable (=0), or not applicable. Coders scored each variable and added these scores to give each category a ranking, then averaged the category scores to provide an overall rating for the website. The overall rating, calculated as the total SAM+CAM score divided by the total possible score, placed each website into categories of superior (>70%), adequate (40–70%), and not suitable (<40%). A “superior” website is considered to be easier to use and understand, to place less literacy demand on patients, and to be more attention-getting and motivational than other websites (Helitzer et al., 2009).

Data analysis

Two coders (YG and KM) independently applied the search strategy and screened websites for eligibility. Primary coding of all the websites included in the study sample was done by YG with independent coding of 20% (six websites) by KM. The interrater reliability was acceptable with 83% agreement and a Kappa=0.71 (p=0.01). To examine website characteristics associated with the SMOG readability score and SAM+CAM quality rating (“superior” vs. “adequate and below”), multivariate linear and logistic regression analyses followed by a backward elimination (p values >0.20) were performed (in separate analysis). The website characteristics (website nature, MODY subtypes, whether or not information on inheritance, treatment, genetic testing, and genetic counseling was included) were selected for inclusion in the multivariable analysis using alpha=0.20 during the univariate analyses (two-sample t-test for SMOG readability score and Fisher’s exact test for SAM+CAM quality rating). In multivariate analyses, two-tailed tests and p values <0.05 were used to draw conclusions regarding statistical significance. Data were analyzed using STATA Version 12.0 (STATA Corp, College Station, Texas).

RESULTS

Website content

Of 1200 websites screened, 966 were excluded because they did not contain educational content about monogenic diabetes targeted to the general public, and 205 of the remaining sites were duplicates, leaving 29 (2%) unique websites for detailed analysis. None of the included websites were labelled specifically as patient-targeted resources. The two search terms with a greater than 50% chance of finding relevant information on monogenic diabetes were “monogenic diabetes” and “maturity onset diabetes of the young.”

The majority of these websites (n=24, 83%) included information on subtypes of monogenic diabetes, while five websites discussed monogenic diabetes in general with little or no information on specific subtypes. Ten websites (34%, 10/29) included the most well-known and common forms of MODY, those caused by mutations in HNF1A (MODY3), HNF4A (MODY1), and GCK (MODY2), and at least one of the rarer types of monogenic diabetes including IPF1 (MODY4), TCF2 (MODY5), NEUROD1 (MODY6), KLF (MODY7), CEL (MODY8), PAX4 (MODY9), INS (MODY10), and BLK (MODY11). Eight websites (28%, 8/29) focused solely on neonatal diabetes, most often caused by a mutation in any one of three genes: KCNJ11, ABCC8 or INS. Six websites (21%, 6/29) had a more comprehensive list of subtypes including the three most common forms of MODY, neonatal diabetes, and at least one rarer type of monogenic diabetes.

Most websites were non-profit (educational institutions or governmental organization; n=28, 97%), offered information about inheritance pattern (n=24, 83%), treatment options (n=23, 79%), and general indications for diagnostic genetic testing (n=21, 72%). Few websites mentioned testing indications for subtypes of monogenic diabetes (n=4, 14%). Genetic counseling services were mentioned in only four websites. None of the websites included social networking links to facilitate idea exchange among site visitors.

Readability

Every website reviewed failed to meet highly recommended reading level standards of less than 6th grade level; overall, the mean Flesch-Kincaid grade level was 10.1 (range: 5.7–14.5 grade level; SD = 2.1) and the average SMOG reading level was 12.8 plus or minus 1.5 grades (range: 9.1 to 16.5 grade level; SD=1.8). The Flesch-Kincaid reading level was strongly correlated with the SMOG score (Pearson correlation coefficient=0.9, p<0.001), although the formulas differ somewhat.

Multiple linear regression analysis showed that including information on neonatal diabetes was associated with a significantly higher SMOG reading level (β=1.6, p=0.01) and accounted for 20% of the variance in websites readability. Other content characteristics were not significantly related to the readability level.

Suitability and Comprehensibility (SAM+CAM)

SAM+CAM results indicated that nine (31%) of the websites were rated as “superior,” 12 (41%) were “adequate,” and 8 (28%) were “not suitable”. The quality ratings by category are shown in Table 1. The two quality categories that were best addressed across websites were numeracy and layout/typography, with scores of 89% and 79%, respectively. The poorest ratings (37%) were related to interactivity and motivation, which refer to the use of an exercise of some sort embedded in educational material that requires active engagement by the reader [e.g., completion of a simple risk assessment check list, a true/false knowledge test (with correct answers provided)], or a list of questions to be asked when seeing a health care provider.

Table 1.

Website SAM+CAM scores by category (N=29)

SAM+CAM category SAM+CAM percent score No. (%) of websites per SAM+CAM category
Mean (95%CI) “superior” “adequate” “not suitable”
Total 0.55 (0.46, 0.64) 9 (31%) 12 (41%) 8 (28%)
Content 0.53 (0.44, 0.63) 9 (31%) 9 (31%) 11 (38%)
Literacy demand 0.45 (0.34, 0.55) 8 (28%) 8 (28%) 13 (45%)
Numeracy 0.89 (0.83, 0.96) 24 (83%) 2 (7%) 3 (10%)
Graphics 0.52 (0.27, 0.77) 5 (17%) 4 (14%) 20 (69%)
Layout/typography 0.79 (0.69, 0.89) 19 (66%) 6 (21%) 4 (14%)
Learning stimulation/motivation 0.37 (0.24, 0.51) 5 (17%) 8 (28%) 16 (55%)
Open in a new tab

Websites were overall ranked “adequate” in content, literacy demand, and graphics categories, although more than one-third of websites failed to meet at least minimum standards on at least one of these three categories. “Not suitable” scores were given to 11 (38%) of the websites for content, 13 (45%) for literacy demand, and 20 (69%) for graphics. Fisher’s exact test results showed that none of the website characteristics significantly differed on characteristics ranked as “superior” and “adequate” or “not suitable.” Table 2 summarizes features of websites ranked as “adequate and above” by SAM+CAM criteria and presented information on the common forms of MODY.

Table 2.

Features of websites ranked as “adequate and above” by SAM+CAM criteria and presented information on common forms of MODY

ID Affiliation Website links Site
type		 Subtypes
of
monogenic
diabetes		 Flesch-
Kincaid
gradea 		SMOG
reading
gradeb 		SAM+
CAM
quality
ratingc 		Inheritance
pattern		 Genetic
counseling		 Genetic testing
in general		 Genetic
testing
for
subtypes		 Treatment
1 National Institute of Diabetes and Digestive and Kidney Diseases (US) http://www.niddk.nih.gov/health-information/health-topics/Diabetes/monogenic-forms-diabetes-neonatal-diabetes-mellitus-maturity-onset-diabetes-young/Pages/index.aspx NP MODY1, 2, 3, 4, 5, 6, neonatal diabetes 10.7 13.1 0.80 Y Y Y N Y
2 The Univeristy of Chicago Medicine Kovler Diabetes Center (US) http://monogenicdiabetes.uchicago.edu/what-is-monogenic-diabetes/ NP MODY1, 2, 3, 4, 5, 6, 7, 10, neonatal diabetes 9.0 12.0 0.68 Y N Y Y Y
3 Islets of Hope (US) http://www.isletsofhope.com/diabetes/symptoms/maturity_onset_diabetes_mody_1.html NP MODY1, 2, 3, 4, 5, 6, neonatal diabetes 11.1 13.6 0.50 Y N Y N Y
4 Drugs.com (US) http://www.drugs.com/health-guide/maturity-onset-diabetes-of-the-young-mody.html NP MODY1, 2, 3, 4, 5, 6 9.0 12.0 0.83 Y N Y N Y
5 children with DIABETE S (US) http://www.childrenwithdiabetes.com/clinic/mody.htm NP MODY1, 2, 3, 4, 5, 6 6.3 10.4 0.69 Y N Y N Y
6 University of Exeter Medical School and Royal Devon and Exeter Hospital (UK) http://www.diabetesgenes.org/content/making-diagnosis-mody NP MODY1, 2, 3, 4, 5, 6 10.7 13.3 0.53 Y N Y N Y
7 “Contact a Family” for families with disabled children (UK) http://www.cafamily.org.uk/medical-information/conditions/d/diabetes-%E2%80%93-monogenic-diabetes/ NP MODY1, 2, 3, 5, neonatal diabetes 11.7 13.4 0.50 Y Y Y N Y
8 Ambry Genetics (US) http://www.ambrygen.com/tests/maturity-onset-diabetes-young-mody-testing C MODY1, 2, 3, 4, 5 11.2 13.0 0.58 Y N Y N Y
9 Diabetes.co.uk (UK) http://www.diabetes.co.uk/diabetes_mody.html NP MODY1, 2, 3, 4, 6 9.2 11.6 0.42 Y N N N N
10 Personal blog by Jenny Ruhl http://www.phlaunt.com/diabetes/14047009.php NP MODY1, 2, 3, 5 7.9 10.3 0.87 Y N Y Y Y
11 Diabetes UK charity (UK) https://www.diabetes.org.uk/Guide-to-diabetes/What-is-diabetes/Other-types-of-diabetes/MODY/ NP MODY1, 2, 3, 5 8.8 11.8 0.81 Y N Y N Y
12 The Newcastle upon Tyne Hospitals NHS Foundation Trust (UK) http://www.newcastle-hospitals.org.uk/services/ng_npcg_common_mody.aspx NP MODY1, 2, 3, 5 9.8 11.9 0.75 Y N Y Y Y
Open in a new tab

NP: Non-for profit; C: Commercial; Y: Yes; N: No.

a

Flesch-Kincaid grade level: 6–8 or below.

b

SMOG reading level: superior (<=6th grade level), adequate (6th–9th grade level), not suitable (>=9th grade level).

c

SAM+CAM quality: superior (>70%), adequate (40–70%), not suitable (<40%).

DISCUSSION

This study provides the first overview of online patient-oriented health information on monogenic diabetes. Our findings highlighted the scarcity of online informational and support resources specific for monogenic diabetes. Although the majority of websites were of acceptable quality (ranked as “adequate and above” by SAM+CAM criteria), all failed to meet readability targets suitable for the broad population of patients with diabetes.

While monogenic diabetes is estimated to affect at least 300,000 people in the U.S., comprehensive information specific to this group of genetic diseases is scarce and difficult to find. Only 2% of 1200 screened websites provided relevant information on monogenic diabetes. Content analysis showed that the study websites differed in the number and subtypes of monogenic diabetes presentation. Overall, most sites presented clinical features, inheritance pattern, treatment options, and general indications for diagnostic genetic testing for at least the most common forms of MODY (1, 2 and 3).

Still, some information categories were underrepresented; for instance, few of the websites (n=4) addressed the utility of meeting with genetic counselors or endocrinologists before or after genetic testing, and psychosocial counseling and family planning were also rarely mentioned. Similar gaps in reference to psychosocial counseling and family planning were noted by Pauer and colleagues in their review of available online information about rare diseases available in the German language or referencing German speaking countries and referrals to specialists. Inclusion of genetic counselors was not coded, even though rare diseases are often investigated in terms of genetic abnormalities. An interesting difference in the German website characterization is that 38% of those websites included social media links. Moreover, the information provided by these support groups and patient advocacy organizations was rated as equal in quality to the information provided by websites associated with medical institution (Pauer et al., 2017).

Of the gaps identified in the current study, the absence of links to patient advocacy and support groups is most striking: none of the sites included specific web-based applications that facilitate information exchange with other patients directly on the site or in links to blogs specific for monogenic diabetes, sites for multimedia sharing (such as YouTube) or social networking (such as Facebook). These types of resources may be especially relevant to patients with monogenic diabetes for whom online social networks may constitute the only source of information about the experience of other patients in terms of interactions with the health care system, symptom management, effective treatments, and what sequelae they might expect [38]. Considering the added benefit of social and emotional support that advocacy groups provide but is often missing from institutional websites, the absence of this feature on monogenic diabetes websites underscores the need for institutional/patient partnerships to support and comprehensibly meet the needs of patients newly diagnosed with monogenic diabetes.

None of the identified websites met the recommended reading level for patient education materials of less than 6th grade; SMOG readability scores hovered around 13 +/1 1.5 (B, 2003). High reading burden was especially notable in websites that included the rarer forms of monogenic diabetes (e.g., neonatal diabetes), consistent with the levels reported in readability studies of print and Web-based health information (Agarwal et al., 2013; Albert, 2000; Friedman & Hoffman-Goetz, 2006; Thomson & Hoffman-Goetz, 2007; Walsh & Volsko, 2008). These levels of readability are problematic for a large segment of the diabetic population; as noted by Overland and colleagues (1993), only 21% of diabetic patients could understand material written at a 9th grade level (Overland, Hoskins, McGill, & Yue, 1993). Another study found that patients with type 2 diabetes were less educated than the general population; the proportion with less than high school education was greater for patients with type 2 diabetes (80%) than people without diabetes (40%) (Cowie & Eberhardt, 1995). Given the demonstrated incongruence between the online information readability and the average diabetic patient’s literacy skills, we can expect that the information provided by the websites analyzed herein would be challenging for the targeted patient population.

The results of SAM+CAM further identified a variety of other suitability, comprehensibility, and communication factors that could be challenging for patients. On a positive note, our findings indicate that the majority (72%) of study websites scored in the “adequate” to “superior” range for suitability and comprehensibility for patients with low literacy skills. As seen in Table 1, the categories of numeracy and layout/typography achieved an overall “superior” ranking, primarily because few numbers, percentages, or proportions were used, and readers were not required to make calculations, such as estimating the recurrence risk of monogenic diabetes for their children. Most of the websites were overall ranked “adequate” in content, literacy demand, and graphics categories, but more than one-third of websites failed to reach an adequate rating in these three categories. In particular, lower content scores were due primarily to a lack of a summary in 86% (n=25) of websites, a gap repeatedly reported in other studies on patient educational materials (Fitzmaurice & Adams, 2000; Ryan et al., 2014; Weintraub, Maliski, Fink, Choe, & Litwin, 2004). When a summary of key points covered is included, users are offered a repetition of key points in different words or examples to assist their comprehension. In addition to the high reading grade level, lower scores in the category of literacy demand often resulted from the lack of vocabulary helpers (45%, n=13). For instance, the medical and technical terms (e.g., monogenic, mutation, autosomal dominant, and hyperglycemia) are used in some of the study websites without definition or further explanation, increasing the literacy demand for patients. Using common, explicit words and tools such as glossaries, tutorials, and internal search engines may facilitate reader understanding. Fewer than one-third of the sites included charts, tables, and pictures, which can be especially useful in conveying genetics concepts such as inheritance. Common problems with pictures included anatomical drawings not placed in the context of the entire body as well as images irrelevant to the text content. These findings indicate that even those health information materials that received an overall ranking of “adequate” or “superior” still need improvement in many areas to diminish the demands they place on users with low literacy skills.

The stimulation and motivation category received the lowest scores of all the categories. This finding is similar to low use of interaction strategies found in other patient educational materials (Fitzmaurice & Adams, 2000; Ryan et al., 2014; Weintraub et al., 2004). Since the information is abstract and relatively unfamiliar – and the reading level is high - use of strategies to enhance reader understanding is especially important, such as incorporating question-and-answer formats, patient narratives and stories, short quizzes and risk-assessments, and modeling of specific behaviors (e.g. questions to ask your doctor) that promotes self-efficacy in learning.

We found that some websites were adequate in terms of SAM+CAM categories despite high reading levels, which is consistent with prior studies in patient health education materials (Finnie et al., 2010). The readability assessment tools may overestimate reading difficulty when the same polysyllabic word (e.g. monogenic, diabetes) is used frequently on a web page. They also do not take into account the use of glossaries when scoring a website. In addition, we found the Flesch-Kincaid scale tended to score website content at a lower reading level than SMOG, although the two scales were highly correlated with each other. This variability is consistent with previous researches (Finnie et al., 2010; Friedman & Hoffman-Goetz, 2006) and further supports the importance of using additional criteria to assess the literacy demand of online information on monogenic diabetes.

Study limitations

Some caution is necessary in interpreting our results. The sample of websites reviewed was small and only included websites in English. None of the study websites were labelled specifically as a patient-targeted resource; so it is possible that some were primarly designed for health care providers and thus resulted in high readability level, low stimulation and motivation category scores, and the lack of links to patient advocacy and support groups. Given the short data collection period and the fact that websites are constantly being updated or removed, our findings only presented a snapshot of websites on monogenic diabetes at the time of review. Information accuracy and completeness are outside the scope of this review, but are important when developing patient educational materials. In addition, although the literacy demand and suitability of websites were assessed, we did not evaluate specific patient information needs or their comprehension of the information being offered on these websites.

Practice implications and research recommendations

In sum, our results show that these online educational resources for monogenic diabetes are written at a high reading level and illustrate specific deficits in informational content, suitability and comprehensibility that require major improvement. Better awareness of the quality of information presented and awareness of the links to superior online resources is needed to help health care providers evaluate and recommend educational materials for their patients. Future research is needed to determine the effective ways to improve website readability and quality, and eventually to increase patient comprehension and use of information related to the screening and diagnosis of monogenic diabetes.

Footnotes

Ethical Statement
  • Funding: This study was funded by NIH U01 grant HG007775.
  • Conflict of Interest: The authors declare that they have no conflict of interest.
  • Ethical approval: This article does not contain any studies with human participants or animals performed by any of the authors.
  • Informed consent: For this type of study formal consent is not required.

References

  1. Classification and Diagnosis of Diabetes. Diabetes Care. 2016;39(Suppl 1):S13–22. doi: 10.2337/dc16-S005. [DOI] [PubMed] [Google Scholar]
  2. Agarwal N, Hansberry DR, Sabourin V, Tomei KL, Prestigiacomo CJ. A comparative analysis of the quality of patient education materials from medical specialties. JAMA Intern Med. 2013;173(13):1257–1259. doi: 10.1001/jamainternmed.2013.6060. [DOI] [PubMed] [Google Scholar]
  3. Albert T. Written information for patients. Lancet. 2000;356(9227):434. doi: 10.1016/S0140-6736(05)73584-4. [DOI] [PubMed] [Google Scholar]
  4. Alexa. The top 500 sites on the web. 2016 Retrieved 01/07, 2016, from http://www.alexa.com/topsites/category/Computers/Internet/Searching/Search_Engines.
  5. BW . Health Literacy: A Manual for Clinicians. Chicago, IL: American Medical Association, American Medical Foundation; 2003. [Google Scholar]
  6. Boulos MN. British internet-derived patient information on diabetes mellitus: is it readable? Diabetes Technol Ther. 2005;7(3):528–535. doi: 10.1089/dia.2005.7.528. [DOI] [PubMed] [Google Scholar]
  7. Center, T. P. R. Health Online 2013. 2013 Retrieved 01/26, 2017, from http://www.pewinternet.org/2013/01/15/health-online-2013/
  8. Chumber S, Huber J, Ghezzi P. A methodology to analyze the quality of health information on the internet: the example of diabetic neuropathy. Diabetes Educ. 2015;41(1):95–105. doi: 10.1177/0145721714560772. [DOI] [PubMed] [Google Scholar]
  9. Cowie CC, Eberhardt MS. Sociodemographic characteristics of persons with diabetes. Diabetes in America. 1995;2:85–116. [Google Scholar]
  10. Doak CC, Doak LG, Root JH. Teaching patients with low literacy skills. Lippincott; 1985. [Google Scholar]
  11. Eysenbach G, Powell J, Kuss O, Sa ER. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA. 2002;287(20):2691–2700. doi: 10.1001/jama.287.20.2691. [DOI] [PubMed] [Google Scholar]
  12. Finnie RK, Felder TM, Linder SK, Mullen PD. Beyond reading level: a systematic review of the suitability of cancer education print and Web-based materials. J Cancer Educ. 2010;25(4):497–505. doi: 10.1007/s13187-010-0075-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Fitzmaurice DA, Adams JL. A systematic review of patient information leaflets for hypertension. J Hum Hypertens. 2000;14(4):259–262. doi: 10.1038/sj.jhh.1001003. [DOI] [PubMed] [Google Scholar]
  14. Flesch R. A new readability yardstick. J Appl Psychol. 1948;32(3):221–233. doi: 10.1037/h0057532. [DOI] [PubMed] [Google Scholar]
  15. Friedman DB, Hoffman-Goetz L. A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Educ Behav. 2006;33(3):352–373. doi: 10.1177/1090198105277329. [DOI] [PubMed] [Google Scholar]
  16. Gimenez-Perez G, Caixas A, Gimenez-Palop O, Gonzalez-Clemente JM, Mauricio D. Dissemination of ‘patient-oriented evidence that matters’ on the Internet: the case of Type 2 diabetes treatment. Diabet Med. 2005;22(6):688–692. doi: 10.1111/j.1464-5491.2005.01522.x. [DOI] [PubMed] [Google Scholar]
  17. Hattersley A, Bruining J, Shield J, Njolstad P, Donaghue KC. The diagnosis and management of monogenic diabetes in children and adolescents. Pediatr Diabetes. 2009;10(Suppl 12):33–42. doi: 10.1111/j.1399-5448.2009.00571.x. [DOI] [PubMed] [Google Scholar]
  18. Ad Hoc Committee on Health Literacy for the Council on Scientific Affairs, American Medical Association. Health literacy: report of the Council on Scientific Affairs. JAMA. 1999;281(6):552–557. [PubMed] [Google Scholar]
  19. Helitzer D, Hollis C, Cotner J, Oestreicher N. Health literacy demands of written health information materials: an assessment of cervical cancer prevention materials. Cancer Control. 2009;16(1):70–78. doi: 10.1177/107327480901600111. [DOI] [PubMed] [Google Scholar]
  20. Hernandez LM. Health literacy, eHealth, and communication: putting the consumer first: workshop summary. National Academies Press; 2009. [PubMed] [Google Scholar]
  21. Hutchinson N, Baird GL, Garg M. Examining the Reading Level of Internet Medical Information for Common Internal Medicine Diagnoses. Am J Med. 2016;129(6):637–639. doi: 10.1016/j.amjmed.2016.01.008. [DOI] [PubMed] [Google Scholar]
  22. Kim S, Love F, Quistberg DA, Shea JA. Association of health literacy with self-management behavior in patients with diabetes. Diabetes Care. 2004;27(12):2980–2982. doi: 10.2337/diacare.27.12.2980. [DOI] [PubMed] [Google Scholar]
  23. Kusec S, Brborovic O, Schillinger D. Diabetes websites accredited by the Health On the Net Foundation Code of Conduct: readable or not? Stud Health Technol Inform. 2003;95:655–660. [PubMed] [Google Scholar]
  24. Ley P, Florio T. The use of readability formulas in health care. Psychology, Health & Medicine. 1996;1(1):7–28. [Google Scholar]
  25. Mc Laughlin GH. SMOG grading-a new readability formula. Journal of reading. 1969;12(8):639–646. [Google Scholar]
  26. Overland JE, Hoskins PL, McGill MJ, Yue DK. Low literacy: a problem in diabetes education. Diabet Med. 1993;10(9):847–850. doi: 10.1111/j.1464-5491.1993.tb00178.x. [DOI] [PubMed] [Google Scholar]
  27. Pauer F, Litzkendorf S, Gobel J, Storf H, Zeidler J, Graf von der Schulenburg JM. Rare Diseases on the Internet: An Assessment of the Quality of Online Information. J Med Internet Res. 2017;19(1):e23. doi: 10.2196/jmir.7056. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Ryan L, Logsdon MC, McGill S, Stikes R, Senior B, Helinger B, … Davis DW. Evaluation of printed health education materials for use by low-education families. J Nurs Scholarsh. 2014;46(4):218–228. doi: 10.1111/jnu.12076. [DOI] [PubMed] [Google Scholar]
  29. Schillinger D, Grumbach K, Piette J, Wang F, Osmond D, Daher C, … Bindman AB. Association of health literacy with diabetes outcomes. JAMA. 2002;288(4):475–482. doi: 10.1001/jama.288.4.475. [DOI] [PubMed] [Google Scholar]
  30. Shepherd M. ‘I’m amazed I’ve been able to come off injections’: patients’ perceptions of genetic testing in diabetes. Practical Diabetes International. 2003;20(9):338–342. [Google Scholar]
  31. Shepherd M, Hattersley AT. ‘I don’t feel like a diabetic any more’: the impact of stopping insulin in patients with maturity onset diabetes of the young following genetic testing. Clin Med (Lond) 2004;4(2):144–147. doi: 10.7861/clinmedicine.4-2-144. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Shields BM, Hicks S, Shepherd MH, Colclough K, Hattersley AT, Ellard S. Maturity-onset diabetes of the young (MODY): how many cases are we missing? Diabetologia. 2010;53(12):2504–2508. doi: 10.1007/s00125-010-1799-4. [DOI] [PubMed] [Google Scholar]
  33. Thakurdesai PA, Kole PL, Pareek RP. Evaluation of the quality and contents of diabetes mellitus patient education on Internet. Patient Educ Couns. 2004;53(3):309–313. doi: 10.1016/j.pec.2003.04.001. [DOI] [PubMed] [Google Scholar]
  34. Thomson MD, Hoffman-Goetz L. Readability and cultural sensitivity of web-based patient decision aids for cancer screening and treatment: a systematic review. Med Inform Internet Med. 2007;32(4):263–286. doi: 10.1080/14639230701780408. [DOI] [PubMed] [Google Scholar]
  35. U.S. Department of Health and Human Services. Web Standards and Usability Guidelines. 2016 Retrieved 3/8, 2016, from https://guidelines.usability.gov/
  36. van Esch SC, Cornel MC, Snoek FJ. Type 2 diabetes and inheritance: what information do diabetes organizations provide on the Internet? Diabet Med. 2006;23(11):1233–1238. doi: 10.1111/j.1464-5491.2006.01963.x. [DOI] [PubMed] [Google Scholar]
  37. Walsh TM, Volsko TA. Readability assessment of internet-based consumer health information. Respir Care. 2008;53(10):1310–1315. [PubMed] [Google Scholar]
  38. Web Accessibility Initiative (WAI) Web accessibility standards and guidelines. 2016 Retrieved 3/8, 2016, from https://www.w3.org/WAI/guid-tech.html.
  39. Weintraub D, Maliski SL, Fink A, Choe S, Litwin MS. Suitability of prostate cancer education materials: applying a standardized assessment tool to currently available materials. Patient Educ Couns. 2004;55(2):275–280. doi: 10.1016/j.pec.2003.10.003. [DOI] [PubMed] [Google Scholar]
  40. Weymann N, Harter M, Dirmaier J. Quality of online information on type 2 diabetes: a cross-sectional study. Health Promot Int. 2015;30(4):821–831. doi: 10.1093/heapro/dau019. [DOI] [PubMed] [Google Scholar]

RESOURCES