Analysis of the quality of online resources for parents of children who are late to talk

Abstract

Background and aims

Internet usage worldwide has become a primary source of health-related information and an important resource for parents to find advice on how to promote their child’s development and well-being. It is important that healthcare professionals understand what information is available to parents online to best support families and children. The current study evaluated the quality of online resources accessible for parents of children who are late to talk.

Method

Fifty-four web pages were evaluated for their usability and reliability using the LIDA instrument and Health on the Net Foundation code of conduct certification, and readability using the Flesch Reading Ease Score and Flesch-Kincaid Grade Level. Origin, author(s), target audience, topics discussed, terminology used, and recommendations were also examined.

Results

The majority of websites scored within the moderate range (50–90%) for total LIDA scores and usability, but scored in the low range for reliability (<50%). Significantly higher reliability scores (p < 0.001) were found for sites with Health on the Net Foundation code of conduct certification. Readability fell within the standard range. The largest proportion of websites were American, written by speech-language pathologists, with the most common topics being milestones, tips and strategies, and red flags. Discrepancies were mostly seen in terminology and misinformation, and when present, usually related to risk factors and causes.

Conclusion

Prior to recommending websites to parents, health professionals should consider readability of the content, check that information is up-to-date, and confirm website sources and reputable authorship. Health professionals should also be aware of the types of unclear or inaccurate information to which parents of children who are late to talk may be exposed online.

Introduction

Internet usage is on the rise. As of 31 December 2017, it was estimated that over 4 billion people worldwide use the Internet, representing approximately 54% of the global population (Internet World Stats, 2018). This is a huge increase from the 20% seen a decade prior (Internet World Stats, 2018). Seventy percent of individuals in Canada use the Internet for medical or health-related information (Statistics Canada, 2012). The majority of searches for online health or medical information occur on generic search engines such as Google, Yahoo, and Bing, rather than medical portals such as the National Institutes of Health (Wang et al., 2012). Approximately 6.75 million health-related searches are performed each day on Google alone (Keogh et al., 2014).

Online health information seeking

The Internet has become an important resource for parents to find parenting advice on how to better promote their child’s development and well-being (Suárez-Perdomo et al., 2018). Online and through social media, parents receive information and counseling from experts and other parents (Plantin & Daneback, 2009). The internet provides a way for parents to access information “quickly, conveniently and privately” (Porter & Edirippulig, 2007, p. 518). However, the responsibility to find and access high-quality, reliable educational content online falls on the parent (Suárez-Perdomo et al., 2018), many of whom may not be equipped to evaluate such factors. Although some parents are skeptical about online information, many miss or do not seek out critical information such as the motives behind a website that can be found in their About Us section (Eysenbach & Köhler, 2002; Plantin & Daneback, 2009).

Concern has been raised that the Internet, rather than healthcare professionals, has become the primary source of health-related information for many people (Tonsaker et al., 2014). Fox and Duggan (2013), for example, found that only half of US adults who go online to look up a medical condition follow up with a visit to a medical professional. Of added concern, Fox (2006) found that three-quarters of Internet users who searched for health information online did not check the source or date of the information. This can be problematic as the Internet is unregulated and may provide misleading or incomplete information (Keogh et al., 2014).

Much of the current research evaluating the quality of online health resources has focused on medical conditions and surgeries such as ear, nose, and throat surgery (Goslin & Elhassan, 2013); tonsillectomies (Roshan et al., 2008); carotid disease (Keogh et al., 2014); orthodontic appliances (Verhoef et al., 2015); and familial adenomatous polyposis (Soobrah & Clark, 2012). The majority of this research has shown an overall poor reliability of online resources (Zhang et al., 2015). To date, the quality of information available online for parents of children who are late to talk has not been explored.

Supporting children who are late to talk

Children tend to be classified as late to talk (or late talkers) if, between roughly 18–35 months of age, they are not reaching typical language milestones. A delay in starting to talk is one of the most common reasons that young children are referred for evaluation by speech-language pathologists (SLPs; Rescorla, 2013). Among clinicians and other healthcare professionals, there is much controversy over the optimal identification criteria of children who are late to talk and when treatment is indicated. Late talkers are typically identified around two years of age, when they are expected to be producing and combining words (Bavin & Bretherton, 2013). Typical identification criteria include a delay in acquiring a child’s first 50 words and producing no two-word combinations by two years of age (Kelly, 1998). Up to 20% of toddlers show delayed onset of spoken language (Bavin & Bretherton, 2013). Many of these children (50–75%) catch up to their peers by four years of age or earlier (Paul & Ellis Weismer, 2013), but roughly 25% of these children show persistent, and oftentimes lifelong, impairments in their language (Duff et al., 2015; Reilly et al., 2018). This persistent, lifelong language impairment is known as developmental language disorder (previously known as specific language impairment; Bishop et al., 2016). It is estimated that between 3% and 7% of children have a developmental language disorder (Norbury et al., 2016; Weindrich et al., 2000). Early identification is crucial to support the child with future challenges in academic, speech, and language abilities (Aram & Nation, 1980).

One important component in supporting children who are late to talk is exposing them to language. Quality exposure to language by parents is crucial, as a positive relationship has been found between the amount of speech children hear and the rate with which their language skills grow (Hoff, 2014). The amount of different words used, grammatical structures, and grammatical complexity of the speech are all positive predictors of vocabulary and grammatical development (Hoff, 2014). Taking this into consideration, it is important that parents understand how to support their child’s language exposure. Parents may find information about speech and language online (McAllister et al., 2011). However, as they may be using generic search engines, they could be accessing poor quality sites that lack evidence-based information or are not parent-friendly (McGill & McLeod, 2019). Accurate information can help parents make informed decisions, which can in turn support their child’s development (Young et al., 2005). Thus, it is important to better understand the quality of online sites parents are most likely to be accessing.

Considerations in website quality evaluations

When evaluating the effectiveness of a website, the likelihood of an individual staying on a site (or returning to it) is a strong indicator of its perceived quality and the overall satisfaction of the user with the website (Kim & Stoel, 2004). Three important features that draw parents to return to websites for information are their usability, reliability, and readability.

Usability reflects a website’s ease of use and contributes to the quality of a user’s experience. High usability is important because if a parent is unable to use a website effectively or find the information they are looking for, they may choose to go elsewhere, or not to return (Al-Taha et al., 2016; McGill & McLeod, 2019; Minervation Ltd, 2007).

Reliability examines whether websites provide comprehensive, relevant, and unbiased information (Minervation Ltd, 2007). Several resources and tools are available for evaluating reliability of online content including the Discern Instrument (Charnock et al., 1999), Journal of American Medical Association benchmarks (Silberg et al., 1997), LIDA tool (Minervation Ltd, 2007), and Trust It or Trash It? (Genetic Alliance, 2010). In addition, the Health on the Net (HON) Foundation, an internationally recognized organization that aims to standardize the reliability of medical and health information available online, has developed the HON Foundation code of conduct (HONcode; www.hon.ch/HONcode/) certification to indicate reliability of health information websites (Soobrah & Clark, 2012).

Readability is the ease with which the content of a site is able to be read and understood by the general public without a medical background (Grewal & Alagaratnam, 2013; Zhang et al., 2015). Readability is an important aspect to take into consideration in the creation of all sources of client information (Al-Taha et al., 2016). The US National Institute of Health recommends that patient information be written between a sixth- and seventh-grade level (Al-Taha et al., 2016; Medline Plus, 2017).

Taking these features into consideration, the current study evaluated the quality of available online information for parents of children who are late to talk, specifically evaluating usability, reliability, and readability. In addition, the reliability of sites with and without HONcode certification was compared.

Methods

Search term formation

Search terms were created by asking 26 individuals (nine speech-language pathology students; seven parents of children who were 5–29 years of age; and 10 non-SLP, non-parent adults): “What would you Google if you had a child and were concerned about their language development (i.e. not talking as much as you’d expect, using few words, etc.)?” From responses collected, three general categories of questions were identified: (1) general (e.g. “Why is my child not talking?”); (2) developmental milestones and norms (i.e. “How many words should my child speak by _ months/years?”); and (3) advice (i.e. “How to enhance my child’s language”). A complete list of questions posed is provided in Supplementary Appendix 1.

Based on this, five Boolean search expressions were created:

1. (child OR kid OR toddler OR baby OR infant) (“not” OR can’t OR isn’t OR doesn’t) (talk OR speak)¹

2. (delay OR slow OR disorder OR late) (child OR kid OR toddler OR baby OR infant) language development “late talker” OR “late to talk” OR “slow to talk”

3. (speech OR language) development (milestone OR stage)

4. (normal OR typical) age (child OR kid OR toddler OR baby OR infant) start (talk OR speak)

Filtering results

All searches were conducted on a single day, 9 May 2018, using the five Boolean search expressions listed above and the top three search engines (Google, Yahoo, and Bing; Verhoef et al., 2015), with the top 10 web pages for each search term/search engine combination saved. The top 10 web pages were used because research has shown that people do not typically search past the first page of results (Kaicker et al., 2013). The searches were conducted from a Canadian-based IP address. All searches were completed in Firefox private browsing mode, with the browser history cleared to ensure that web searches were unbiased. This prevented a “filter bubble” effect from occurring where previous searches influence subsequent search results (Wiley et al., 2017).

A flow chart of the systematic website search is provided in Figure 1. A total of 150 web page results were examined. The Boolean search “‘late talker’ OR ‘late to talk’ OR ‘slow to talk’” did not retrieve any relevant web pages in the Yahoo or Bing search engines (n = 20 sites), leaving 130 web pages. Web pages were examined for duplicates (same web page) using the program Zotero (Roy Rosenzweig Center for History and New Media, 2016) and 55 duplicates were removed. Of the remaining 75 web pages, 21 were excluded. Excluded web pages were unrelated to the search topic (n = 7 web pages); had an access fee or login requirement (n = 1); or were promotional product web pages (n = 2), videos (n = 1), journal articles (n = 1), Wikipedia entries (n = 2), or primarily discussion forums or discussion-based blogs (n = 7). These criteria are commonly used reasons for exclusion in evaluations of online health resources (Zhang et al., 2015). For example, although academic journal articles contain evidence-based information regarding the topic area, the level of discussion is often above the reading level of the majority of parents (Hayes, 1992), and frequently are inaccessible to the general public (Khabsa & Giles, 2014). Parent discussion forums and Wikipedia web pages were excluded due to their user-generated content. It is important to note that of the two Wikipedia entries, only one (related to Language delay) shared accurate information. The other, an entry entitled Late talker, contained inaccurate descriptions of late talkers as children who are “exceptionally bright” and “lagging in social development,” which is not the case in many occurrences of late talking (Camarata, 2014). Unfortunately, such inaccuracies could mislead parents into false assumptions about their child’s language, intellectual, and social abilities (Ellis Weismer, 2007).

Figure 1.

PRISMA diagram of the systematic website search.

Analysis

Web pages were analyzed from 18 May to 22 June 2018 for their usability using the Minervalidation tool (LIDA instrument; Minervation Ltd, 2007), for their reliability using both the LIDA instrument and HONcode certification status, and for their readability using the Flesch Reading Ease (FRE) Score and the Flesch-Kincaid Grade (FKG) level (My Byline Media, n.d.). Web pages were also analyzed for topics discussed and other website characteristics.

Website characteristics and topic evaluation

Web pages were analyzed for location of origin, author(s), target audience, and website type. Additionally, web pages were analyzed for topics discussed, terminology used, and what, if any, recommendations were provided.

Usability and reliability

The LIDA instrument is a validated 41-question tool that can be used to assess the accessibility, usability, and reliability of health care websites (Minervation Ltd, 2007). At the time of this analysis, the LIDA accessibility evaluation tool was no longer available online and, therefore, was not included in the present analysis. Each question was rated on a four-point Likert scale: 0 (Never), 1 (Sometimes), 2 (Mostly), and 3 (Always).

Usability was assessed through 18 questions pertaining to: clarity of information (six questions); consistency of website design (three questions); functionality of browsing and search functions (five questions); and the overall engagability of the web page (four questions). The maximum score for usability is 54, which was then converted into a percentage.

Reliability was assessed via 17 questions on website update frequency (three questions), conflicts of interest (three questions), and methodology of content production (three questions). Eight additional supplemental questions were used to evaluate how the content was created (five questions), as well as the accuracy of content (three questions). Including the supplementary questions, the maximum score for reliability is 51, which was converted into a reliability percentage score.

Usability and reliability scores were totaled to create an overall LIDA score out of 105.² LIDA scores were recorded as percentages and web pages were classified as high (>90%), moderate (50–90%), or low (<50%). For example, if a web page scored 12/18 on questions about clarity, 9/9 in consistency, 10/15 in functionality, and 10/12 in engagability, the usability score would total to 41/54 (76%), placing it in the moderate range for usability. Extending on this, if the same web page scored 15/27 on reliability questions (summing results from currency, conflicts, and content production) and 9/24 on supplementary questions, this would sum to a reliability score of 24/51 (47%), placing it in the low range. Adding usability and reliability total scores together, the total LIDA score would be 65/105 (62%), placing it in the moderate range.

Inter-rater reliability of LIDA scoring

Usability and reliability were rated by the first author (CC) and 20% of the web pages (n = 12) were also evaluated by an independent graduate student in the field of speech-language pathology. Intraclass correlations between the two raters were calculated for each variable. In instances where a given variable did not receive an inter-rater reliability score of 0.7 or higher, the ratings were discussed until consensus was achieved. Table 1 includes inter-rater reliability with intraclass correlation coefficients. Inter-rater reliability exceeded .83 on all ratings.

Table 1.

Intraclass correlation coefficients for interrater reliability, absolute-agreement, 2-way random-effects model for coding of web pages included in analyses (n = 54).

	Intraclass correlation	95% confidence interval		F test with true value 0
		Lower bound	Upper bound	Value	df1	df2	Sig
Total usability	0.83	0.499	0.949	12.9	11	9.53	0.000236
Clarity	0.821	0.49	0.945	11.7	11	10.3	0.000225
Consistency	0.879	0.648	0.963	15.5	11	12	0.0000194
Functionality	0.93	0.763	0.98	32.8	11	9.52	0.00000383
Engagability	0.844	0.552	0.952	11.3	11	11.6	0.000131
Total reliabilitya	0.899	0.451	0.975	32.3	11	4.16	0.00176
Clarity	0.833	0.538	0.948	11.5	11	11.9	0.000103
Conflicts of interest	0.884	0.661	0.965	16.3	11	12	0.0000152
Content production	0.935	0.784	0.981	34.7	11	9.97	0.00000183
Total supplementary	0.853	0.449	0.959	18	11	6.12	0.000924
Content production	0.888	0.546	0.97	24.3	11	6.04	0.000424
Output of content	0.949	0.84	0.985	38.1	11	12	0.000000132

^aTotal including supplementary scores.

As a secondary evaluation of reliability, each website was also checked for presence of HONcode certification, which is awarded to sites that adhere to the eight HONcode principles: authoritative (indicates qualification of the authors), complementary (information should support, not replace the doctor–patient relationship), privacy (respects the privacy and confidentiality of personal data), attribution (cites the source of published information, dates medical and health pages), justifiability (backs up claims relating to benefits and performance), transparency (accessible presentation, accurate contact information), financial disclosure (identifies funding sources), and advertising policy (distinguishes advertising from editorial content; Grewal & Alagaratnam, 2013). The HONcode toolbar plug-in³ was installed in Google Chrome. This plug-in allowed easy access to the HONcode seal of approval. HONcode certification is only visible when the HONcode toolbar plug-in has been installed by the user or if a website address is searched via the HONcode website. To evaluate whether HONcode certification does indeed reflect better webpage reliability, LIDA reliability scores of web pages with and without HONcode certification were compared using a t-test.

Readability

Readability was assessed using FRE score rating and the FKG level, which take syllable and word count into consideration when calculating the readability of English text. A sample of text from each web page was analyzed automatically via the website: www.readabilityformulas.com/free-readability-formula-tests.php, which has been used in previous research (Verhoef et al., 2015). The FRE score rates English text on a 100-point scale, where higher scores indicate material that is easier to read (Grewal & Alagaratnam, 2013). Scores between 60 and 70 represent a standard readability level, easily understood by 13- to 15-year-old students. FRE scores range from 0 to 100 and are categorized according to: very confusing (0–29), difficult (30–49), fairly difficult (50–59), standard (60–69), fairly easy (70–79), easy (80–89), and very easy (90–100). FKG level uses the same core measures of FRE score, with a different weighting, translating the score into a US grade level (Grewal & Alagaratnam, 2013). For example, a score of 8.2 would be understandable by an average eighth grader (12–14 years old; Grewal & Alagaratnam, 2013).

Results

Of the 54 web pages evaluated, several were identified as being from within the same broader website (i.e. a collection of web pages). The most frequent website was babycenter.com, from which five web pages were included in the current analysis. Other websites with multiple included web pages are listed in Table 2.

Table 2.

Websites with multiple web page occurrences.

Website title	Website address	Number of unique web pages identified by search and included in analysis
Babycenter (USA)	https://babycenter.com	5
The Hanen Centre (Canada)	http://hanen.org	3
Kid Sense (Australia)	https://childdevelopment.com.au	2
We Have Kids (unknown)	https://wehavekids.com	2
American Speech-Language-Hearing Association (USA)	https://asha.org	2
Parents (USA)	https://parents.com	2
WebMD (USA)	https://webmd.com	2

Website characteristics

Table 3 displays the characteristics of the web pages including author, target audience, and category (website type). Website type and page authorship were characterized based on declared authorship and website organizational information. In contrast, target audience was evaluated based on who the specific web page was directly written for. A complete breakdown of characteristics by web page number can be seen in Supplementary File 2.⁴

Table 3.

Web page characteristics.

	Number (%) of web pages
Author
Speech-language pathologist	14 (25.9)
Parent	10 (18.5)
Physician	9 (16.7)
Not listed	8 (14.8)
Organization	5 (9.3)
Non-expert	4 (7.4)
Clinic	3 (5.6)
Psychologist	1 (1.9)
Target audience
Parents	42 (77.8)
Professionals	10 (18.5)
General public	8 (14.8)
Teachers	3 (5.6)
Website type
Parent	24 (44.4)
Speech-language pathologist	12 (22.2)
Clinic	6 (11.1)
General medicine	4 (7.4)
Government	3 (5.6)
Education	2 (3.7)
Other	2 (3.7)
Product	1 (1.9)

Location, author, and target audience

The majority of the web pages (63%) were based in the United States, with the second most frequent location of origin being Canada (13%). The remaining sites were based in Australia (9%) and the United Kingdom (6%) or were unknown (6%). Of the 54 web pages, 14 (26%) were written by SLPs (currently practicing or retired). The second most common authorship was parents (19%). Background information discussing the expertise of the author was only provided in 12 of the 54 (22%) web pages. We used the category Physician to represent web pages written by a pediatrician or physician, or reviewed by a medical advisory board. The majority of web pages (42/54; 78%) were targeted at parents. This was deduced from Mission Statements and About Us sections of the websites. Ten of the web pages (19%) were targeted at professionals. Professionals in this context included early childhood educators, SLPs, and audiologists.

Website type

Most web pages were from parent (directed) websites (24/54; 44%), indicating that not only was the web page targeted to parents, but the complete website was designed specifically for parents. These parent sites included a range of advice for parents for various situations they would encounter across their child’s development. Four of the web pages were characterized during analysis as from general medical sites (e.g. WebMD, Healthline), which provided information about a variety of health conditions. The category Other included two web pages of authors Paula Spencer Scott and Janet Lansbury, respectively, that did not fit into any of the other classifications. Paula Spencer Scott (see web page #19) is a “wellness, brain health, and family-care specialist”, whose site discusses parenting and dementia care. Janet Lansbury (see web page #41) is an “actor turned infant educarer” whose site focuses on her philosophy of child care and offers private one-on-one consultations and resources based on the Resources for Infant Educarers framework.

Topic evaluation

Supplementary File 3⁵ provides a complete breakdown of terminology used, topics discussed, and recommendations provided, sorted by web page ID number (used throughout to refer to specific web pages). Proportion of web pages in each of these areas is summarized in Table 4.

Table 4.

Terminology used, topics discussed, and recommendations made for who to consult with concerns.

	Number (%) of web pages
Terminology used
Language delay	27 (50)
Speech delay	20 (37)
Late talker	15 (28)
Expressive language delay	3 (6)
Developmental delay	3 (6)
Receptive ± expressive delay	3 (6)
Late bloomer	1 (2)
Expressive language disorder	4 (7)
Developmental language disorder	1 (2)
specific language impairment	1 (2)
Topics discussed
Milestones	38 (70)
Tips and strategies	30 (56)
Red flags	27 (50)
Causes/risk factors	19 (35)
Importance of early intervention	10 (19)
Associated difficulties	6 (11)
Consequences of no treatment	5 (9)
Who to consult with concerns
Physician (pediatric or family doctor)	31 (57)
Speech-language pathologist	26 (48)
Early intervention program (including  preschool speech and language program)	8 (15)
Audiologist	8 (15)
Hearing evaluation	7 (12)
Speech and language evaluation	4 (7)
Psychologist	3 (6)
Nurse	2 (4)
Teachers	2 (4)
Occupational therapist	2 (4)
Developmental screening	1 (2)

Terminology and topics discussed

The terminology most commonly used when discussing a child who was not speaking at an expected time point was “language delay” (used in 50% of the web pages), closely followed by “speech delay” (37%). It is important to note that 4 out of the 20 web pages (20%) differentiated speech delay from language delay (the remaining 16 used them interchangeably). Six of the 54 web pages (11%) discussed the possibility of a division between children who are late to talk (or “late bloomers” as described in one web page) and those who may continue to have difficulties. The most common choice of terminology to describe persistent language difficulties into school age was “expressive language disorder” (4/54 web pages). Three sites used “developmental language disorder” and one web page (web page #38) used the term “specific language impairment”.

The majority of the web pages (38/54; 70%) described speech and language milestones. This included children producing their first word by one year of age (web pages #1, #10, #19, #29, #33, #49, #54), using at least 20 words by 18 months of age (web pages #1, #3, #9, #17, #36, #39, #54), and saying at least 50 words by two years of age (web pages #1, #2, #6, #9, #10, #18, #21, #22, #24, #25, #26, #37, #51, #54), while also starting to put two words together (web pages #1, #2, #6, #9, #10, #17, #18, #21, #22, #24, #25, #26, #33, #36, #37, #43, #49, #51, #54).

The second most common piece of information shared on the web pages was tips and strategies to help with child speech and language acquisition (30/54; 56% of web pages). This included encouraging the use of “parentese” (also known as “motherese”, infant-directed speech, or child-directed speech; defined as the manner in which adults modify their speech when talking to children including the use of a higher-pitched voice, wider range of pitches, longer pauses, and shorter phrases; Hoff, 2014), narrating daily activities (describing things you see and actions you do with your child), reading with children, and expanding on children’s utterances.

Recommendations

A wide range of recommendations were provided to individuals should they be concerned. The majority of the web pages (33/54; 61%) suggested that individuals see their physician if they have any concerns about their child’s speech and language. Six of these web pages solely recommended a doctor visit (without the possibility of seeing other professionals). Just under half (48%) advised consulting a SLP. Eight web pages (15%) provided no recommendation as to whom to seek out if there were concerns about speech and language.

LIDA instrument: Usability and reliability

Overall

For all 54 of the web pages analyzed, the mean LIDA score was 60/105 (57%), ranging from 48–77 (46–73%, SD ± 7%), which falls within the moderate range. Figure 2 illustrates the distribution of LIDA scores overall, as well as usability, reliability, and FRE scores. Figure 3 shows the total LIDA scores, as well as usability and reliability scores, divided by classification of low, moderate, and high percentages. In general, total scores for usability, reliability, and the overall LIDA fell in the ranges of low and moderate, with none scoring greater than 90% (high range). Distribution of LIDA subset scores (by low, moderate, and high) can be seen in Table 5.

Figure 2.

Median box distribution of rating scores for the 54 web pages included in the current analysis: FRE score (Flesch Reading Ease Score; My Byline Media, n.d.) for readability, LIDA (Minervalidation tool; LIDA instrument total = Reliability + Usability scores; Minervation Ltd, 2007), Reliability (from LIDA instrument), and Usability (from LIDA instrument).

Source: reproduced with permission Minervation Ltd. (2007) and My Byline Media (n.d.).

Figure 3.

Percentage of web pages (total n = 54) achieving high, moderate, and low LIDA scores for each LIDA category (from Minervalidation tool; Minervation Ltd, 2007). Supplementary scores included in reliability score.

Source: reproduced with permission Minervation Ltd. (2007).

Table 5.

Distribution of web pages according to LIDA subset percentage scores.

	High score (≥90%)No. of web pages	Moderate score (50–90%)No. of web pages	Low score (<50%)No. of web pages
Usability	0	53	1
Clarity	1	53	0
Consistency	36	18	0
Engagability	3	25	26
Functionality	0	43	11
Reliabilitya	0	22	32
Reliabilityb	0	25	29
Conflict of interest	9	44	1
Currency	0	8	46
Content production	2	14	38
Supplementary	0	17	37
Content productionc	0	1	53
Output content	16	34	4
Total LIDA scores	90	41	13

Note: Score ranges based on LIDA instrument (Minervalidation tool) instructions from Minervation Ltd. (2007).

Source: reproduced with permission Minervation Ltd. (2007).

^aReliability score including supplementary scores.

^bReliability score excluding supplementary scores.

^cSupplementary.

Usability

The mean LIDA score for usability was classified as moderate, at 37/54 (68%), with a range from 25–45 (46–83%, SD ± 9%). The lowest scores were seen in engagability, with 26/54 (48%) of web pages scoring in the low range. Engagability includes interactiveness of the website, ability to personalize experience using the website, and use of non-textual media (i.e. images and videos). As seen in Table 5, the highest scores were observed for consistency (i.e. whether a consistent design is used throughout the website for links, page layout, and site structure) across the web pages.

Reliability

The mean LIDA score for reliability was found to be low, at 23/51 (46%), with a range from 12–38 (24–75%, SD ± 12%). Twenty-four of the 54 web pages (44%) provided references. Information about the date that material was written or updated was provided in 33 of the 54 (61%) web pages; however, 10 of these 33 web pages (30%) were created more than five years ago. As seen in Table 5, the majority of the web pages (38/54; 70%) disclosed very little information about their content production method, scoring them in the low range.

HONcode

Fourteen of the 54 web pages (26%) had been HONcode-certified and maintained their certification (through renewal after the initial certification period expired). As seen in Table 6, HONcode-certified web pages had, on average, significantly higher reliability (p < 0.001) and overall LIDA scores (p < 0.001) than web pages from websites without HONcode certification. Five of the seven web pages scoring in the top five for reliability were HONcode-certified (21 rankings total; Table 7). A complete list of readability, usability, reliability scores, and rankings, including information about which web pages had HONcode certification, can be seen in Supplementary File 2 (sorted by web page ID number).

Table 6.

Comparison of average scores of web pages according to HONcode (Health on the Net Foundation code of conduct) certification status.

	HONcode certified (n = 14)	Not HONcode certified (n = 40)	p Values
Mean usability score (SD)	72 (8)	67 (9)	0.06
Mean reliability score (SD)	57 (8)	41 (10)	<.001*
Mean LIDA score (SD)	65 (4)	55 (6)	<.001*
Mean FRE score (SD)	63 (11)	58 (16)	0.21

Note: Standard deviation (SD) included in parentheses. Scores represented as percentages (out of 100), with higher scores indicating greater reliability.

Table 7.

Top five ranked web pages by LIDA reliability score.

Web address	HONcode certification	Reliability score (%)
http://mayoclinic.org/healthy-lifestyle/infant-and-toddler-health/in-depth/language-development/art-20045163	Yes	74
https://webmd.com/parenting/baby/recognizing-developmental-delays-birth-age-2	Yes	68
http://raisingchildren.net.au/articles/language_delay.html	No	67
https://webmd.com/parenting/baby-talk-your-babys-first-words	Yes	67
https://healthychildren.org/English/ages-stages/toddler/Pages/Language-Delay.aspx	Yes	63
https://medicinenet.com/script/main/art.asp?articlekey=52130	Yes	63
https://cdc.gov/ncbddd/actearly/milestones/index.html	No	59

Note: Reliability scores created using LIDA instrument (Minervalidation tool; Minervation Ltd., 2007).

Source: reproduced with permission Minervation Ltd (2007).

Readability

The FRE score average across all web pages was 59.5 out of 100 (3.5–89.6, SD ± 15), putting the overall average in the standard range (Figure 2). As seen in Figure 4, the highest proportion of these web pages (19/54) fell in the fairly difficult range. Similarly, the FKG mean score was evaluated as at a grade 10.5 reading level (4.1–32.9, SD ± 5), which is high given the standard recommendation that material be at a grade 8 reading level.

Figure 4.

Distribution of FRE (Flesch Reading Ease; My Byline Media, n.d.) scores in reviewed web pages.

Source: reproduced with permission My Byline Media (n.d.).

Discussion

In this age of widespread Internet usage, it is important for SLPs to help guide families to the optimal and most accurate sources of information. The vast collection of easily accessible and usable information makes the Internet appealing for professionals and parents alike (Cline & Haynes, 2001). This ease of access marks the emergence of the “self-educated” patient (Al-Taha et al., 2016), who arrives to appointments with a myriad of information previously collected online. Now, more than ever before, it is important to know what kind and quality of information is available to the general public online. Results from the current study provide key information into what parents of children who are late to talk may be retrieving online and important considerations for recommending websites to them.

Content retrieved: The good, the bad, and the ugly

The first goal of this study was to retrieve web pages that parents may seek out when concerned about their child’s language development. The topics discussed in the web pages retrieved included information on typical communication development milestones, red flags, where to receive services if concerned, and strategies to support children’s communication development. This aligns with the content that participants noted they wanted to see online in McGill and McLeod’s (2019) study about web-based information needs for children and families waiting for speech-language pathology services.

This information is important for parents to have access to because many children wait to receive assessments or intervention after referrals (Pickstone, 2014; Ruggero et al., 2012; Rvachew & Rafaat, 2014). As speech and language impairments are known to increase a child’s risk of later reading and behavioral problems, it is important that intervention is provided early, prior to entering school (Morgan et al., 2016). Parents may also not be aware of what services are available to them or may not know where to begin the process. In the current study, the majority of web pages recommended that parents visit a doctor/pediatrician if concerned, with a proportion of these discussing the possibility of further referrals such as to a SLP. Two web pages (#10 and #18) discussed the possibility that a child’s pediatrician may pass off late talking as not a concern, claiming, “he’ll talk when he’s ready”. These web pages encouraged seeking a second opinion if still concerned.

While waiting for services, receiving information on strategies to support children’s language development may empower families to engage in active waiting and alleviate some of the feelings of stress and uncertainty, which could ultimately lead to improved language outcomes (Fordham et al., 2012; McGill & McLeod, 2019). Strategies discussed in the current study included ones previously shown to be successful in parent–child interaction, including family-focused programs for parents such as the Hanen Programs® (Roberts & Kaiser, 2011). This includes encouraging face-to-face talking (get down on their level), reading together, narrating day-to-day activities with their child, and expanding on their child’s utterances.

In addition to appropriate strategies, the web pages reviewed in this study provided fairly accurate information regarding typical speech and language milestones. Discrepancies of information were mostly seen in risk factors/causes. One common misconception discussed (see web page #12) was that second-born or later-born children develop language later than their older siblings. Several web pages discussed the need for screening for autism spectrum disorder when late talking occurs (see web pages #11, #19, #21, #22, #42, #53). Although late talking is a common feature in children with autism, the majority of late talking children do not have autism (Camarata, 2014). One web page (#19) discussed late talking as being due to a visual-spatial-analytical learning style, with those on the extreme end of the spectrum being extremely bright with “Einstein syndrome”. Most children who are late to talk do not fit this pattern (Camarata, 2014) and the concept of learning styles does not have scientific support (Willingham et al., 2015). Eight other web pages suggested that late talking was a result of ear infections and fluid build-up (see web pages #9, #15, #17, #18, #20, #22, #35, #44). While recurrent middle ear infections (otitis media) in children with documented hearing loss or other developmental issues may confer risk, they do not alone have any meaningful associations with speech and language development in otherwise healthy children (Roberts et al., 2004; Rosenfeld et al., 2016). In addition, another web page (#47) claimed there has been a rise in language problems among children, which they noted was due to increased television time, forward facing buggies (reducing face-to-face time with parents), and busy parents who communicate through rhetorical questions. These claims are unsupported in the literature and may lead to unwarranted panic or misplaced blame among parents. Finally, one web page (#18) claimed that parents should be concerned if the child was not using all consonant sounds in the first year of life. This is unrealistic as consonant sound acquisition continues well beyond the first year (up until six years) in typical development (Smit et al., 1990). Ultimately, these results show that clinicians should be prepared for a myriad of possible concerns and queries from parents based on misinformation that is readily available online. Awareness and accuracy of what materials and information is shared online is essential for clinicians to provide support, clarification, and correction of misinformation.

Clinical significance: Increasing parent website quality awareness

When discussing information retrieved online with parents, or when recommending websites, a number of considerations should be made by clinicians. Important discussions about what constitutes a quality website include trustworthiness of sources, authorship, location of website origin, readability, and overall usability of website.

Increasing parents’ awareness of what constitutes a quality website should begin with discussions around reliability or trustworthiness of sources. Web pages scored in our study rated in the low range for reliability. A lack of quality control standards online is a phenomenon that has been observed in websites of many other specialties (Borgmann et al., 2017; Goslin & Elhassan, 2013; Roshan et al., 2008; Soobrah & Clark, 2012; Zhang et al., 2015). However, several indicators can be used to easily help parents determine if a website can be trusted. In the present study, factors found to contribute to these lower reliability scores included lack of referencing, lack of disclosure of content production method, and out of date and/or unreliable information. Discussing with parents the need to check for sources and up-to-date information is a starting point to improving information seeking behaviors and media literacy. Another indicator of higher quality reliable websites is whether the site has received HONcode certification. In the current study, a clear discrepancy was seen in reliability and total LIDA scores, with HONcode-certified sites showing significantly higher quality of information than those that were not certified. In the future, encouraging sites to strive for HONcode certification may significantly increase the standardization and quality of healthcare websites, promoting higher reliability of web content. As well, showing parents how to install the HONcode plugin to ensure the websites they are using have been vetted will increase the likelihood that they are accessing accurate content.

Another important discussion is around terminology. It is important for clinicians to be aware of the terminology to which parents may have been previously exposed and may be using. This enables clearer discussions addressing concerns. Although web pages in the current study described “language delays”, the terminology “language delay” and “speech delay” were used interchangeably when discussing a child who had not reached appropriate milestones. Speech delays refer to when children produce a restricted range of speech sounds, making speech unintelligible, whereas a child with language delays may not have difficulties producing sounds but rather have poor understanding of what others say, limited vocabulary, and immature sentences (Paul, 2012). One possible reason for differences in terminology may be due to authorship. Expert authorship is important when seeking information and advice concerning typical development and how to support language acquisition (Suárez-Perdomo et al., 2018). The current study found that approximately one-quarter of web pages reviewed were authored by SLPs. In contrast, the second most common authors were parents. This suggests that much of the information parents may be receiving online is coming from fellow parents rather than experts. Discussions surrounding seeking out (or checking for) reputable authors is important, as non-expert authors may be sharing information that is based on personal experiences and is not generalizable to others.

It is important that clinicians and parents are aware of the geographic location or origin of websites. In the current study, the majority of the websites available to Canadian parents online came from locations in the United States. Location is particularly important to be aware of when considering the recommendations provided to parents. For instance, several of the web pages recommended that individuals contact their state’s early intervention agency. This is a service that is not necessarily available outside of the United States. For example, in Ontario, Canada, children would be directed to their local, publicly-funded community preschool speech and language services, but this would be different in other Canadian provinces and presumably in other countries as well. Lack of information about the service delivery model in their area may frustrate parents and discourage them from seeking out help when concerned.

Prior to referring parents to websites, it is also recommended that clinicians assess the readability of sites and ensure they are at the appropriate level (Woods, 2019). Although the web pages retrieved in this study were, on average, written in the appropriate, standard grade eight reading level for FRE scores, more than half fell within the fairly difficult or more difficult ranges, with FKG scores also higher (on average) at a grade 11 reading level. These higher reading levels may make it difficult for parents to retrieve and interpret relevant and necessary information (Al-Taha et al., 2016).

High quality, reliable, readable information is vital. However, it is equally important to have the information presented in a manner that is easy both to navigate and to understand. Websites that have higher usability are more enjoyable to access and parents may be more likely to return to them to seek out information on future questions they may have. In the current study, almost all web pages scored within the moderate range for usability on the LIDA instrument. Only one web page scored within the low range (see web page #1), which was a result of poor navigability (functionality score). Overall, this indicates that most of the web pages were fairly easy to use. The lowest usability scores fell within the categories of engagability and functionality, two fundamental characteristics of usability. Several sites were difficult to navigate due to poor browsing design and a lack of search options (or poor functioning searches). Improving on site functionality through better site organization, more efficient search engines, and more engageable non-static content would likely increase the number of return visitors to sites (Keogh et al., 2014). Recommending websites that have ease of access, good navigability, and generally high usability will aid parents in more easily finding information to support their child’s language development.

Study limitations

Several limitations should be considered when interpreting these findings. First, readability scores such as FRE score and FKG rely on formulae that take into consideration only the number of syllables in a word and the number of words in a sentence. Thus, such scores may not accurately reflect the actual reading level by lacking consideration of writing style and explanations of jargon (Soobrah & Clark, 2012). Factors such as illustrations and layout may also affect a reader’s comprehension of the material (Keogh et al., 2014), but were not considered in the current study.

Second, although the LIDA instrument provides several criteria to consider, it is a subjective analysis. Thus, there is a possibility of bias during assessment (Keogh et al., 2014). In order to prevent this, an independent scorer was trained and inter-rater reliability was calculated.

Third, based on the ever-changing nature of the Internet and search algorithms, the web pages retrieved in this study are time-specific, corresponding with the specific day on which the search was conducted. Additionally, the web pages retrieved are merely a subset of the information available online to parents. There is a high likelihood that many parents would perform multiple searches with varied search terms to gain the relevant information they seek (Hölscher & Strube, 2000). On this note, there is a wide-range of variability in how people formulate their searches. As can be seen in Supplementary Appendix 1, the majority of the search expressions provided by the individuals we polled were full sentences. We elected to use keywords based on these sentences in order to optimally retrieve results through a variety of word choice possibilities. Search engine optimization was taken into consideration in the creation of the five Boolean search expressions used, exploring key word frequency as well. Although the terms we searched were seen in the search questions posed by those we polled, the websites we retrieved may not necessarily represent sites that individual parents will retrieve.

Most parents would not clear their browser history when conducting searches to ensure unbiased results, as we did in this study (Wiley et al., 2017). Thus, a filter bubble effect will occur where algorithms used by the search engine, such as Google’s Personalization Search function, will return results based on sites the parent has previously visited, rather than solely on the relevance of the website (Wiley et al., 2017).

Finally, it is important to note that of the three search engines used, Bing and Yahoo search results differed only by two web pages. This is because Yahoo is powered by Bing (i.e. the two use the same index; Seymour et al., 2011). The search engine DuckDuckGo, which does not track any personal information, and emphasizes protecting searchers’ privacy, was taken into consideration as a search engine option (Nandy et al., 2016). It was unclear at the time of the study how many parents would be aware of the possibility of DuckDuckGo as a search engine or would be using it. Thus, Google, Yahoo, and Bing were chosen, based on the fact that they are the top three search engines used worldwide (Al-Taha et al., 2016).

Conclusion

Overall, this study shows that several of the web pages parents might access for information on children who are late to talk do not necessarily contain reliable information. However, web pages with HONcode certification had on average higher reliability scores than those that were not certified, suggesting the need for greater use of or more standardized certifications for web content. Awareness of tools such as HONcode for parents might serve to increase the probability that they are retrieving accurate, reliable information. Our findings also showed that web pages were moderately usable and varied in readability scores. Better use of images, videos, and improved navigability and searchability might ease use and increase education (Palma et al., 2016). Prior to SLPs recommending websites to parents, readability of content should be screened, up-to-date information should be checked, and sources and reputable authorship should be confirmed.

Information shared on the web pages examined included typical speech and language milestones, tips and strategies for expanding language, and red flags. Overall, the sites provided fairly accurate information regarding milestones and strategies, although less than half suggested consulting a SLP about concerns. Discrepancies and misinformation, when present, were mostly seen in terminology and risk factors. Further studies examining online resources relevant to speech-language pathology and the effectiveness of clinician initiatives to improve parent web page choices using the suggestions outlined in this paper are warranted.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Caitlin Coughler https://orcid.org/0000-0002-3260-2423

Janis Oram Cardy https://orcid.org/0000-0002-7170-6145

Supplemental material

Supplemental material for this article is available online.