Role of Information Available Over the Internet: What are the Parents of Children Undergoing Tonsillectomy Likely to Find?

Abstract

INTRODUCTION

The aim of this study was to assess the quality of information available on the world wide web to parents of children undergoing tonsillectomy.

MATERIALS AND METHODS

The main data source was from internet searches using the five most popular search engines and the keyword ‘tonsillectomy’ with default settings. The first 50 web links in each search were evaluated with the LIDA Instrument (assessing accessibility, usability and reliability criteria). We also assessed the readability of the sites using the Flesch reading ease score (FRES).

RESULTS

Of the 250 possible links, 113 new links were included the remaining being repetitions, inactive links or restricted access sites. The websites had an average accessibility score of 42/63 (66.7%; range, 26–57), a usability score of 29/54 (53.7%; range, 7–49), and are liability score of 17/51 (33.3%; range, 0–49). The Flesch reading ease score was 43.8 (range, 3–84.4).

CONCLUSIONS

Health information available on the internet varies greatly. Highly ranked websites on popular search engines may not be the most reliable. Overall, the websites had low scores for reliability, with poor engagability, content production and conflict of interest declaration. Patients should be given previously assessed references on the internet to prevent them being misled by in accurate or commercially motivated information.

The explosion of internet usage by patients and the amount of healthcare information publicly available has led to new challenges for practising physicians.^1,2 The information available from a third party has important implications for the doctor–patient relationship and can be used to enhance the healthcare experience.^2,3

Freedom of information on the internet dictates that anyone can publish information. Therefore, there is a risk that such information, through ignorance or bias, may be incorrect or misleading.^4–6 Information of dubious quality or that is commercially motivated can be misleading, or even potentially lethal.^7,8 Many tools are available to assess healthcare information on the internet and, although there is consensus on key criteria, they are still being developed.^9–11 The LIDA Instrument is a validated method of evaluating healthcare websites based on three important areas–accessibility, usability and reliability.¹²

Search engines are the most commonly used method to find information on the world wide web. Information exists about the usage characteristics of search engines in the UK. Ratings such as the Neilson's Net rating¹³ provide data for the most widely used search engines, and patterns therein. The five most visited search engines (Google, Yahoo, MSN, AOL and AskJeeves) field > 90% of all search questions according to these surveys. Although searches often reveal thousands of links, a typical user is likely to visit the most highly ranked on the list.¹⁴ In addition, a major concern for website developers is how to improve their rankings on popular search engines, their motives not necessarily being altruistic.¹⁵ Thus, a higher ranked website is no indication of reliability or relevance.

Another important factor is the structure of websites – can people understand what information is being conveyed?¹⁶ For information directed at the public, it is important that understanding is not lost behind complex scientific vocabulary. Readability tests have been designed to indicate how difficult a reading passage is to understand.¹⁷ In the Flesch Reading Ease test, higher scores indicate material that is easier to read; lower numbers mark harder-to-read passages. Use of this scale is so ubiquitous that it is bundled with popular word processing programs such as Microsoft Word®.

Tonsillectomy is the most commonly performed ENT operation accounting for 50,531 procedures in English NHS Trusts (2003–2004).¹⁹ Tonsillectomy accounts for over 20% of all ENT surgery, and is the most commonly performed operation on children in the UK. It is a low-risk operation with few complications,²⁰ the majority of which are not serious. Complications include difficulty swallowing, vomiting, fever and excessive pain. Postoperative bleeding may also occur, either soon after the operation while the patient is in hospital or, classically, 7–10 days post surgery. Post-tonsillectomy sequelae are often harmless, but lead to in appropriate calls if the parents are not forewarned. The availability of accurate information on the internet (when to seek medical advice, and what to expect after surgery) could be useful to allay parents fears and minimise unnecessary attendances.

Materials and Methods

In October 2006, we searched the world wide web using the five most commonly used search engines (Google,²¹ Yahoo,²² MSN,²³ AOL²⁴ and AskJeeves²⁵) with the keyword ‘tonsillectomy’. We did not change any of the default settings, use any plug-ins or use any of the advanced search options. Thus, the searches were in all languages, and not limited to the UK. The LIDA Instrument, under the subsets of accessibility, usability and reliability, was used to assess the first 50 web links in each of the five searches.

Accessibility

Can all people access the website? The websites should meet legal accessibility requirements, including W3C and Bobby standards. The information available should be without restrictions and outdated HTML code. We checked this portion of the accessibility test using an online tool for the LIDA Instrument.¹² Additionally, they should work on all commonly used browsers and platforms (we checked usage with Macintoshes and Windows operating systems, and four browsers–Internet Explorer, Safari, Firefox and Opera). The information should be available full text, without registration, login or subscription, as this has a higher impact than information which has restricted access.²⁶ The maximum possible score was 63.

Usability

Can users find what they need to know? The aspects assessed included clarity of presentation, consistency of web-page design, functionality including intuitive browsing and search facilities, and engagability. The maximum possible score was 54.

Reliability

Does the site provide comprehensive, relevant and unbiased information? Features that are indicative of reliability are regular updates (currency), clear declaration of conflicts of interest, rigorous methodology for content production and output. The maximum possible score was 51.

The LIDA scores were considered to be ‘high’ if they were more than 90%, ‘moderate’ if they were between 50% and 90%, and ‘low’ if they were below 50%.¹²

The main text of each website was cut and pasted in to a word processor (Microsoft Word 2004 for Windows®) after cleaning the text of HTML tags and links to a reduced format. Using the grammar checker, a Flesch Reading Ease Score (FRES) was obtained. The Flesch–Kincaid grade level formula uses two variables, the average sentence length (ASL) and the average number of syllables per word (ASW).

$$\text{FRES}=206.835-1.015\left(\text{ASL}\right)-84.6\left(\text{ASW}\right)$$ Eq. 1

The formula requires only sentence and syllable data, but creates a very reproducible and predictable score. Scores range from 0 (zero) to 100. Standard writing averages approximately 60–70. The higher the score, the better the readability. For example, the average FRESs for comic strips, Reader's Digest, the Wall Street Journal, and the Harvard Law Review are 92, 65, 43 and 32, respectively.¹⁷

Information so gathered from the five search engines was pooled and the data analysed.

Results

There were 250 possible web links (50 from each of the five search engines). Considerable overlap amongst the websites was identified, especially with the AOL search that is enhanced by Google. Of the 250 web links, 137 were repetitions, previously evaluated or restricted access sites. Thus, 113 web links were evaluated.

All web links were in English, the majority (58) having been created by centres in the UK. The US contributed 48 websites, with the remainder from Canada (4) and Australia (3).

Results are summarised in Table 1 and Figure 1.

Table 1.

Distribution of number of websites according to LIDA subset scores

		High score (≥ 90%)	Moderate score (≥ 50%, < 90%)	Low score (< 50%)
Accessibility		1	112	0
Usability		1	69	43
	Clarity	1	79	33
	Consistency	6	91	22
	Functionality	2	80	31
	Engagability	1	24	88
Reliability		2	22	89
	Currency	1	24	88
	Conflict of interest	3	54	56
	Content production	2	20	91
	Content production (suppl)	1	25	87
	Output content	4	29	80
Total LIDA score		0	66	47

Figure 1.

Median box distribution

Accessibility

The average accessibility score was 42 out of a possible 63 (66.7%). This was fairly well distributed amongst the websites, with only one website having a high accessibility score of more than 57 (90%), the remainder being moderate scores. All websites worked on the four browsers we tested, and across the Windows and OS X platforms. Nineteen websites required free registration for access, while five required a paid registration.

Usability

The average usability score was 29 out of a possible 54 (53.7%). The distribution of this variable was over a wider range between 7–49. Again, only one website consistently achieved high scores on clarity, consistency, functionality and engagability, the majority being in the moderate score group (69 sites), with the remainder achieving low scores (43 sites).

Reliability

The average reliability score was 17 out of a possible 51 (33.3%). The range was 0–49, with 89 of the websites being in the ‘low’ score category. Specifically, the websites scored low averages in disclosure of sponsorship, and regular updating of content (6-monthly update for treatment options, longer for diagnosis and background information).

The overall average LIDA score for the websites was 88.2 (52.5%), bordering on a low score overall. Although this does not tell us the quality of a specific website, it does give us an indication of the quality of the websites likely to be encountered.

Flesch Reading Ease Score

The average FRES was 43.8 (range, 3–84.4). The websites that had good usability scores also had higher FRESs.

While there were some poor examples of health information websites about tonsillectomy, there were two websites that consistently performed well in all the tests. The first, <www.besttreatments.co.uk>, from the BMJ publishing group had accessibility, usability, reliability and FRE scores of 80%, 83.3%, 96%, and 83.4, respectively.²⁷ Similarly, a very good interactive and multimedia-enhanced website was from MedLine Plus from the National Library of Medicine (NLM). This website had scores of 69.8%, 90.7%, 94.2% and 61.7, respectively.²⁸

Discussion

There is a wealth of information and resources available on the internet. The quality of such information is subject to pressures similar to other print or broadcast media.¹⁶

Search engines are a popular way to find information on the world wide web, but the rankings on searches are not necessarily indicative of quality. Highly ranked websites are not necessarily the best, although they are more likely to be the ones first visited. The converse is also true. Worrying examples included the highly ranked web links of three commercially motivated sites. These websites were funded by surgical instrument manufacturers directed at patients, championing the use of coblation diathermy, laser-assisted tonsillectomy, and the harmonic scalpel as better alternatives to routine tonsillectomy. Experience from the National Prospective Tonsillectomy Audit²⁰ has shown that coblation has a 3-fold higher risk of postoperative bleeding as compared to routine cold steel dissection and haemostasis with ties. For anxious parents without access to reliable information, this has potentially serious implications.

On the other hand, not all information is bad. We found websites that provided easily understood and accurate information directed at patients. Two websites^27,28 stood out amongst the rest, but did not feature in all of the searches done. In addition, their rankings were variable, not necessarily being highly ranked.

This is the first study to look at quality of specific ENT information on the internet. We evaluated web links that were most likely to be seen by our average patient. An exhaustive search beyond the highly ranked links on the commonly used search engines is unlikely.¹⁴

Methods to determine quality of web sites, and especially those with medical content, are variable. There are many tools available, but few have been tested for reliability; those that have are mostly shown to be unreliable. We chose LIDA as it addresses the issues of accessibility and us ability as well as reliability, and has been validated.

The default settings of search engines are variable according to the country of origin, and the setting determined on installation if a plug-in is used.

The FRES is a validated tool in assessing overall read-ability, but should be used as an indicator only. Amonotonous succession of short sentences and simple words can give high FRES, but may not hold the attention of a reader. Readability scores are notoriously variable between different software, and manual calculation.

Other studies have criticised the available information on the public domain in other medical branches. Impicciatore et al.⁴ evaluated the advice available over the internet to parents on managing fever in children at home, and found only four of 41 identified public oriented sites adhered to published guidelines. Soot et al.²⁹ found a third of patient-oriented web sites related to peripheral vascular disease contained misleading information. Greene et al.³⁰ found less than 10% of internet sites relating to lumbar disc herniation to be of good factual quality, and 34.3% of the sites to be commercially motivated. Our figures show a similar trend in information concerning tonsillectomy.

We believe it is important for clinicians to know what information is available to their patients.³¹ It is easy for anonymous authors to conceal commercial or other conflicts of interest, but is not easy for lay-people to discern genuine insight from otherwise motivated information. Giving patients references on the internet that have been previously assessed by the clinician may be away to overcome potentially misleading information.