Quality of patient health information on the Internet: reviewing a complex and evolving landscape

Abstract

Background

The popularity of the Internet has enabled unprecedented access to health information. As a largely unregulated source, there is potential for inconsistency in the quality of information that reaches the patient.

Aims

To review the literature relating to the quality indicators of health information for patients on the Internet.

Method

A search of English language literature was conducted using PubMed, Google Scholar and EMBASE databases.

Results

Many articles have been published which assess the quality of information relating to specific medical conditions. Indicators of quality have been defined in an attempt to predict higher quality health information on the Internet. Quality evaluation tools are scoring systems based on indicators of quality. Established tools such as the HONcode may help patients navigate to more reliable information. Google and Wikipedia are important emerging sources of patient health information.

Conclusion

The Internet is crucial for modern dissemination of health information, but it is clear that quality varies significantly between sources. Quality indicators for web-information have been developed but there is no agreed standard yet. We envisage that reliable rating tools, effective search engine ranking and progress in crowd-edited websites will enhance patient access to health information on the Internet.

What this study adds:

1. What is known about this subject?

   The enormous volume of health information on the Internet has revolutionised patient education. A large body of research exists analysing the quality of this information, but it is unclear how effective these quality evaluation tools are.

2. What new information is offered in this study?

   Quality evaluation tools are not reliable in predicting high quality information. Ranking tools such as Google are important predictors of what information reaches patients.

3. What are the implications for research, policy or practice?

   Improving the quality of health information that reaches patients on the Internet will require a multi-faceted approach, encompassing better patient education, and guidelines to support the development of online health resources.

Introduction

Since its inception, the internet has rapidly become the single largest source of information worldwide. Health information has seen an exponential increase in accessibility with the advent of this technology. In the past, patient health information had been limited to the discretion of a treating doctor. Today, patients have access to more information than ever. This changing trend has been a double-edged sword. A more involved and informed patient is unarguably a positive change, but sources of information on the internet are varied and largely unregulated. The result is inconsistency in the quality of information that reaches the patient.

In this article, we review the literature relating to the quality of health information on the internet. This is an introduction to what is an expanding field of research.

History of the research field

In the mid-90s, approximately 10 million people had access to the internet, while current estimates exceed two billion users worldwide.¹ A new concept to many, early articles in the medical literature focused on educating healthcare providers about the Internet.²

Later, studies began to emerge which looked at specific medical topics, and evaluated the quality of information available to patients. The first of these was a 1997 study by Impicciatore et al,³ which assessed the accuracy and completeness of website information relating to the management of fever in children. According to the authors, out of 41 websites, only four provided complete and accurate information on the topic. This landmark study provided a framework for articles that followed, produced by researchers in various disciplines, exploring the quality of information in their own specialty.

However, there was little consistency between these studies in the criteria used to assess quality of information. This issue became the focus for later research which considered ‘indicators of quality’⁴ - criteria which could serve as predictors of accurate information. The next section considers this literature in detail.

Indicators of quality

A patient who seeks information on a health topic may find it difficult to gauge the reliability of a webpage. Indicators of quality aim to guide patients towards information of a high calibre. ‘Core indicators’ directly assess the quality of information. They are accuracy, completeness and currency of information.⁵ These criteria are the gold standard for evaluating health information on the internet. However, their use is limited because of the need for expert input - evaluating every health website using these criteria would be impossible.

And so, ‘proxy indicators’ were born out of necessity. These are indirect measurements of quality, for example readability, design and disclosures, which aim to predict the quality of information.⁵ Proxy indicators are useful because they are objective and easy to assess. Their popularity was demonstrated in a study by Eysenbach,⁴ which systematically reviewed 85 articles that had evaluated quality of patient health information. Proxy indicators were more commonly used than core indicators in these studies.

However, proxy indicators may not provide a reliable guide to information quality. For example, readability is based on formulae such as the Flesh-Kincaid reading level,⁴ which calculates readability using average sentence length and number of syllables per word. There are clearly advantages to automatic assessment by a computer-based algorithm. However, these formulae are limited because they do not consider other important factors like use of jargon or appropriate tone, and may not accurately gauge practical readability.

Quality evaluation tools

Scoring systems have been developed based on the above indicators of quality to standardise evaluation. These scoring systems – or quality evaluation tools – are essentially a set of indicators of quality applied to a website in order to derive a quality score. There are many quality evaluation tools – 273 were identified in a 2004 review,⁶ compared to 47 in 1998.7 We will discuss three tools that are widely used today; the HON Code, the JAMA benchmarks and the DISCERN tool.

The oldest quality evaluation tool is the HONcode. The HONcode is an 8-point code of conduct created by the Health on the Net Foundation in 1995, which allows approved websites to display an award-like badge. As of 2010 there are over 7400 sites certified, and it was accepted as the official health information certification body of France in 2007.⁸ HON certification is subject to a process of annual review by the HON foundation, who also respond to any reported violations by Internet users.⁸

Codes such as these do not claim to be able to evaluate accuracy of medical information, but instead relate to ethics of the information presented.⁸ Nevertheless, multiple studies have found that HONcode certification was associated with an improved quality of information and increased reliability.^9-11 However, there have also been studies that found many websites displaying the badge to not comply fully with all eight criteria.¹²

Another widely used tool, the JAMA (Journal of the American Medical Association) benchmarks, arose from a 1997 article by Silberg et al which presented a set of criteria upon which health information could be evaluated.¹³ These included display of authorship, source, date of update, disclosure of ownership, sponsorship, advertising policies and conflict of interest. Despite its popularity as a rating tool, a 2010 study by Barker suggested that websites meeting all JAMA benchmarks actually contained more incorrect information than other sites. ¹⁴

Table 1. Presentation of the HONcode Principle (summarised).

1.	Authoritative: indicate the qualifications of the authors
2.	Complementarity: information should support, not replace, the doctor-patient relationship, the mission and the audience are explicated
3.	Privacy: Respect the privacy and confidentiality of personal data submitted to the site by the visitor
4.	Attribution: Cite the source(s) of published information, data and medical and health pages
5.	Justifiability: Site must back up claims relating to benefits and performance
6.	Transparency: Accessible presentation, accurate email contact
7.	Financial disclosure: Identify funding sources
8.	Advertising policy: Clearly distinguish advertising from editorial content

Reproduced with permission. From Evolution of Health Web certification through the HONcode experience (Boyer, Stud Health Technol Inform. 2011;169:53-7.)

DISCERN is a tool created by the Division of Public Health and Primary Care at Oxford University, London. It was launched in 1998, and consists of a 16-item questionnaire for patients to complete for each site they wish to evaluate. This type of system may help to educate patients in terms of quality criteria, as they apply each aspect of the questionnaire to a website. However, leaving this responsibility with the patient may be inconvenient and unreliable. For instance, a study by Rao et al found poor inter-rater reliability when using the DISCERN tool for dengue related health information.¹⁵

Tools such as the HONcode allow compliant websites to display an award-like badge. In fact, most quality evaluation tools exist as part of a ‘seal of approval’. However, many of these seals do not publicly disclose their criteria. A 2004 study showed that participants believed seals of approval to be based on core indicators, rather than the proxy indicators upon which they more commonly are.¹⁵

Although a number of key quality evaluation tools have been developed, it is unclear if these tools can accurately identify quality information. It may be impossible to develop a universal rating tool, given the ever-changing landscape of the Internet. These difficulties have effected a shift in focus to consider more the practical habits of the patient surfing the web. Nothing on the Internet is more habitual for us today than Google and Wikipedia.

Google ranking

The likelihood that a patient will view a particular web site may be influenced by its order of appearance on major search engines. It has been shown that most web users only visit the top 10 web sites listed in search results.¹⁶ With such influence it seems appropriate that we consider the way in which search results are ordered.

Google is the most popular search engine worldwide, and the most-visited site on the internet.¹⁷ One way in which Google ranks its search results is based on link popularity (or ‘PageRankTM’). For any given website, the number of hyperlinks pointing to it from other webpages will improve its rank in a Google search. More ‘important’ linking webpages cast a heavier vote for the rank of the website they are linking to. ¹⁸

Link popularity has been explored by a few articles^19-20 in terms of its association with the content and quality of websites, posing the question: is it a valid measure of website quality? Studies assessing website information on menopause¹⁹ and post-menopausal osteoporosis²⁰ found that link popularity did not correlate with measures of quality. The link popularity of breast cancer related websites was found to be associated with type of information rather than quality of content. ¹² For example, sites with higher link popularity were more likely to provide updates on breast cancer research and information on legislation.

But link popularity is not the full picture. Google combines PageRank with ‘sophisticated text-matching techniques’ to produce a search result which is both relevant and ‘important’.¹⁸ However, based on the limited evidence available, it seems these efforts may not be particularly good at identifying quality health information. The overall Google rank did not significantly predict higher quality information in any of the studies mentioned above. ^19,20

Wikipedia

Wikipedia is a multilingual, web-based, free-content encyclopaedia project based on an openly editable model. It has generated attention from international groups aiming to harness its potential for global public health promotion.²¹ Wikipedia is the sixth most accessed website on the internet¹⁷ and its popularity may be a result of its high ranking among top search engines.²²

As an unregulated source of health information used by many Internet users, it has been suggested that medical topics on the website could be edited by experts in order to improve the quality of information.²³ Other sources have found that accuracy and completeness did not differ much between Wikipedia articles and expert-generated websites.²⁴ Given the potential for inconsistency in the non-peer-reviewed nature of Wikipedia, it may be useful to ensure that links are provided to more definitive sources of health information.²⁵ There is no doubt that Wikipedia could serve as an effective medium for patient education if properly reviewed.

Specialty Wiki pages such as SurgWiki (www.surgwiki.com) and the Cancer Guidelines Wiki produced by Cancer Council Australia (http://www.wiki.cancer.org.au), represent an innovative approach to the Wiki concept, with information edited only by experts. At this stage, however, these websites have not gained the popularity enabling them to secure high Google ranks. While the information presented is of a high calibre, it may not reach the patient who will typically access only the first 10 hits from a Google search.

Despite its limitations, Wikipedia is potentially an important medium to disseminate patient health information. As the World Wide Web continues to expand, our ability as health care providers to regulate the quality of Internet health information will continue to diminish. The openly editable content of Wikipedia has many advantages and draws on import at an enormous scale. There is a high likelihood that its popularity will continue to increase at least for the near term. It has been suggested by some²³ that as health care providers trying to regulate the quality of health information that reaches patients, our efforts would be best spent editing and updating Wikipedia pages.

Discussion and conclusion

There have been many studies assessing the quality of health information on the Internet. Attempts have been made at developing rating scores for Internet health information; however, it is not clear how useful these rating tools are. HONcode is the most widely used quality evaluation tool.

The Google rank system is important in determining which information will reach patients, but is a poor predictor of good quality information. Wikipedia is a frequently accessed website and ranks highly on Google searches. Despite being a potentially important source of health information for patients, there is a risk of inaccurate or commercially biased information due to its crowd-edited nature.

Practice implications

In the internet era, our role as doctors in guiding patients towards high-quality health information has expanded into the digital setting. It is difficult to make recommendations on an optimal approach to this complex and evolving environment. However, from this review, the potential inaccuracy of information on the internet suggests some key areas in which doctors can contribute.

In the clinical setting, more time spent in discussion with patients on aspects of diagnosis and treatment may be a beneficial, individualised alternative to internet information. For computer literate patients, guidance towards high quality, accurate web resources is a reasonable approach which may help patients to navigate the myriad of information available to them while promoting autonomy. As discussed earlier, becoming actively involved in editing and updating resources such as Wikipedia may also be an effective way to disseminate our knowledge to patients.

On a broader scale, protocols for the development of web resources, to guide the process of writing and design may be beneficial for patient understanding, and complementary to grading systems such as the HONcode.

Overall, it is likely that a combination of these approaches is required. Guidelines do not currently exist on the approach of the medical profession to health information on the Internet. Such guidelines could potentially advise on strategies from the level of the individual doctor to professional bodies and policy makers. This may be necessary in order to stay ahead of the growing body of inaccurate material on the Internet and also to harness its power as a source of health information.