Appraising the quality of consumer health information leaflets

Jane Shaddock

Accreditation Manager, Centre for Health Information Quality, The Help for Health Trust, Highcroft, Romsey Road, Winchester SO22 5DH, UK. E‐mail: jshaddock@hfht.org

Background

The aim of this article is to illustrate information quality issues with particular reference to a selection of leaflets produced by the Consumers' Association, `Treatment Notes'. A trained appraiser assessed the leaflets against specific criteria of quality, identifying existing strengths and offering suggestions for ways in which the material might be improved. Tools used included the DISCERN instrument, ¹ the Flesch test and the Centre for Health Information Quality's health information appraisal guidelines. ² The Centre for Health Information Quality (C‐H‐i‐Q) is part of the Help for Health Trust, an independent organization providing appraisal services to organizations who deal with health information for the public. C‐H‐i‐Q believes health information should be clearly communicated and evidence‐based, with members of the public involved at key stages of production.

Methodology

The Consumers' Association Treatment Notes are in the form of A5‐size leaflets. Fifteen titles, published between January 2000 and July 2001, were assessed (Fig. 1).

Figure 1.

 Consumers' Association leaflets assessed.

Overview

Treatment Notes provide concise, thorough overviews of a variety of topics. The language used is simple and chatty and offers advice in a straightforward and easy style. Treatment Notes are easy to navigate, and follow a natural progression from definition of conditions and explanation of the disorder, through causes and symptoms to treatments, self‐help and after care. Distinctive diagrams and drawings are used to good effect to illustrate the text and clarify complex messages. Every Treatment Note features useful side columns giving additional information, tips and summaries. Each includes references to additional sources of information such as useful websites, appropriate self‐help groups and Health Education Authority leaflets. The dates of publication and revision are clearly shown on each publication.

The C‐H‐i‐Q guidelines

The guidelines can be found on the C‐H‐i‐Q website, at www.chiq.org. Using this tool, 14 themes of quality are considered across the Treatment Notes leaflets.

Accessibility

Aim: the information is in an accessible format for the target audience

All the Treatment Notes are clearly laid out with an index and clear headings. Additional tips, summaries and safety information are presented in side columns tinted a different shade to the rest of the leaflet. There is no information about alternative formats for the information or whether it is available in languages other than English.

Appropriateness

Aim: the information is relevant to the target audience

Some of the Treatment Notes make the target audience for the resource clear in their title, e.g. `When your child has epilepsy'. The front page of every Treatment Note explains in detail what it is about and the topic areas covered.

Availability

Aim: the information is available to the widest possible audience

It is important to ensure high‐quality information is available to the widest possible audience. This involves considering the format in which the material is produced (leaflets, web pages, audio tapes, etc.) and also the means used to distribute the material. To address the issue of availability, the C‐H‐i‐Q appraiser would need to know in which additional formats the information is being produced. This is not specified in the Treatment Notes.

Communication

Aim: the information communicates the subject matter clearly

The front of each leaflet gives a clear and broad overview of the condition involved and a list of the topics covered by the leaflet. The Treatment Notes make good use of the active voice and personal pronouns which make the text easier to read and understand. All leaflets use a variety of communication devices to get their messages across. In particular, bullet points, columns, images and photographs are used to good effect. Boxes with reinforcement messages, practical tips and summaries are also useful. Where appropriate, simple terms and expressions are used throughout the text with the technical equivalent alongside in parentheses. For instance, `…breathing tubes (bronchi) leading to the lungs'. More complex material, such as descriptions of conditions and treatments, is explained simply and clearly in separate boxes.

Comprehensiveness

Aim: the information covers all aspects of the subject matter

The consumer needs to have access to all the relevant information about a condition and available treatments in order to make an informed choice. This includes giving the consumer access to information about the risks and benefits of each treatment and the likely outcome if no treatment was accepted. The Treatment Notes are particularly thorough on this point, discussing the various issues and treatment options clearly and plainly, including potential side‐effects and advice on self‐help.

Consistency

Aim: the information is presented in a consistent style throughout

The distinctive style of the Consumers' Association leaflets is recognizable throughout the range with consistent leaflet size, fonts, style and layout. Each leaflet uses a different colour for shaded side columns, headings, subheadings and front page text. (It should be noted that this attractive concept should be used with care to avoid compromise of legibility.)

Continuity

Aim: the information is presented in context with other resources

Almost all the leaflets examined had a useful Further Information section with contact details for additional sources of information and self‐help groups. There was also mention of other Consumers' Association leaflets on related topics, where appropriate.

Currency

Aim: the information is up‐to‐date

Ten of the leaflets were published in 2000, five in 2001. The date of publication of each leaflet is clearly shown and, where appropriate, the date when the leaflet was revised is also shown.

Imagery

Aim: images are used to good effect

Many of the leaflets use diagrams, charts, photographs and illustrations where appropriate. These are used effectively to communicate complex and difficult messages or to break up the text, making the material more accessible.

Legibility

Aim: written information is clearly presented

The Royal National Institute for the Blind (RNIB) suggests using fonts of at least 12‐point and paying particular attention to colour combinations and shading that may compromise legibility. The RNIB website (www.rnib.org.uk) offers further guidelines on legibility. A number of different sans‐serif fonts and font sizes have been used in the Treatment Notes to achieve clearly defined areas of information. However, some of the font sizes used are small and may present difficulties for someone with sight problems. This is particularly noticeable in some of the side panels that often contain important safety information and summaries. Similarly, the leaflet `Which Pill?' uses a lilac shade for side columns, heading and some of the text. This might also be difficult for someone with sight problems.

Originality

Aim: information has not already been produced for the same audience and in the same format

Many of the topics of Treatment Notes are frequently discussed at length in alternative resources (e.g. those areas covered in `When and how to lose weight'). However, the Treatment Notes offer an independent and reliable overview of specific topics, providing a useful introduction to a condition and pointing the way to more in‐depth advice.

Patient involvement

Aim: the information is specifically designed to meet the needs of the patient

Consumer health information should address the needs of the patients so it is particularly important to involve consumers in all key stages of production. The authors do not give any information on the extent of public involvement in the development of these leaflets.

Readability

Aim: words and sentences are kept short where possible

The language used should be written in plain English, free from jargon and should not patronize the reader. Several leaflets were tested using the Flesch reading score. This rates the readability of the text on a 100‐point scale; the higher the score, the easier it is to understand the document. For most standard documents, a score of approximately 60–70 is recommended.

Three leaflets were randomly selected and three 100‐word samples were taken from each, one from the beginning, one from the middle and one from the end. The readability scores generated by the Microsoft Word software were then averaged to obtain an overall score for each leaflet tested. The following Consumers' Association leaflets scored as follows:

• Bandage treatment for leg ulcers scored an average 73.7%;

• Helping your heart scored an average 71.8%;

• When your child has epilepsy scored an average 61.3%.

Reliability

Aim: the information is based on the best available evidence

The Treatment Notes leaflets clearly note that the material is based on matching information from the Drug and Therapeutic Bulletin (DTB), a monthly publication for doctors and pharmacists, which is also produced by the Consumers' Association. Full details are given on how to obtain a copy of the original article and copies are available from the Consumers' Association for a fee. Although the leaflets are offered as an independent viewpoint, the material is drawn from the Consumers' Association's own publication (DTB). Clinical trials are discussed in the text but no references are included so readers have no direct access to the evidence sources. In order to find out more about the evidence, readers would have to get hold of the relevant copy of DTB or purchase the article. Additionally, no information is given about the actual authors of the Treatment Notes. Are they doctors, health journalists or informed laypersons? Knowing the background and credentials of the authors helps the readers to have more confidence in the material.

Conclusions and recommendations

One of the great difficulties with preparing health information for the public is that the public is not a homogeneous group. Reading aptitude, education levels and physical capabilities vary considerably and cultural, language and age differences mean that it is virtually impossible to produce a resource that suits everyone. Criteria, appraisal tools and guidelines are designed to help us to reject the poorest quality material and signpost the valuable resources. With this in mind, the Consumers' Association Treatment Notes may benefit from the following:

• including reference sources for the evidence;

• giving the authors' names and credentials;

• avoiding small font sizes and pale shading;

• adding contact details and background information for the Consumers' Association;

• involving the public in production of the leaflets.

After assessing this selection of health information, the appraiser would recommend the Consumers' Association Treatment Notes as some of the more valuable health information resources available. The leaflets were developed to be naturally flowing with easy language and consistent style, colour and images to ensure the information is accessible to a wide audience.