Consent information leaflets – readable or unreadable?

Abstract

Objective

The objective of this article is to assess the readability of leaflets about urological procedures provided by the British Association of Urological Surgeons (BAUS) to evaluate their suitability for providing information.

Methods

Information leaflets were assessed using three measures of readability: Flesch Reading Ease, Flesch-Kincaid and Simple Measure of Gobbledygook (SMOG) grade formulae. The scores were compared with national literacy statistics.

Results

Relatively good readability was demonstrated using the Flesch Reading Ease (53.4–60.1) and Flesch-Kincaid Grade Level (6.5–7.6) methods. However, the average SMOG index (14.0–15.0) for each category suggests that the majority of the leaflets are written above the reading level of an 18-year-old. Using national literacy statistics, at least 43% of the population will have significant difficultly understanding the majority of these leaflets.

Conclusions

The results suggest that comprehension of the leaflets provided by the BAUS is likely to be poor. These leaflets may be used as an adjunct to discussion but it is essential to ensure that all the information necessary to make an informed decision has been conveyed in a way that can be understood by the patient.

Introduction

It is a general ethical and legal principle that valid consent must be obtained before starting a procedure, reflecting the patient’s right to autonomy. For consent to be valid, it must be given voluntarily by an appropriately informed patient who has the capacity to consent to the particular intervention.¹ In order to be appropriately informed, the patient should be provided with the necessary information in a way that can be understood by them. If any of these elements are not met, the consent is deemed invalid.

Traditionally, the main mechanism of information transfer would be via discussion at the clinic and completion of a consent form prior to surgery. More recently, patients are increasingly being directed towards other information resources to access at their convenience in order to aid the decision-making process.

In recent years the British Association of Urological Surgeons (BAUS) produced information leaflets and consent forms for urological procedures. These are updated and renewed annually. Many centres have adopted them as part of the consent process. BAUS provides information leaflets for over 150 different procedures and interventions.² The reliance on these leaflets or other methods of information provision will vary depending on the surgeon, institution and the patients’ requirements. Nevertheless, given that these leaflets may account for a significant part of the information transfer process, it is important to assess how well they are likely to be understood by patients.

In this study, the readability of the BAUS procedure-specific information leaflets was analysed using three scoring methods to predict the ease with which they can be read and understood. The scores give an indication of the educational level required to read the leaflets; these can be compared with national literacy levels to assess the suitability of the leaflets for information transfer.

Methods

Procedure-specific information leaflets were downloaded from the BAUS website.² These each contain information about the specific procedure with a space at the end for the patients to sign, confirming that they have read the booklet and accept the information it provides. Patients also sign separate consent forms, which were not analysed. Information common to all leaflets was removed (e.g. headers, page numbers, references and common sections of text) prior to analysis of each leaflet.

Readability was assessed using the Flesch Reading Ease and Flesch-Kincaid Grade Level formulae and the SMOG index.^3,4 The Flesch Reading Ease and Flesch-Kincaid Grade Level are calculated using formulae based on word and sentence length. Automated software within Microsoft Word 2010 was used for these calculations. This has been proven to be reliable and valid.⁵

The SMOG index incorporates the number of polysyllabic words into the formula to assess readability. The SMOG scores were assessed using an online tool supported by the National Institute of Adult Continuing Education (NIACE) and the University of Nottingham.⁶ This tool differs from other SMOG scoring systems by adding five to every score. Five was subtracted from every result to give a United States (US) grade level in order to make it consistent with other SMOG scoring systems.

Scores were calculated for each leaflet and means were then calculated for each category (e.g. bladder, fertility and infertility, kidney and adrenal) and sub-sections within these (e.g. within the ‘bladder’ category: bladder instillations, catheter procedures and information, cystoscopy).

Interpretation of the scores

The Flesch Reading Ease provides a readability score from 0 to 100. The higher the score, the more readable the document. Tabloid newspapers, for example, have a score of around 58 whereas journal articles have scores of around 20 (Table 1). The Flesch-Kincaid Grade Level and SMOG formulae give scores as US education grade levels from 0 to 12, indicating the level of education or reading age required to understand the analysed text. For example, a score of 8.0 means that a student in the eighth grade (around 13 years old) can understand the document. The US grade levels can be compared with the respective United Kingdom (UK) school year for age (Table 2).

Table 1.

Indicative scores for different types of reading material.^7,8

Reading material type	Flesch Reading Ease	Flesch-Kincaid Grade Level	SMOG score
UK tabloid newspaper	58	9.9	<9
UK broadsheet newspaper	41.9	11.8	>12
Random journal articles	23.7	11.8
Nature Medicine	20.6	15.9	19.6
New England Journal of Medicine	18.1	17.3	21.0

SMOG: Simple Measure of Gobbledygook; UK: United Kingdom.

Table 2.

Comparison of US and UK school grades.⁹

Age	Level of study	US grade	UK year
3–4	Pre-school	N/A	Nursery school
5–10	Elementary/primary school	Kindergarten–5th	Years 1–6
11–13	Middle school	6th–8th	Years 7–9
14–18	High school	9th–12th	Years 10–13

US: United States; UK: United Kingdom.

In the UK, the average reading age is around US grade 8 (13–14 years, UK year 9) and the recommended level at which patient medical information should be provided is US grade 5 (10–11 years, UK year 6).⁵

To place this in context, the scores of different types of reading material can be analysed (Table 1).^7,8

Table 3 shows adult literacy levels in England, together with equivalent SMOG scores. A total of 85% of the adult population in England achieve the equivalent reading level of US school grade 6–7 and above (Flesch-Kincaid and SMOG readability scores of over 6). Fifty-seven per cent of the population have a reading level above US grade 9–10 (Flesch-Kincaid and SMOG scores of 9–10). However, 15% (around 5.1 million people) of the population in England have literacy levels at or below US school grade 5. At this level, individuals are described as being functionally illiterate. Five per cent of UK adults (around 1.1 million people) read at the level expected of 5- to 7-year-olds, equivalent to a US education grade of kindergarten to 2nd grade.

Table 3.

Comparison of reading levels in England and US school grades.¹⁰

Reading level in England	Adults in England reading at this level	Equivalent US school grade/SMOG score
Entry level 1 (the equivalent of National Curriculum expectations of 5- to 7-year-olds)	5%	Kindergarten–2
Entry level 3 (level expected of an 11-year-old) i.e. functionally illiterate	10%	4–5
Level 1 (GCSE D–G)	28%	6–7
Level 2 or above (GCSE A*–C)	57%	9–10

US: United States; SMOG: Simple Measure of Gobbledygook; GCSE: General Certificate of Secondary Education.

Results

In total, 155 leaflets were analysed. The average Flesch Reading Ease, Flesch-Kincaid Grade Level and SMOG index scores for all leaflets are 57.5, 6.8 and 14.3, respectively. Analysis of the text common to each leaflet was undertaken separately, giving Flesch Reading Ease, Flesch Kincaid Grade Level and SMOG index scores of 54.3, 7.6 and 15.6, respectively.

The average readability statistics for each category are shown in Table 4. The sub-category scores are shown in Table 5. Overall, the readability scores for each category are similar with good readability demonstrated using the Flesch Reading Ease (53.4–60.1) and Flesch-Kincaid Grade Level (6.5–7.6) scores. These scores suggest that, on average, the forms can be understood by individuals with a reading level of 11–13 years; National Literacy statistics show that 85% of the population in England should be able to understand these scores.

Table 4.

Average scores for each category of information leaflets.

Leaflet	Flesch Reading Ease	Flesch-Kincaid Grade Level	SMOG index
Bladder	60.0	6.5	14.1
Fertility and infertility procedures	56.9	6.8	14.1
Kidney and adrenal	53.7	7.3	15.0
Miscellaneous procedures	55.0	7.1	14.0
Penis procedures	58.8	6.8	14.1
Prostate procedures	56.5	7.0	14.0
Retroperitoneal procedures	56.7	6.9	14.6
Stone procedures	60.1	6.5	14.0
Testis and scrotum	57.6	6.6	14.3
Transplantation and dialysis procedures	53.4	7.3	14.9
Ureter	54.7	7.2	15.0
Urethral procedures	54.2	7.6	14.6
Grand average	57.5	6.8	14.3

SMOG: Simple Measure of Gobbledygook.

Table 5.

Average scores for each sub-category of leaflets.

Leaflet	Flesch Reading Ease	Flesch-Kincaid Grade Level	SMOG index
Bladder	60.8	6.5	13.5
Catheter procedures and instillation	60.9	6.3	14.1
Cystoscopy (inspection of the bladder)	59.2	6.8	14.4
Cystoscopy and other procedures	59.4	6.5	14.4
Procedures for urinary incontinence	57.2	7.0	14.1
Removal of the bladder	59.5	6.6	14.9
Urinary diversion	54.9	7.1	14.9
Other non-procedural information	72.2	4.9	12.5
Ejaculatory disorders	50.7	7.3	14.9
Family planning procedures	57.7	6.7	14.0
Adrenal procedures	50.0	7.7	15.1
On the kidney	52.0	7.6	15.5
Open procedures	55.3	7.0	15.0
Radiological procedures	55.8	7.1	13.7
Procedures not classifiable by anatomical area	55.0	7.1	14.0
Procedures for erectile dysfunction (impotence)	58.3	6.7	14.1
Procedures for penile straightening	64.5	5.8	14.4
Cancer	58.8	6.7	14.4
Procedures on the foreskin (prepuce)	62.7	6.0	14.2
Procedures on the urinary opening (meatus)	54.7	8.1	14.8
Other non-procedural information	56.8	7.0	13.3
Endoscopic procedures	52.0	7.6	14.5
Laparoscopic procedures	54.8	7.2	14.0
Open procedures	54.8	7.1	15.2
Ultrasound-guided procedures	52.1	7.6	14.5
Other non-procedural information	68.3	5.6	12.2
Procedures on the retroperitoneum	56.7	6.9	14.6
Procedures for kidney stones	56.9	6.8	14.7
Procedures for ureteric stones	60.7	6.3	14.6
Shockwave lithotripsy (ESWL)	54.7	7.5	14.0
Other non-procedural information	68.2	5.5	11.8
Procedures for benign conditions	57.5	6.6	14.5
Procedures for suspected tumour	50.9	7.8	14.1
Other non-procedural information	61.8	6.3	13.1
Access surgery for dialysis	56.5	7.0	14.7
Kidney donation	49.3	7.8	15.3
Transplantation	55.5	7.2	14.4
Other procedures	54.7	7.2	15.0
Procedures for urethral stricture	55.2	7.1	14.8
Procedures for urethral lesion	52.5	8.3	14.4
Grand average	57.5	6.8	14.3

SMOG: Simple Measure of Gobbledygook; ESWL: extracorporeal shock wave lithotripsy.

The average SMOG index results (14.0–15.0) for each category, although similar amongst categories, give a significantly different impression of readability. These scores indicate the average readability is above school level: the majority of leaflets are written above the reading level of an 18-year-old school-leaver. Indeed, only eight leaflets had a score of 12 or below and the lowest score was 10.9. Given the national literacy statistics (Table 3), the SMOG scores suggest that at least 43% of the population may struggle to understand all of the leaflets.

The National Literacy Strategy has improved literacy levels in the UK.¹¹ The national statistics may therefore underestimate and be less applicable to the literacy of the middle-aged-elderly population, a significant proportion of urology patients.

Discussion

For consent to be valid, patients must be informed of the benefits, available options and risks. There is a vast amount of information available to patients and so directing them to particular well-trusted sources is important. Frequently, urologists will refer patients to leaflets published by BAUS. Reliance on these leaflets as a source of information has both ethical and legal implications. If these leaflets form the majority of information provision, it is essential that they are easily understood. It is therefore important to assess the readability to see how useful they are to the general patient population.

Although each scoring system shows similar results between categories, the assessment using Flesch Reading Ease and Flesch-Kincaid Grade Level differs from the analysis using the SMOG tool.

The Flesch Reading Ease and Flesch-Kincaid Grade Level scores appear attractive because they are widely utilised and are routinely incorporated into word processing software.

However, the SMOG score is considered a more exacting measure of readability, accurately scoring for the grade level required for complete text comprehension.¹² In addition, the SMOG score is more consistent than the Flesch-Kincaid,¹³ demonstrating strong correlation with the required level in validation studies.¹⁴

None of these scores capture the difficulty of the concepts being conveyed; as health care material is likely to contain challenging/novel topics, these scores will likely underestimate the readability. Furthermore, the Flesch-Kincaid Grade Level score has been reported to significantly underestimate reading difficulty.¹²

The SMOG score is therefore recommended as the preferred measure of readability, particularly in the evaluation of health care literature.¹² However, it primarily assesses the number of polysyllabic words, which may lead to bias as these are frequently utilised in the health care vocabulary. Artificially reducing the number of specialist polysyllabic words could result in loss of precision and increased ambiguity. Nevertheless, a high polysyllable count (and high use of specialist terminology) indicates that patients may be less likely to understand the text. The words must be explained in more simple terms.

With the average SMOG scores for each category ranging from 14.0 to 15.0, the majority of leaflets are written to a level above that expected of 18-year-olds leaving school education. In the UK, 21% of the population are educated to A-level standard (18 years old) and 38% are graduates.¹⁵

Evaluating individual leaflet scores, not one has a SMOG score of 10 or below and only eight have a score of 12 or below. National literacy statistics (Table 3) suggest that at least 43% of the 16- to 65-year-old population in England will not easily understand these leaflets. As mentioned, these statistics will underestimate literacy in the over-65s (a significant proportion of urology patients) reducing the potential numbers who understand these leaflets.¹¹

Comparing the SMOG scores with those estimated for national newspapers, the readability of all the leaflets is lower than tabloids such as the Sun and the vast majority of leaflets have a lower readability than broadsheets such as the Telegraph (Table 2).

However, the scores capture only some of the factors that contribute to the readability of a document. The formulae do not account for the overall cohesion of a sentence and the scores do not convey the complexity of the topics. Furthermore, text layout, font type and size and patient motivation are not considered.

Despite the limitations of these readability formulae, the scores raise concerns that a substantial proportion of the UK population will find these leaflets difficult to understand.

The specific content of the leaflets was not assessed to determine if they contain all of the information necessary for informed consent. This is essential as even if a particular leaflet is understood, it may not contain all of the relevant information and so consent may not be fully informed.

Information leaflets may be used as an adjunct to discussion, providing useful, reliable information for patients to access away from the clinic. However, the leaflets provided by BAUS require a relatively high level of education to be understood, preventing their use as the only source of information for a substantial proportion of the population.

To improve readability, the leaflets could be modified in consultation with representative lay patient groups, writing succinctly and clearly according to published guidelines.¹⁶ Additional explanatory information for complex/unfamiliar words should be given, thereby improving readability without loss of precision.

Importantly, 15% of the population in England are functionally illiterate, limiting the use of these reading materials as forms of information transfer. The clinician must work in partnership with the patient, tailoring their approach to the patient’s requirements and wishes. In some situations, it may be more appropriate to use the leaflets as an aide memoire for the clinician rather than the main mechanism of information transfer. Information leaflets may be useful in selected cases but it is essential that discussion confirms the patient is informed and that consent is therefore valid.