Readability of consent forms in veterinary clinical research

Abstract

Background

“Readability” of consent forms is vital to the informed consent process. The average human hospital consent form is written at a 10th grade reading level, whereas the average American adult reads at an 8th grade level. Limited information currently exists regarding the readability of veterinary general medical or clinical research consent forms.

Hypothesis/Objectives

The goal of this study was to assess the readability of veterinary clinical trial consent forms from a group of veterinary referral centers recently involved in a working group focused on veterinary clinical trial review and consent. We hypothesized that consent forms would not be optimized for client comprehension and would be written above the National Institutes of Health‐recommended 6th grade reading level.

Animals

None.

Methods

This was a prospective study assessing a convenience sample of veterinary clinical trial consent forms. Readability was assessed using 3 methods: the Flesch‐Kincaid (F‐K) Grade Level, Flesch Reading Ease Score (FRES), and the Readability Test Tool (RTT). Results were reported as mean (±SD) and compared across specialties.

Results

Fifty‐three consent forms were evaluated. Mean FRES was 37.5 ± 6.0 (target 60 or higher). Mean F‐K Grade Level was 13.0 ± 1.2 and mean RTT grade level was 12.75 ± 1.1 (target 6.0 or lower). There was substantial agreement between F‐K and RTT grade level scores (intraclass correlation coefficient 0.8).

Conclusions and Clinical Importance

No form evaluated met current health literacy recommendations for readability. A simple and readily available F‐K Microsoft‐based approach for evaluating grade level was in substantial agreement with other methods, suggesting that this approach might be sufficient for use by clinicians and administrators drafting forms for future studies.

Abbreviations

AMA

American Medical Association

COHA

CTSA One Health Alliance

F‐K

Flesch‐Kincaid

FRES

Flesch Reading Ease Score

NIH

National Institutes of Health

RTT

Readability Test Tool

SMOG

Simple Measure of Gobbledygook

1. INTRODUCTION

The concept of informed consent is a core ethical principle in both human and veterinary clinical research.1, 2, 3 In veterinary medicine, informed consent is given by the animal's owner, and requires the owner to receive and comprehend appropriate information about the study's requirements, risks, and potential benefits. One common way of documenting this process is by having the owner sign a consent form that delineates, in writing, the information deemed necessary by investigators and institutional approval bodies for informed consent. Although written consent forms are a cornerstone of the consenting process across species, research in the human medical setting indicates that many forms have poor “readability” suggesting that they are not optimized for patient comprehension.4 “Readability” is an objective measure of whether a written text is understandable for an individual with a particular level of reading skills.5 In the context of clinical research consent forms, this definition is extended to mean that written information is presented in such a way that it can be read and understood by the patient, or in the example of veterinary medicine, the pet owner. Consent forms that are not optimized for readability could jeopardize the informed consent process by not adequately conveying adverse event possibility and prevalence.6, 7

An assortment of hand calculated and computerized measures of readability, designed to provide objective measures of text difficulty, are available. The Flesch‐Kincaid (F‐K) Grade Level method is commonly used in health literacy research to assign a US educational grade level at which a particular document can be expected to be read and understood and relies on assessment of sentence length and word complexity (using syllables as a surrogate marker) to assign a score.8 A computerized F‐K Grade Level assessment can be easily obtained in Microsoft Word through the Spelling and Grammar tool, which, after an adjustment to settings, can be asked to routinely display both the F‐K Grade Level and another common readability statistic, the Flesch Reading Ease Score (FRES). The FRES applies a different mathematical equation to sentence length and syllables in order to make an assessment.9, 10, 11 A third common readability statistic used in health literacy research is the Simple Measure of Gobbledygook (SMOG) index, which can be applied by hand or in a computerized fashion to a piece of written material.12 The SMOG index evaluates word length in the first, middle, and last 10 sentences of a document to arrive at a grade level score. Microsoft‐derived scores have the benefit of being automated and ubiquitously available while taking in to account the entirety of a document; however, they have been found in some cases to underestimate grade level compared to hand‐calculated SMOG indices.13 More recently, web‐based readability calculators have become widely available. Some of these are freely accessible online and calculate readability in a variety of ways, many via a complex algorithm that incorporates F‐K Grade Level, FRES, SMOG index, and several other methods to derive a composite “grade level” score for a piece of written material.

A wealth of recent publications describing consent in human clinical trials has focused on readability and comprehension of medical consent forms.4, 14, 15, 16 A survey conducted by the National Assessment of Adult Literacy concluded that approximately half of Americans have only basic literacy skills and that 14% have literacy skills that are less than basic.17 As such, medical groups such as the National Institutes of Health (NIH), American Medical Association (AMA), and Centers for Disease Control (CDC) have established evidence‐based guidelines which recommend that health care‐related materials be written at no higher than a 6th grade reading level.18, 19

There is currently a void of information in the veterinary literature with respect to readability of medical consent forms. A single recent study evaluated the readability of other veterinary health literature, in the form of client education handouts, and found that 90% were written above the recommended readability range for maximizing comprehension.20 The objective of the current study was to conduct a multicenter assessment of the readability of consent forms used in veterinary clinical research employing 3 readily available readability statistics in order to identify opportunities to improve the written consent process. An additional aim was to compare grade‐level scores as assigned to the same forms by Microsoft and web‐based methods to determine an optimal approach for clinicians interested in assessing form readability. We hypothesized that consent forms for veterinary clinical research would be written above the recommended 6th grade reading level. Additionally, we hypothesized that there would be strong agreement between Microsoft‐derived grade level scores and grade level scores obtained using a web‐based approach.

2. MATERIALS AND METHODS

Consent forms were solicited via email from 5 large public universities with federally funded research activity. All are Clinical and Translational Sciences Award One Health Alliance (COHA) veterinary academic centers recently involved in a working group focused on veterinary clinical research review and consenting. Institutions were asked to contribute consent forms associated with their 2 most recently approved veterinary clinical studies for the specialties of: internal medicine, neurology, oncology, small animal surgery, equine, and food animal. They were also provided the option to submit consent forms from other specialties in place of those requested if none from the requested specialty was available at their institution. Along with an unsigned copy of the form, institutions were also asked to provide the date of study approval associated with each form.

2.1. Consent form cleaning and conversion

Consent forms were received and initially processed by a single investigator (SM), who converted each form from a PDF to a Microsoft Word document by using the “save as” function and selecting “Word document” in order to create a new converted file. After conversion, each Word document was de‐identified by deleting investigator and institution name and then the remainder of the form, in its entirety, was passed on for evaluation of readability by a different investigator (JS).

2.2. Readability assessments

Readability was assessed using 3 methods: the Microsoft‐derived F‐K Grade Level, Microsoft‐derived FRES,8 and the Readability Test Tool (RTT; a free online readability calculator which can be found at https://www.webfx.com/tools/read-able/).

2.3. FRES and F‐K Grade Levels

Scores were obtained from each document within Microsoft Word using a standardized method that can be optionally displayed within the program. This feature was activated by navigating within the document to “file,” then “options,” then selecting the proofing tab and ensuring that “show readability statistics” was selected. After that, both FRES and F‐K scores were automatically displayed along with more typical statistics such as word count when using the Spelling and Grammar check function (Figure 1). Flesch‐Kincaid Grade Level is calculated as:

$$\mathrm{F}–\mathrm{K}\text{Grade Level}=\left(0.39\times \mathrm{ASL}\right)+\left(11.8\times \mathrm{ASW}\right)-15.59$$

Figure 1.

Example data output for Flesch Reading Ease Score (FRES) and Flesch‐Kincaid (F‐K) Grade Level as evaluated using a Microsoft Word approach

where ASL refers to “average sentence length” and ASW refers to “average number of syllables per word.” Flesch Reading Ease Score is calculated as:

$$\text{FRES}=206.835-\left(1.015\times \mathrm{ASL}\right)-\left(84.6\times \mathrm{ASW}\right)$$

Possible FRES values range from 0 to 100 with higher scores denoting “more readable” material. A value of 60 or above (which equates to a 6th grade reading level) was considered adequate readability based on NIH and AMA recommendations. Possible F‐K values range from 0 to 20, equating to the US educational grade level at which a piece of written material is expected to be read and understood. An F‐K Grade Level of 6th grade or lower was considered adequate for the purposes of the study based on NIH and AMA recommendations.

2.4. Readability Test Tool

Each consent form was first opened as a Word document, and the “ctrl A” function was used to select the entirety of the document. All text was then copied and pasted directly into the “text by direct input” window provided on the web‐based calculator. A single investigator (JS) performed this task for each form evaluated. The RTT utilizes 5 different grade level indicators, the lists of which, along with their associated formulas, are summarized in Table 1. The grade level assigned using the RTT represents an averaged value of these 5 grade level indicators.

Table 1.

Readability tests and equations used by the web‐based Readability Test Tool (RTT) to assign a grade level reading score for a text

Readability test	Formula
F‐K Grade Level	(0.39 × ASL) + (11.8 × ASW) − 15.59
SMOG index	1.043 × √ [#complex words × (30/ #sentences)] + 3.129
Gunning Fog index	0.4 x {(#words/ #sentences) + [100 × (#complex words/ #words)]}
Coleman‐Liau index	(0.0588 × mean #letters per 100 words) − (0.296 × mean #sentences per 100 words) − 15.8
Automated readability index	4.71 (characters/words) + 0.5 (words/sentences) − 21.43

Complex words are defined as words with 3 or more syllables; a character is defined as any letter or number.

Abbreviations: F‐K, Flesch‐Kincaid; SMOG, Simple Measure of Gobbledygook.

Possible values range from 0 to 20, equating to grade level as described above. A grade level of 6th or lower was considered adequately readable.

2.5. Statistical evaluation

Data were assessed for normality using the Shapiro‐Wilk test and found to be normally distributed. Descriptive statistics were calculated as mean ± SD for aggregate and specialty‐specific data from each readability test and compared across specialties using a 1‐way ANOVA. Agreement between Microsoft‐derived F‐K Grade Level and web‐based grade level scores was compared visually by constructing a Bland‐Altman plot and by calculating an intraclass correlation coefficient (ICC; 2‐way model for consistency of single measures) where ≤0.2 was considered slight agreement, 0.21‐0.40 was considered fair agreement, 0.41‐0.60 was considered moderate agreement, 0.61‐0.80 was considered substantial agreement, and 0.81‐1.00 was considered almost perfect agreement.21, 22 Statistical analysis was performed using GraphPad Prism software (Version 6.0, La Jolla, California).

3. RESULTS

A total of 53 consent forms from 4 veterinary academic centers were received and evaluated, constituting an 88% response rate with respect to total number of forms originally solicited and an 80% response rate with respect to number of institutions originally contacted. The breakdown of included medical specialties was as follows: 8 forms each from internal medicine, oncology, and surgery; 7 each from cardiology and neurology; 6 from equine; 5 from food animal, and 2 each from dermatology and ophthalmology (grouped together and analyzed as “Other”). All consent forms were reviewed and approved by their respective institutional approval bodies between October 2013 and May 2018.

Mean Microsoft‐derived FRES for all forms was 37.5 ± 6. Specialty‐specific FRES are summarized in Figure 2A. All FRES fell below the recommended cutoff of ≥60. Mean Microsoft‐derived F‐K Grade Level for all forms was 13.0 ± 1.2, and mean web‐based RTT grade level for all forms was 12.75 ± 1.1. Specialty‐specific F‐K and RTT grade level scores are summarized in Figure 2B,C. All forms were found to be written above the recommended 6th grade level using either method of grade level assessment. Flesch Reading Ease Score (P = .12) and Microsoft‐derived F‐K Grade Level (P = .08) did not differ significantly between specialty; however, the web‐based RTT assessment of grade level for forms in the “Other” category was significantly lower than other groups (P = .01).

Figure 2.

Box and whisker plots of readability scores for combined (n = 53) and specialty‐specific clinical research consent forms. Scores were above recommended ranges for all forms evaluated based on the Flesch Reading Ease Score (FRES; A), Flesch‐Kincaid Grade Level (F‐K; B), and web‐based Readability Test Tool (RTT; C). Target range for acceptable readability is indicated by the green box. Readability did not differ significantly between groups with the exception of the web‐based grade level (*P = .01)

There was substantial agreement (ICC 0.8) between grade level scores assigned using Microsoft‐based F‐K and web‐based RTT grade level assignments, although web‐based grade level assessments tended to be subjectively lower than Microsoft‐derived scores for the same document (Figure 3).

Figure 3.

Comparison of Microsoft‐derived Flesch‐Kincaid (F‐K) and web‐based Readability Test Tool (RTT) grade level as assigned to the same veterinary clinical research consent document. There was substantial agreement between the 2 methods (ICC = 0.8; A); however, visual inspection of the Bland‐Altman plot (B) of the 2 scoring methods suggests that the web‐based approach tends to assign a slightly lower score than the Microsoft‐based F‐K method to the same document, particularly for forms written at an inherently higher grade level (plot shows the difference in score between Microsoft‐based F‐K and web‐based methods graphed against the average score of the 2 methods together)

4. DISCUSSION

All 53 consent forms evaluated in the present study fell outside the recommended parameters for readability based on 3 common assessment methods: the FRES, Microsoft‐derived F‐K Grade Level, and the RTT web‐based grade level assessment. Moreover, mean grade level scores assigned by either Microsoft F‐K or web‐based methods were approximately 13, suggesting that most forms were written at a level requiring a college education to read and understand and are well above the 6th grade reading level that is considered best practice to optimize comprehension by the general public.

Consent forms for most specialties were similar with respect to their readability, although forms in the “Other” category (dermatology and ophthalmology) were written at a significantly lower grade level than the other specialties based on the web‐based approach to grade level assignment (mean score of 11 versus 12.25‐13.6 for other groups). The reason for this difference was not evaluated in the present study; however, these forms were still written well above the recommended grade level.

Grade level reading scores as determined by Microsoft‐based and web‐based RTT methods showed substantial agreement, suggesting that a simple Microsoft Word‐based approach to assessing grade level is likely sufficient for clinicians drafting consent forms. Although agreement was substantial between the 2 methods, the web‐based approach appeared to subjectively assign slightly lower grade level scores for some forms underscoring subtle differences in how grade level is assessed by this method.

The web‐based approach employed in the current study uses a combination of readability indices to arrive at a composite grade level score. These include the F‐K Grade Level, FRES, Gunning Fog Score, SMOG index, Coleman Liau Index, and Automated Readability Index. Differences in the method by which each index calculates grade level/readability can lead to inherent differences in assigned score. For example, when the Introduction section of this manuscript is evaluated using the web‐based RTT, it receives a composite grade level of 17, but its SMOG index is calculated as 15.1, and its F‐K Grade Level is calculated as 17.8. Interestingly, previous comparisons of grade level scoring indices have suggested that the SMOG index might overestimate grade level relative to computerized F‐K scores13; however, in the present study, incorporating SMOG along with several other indices into the web‐based grade level assignment actually produced a grade level score which trended lower than the F‐K Grade Level score for the same documents. Further work should focus on which indices are ideal for assessing readability specific to veterinary documents, but the ease with which an author can obtain an F‐K Grade Level score within Microsoft Word makes it an attractive tool for at least an initial evaluation of draft documents.

There are several limitations to the current study, the first being that consent forms included in this analysis were not randomly sampled. Because they were solicited from specific institutions, they might not represent the entirety of the veterinary clinical research landscape. However, institutions involved in the present study (all COHA member institutions) have an expressed interest in conducting high‐quality veterinary clinical research and have participated in in‐depth discussions of best practices in the informed consent process. Therefore, 1 might expect that data associated with these institutions, if anything, represent an underestimation of the current challenges with consent form readability across veterinary medicine.

An additional consideration is that written materials, such as consent forms, encompass only 1 aspect of how information related to the consent process is conveyed and understood. Other factors shown to influence a patient's understanding of health‐related information in the human health care setting include verbal communication (including language barriers or lack thereof), cultural awareness by health care providers, educational background of the patient, and situational factors (stress, general health status, environment in which the consenting process occurs).23, 24 Although in‐depth studies of these factors as they relate to informed consent do not exist in veterinary medicine, it is reasonable to suspect that they also play a part in an owner's comprehension of their pet's health care plan. Our study did not assess owner, veterinary health care provider, or situational factors as they relate to comprehension of clinical research consent forms, and clinicians should be cognizant of the fact that generating readable consent forms is only 1 aspect of optimizing the informed consent process.

Our results highlight substantial opportunity to improve readability of client consent forms in veterinary clinical research. Some previously published suggestions for clinicians and administrators crafting consent forms include using “plain language” as much as possible, keeping sentences to a maximum of 8‐10 words, limiting the use of words with 3 or more syllables, eliminating jargon and defining technical terms when use is required, using headings and subheadings to divide text into smaller sections, using questions as subheadings, leaving ample white space to avoid cramped text, and using active voice.20 Additionally, provision of informed consent templates by institutions can help investigators navigate the process of drafting readable consent forms,4 and provision of forms for study participants to read and digest in advance of their hospital visit could assist with understanding. Lastly, assessing readability of draft forms using mechanisms already available in Microsoft Word, such as the F‐K Grade Level and FRES, should help authors gauge appropriateness of their content.

5. CONCLUSION

Veterinary clinical study consent forms evaluated in the present study were not optimized for client comprehension based on readability scores assigned via 3 common methods. Furthermore, our results suggest that the average consent form would require a college level education to read and comprehend. Future work should focus on optimizing readability of veterinary clinical research consent forms, and clinicians and administrators should be aware of simple and validated methods, such as F‐K Grade Level, which are readily available to them in their word processing program.

CONFLICT OF INTEREST DECLARATION

Authors declare no conflict of interest.

OFF‐LABEL ANTIMICROBIAL DECLARATION

Authors declare no off‐label use of antimicrobials.

INSTITUTIONAL ANIMAL CARE AND USE COMMITTEE (IACUC) OR OTHER APPROVAL DECLARATION

Authors declare no IACUC or other approval was needed.

HUMAN ETHICS APPROVAL DECLARATION

Authors declare human ethics approval was not needed for this study.

Sobolewski J, Bryan JN, Duval D, et al. Readability of consent forms in veterinary clinical research. J Vet Intern Med. 2019;33:350–355. 10.1111/jvim.15462