Readability of Participant Informed Consent Forms and Informational Documents

Abstract

Objective

To assess the readability of the informed consent forms from the phase 3 COVID-19 vaccine trials conducted in the United States.

Patients and Methods

English consent forms were used for patients in phase 3 COVID-19 vaccine clinical trials. Consent forms were obtained in October 2020. Using Microsoft Word tools, we analyzed the readability (ie, the ease of reading) of written consent forms and informational documents from phase 3 COVID-19 vaccine clinical trials in the United States from the following manufacturers: AstraZeneca, Moderna, Pfizer, Johnson & Johnson, and Novavax.

Results

Owing to low readability and several format factors, this study determined that none of the consent forms or informational documents from the recent phase 3 COVID-19 vaccine clinical trials conducted in the United States met readability standards at the recommended 7th grade readability level for the average vaccine research volunteer in any readability category. The average English-speaking vaccine trial volunteer would have great difficulty comprehending the information provided in the consent forms and informational documents. To ensure that study subjects receive and fully comprehend information regarding a clinical study and can provide reliable consent, greater attention should be given to the development and use of simplified consent forms, multimedia formatting, personal discussion, and comprehension assessments.

The 2019 coronavirus disease (COVID-19) is a novel respiratory illness caused by severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) and is responsible for the current global pandemic.¹ In response, pharmaceutical companies implemented phase 3 clinical trials in the United States and other countries to assess the efficacy and safety of COVID-19 vaccine candidates. Before enrollment in these vaccine trials, informational documents and consent forms are provided to potential study subjects. These documents provide subjects with basic study information, including necessary precautions, privacy measures, and potential vaccine benefits, risks, and side effects. By providing their signatures on the consent forms, study subjects acknowledge that they understand the potential consequences of participating in the study and are willing to accept the risks associated with the study. In this paper, we analyze the readability of these informational documents and consent forms.

During the informed consent process, United States law requires volunteer study subjects to be informed and educated about the risks, benefits, and alternatives of the experimental medical intervention and procedures.² For informed consent to be valid, the patient or study volunteer must adequately understand the risks of participating in the clinical trial and voluntarily provide consent.³ In fact, any intervention without clear patient consent could be considered assault.⁴ Several factors may impede informed consent, including the subject’s education level, inadequate or complicated consent information, or ineffective communication between subjects and research professionals. Because study information packets and consent forms are the primary method by which study information and procedures are communicated regarding the clinical trial, the readability and comprehension of these materials are critical. If a study volunteer is unable to read and comprehend the information provided in the consent form, the provision of consent is misguided and invalid. Thus, informed consent documents must be written in a manner that accommodates trial participants in regard to their reading ability and comprehension.

Assessing consent form readability is a critical step toward ensuring that the text is at an appropriate level for readers who have varying proficiency.⁵ Data collected by the National Work Group on Literacy and Health indicate that approximately a quarter of US citizens have rudimentary reading skills and are likely unable to read and understand even medication instructions, a bus schedule, or directions on cleaning products.⁶ The average UK citizen reads at a lower level than the average US citizen;⁷ however, this does not mean that these people should be excluded from clinical trial research. For clinical research conducted in the United States, the Office of Human Research Protection (OHRP), Department of Health and Human Services (DHHS), and the Food and Drug Administration (FDA) mandate the use of consent forms as described in the Code of Federal Regulations (CFR).⁸ According to a study conducted by the National Adult Literacy Agency, consent materials should have readability levels at the 7th grade, which is the average reading grade level for adults in the United States.⁹

Other consent form factors, such as font size, document layout, charts, pictures, vocabulary, and word length, can also greatly affect readability.¹⁰ Many experts recommend using plain vocabulary, culture- and gender-neutral wording, short sentences, active voice, bulleted lists, and present tense in documents intended for the public.¹¹ The purpose of this study was to analyze and compare the readability of the consent forms and informational documents from multiple phase 3 COVID-19 clinical vaccine trials being conducted in the United States to determine if they meet the guidelines for adequate public readability.

Methods

We acquired electronic consent forms and informational documents from AstraZeneca, Johnson & Johnson, Moderna, Novavax, and Pfizer for the phase 3 COVID-19 vaccine trials recently conducted in the United States. All 5 companies provided an informed consent document written for adult study subjects. Two companies also provided informational documents given to study subjects, and 1 company provided a consent and assent form for their vaccine study involving adolescents.

Using the readability tools available in Microsoft Word, we evaluated formatting metrics and readability statistics in each consent form and informational document. Two authors independently analyzed formatting and readability to reduce bias. Assent form F was used in a vaccine trial on adolescents. Because of this, it has significantly higher (ie, easier) readability scores in all of the categories analyzed, and therefore we did not include the readability scores for the adolescent assent form in our report outlining the mean metrics of the adult consent forms.

We used 3 readability formulas to determine the difficulty of a text. The Flesch Reading Ease score is a scale that predicts reading ease on a scale from 1 to 100, with 70 being an appropriate level for the average reader in the United States.¹¹ Higher scores indicate easier readability. This formula, which is the most tested and reliable readability formula, is widely used (Table 1 ).

Table 1.

Flesch Reading Ease Score Comparison¹¹

Style	Flesch Reading Ease score	Average sentence length in words	Average number of syllables per 100 words	Type of magazine	Estimated school grade completed	Estimated percent of US adults
Very easy	90-100	8 or less	123 or less	Comics	4th grade	93
Easy	80-90	11	131	Pulp fiction	5th grade	91
Fairly easy	70-80	14	139	Slick fiction	6th grade	88
Standard	60-70	17	147	Digests	7th or 8th grade	83
Fairly difficult	50-60	21	155	Quality	Some high school	54
Difficult	30-50	25	167	Academic	High school to college	33
Very difficult	0-30	29 or more	192 or more	Scientific	College	4.5

The Flesch-Kincaid Grade Level score is a readability score developed by the US Navy. It predicts the reading ease of a passage on a scale from 1 to 12, which indicates the education grade level necessary to read the passage.¹¹ A 7th grade level is recommended for the average reader in the United States.¹² Although the Flesch Reading Ease score and Flesch-Kincaid Grade Level score are both based on word length and sentence length, they have different weighting factors. We recognize a difference in grade level assessment between the mean Flesch-Kincaid Grade Level and mean Gunning Fog Index. The Flesch-Kincaid Grade Level truncates readability at the 12th grade level and may, in fact, underestimate the actual reading level required for complex text.¹² Table 1 compares the Flesch Reading Ease score with the Flesch-Kincaid Grade Level score.

Finally, The Gunning Fog index, which is similar to the Flesch-Kincaid Grade Level, is a readability test that scores text from 1 to 20. Scores are based on the number of complex words per sentence in a ~100 word passage. A complete score is obtained by averaging the Gunning Fog index score for three random 100 word passages throughout the consent forms. Texts requiring near-universal understanding generally require an index of 7 (Table 2 ).¹³

Table 2.

Gunning Fog Index Grade Comparison

Gunning Fog index	Reading level by grade
17	College graduate
16	College senior
15	College junior
14	College sophomore
13	College freshman
12	High school senior
11	High school junior
10	High school sophomore
9	High school freshman
8	8th grade
7	7th grade
6	6th grade

Results

We obtained and reviewed 2 informational documents and 7 consent forms, including the adolescent assent and adult consent forms for an adolescent study, used in the 5 phase 3 COVID-19 vaccine clinical trials conducted in the United States. To provide confidentiality to the pharmaceutical companies, each consent form was randomly assigned a letter (A to G). Informational documents were randomly assigned either Y or Z. No significant difference in any scoring metric was detected by the 2 independent persons who scored readability and formatting metrics.

The lengths of the 7 consent forms and 2 informational documents ranged from 7 to 36 pages, with a mean of 24.6 pages. The number of words (including headers) ranged from 1839 to 10,337, with a mean of 8619.8 words. Words per page ranged from 240.4 to 462.2 (mean 350.2), and words per paragraph ranged from 16.1 to 100.4 (mean 38.8). Font size ranged from 10 to 12 points. Consent form C used font size 10; consent form A used font size 11; and consent forms B, D, E, F, and G used font size 12 (mean 11.4). Consent forms A, B, C, and Y used Times New Roman font, whereas consent forms D, E, F, G, and Z used Arial or Calibri fonts. The mean word length ranged from 4.3 to 5 characters (mean 4.8). Mean sentence length ranged from 16.9 to 20.4 words (mean 19.1) (Table 3 ).

Table 3.

Phase 3 COVID-19 Vaccine Trial Document Statistics

	Total pages	Total paragraphs	Total words	Words/page	Words/paragraph	Font style	Font size	Word length (characters)	Sentence Length (Words)
Adult Consent Forms
A	24	608	9789	407.9	16.1	Times new roman (TNR)	11	4.7	16.9
B	27	348	9575	354.6	27.5	TNR	12	4.8	19.6
C	22	502	10,168	462.2	20.3	TNR	10	4.8	19
D	36	421	9736	270.4	23.1	Arial	12	4.8	20.4
E	30	103	10,337	344.6	100.4	Arial	12	4.7	17.5
G	31	139	9957	321.2	71.6	Arial	12	4.6	18.5
Adult Informational documents
Y	7	100	1839	262.7	18.4	TNR	10	5	19.2
Z	29	230	7557	260.6	32.9	Calibri	12	4.7	21.7
Mean	24.63	306.38	8619.75	350.18	38.78	TNR	11.38	4.76	19.1
Adolescent assent form
F	8	107	2345	293.1	21.9	Arial	12	4.3	17.7
Recommended levels	<15	None	None	None	None	Arial/Calibri	14	None	15

The percent of passive sentences ranged from 0% to 42.1% (mean 33%). The Flesch Reading Ease Score ranged from 43.3 to 68.3 (mean 49.6). Flesch-Kincaid Grade Level ranged from 8 to 11.8 (mean 11). The Gunning Fog Score ranged from 9.9 to 15.6 (mean 13.3) (Table 4 ).

Table 4.

Phase 3 COVID-19 Vaccine Trial Readability Scores

	% Passive	Flesch Reading Ease	Flesch-Kincaid Grade Level	Gunning Fog Index
Adult consent forms
A	29.60%	54.6	9.7	13.23
B	42.10%	49.2	11.2	14.42
C	38.50%	48.2	11.1	12.49
D	37.10%	51.8	11	15.62
E	32.80%	56.1	9.7	10.62
G	0%	44	11.7	14.25
Adult informational documents
Y	46%	43.3	11.8	11.96
Z	41.20%	49.2	11.7	14.15
Mean	33%	49.55	10.99	13.34
Adolescent assent form
F	18.30%	68.3	8	9.93
Recommended levels	0%	70	7	7

Discussion

To our knowledge, this study is the first systematic analysis of informed consent readability for the phase 3 COVID-19 vaccine trials conducted in the United States. We found that none of the consent forms or informational documents had readability levels that met recommended standards for average English-speaking vaccine volunteers in any category. In addition, very few met the recommended criteria in page number, sentence length, font style, and percentage of passive-voice sentences. Although assent form F for the adolescent study did not reach desired readability metrics, it scored notably better in readability metrics, demonstrating that improved readability is possible.

Study subjects are less likely to fully comprehend informational documents that are excessively detailed and lengthy.¹⁴ Thus, the number of pages in informational materials should be fewer than 15.¹⁵ The informed consent documents we reviewed ranged from 8 to 36 pages (mean 24.6). This average exceeds the recommended number of 15 by nearly 10 pages.

High word density can also lead to incomprehensibility because it raises the difficulty level of a text. Too many words in a sentence or on a page can hinder or prevent comprehension of key content. For this reason, it is recommended that sentences be between 12 and 17 words.¹⁶ The mean sentence length ranged from 16.9 to 20.4 words (mean 19.1), several words more than the recommended 12 to 17 word limit.

Other factors, such as font style and size, can also play a role in assessing the readability of a text. Sans serif fonts, such as Arial or Calibri, are preferred over serif fonts such as Times New Roman. In addition to this, larger size fonts, especially size 14, are easier to read than smaller fonts.¹⁷ The font size in all of the consent forms and readability documents was too small; however, some did use appropriate font style (eg, Arial). Although studies demonstrate that 14 point font is preferred,¹⁷ all the consent forms and informational documents in this study varied between 10 and 12 point font. Documents D, E, F, G, and Z used the favorable sans serif (eg, Arial or Calibri) fonts, whereas documents A, B, C, and Y used serif (Times New Roman) font.¹⁸

A high percentage of passive-voice sentences, or sentences in which the subject receives the action of the verb, also has a negative impact on readability.¹¹ The percentage of passive-voice sentences in the consent forms ranged from 0% to 42.1% (mean 33%). Only consent form G had 0% sentences in passive voice. Assent form F was marginally successful in this category, with only 18.3% written in passive voice.

It is critical that research subjects are able to comprehend informed consent material. Many studies indicate that research volunteers often do not understand critical aspects of the research in which they are participating, suggesting that “informed” consent is imperfectly realized.¹⁹ This is largely due to the increasing length and complexity of consent forms. Providing too much information can have similar consequences to not providing enough information. Consequently, the informed consent process may not accomplish the goal of adequately informing prospective participants about the nature of a study and its potential risks and benefits. As almost half of American adults read at or below the 8th grade level,¹² this study demonstrates that the consent form information used in the phase 3 clinical trials in the United States is too difficult to ensure comprehension for the majority of adult readers. Assessing the study participants’ knowledge about the study at the end of the consent process could help improve their consent form comprehension. If subjects fail to answer questions critical to indicating an understanding of the study correctly, it would indicate a lack of understanding the consent documents. Engaging with study subjects may also help investigators learn about limitations of their consent forms.

Aside from readability that is too complex and text that is too long, the consent forms analyzed in this study also lacked instructional graphics and multimedia including pictures, charts, videos, or slideshows, which increase both appeal and comprehension levels.²⁰ Verbal discussion between volunteers and study educators also appears to be an effective way of improving research participants’ understanding.²¹ Further research is necessary to determine the most effective methods to educate patients on study information.

Consent form readability could be significantly improved by using standard readability formulas, applying rewriting techniques, and being aware of subjects' reading comprehension levels.²² The biopharmaceutical industry, FDA, National Cancer Institute, and National Institutes of Health often advise developing consent forms appropriate for a 6th to 8th grade reading level.⁸ Using readability formulas such as the Flesch Reading Ease Score, Flesch-Kincaid Grade Level, and the Gunning Fog Index would enable preanalysis of readability and allow development of reliable and understandable consent forms. As many factors influence text readability,¹⁰ using several of these formulas as well as other physical formatting aspects of the text can assist in providing highly readable consent forms.

Limitations

The limitations of our study include analyzing readability statistics primarily based on the population of the United States; therefore, these reading scores only pertain to analysis of such documents written in the English language. Studies in other nations may have different population education or language readability statistics. This study does not consider that readability scores may be further affected based on specific institutional review board specifications in a given region or country. This study also does not evaluate whether some people desire more information than presented in the consent forms. Studies show that although some people believe that vaccine pamphlets provide too much information, others may not think the information received is adequate, and at least 1 study has evaluated provision of vaccine related material based on the subject choosing the grade level complexity they prefer.²³ The strengths of our study include comparing consent forms for all 5 phase 3 COVID-19 vaccines conducted in the United States.

Conclusion

The results of our study indicate that an appreciable percent of the study participants involved are likely to have not understood the consent forms used in phase 3 COVID-19 vaccine trials in the United States. This reinforces the need to address—and improve—readability of informed consent forms and informational documents. Further, the role of readability when enrolling the elderly, illiterate, or disadvantaged minority communities in clinical trials who may have lower reading levels, adds further significance to this issue. Additional strategies, such as multimedia formatting, personal discussion, and comprehension assessments to improve the public understanding of consent forms in clinical trials, are also warranted. We conclude that informational documents and consent forms for the current phase 3 COVID-19 vaccine studies are not written in a way that promotes participant comprehension and understanding. Given the vital role these trials have in combating the COVID-19 pandemic, significant work needs to be done to improve the readability and comprehension of informed consent documents.