Informed consent document improvement does not increase patients' comprehension in biomedical research

Abstract

AIMS

International guidelines on ethics in biomedical research require that the informed consent of all enrolled participants is obtained. A written document describing the research, the informed consent (IC) document, must be given to all participants by the investigator. Most IC documents are long, containing much information. The aim of the present study was to determine whether the modification of the IC document by a working group or systematic improvement in its lexicosyntactic readability can improve comprehension of the written information given to patients participating in biomedical research.

METHODS

One hundred and fifty-nine patients were randomized to read one of the three versions of the IC document: unchanged document, document modified using systematic improvement of lexicosyntactic readability and document modified by a working group.

RESULTS

Neither the improvement in the lexicosyntactic readability, nor the intervention of the working group significantly improved the score of objective comprehension for the subjects included in this study: it was 66.6 (95% confidence interval 64.0, 69.2) for the control group, 68.8 (66.2, 71.4) for the group with the document improved for lexicosyntactic readability and 69.2 (66.0, 72.4) for the group who read the document improved by the working group (P= 0.38).

CONCLUSIONS

We failed to show that improving IC document comprehension through a lexicosyntactic approach or by a working group leads to better comprehension.

WHAT IS ALREADY KNOWN ABOUT THIS SUBJECT

• Before enrolment, an informed consent (IC) document must be given to all participants by the investigator.

• Most IC documents are long, contain much information and are difficult to understand.

• Some methods to improve comprehension were tested, but they had limited effects.

WHAT THIS STUDY ADDS

• We failed to show that improving IC document comprehension through a lexicosyntactic approach or by a working group leads to better comprehension in our patient population, whichever method was used.

• We confirm too that a study in real conditions is necessary.

Introduction

International guidelines on ethics in biomedical research require that the informed consent of all enrolled participants is obtained [1]. In France, the national legislation was recently modified to comply with the European directive [2]. A written document describing the research, the informed consent (IC) document, must be given to all participants by the investigator. French law details the elements that the IC document must contain: the objective of the study, the methodology and duration of the research and its location, the expected benefits, constraints and predictable risks, possible alternative treatments, medical care after the study, confirmation that the sponsor has subscribed to an insurance policy, that the study has been approved by the ‘Comité de Protection des Personnes’[CPP, the French equivalent of an Institutional Review Board (IRB)] and authorization by the competent authority. In addition, if required, the rules forbidding simultaneous participation in another research study and the appropriate exclusion period need to be stated, as well as the participant's right to refuse to participate in the research, their right to withdraw consent at any moment and the possibility of delay for reflection. Including all these details implies that most IC documents are lengthy and contain much information. As a consequence, doctors who experience the process of providing written information often have the impression that participants do not grasp the appropriate information from the IC document.

We have previously shown that the lexicosyntactic readability of French IC documents is lower than the readability score of the most difficult reference texts (university level) [3]. More recently, we have shown that while the readability of IC document is low, the density of the information provided is also low [4].

In 2004, Flory et al.[5] published a systematic review of studies performed in order to test the different methods that have been tried to improve the comprehension of documents given to participants in biomedical research. The five categories of intervention were: multimedia, enhanced consent form, extended discussion, test/feedback and miscellaneous. The use of multimedia and an enhanced consent document had limited success and it seems to be better to have a third party who spends more time discussing the proposed research with the participants. However, Flory et al. concluded that further research was required due to the poor quality of some studies. We recently conducted a randomized controlled study on healthy volunteers, in a simulated situation, to evaluate the impact of two methods of modification of the IC document on their comprehension [6]. The interventions were: a systematic improvement of the lexicosyntactic readability and improvement by a working group. We showed that improving the IC document in Phase I biomedical research led to better comprehension, whichever method was used. However, the combination of both methods did not further improve comprehension. Given this interesting result, we decided to conduct a similar study in a population of patients participating in Phase III biomedical research, QuIP-4.

The aim of the present study was to determine whether the modification of the IC document by a working group or systematic improvement in its lexicosyntactic readability can improve comprehension of the written information given to patients participating in biomedical research.

Subjects and methods

Questionnaire QECIEM

The Questionnaire d'Evaluation de la Compréhension de l'information Ecrite chez des Malades (QECIEM) used here with patients is an adaptation of the Quality of Informed Consent (QuIC) [7] for use in French. Briefly, the adaptation was made in five consecutive logical steps: translation, scientific validation, lexical validation, edition of gold-standard answers and a pilot study. The QECIEM is composed of 28 questions in part A (objective comprehension, i.e. what the participant really understood) and 12 questions in part B (subjective comprehension: i.e. what the participant thinks they understood). The six domains of information are found in both parts: ‘Notion of experimentation’, ‘Objective of the study’, ‘Methodology’, ‘Legal obligations’, ‘Benefit/risk and constraints’ and ‘Subject protection’. For scoring, we used the same method as for the Questionnaire de Compréhension des Formulaires d'information et de consentement (QCFic). We determined a global score of comprehension, a score of objective comprehension for part A, a score of subjective comprehension for part B and six secondary scores for each domain of information. Each score was out of 100 points.

Fourteen patients were included in a preliminary pilot study. Six were diabetic and eight had presented stroke in the 15 days before inclusion. The time to read the text was 31.1 (95% CI 30, 32.2) min and the time to answer the questions 16.8 (95% CI 13.9, 19.7) min. The score of objective comprehension was 70.8 (95% CI 67, 74.7). The score of subjective comprehension was 85.1 (95% CI 78.7, 91.5) and the global score 78.0 (95% CI 73.3, 82.7).

Subjects

Subjects were recruited between 8 August 2007 and 11 June 2008 in five French clinical research centres: Grenoble, Strasbourg, Lyon, Créteil and Toulouse. The inclusion criteria were inpatients presenting a stroke <2 weeks before or diabetes mellitus or obstructive sleep apnoea syndrome (OSAS), agreeing to read an IC form and to answer the QECIEM, and age >18 years. Exclusion criteria were illiteracy, refusal to participate, neurological disorders that made reading impossible and age <18 years. Although participants were informed about the study, the information process differed from that usually performed for a clinical trial. All participants were informed orally, and no signed consent was required. This was in accordance with French law, including favourable advice from the Rhône-Alpes Auvergne IRB (IRB no. 5045) on 2 May 2007. We intentionally used such a process to avoid confusion.

Design

One IC document was randomly chosen for each disease (stroke, OSAS and insulin-dependent and non-insulin-dependent diabetes) among recently ended Phase III clinical studies. Only one document was chosen for each disease and used in the present study in all participating centres. Participants received only the IC document appropriate to their disease. All IC documents concerned a genuine closed study and thus had been approved by a CPP. All references to the sponsor and the drug tested were modified to ensure confidentiality. Participants were randomized in a ratio 1:1:1 using centralized electronic randomization based on a computerized random-number generator provided by Clininfo S.A.® (Lyon, France). Stratification by clinical research centre and pathology was performed through the centralized electronic randomization. The size of the blocks was randomly 3, 6 or 9, and was unknown to the study coordinators.

Patients were asked to read one of the three versions of the IC document: unchanged IC document (A), IC document modified using systematic improvement of lexicosyntactic readability based on an increase in the Flesch index (B), and IC document modified by a working group composed of a member of a ‘Comité de Protection des Personnes’ qualified in ethics, a clinical research assistant and a users' representative (C). The methods employed to improve lexicosyntactic readability were the same as in the previous study. Briefly, words composed of three or more syllables were replaced by shorter synonyms when possible and the length of the sentences was reduced. The method used by the working group was dependent on the people participating and no directives were given.

Patients were left to read the IC document for at least 30 min (maximum 1 h). As this was a simulation, the patients were informed that they did not have to make a decision about participation in the study described by the IC document, but that they had just to read the document. Next, they were asked to complete the QECIEM.

Objectives and end-points

The main objective of the QuIP-4 study was to find a method to improve the comprehension of written information given to patients in biomedical research. The primary end-point was the objective comprehension of each study group. Second, we studied the potential effect of different characteristics of the population such as gender, educational level and professional sphere. Secondary end-points were the global score, the score of subjective comprehension and the scores in the different domains of information.

Statistical analysis

Sample size calculations were based on the objective to detect a difference in the objective comprehension score of at least 5 units between groups, with α= 0.05 and power (1-β) = 95%. As the mean objective comprehension score was 70.8 (SD = 7.4) in the 14 volunteers enrolled in our pilot study, a total of 58 subjects per group was necessary, i.e. a total of exploitable questionnaires of 174 (NQueryAdvisor 6.01®). We chose to include 210 volunteers in total to consider potential non-exploitable questionnaires and patients drop-outs. The number of subjects was calculated assuming that improving comprehension would aim at increasing the comprehension level from secondary school to high school, i.e. an increase of 5 of the comprehension score as suggested by our previous study in healthy volunteers.

Quantitative data were described by mean and 95% confidence interval if the distribution was normal, and by median, 10th and 90th percentiles in the other cases. Qualitative data were described by size and percentage. For the primary end-point, analysis of variance was performed to identify influencing factors. If significant difference was identified, pairwise post hoc tests were performed using Bonferroni correction. We planned a priori to compare the two methods with the control, i.e. only two pairwise comparisons. Thus a P-value <0.05, or <0.05/2, i.e. 0.025 following Bonferroni correction for post hoc multiple comparisons, was considered as statistically significant. P-values were two-sided.

Results

Description of the population

One hundred and seventy-one patients were included between August 2007 and July 2008 in the five centres for the study. Due to delays in enrolment, we did not include the 210 patients as originally planned. Twelve patients withdrew shortly after inclusion and finally 159 subjects read the IC document and 153 QECIEMs were exploitable (Figure 1). One hundred and one patients were included by the Grenoble centre, 44 at Strasbourg, 10 at Créteil, 10 at Lyon and six at Toulouse. The repartition between each class of IC document was homogenous in the centres due to stratification. The characteristics of the population are described in Table 1.

Figure 1.

Flow-chart. IC, informed consent

Table 1.

Study population

	Unchanged informed consent (n= 58)	Improved readability of informed consent (n= 57)	Informed consent modified by a working group (n= 56)
Age, years (mean [95% CI])	51.7 [48.3, 55.1]	54.5 [50.1, 58.9]	51.5 [47.3, 55.7]
Women (n, %)	24 (45.3)	22 (40.0)	21 (41.2)
Educational level (n, %)
Secondary school	21 (41.2)	18 (33.3)	13 (26.0)
High school	13 (25.5)	21 (38.9)	14 (34.0)
Higher education	17 (33.3)	15 (27.8)	20 (40.0)
Professional sphere (n, %)
Medical, paramedical	8 (15.4)	3 (5.46)	3 (5.9)
Other	44 (84.6)	52 (94.55)	48 (94.1)
Disease (n, %)
IDDM	19 (32.8)	16 (28.1)	18 (32.1)
NIDDM	28 (48.3)	27 (47.4)	24 (42.9)
Stroke	7 (12.1)	10 (17.5)	12 (21.4)
OSAS	4 (6.9)	4 (7.0)	2 (3.6)
Participation in a previous study (n, %)	9 (17.3)	4 (7.3)	7 (14.0)
Time to read the IC document minutes (mean [95% CI])	32.3 [30.2, 34.4]	32.6 [30.1, 35.1]	33.7 [30.8, 36.6]
Time to answer the QECIEM minutes (mean [95% CI])	16.7 [14.3, 19.1]	15.9 [14.3, 17.5]	17.0 [15.0, 19.0]

IC, informed consent; IDDM, insulin-dependent diabetes mellitus; NIDDM, non-insulin-dependent diabetes mellitus; OSAS, obstructive sleep apnoea syndrome; QECIEM, Questionnaire d'Evaluation de la Compréhension de l'Information Ecrite chez des Malades.

Primary end-point

Neither the improvement in lexicosyntactic readability, nor the intervention of the working group significantly improved the score of objective comprehension for the subjects included in this study: it was 66.6 (64.0, 69.2) for the control group, 68.8 (66.2, 71.4) for the group with the IC document improved for lexicosyntactic readability and 69.2 (66.0, 72.4) for the group who read the IC document improved by the working group (anovaP= 0.38). The score of objective comprehension for patients who read the IC document improved for lexicosyntactic readability was 2.2 (–1.4, 5.9) points higher than that for patients who read original IC document. The score of objective comprehension for the patients who read the IC document improved by the working group was 2.6 (–1.5, 6.8) points higher than that for patients who read the original IC document.

Secondary end-points

There was no statistically significant difference between the three study groups for either the global QECIEM score or each of the six domains for both objective and subjective parts (Table 2).

Table 2.

Score of the different domains and parts of the QECIEM (results are expressed mean [95% confidence interval])

	Unchanged informed consent (n= 51)	Improved readability of informed consent (n= 53)	Informed consent modified by a working group (n= 49)	P-value
Global score	75.4 [73.1, 77.7]	74.9 [71.8, 78.0]	77.2 [68.5, 75.9]	NS
Objective comprehension	66.6 [64.0, 69.2]	68.8 [66.2, 71.4]	69.2 [66.0, 72.4]	NS
Notion of experimentation	65.9 [58.9, 72.9]	68.1 [61.4, 74.8]	76.1 [69.0, 83.2]	NS
Objective of the study	50.3 [46.2, 54.4]	55.2 [50.6, 59.8]	55.2 [50.3, 60.1]	NS
Methodology	66.9 [63.5, 70.3]	69.7 [66.0, 73.4]	68.8 [64.3, 73.3]	NS
Legal obligations	68.1 [60.9, 75.3]	70.5 [62.9, 78.1]	73.4 [66.2, 80.6]	NS
Benefit/risk and constraints	68.4 [62.1, 74.7]	61.4 [56.6, 66.2]	62.0 [57.4, 66.6]	NS
Subject protection	77.0 [72.7, 81.3]	78.7 [75.2, 82.2]	78.1 [73.3, 82.9]	NS
Subjective comprehension	83.1 [79.5, 86.7]	80.9 [75.9, 85.5]	85.2 [80.2, 90.2]	NS
Notion of experimentation	93.1 [89.2, 97.0]	85.4 [78.9, 91.9]	88.4 [81.1, 95.7]	NS
Objective of the study	90.2 [85.8, 94.6]	82.1 [76.0, 88.2]	83.2 [75.1, 91.3]	NS
Methodology	80.1 [74.8, 85.4]	79.0 [72.5, 85.5]	84.5 [78.9, 90.1]	NS
Legal obligations	77.5 [70.7, 84.3]	76.0 [68.5, 83.5]	81.6 [74.5, 88.7]	NS
Benefit/risk and constraints	82.6 [76.3, 88.9]	79.5 [73.7, 85.3]	86.7 [80.5, 92.9]	NS
Subject protection	92.2 [89.0, 95.4]	90.6 [85.8, 95.4]	89.5 [83.8, 95.2]	NS

QECIEM, Questionnaire d'Evaluation de la Compréhension de l'Information Ecrite chez des malades; NS, not significant.

We conducted a two-way anova to identify any characteristics of the population that could explain the objective score. There was no effect of gender, pathology, centre or professional sphere. The fact that the patients had already taken part in a clinical study had no significant influence on their comprehension.

Nevertheless, on the objective score of comprehension, there was an effect of educational level [62.1 (59.8, 64.4), 69.8 (67.3, 72.3) and 72.6 (69.7, 75.5) for secondary school, high school and higher education respectively; P < 0.01] and professional category [workman 65.1 (62.5, 67.7), intermediate profession 69.3 (66.6, 72.0), manager 72.8 (69.4, 76.2) and unemployed 66.4 (61.2, 71.6); P < 0.01], but no interaction with the type of IC document. In addition, age was inversely correlated to the comprehension score (Figure 2; P < 0.01). Educational level and profession were closely related as expected, and the difference observed relied on the educational level when both were introduced as covariates in a multivariate analysis, profession being not significant. However, age and educational levels were independently associated with the objective comprehension score, with no interaction.

Figure 2.

Correlation between age and objective comprehension. QECIEM, Questionnaire d'Evaluation de la Compréhension de l'Information Ecrite chez des Malades, R, coefficient of correlation

Discussion

We failed to show that improving the IC document through a lexicosyntactic approach or by a working group leads to better comprehension by our patient population, whichever method was used.

Several authors have studied the quality of IC documents to examine the relationship between readability and comprehension and to propose methods to improve both. In the USA, the readability and length of 107 IC documents in the field of cancer research were analysed [8]. No document had a readability index of less than the 8th degree of the Flesh Kincaid scale (this degree corresponds to the ‘brevet des colleges’ in France, usually taken at age 15 years). More recently, the analysis of the readability of 154 IC documents in biomedical research relating to mental illness showed that 35% of the population concerned in these studies did not have the minimal school level required to read the documents properly [9]. In France, we compared the lexicosyntactic readability of French IC documents with reference texts corresponding to five educational levels. We concluded that the lexicosyntactic readability of French IC documents was lower than the readability score of the most difficult reference texts (university level) [3]. More recently, we have shown that while the readability of IC documents is low, the density of the information given is also low [4].

In terms of comprehension, <10 weeks after the signing of the consent form, 28% of 156 veterans did not know that they were included in a study and only 10% were able to explain correctly the objective of the research [10]. In the same spirit, 74% of patients included in a study did not understand that the treatment proposed was not standard [11].

The quality of IC documents is obviously insufficient and methods are required to improve them in order to improve patients' comprehension. The legislators are conscious of this problem and French law requires that the CPPs (IRBs) ensure that the intelligibility of the IC document is satisfactory. To examine the effectiveness of CPP intervention, we performed a study comparing the lexicosyntactic readability of French IC documents before and after review by a CPP [12]. The readability was not improved and the length of the text was increased.

The present results contradict those obtained in our previous study [6], but are in line with other studies suggesting that IC document lexicosyntactic improvement does not increase patients' comprehension in biomedical research [13]. Moreover, our results confirm some of the previous studies [14, 15]. A comparison of the published studies is difficult because of the diversity of measurement tools used. Despite this, in 2007 Cohn et al.[16] performed a systematic review of studies conducted over the last 10 years to test methods of improving comprehension in biomedical research. As well as the diversity of the tools employed, they also highlighted differences in the definition of comprehension. Like Flory 3 years earlier [5], they concluded that the best method seems to be the intervention of a third person, although even this was not entirely sufficient. Concerning the definition of comprehension, some studies have evaluated subjective comprehension (what the patients think they understand) and others objective comprehension (what the patients really understand). In the present study, we chose to evaluate objective comprehension. We aimed to involve people who enter into a biomedical research programme as real partners and not just subjects for data collection. In this respect, we believe that it is extremely important that volunteers genuinely understand the project and not simply that they have the sensation of having understood. In the process of deciding to participate in biomedical research not just factual elements, but also psychological elements play an important role.

Our results raise the question whether our comprehension questionnaire was sensitive enough. However, the fact that we were able to detect a population of patients having a lower comprehension score suggests that it is a sufficiently sensitive tool. Furhtermore, we were able to show an age-dependent decrease in patient comprehension. Finally, in a previous study using a similar questionnaire with healthy subjects, we showed an improvement in comprehension [6].

Detailed examination of our results shows that there is a nonsignificant trend towards better objective comprehension. In other words, while we show that the impact of the intervention was at least minimal, we cannot rule out the hypothesis that this study was underpowered to detect a smaller difference or the question whether such a small difference would have any clinical impact. In fact, with a mean objective comprehension score of 66.6 ± 9.3 and a difference of 2.6 between the two groups, our study was underpowered to detect such a small difference (power = 28%). Indeed, SDs in this study were larger than those in the pilot study used to calculate power a priori. To achieve a power of 80%, 282 subjects would have been necessary.

Both our results and the review of the literature raise the question as to whether it is possible to improve comprehension by working on the IC document. Improving the document does not always work and resorting to a multimedia approach is not necessarily a good tool. Thus, the only method that may be efficient is the intervention of a third person (from the study team or an independent individual), someone who can spend more time discussing the study information with the participant. Indeed, it is important to remember that although written information must be given to the participant, it is also given orally by the investigator or their representative. In the present study the written information was not supported by oral information.

To conclude, we failed to show that improving IC document comprehension through a lexicosyntactic approach or by a working group leads to a better comprehension.

Competing interests

None to declare.

We thank Aurélie Thollet, Hugo Borrell, Amandine Scata, Sébastien Gillard, Pauline Engel, Séverine Niglis, Laurie Maurin, Pauline Jouany and Frédéric Sakr for their help in recruiting volunteers; Pierre-Yves Benhamou, Jean-Louis Pépin, Olivier Detante and Wilfried Vadot for their help in the conception of the case report forms, and Alison Foote for correcting the manuscript. This study was supported by a grant from the University Hospital of Grenoble, France.