Improving the Informed Consent Process for Research Subjects with Low Literacy: A Systematic Review

ABSTRACT

BACKGROUND

Inadequate health literacy may impair research subjects’ ability to participate adequately in the informed consent (IC) process. Our aim is to evaluate the evidence supporting interventions, to improve comprehension of the IC process in low literacy subjects.

METHODS

We performed a MEDLINE database search (1966 to November 2011) supplemented by manual searches of bibliographies of key relevant articles. We selected all studies in which a modification of the IC was tested to improve comprehension in low literacy populations. Study design, quality criteria, population, interventions and outcomes for each trial were extracted. The main outcome evaluated was comprehension, measured using a written test or verbal comprehension.

RESULTS

Our search strategy yielded 281 studies, of which only six met our eligibility criteria. The six studies included 1620 research participants. The studies predominantly included populations that were older (median age 61, range 48–64), ethnic minority, and with literacy level of 8th grade or below. Only one study had a randomized design. The specific intervention differed in each study. Two of the studies included the teach-back method or teach to goal method and achieved the highest level of comprehension. Two studies changed the readability level of the IC and resulted in the lowest comprehension among study subjects.

CONCLUSIONS

The evidence supporting interventions to improve the informed consent process in low literacy populations is extremely limited. Among the interventions evaluated, having a study team member spend more time talking one-on-one to study participants was the most effective strategy for improving informed consent understanding; however, this finding is based on the results of a single study.

Informed consent is key to ethical research and is one of the multiple criteria for Institutional Board Review (IRB) approval.¹ Despite federal laws mandating its use, a body of research has documented that inadequate comprehension of informed consent is common among study subjects.^2–5

Nearly half of the United States adult population has marginal (i.e., below basic or basic) health literacy.⁶ The association between limited health literacy and poor health has been supported across acute and chronic diseases.^7,8 Limited health literacy has been consistently related to determinants of self-care behavior, poor performance of self-care behavior, and worse health outcomes.^9,10

Research participants with inadequate or marginal health literacy may not be able to fully comprehend the information disclosed in consent forms.¹¹ In order to facilitate informed consent form comprehension amongst research participants, different types of interventions have been evaluated to aid those with low health literacy.

A previous systematic review reported on studies that included interventions that enhanced consent forms, extended informed consent discussion, test/feedback, and found that spending more time on the informed consent improved participant’s comprehension.¹² This earlier review did not report on research subjects with limited health literacy, however. The objective of this study is to synthesize the evidence from studies that have implemented various intervention methods aimed at improving the understanding of informed consent among participants with low health literacy.

METHODS

Search Strategy

A search was conducted through the MEDLINE database by using PubMed, which contained articles from 1966 to November, 2011. This search was conducted by filtering all articles, except those containing key terms such as informed consent, health literacy, research, and comprehension. More specifically, the search was performed by entering: ("informed consent"[MeSH Terms] OR ("informed"[All Fields] AND "consent"[All Fields]) OR "informed consent"[All Fields]) AND ("health literacy"[MeSH Terms] OR ("health"[All Fields] AND "literacy"[All Fields]) OR "health literacy"[All Fields]) OR ("educational status"[MeSH Terms] OR ("educational"[All Fields] AND "status"[All Fields]) OR "educational status"[All Fields]) AND ("research"[MeSH Terms] OR "research"[All Fields]) AND ("comprehension"[MeSH Terms] OR "comprehension"[All Fields]) AND ("adult"[MeSH Terms] OR "adult"[All Fields] OR "adults"[All Fields]). All searches were conducted in November, 2011 and were supplemented by manual searches of bibliographies of key relevant articles.

Selection Criteria

Two investigators reviewed the abstract of each identified citation. When either investigator selected an article for full text review, the full text was also reviewed by two investigators. Agreement on whether to review the full text or include the article in the evidence table was calculated using inter-rater agreement. Articles were considered for inclusion if they reported on original data where a change in the informed consent was used to improve comprehension about the informed consent for research purposes in low literacy populations.

Data Abstraction

One investigator was responsible for completing the evidence table (LT), and the second confirmed the accuracy of the data abstracted (AP). Differences between the two reviewers were resolved by consensus.

Definition of Health Literacy

The key exposure variable was an intervention used in a low literacy population. We included studies that reported the outcome by health literacy level. Literacy was defined by either a validated method of measuring health literacy or self reported educational level. The use of educational level has been reported as a surrogate marker of health illiteracy.^6,13 We defined limited health literacy as a reading level of 8th grade or below.

Description of the Intervention

We considered an intervention if a study changed the informed consent to improve comprehension. We included randomized trials where those changes were evaluated against a control group, and also non-random studies where the comprehension to changes to the consent document or process were evaluated without a control group.

Outcome

The outcome variable of interest was comprehension, measured using either a written test or verbal comprehension. Comprehension was defined as the average understanding scores for participants in both the intervention group and the control group when applicable. All scores are presented as the average of the scores on the individual tested questions.

Quality Evaluation

We evaluated five quality criteria for each study: whether a trial was randomized, whether the trial evaluated real or simulated informed consent processes, the number of participants in the trial, method of evaluating comprehension and the method of evaluating health literacy.¹² Simulated trials, which asked volunteers to consider a hypothetical decision to enroll in a study, were considered less realistic, and therefore of lower quality, than those that evaluated comprehension of real informed consent processes. Randomized studies were given the highest quality and those with large sample sizes were considered to have the greatest validity. The method of evaluating the key exposure and outcome are of critical importance. The use of validated tools for these purposes was considered a high quality indicator. We chose not to use more formal quality rating systems because the best-validated systems are not clearly relevant to this set of trials.¹⁴

RESULTS

Literature Search

Figure 1 shows the results of our literature search. Our search yielded 281 abstracts. We excluded 250 on the abstract level and selected 31 for full text review; of the 31 selected for full text review, we included five studies and excluded 26 for the following reasons: did not study an informed consent related to research (n = 5); did not report health literacy (n = 9); did not involve modification of the IC process (n = 6); did not report quantitative results (n = 4); and did not report their results in English ( = 2). Inter-rater agreement between reviewers on inclusion vs. exclusion of abstracts was 92 %; 95 % C.I 87-94. The reviewers also identified one 1 citation for review from full text article references. Therefore, we included only six articles into our systematic review. Inter-rater agreement between reviewers on inclusion vs. exclusion of full text articles reviewed was 87.1 %; 95 % C.I 71-95.

Figure 1.

Details of the literature search.

Quality of the Studies

Table 1 reports the qualitative results of our six included studies. Only one study reported on the effects of an intervention on informed consent comprehension using a randomized design.¹⁵ Three of the studies evaluated the improved comprehension on research studies that were not simulated for the purpose of the evaluation. The median sample size was 197, with a range of 29–606 participants per study. Two studies used educational level as a surrogate marker of health literacy, and the most commonly used health literacy tool was Rapid Estimates of Adult Literacy in Medicine (REALM) in four studies. Only one study used a validated tool (Brief Informed Consent Evaluation Protocol, (BICEP)) to evaluate comprehension; ¹⁵ most studies used non-validated questionnaires developed for the purpose of the study; and only one study evaluated comprehension using multiple evaluators.¹⁶

Table 1.

Qualitative Results

Author, year	Study design	Scenario	Sample size	Measure of health literacy	Measurement of comprehension
Bickmore, 2009	Randomized	Simulated	29	REALM	Validated questionnaire(BICEP) and  taken by a single evaluator
Kripalani, 2008	Nonrandom	Real	408	REALM	Oral recall
Sudore, 2006	Nonrandom	Real	204	TOHFLA	Oral recall
Young, 1990	Nonrandom	Real	666	Educational level	NR
Davis, 1998	Nonrandom	Simulated	183	REALM	Non-validated questionnaire
Chong, 2004	Nonrandom	Simulated	190	Educational level	Non-validated by multiple evaluators

NR: Not reported

Baseline Characteristics of the Included Studies and Health Literacy Level

Table 2 reports the baseline characteristics of the six eligible studies. The median age of the participants was 61 (48–64), with females representing the majority of the sample (53 % (36–66)), and ethnic minorities representing the majority of the recruited sample in at least three studies.

Table 2.

Baseline Characteristics of the Studies

Source	Population	Age	% Minorities	% Female	Percentage of participants with inadequate health literacy
Bickmore, 2009	Volunteers	60	NR	66	45
Kripalani, 2008	Coronary heart disease patients	64	90	55	40
Sudore, 2006	Vulnerable patients	61	55	53	40
Young, 1990	Volunteers	18–49			38
Davis, 1998	Oncology patients	48	56	44	73
Chong, 2004	Psychiatric patients	NR	18	36	17

NR: Not reported

The sample combining all eligible studies had 1620 participants. Of this overall sample, the median percentage of participants identified as having inadequate health literacy was 40 % (17–73). Three studies included subjects with specific clinical conditions and two included volunteers.

Description of the Intervention

Table 3 describes the interventions reported in the 6 eligible studies. Bickmore et al.¹⁵ compared a computerized agent against human interaction explaining consent for participating in a hypothetical genetic repository study. The computer animated computer agent had nonverbal behavior synchronized with a text to a speech engine, and patient contributions were done using a touch screen. Two articles described the results of nested randomized studies involving “teach-back” or “teach to goal” methods.^17,18 Both methods involve asking the patient to verbalize or demonstrate their understanding of health information that was just communicated to them. In the first article, Kripalani et al. evaluated the effect of having an interviewer use a script to conduct an overview of the consent and HIPAA authorization for participation in a study to improve cardiovascular medication adherence, and then completed cycles of “teach back.” In the second paper, Sudore et al. studied the impact of reading alongside potential participants of an advance directive study, a 6th grade level informed consent form. They then used the “teach to goal” method to ensure comprehension. Two studies reported on changing the reading level of the informed consent. The first, by Young et al.,¹⁹ used a 6th grade level, and the second by Davis et al.²⁰ compared a 5th grade reading level to a 12th grade reading level. The last study, by Chong, followed the informed consent with an educational module that discussed research terminology.

Table 3.

Interventions in Low Literacy Populations and Comprehension

Source	Type of consent	Intervention	% Comprehension score
Bickmore,  2009	Genetic  repository	Computer  agent	25
		Human  interaction	30
		Self evaluation  of consent	26
Kripalani,  2008	Cardiovascular  medication  adherence trial	Teach back  method	31
Sudore,  2006	Advanced  directive study	Teach to goal  method	33
Young,  1990	Consumer  preference  study	Changing IC to  a 6th grade  reading level	13
Davis,  1998	Cancer study	Cancer patient  input IC with a  5th grade  reading level	45
		IC with a 12th  grade reading  level	43
Chong,  2004	Schizophrenia  treatment  consent	IC followed by  educational  module on  research  terminology	17

IC: Informed consent

Comprehension of Informed Consent

Table 3 reports the comprehension of the informed consent in each study. Of the two studies that evaluated “teach back” or “teach to goal” methods, Kripalani et al. adequately evaluated “teach back” on issues of randomization, study design and disclosure of data in the HIPAA authorization, and reported first pass comprehension of 31 %. The areas where participants had difficulty comprehending were randomization and disclosure of information. Sudore et al.’s ¹⁸ evaluation of the “teach to goal” method assessed comprehension of specific tasks asked of patients, such as blood draws and filling out medical forms, and reported comprehension and first pass comprehension of 33 %. In this study, the reported comprehension after three passes of the “teach to goal” method was 90 % in those who had marginal and inadequate health literacy.

The randomized study by Bickmore et al.¹⁵ comparing an interactive computer agent to human intervention showed that comprehension was higher in the human interaction arm (30 % vs. 25 %); however, the difference was not statistically significant. The studies that decreased the reading level of the informed consent found that subjects that read 6th grade level forms had a comprehension of 13 %, whereas the 5th to 12th grade level comparison revealed a comprehension of 45 % and 43 %, respectively. The study by Chong et al. had a comprehension of 17 %.¹⁶

DISCUSSION

This review synthesizes the evidence of interventions to improve informed consent comprehension and identifies gaps in knowledge that can potentially be used to change policy and regulations. First, our study found that the evidence supporting interventions to improve the comprehension of the informed consent process in low literacy populations is limited. We could only identify six studies that targeted comprehension of a research informed consent in low literacy populations. In each of these studies, the intent was not to evaluate the effect of the intervention on low literacy subjects; rather, low literacy was a subset of the results, and it was treated as a confounder. Only two studies had comparison arms, only one study was randomized, and the method of evaluation of comprehension was usually subjective. There is clearly a need for new comparative and innovative studies that address this important gap in knowledge.

Second, our review found that interventions where a study team member spent more time talking one-on-one to study participants were the most effective at improving research participants’ understanding. This conclusion is supported by the findings of one study where the “teach to goal” method was used and revealed improvement in comprehension from 33 % of subjects after first pass, to 90 % after three passes of the method. However, the concepts that required understanding in this particular study were easier to grasp than those in the other studies. This difference in difficulty may have served as a confounder in the evaluation of this intervention.

Third, a previous systematic review in 2004 reported on interventions to improve research participants’ understanding of informed consent for research.¹² Even though that review did not specifically focus on subjects with low health literacy, its findings are pertinent to our investigation. That review identified 42 studies and found that having extended informed consent discussions through extra meetings reported a median comprehension score of 68 % (63–93 %) for the intervention group, and a score of 60 % (51–73 %) for the control group. Comparing those scores to the scores of 30 %(13–45 %) reported for the interventions included in this analysis illustrates an important gap in understanding between those with low health literacy and all potential research participants.

Fourth, although the institutional review board (IRB) system was designed to assure the ethical conduct of research, IRBs typically do not facilitate the identification of people with low health literacy as vulnerable subjects, do not take into account the reading difficulty of informed consent documents,²¹ and do not enforce the assurance of comprehension of informed consents as good clinical/research practice.²²

Research informed consents, when compared to other consents, are particularly problematic because of the inherent mistrust of minorities participating in clinical research and because the research informed consents contain statements that are frequently lengthy and complex, have unfamiliar medical and legal terminology, and have risk information that requires numeracy skills.

Our study has several limitations. First, the small number of published studies limits our ability to comment on the different interventions. Second, health literacy was measured using different methods, and this could have misclassified subjects. Third, the generalizability of the study is subject to the findings. Fourth, although review methods were systematic, one study was not identified using a broad search of MEDLINE, indicating that we could have missed studies. Fifth, there is no single objective way of weighing the results based on quality. Finally, small sample size and heterogeneity prevented mathematical pooling of the data.

Poor literacy skills may directly affect not only participation in clinical research, but also understanding of potentially beneficial procedures needed for clinical care. In fact, studies documenting factors that affect comprehension come from the use of informed consent in non-research settings, where there are inconsistent results on the timing of the assessment and the instruments used.¹³ A potential problem is that health literacy goes beyond just literacy to encompass an understanding of health-related processes, including the pathophysiology of disease and how to navigate the health system. Therefore, a patient or research subject could theoretically “teach back” risks and benefits, but might not have good insight into the long-term implications of those risks and benefits.

In conclusion, the field of informed consent comprehension in low literacy populations needs to be addressed by all stakeholders, including investigators, research participants and regulatory personnel. Institutional Review Boards need to identify evidence-based interventions and conduct evidence-based regulation review to ensure this important deficiency is corrected. Future studies should focus on understanding the prevalence of low literacy volunteering in clinical research, patient-centered approaches to developing new interventions, and comparative effectiveness of interventions in low literacy populations.