Abstract
Objectives:
To evaluate patient attitudes towards literacy screening, agreement between literacy tests, and associations between literacy, informed consent comprehension, and health-related quality of life (HRQL).
Methods:
Participants completed three literacy tests, read a sample consent form, and reported their HRQL, experiences, and attitudes.
Results:
We enrolled 97 cancer patients, of whom 66% were female, 67% were African American, and 65% were high school graduates. Sixty percent of patients with lower reading comprehension had trouble reading health information, and 31% had trouble reading everyday written material. Even patients with higher reading comprehension had trouble reading health information (29%) and everyday written material (10%). Low-literacy patients were more likely to feel anxious about literacy screening. However, the majority of patients (84%) would be willing to have literacy results given to providers. Comprehension of informed consent increased with higher literacy. There were no HRQL differences.
Conclusions:
Patients report difficulty comprehending written health information. Literacy assessment is acceptable and it is considered important for providers to be aware of their patients’ reading abilities. Patient preference data should be used to improve literacy testing strategies and measures. Enhancing detection of low literacy can facilitate interventions to reduce health disparities.
Keywords: health literacy, screening, informed consent, reading, writing
Introduction
Low literacy is a widespread but neglected problem in the US. Almost half of the adult population experiences difficulty in reading, speaking, writing, and using computational skills in everyday life situations.1 “Health literacy”, the constellation of skills required to function in the health care environment, may be significantly worse than functional literacy because of the unfamiliar context and vocabulary of the health care system.2
Despite its prevalence, low literacy is often underrecognized and underreported. For example, the majority of adults who scored low on the National Adult Literacy Survey described themselves as being able to read or write English “well” or “very well”.1 Many low-literate individuals are ashamed of their reading difficulties and try to hide the problem. Moreover, because there are generally only moderate reading demands in everyday life, individuals may not be aware of their reading problems until a literacy-challenging event (such as reviewing treatment options, reading a consent document, or completing forms) occurs.3
Lack of recognition and denial of reading problems creates a barrier to health care and health research. Health care providers rarely assess literacy skills and low-literate patients are often uncomfortable disclosing their reading deficiencies. Because they are ashamed of their reading difficulties, low-literate patients have acknowledged avoidance of medical care.3–4 Likewise, they are often excluded from research studies due to reading and writing ability being a common inclusion criterion. Such selection bias distorts estimates of study findings and limits generalizability.5 In addition, health care providers routinely distribute to patients educational materials that require advanced reading skills for comprehension.6 Similarly, consent forms for clinical research are often written at or above a college reading level, and many patients have difficulty understanding them.7 Thus, patients without adequate literacy skills may not be able to assess accurately the risks and benefits of a study. Increasing recognition is being paid to the mismatch in providers’ communication processes regarding how information is given and patients’ communication processes for understanding, remembering, and applying information.8 It is generally recommended that providers accept primary responsibility for ensuring that patients have adequate understanding.8,9
Low literacy is associated with health disparities, such as reduced access to health information, poorer understanding of illness and treatment, less effective disease management, less understanding and use of preventive services, advanced disease at diagnosis, lower medication adherence, poorer physiologic health markers, increased hospitalizations, and higher financial costs.6,10–17 Evidence about the relationship between literacy and patient-reported outcomes, such as health-related quality of life (HRQL), is more limited and inconclusive.18 Some studies have found an association between poorer reading ability and poorer HRQL/perceived health status,10,19–21 while others have found that low literacy is not an independent risk factor for poorer HRQL outcomes.22 More research, using valid literacy measures, is needed in this area.
Educational level, which has itself been weakly associated with poor health and mortality,23 is often used as a literacy proxy measure. However, literacy and years of formal education are only modestly correlated,1,7,24,25 and the association between education and health may disappear after adjusting for literacy.10,11 Years of education reflect the educational level attempted, but literacy is an indicator of the educational level attained. Literacy is thus a better indicator of the ability to acquire new knowledge and cope with societal demands.10 Reading comprehension tests are considered the best measure of literacy although the results do not correspond well with years of education.22,26,27
The best approach to assessing literacy in health care settings in a way that promotes patients receiving the best possible care remains unclear. To address this important issue, we evaluated three different types of literacy assessment, ie, a reading recognition test, a reading comprehension test, and an assessment of functional health literacy. Our primary purpose was to evaluate patient preferences and attitudes towards literacy screening. We also evaluated the agreement between these tests, clarified the advantages and disadvantages of each, evaluated their ability to predict comprehension of informed consent, and evaluated the effects of literacy level on self-reported HRQL. Further research in this area should help people who lack sufficient literacy to understand medical treatment options and receive care that meets their needs.
Methods
Interviewer training
Three study interviewers were responsible for recruitment, and were experienced bachelors-level employees of our research center. They participated in several training sessions that included general interviewing techniques, administration of surveys and questionnaires, administration and scoring of literacy tests, and sensitization to issues of low literacy, such as shame and avoidance.
Patient enrolment
We enrolled a convenience sample of adult cancer patients (at least 18 years of age) at two Chicago cancer clinics that provide care to low-income, uninsured, and vulnerable populations. From June 2002 to July 2003, the study interviewers spent one or two days each week at each clinic and approached patients in the waiting room. The Institutional Review Boards gave permission to gather a small amount of information (age, gender, race, education, diagnosis) from patients who were not enrolled. All patients who agreed to participate provided informed consent and received $20. Eligibility criteria included having any cancer type, stage or treatment; any age, gender or race; and sufficient English language ability and absence of cognitive impairment based on the judgment of the trained study interviewer. Reading ability was not an eligibility criterion, and was not assessed until the patient signed a consent form to participate in the study.
Patient assessment
After enrolment, visual acuity of at least 20/70 was confirmed using a Rosenbaum hand-held vision card. Sociodemographic and clinical information was collected from the patient and medical records. The study instruments are described below in their order of administration, although the three literacy tests were randomly ordered.
Instruments
Eastern Cooperative Oncology Group Performance Status Rating
The Eastern Cooperative Oncology Group (ECOG) Performance Status Rating (PSR)28 is a widely used single-item rating of the degree to which patients feel they are able to participate in typical activities without the need for rest. It uses a five-point ordinal scale ranging from “normal activity” to “unable to get out of bed”.
Functional Assessment of Cancer Therapy-General
The Functional Assessment of Cancer Therapy-General (FACT-G)29–31 is a 27-item measure of HRQL. All items use a five-point Likert response scale (“not at all” to “very much”). In addition to a total score, there are four subscales, ie, physical, functional, social, and emotional well-being. Responses to negative items (eg, “I have nausea”) were reversed so that a higher HRQL score represents better HRQL. The FACT-G has demonstrated strong psychometric properties, including reliability, validity, and sensitivity to change. The FACT-G was administered by interview unless patients requested to complete it on their own.32
Rapid Estimate of Adult Literacy in Medicine
The Rapid Estimate of Adult Literacy in Medicine (REALM)33 is a reading recognition test that measures a person’s ability to pronounce words in increasing order of difficulty; it does not measure comprehension. It uses 66 common medical terms, and can be administered with minimal training. The REALM has a high concentration of items at lower levels to increase its discriminatory power for detecting low literacy.
Woodcock Language Proficiency Battery-passage comprehension subtest
The Woodcock Language Proficiency Battery (WLPB) is a multi-item battery of nine subtests taken from the Woodcock-Johnson Psycho-Educational Battery-Revised. The WLPB manual reports impressive reliability and validity statistics. The Passage Comprehension (WLPB-PC)34 subtest is a self-contained reading comprehension test that can be administered adaptively in a relatively short period of time, ie, 10 minutes.
Short Test of Functional Health Literacy in Adults
The TOFHLA7,35,36 consists of a 50-item reading comprehension and a 17-item numeracy test developed using actual hospital materials such as prescription labels, instructions for preparation for medical tests, and informed consent forms. An abbreviated version (S-TOFHLA) consists of 36 reading comprehension and four numeracy items. We used the S-TOFHLA in this study. We also administered all 17 numeracy items to use for psychometric analyses for a related project.
Preference survey
After administration of the three literacy assessments, participants were asked to rate them in terms of ease of completion, interest, and preference. Rather than using a bipolar rating scale (eg, “easy” to “hard”), patients were asked to select the easiest test, the hardest, the most interesting, the most boring, the one they would most prefer to do again, and the one they would least prefer to do again. This interviewer-administered survey also included questions about the literacy testing experience, including ease of completion and duration.
Informed consent form and comprehension interview
We adapted an “easy to read” consent form that has been tested in ECOG cancer clinical trials.37 The names of the diagnoses and drug regimens were changed to “disease X”, “drug A”, “drug B”, etc. The seven-page consent form was described to patients in this way: “This is a sample of a consent form. We are not asking you to participate in a treatment study today. I would like you to read this sample consent form from start to finish, and I will ask you questions about the information contained in the sample consent form when you are through. Don’t worry about memorizing the entire packet. You will be able to refer back to the packet when answering the follow-up questions”. Patients were allowed up to 15 minutes to read the consent form. A comprehension interview was then conducted, using a subset of questions from the ECOG study. There were three questions about the clarity of the consent form, and 20 comprehension questions. The comprehension questions were aggregated into eight subscales, ie, privacy (two items), side effects (three), personal benefit (six), benefit to others (one), choice (four), financial (two), randomization (one), reason off-study (one), as well as a total score (20).
Final interview
The final set of questions asked about the patient’s experiences, attitudes, and self-assessed literacy skills (eg, “Do you ever have trouble reading printed information given to you by doctors or nurses, for example, booklets, forms, or instructions for medication?” “Do you ever have trouble reading everyday things like the newspaper or signs on the road?”).
Statistical analyses
All instruments were scored in accordance with the developers’ instructions. One patient who reported being unable to read at all was assigned a score of zero for all three literacy tests. The literacy test scores were also dichotomized to create low- and high-literacy groups. Specifically, the REALM was dichotomized at 45 (approximately a seventh-grade level), the S-TOFHLA was dichotomized into “adequate” versus “inadequate” or “marginal” functional health literacy, and the Woodcock was dichotomized at the seventh-grade level. Education was also dichotomized at the seventh-grade level. We used the seventh-grade criterion because it corresponds with the reading level of the FACT-G. For each participant we calculated the proportion of comprehension questions answered correctly in each subscale and overall. Differences between literacy groups were reported as effect sizes (mean difference divided by the common standard deviation). An effect size of 0.2 is considered small, 0.5 is moderate, and 0.8 is large.38 Ordinal response scales were dichotomized as follows: “very much so” versus “not at all/somewhat/ moderately so”; “strongly agree/agree” versus “disagree/ strongly disagree”; “some/most/all of the time” versus “none of the time”; “a little bit/somewhat/quite a bit/very much” versus “not at all”.
Sociodemographic and clinical characteristics were compared between low- and high-literacy groups, and between enrolled and nonenrolled patients, using an independent t-test or Wilcoxon rank-sum test for continuous variables, a Pearson chi-square statistic or Fisher’s exact test for nominal variables, and a Mantel-Haenszel chi-square statistic for ordinal variables. Rank correlation coefficients (rs), McNemar’s test for correlated proportions and the Kappa (κ) statistic were computed for the associations between literacy and education, and to assess pairwise agreement between literacy tests. Values for κ below 0.40 indicate poor agreement.39 Patient ratings of the literacy tests, their experiences and attitudes, and their ratings of the clarity of the consent form were compared across literacy levels with a Pearson chi-square statistic or Fisher’s exact test. Mean comprehension scores were compared between low- and high-literacy groups using a Wilcoxon rank-sum test.
Results
We approached 128 patients and enrolled 97 (76%). The most common reasons for refusal to participate were the time commitment required for the study (n = 10) or feeling too ill (n = 7). Refusers were comparable with enrolled patients in terms of gender, race, education, and diagnosis, but were slightly older. Enrolled patients were primarily female (66%), African American (67%), and high school or college graduates (65%). Sociodemographic and clinical characteristics are summarized in Table 1 for all patients and separately by literacy level on the WLPB-PC.34 Low-literacy patients were slightly older than high-literacy patients, and included a higher proportion of African Americans, lower education level and poorer performance status. There were no differences in HRQL. Results were similar using the REALM or S-TOFHLA literacy levels.
Table 1.
Low literacy (n = 45) | High literacy (n = 52) | Pb | All patients (n = 97) | |
---|---|---|---|---|
Female | 31 (68.9%) | 33 (63.5%) | 0.574 | 64 (66.0%) |
Mean age ± SD | 53.9 ± 10.3 | 48.2 ± 12.3 | 0.017 | 50.8 ± 11.7 |
Race/ethnicity | ||||
Hispanic, any race | 3 (6.7%) | 10 (19.2%) | 0.0187 | 13 (13.4%) |
Black/African American, non-Hispanic | 37 (82.2%) | 28 (53.9%) | 65 (67.0%) | |
White, non-Hispanic | 5 (11.1%) | 12 (23.1%) | 17 (17.5%) | |
Multiracial | – | 2 (3.8%) | 2 (2.1%) | |
Highest level of education | ||||
<7th grade | 1 (2.2%) | 1 (1.9%) | <0.0001 | 2 (2.1%) |
7th to 11th grade | 23 (51.1%) | 9 (17.3%) | 32 (33.0%) | |
High school/GED | 17 (37.8%) | 13 (25.0%) | 30 (30.9%) | |
College | 4 (8.9%) | 29 (55.8%) | 33 (34.0%) | |
Current occupational status | ||||
Employed, student, homemaker | 11 (24.4%) | 13 (25.0%) | 0.139 | 24 (24.7%) |
Retired, disability, leave of absence | 13 (28.9%) | 24 (46.2%) | 37 (38.1%) | |
Unemployed | 21 (46.7%) | 15 (28.8%) | 36 (37.1%) | |
Cancer diagnosis | ||||
GI, solid tumors | 17 (37.8%) | 16 (30.8%) | 0.896 | 33 (34.0%) |
Gender-specific tumors | 18 (40.0%) | 22 (42.3%) | 40 (41.2%) | |
Hematologic | 9 (20.0%) | 12 (23.1%) | 21 (21.7%) | |
Other | 1 (2.2%) | 2 (3.8%) | 3 (3.1%) | |
Currently receiving chemotherapy | 30 (66.7%) | 30 (57.7%) | 0.364 | 60 (61.9%) |
Performance status | ||||
Normal activity, no symptoms | 4 (8.9%) | 5 (9.6%) | 0.035 | 9 (9.3%) |
Some symptoms | 12 (26.7%) | 29 (55.8%) | 41 (42.3%) | |
Bed rest less than half of the day | 24 (53.3%) | 14 (26.9%) | 38 (39.2%) | |
Bed rest more than half of the day | 5 (11.1%) | 4 (7.7%) | 9 (9.3%) | |
FACT-G | ||||
Physical wellbeing | 17.3 ± 6.9 | 19.0 ± 7.6 | 0.244 | 18.2 ± 7.3 |
Functional wellbeing | 15.2 ± 7.0 | 17.4 ± 6.9 | 0.134 | 16.4 ± 7.0 |
Social wellbeing | 20.8 ± 5.8 | 22.4 ± 5.1 | 0.153 | 21.7 ± 5.4 |
Emotional wellbeing | 18.6 ± 5.2 | 19.3 ± 4.6 | 0.491 | 19.0 ± 4.9 |
FACT-G total | 72.0 ± 19.3 | 78.1 ± 17.6 | 0.105 | 75.2 ± 18.6 |
Notes: Entries in table represent number of patients (percentage), unless otherwise specified.
low literacy: below seventh-grade reading level; high literacy: seventh-grade level and above.34
P value for statistical tests to compare low- and high-literacy groups.
Patient literacy scores, preferences, and experiences
There were moderate correlations between literacy scores and years of education (rs, 0.49–0.62), and between the literacy scores on the three tests (rs, 0.57–0.67). The proportions of patients classified as low- or high-literacy varied considerably across the literacy tests (Table 2). McNemar and Kappa (κ) statistics showed poor agreement between literacy tests (McNemar P < 0.001 and κ, 0.12–0.37 for all three pairwise tests). Similarly, there was poor agreement between dichotomous education and the literacy tests (κ, 0.003–0.26 for all three pairwise tests). For example, only two patients had educational attainment below the seventh-grade level, yet 45 demonstrated reading comprehension below the seventh grade.
Table 2.
Low literacy | High literacy | |
---|---|---|
Reading recognition (REALM)33 | 14 (14.4%) | 83 (85.6%) |
Reading comprehension (WLPB-PC)34 | 45 (46.4%) | 52 (53.6%) |
Functional health literacy (S-TOFHLA)7,35–36 | 5 (5.2%) | 92 (94.8%) |
Notes:
REALM and WLPB-PC: <7th grade (low literacy); ≥7th grade (high literacy), S-TOFHLA: inadequate/marginal (low literacy); adequate (high literacy). Entries in Table represent number of patients (percentage).
Abbreviations: REALM, Rapid Estimate of Adult Literacy in Medicine; S-TOFHLA, Short Test of Functional Health Literacy in Adults, abbreviated version; WLPB-PC, Woodcock Language Proficiency Battery-Passage Comprehension subtest.
Patients also provided self-assessments of their literacy skills. Among those with lower WLPB-PC reading comprehension, 60% reported that they have trouble reading printed health information some or all of the time, and 31% reported having trouble reading everyday things like the newspaper. Patients with higher reading comprehension also sometimes have trouble reading health information (29%) or everyday things (10%). Similar trends were observed using the REALM or S-TOFHLA. Among the 19 patients who reported having trouble reading everyday things, most (79%) said they felt ashamed (a little bit or somewhat) about having trouble reading; this did not differ across literacy levels.
The majority of patients said that the REALM reading recognition test was the easiest (81%), and the WLPB-PC reading comprehension test was the hardest (69%). There tended to be more variability in the low-literacy group, eg, among those with a low WLPB-PC score, 60% said this test was the hardest, whereas 78% of those with a high WLPB-PC score said it was the hardest (P = 0.007). Results were similar using the REALM or S-TOFHLA literacy tests.
Most patients (67%) said the health literacy (S-TOFHLA) assessment was the most interesting, compared with the reading recognition (REALM, 5%) and reading comprehension tests (WLPB-PC, 28%). Over half (52%) of the patients said the reading comprehension test (WLPB-PC) was the most boring, compared with the reading recognition (REALM, 36%) and health literacy tests (S-TOFHLA, 11%). If asked to do a literacy assessment again, 49% would prefer the S-TOFHLA, 40% would prefer the REALM, and 11% would prefer the WLPB-PC. These results did not differ by literacy levels.
Overall, low-literacy patients were more likely to report that literacy screening made them feel anxious or stressed compared with high-literacy patients. For example, 20% with low reading comprehension felt anxious and 24% said it was stressful, whereas the proportions among those with high reading comprehension were 6% and zero, respectively. However, nearly all patients (96%) agreed that “It is important for doctors and nurses to know about their patients’ reading abilities.” Most (84%) would be willing to have literacy assessment results given to their doctors/nurses. These results did not differ by literacy levels.
Comprehension of informed consent
Comprehension varied across issues on a sample consent form and was nearly always higher for high-literacy patients (Table 3). Most patients correctly answered the item about benefit to others (“What is learned from the treatment study will help future patients”). Very few patients correctly answered the item about the reason off-study (“The only reason you will be taken off the study, or have your treatment changed, is if the drugs make you so sick that it’s unbearable”). The range of correct mean responses for the subscales was 8% to 100% for low reading comprehension, and 13% to 98% for high reading comprehension, and many of the differences between literacy groups were statistically significant at P < 0.05. Effect sizes for many differences were moderate (approximately 0.5). Results were similar using the REALM or S-TOFHLA literacy levels. Many patients favorably rated the clarity of the consent form. Specifically, 40% reported that the descriptions of the treatments were very clear, 36% reported that the form was very easy to read, and 36% reported that the form was very easy to understand. Patients with high WLPB-PC reading comprehension tended to rate the clarity of the form more favorably than those with low reading comprehension, eg, 48% and 23%, respectively, rated it as very easy to understand (P = 0.016). When asked to describe what made the consent form easy or hard to understand, patients commented on the format (“It’s broken into sections; it includes the questions and answers a patient might have”) and the wording (“Uses simple words, no medical terminology” and “It’s written at a low reading level, which makes it easier for me to understand”). There were only a few negative comments and these were mostly focused on the description of the treatment groups (“The explanation it gives about what treatment you will be receiving is confusing.” “The section that explains the differences between the four treatment groups was a little confusing; I had to read it twice”).
Table 3.
Comprehension subscale and items (correct response) | Low literacy (n = 40)b | High literacy (n = 52) | Effect sizec | Pd value |
---|---|---|---|---|
Mean % correct | ||||
Privacy (two items)e | 35.0 | 47.0 | 0.46 | 0.030 |
Only staff from the hospital or clinic can look in your medical records for information about your treatment. (F) | 5.0 | 15.4 | ||
Published information about this study may include your name. (F) | 65.0 | 78.8 | ||
Side effects (three items)e | 88.3 | 95.0 | 0.37 | 0.060 |
The drugs that are part of the study may cause your hair to fall out. (T) | 92.5 | 96.2 | ||
The drugs in the study will have no effect on your level of energy. (F) | 85.0 | 94.2 | ||
The side effects from the drugs will affect all patients in the same way. (F) | 87.5 | 94.2 | ||
Personal benefit (six items)e | 65.8 | 71.8 | 0.29 | 0.095 |
The consent form stated that the study is being done to find the treatment that will most benefit you. (F) | 25.0 | 27.5 | ||
According to the consent form the purpose of the study is to see which of the treatments being tested is better at slowing the growth of your disease. (T) | 92.5 | 96.2 | ||
According to the consent form the reason the study is being conducted is to find the treatment that will cure your disease. (F) | 37.5 | 61.5 | ||
It is unlikely that the drugs that are part of the study will cure your disease. (T) | 62.5 | 65.4 | ||
The drugs that are part of the study might not be better for treating your disease than other drugs that are available. (T) | 92.5 | 86.5 | ||
Your disease will be cured as a result of the treatment study. (F) | 85.0 | 94.2 | ||
Benefit to others (one item)e | 100.0 | 98.0 | −0.20 | 0.395 |
What is learned from the treatment study will help future patients. (T) | ||||
Choice (four items)e | 79.5 | 87.5 | 0.41 | 0.012 |
It’s up to you to decide if your will be part of the treatment study. (T) | 97.5 | 98.1 | ||
If you agree to be part of the study you must stay on the treatment until the study is over. (F) | 40.0 | 73.1 | ||
There are no treatment options for your disease other than this study. (F) | 80.0 | 84.6 | ||
You could choose not to have any treatment at all. (T) | 100.0 | 94.2 | ||
Financial (two items)e | 50.0 | 68.5 | 0.49 | 0.027 |
You may be responsible for some of the treatment costs not covered by your insurance or the study. (T) | 62.5 | 76.9 | ||
You will be paid for taking part in the treatment study. (F) | 37.5 | 59.6 | ||
Randomization (one item)f | 45.0 | 69.0 | 0.48 | 0.022 |
If you agree to be part of the treatment study, how is it decided what treatment group you are in? (3: Chosen by chance) | ||||
Reason off-study (one item)e | 8.0 | 13.0 | 0.16 | 0.371 |
The only reason you will be taken off the study, or have your treatment changed, is if the drugs make you so sick that it’s unbearable. (F) | ||||
Total (20 items) | 65.0 | 73.9 | 0.65 | <0.001 |
Notes:
Low literacy: below seventh-grade reading level; high literacy: seventh-grade level and above 34;
Data were missing for five low-literacy patients who did not complete the comprehension interview;
Effect size: mean difference between groups (high literacy minus low literacy), divided by the common standard deviation;
P value for Wilcoxon rank-sum test to compare comprehension scores between low- and high-literacy groups;
Item response scale: T = true, F = false;
Item response scale: 1 = you decide on the treatment group, 2 = your doctor decides on the treatment group, 3 = the treatment group is chosen by chance.
Discussion
Despite its prevalence and negative impact on health outcomes, low literacy remains to be routinely addressed in clinical practice or research. Appropriate screening tools are clearly needed, given that patients do not tend to report their literacy levels spontaneously and accurately, many providers and researchers are hesitant to inquire about literacy, and educational level has proven to be an inadequate proxy for literacy. A crucial first step to addressing better literacy in health care is to identify reliable and valid measures that are deemed acceptable for routine use. Existing measures assess different aspects of literacy (with reading comprehension tests favored) and have demonstrated acceptable or favorable psychometric properties. However, no gold standard has been established. Furthermore, better understanding of the relationships between available literacy measures and relevant comprehension tasks, as well as patient preferences and attitudes regarding literacy screening, is necessary prior to recommending a standardized means of screening for inadequate literacy in health care settings.
We evaluated three widely used literacy measures and found patient scores on them to be only moderately correlated. Likewise, classification of patients as low- or high-literacy varied widely and showed poor agreement across the tests. These results substantiate that the WLPB-PC, REALM and S-TOFHLA assess different components of literacy, but do not identify which one is best suited as a screening tool. For any literacy measure to be recommended above others, both its relationships with literacy-related tasks and patient feedback need to be considered.
Patients’ scores on all three literacy measures demonstrated poor agreement with their educational levels, further substantiating that education is not a suitable proxy for literacy. However, scores on all three literacy tests did correspond with patient self-assessments of their literacy skills, with those categorized as low-literacy tending to acknowledge reading difficulties more often than those categorized as high-literacy. Likewise, for all three measures, patients categorized as low-literacy scored significantly lower on the comprehension tasks and rated the clarity of the consent form less favorably than did high literacy patients. However, of note, a considerable proportion of patients categorized as high-literacy also misunderstood certain comprehension tasks and reported health-related reading difficulties. Also of interest, a majority of patients reporting reading difficulties acknowledged shame about this.
The fact that patients across the literacy spectrum sometimes have trouble reading printed health information suggests that the reading level of such information may need to be decreased, and the format clarified, for all patients. Such an approach is in line with universal precaution recommendations such as using easy-to-read consent forms as a way of decreasing consent anxiety,37 and evaluating literacy for all patients as standard practice. The aspects of the easy-to-read consent form (eg, avoiding medical jargon, breaking text into small sections) that patients in this study indicated improved its understandability coincide with recommendations in the literature on making written health care materials accessible to patients with low literacy.40–42 Our results also indicate that certain content in consent forms (eg, benefits of the study) are easier to understand than others (eg, reasons to be taken off study). Given that the understandability of particular consent forms and other written health care materials will differ based on their content, wording and formatting, we recommend that their usability and acceptability be pretested with patients. It is also useful for clinicians and researchers to have a sense of the literacy levels of their patient populations before developing written materials.
Before literacy screening can be instituted in clinical and research settings, more information is needed about patient attitudes and preferences regarding specific measures and the assessment process. The majority of patients in our study found the REALM to be the easiest, and the WLPB-PC to be the most difficult and most boring of the three measures. Most patients demonstrated a preference for the S-TOFHLA, endorsing it as the most interesting of the three and the one they would prefer to take again. Thus, our results indicate that patients may prefer a literacy test with health-related content and that is perceived as neither particularly easy nor difficult.
In terms of feedback regarding the screening process, patients with low-literacy were more likely than high-literacy patients to report that the tests were stressful and anxiety-provoking. However, those reporting test-related anxiety or stress were still a minority of the patients categorized as low-literacy. Moreover, the vast majority of participants, regardless of literacy level, endorsed clinicians knowing the reading ability of their patients as being important, and indicated that they would be willing for their literacy assessment results to be given to the clinicians treating them. Taken together, these findings suggest that literacy screening is deemed acceptable by patients. This is in sharp contrast with the prevailing notion that literacy is too sensitive or unwelcome a topic to be addressed routinely in health care.
This study is not without limitations. We used a convenience, not a representative, sample of patients with cancer and results may not be generalizable. More research is needed with diverse patient populations to establish the level of agreement between widely used literacy tests and their ability to predict health-related comprehension tasks, and to understand better patient preferences regarding literacy screening.
Accurate detection of low literacy in clinical settings must take place before interventions or programs can be implemented to reduce literacy-related health disparities. The literacy levels of clinical and study populations must be known in order to establish whether written materials are at appropriate reading levels. Valid and appropriate measures are needed if literacy level is to become another health care vital sign. Existing tests appear to assess different aspects of literacy. More research is needed to examine how different literacy measures correspond with relevant comprehension tasks and health outcomes, and how well they discriminate between individuals across the literacy spectrum. Also, as demonstrated in our study, patient input can serve as a valuable tool when evaluating literacy measures and written health care materials.
Acknowledgments
The Coleman Foundation funded this research which was presented in part at the International Conference on Communication in Healthcare, Chicago, Illinois in October 2005. The authors thank Dr Peter Raich for providing the ECOG “easy to read” consent form, comprehension questions, and study protocol, and Patricia Diaz, Kimberly Rosenfeld, and Veronica Valenzuela for recruiting and interviewing patients. We also thank all of the patients who participated in this study.
Footnotes
Disclosures
The authors report no conflict of interest in this work.
References
- 1.Kirsch I, Jungeblut A, Jenkins L, Kolstad A. Adult literacy in America: A First Look at the Results of the National Adult Literacy Survey. Washington, DC: National Center for Education Statistics, US Department of Education; 1993. [Google Scholar]
- 2.Ad Hoc Committee on Health Literacy for the Council on Scientific Affairs, American Medical Association Health literacy: Report of the Council on Scientific Affairs. JAMA. 1999;281(6):552–557. [PubMed] [Google Scholar]
- 3.Baker DW, Parker RM, Williams MV, et al. The health care experience of patients with low literacy. Arch Fam Med. 1996;5(6):329–334. doi: 10.1001/archfami.5.6.329. [DOI] [PubMed] [Google Scholar]
- 4.Parikh NS, Parker RM, Nurss JR, Baker DW, Williams MV. Shame and health literacy: The unspoken connection. Patient Educ Couns. 1996;27(1):33–39. doi: 10.1016/0738-3991(95)00787-3. [DOI] [PubMed] [Google Scholar]
- 5.Kleinbaum DG, Kupper LL, Morgenstern H. Epidemiologic Research: Principles and Quantitative Methods. Belmont, CA: Lifetime Learning Publications; 1982. [Google Scholar]
- 6.National Work Group on Literacy and Health Communicating with patients who have limited literacy skills. Report of the National Work Group on Literacy and Health. J Fam Pract. 1998;46(2):168–176. [PubMed] [Google Scholar]
- 7.Williams MV, Parker RM, Baker DW, et al. Inadequate functional health literacy among patients at two public hospitals. JAMA. 1995;274(21):1677–1682. [PubMed] [Google Scholar]
- 8.Doak CC, Doak LG, Friedell GH, Meade CD. Improving comprehension for cancer patients with low literacy skills: Strategies for clinicians. CA Cancer J Clin. 1998;48(3):151–162. doi: 10.3322/canjclin.48.3.151. [DOI] [PubMed] [Google Scholar]
- 9.Davis TC, Williams MV, Marin E, Parker RM, Glass J. Health literacy and cancer communication. CA Cancer J Clin. 2002;52(3):134–149. doi: 10.3322/canjclin.52.3.134. [DOI] [PubMed] [Google Scholar]
- 10.Baker DW, Parker RM, Williams MV, Clark WS, Nurss J. The relationship of patient reading ability to self-reported health and use of health services. Am J Public Health. 1997;87(6):1027–1030. doi: 10.2105/ajph.87.6.1027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Bennett CL, Ferreira MR, Davis TC, et al. Relation between literacy, race, and stage of presentation among low-income patients with prostate cancer. J Clin Oncol. 1998;16(9):3101–3104. doi: 10.1200/JCO.1998.16.9.3101. [DOI] [PubMed] [Google Scholar]
- 12.Kalichman SC, Rompa D. Functional health literacy is associated with health status and health-related knowledge in people living with HIV-AIDS. J Acquir Immune Defic Syndr. 2000;25(4):337–344. doi: 10.1097/00042560-200012010-00007. [DOI] [PubMed] [Google Scholar]
- 13.Baker DW, Gazmararian JA, Williams MV, et al. Functional health literacy and the risk of hospital admission among Medicare managed care enrollees. Am J Public Health. 2002;92(8):1278–1283. doi: 10.2105/ajph.92.8.1278. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Schillinger D, Grumbach K, Piette J, et al. Association of health literacy with diabetes outcomes. JAMA. 2002;288(4):475–482. doi: 10.1001/jama.288.4.475. [DOI] [PubMed] [Google Scholar]
- 15.Parker RM, Ratzan SC, Lurie N. Health literacy: A policy challenge for advancing high-quality health care. Health Affairs (Millwood) 2003;22(4):147–153. doi: 10.1377/hlthaff.22.4.147. [DOI] [PubMed] [Google Scholar]
- 16.Berkman ND, DeWalt DA, Pignone MP, et al. Literacy and Health Outcomes. Evidence Report/Technology Assessment No. 87. Rockville, MD: Agency for Healthcare Research and Quality; 2004. AHRQ Publication No. 04-E007-2. [Google Scholar]
- 17.Committee on Health Literacy. Nielsen-Bohlman L, Panzer AM, Kindig DA. Health Literacy: A Prescription to End Confusion. Washington, DC: The National Academies Press; 2004. [PubMed] [Google Scholar]
- 18.DeWalt DA, Berkman ND, Sheridan S, Lohr KN, Pignone MP. Literacy and health outcomes: A systematic review of the literature. J Gen Intern Med. 2004;19(12):1228–1239. doi: 10.1111/j.1525-1497.2004.40153.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Weiss BD, Hart G, McGee DL, D’Estelle S. Health status of illiterate adults: Relation between literacy and health status among persons with low literacy skills. J Am Board Fam Pract. 1992;5(3):257–264. [PubMed] [Google Scholar]
- 20.Gazmararian JA, Baker DW, Williams MV, et al. Health literacy among Medicare enrollees in a managed care organization. JAMA. 1999;281(6):545–551. doi: 10.1001/jama.281.6.545. [DOI] [PubMed] [Google Scholar]
- 21.Wolf MS, Gazmararian JA, Baker DW. Health literacy and functional health status among older adults. Arch Intern Med. 2005;165(17):1946–1952. doi: 10.1001/archinte.165.17.1946. [DOI] [PubMed] [Google Scholar]
- 22.Hahn EA, Cella D, Dobrez DG, et al. The impact of literacy on health-related quality of life measurement and outcomes in cancer outpatients. Qual Life Res. 2007;16(3):495–507. doi: 10.1007/s11136-006-9128-6. [DOI] [PubMed] [Google Scholar]
- 23.Cella DF, Orav EJ, Kornblith AB, et al. Socioeconomic status and cancer survival. J. Clin Oncol. 1991;9(8):1500–1509. doi: 10.1200/JCO.1991.9.8.1500. [DOI] [PubMed] [Google Scholar]
- 24.Powers RD. Emergency department patient literacy and the readability of patient- directed materials. Ann Emerg Med. 1988;17(2):124–126. doi: 10.1016/s0196-0644(88)80295-6. [DOI] [PubMed] [Google Scholar]
- 25.Weiss BD, Reed RL, Kligman EW. Literacy skills and communication methods of low-income older persons. Patient Educ Couns. 1995;25(2):109–119. doi: 10.1016/0738-3991(95)00710-h. [DOI] [PubMed] [Google Scholar]
- 26.Jackson RH, Davis TC, Bairnsfather LE, George RB, Crouch MA, Gault H. Patient reading ability: An overlooked problem in health care. South Med J. 1991;84(10):1172–1175. doi: 10.1097/00007611-199110000-00004. [DOI] [PubMed] [Google Scholar]
- 27.Anastasi A, Urbina S. Psychological Testing. Upper Saddle River, NJ: Prentice Hall; 1997. [Google Scholar]
- 28.Oken MM, Creech RH, Tormey DC, et al. Toxicity and response criteria of the Eastern Cooperative Oncology Group. Am J Clin Oncol. 1982;5:649–655. [PubMed] [Google Scholar]
- 29.Cella DF, Tulsky DS, Gray G, et al. The Functional Assessment of Cancer Therapy scale: Development and validation of the general measure. J Clin Oncol. 1993;11(3):570–579. doi: 10.1200/JCO.1993.11.3.570. [DOI] [PubMed] [Google Scholar]
- 30.Cella DF, Bonomi AE, Spilker B. The Functional Assessment of Cancer Therapy (FACT) and Functional Assessment of HIV Infection (FAHI) quality of life measurement system. In: Spilker B, editor. Quality of Life Pharmacoeconomics in Clinical Trials. 2nd ed. Philadelphia, PA: Lippincott-Raven; 1996. [Google Scholar]
- 31.Cella D. Manual of the Functional Assessment of Chronic Illness Therapy (FACIT Scales). Version 4. Available at: http://facit.org/. Accessed March 09, 2010. [DOI] [PubMed]
- 32.Hahn EA, Rao D, Cella D, Choi SW. Comparability of interview- and self-administration of the Functional Assessment of Cancer Therapy-General (FACT-G) in English- and Spanish-speaking ambulatory cancer patients. Med Care. 2008;46(4):423–431. doi: 10.1097/MLR.0b013e3181648e6e. [DOI] [PubMed] [Google Scholar]
- 33.Davis TC, Long SW, Jackson RH, et al. Rapid estimate of adult literacy in medicine: A shortened screening instrument. Fam Med. 1993;25(6):391–395. [PubMed] [Google Scholar]
- 34.Woodcock RW. Examiner’s Manual: Woodcock Language Proficiency Battery-Revised. Allen, TX: DLM Publisher; 1991. [Google Scholar]
- 35.Parker RM, Baker DW, Williams MV, Nurss JR. The test of functional health literacy in adults: A new instrument for measuring patients’ literacy skills. J Gen Intern Med. 1995;10(10):537–541. doi: 10.1007/BF02640361. [DOI] [PubMed] [Google Scholar]
- 36.Baker DW, Williams MV, Parker RM, Gazmararian JA, Nurss J. Development of a brief test to measure functional health literacy. Patient Educ Couns. 1999;38(1):33–42. doi: 10.1016/s0738-3991(98)00116-5. [DOI] [PubMed] [Google Scholar]
- 37.Coyne CA, Xu R, Raich P, et al. Randomized, controlled trial of an easy-to-read informed consent statement for clinical trial participation: A study of the Eastern Cooperative Oncology Group. J Clin Oncol. 2003;21(5):836–842. doi: 10.1200/JCO.2003.07.022. [DOI] [PubMed] [Google Scholar]
- 38.Cohen J. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: L. Erlbaum Associates; 1988. [Google Scholar]
- 39.Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174. [PubMed] [Google Scholar]
- 40.Plain Language Action and Information Network (PLAIN) Improving Communication from the Federal Government to the Public. Available at: http://www.plainlanguage.gov. Accessed September 21, 2009.
- 41.Centers for Disease Control and Prevention Scientific and Technical Information: Simply Put. Available at: http://www.cdc.gov/od/oc/simp-put.pdf. Accessed September 21, 2009.
- 42.Doak CC, Doak LG, Root JH. Teaching Patients with Low Literacy Skills. Philadelphia, PA: J.B. Lippincott; 1996. [Google Scholar]