Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Feb 12.
Published in final edited form as: J Health Commun. 2014;19(0 2):267–284. doi: 10.1080/10810730.2014.936569

Development and Validation of the Assessment of Health Literacy in Breast and Cervical Cancer Screening

Hae-Ra Han 1, Boyun Huh 2, Miyong T Kim 3, Jiyun Kim 4, Tam Nguyen 5
PMCID: PMC4751992  NIHMSID: NIHMS737096  PMID: 25315598

Abstract

For many people limited health literacy is a major barrier to effective preventive health behavior such as cancer screening, yet a comprehensive health literacy measure that is specific to breast and cervical cancer screening is not readily available. The purpose of this paper is to describe the development and testing of a new instrument to measure health literacy in the context of breast and cervical cancer screening, the Assessment of Health Literacy in Cancer Screening (AHL-C). The AHL-C is based on Baker’s conceptualization of health literacy and modeled from the two most popular health literacy tests, the Rapid Estimate of Adult Literacy in Medicine and the Test of Functional Health Literacy in Adults. The AHL-C consists of four subscales; print literacy, numeracy, comprehension, and familiarity. We used baseline data from 560 Korean American immigrant women who participated in a community-based randomized trial designed to test the effect of a health literacy-focused intervention to promote breast and cervical cancer screening. Rigorous psychometric testing supports that the AHL-C is reliable, valid, and significantly correlated with theoretically selected variables. Future research is needed to test the utility of the AHL-C in predicting cancer screening outcomes.

Keywords: health literacy, instrument, cancer, screening

Over the last decade, health literacy research has gained increasing attention because it is a significant determinant of health outcomes. Defined as “the degree to which individuals have the capacity to obtain, process, and understand basic health information and services to make appropriate health decisions (U.S. Department of Health and Human Services, 2000),” health literacy has been associated with better compliance with healthcare provider recommendations, health appointments, and adoption of preventive care such as cancer screening (Bennett et al., 1998; Lindau, Tomori, McCarville, & Bennett, 2001; Weiss, Hart, McGee, & D'Estelle, 1992). Those with a lower level of education and income, non-English-speaking immigrants, and the elderly are disproportionately affected by low health literacy (Institute of Medicine, 2004). Low health literacy has been found to be a strong predictor of inadequate utilization of health care resources and poor health outcomes in vulnerable populations, especially among non-English-speaking immigrants (Institute of Medicine, 2004; Weinick, Zuvekas, & Cohen, 2000).

Several health literacy measures in current use have yielded valuable insights. In particular, the Rapid Estimate of Adult Literacy in Medicine (REALM) (Davis et al., 1993) and the Test of Functional Health Literacy in Adults (TOFHLA) (Parker, Baker, Williams, & Nurss, 1995) are the two most popular tests to assess health literacy in general populations. Nevertheless, they fail to address the particular health context in which health literacy needs to be screened. For example, REALM features general medical words and has limited utility in assessing health literacy in a specific context such as cancer or diabetes care. Likewise, the TOFHLA which measures the ability to apply information in the health care environment includes non-specific/general items and passages that are unique to the U.S. health care system (i.e., Medicaid application questions). As such, it cannot assess health literacy in a specific context. Moreover, some of the passages on the TOFHLA may cause difficulty for immigrants because the content is not relevant in countries outside the U.S. (Han, Kim, Kim, & Kim, 2011).

The characteristics of the health care settings in which individuals navigate and interact to obtain necessary information and care can vary depending on the conditions for which they seek care (e.g., cancer screening versus diabetes care). A context-specific health literacy assessment may be more useful when it is applied to individuals who are in need of obtaining further education and care for a certain condition (Institute of Medicine, 2004). The context-specific assessment can also function as an effective evaluation tool for targeted disease-specific interventions. Assessing health literacy within a specific context has been a growing trend. For example, the following context specific health literacy tools have been developed: the Literacy Assessment for Diabetes (Nath, Sylvester, Yasek, & Gunel, 2001), Diabetes Numeracy Test (Huizinga et al., 2008), Asthma Numeracy Questionnaire (Apter et al., 2006), Rapid Estimate of Adult Literacy in Genetics (REAL-G) (Erby, Roter, Larson, & Cho, 2008), High Blood Pressure-Focused Health Literacy Scale (Kim et al., 2012), and HIV-Related Health Literacy Scale (HIV-HL) (Ownby et al., 2013).

Several health literacy instruments are available in the context of cancer. Specifically, they include the Assessment of Colon Cancer Literacy (Pendlimari, Holubar, Hassinger, & Cima, 2012), the Cancer Literacy Score (Diviani & Schulz, 2012), and Cancer Literacy Assessment Tools (Williams, Reckase, & Rivera-Vasquez, 2008). The first two cancer specific instruments assess colon cancer-related health literacy (Pendlimari et al., 2012) and knowledge about general cancer risk, diagnosis, and treatment as well as coping with cancer (Diviani & Schulz, 2012), respectively. The Cancer Literacy Assessment Tools (Williams et al., 2008) measure women’s understanding of their personal and familial risk of breast and cervical cancers. The Cancer Literacy Assessment Tools were pilot-tested in 16 community health workers and their respective supervisors with a Cronbach’s alpha of 0.91. The developers then revised the original version to develop the Breast Cancer Literacy Assessment Tool (B-CLAT) (Williams, Templin, & Hines, 2013) and the Cervical Cancer Literacy Assessment Tool (C-CLAT) (Williams & Templin, 2013). Even though the Cancer Literacy Assessment Tools and the revised tests (B-CLAT and C-CLAT) are breast and cervical cancer-specific, the items included in the measures primarily focus on knowledge (e.g., different types of exams available to detect breast cancer) and fail to assess the comprehensive range of skills needed for health literacy (Institute of Medicine, 2004). Consequently, a comprehensive health literacy measure that is specific to breast and cervical cancer screening is not readily available.

Therefore, we sought to develop a comprehensive cancer screening-specific health literacy measure that assesses individual’s comprehension, familiarity, and recognition of words related to breast and cervical cancer screening. The development of this measure was firmly grounded in a theoretical framework, with the ultimate goal of creating a measure that is sensitive enough to capture the intervention effects of a health literacy-focused breast and cervical cancer control education and counseling program. This paper reports the development process and the psychometric properties of the breast and cervical cancer screening-specific health literacy instrument, the Assessment of Health Literacy in Cancer Screening (AHL-C), using a sample of first-generation Korean American women. In addition, preliminary evidence of its utility for evaluating cancer screening interventions as an outcome measure is presented.

Methods

We employed a two-phase instrumentation design process: instrument development and testing. The instrument development phase began with generating items for the AHL-C as described in the following section.

Development phase

Selection of items

We used multiple methods to create a comprehensive pool of test items for the AHL-C. First, we searched the literature for current practice guidelines on breast and cervical cancer screening (U.S. Preventive Services Taskforce, 2010). Relevant words were also collected from educational materials (e.g., pamphlets, brochures, websites) published by the Centers for Disease Prevention and Control, National Cancer Institute, and American Cancer Society. In addition, we used participant observation to explore common clinical encounters experienced by women when they go through breast and cervical cancer screening. In consultation with practicing clinicians, the study team selected nine settings where women seek care for breast and cervical cancer screening (e.g., primary care office, OB/GY clinic, breast center). A trained research assistant observed at least one encounter for each setting and completed field notes to inform the essential terminology needed to be understood by women when undergoing breast and cervical cancer screening. A panel of 10 people including Korean American women with previous breast and/or cervical cancer screening experiences, nurses, an OB/GYN, a nurse practitioner, and community health workers assessed the face validity and the appropriateness of the initial pool of items generated using the combined methods. Then, an expert panel (N=8) of clinicians and researchers with extensive experience in women’s breast and cervical cancer screening rated each item for its relevance using a 4-point scale, with 1 being “not relevant” and 4 being “very relevant (Lynn, 1986).” A content validity index was calculated by the proportion of experts who gave an item a rating of 3 or 4. Items with at least 80% of endorsement rates were retained (Lynn, 1986), yielding 47 words specific to breast and cervical cancer. A pilot sample of 12 Korean American women was then asked to pronounce the 47 words. Twelve of the most frequently missed words with a 0.5 or greater item-total correlation (Erby et al., 2008) were retained on the final list to maximize discrimination.

AHL-C structure

Building on the conceptual model of health literacy presented by the Institute of Medicine (2004), Baker (2006) suggests that there are two sub-domains of health literacy at the individual level. They include reading fluency and prior knowledge. Reading fluency refers to “the ability to mentally process written materials and form new knowledge (p. 898)” (Baker, 2006). Reading fluency includes a variety of skill sets such as the ability to read and understand written documents (print literacy) and to apply arithmetic operations and use numerical information in printed materials (numeracy). On the other hand, prior knowledge is defined as “an individual’s knowledge at the time before he/she reads health-related materials or speaks to a health care professional (p. 879),” and includes vocabulary (comprehension—knowing which words mean what) and conceptual knowledge (familiarity—understanding aspects of the world) (Baker, 2006).

The structure of the AHL-C was developed based on the Baker’s conceptualization and includes the following subscales: print literacy, numeracy, comprehension, and familiarity. The print literacy subscale was modeled on the two most popular global health literacy tests, the REALM and the TOFHLA (see Appendix A). First section of the print literacy subscale (Reading test) includes simple questions in a REALM-like instrument to test the individual's reading ability (12 items). The second section of the subscale requires examinees to complete 12 clause-type items (i.e. fill in the blank sentences) as in the case of the TOFHLA. The passages used for the second section of the print literacy subscale are based on the navigational trajectory required for cancer screening (e.g., check-in at OB/GYN clinic, communication between a woman and a doctor about breast self-examination and risk factors for breast and cervical cancers, appointment making for a mammogram test). The items on the print literacy subscale were scored as correct/incorrect, with total possible scores ranging from 0 to 24.

The numeracy subscale included four items. The scale was adapted from a popular numeracy assessment tool, the Newest Vital Sign (NVS) (Weiss et al., 2005). We modified the items on the NVS to make them more relevant to cancer prevention. For example, the original NVS asks questions about an ice cream nutrition label. In the AHL-C, we used a nutrition label for pizza since weight control and reduced calorie intake are important dietary behaviors for preventing cancer (see Appendix B). Total possible scores ranged from 0 to 4, with one point assigned for each correct response.

The comprehension subscale consisted of 12 items which require an individual to associate each term to a word or a picture of the meaning to demonstrate comprehension. As was the case for the print literacy and numeracy subscales, the items on the comprehension subscale were scored as correct/incorrect, with total possible scores ranging from 0 to 12.

Finally, the familiarity subscale asked participants their level of familiarity with the 12 items in the print literacy subscale on a 5-point Likert scale from 0 (not at all familiar—never heard of the word before) to 4 (very familiar—can use the word proficiently). For ease of understanding, scores on the familiarity subscale were recalibrated for each item on a 0–1 point scale so that total possible scores ranged from 0 to 12 (see Appendix C). The AHL-C was written at the 6th grade reading level.

Testing Phase

Sample and Design

Data were obtained from a sample of Korean American women who participated in a community-based intervention study—Better Breast and Cervical Cancer Control for Korean American Women—which was designed to promote breast and cervical cancer screening among non-adherent Korean women. The parent study is a cluster-randomized controlled trial in which trained lay health workers deliver health literacy-focused education and navigation assistance to Korean women in the study for 6 months. Eligibility was based on the following inclusion criteria: a) self-identified as a Korean American woman; b) aged 21–65 years; c) have no mammogram and/or Pap test within the last 24 months; d) able to read and write Korean or English; and e) willing to provide written informed consent to allow the researchers to audit medical records for mammography and/or Pap test use. Participants were recruited from ethnic churches located in the Baltimore-Washington Metropolitan Area. A total of 560 eligible Korean American women completed the study questionnaire at baseline. At 6 months follow-up, 527 women participated in the final data collection. We used the baseline data to test the psychometrics of the newly developed AHL-C measure and the 6-month follow-up data to evaluate the utility of the AHL-C as an intervention outcome assessment tool.

Procedures

After approval of the study protocol by the Institutional Review Board, potential participants were identified by trained lay health workers. Trained bilingual research staff approached these participants, briefly explained the study to them, verified eligibility, and asked those eligible to participate in the study. The research staff then obtained informed consent from each woman who agreed to participate, and administered study instruments. To assess the print literacy subscale, participants were given a laminated list of words and instructed to pronounce each word. Trained research staff scored answers, checking (+) for each item correctly pronounced and (−) for any word that was either not attempted or was mispronounced. If the participant took more than 5 seconds on a certain item, they were told to skip the item and proceed to the next word. The number of correctly pronounced words (+) was then counted to calculate a sum score. Similarly, to assess comprehension, participants were instructed to complete the clause test to apply their health literacy in the navigational trajectory required for cancer screening. To assess numeracy, research staff showed a laminated nutritional label to participants and recorded their responses to the four questions. Finally, participants selected their level of familiarity with the 12 items in the print literacy subscale. It took about 10–15 min to complete the AHL-C. Participants received $20 for their participation.

Measurements

A study questionnaire was developed to collect basic socio-demographic information such as age, education, marital status, employment status, English proficiency, and years of residence in the United States. Cancer-specific health literacy was measured using the new tool, the AHL-C. We also administered the standard REALM (Davis et al., 1993) as an additional outcome measure of health literacy. The REALM is one of the most popular validated instruments and has been used to assess the individual’s ability to pronounce 66 medically relevant English words (i.e., words recognition test). The REALM compares favorably with other health literacy assessments with correlation coefficients of 0.80–0.90 (Davis et al., 1993). The REALM was validated in Korean American women (Han, Kim, Kim, & Kim, 2011). The REALM yielded a reliability coefficient of 0.98 in the current sample of Korean women. Cancer knowledge was assessed using two validated tests: the Breast Cancer Knowledge (BCK) Test (McCance, Mooney, Smith, & Field, 1990) and the Cervical Cancer Knowledge (CCK) Test (Park, Chang, & Chung, 2005). The BCK Test consists of 18 true or false items (e.g., “Most breast cancer is associated with hereditary factors.”). Similarly, the CCK Test consists of 10 true or false items (e.g., “Heavy smokers have an increased risk of cervical cancer)”. The BCK and CCK tests have been validated in Korean women, with reliability coefficients of 0.80–0.89 (Park, Chang, & Chung, 2005). The BCK and CCK tests yielded respective internal consistency reliability coefficients of 0.81 and 0.74 in the study sample.

Analysis

Descriptive statistics were used to summarize participant’s socio-demographic characteristics and health literacy scores. Coefficients for internal consistency reliability of the AHL-C were estimated by KR-20 for Yes/No responses and Cronbach’s alpha for Likert-type responses. Internal consistency reliability coefficients of 0.70 or greater were considered acceptable (Nunnally, 1994). Item analysis was also performed. Item-total correlations greater than 0.15 were considered adequate (Nunnally, 1994). Pearson’s correlation coefficients were used to calculate the correlations between the AHL-C subscales and known covariates such as age, educational level, English proficiency, and cancer knowledge to demonstrate evidence of construct validity (Institute of Medicine, 2004; Lindau et al., 2002; Scott, Gazmararian, Williams, & Baker, 2002). Analyses of covariance (ANCOVAs) were then performed to compare the magnitude of within-group changes in the level of health literacy after 6-month intervention with baseline controlled. Statistical significance was determined at the level of p-value less than 0.05.

Results

Participant characteristics

Socio-demographic characteristics of the participants are presented in Table 1. The majority of the sample were middle-aged (mean age=46.0±8.6 years), married or partnered (85.7%), and were well educated with nearly two thirds (64.8%) having completed some college or more. Three out of five (60%) were employed full- or part-time and all were born in Korea. The mean length of stay in the US was 16.5 (±9.7) years with about 70% of women having been in the US for 10 years or more. While less than one-third of the participants felt “very comfortable” or “comfortable” about their household income level, only 37.9% of the sample reported having medical insurance and 34.5% reported having a primary care provider. As for English proficiency, most Korean American women in the study reported their English as being poor or fair, and less than one in four (23.4%) rated their English proficiency as fluent.

Table 1.

Sample characteristics (N=560)

Variables n (%) Mean ± SD
Age, years (range=21–65) 46.0 ± 8.6
  20–30s 145 (25.9)
  40s 199 (35.5)
  50–60s 216 (38.6)
Marital status
  Married/Partnered 480 (85.7)
  Separated/Widowed/Divorced 61 (10.9)
  Never married 19 (3.4)
Years of education (range=4–24) 14.6 ± 2.7
  ≤High school graduate 197 (35.2)
  Some college+ 363 (64.8)
Employment
  Working full- or part-time 336 (60.0)
  Unemployed or retired 224 (40.0)
Length of stay in the US, years (range=0.1–62.3) 16.5 ± 9.7
  <10 yrs 166 (29.6)
  10–19 yrs 193 (34.5)
  20–29 yrs 140 (25.0)
  30+ yrs 61 (10.9)
Income level
  Very comfortable/Comfortable 150 (26.8)
  Just OK 193 (34.5)
  Uncomfortable/Very uncomfortable 217 (38.7)
Have health insurance 212 (37.9)
Have primary care physician 193 (34.5)
English proficiency (range=1–4) 2.7 ± 0.9
  Not at all/Poor 227 (40.5)
  Fair 202 (36.1)
  Fluent 131 (23.4)
Open in a new tab

Descriptive psychometrics of the AHL-C

Table 2 shows the means, standard deviation, and reliability coefficients of the AHL-C scales. Internal consistency reliability coefficients ranged from 0.70 for the Numeracy scale to 0.96 for the familiarity scale with a reliability coefficient of 0.96 for the total scale, indicating excellent internal consistency reliability. The item-total correlations for all items on the AHL-C scale also met the a priori cut-off score, ranging from 0.18 to 0.86.

Table 2.

Reliability testing of the AHL-C (N=560)

Mean ± SD Range Reliability
coefficient		 Item-total
correlation
Reading fluency1 14.59 ± 7.78 0–28 0.93 0.25–0.69
  Reading test 5.38 ± 3.83 0–12 0.89 0.50–0.68
  Clause test 7.06 ± 4.06 0–12 0.92 0.47–0.76
  Numeracy 2.14 ± 1.36 0–4 0.70 0.49–0.55
Prior knowledge2 5.27 ± 5.92 0–24 0.95 0.41–0.77
  Comprehension 3.24 ± 3.68 0–12 0.91 0.48–0.70
  Familiarity 2.04 ± 2.86 0–12 0.96 0.63–0.86
AHL-C total3 19.87 ± 12.52 0–52 0.96 0.18–0.69
Open in a new tab
1

Sum of Reading test, Clause test, and Numeracy.

2

Sum of Comprehension and Familiarity.

3

Sum of Reading fluency and Prior knowledge.

Validity testing

We tested the construct validity of the AHL-C by correlating its total scores with theoretically relevant variables including age, education, English proficiency, and breast and cervical cancer knowledge. Table 3 presents correlation coefficients between the theoretically relevant variables and AHL-C scales. The AHL-C scales showed statistically significant negative correlations with age and positive correlations with education and English proficiency (p<0.05 for all correlation coefficients). Health literacy as measured by the AHL-C was also significantly positively correlated with both breast (r=0.11–0.26, p<0.05) and cervical cancer knowledge (r=0.11–0.24, p<0.05). Finally, the reading fluency subscale was positively correlated with the prior knowledge subscale as well as the total AHL-C (r=0.66 and 0.93, respectively; p<0.05 for both), indicating convergent validity.

Table 3.

Correlation coefficients for selected variables and AHL-C subscales (N=560)

1 2 3 4 5 6 7 8 9 10 11 12 13
1. Age --
2. Education −.31** --
3. English proficiency −.19** .33** --
4. Breast cancer knowledge −.04 .20** .08 --
5. Cervical cancer knowledge −.09* .25** .09* .62** --
6. Reading fluency −.34** .49** .62** .19** .20** --
7. Reading test −.27** .45** .55** .14** .17** .88** --
8. Clause test −.31** .43** .59** .20** .19** .90** .64** --
9. Numeracy −.22** .25** .27** .11* .11* .54** .33** .39** --
10. Prior knowledge −.23** .35** .49** .26** .24** .66** .58** .63** .26** --
11. Comprehension −.26** .34** .44** .26** .22** .66** .57** .65** .26** .93** --
12. Familiarity −.14** .27** .44** .21** .22** .52** .47** .47** .21** .88** .64** --
13. AHL-C total −.32** .47* .62** .24** .24** .93** .82** .86** .46** .88** .85** .74** --
Open in a new tab
*

p<0.05;

**

p<0.01

Evaluation of the AHL-C as an outcome measure

Despite randomization, differences existed in the following health literacy subscale scores at baseline, all favoring the control group except for Numeracy: Clause Test, Numeracy, and Familiarity. Table 4 compares adjusted health literacy change scores between the intervention and control groups. As shown in Table 4, after controlling for baseline values of the health literacy outcome, the intervention group had significantly improved health literacy scores across all AHL-C subscales and the total scale as compared to the control group. However, no group difference emerged at 6 months when health literacy was measured by the REALM (p=0.179).

Table 4.

Adjusted health literacy change scores at 6 months (N=527)

Health literacy Intervention
(n=261)
mean (SD)		 Control
(n=266)
mean (SD)		 pb
Reading fluency 5.39 (7.21) 2.11 (4.65) <.001
  Reading test 2.69 (3.97) 1.59 (2.93) <.001
  Clause testa 2.59 (4.01) 0.52 (2.47) <.001
  Numeracya 0.11 (1.66) 0.00 (1.53) .026
Prior knowledge 5.55 (4.49) 2.14 (3.04) <.001
  Comprehension 0.96 (0.87) 0.63 (0.86) <.001
  Familiaritya 4.58 (4.22) 1.51 (2.78) <.001
AHL-C total 10.97 (10.44) 4.25 (6.27) <.001
REALM total 3.21 (16.77) 4.77 (11.18) .295
Open in a new tab
a

Indicates subscales for which there were baseline differences

b

Between-group difference in change scores (calculated by scores at 6 months minus scores at baseline) after controlling for baseline values

Discussion

Data from this developmental study suggests that AHL-C tool is a reliable and valid measure of health literacy in the context of breast and cervical cancer screening. Results show that the AHL-C was significantly correlated with both breast and cervical cancer knowledge and known covariates of health literacy such as age, education, and English proficiency. All the item-total correlations as well as internal consistency reliability coefficients were also above the acceptable range (Nunnally, 1994). Finding ideal ways of measuring health literacy is critical to address health disparity gaps observed in racial/ethnic minorities, particularly those with limited health literacy (Innos & Horn-Ross, 2003; Institute of Medicine, 2004; Liu, Zhang, Deapen, Bernstein, & Ross, 2003; Liu, Zhang, Wu, Pike, & Deapen, 2012). This is particularly important given the wide range of adverse health outcomes resulting from limited health literacy (Bennett et al., 1998; Institute of Medicine, 2004; Lindau et al., 2001; Weiss et al., 1992). Our findings suggest the AHL-C has sound psychometric properties and is a comprehensive measure of health literacy in breast and cervical cancer screening.

Health literacy encompasses multifaceted skill sets such as print literacy, numeracy, comprehension, and conceptual knowledge (or familiarity) (Baker, 2006). Yet, a recent systematic review of ten studies which examined health literacy and cancer screening (Oldach & Katz, 2014) revealed that while the health literacy instruments used in the study were valid, they only addressed a subsample of the health literacy skill set. Indeed, most existing health literacy measures focus on reading fluency (Oldach & Katz, 2014), whereas the AHL-C includes an additional domain of health literacy, prior knowledge (i.e., comprehension and familiarity subscales) (Baker, 2006). Prior knowledge is of particular relevance to those who are naïve to the US healthcare system (e.g., recent immigrants) as their lack of prior knowledge can create cognitive as well as emotional barriers to understanding health information and navigating the healthcare system (Poureslami, Rootman, Doyle-Waters, Nimmon, & Fitzgerald, 2011). In fact, prior knowledge yielded slightly stronger correlations with both breast (r=0.26 vs. 0.19) and cervical cancer knowledge (r=0.24 vs. 0.20) than reading fluency in the study sample. Future research needs to address if prior knowledge leads to better health behavior and outcomes.

Results from this study suggest that the AHL-C may be an effective evaluation tool for assessing interventions designed to improve health literacy in the context of breast and cervical cancer screening. However, we were only able to look at short-term effects over a six-month period. Traditional health education programs involving individuals with low health literacy have primarily focused on adjusting the reading levels, using interpreters, or incorporating the use of video, audiotapes, or web-based decision aids (McCray, 2005; Bischoff, Perneger, Bovier, Loutan, & Stalder; 2003; Santo, Laizner, & Shohet, 2005; Miller et al., 2011). Few have included topic contents or activities that are directly focused on improving health literacy due, in part, to the lack of a sensitive instrument to measure health literacy (Kim et al., 2012). The AHL-C may be a useful evaluation tool for clinicians and researchers who work with populations in need of improving cancer screening health literacy.

While all the test instructions were given in Korean, the actual test items on the AHL-C were written in English. This may have caused some readers to question whether using test items in English is a valid approach because the traditional approach has been that an English instrument is typically translated into the language of a target population (such as Korean). Our decision was based on prior investigations in which the translation of the REALM and S-TOFHLA into Korean did not lead to a valid assessment of health literacy (Han et al., 2011)—with results suggesting marginal reliability and questionable validity, and a highly skewed distribution of health literacy scores due, in large part, to the fact that Korean is a phonetic language. The feature of a phonetic language (i.e., one can pronounce words in the language so long as one can recognize its letters or alphabets) violates the design basis of popular health literacy tests based on reading fluency such as the REALM, which assumes a high correspondence between reading ability and comprehension (Han et al., 2011). An effort to translate the REALM into another phonetic language, Spanish, revealed similar problems (Nurss, Baker, Davis, Parker, & Williams, 1995). In addition, in the context of cancer screening, an individual navigates a health system in English at least partially, if not all, to obtain a screening test. Bilingual health personnel are difficult to identify in a minority community, let alone cancer screening facilities (e.g., mammogram facility). The sound evidence of reliability and validity of the AHL-C demonstrated in this study supports the approach taken in designing this measure.

Limitations of the study should be acknowledged. First, we included only one ethnic group whose educational level was somewhat higher than that of general U.S. women aged 25 years or older (about 65% having completed some college or more in our sample vs. 59% in the general population) (U.S. Census Bureau, 2013). In addition, the majority of our sample (about 77%) reported limited English proficiency in comparison to 42% of the general U.S. population and 56% of general Korean Americans who spoke a non-English language at home (Ryan, 2013). Hence, we are not able to assess the extent to which our findings are generalizable to other ethnic groups or to those without English difficulty. Nevertheless, the educational level reported in our study sample is comparable to 68% of Asian women (25+ years) who reported having completed some college or more (U.S. Census Bureau, 2013). We deliberately chose our sample based on cancer disparities experienced by Korean American women. Further research is warranted to validate the AHL-C in a wider range of ethnic samples with varying levels of educational attainment and English proficiency.

Second, the study sample was mainly drawn from ethnic churches. We decided that churches would be ideal recruitment and intervention sites for the study population since the majority of Korean Americans (80%) regularly attend an ethnic church (Han, Kang, Kim, Ryu, & Kim, 2007; Kim, Kim, Juon, & Hill, 2000). The church is a central feature of life for a number of minority and recent immigrant groups, functioning as a main source of social support and information sharing (Jo, Maxwell, Yang, & Bastani, 2010; Lee, Hanner, Cho, Han, & Kim, 2008). It is possible that church goers might have unique experiences that differentiate them from non-church goers such as increased opportunities to learn about health information and exposure to medical terminologies or procedures which might affect their health literacy.

Third, in its current form, the AHL-C may be too long to use in a busy clinical setting, hence further efforts need to be made to shorten the scale as appropriate. Finally, we acknowledge that there are competing definitions of health literacy such as the definition by Williams (2008) that focuses on knowledge of personal and familial risk of cancer and knowledge about cancer screening test. We developed the AHL-C based on Baker’s (2006) conceptualization of health literacy with print literacy, numeracy, comprehension, and familiarity domains.

As the field is moving forward with more sophisticated conceptualization of health literacy (von Wagner, Steptoe, Wolf & Wardle, 2009), it will be important to continue investigating particular conceptualization and domains of health literacy that are most salient to cancer screening behavior and refine the concept definition accordingly.

Further development of the AHL-C will focus on improving the measure’s feasibility by refining it based on item response theory. In the next phase, it would be important to test the sensitivity of the new instrument as an outcome measure in diverse populations. Additionally, while existing health literacy instruments are often time-consuming and costly to use in clinical practice (Johnson & Weiss, 2008), systematic investigation of the AHL-C using different test administration approaches (e.g., paper-pencil vs. computer-based) is warranted to enhance efficiency of the tool for use in multiple healthcare settings.

Acknowledgments

This study was supported by a grant from the National Cancer Institute (R01CA129060, Clinical Trials Registry NCT00857636). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The authors are grateful to all Korean American women and community health workers who participated in this study, and to the research staff, Jeong-A Ahn, Hyekyung Chin, and Myunghee Kim.

Appendix A

Proportion of correct response on the Reading fluency subscale

Classification Items Response type Correct
(%)
Direction: 아래 단어들을 발음해 주십시오 (Please read out loud the listed words below).
Reading test 1. Benign correct/incorrect 24.4
2. Biopsy correct/incorrect 53.0
3. Cervix correct/incorrect 66.9
4. Gynecology correct/incorrect 34.7
5. Human Papillomavirus correct/incorrect 29.0
6. Hysterectomy correct/incorrect 33.5
7. Malignant correct/incorrect 40.4
8. Mastectomy correct/incorrect 56.0
9. Metastasis correct/incorrect 40.8
10. Pelvic correct/incorrect 69.2
11. Uterus correct/incorrect 51.7
12. Vagina correct/incorrect 37.6
Average correct response for Reading test (%) 44.8
Direction: 다음 문장을 읽고 빈칸에 알맞은 단어를 골라 보십시오 (Please read the following and select the right response).
Clause test 13. Please sit down and roll up your sleeve. I will measure your ______. correct/incorrect 68.8
14. Please have a seat. While you are waiting, please fill out this ______. correct/incorrect 65.7
15. Do you have your ______ card with you? correct/incorrect 77.3
16. Doctor: Please tell me whether you have abnormal symptoms such as ______ in your breast or correct/incorrect 56.6
17. _______from your nipple. correct/incorrect 28.4
18. If you feel any changes in your breasts during self-examination, call us to make an appointment for _____. correct/incorrect 65.5
19. For what test appointment does Mrs. Kim make a call? correct/incorrect 78.2
20. Where does Mrs. Kim call for this test? correct/incorrect 77.7
21. Does anybody in your family have cervical, ovarian or breast cancer? Mrs. Kim: Yes. My sister had a _______ because she had a 22.______in the uterus. I heard about the virus that causes cervical cancer. What is it? correct/incorrect
correct/incorrect		 44.8
47.9
23. What is the answer that the doctor gave to Mrs. Kim? correct/incorrect 50.4
24. What is the right test for the explanation? correct/incorrect 44.8
Average correct response for Clause test (%) 58.8
Direction: 아래의 정보는 피자의 영양라벨입니다. 잘 읽고 대답해 주십시오 (Below is the food label of pizza (Appendix B). Please read and answer the questions).
Numeracy test 25. If you eat a piece of pizza, how many calories will you take? correct/incorrect 65.2
26. If you eat a piece of pizza for lunch, how many calories from saturated fat will you take? correct/incorrect 26.8
27. The doctor advises you to reduce cholesterol. If you do not eat a slice of pizza following that advice, how many calories would you reduce? correct/incorrect 69.8
28. If you eat a whole box of pizza, how many calories would you take? correct/incorrect 52.5
Average correct response for Numeracy test (%) 53.6
Open in a new tab

Appendix B

Nutrition label

Nutrition Facts *Percent Daily Values are based on a 2,000 calorie
diet. Your daily values may be higher or lower
depending on your calorie needs:
Serving Size ⅓ pizza (116g)
Serving Per Container 3 Calories 2,000 2,500
Amount Per Serving Total Fat Less than 65g 80g
Calories 330 Calories from Fat 150   Sat Fat Less than 20g 25g
% Daily Value* Cholesterol Less than 300g 300g
Total Fat 17g 18% Sodium Less than 2,400mg 2,400mg
  Saturated Fat 7g 30% Total Carbohydrate 300g 375g
  Trans Fat 0g   Dietary Fiber 25g 30g
Calories per gram:
  Fat 9 ● Carbohydrate 4 ● Protein 4
Cholesterol 35mg 3% INGREDIENTS
CRUST: Enriched wheat flour (malted barley flour, niacin, ferrous sulfate, thiamin mononitrate, riboflavin, folic acid), water, soybean oil, contains less than 2% sugar, yeast, salt, baking powder (corn starch, sodium bicarbonate, sodium aluminum sulfate, monocalcium phosphate), soy flour, cornmeal.
SAUCE: Water, tomato paste, contains less than 2% sugar, salt, spices, dehydrated garlic and onion, soybean oil, modified corn starch, guar gum, parmesan cheese (cultured milk, salt, enzymes), xanthan gum, paprika, citric acid, soy flour.
PIZZA TOPPING: Italian Sausage: Pork, seasonings (spices, corn syrup solids, salt, garlic powder, chili pepper, caramel color), water, textured vegetable protein (soy protein concentrate, caramel color).
Mozzarella Cheese Substitute: Water, partially hydrogenated soybean oil, rennet casein, sodium aluminum phosphate, corn starch, lactic acid, natural flavor, sodium citrate, sorbic acid (preservative), artificial color, enrichment blend (magnesium oxide)
Sodium 690mg 50%
Total Carbohydrate 30g 13%
  Dietary Fiber 2g 8%
  Sugar 6g
Protein 14g
Vitamin A 8% ● Vitamin C 2%
Calcium 25% ● Iron 6%
Open in a new tab

Appendix C

Proportion of response on the Prior knowledge subscale

Classification Items Response type Correct (%)
Direction: 단어와 단어의 정확한 뜻을 연결하세요 (Please find the correct meaning of each word listed below).
Comprehension 1. Benign correct/incorrect 17.9
2. Biopsy correct/incorrect 17.1
3. Cervix correct/incorrect 29.1
4. Gynecology correct/incorrect 38.8
5. Human Papillomavirus correct/incorrect 37.1
6. Hysterectomy correct/incorrect 22.7
7. Malignant correct/incorrect 22.5
8. Mastectomy correct/incorrect 23.8
9. Metastasis correct/incorrect 21.3
10. Pelvic correct/incorrect 22.9
11. Uterus correct/incorrect 30.5
12. Vagina correct/incorrect 40.9
Average correct response for Comprehension (%) 27.1
Direction: 다음 단어들이 얼마나 익숙한지 한가지만 골라 응답해 주세요 (The following questions are asking about how familiar you are with the words below. Please check the answer that is most appropriate).
Not at all familiar
(%)		 Item mean
(range=0–1)*
Familiarity 13. Benign 73.0 0.14
14. Biopsy 55.4 0.24
15. Cervix 66.8 0.18
16. Gynecology 62.1 0.21
17. Human Papillomavirus 70.0 0.15
18. Hysterectomy 72.7 0.14
19. Malignant 75.7 0.13
20. Mastectomy 79.1 0.11
21. Metastasis 80.1 0.10
22. Pelvic 70.1 0.17
23. Uterus 64.8 0.20
24. Vagina 52.8 0.28
Average item mean for Familiarity 0.17
Open in a new tab
*

Recalibrated from 0–4 points to 0–1 point for ease of understanding.

Contributor Information

Hae-Ra Han, The Johns Hopkins University School of Nursing, Baltimore, MD, USA.

Boyun Huh, The Johns Hopkins University School of Nursing, Baltimore, MD, USA.

Miyong T. Kim, University of Texas at Austin School of Nursing, Austin, TX, USA.

Jiyun Kim, Gachon University Department of Nursing, Seongnam-Si, Gyeonggi-Do, Korea.

Tam Nguyen, Boston College School of Nursing, Boston, MA, USA.

References

  1. Apter AJ, Cheng J, Small D, Bennett IM, Albert C, Fein DG, et al. Asthma numeracy skill and health literacy. Journal of Asthma. 2006;43(9):705–710. doi: 10.1080/02770900600925585. [DOI] [PubMed] [Google Scholar]
  2. Baker DW. The meaning and the measure of health literacy. Journal of General Internal Medicine. 2006;21(8):878–883. doi: 10.1111/j.1525-1497.2006.00540.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bennett CL, Ferreira MR, Davis TC, Kaplan J, Weinberger M, Kuzel T, et al. Relation between literacy, race, and stage of presentation among low-income patients with prostate cancer. Journal of Clinical Oncology. 1998;16(9):3101–3104. doi: 10.1200/JCO.1998.16.9.3101. [DOI] [PubMed] [Google Scholar]
  4. Bischoff A, Perneger TV, Bovier PA, Loutan L, Stalder H. Improving communication between physicians and patients who speak a foreign language. British Journal of General Practice. 2003;53(492):541–546. [PMC free article] [PubMed] [Google Scholar]
  5. Davis TC, Long SW, Jackson RH, Mayeaux EJ, George RB, Murphy PW, et al. Rapid estimate of adult literacy in medicine: A shortened screening instrument. Family Medicine. 1993;25(6):391–395. [PubMed] [Google Scholar]
  6. Diviani N, Schulz PJ. First insights on the validity of the concept of cancer literacy: A test in a sample of Ticino (Switzerland) residents. Patient Education and Counseling. 2012;87(2):152–159. doi: 10.1016/j.pec.2011.08.016. [DOI] [PubMed] [Google Scholar]
  7. Erby LH, Roter D, Larson S, Cho J. The rapid estimate of adult literacy in genetics (REAL-G): A means to assess literacy deficits in the context of genetics. American Journal of Medical Genetics. Part A. 2008;146A(2):174–181. doi: 10.1002/ajmg.a.32068. [DOI] [PubMed] [Google Scholar]
  8. Han HR, Kim J, Kim MT, Kim KB. Measuring health literacy among immigrants with a phonetic primary language: A case of Korean American women. Journal of Immigrant and Minority Health. 2011;13(2):253–259. doi: 10.1007/s10903-010-9366-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Han HR, Kang J, Kim KB, Ryu JP, Kim MT. Barriers to and strategies for recruiting Korean Americans for community-partnered health promotion research. Journal of Immigrant and Minority Health. 2007;9(2):137–146. doi: 10.1007/s10903-006-9022-x. [DOI] [PubMed] [Google Scholar]
  10. Huizinga MM, Elasy TA, Wallston KA, Cavanaugh K, Davis D, Gregory RP, et al. Development and validation of the diabetes numeracy test (DNT) BMC Health Services Research. 2008;8:96. doi: 10.1186/1472-6963-8-96. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Innos K, Horn-Ross PL. Recent trends and racial/ethnic differences in the incidence and treatment of ductal carcinoma in situ of the breast in California women. Cancer. 2003;97(4):1099–1106. doi: 10.1002/cncr.11104. [DOI] [PubMed] [Google Scholar]
  12. Institute of Medicine. Health literacy: A prescription to end confusion. Washington, DC: The National Academies Press; 2004. [PubMed] [Google Scholar]
  13. Jo AM, Maxwell AE, Yang B, Bastani R. Conducting health research in Korean American churches: Perspectives from church leaders. Journal of Community Health. 2010;35(2):156–164. doi: 10.1007/s10900-009-9213-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Johnson K, Weiss BD. How long does it take to assess literacy skills in clinical practice? Journal of the American Board of Family Medicine. 2008;21:211–214. doi: 10.3122/jabfm.2008.03.070217. [DOI] [PubMed] [Google Scholar]
  15. Kim MT, Kim KB, Juon HS, Hill MN. Prevalence and factors associated with high blood pressure in Korean Americans. Ethnicity & Disease. 2000;10(3):364–374. [PubMed] [Google Scholar]
  16. Kim MT, Song HJ, Han HR, Song Y, Nam S, Nguyen TH, et al. Development and validation of the high blood pressure-focused health literacy scale. Patient Education and Counseling. 2012;87(2):165–170. doi: 10.1016/j.pec.2011.09.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Lee HB, Hanner JA, Cho SJ, Han HR, Kim MT. Improving access to mental health services for Korean American immigrants: Moving toward a community partnership between religious and mental health services. Psychiatry Investigation. 2008;5(1):14–20. doi: 10.4306/pi.2008.5.1.14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Lindau ST, Tomori C, Lyons T, Langseth L, Bennett CL, Garcia P. The association of health literacy with cervical cancer prevention knowledge and health behaviors in a multiethnic cohort of women. American Journal of Obstetrics and Gynecology. 2002;186(5):938–943. doi: 10.1067/mob.2002.122091. [DOI] [PubMed] [Google Scholar]
  19. Lindau ST, Tomori C, McCarville MA, Bennett CL. Improving rates of cervical cancer screening and Pap smear follow-up for low-income women with limited health literacy. Cancer Investigation. 2001;19(3):316–323. doi: 10.1081/cnv-100102558. [DOI] [PubMed] [Google Scholar]
  20. Liu L, Zhang J, Deapen D, Bernstein L, Ross RK. Cancer in Los Angeles County: Incidence and mortality by race/ethnicity 1988–2000. Los Angeles, CA: Los Angeles County Cancer Surveillance Program, University of Southern California; 2003. [Google Scholar]
  21. Liu L, Zhang J, Wu AH, Pike MC, Deapen D. Invasive breast cancer incidence trends by detailed race/ethnicity and age. International Journal of Cancer. 2012;130(2):395–404. doi: 10.1002/ijc.26004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Lynn MR. Determination and quantification of content validity. Nursing Research. 1986;35(6):382–385. [PubMed] [Google Scholar]
  23. McCance KL, Mooney KH, Smith KR, Field R. Validity and reliability of a breast cancer knowledge test. American Journal of Preventive Medicine. 1990;6(2):93–98. [PubMed] [Google Scholar]
  24. McCray AT. Promoting health literacy. Journal of American Medical Informatics Association. 2005;12(2):152–163. doi: 10.1197/jamia.M1687. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Miller DP, Jr, Spangler JG, Case LD, Goff DC, Jr, Singh S, Pignone MP. Effectiveness of a web-based colorectal cancer screening patient decision aid: a randomized controlled trial in a mixed-literacy population. American Journal of Preventive Medicine. 2011;40(6):608–615. doi: 10.1016/j.amepre.2011.02.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Nath CR, Sylvester ST, Yasek V, Gunel E. Development and validation of a literacy assessment tool for persons with diabetes. Diabetes Educator. 2001;27(6):857–864. doi: 10.1177/014572170102700611. [DOI] [PubMed] [Google Scholar]
  27. Nunnally JCBI. Psychometric theory. 3rd. New York: McCraw-Hill; 1994. [Google Scholar]
  28. Nurss JR, Baker DW, Davis TC, Parker RM, Williams M. Difficulties in functional health literacy screening in Spanish-speaking adults. Journal of Reading. 1995;38:632–637. [Google Scholar]
  29. Oldach BR, Katz ML. Health literacy and cancer screening: A systematic review. Patient Education and Counseling. 2014;94(2):149–157. doi: 10.1016/j.pec.2013.10.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Ownby RL, Waldrop-Valverde D, Hardigan P, Caballero J, Jacobs R, Acevedo A. Development and validation of a brief computer-administered HIV-related health literacy scale (HIV-HL) AIDS and Behavior. 2013;17(2):710–718. doi: 10.1007/s10461-012-0301-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Park S, Chang S, Chung C. Effects of a cognition-emotion focused program to increase public participation in Papanicolaou smear screening. Public Health Nursing. 2005;22:289–298. doi: 10.1111/j.0737-1209.2005.220404.x. [DOI] [PubMed] [Google Scholar]
  32. Parker RM, Baker DW, Williams MV, Nurss JR. The test of functional health literacy in adults: A new instrument for measuring patients' literacy skills. Journal of General Internal Medicine. 1995;10(10):537–541. doi: 10.1007/BF02640361. [DOI] [PubMed] [Google Scholar]
  33. Pendlimari R, Holubar SD, Hassinger JP, Cima RR. Assessment of colon cancer literacy in screening colonoscopy patients: A validation study. The Journal of Surgical Research. 2012;175(2):221–226. doi: 10.1016/j.jss.2011.04.036. [DOI] [PubMed] [Google Scholar]
  34. Poureslami I, Rootman I, Doyle-Waters MM, Nimmon L, Fitzgerald JM. Health literacy, language, and ethnicity-related factors in newcomer asthma patients to Canada: A qualitative study. Journal of Immigrant and Minority Health. 2011;13(2):315–322. doi: 10.1007/s10903-010-9405-x. [DOI] [PubMed] [Google Scholar]
  35. Ryan C. Language use in the United States: 2011. [Retrieved May 23, 2014];American Community Survey Reports. 2013 from http://www.census.gov/prod/2013pubs/acs-22.pdf.
  36. Santo A, Laizner AM, Shohet L. Exploring the value of audiotapes for health literacy: a systematic review. Patient Education and Counseling. 2005;58(3):235–243. doi: 10.1016/j.pec.2004.07.001. [DOI] [PubMed] [Google Scholar]
  37. Scott TL, Gazmararian JA, Williams MV, Baker DW. Health literacy and preventive health care use among Medicare enrollees in a managed care organization. Medical Care. 2002;40(5):395–404. doi: 10.1097/00005650-200205000-00005. [DOI] [PubMed] [Google Scholar]
  38. U.S. Census Bureau. Educational attainment in the United States: 2013. [Retrieved May 23, 2014];2013 from http://www.census.gov/hhes/socdemo/education/
  39. U.S. Department of Health and Human Services. Health literacy definition. Washington, DC: U.S. Department of Health and Human Services; 2000. [Google Scholar]
  40. U.S. Preventive Services Taskforce. A-Z topic guide. [Retrieved February 20, 2012];2010 from http://www.uspreventiveservicestaskforce.org/ [Google Scholar]
  41. von Wagner C, Steptoe A, Wolf MS, Wardle J. Health literacy and health actions: a review and a framework from health psychology. Health Education & Behavior. 2009;36(5):860–877. doi: 10.1177/1090198108322819. [DOI] [PubMed] [Google Scholar]
  42. Weinick RM, Zuvekas SH, Cohen JW. Racial and ethnic differences in access to and use of health care services, 1977 to 1996. Medical Care Research and Review. 2000;57(Suppl 1):36–54. doi: 10.1177/1077558700057001S03. [DOI] [PubMed] [Google Scholar]
  43. Weiss BD, Hart G, McGee DL, D'Estelle S. Health status of illiterate adults: Relation between literacy and health status among persons with low literacy skills. Journal of the American Board of Family Practice. 1992;5(3):257–264. [PubMed] [Google Scholar]
  44. Weiss BD, Mays MZ, Martz W, Castro KM, DeWalt DA, Pignone MP, et al. Quick assessment of literacy in primary care: The Newest Vital Sign. Annals of Family Medicine. 2005;3(6):514–522. doi: 10.1370/afm.405. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Williams KP, Reckase M, Rivera-Vasquez O. Toward the development of cancer literacy assessment tools. Michigan Journal of Public Health. 2008;2:21–31. [Google Scholar]
  46. Williams KP, Templin TN, Hines RD. Answering the call: A tool that measures functional breast cancer literacy. Journal of Health Communication. 2013;18:1310–1325. doi: 10.1080/10810730.2013.778367. [DOI] [PubMed] [Google Scholar]
  47. Williams KP, Templin TN. Bringing the real world to psychometric evaluation of cervical cancer literacy assessments with Black, Latina, and Arab women in real-world settings. Journal of Cancer Education. 2013;28:738–743. doi: 10.1007/s13187-013-0549-y. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES