Measuring Health Literacy Among Immigrants with a Phonetic Primary Language: A Case of Korean American Women

Abstract

While the need for understanding the issue of health literacy among ethnic minority groups with limited English skills is rapidly increasing in the US, it is difficult to find valid and useful health literacy tools for certain linguistic minorities. This study was designed to validate the Korean translation of Rapid Estimate of Adult Literacy in Medicine (REALM) and Test of Functional Health Literacy in Adults-Short form (S-TOFHLA). Korean REALM and S-TOFHLA were administered to 98 Korean American women, together with REALM-English. Participants were first-generation immigrants who were educated in Korea. Both Korean REALM and S-TOFHLA resulted in a negatively-skewed distribution. REALM-English yielded well-distributed groups with significant correlations with Korean REALM and S-TOFHLA (Spearman’s rho = 0.30, P = 0.003 and 0.22, P = 0.03, respectively). Educational level was significantly correlated with REALM-English and Korean S-TOFHLA (Spearman’s rho = 0.39, P = 0.000 and 0.25, P = 0.014), but not with REALM-Korean. The translation of REALM and S-TOFHLA into the Korean language did not lead to a valid assessment of health literacy. A more systematic approach is needed to assess health literacy in immigrants with limited English skills, particularly those with a phonetic primary language. Meanwhile, REALM-English could be used as a crude health literacy test for individuals with some English skills.

Introduction

Health literacy is defined as “the degree to which individuals have the capacity to obtain, process, and understand basic health information and services to make appropriate health decisions [1].” Foreign-born, non-English-speaking immigrants, those with a lower level of education, and the elderly are disproportionately affected by low health literacy [2]. In particular, foreign-born Americans represent at least 15% of the 90–94 million people who are illiterate in the United States [2]. Limited health literacy has been associated with inadequate utilization of preventive services [3, 4], poor chronic disease management outcomes [5, 6], and higher medical costs [7, 8].

Today’s Korean Americans (KA) are predominantly first-generation immigrants (92.5% are foreign-born) and monolingual (70% do not speak English well) [9, 10]. Low educational attainment is notable among KA women; 26–54% have completed less than a high school education, compared to 11% of KA men and 20% of the general US population [11, 12]. As is often true for many adult immigrants with limited English skills, KA women frequently rely on assistance from their children or acquaintances who may lack adequate time or English skills to communicate successfully with health care providers [13]. In addition, previous studies revealed that a feeling of fear of having to read or complete hospital paperwork in English is one of main barriers to utilizing the health care system among KA women [14, 15]. Accurate assessment of health literacy is the first step toward understanding and addressing health issues associated with low health literacy among KA women.

To date, most health literacy research has focused on English- or Spanish-speaking populations, with no health literacy test available for Korean. Rapid Estimate of Adult Literacy in Medicine (REALM) [16] and Test of Functional Health Literacy in Adults-Short form (S-TOFHLA) [17, 18] are the two most popular health literacy tests. Both health literacy tests have been translated into Spanish [19, 20]. In this study, we attempted to validate the Korean translation of REALM and S-TOFHLA in a sample of KA women.

Methods

Participants/Data Collection

Participants were recruited from health fairs in the local Korean community. At each fair, which lasted about 2–4 h, a variety of health seminars and services were made available by county health department personnel, Korean-speaking clinicians, university academicians, and community volunteers. Seminar topics included bone health, mental health, health promotion, and smoking cessation. Health services offered were on-the-spot blood sugar test, bone density screening, and blood pressure monitoring. Women were recruited from these events while they were waiting for their turn for health services or after they received the services. Given the nature of the health literacy tests, data were collected via face-to-face interview. All interviews were performed in Korean. Each interview took less than 15–20 min.

Study protocol was approved by the Institutional Review Board. Trained research staff approached potential participants to briefly introduce themselves and the study. Research staff then asked whether they would be interested in participating in the study. Korean women 18 years of age or older and who could read and write Korean were eligible to participate. For those who were interested and eligible, detailed explanation about the study was given. A total of 99 KA women were enrolled in the study. Informed consent was obtained from each participating woman.

Measures

A sociodemographic data collecting tool was developed for gathering pertinent information on study participants. The tool included questions asking basic demographic information about the participant such as age, education, marital status, employment status, and years of residence in the United States.

Rapid Estimate of Adult Literacy in Medicine (REALM) [16] is the most popular instrument that has been validated and used for the assessment of literacy skills in multiple contexts [16, 21, 22]. REALM tests the individual’s ability to pronounce 66 commonly used medically relevant words (i.e., medical and lay terms for body parts and illnesses). Previous studies have shown that REALM compares favorably to other formal reading assessments and to assessments that test other skills, such as comprehension, with correlation coefficients ranging from 0.80 to 0.90 [16–18]. In addition, it is easy to administer and takes less than 5 min to complete. Raw scores range from 0 to 66 and can be converted into four grade range estimates of literacy: (1) 3rd grade level and below (scores 0–18), (2) 4–6th grade level (scores 19–44), (3) 7–8th grade level (scores 45–60), and (4) high school level (scores 61–66) [16].

Test of Functional Health Literacy in Adults-Shortened version (S-TOFHLA) [17] uses actual materials that patients might encounter in the health care setting. It consists of two parts: (1) The reading comprehension section is a 36-item test using two prose passages. The passages on the S-TOFHLA use a modified Cloze procedure where every fifth to seventh word is omitted and individuals select the correct word from a set of four options. This section measures a patient’s ability to read and understand prose passages selected from instructions for preparation for an upper gastrointestinal tract X-ray procedure, and the patient “rights and responsibilities” section of a Medicaid application. (2) The numeracy section is a 4-item test using actual hospital forms and labeled prescription vials. It tests a patient’s ability to comprehend directions for taking medicines and keeping clinic appointments. Correct responses for each question in the reading and numeracy sections receive one point, and incorrect responses do not receive any points. Each item in the reading comprehension is multiplied by 2 (×36 items) to create a score from 0 to 72, and each numeracy question is multiplied by 7 (×4 items) to create a score from 0 to 28. The sum of the two sections yields the S-TOFHLA score, which ranges from 0 to 100. Scores on S-TOFHLA are classified and interpreted as follows: inadequate health literacy (scores 0–53), marginal health literacy (scores 54–66), and adequate health literacy (scores 67–100). The maximum time for administration by face-to-face interview is 12 min. S-TOFHLA has been validated in both English and Spanish [18, 20, 23].

REALM and S-TOFHLA were translated into Korean and back-translated into English to ensure accuracy of translation. Due to differences in the basic structure of English and Korean, passages in S-TOFHLA were rewritten keeping the consistency in the order of subject-object-verb words, in contrast to English which has a subject-verb-object word order. In addition, original REALM-English was used for validation purposes. For all tests, the word lists and passages were printed in large letters and presented to the KA woman to read on laminated cards. The interviewer then asked the woman to read words on REALM or answer the comprehension and numeracy questions on S-TOFHLA.

Analysis

Analysis was done on 98 KA women who completed REALM and S-TOFHLA. One woman initiated but did not complete the health literacy tests. Descriptive statistics were used to summarize sample characteristics and health literacy scores. Internal consistency was assessed by KR-20 [24]. Spearman’s correlation coefficients were calculated for scores between REALM-English and S-TOFHLA, between REALM-English and REALM-Korean, and between REALM-Korean and S-TOFHLA. We also used Spearman rank correlation to examine the relationships between sociodemographic characteristics and the categories for REALM and S-TOFHLA. A P value of <.05 was considered significant for all tests.

Results

KA women in the study were mostly middle-aged, with a mean age of 49.1 (SD = 6.9) years (range = 34–68 years). Most women were married (83.7%) and 54.1% were unemployed. Less than 5% did not complete high school; one-third completed high school; more than 59% reported more than a high school level of education, with all women receiving their education in Korea. Nearly two thirds of the sample had lived in the US for 10 years or more, with an average length of residence being 15.2 (SD = 8.6) years (range = 1–36 years) (Table 1).

Table 1.

Sample characteristics (N = 98)

Variable	%	Mean (SD)
Age, years		49.1 (6.9)
30–39	6.1
40–49	46.9
50–59	39.8
60+	5.1
Missing	2.0
Married	83.7
Employed full- or part-time	45.9
Level of education
Less than high school	4.1
High school	33.7
More than high school	59.2
Missing	3.1
Years in the US, years		15.2 (8.6)
Less than 10 years	31.6
10 years or more	65.3
Missing	3.1
REALM-English (range = 0–66)		49.1 (18.4)
REALM-Korean (range = 55–66)		65.3 (1.8)
S-TOFHLA (range = 32–96)		85.4 (10.2)
Numeracy (range = 0–28)		21.5 (6.3)
Comprehension (range = 32–72)		63.9 (7.0)

REALM-English resulted in generally well-distributed groups with a reliability coefficient of .98 and a mean score of 49.1 (SD = 18.4): 9.2% of women scored 0–18, 17.3% scored 19–44, 37.8% scored 45–60, and the remaining 35.7% scored 61–66 (Table 2). In contrast, scores on the Korean REALM and S-TOFHLA were negatively skewed, with the majority of the sample achieving scores on the high end. For example, more than 9 out of 10 KA women in the study scored 61–66 (mean = 65.3, SD = 1.8) on REALM-Korean. Likewise, nearly 90% answered 3 or all 4 numeracy items correctly on Korean translation of S-TOFHLA. For S-TOFHLA reading passages in Korean, the majority of the sample (89.8%) answered three quarters of the Close items correctly for both passages A and B. Based on S-TOFHLA cutoffs, most women (93.9%) were categorized as having adequate health literacy skills, whereas 4.1% had marginal and 2% had inadequate literacy skills (mean = 85.4, SD = 10.2). Reliability coefficients yielded 0.80 for REALM-Korean, 0.64 for the numeracy, and 0.79 for the reading comprehension sections of S-TOFHLA, respectively.

Table 2.

Proportion of Korean women with correct responses on the REALM and S-TOFHLA (N = 98)

Variable	Correct (%)
REALM-English items
0–18	9.2
19–44	17.3
45–60	37.8
61–66	35.7
REALM-Korean items
45–60	3.1
61–66	96.9
Numeracy items
0	4.1
1	3.1
2	3.1
3	61.2
4	28.6
Reading comprehension passages
A: Preparation of an upper GI X-ray
0–8	1.0
9–12	9.2
13–16	89.8
B: Medicaid application
0–10	3.1
11–15	7.1
16–20	89.8

Predictive validity of the Korean health literacy tests was examined by correlations between the Korean version and REALM-English. Korean translations of REALM and S-TOFHLA were both significantly correlated with REALM-English (Spearman’s rho = 0.30, P = 0.003 and 0.22, P = 0.03, respectively); however, correlation between the two Korean health literacy tests was not statistically significant. To test construct validity, the level of health literacy was compared in relation to the woman’s age, education, and years of residence in the United States. The proportion of women with higher levels of health literacy tended to be greater among younger (<50 years) and more educated (>high school) women, though none of the literacy tests were correlated with age. The level of education was significantly associated with REALM-English and S-TOFHLA (Spearman’s rho = 0.39, P = 0.000 and 0.25, P = 0.014), but not with REALM-Korean. The length of residence in the United States was not associated with health literacy in this sample (Tables 3, 4).

Table 3.

Health literacy categories by age, level of education, and years in the United States

Health literacy categories	Age		Education			Years in the US
	< 50 years (%)	≥50 years (%)	< High school (%)	High school (%)	> High school (%)	< 10 years (%)	≥10 years (%)
REALM-English
≤3rd grade level	3.8	13.6	50.0	15.2	1.7	6.5	9.4
4–6th grade level	13.5	20.5	25.0	30.3	6.9	16.1	15.6
7–8th grade level	42.3	34.1	–	30.3	46.6	45.2	35.9
High school level	40.4	31.8	25.0	24.2	44.8	32.3	39.1
REALM-Korean
7–8th grade level	1.9	4.5	–	6.1	1.7	3.2	3.1
High school level	98.1	95.5	100.0	93.9	98.3	96.8	96.9
S-TOFHLA
Inadequate	–	4.5	–	6.1	–	–	3.1
Marginal	1.9	4.5	25.0	6.1	–	3.2	3.1
Adequate	98.1	90.9	75.0	87.9	100.0	96.8	93.8

Table 4.

Spearman’s correlation coefficients for demographic characteristics and categories for health literacy tests

	Age	Education	Years in the US	REALM-English	REALM-Korean	S-TOFHLA
Age	–
Education	−.259 (.011)*	–
Years in the US	.342 (.001)**	−.126 (.222)	–
REALM-English	−.096 (.351)	.387 (.000)***	.182 (.077)	–
REALM-Korean	.087 (.398)	−.026 (.801)	.124 (.230)	.300 (.003)**	–
S-TOFHLA	−.193 (.059)	.252 (.014)*	−.023 (.823)	.219 (.030)*	−.006 (.952)	–

Spearman’s rho (p)

^*P < .05,

^**P < .01,

^***P < .001

Discussion/New Contribution to the Literature

To the best of our knowledge, this is the first study to validate REALM and S-TOFHLA in non-English speaking Korean immigrants. The translation of REALM and S-TOFHLA into Korean did not lead to a valid assessment of health literacy—the distribution of both tests was considerably negatively skewed, with more than 90% of KA women being categorized in the highest score range, yielding little variance in health literacy scores. In particular, REALM-Korean scores did not correlate with the woman’s level of education at all. One of the most plausible explanations may be that Korean is a phonetic language; unlike English, the Korean language system is made in such a way that one sound is represented by one letter, and each letter is matched to the Korean spoken language. Hence, if one learns Korean alphabets with the matching sounds, one can easily pronounce and read words in Korean without comprehending the meaning of the word. Indeed, available statistics indicate most Korean nationals can read and write; literacy rates are 97% for Korean women and 99% for Korean men [25]. For this reason, it is highly unlikely that a simple reading test in Korean—even in the context of health care—would have an ability to discriminate individuals with and without comprehension of the word and, hence, result in a meaningful assessment measure of health literacy.

An earlier effort to translate the REALM into Spanish was also unsuccessful [19]. The authors explained the finding as a result of the phonetic structure of the Spanish language, similar to the case of the Korean language. It was noted that the feature of the Spanish language based on the regular phoneme–grapheme correspondence (one can pronounce words in Spanish so long as one can recognize letters) violates the design basis of the REALM, which assumes a high correspondence between reading ability and comprehension based on the English linguistic structure.

Korean translation of S-TOFHLA, a functional literacy test, was more successful than the REALM-Korean, with more variability in the responses. Several researchers have recognized the importance of functional literacy as a more accurate indicator of a patient’s reading ability as it pertains to the ability to read and understand health-related materials and numerical information [17, 18]. Indeed, S-TOFHLA scores significantly correlated with the level of education in KA women, though it was a weak correlation. Similarly, in a recent study of South African patients [26], comprehension (P = 0.001) had a stronger association with education than pronunciation (P = 0.016). Nevertheless, the issue of regular phoneme-grapheme correspondence remained the same. Further, the use of some of the standardized passages for S-TOFHLA (e.g., Medicaid application) may not be necessarily common even among American natives [27], causing more difficulty for immigrants and increasing measurement errors.

REALM-English resulted in a better distribution of scores that also coincided with the level of education in this sample. It was assumed that KA women in the sample would have some English skills since English education, traditionally focused more on grammatical skills rather than English verbal abilities, is mandatory in Korea from the third year of elementary school (from middle school before 1995) up to high school [28]. The traditional approach for an instrument developed in English has been that the instrument gets translated into the language of a target population. In the case of health literacy, however, the expectation is that an individual navigates a health system in English at least partially, if not all. Therefore, it would be important to incorporate critical medically relevant English words in the application of health literacy test for immigrants with limited English skills. Although the attempt to use REALM-English among KA women was more successful than the Korean translation of REALM and S-TOFHLA in terms of discriminating subgroups, the English version also has a major limitation in its application to the KA population: KAs, especially those in older age group, have no or very limited English skills and are often intimidated by the attempt to assess their literacy skills in English [29]. Considering these, the health literacy scale that is designed as a simple reading test may not be a sensitive tool for an individual who can read words aloud on REALM without comprehension of the words [27], which can make it difficult to assess the person’s true health literacy level. As noted by Nurss et al. [19], a test of reading comprehension in an actual medical context may be more useful as an indicator of true health literacy for non-English speaking immigrants, particularly those with a phonetic primary language.

Construct validity of the health literacy instruments was also examined in relation to the woman’s age and years of residence in the US. While KA women with greater health literacy tended to be younger (<50 years), the relationship was not significant. The finding is inconsistent with earlier research where the correlation between younger age and higher health literacy persisted even after controlling for education, race, ethnicity, gender, and immigration status [30]. A possible explanation may be that our sample included mostly middle-aged women with more than half aged <50, as compared to the previous study which included older participants (average = 55 years) with only about one third (34%) aged 50 or less.

We used years of residence in the US as a proxy measure of English ability. As with the case of age, length of residence in the US was not associated with health literacy in the KA sample. Residence in the US has not been necessarily associated with the level of English proficiency in Korean women, even after residing in the US for more than 20 years [31, 32]. This may be due, in part, to the fact that today’s KAs are predominantly first-generation immigrants who come to the US, for the most part when they are 20–40 years of age [33]. Studies have documented that acquisition of English skills is predicted by age of arrival in the US, with early arrivals achieving greater proficiency than late arrivals [34, 35]. Future research should include a more accurate assessment of English ability in association with health literacy.

The study findings should be interpreted with caution, given that our sampling design was convenience-based, thus limiting the extent to which generalization can be made beyond the target minority women. In particular, KA women in the study were mostly middle-aged, with a relatively high level of education (less than 5% did not complete high school). In future research, women with more diverse educational background representing different age groups should be considered to further validate the health literacy tests used in this study. The study sample was recruited from local health fairs. Those who attended a health fair were likely the ones who were interested in their health. Future recruitment strategies should include utilizing various sites with better access to the target population such as ethnic churches [29].

Finding the ideal way of measuring health literacy among groups with limited English skills is one of main scientific goals of many investigators. Our findings indicate that the translation of the standardized measures of health literacy, REALM and S-TOFHLA, has a limited utility in applying them for certain linguistic minority populations (e.g., those whose primary language structure is predominantly phonetic, such as Korean). In fact, the use of Korean REALM and S-TOFHLA may not be fruitful in future research endeavors because of its poor performance with questionable validity. Meanwhile, REALM-English could be used as a crude tool to assess the level of health literacy among individuals with some English skills. Assessing health literacy in non-English speaking Korean immigrants may require a more systematic approach. Future health literacy instruments targeting immigrants with a phonetic primary language need to incorporate questions to measure the examinee’s comprehension of commonly used English medical terms or scripts in a given context.