Question Context and Priming Meaning of Health: Effect on Differences in Self-Rated Health Between Hispanics and Non-Hispanic Whites

Abstract

Objectives. We examined the implications of the current recommended data collection practice of placing self-rated health (SRH) before specific health-related questions (hence, without a health context) to remove potential context effects, between Hispanics and non-Hispanics.

Methods. We used 2 methodologically comparable surveys conducted in English and Spanish that asked SRH in different contexts: before and after specific health questions. Focusing on the elderly, we compared the influence of question contexts on SRH between Hispanics and non-Hispanics and between Spanish and English speakers.

Results. The question context influenced SRH reports of Spanish speakers (and Hispanics) significantly but not of English speakers (and non-Hispanics). Specifically, on SRH within a health context, Hispanics reported more positive health, decreasing the gap with non-Hispanic Whites by two thirds, and the measurement utility of SRH was improved through more consistent mortality prediction across ethnic and linguistic groups.

Conclusions. Contrary to the current recommendation, asking SRH within a health context enhanced measurement utility. Studies using SRH may result in erroneous conclusions when one does not consider its question context.

Hispanics in the United States have emerged as an important group for public health research because of their noteworthy population growth. The past decade saw a rapid increase of the Hispanic population from 35.3 million to 50.5 million, corresponding to 12.5% and 16.3% of the total population.¹ In states such as California, New Mexico, and Texas, Hispanics make up more than 35% of the population. Although not the majority in the general population, Hispanics contributed more than half of the US population growth.

One distinctive characteristic of Hispanics is their language use. Four out of 10 Hispanics are reported to speak English less than very well, hence being classified as “linguistically isolated.”² The linguistic isolation rate is estimated to be more than 90% for some Hispanic subgroups, such as older low-income Cuban women in Miami.³ Although not a health risk factor itself, low English proficiency (LEP) is related to many health outcomes through the socioeconomic gradient, such as education, poverty, and access to health care. Because the failure to capture LEP persons produces data misrepresenting the population,^4–6 it has become standard practice for government and academic surveys in the United States to conduct interviews in both English and Spanish. The National Health Interview Survey (NHIS), for example, has conducted Spanish interviews consistently since 1997 and with standardized translated questionnaires since 2004.

Hispanics’ health has been compared with that of other racial/ethnic groups,^7–11 creating a famous term, “Hispanic paradox.” Even though correlates of health, such as income and education, are estimated to be lower for Hispanics than for non-Hispanic Whites, Hispanics show better health outcomes than non-Hispanic Whites or comparable health outcomes to non-Hispanic Whites.^12–18 One exception to the paradox is the measure self-rated health (SRH), which consistently shows less favorable outcomes for Hispanics compared with non-Hispanic Whites.^7,19,20 Self-rated health is a simple survey item asking respondents for their subjective assessment about their own health by using some variations of a 4- or 5-point Likert response scale. One of the scales ranging from “excellent,” “very good,” “good,” “fair,” to “poor” is popular in the United States, and another scale using “very good” to “very poor,” supported by the World Health Organization, is popular elsewhere.²¹

The popularity of SRH not only in health research^22–26 but also in other social sciences^27–31 led the US National Center for Health Statistics to organize a conference dedicated to this particular item, the Conference on the Cognitive Aspects of the Self-Rated Health Status, in 1993.³² Because of its proven utility as a strong and independent predictor of subsequent mortality,^33–39 various health conditions,^40-43 and health care utilization,^44–46 the World Health Organization,⁴⁷ the US Centers for Disease Control and Prevention,⁴⁸ and the European Commission⁴⁹ have recommended SRH as a reliable measure of monitoring population health.

Self-rated health is also used as a practical tool for comparing various population groups associated with country,^50–52 gender,⁵³ race,⁴¹ socioeconomic status,^54,55 educational attainment,⁵⁶ poverty status,⁵⁷ and immigration status,⁵⁸ often leading to discussions about health disparities. Although it is critical to use items with comparable measurement properties across comparison groups, the measurement utility of SRH has been mostly examined with English speakers or northern Europeans.^44,59–61 Beyond these groups, the performance of SRH has been found to be inconsistent, leading to questionable comparability.^{9,21,25,29,62–68}

The literature on the utility of SRH for US Hispanics is spotty and does not provide clear conclusions and appears to have overlooked methodological limitations in the data.^8,10,64,69 Some studies have used data that did not include LEP Hispanics,⁶⁴ and some used SRH asked in different contexts.^8,10,69 The former is no longer a serious issue because the current survey practice accommodates LEP Hispanics. However, the question context may raise a concern as it has been suggested as a future research topic for SRH,^70,71 including the seminal work by Idler and Benyamini.³⁶

Particularly for Hispanics, a recent study by Lee and Grant⁷² suggests troublesome implications. They conducted an experiment where the order of SRH in a questionnaire was randomized: SRH was asked as either a first health-related item after a few demographic questions (i.e., without a health context) or after a series of questions on chronic health conditions (i.e., within a health context). Whereas English-speaking respondents’ SRH reports remained consistent regardless of the question context, Spanish-speaking respondents’ reports were found to be unstable depending on the context. Specifically, Spanish-speaking respondents reported significantly and substantively better health on SRH asked within than without a health context. Reflecting LEP among Hispanics, their SRH rating was also affected by the question context. The question context effect is of concern in its own right. When the context interacts with interview language or respondents’ cultural background as in this example, it becomes even more important, because systematic incomparability is introduced.

We further examined the effect of SRH question contexts with the US elderly population. Using data from surveys conducted in both English and Spanish, we examined how the question context affects (1) the estimates of SRH for each linguistic group, (2) the comparisons of health between 2 linguistic groups, and (3) the predictive power of SRH for subsequent mortality. Because Spanish-language use is tightly related to ethnicity, we also included Hispanics and non-Hispanics in the study.

METHODS

We focused on the elderly population aged 50 years and older and used data from 2 sources: NHIS and the Health and Retirement Study (HRS). These 2 surveys are methodologically comparable in that they are conducted face to face with area probability samples and Spanish has been introduced as an interview language since 1992 in HRS and 1997 in NHIS. At the survey level, there are no mode-, sampling-, and language-related confounders that may dampen comparisons. The HRS is a panel survey targeting the elderly population aged 50 years and older, and NHIS is a cross-sectional survey targeting the general population. When focusing on those aged 50 years and older, both HRS and NHIS samples represent the same elderly population once sample adjustment weights are applied. Examination of basic demographic characteristics, such as age, gender, and education showed no difference between the 2 surveys.

At the SRH item level, the question wording is virtually identical in both English and Spanish. In English, the questions are “Would you say your health is excellent, very good, good, fair, or poor?” in HRS and “Would you say your health in general is excellent, very good, good, fair, or poor?” in NHIS. The Spanish versions are “¿Diría que su salud es excelente, muy buena, buena, regular, o mala?” in HRS and “¿Diría que, en general, su salud es excelente, muy buena, buena, regular, o mala?” in NHIS. The only methodological incomparability for SRH between HRS and NHIS is the question context. The HRS places SRH after demographic questions and before any health-related questions—therefore, without a health context. In NHIS, SRH is asked within a health context after a series of health limitation questions. Differences in SRH rates between the 2 surveys can be regarded mostly as a result of the differential question context effect. At the point when answering SRH, NHIS respondents have already gone through health limitation questions and, hence, have a chance to infer the meaning of health from these antecedent questions. On the other hand, HRS respondents have completed only demographic questions and need to define health on their own. In other words, the difference in the SRH measurement between the 2 surveys is that NHIS respondents are primed with meaning of health, but HRS respondents are not.

We carried out the analysis in 2 steps. The first analysis used NHIS and HRS from 2004. We compared SRH rates between the 2 surveys separately for English- and Spanish-speaking respondents, which indicated the magnitudes of context effects for each group and the difference in the context effects. A total of 16 709 and 816 respondents were interviewed in English and Spanish in HRS, respectively. The respective figures were 12 039 and 688 for NHIS. (NHIS interview language was summarized at the household level whereas HRS was measured at the person level.) As Spanish language usage is tightly linked to ethnicity in the United States, more than 97% of the Spanish-language sample in both surveys was Hispanic. We analyzed SRH rates separately for English-speaking and Spanish-speaking Hispanics to examine whether the context effect was related to language use among Hispanics. Roughly 1 out of 2 Hispanics was interviewed in Spanish (51.3% in HRS and 45.6% in NHIS). Mexicans constitute a large proportion of the Hispanic sample (59.7% in HRS and 55.5% in NHIS). We compared Mexicans by interview language to examine whether the context effect is specific to certain ethnic groups within Hispanics.

We then examined how the question context influences the level of health disparities by comparing the SRH rate of non-Hispanic Whites to that of Hispanics, non-Hispanic Blacks, and non-Hispanic other races within each survey and comparing the disparities between the 2 surveys. Because the population-level inference is of analytic interest, we conducted weighted analyses.

The second set of analyses examined the power of SRH for predicting mortality by its question context. By design, mortality data are collected for all sample persons in HRS, and the most up-to-date mortality status comes from 2008. Unlike HRS, NHIS itself does not collect mortality information directly, but the survey data can be linked to external mortality data.⁷³ The most recent mortality status available is for the year 2006 which includes NHIS respondents sampled through 2004. We used the mortality data and linked them to respective survey data from NHIS 1997–2004 and all cycles of HRS (1992–2008). For doing so, we took the first year when the sample person entered the panel as the baseline survey year for HRS and the sample year as the baseline for NHIS.

We predicted mortality by using SRH as a predictor in simple logistic regression as well as multivariate logistic regression that controlled for age, gender, education, and health condition. Because not all variables measured in these surveys are identical, we used different measures for the health condition variable: the total number of chronic conditions asked after SRH in HRS, and an indicator summarizing all the health limitations asked before SRH in NHIS. All variables in this analysis came from the baseline survey, except for mortality status. We fitted the models separately for those interviewed in English and Spanish, English-speaking Hispanics, and all Hispanics. We compared the estimated odds ratios between question contexts within linguistic or ethnic group. Because the ability to predict mortality is the most praised measurement utility of SRH, the relationships examined in the multivariate model outlined previously were expected to shed light on the “better” question context for SRH. The analysis involved data from different years, and the goal was not to represent the population but to understand the role of context effects on the relationship between SRH and subsequent mortality. Therefore, analyses were unweighted. In addition, we only included self-respondents in analyses to eliminate the effect of inconsistent proxy responses on health prediction.^74,75

RESULTS

The rate of combined “excellent,” “very good,” and “good” health for the population aged 50 years and older was 74.3% (SE = 0.4%) according to HRS and 80.0% (SE = 0.4%) according to NHIS. The difference in these rates was significant (P < .001). Asking SRH within a health context as in NHIS produced a slightly more positive health rating than asking SRH without a health context as in HRS.

Self-Rated Health Measures in Different Question Contexts

We further examined the question context effect by interview language separately in Table 1. Overall, the context effects were in the same direction for both groups in that HRS suggested a lower level of positive health than did NHIS. However, their magnitudes varied. For English speakers, the question context effect was smaller as the positive health reports were in the range of 75.4% to 80.5%. For Spanish speakers, the effect was much larger: depending on the question context, their positive health rate may be as low as 41.3% or as high as 68.3%, resulting in a 27.0 percentage-point difference. The context effects differed between English and Spanish speakers significantly (P < .001).

TABLE 1—

Percentage of Self-Rated Good Health: National Health Interview Survey and Health and Retirement Study for Ages 50 Years and Older, United States, 2004

Variable	HRS (SRH Asked Without Health Context) No., % (SE)	NHIS (SRH Asked Within Health Context) No., % (SE)	Difference Between HRS and NHIS, Percentage Points
Languagea
English speakers	16 343, 75.4 (0.5)	12 039, 80.5 (0.4)	-5.2**
Spanish speakers	801, 41.3 (2.2)	688, 68.3 (2.2)	-27.0**,b
Language of Hispanics and Mexicans
English-speaking Hispanics	735, 61.4 (2.5)	804, 76.1 (1.8)	-14.8**
Spanish-speaking Hispanics	779, 40.4 (2.2)	676, 68.0 (2.2)	-27.6**,c
English-speaking Mexicans	473, 60.3 (2.9)	463, 75.8 (2.3)	-15.5***
Spanish-speaking Mexicans	431, 33.8 (2.2)	351, 69.4 (3.0)	-35.6***,d
Race/ethnicity
Non-Hispanic Whites	12 937, 77.8 (0.4)	9346, 82.1 (0.4)
All Hispanics	1514, -26.0***,e	1481, -9.7***,e,f
Non-Hispanic Blacks	2410, -17.1***,e	1566, -14.0***,e
Non-Hispanic other races	268, -9.7**,e	365, -1.6eg

Note. HRS = Health and Retirement Study; NHIS = National Health Interview Study; SRH = self-rated health. The Excellent, Very Good, and Good categories were combined.

^aDifference between English and Spanish speakers by percentage points was HRS (SRH asked without health context) = -34.0 (P < .001) and NHIS (SRH asked within health context) = -12.2 (P < .001).

^bSignificantly different from the difference between HRS and NHIS for English speakers (P < .001).

^cSignificantly different from the difference between HRS and NHIS for English-speaking Hispanics (P = .003).

^dSignificantly different from the difference between HRS and NHIS for English-speaking Mexicans (P = .001).

^eDifference from non-Hispanic Whites by percentage points.

^fSignificantly different from HRS (P < .001).

^gSignificantly different from HRS (P = .047).

**P < .005; ***P < .001.

When we examined Hispanics by interview language (Table 1), the differences in positive health between HRS and NHIS for both English- and Spanish-speaking Hispanics were 14.8 and 27.6 percentage points (P < .001) but significantly larger for Spanish-speaking Hispanics (P < .003). When we examined Mexicans separately by interview language, we saw similar patterns of the question context effect as with overall Hispanics.

If we were to measure health disparities by using SRH, Hispanics would show significantly lower SRH in both surveys, as shown in Table 1 (P < .001). However, Hispanics’ and non-Hispanic Whites’ health would appear much more comparable in NHIS than HRS. Specifically, Hispanics’ positive health rate was lower than non-Hispanic Whites’ rate by 26.0 percentage points in HRS. The disparities significantly decreased by almost two thirds to 9.7 percentage points in NHIS (P < .001). The likely driver of this drastic discrepancy was SRH question contexts, which influenced responses by Hispanics and, in particular, Spanish-speaking Hispanics. The contexts did not appear to influence non-Hispanic Blacks as the difference in positive health rates between non-Hispanic Whites and non-Hispanic Blacks was similar at 17.1 and 14.0 percentage points based on HRS and NHIS, respectively. In addition, whereas the rate of positive health for non-Hispanic other races was less than 2 percentage points lower than that for non-Hispanic Whites in NHIS, the difference in HRS was almost 10 percentage points. These were significantly different (P = .047). (We also conducted analyses with 2006, 2008, and 2010 data and with age–gender adjustments. The results are not shown here because they provide the same conclusion.)

Self-Rated Health Question Context and Mortality Prediction

We explored whether it is better to ask SRH within or without a health context by examining the subsequent mortality prediction. The overall mortality rates were 332 (95% confidence interval [CI] = 326, 327) and 148 (95% CI = 146, 150) per 1000 from HRS and NHIS, respectively. The rates were 336 (95% CI = 329, 342) and 252 (95% CI = 235, 269) for non-Hispanic Whites and Hispanics from HRS and 154 (95% CI = 152, 157) and 103 (95% CI = 98, 108) for non-Hispanic Whites and Hispanics from NHIS, respectively. The gaps between the 2 surveys were not surprising, because HRS baseline survey data covered a longer time period (1992–2008) than did NHIS (1997–2004). (When we restricted HRS survey years to match those of NHIS, the mortality rates become fairly comparable at 124 [95% CI = 117, 131], 130 [95% CI = 121, 138], and 73 [95% CI = 57, 90] for the overall sample, non-Hispanic Whites, and Hispanics, respectively. Restricting HRS data to 1997–2004 would have been ideal; however, this would have reduced sample sizes dramatically from 28 811 to 8850, limiting analytical power. Therefore, we used all cycles of HRS data.)

The results from a simple logistic regression model in Table 2 show the odds ratio of those reporting positive health being deceased at later years significantly less than 1 across all subgroups regardless of the SRH question context. This suggests that SRH is a significant predictor of mortality. However, the level of predictive power varies. The odds ratios indicate that SRH predicts mortality at a significantly lower level for Spanish speakers than for English speakers (0.28 vs 0.58 in HRS and 0.35 vs 0.51 in NHIS). The gap in the prediction level appeared larger in HRS where SRH was asked without a health context.

TABLE 2—

Odds Ratios of Good Health in Logistic Regressions Predicting Subsequent Mortality for Linguistic and Ethnic Subgroups in 1997–2004 National Health Interview Survey and 1992–2008 Health and Retirement Study for Ages 50 Years and Older: United States

Variable	Simple Logistic Regression, OR (95% CI)	Multivariate Logistic Regression,a OR (95% CI)
English speakers
HRS (n = 27 636)	0.28* (0.26, 0.29)	0.42* (0.39, 0.45)
NHIS (n = 100 650)	0.35* (0.33, 0.36)	0.46* (0.45, 0.49)
Spanish speakers
HRS (n = 1245)	0.58* (0.45, 0.75)	0.92 (0.65, 1.29)
NHIS (n = 3656)	0.51* (0.42, 0.63)	0.71* (0.56, 0.89)
English-speaking Hispanics
HRS (n = 1325)	0.35* (0.27, 0.45)	0.54* (0.38, 0.76)
NHIS (n = 6448)	0.43* (0.37, 0.51)	0.58* (0.48, 0.71)
All Hispanics
HRS (n = 2543)	0.44* (0.37, 0.53)	0.71* (0.55, 0.90)
NHIS (n = 12 178)	0.46* (0.41, 0.51)	0.63* (0.55, 0.72)

Note. CI = confidence interval; HRS = Health and Retirement Study; NHIS = National Health Interview Study; OR = odds ratio. All predictors are from the baseline survey.

^aWe controlled for age, gender, education, and health condition and used the total number of chronic conditions ranging from 0 to 8 for HRS and an indicator describing whether limited in any way in NHIS.

*P < .01.

Table 2 shows the predictive power of SRH after we controlled for age, gender, educational attainment, and specific health measures in a multivariate logistic regression. Once we introduced these control variables, the odds ratios of SRH become closer to 1, indicating lowered predictive power. Nonetheless, SRH was still a significant predictor of mortality, echoing the literature. One exception was the HRS Spanish-language sample; SRH was no longer a significant predictor with an odds ratio ranging from 0.65 to 1.29 under the 95% CI, indicating that Spanish speakers who reported positive health were not significantly different from those reporting negative health in terms of subsequent mortality. On the other hand, SRH was a significant predictor for the NHIS Spanish-language sample with an odds ratio of 0.71 (95% CI = 0.56, 0.89). In summary, when SRH was asked within a health context, the predictability of SRH appeared consistent, and when SRH was asked without a health context, SRH was a significant independent predictor of mortality for English speakers but not for Spanish speakers.

DISCUSSION

Our study suggests that (1) Hispanics, especially Spanish-speaking Hispanics, are subject to the SRH question context at a systematically higher level than non-Hispanics and English speakers, (2) the question context effect is not necessarily driven by a particular Hispanic subgroup, and (3) if consistent mortality prediction is a criterion for determining the better question context, then priming respondents with meaning of health by asking SRH after specific health questions appears to be a better strategy.

Depending on the SRH question context, Hispanic health disparities appeared drastically different. This may explain some of the anomaly noted in the previous Hispanic health disparity studies using SRH from NHIS and the Behavioral Risk Factor Surveillance System.^8,10,69 These 2 surveys ask SRH in different contexts: within a health context in NHIS and without a health context in the Behavioral Risk Factor Surveillance System. (This appears to be incorrectly reported in Viruell-Fuentes et al.⁶⁹) In addition, non-Hispanic other races, who are mostly Asians, show larger disparities in HRS than in NHIS. If we consider that Asian languages are not provided as standard interview languages in these surveys and LEP among Asians, it is conceivable that the context effect may play a larger role if LEP Asians are included in these surveys.

It is not surprising that existing research on the question context for SRH fails to provide clear guidance, as the data were collected entirely from English speakers.^76–78 However, the current recommendation in the literature is to ask SRH without a health context.^79–81 This reflects a general norm in survey methodology literature that it is advantageous to ask a general concept question before specific ones.⁸² The results of this study, however, challenge this view by showing that asking SRH after specific health questions improves the comparability of SRH across different population subgroups and the mortality prediction. In fact, a quality-of-life instrument, Quality of Life Questionnaire–Core 30, introduced by the European Organization for Research and Treatment of Cancer⁸³ places SRH at the end of the questionnaire to promote the usage of answers to preceding questions as a frame of reference for SRH.⁸⁴

There have been numerous studies examining the measurement properties of SRH with psychometric approaches focusing on the item-by-item reliability and validity.^63,85–87 Unfortunately, these approaches do not consider 2 important factors. First, the construct of health is neither clear nor fixed. In fact, there is little agreement on the conceptualization of health among health professionals.⁸⁸ People may have only a vague notion about what health means.^35,89–92 This may explain why standardizing respondents’ interpretation of SRH by priming them with specific health-related questions increases its utility for mortality prediction, as this study suggests. Second, focusing on the SRH item alone overlooks the dynamic interactions among questions that influence measurement properties. Adding a cultural element further complicates the picture, but the focus of cross-cultural assessment of SRH has yet to extend beyond the item-level translation.^63,93–95 Satisfying these technical criteria alone does not guarantee functional comparability in cross-cultural measurements.^96–98

This study presents evidence for an interaction between the question context effect and respondents’ linguistic or ethnic backgrounds, but its causes remain to be explored. One may hypothesize that Hispanics’ conceptualization of health does not align with that of non-Hispanics. As an alternative, respondents’ cognitive processes used to comprehend SRH as a question and formulate their answers may be hypothesized to differ between Hispanics and non-Hispanics. A qualitative approach, such as in-depth cognitive interviews, may serve as a reasonable platform for testing these hypotheses as they provide rich insight into respondents’ cognitive strategies when faced with a survey task.^36,99,100 (In fact, this is exactly the theme of NCHS conference on SRH mentioned earlier.) In doing so, it is imperative to consider SRH as an element within a questionnaire rather than in isolation from other question items. Furthermore, other question contexts different than the 2 examined in this study may be introduced in the investigation because the presence and magnitudes of the SRH question context effects may change depending on the type of contexts. For example, asking SRH immediately after mental health questions may result in different context effects. In addition, this study can be extended beyond the US Hispanic population.

Limitations

There are a few limitations with the data. First, NHIS and HRS were not conducted for the purpose of the study and may not be completely comparable. For instance, the interview language is measured at the household level in NHIS but at the person level in HRS. We acknowledge that a randomized experiment on SRH question contexts that includes all age, linguistic, and racial/ethnic groups and can be linked to subsequent mortality status would be ideal for this study. Unfortunately, we are not aware of such a data source. Although the data of this study are not ideal, the interaction between the question context effect and respondents’ linguistic or ethnic backgrounds in this study was replicated in the 2006, 2008, and 2010 data. Moreover, it is consistent with the interaction reported in Lee and Grant,⁷² which used a randomized experiment. Therefore, it is unlikely that the question context pattern in this study was a result of some unknown confounding factors or sampling oddity.

Second, the results are applicable only to the elderly population. However, it should be noted that the findings match those in Lee and Grant,⁷² which included all adult age groups. If one considers the frequent usage of SRH in gerontology, the age restriction in the study may provide a higher level of relevance. Lastly, we used mortality prediction to determine the “better” question context. We note that this is just one of many ways to evaluate measurement utilities of SRH. We examined the relationship between SRH and the number of doctor visits but did not find the same type of support for the SRH question context. Therefore, other criteria may be explored further to determine a “better” context.

Conclusions

Overall, our findings challenge the current practice of SRH measurement that implicitly assumes the measurement properties examined with the English-speaking population apply to other linguistic groups or LEP racial/ethnic groups. We found that the subtle changes in the SRH question contexts drastically changed the inference about Hispanics’ health, and their health disparities and the level of mortality prediction for Spanish speakers. To better utilize SRH for a diverse population, in-depth studies examining measurement properties of SRH with not only Hispanics but also other racial/ethnic groups are needed. Because the minority population in the United States is projected to grow,¹⁰¹ such information is vital in increasing the utility and validity of SRH for monitoring the population health.