Benchmarking Weighted Estimates of Social Determinants of Health in the National Health and Aging Trends Study (NHATS)

For reasons of feasibility and efficiency, studies designed to be nationally representative of a specific target population often use complex sampling strategies and apply weights to adjust for differential probabilities of selection and nonresponse.¹ These weights are primarily based on demographic characteristics, with socioeconomic factors, particularly those that are not available in administrative records, often underrepresented in the weighting process. The goal of the current study was to benchmark the weighted prevalence estimates of a set of individual-level and contextual-level indicators of socioeconomic disadvantage among community-living older persons in the National Health and Aging Trends Study (NHATS) against nationally representative values from the US Census Bureau.

METHODS

On September 30, 2010, the National Health and Aging Trends Study (NHATS) drew a random sample of persons 65 or older in the contiguous US from the Medicare enrollment file.² Medicare beneficiaries were selected using a stratified, multistage sampling design that included geographic clustering and oversampling of the oldest age groups and non-Hispanic Blacks.³ Round 1 assessments, completed from May through November 2011, yielded a sample of 8,245 persons with a 71% weighted response rate.

Measures

The individual-level indicators of socioeconomic disadvantage included self-reported educational level and Medicaid coverage. The contextual-level indicator of socioeconomic disadvantage was the area deprivation index (ADI), which includes 17 education, employment, housing quality, and poverty measures from the American Community Survey (ACS), a nationally representative survey conducted by the US Census Bureau.⁴ These measures are weighted and summed to yield a score for each neighborhood at the census-block level.⁵ We linked block group data from NHATS⁶ and 2010 Census to the 2013 ADI file, representing ACS data from 2009 to 2013, using the Neighborhood Atlas.⁷ The ADI scores were classified into quintiles, with higher scores denoting greater socioeconomic disadvantage and highest quintile denoting neighborhood disadvantage. Because block group data were not available, ADI scores could not be obtained for 104 (1.4%) of the 7,609 community-living participants in NHATS or 632,292 (1.6%) of the 38,725,224 age-eligible community-living persons in the 2010 Census.

Statistical Analysis

For the NHATS sample, weighted percentages were calculated for the demographic characteristics and individual-level and contextual-level indicators of socioeconomic disadvantage. These estimates incorporated the NHATS analytic weights, which account for differential probabilities of selection (based on age, race, and geography), strata (based on region and county-level poverty and racial composition), and clustering elements (based on zip codes).⁸ As a reference standard, we used the 2008–2012 ACS 5-year estimates from the Public Use Microdata (PUMS) file with person-level weights⁹ (for the demographic characteristics and individual-level indicators) and 2010 Census data (for the contextual-level indicator), focusing on community-living persons 65 or older in the contiguous US. Unlike the ACS, the 2010 Census does not include data on socioeconomic indicators or identify persons 90 or older. Unlike the 2010 Census, the ACS (PUMS file) does not include census block group data, precluding linkage to the Neighborhood Atlas to obtain ADI scores. Persons without Medicare were omitted from the ACS but not the 2010 Census, based on the availability of data. For age and sex comparisons with ACS values, NHATS weights were adjusted to the 2010 Census frame.¹⁰

RESULTS

For the demographic characteristics (Table), the weighted NHATS estimates were comparable to those of the ACS for age (other than the 65–69 group), female sex, and race/ethnicity. For the two individual-level indicators of socioeconomic disadvantage, the weighted NHATS sample included a lower percentage of persons with beyond high school education, a higher percentage of college graduate or beyond, and a lower percentage of persons with Medicaid, relative to the ACS.

Table.

Characteristics of Community-living Older Persons in NHATS, ACS and 2010 Decennial Census^a

Characteristic	NHATS b	ACS c			2010 Census d
Number	34,742,661	37,831,933			38,092,932
	Percent (95% CI)		Difference (95% CI) e	P-value f
Demographic
Age group, years				0.001
65–69	31.7 (31.1, 32.3)	30.6 (30.5, 30.6)	1.1 (0.5, 1.7)
70–74	23.3 (22.6, 23.9)	23.9 (23.9, 24.0)	−0.6 (−1.3, 0.1)
75–79	18.2 (17.6, 18.8)	18.6 (18.6, 18.7)	−0.4 (−1.0, 0.2)
80–84	14.2 (13.7, 14.6)	14.3 (14.2, 14.3)	−0.1 (−0.6, 0.4)
85–89	8.7 (8.3, 9.0)	8.4 (8.4, 8.5)	0.3 (−0.1, 0.7)
90 or older	4.0 (3.7, 4.4)	4.2 (4.2, 4.3)	−0.2 (−0.6, 0.2)
Female	56.5 (56.2, 56.9)	56.6 (56.5, 56.6)	0 (−0.4, 0.4)	1.0
Race/ethnicity				0.394
Hispanic	6.6 (5.5, 7.6)	6.7 (6.6, 6.7)	−0.1 (−1.2, 1.0)
Non-Hispanic Black	8.1 (7.9, 8.4)	8.2 (8.2, 8.3)	−0.1 (−0.4, 0.2)
Non-Hispanic White	80.7 (79.3, 82.2)	80.9 (80.8, 80.9)	−0.2 (−1.7, 1.3)
Other g	3.4 (2.6, 4.3)	4.2 (4.2, 4.3)	−0.8 (−1.7, 0.1)
Missing	1.1 (0.6, 1.7)	N/A	N/A
Socioeconomic Disadvantage
Individual level
Educational level				<0.001
Less than high school	21.5 (19.8, 23.2)	21.6 (21.5, 21.7)	−0.1 (−1.8, 1.6)
High school or equivalent	35.0 (33.4, 36.5)	34.3 (34.2, 34.3)	0.7 (−0.9, 2.3)
Beyond high school h	18.5 (17.3, 19.6)	22.7 (22.6, 22.7)	−4.2 (−5.4, −3.0)
College graduate or beyond	23.8 (21.7, 25.8)	21.5 (21.4, 21.6)	2.3 (0.2, 4.4)
Missing	1.3 (0.7, 1.8)	N/A	N/A
Medicaid coverage i	11.8 (10.6, 13.0)	15.6 (15.5, 15.6)	−3.8 (−5.0, −2.6)	<0.001
Missing	2.5 (2.0, 3.1)	N/A	N/A
Contextual level
Quintiles of ADI Scores j
Lowest (best)	21.2 (16.7, 25.7)				20.7
Second	21.1 (18.2, 24.0)				21.7
Third	21.7 (18.8, 24.6)				21.5
Fourth	20.2 (17.9, 22.5)				19.5
Highest (worst) k	15.8 (13.6, 18.1)				16.6

Notes. NHATS = National Health and Aging Trends Study; ACS = American Community Survey; CI = confidence interval; N/A = not applicable; ADI = Area Deprivation Index

^aFor each of the data sources, community-living excluded persons living in nursing homes. Values were accompanied by 95% confidence intervals for the NHATS sample and ACS, but not the 2010 Census, which was based on a complete enumeration rather than a random sample of the population.

^bCharacteristics were obtained from the Round 1 assessment. Estimates incorporated the analytic weights, which account for differential probabilities of selection (based on age, race, and geography), strata (based on region and county-level poverty and racial composition), and clustering elements (based on zip codes). For age and sex comparisons with ACS values, NHATS weights were adjusted to the 2010 Census frame.

^cBecause data available from the 2010 Census do not include socioeconomic indicators or identify persons 90 or older, person-level weighted values were obtained from the 2008–2012 5-year ACS Public Use Microdata Sample (PUMS) file, which includes annual nationally representative surveys conducted by the US Census Bureau. Persons without Medicare were excluded. ADI scores could not be obtained because of the absence of census block group data.

^dData were not available to exclude age-eligible persons without Medicare.

^eCategory-level differences (percentage points) between the NHATS and ACS values are shown with 95% CIs based on pooled standard errors.

^fFor each characteristic, an omnibus chi-square test assessed overall differences in category distributions. Missing NHAT values are shown descriptively but were excluded from the tests.

^gIncludes those who reported their race/ethnicity as Asian, American Indian, Native Hawaiian, Pacific Islander, other, do not know, or more than 1 race and ethnicity.

^hAssociate’s degree is classified as beyond high school for both NHATS and ACS. The ACS did not include a category for vocational/technical/business/trade certificate. Although this category was classified by NHATS as beyond high school, we have included it in the high school or equivalent category to enhance comparability with the ACS. Without this change, the NHATS values would be 27.4 (26.2, 28.7) for high school or equivalent and 26.0 (24.8, 27.2) for beyond high school.

ⁱThe NHATS and ACS questions differed slightly. For NHATS, the question was “Medicaid [, also known as [STATE NAME FOR MEDICAID PROGRAM],] is a state program for low-income people or for people on public assistance. Sometimes people with very large medical bills are also covered by Medicaid. [Are you/Is study participant] now covered by [Medicaid/[STATE NAME FOR MEDICAID PROGRAM]]?” For ACS, the question was “Is this person CURRENTLY covered by any of the following types of health insurance or health coverage plans?” The relevant response option was “Medicaid, Medical Assistance, or any kind of government-assistance plan for those with low incomes or a disability?”

^jFor both NHATS and 2010 Census, block group data were linked to the 2013 ADI file using the Neighborhood Atlas. Because block group data were not available, ADI scores could not be obtained for 104 (1.4%) of the 7,609 community-living participants in NHATS or 632,292 (1.6%) of the 38,725,224 age-eligible, community-living persons in the 2010 Census.

^kDenotes neighborhood disadvantage.

For all quintiles of ADI scores, values from the weighted NHATS sample were comparable to those of the 2010 Census, with the largest, albeit not statistically significant difference observed for the highest quintile (15.8% vs. 16.6%), denoting a slightly lower likelihood of neighborhood disadvantage.

DISCUSSION

In this study, we found that the weighted prevalence estimates of two individual-level indicators of socioeconomic disadvantage were lower in NHATS participants relative to nationally representative values from the US Census Bureau. Most notably, NHATS participants were more likely to be college graduates and less likely to have Medicaid coverage. These differences are unlikely to be attributable to differences in the demographic compositions of the two populations, which were comparable. However, the weighted NHATS values closely approximated the 2010 Census values for the ADI quintile scores, including neighborhood disadvantage. Because it is readily accessible based on 9-digit zip codes and strongly associated with individual-level indicators of socioeconomic disadvantage, the ADI could be a valuable factor for sampling and weighting in population-based studies to enhance the representativeness of participants on key social determinants of health.

To our knowledge, no prior study has directly benchmarked NHATS survey responses against census external standards. Nonetheless, this study has several potential limitations. Based on the availability of data for the demographic characteristics and individual-level and contextual-level indicators of socioeconomic disadvantage, the reference standard for comparisons with the weighted NHATS sample included two related but distinct sources of census data—the ACS and 2010 Census. Second, information on Medicaid coverage and education was self-reported, and the questions, categories, and time frames differed. For Medicaid, the question was broader in ACS than NHATS, which may have contributed to its higher prevalence. However, the other elements of the Medicaid question in ACS—Medical Assistance, or any kind of government-assistance plan for those with low incomes or a disability—are much less applicable in older than younger persons. For education, the ACS did not include a category for vocational/technical/business/trade certificate. While this difference may have affected estimates for the high school or equivalent and beyond high school groups, it should have had no effect on the college graduate or beyond group. Although 2011 (NHATS) is between 2008 and 2012 (ACS), the slight difference in time frames between these two sources of data could have also led to spurious differences in educational attainment. Third, it was not possible to exclude persons without Medicare from the 2010 Census. According to NHATS, only about 4% of age-eligible persons did not have Medicare in 2010, including those who opted to defer enrollment and those who did not meet program eligibility.² Fourth, ADI values were not available for comparably small percentages of NHATS participants (1.4%) and older persons in the 2010 Census (1.6%). Whether or how these limitations affected our results is uncertain.