Improvements in Uninsurance Estimates for Fully Imputed Cases in the Current Population Survey Annual Social and Economic Supplement

Abstract

In 2019, the Current Population Survey Annual Social and Economic Supplement introduced updates to data processing, including to the imputation of health insurance for cases with no reported health insurance information. This article examines the impact on health insurance estimates of modernized imputation procedures that were part of a redesign of the Current Population Survey Annual Social and Economic Supplement. We use descriptive analysis and multinomial logistic regression to examine whether imputation biases estimates of health insurance coverage using data from the 2017 Current Population Survey Annual Social and Economic Supplement, which used legacy methods, and the 2017 Current Population Survey Annual Social and Economic Supplement Research File, which debuted the processing redesign. We find that cases with all of their health insurance information imputed using legacy methods were more likely to be uninsured or to be covered by multiple insurance types after adjusting for factors associated with having missing data. With the processing updates, fully imputed cases do not differ from other cases in their likelihood of being uninsured, having private coverage, having public coverage, or in having private and public coverage. Processing updates in the Current Population Survey Annual Social and Economic Supplement improved data quality by increasing the percent of people with any health insurance coverage and decreasing the percent of people with multiple types of coverage, especially among fully imputed cases.

• What do we already know about this topic?

• Before the processing system redesign, imputation in the Current Population Survey Annual Social and Economic Supplement (CPS ASEC) contributed to biased estimates of health insurance.

• How does your research contribute to the field?

• This study is the first to show that redesigned imputation methods help to address previously noted biases in the CPS ASEC’s estimates of the uninsured.

• What are your research’s implications toward theory, practice, or policy?

• The redesign of the CPS ASEC health insurance imputation improves the survey’s data quality and estimates of the health insurance coverage in the US population.

The Current Population Survey Annual Social and Economic Supplement (CPS ASEC) is an important and widely used source of information on the nation’s income, labor force participation, and health insurance status. Data from the survey have been used widely by the research and policy community for a variety of purposes,^1,2 and the CPS ASEC has been noted for its many strengths including its large sample size, rich set of demographic, social, and economic measures, and timely release of data.

Despite these strengths, the CPS ASEC also had some notable limitations. Namely, in past years, the survey had uninsured estimates higher than other federal surveys and higher reports of multiple types of coverage.^3-6 To address these limitations and improve data quality, the CPS ASEC was redesigned in 2 stages.^7,8 First, in 2014, the questionnaire was redesigned for calendar year 2013. While these new health insurance questions improved data quality,^9,10 the CPS ASEC continued to underestimate health insurance coverage as the data processing system could not take full advantage of these improvements. In 2019, an “updated” processing system was introduced to incorporate the more detailed information from redesigned income, health insurance, and demographic content.^11,12 The 2017 CPS ASEC Research File marks the first time these improvements in processing and imputation were fully implemented.

In this article, we focus on how one aspect of the updated processing system, the imputation procedures for cases with fully missing health insurance information, affected health insurance estimates in the CPS ASEC. This imputation was a key component of updates to the health insurance content in the CPS ASEC. We focus on the prevalence of coverage in 2016 using data from 2 files. The first, the 2017 CPS ASEC (which we call the “production file”), features data collected using the improved health insurance questions but processed using procedures that predate the questionnaire redesign (called the “legacy” processing system). The second file, the 2017 CPS ASEC Research File, features data collected using the improved health insurance questions and processed using procedures that were designed to use the detailed information obtained from the redesigned questionnaire (called the “updated” processing system). Both sets of files are based on the same underlying data, but the Research File reflects changes to data processing and imputation procedures. This article is the first to investigate whether the prevalence of major insurance types differs by imputation status with the updated processing system.

Overview of the CPS ASEC Data Collection and Processing Systems

The Current Population Survey (CPS) is a monthly survey conducted by the Census Bureau and sponsored by the Bureau of Labor Statistics. The Annual Social and Economic Supplement (ASEC) is added to the CPS survey from February through April and collects information on a variety of topics including income from the prior (calendar) year as well as information on health insurance and health status.¹³ The data collected from this instrument is subsequently processed, that is, reformatted, imputed, and logically edited as necessary and recoded for release on a microdata file. The data processing system helps to adjudicate which cases are in sample, extract data from the instrument, reconcile conflicting reports, and impute missing or incomplete data.

Imputation procedures are a particularly important part of processing. In recent years, rates of item non-response to federal surveys, including the CPS ASEC, have increased.¹⁴ For cases where a full or partial interview was conducted, the Census Bureau imputes missing information for demographic, income, and health insurance measures.¹³ While income and demographic questions in the CPS ASEC have also undergone recent questionnaire and processing redesigns,^15,16 the focus of this article is on changes to health insurance.

In the 2017 CPS ASEC, 22% of cases were fully imputed, that is, all of their health insurance coverage information was imputed. Given that about 1-in-5 CPS ASEC cases has fully imputed coverage, and the procedures for imputing this information changed as part of the updated processing system, it is important to evaluate the quality of these procedures.

Following a decade of research and evaluation, the 2014 CPS ASEC debuted a fully redesigned health insurance questionnaire.^9,10 However, the redesigned questionnaire was processed using existing (legacy) methods of extracting, reformatting, and imputing the data. With the 2017 CPS ASEC Research File, the Census Bureau unveiled an updated processing procedure designed to complement the redesigned health insurance questionnaire, in addition to redesigns of the income questionnaire (fully implemented in the 2015 CPS ASEC) and the demographic questionnaire (fully implemented in the 2017 CPS ASEC).^15,16

For calendar-year 2016 estimates, the Census Bureau released 2 files based on the same underlying data. The first file, the 2017 CPS ASEC production file, used legacy imputation and processing procedures that predate the redesign of the CPS ASEC health insurance questionnaire. The second file, the 2017 CPS ASEC Research File, features updated processing procedures designed to maximize information obtained from the redesigned health insurance questionnaire and improve the imputation of missing data.⁸

Health Insurance Imputation Procedures in the CPS ASEC

In the CPS ASEC, the Census Bureau imputes missing health insurance data using hotdeck imputation.¹⁷ Hotdeck imputation is a type of model-based imputation which takes information from cases with reported data, “donors,” and assigns it to cases with missing data “recipients.”¹⁸ Donors and recipients are matched so that they are comparable on selected demographic and social characteristics. Hotdeck imputation, like many other imputation strategies, assumes that data are missing at random.¹⁴ That is, controlling for relevant observed characteristics, the uninsured rate for imputed cases should be comparable to cases with reported values.^5,18

With legacy processing procedures, imputation of health insurance information for individuals missing all health insurance information consisted of 2 parts.¹⁹ First, public insurance was imputed prior to private insurance. This information, in turn, was factored into the hotdecks when imputing private coverage. Second, for private coverage, policyholder status was first imputed. If a person was identified as a policyholder, they were then imputed to have either an individual or family plan. For policyholders of family plans, coverage was copied to other family members. All non-policyholders were also allowed to be imputed as dependents on health insurance plans. Variables used in the hotdecks accounted for many different family configurations, income levels, and, importantly for private health insurance, a measure of family size. This approach, in other words, imputed each coverage type separately and independently, increasing the prevalence of multiple coverage types among fully imputed cases. That is, there was a higher-than-expected prevalence of dual health insurance coverage for fully imputed cases.

As part of the updated processing system, the health insurance imputation procedure was overhauled to address these limitations for fully imputed cases.⁸ Starting with the 2017 CPS ASEC Research File, the Census Bureau constructed health insurance units (HIUs) based on individuals who were eligible to share insurance coverage using guidelines jointly developed by the Census Bureau and SHADAC.²⁰ Some groups of people considered eligible to share coverage include parents and children (when the child is under the age of 26 and does not have dependents living in the household), as well as spouses. If no one in the HIU reported health insurance information, data for the entire HIU was imputed simultaneously by matching this recipient HIU with a donor HIU with health insurance information.

This procedure addressed limitations of prior imputation strategies in 3 key ways. First, simultaneous imputation for all members of a HIU allows the distribution of coverage to better match across imputed and non-imputed cases at the person level and HIU level. Second, the number of characteristics included in the hotdeck has been expanded. For example, the hotdecks now include additional information about nativity and the age of adults in the household. Third, the revised procedures allow better matching of multiple insurance types across imputed and reported cases. Rather than imputing separately by type, the updated processing system now imputes concurrently across types.

Analytic Approach

To examine how changes to the health insurance imputation affected estimates of coverage, we use the 2017 CPS ASEC production file and 2017 CPS ASEC Research File. In this analysis, we make 2 adjustments; first, we only include people in-sample for both files.²¹ The updated processing system does not produce coverage information for infants who were born after the calendar-year reference period, and the imputation of certain demographic characteristics moved a small number of cases in or out of sample.^8,21 This restriction allows us to examine changes net of universe differences. Second, we treat all military coverage as public coverage for the purposes of this analysis.⁸ This reclassification of TRICARE allows us to compare changes in the prevalence of public and private coverage due to processing changes, not changes in categorization.

Our analysis consists of 2 parts. First, we perform a descriptive analysis of health insurance status for fully imputed cases and all other cases (either fully reported or partially imputed) in the both CPS ASEC files. We examine health insurance at the person level, estimating the prevalence of any, private, public, and combined (private and public) health insurance by imputation status across files. Although not the focus of this article, presenting estimates of health insurance by imputation status allows us to highlight which aspects of the processing system may have contributed to observed changes in estimates.

Because one major change with the updated processing system for fully imputed cases is the imputation of health insurance information at the HIU level, rather than the individual level, we also examine health insurance coverage patterns in households that are fully imputed versus those with some reported data across files. We examine variation at the household level, as HIUs were not available in the production file. Specifically, we look at the percent of households where everyone in the household is (1) uninsured, (2) covered by private insurance, (3) covered by public insurance, (4) covered by a combination of private and public insurance, or (5) has some other coverage configuration (eg, one person is uninsured and another has private insurance) by household imputation status and household size.

We pay particular attention to this final insurance category (some other coverage configuration) because we expect that legacy imputation procedures, which imputed at the person level, may have inflated the percent of households where people were covered by different types of insurance. We expect that these differences will be more pronounced in households that had more people with imputed data. The updated processing addresses this limitation by imputing coverage at the HIU level, which should reduce differences by imputation status in the percent of households where people are covered by different health insurance types.

We next move to a regression-based analysis. We perform a multinomial logistic regression analysis, examining the association between imputation status and health insurance coverage. Specifically, we examine 3 outcomes: (1) no insurance (reference category), (2) public insurance alone or in combination with private insurance, (3) private insurance. As health insurance is almost universal for adults over the age of 65, and individuals under the age of 26 are eligible to be covered as dependents, we restrict our sample to people under the age of 65 and run separate models for people under age 26 and adults aged 26 to 64. The primary covariate of interest in these models is a dichotomous variable indicating whether a person has all of their health insurance coverage imputed. Models for people under the age of 26 include age, race, sex, self-rated health, household poverty status, citizenship status, and country of birth as controls. Models for adults aged 26 to 64 also account for marital status, educational attainment, disability status, and labor force participation.

For this model, we report the relative risk ratio (RRR) between imputation status and insurance coverage. As having no insurance is the reference category, this RRR represents the ratio of the probability of having an insurance type over the probability of being uninsured by imputation status, that is, for the fully imputed compared to those with some reported information. An RRR significantly different from 1.00 indicates bias in the imputation procedure. If the RRR is significantly below 1.00, then people with fully imputed information are less likely to have a specific insurance type versus being uninsured. If the RRR is significantly above 1.00, then people with fully imputed information are more likely to have a specific insurance type versus being uninsured. An RRR not significantly different from 1.00 indicates that the imputation procedure is unbiased. That is, to be consistent with the missing-at-random assumptions of hotdeck imputation, being fully imputed should not be associated with insurance status after controlling for relevant respondent characteristics. This analytic approach is consistent with previous research evaluating the quality of imputation in the CPS ASEC.^5,19

Results

Table 1 shows the person-level distribution of insurance coverage by imputation status across the 2017 CPS ASEC Research File and production file. Updates to the processing system attenuate differences between fully imputed and not fully imputed cases. Consistent with prior work,^5,19 the production file shows large differences in the prevalence of any, public, private, and combined public and private coverage by imputation status at both the person level and household level. For example, coverage was 5.76 percentage points lower for fully imputed cases than for other cases: 86.82% of fully imputed cases had coverage, compared with 92.58% of not fully imputed cases. Differences by imputation status were especially large for estimates of the percentage of people with both public and private coverage. Fully imputed cases were 8.83 percentage points more likely to have this coverage pattern.

Table 1.

Health Insurance for People by Imputation Status in the 2017 Current Population Survey Annual Social and Economic Supplement Production and Research Files.

	Production File					Significance	Research File					Significance
	Fully imputed		Not fully imputed		Fully imputed-not fully imputed		Fully imputed		Not fully imputed		Fully imputed-not fully imputed
Coverage Type	%	SE	%	SE	%		%	SE	%	SE	%
Any coverage	86.82	0.29	92.58	0.11	−5.76	***	91.61	0.24	92.29	0.11	−0.68	**
Private coverage only	48.65	0.45	55.66	0.23	−7.01	***	55.98	0.41	55.73	0.24	0.25
Public coverage only	17.82	0.32	25.40	0.23	−7.58	***	26.08	0.36	26.24	0.24	−0.16
Combination of private and public coverage	20.35	0.34	11.52	0.12	8.83	***	9.54	0.21	10.32	0.12	−0.77	***

Source. US Census Bureau, 2017 Current Population Survey Annual Social and Economic Supplement and 2017 Current Population Survey Annual Social and Economic Supplement Research File.

^*P < .05. **P < .01. ***P < .001.

In the Research File, differences by imputation status were greatly attenuated. Specifically, 91.61% of fully imputed cases had insurance coverage; 0.68 percentage points lower than other cases. Differences in multiple coverage rates by imputation status were also significantly reduced; in contrast to the production file, fully imputed cases in the Research File were less likely to have a combination of private and public coverage: 9.54% of imputed cases had both private and public coverage compared with 10.32% of cases with some reported health insurance data. There were also no significant differences in the prevalence of private coverage only and public coverage only.

Table 2 shows distributions of insurance coverage by imputation status at the household level. The production file which used person-level imputation shows large differences between fully imputed and other households for most insurance types examined, with the largest differences found in households containing 3 or more people. The uninsured rate for fully imputed and not fully imputed households differed across all household sizes. Also notable, the production file showed large differences by imputation status in the percent of households that contained people covered by different insurance types. In 2-person households, fully imputed cases were 6.48 percentage points more likely to have different coverage types; in households with 3 or more people, fully imputed cases were 13.14 percentage points more likely to have this profile.

Table 2.

Health Insurance for Households by Imputation Status and Household Size in the 2017 Current Population Survey Annual Social and Economic Supplement Production and Research Files.

	Production file				Fully imputed-non-fully imputed		Research file				Fully imputed-non-fully imputed
	All household members fully imputed		Not all household members fully imputed				All household members fully imputed		Not all household members fully imputed
Household and Coverage Type	%	SE	%	SE			%	SE	%	SE
1-person household
Uninsured	12.44	0.6	6.23	0.23	6.21	***	9.62	0.58	6.23	0.23	3.39	***
Member covered by public insurance only	30.92	1.00	32.35	0.52	−1.43		33.90	1.08	33.07	0.52	0.82
Member covered by private insurance only	35.22	1.00	42.28	0.48	−7.06	***	40.44	0.95	42.07	0.47	−1.63
Member covered by private and public insurance	21.41	0.82	19.14	0.39	2.27	*	16.04	0.73	18.63	0.39	−2.59	**
Other insurance coverage pattern	—	—	—	—	—	—	—	—	—	—	—	—
2-person household
Fully uninsured	7.45	0.46	2.78	0.13	4.67	***	3.25	0.29	2.95	0.13	0.29
All members covered by public insurance only	16.06	0.64	17.03	0.37	−0.97		17.60	0.62	17.66	0.37	−0.06
All members covered by private insurance only	31.52	0.87	42.06	0.40	−10.54	***	41.18	0.82	42.17	0.39	−0.98
All members covered by private and public insurance	14.45	0.64	14.08	0.30	0.37		12.56	0.52	13.86	0.28	−1.31	*
Other insurance coverage pattern	30.52	0.83	24.05	0.35	6.48	***	25.41	0.69	23.36	0.37	2.05	*
3+-person household
Fully uninsured	3.44	0.27	2.24	0.12	1.20	***	2.33	0.22	2.63	0.12	−0.30
All members covered by public insurance only	5.62	0.33	11.26	0.25	−5.64	***	11.11	0.42	12.08	0.27	−0.97	*
All members covered by private insurance only	34.27	0.67	48.90	0.36	−14.64	***	47.58	0.75	49.41	0.35	−1.84	**
All members covered by private and public insurance	7.55	0.38	1.62	0.10	5.93	***	0.79	0.12	0.96	0.07	−0.18
Other insurance coverage pattern	49.11	0.67	35.97	0.36	13.14	***	38.2	0.70	34.91	0.34	3.28	***

Source. US Census Bureau, 2017 Current Population Survey Annual Social and Economic Supplement and 2017 Current Population Survey Annual Social and Economic Supplement Research File.

^*P < .05. **P < .01. ***P < .001

In the Research File, there were fewer significant differences in household health insurance coverage patterns. Consistent with the shift from an individual-level to HIU-level imputation, there were no significant differences in the uninsured rate for 2- and 3-person households. A significant difference remained for 1-person households, but the difference is of a smaller magnitude than in the production file. The Research File also shows some significant differences in households where people within the household have different coverage types. In 2-person households, fully imputed cases were 2.05 percentage points more likely to have different insurance types, and 3-person households were 3.28 percentage points more likely to have this profile. These differences between fully imputed and other households are smaller than in the production file and are perhaps expected as the new imputation procedure simultaneously imputes information for people eligible to share health insurance coverage and some large households may contain multiple HIUs.

To detect whether either imputation procedure is biased, a regression-based analysis is needed. Table 3 shows results from the multinomial logistic regression analysis predicting coverage type. In the production file, fully imputed cases were less likely to have public coverage, alone or in combination with private, or private coverage, even after adjusting for other factors that should predict insurance status. For people under the age of 26 with imputed data, the RRRs were 0.78 for any public insurance and 0.54 for any private insurance, relative to being uninsured. For adults between the ages of 26 and 64, the RRRs for fully imputed cases were 0.62 for public insurance alone or in combination and 0.41 for having exclusively private insurance. In contrast, the Research File shows no such relationship between imputation status and insurance status. The RRRs for imputation status do not significantly differ from 1.00 for any of the coverage patterns examined or for any age group.

Table 3.

Association of Imputation Status With Health Insurance Status in the 2017 Current Population Survey Annual Social and Economic Supplement Production and Research Files: Results from Multinomial Logistic Regression Models.

	Production File		Research File
Characteristic	Ages 0 to 25	Ages 26 to 64	Ages 0 to 25	Ages 26 to 64
Public insurance alone or in combination (relative to uninsured)
RRR: fully imputed	0.78***	0.62***	0.99	0.97
	(0.05)	(0.02)	(0.06)	(0.04)
Private insurance (relative to uninsured)
RRR: fully imputed	0.54***	0.41***	0.89	0.95
	(0.03)	(0.02)	(0.06)	(0.04)
Controls
Age	X	X	X	X
Race	X	X	X	X
Sex	X	X	X	X
Household income-to-poverty ratio	X	X	X	X
Self-rated health	X	X	X	X
Citizenship status	X	X	X	X
Nativity	X	X	X	X
Disability status		X		X
Marital status		X		X
Labor force participation		X		X

Source. US Census Bureau, 2017 Current Population Survey Annual Social and Economic Supplement and 2017 Current Population Survey Annual Social and Economic Supplement Research File.

Note. RRR = relative risk ratio.

^*P < .05. **P < .01. ***P < .001.

Discussion

The CPS ASEC is a leading source of information on the nation’s health insurance coverage patterns.^1,2 However, prior research had noted that the uninsured rate was consistently higher than other federal surveys^3-6 in part due to limitations in terms of question wording and data processing, including imputation procedures for fully imputed cases.^5,19 Specifically, by imputing each insurance type separately and at the person level, the processing system contributed to an overestimate of the number of people who were uninsured, while also inflating the prevalence of dual coverage and having multiple insurance types within a household.^5,19

Prior research has highlighted the importance of the CPS ASEC questionnaire redesign for improving data quality. The current paper makes 2 important empirical contributions. First, we show that limitations of the imputation procedure continued to bias estimates of health insurance with the legacy processing system; people with fully imputed health insurance data were more likely to be uninsured, even after adjusting for relevant characteristics. Fully imputed households were also more likely to have heterogeneity in insurance coverage than households with some reported health insurance coverage. Second, we show the bias in the health insurance imputation procedure for fully imputed cases appears largely addressed in the updated processing system. In the 2017 CPS ASEC Research File, differences between fully imputed and other cases were greatly attenuated. Moreover, after accounting for factors predictive of insurance status, there were no significant differences between fully imputed cases versus those that had some data reported. Not only does the revised imputation improve distributions at the person level, but imputation of HIUs improves the distribution of coverage for entire households.

The changes introduced in the 2017 CPS ASEC Research File are particularly consequential for estimates of dual coverage. Prior to the introduction of the processing improvements, the CPS ASEC had estimates of multiple forms coverage higher than other federal surveys. Given the cost of multiple types of coverage, few people are expected to pay for multiple types of comprehensive coverage.²² The improvements introduced in the Research File lower the prevalence of dual coverage,⁸ thus improving data quality.

An important limitation of this article is that we are not able to disentangle changes to the imputation procedure from other changes in processing because both were implemented simultaneously. Moreover, individuals in households missing all health insurance data also were more likely to be missing demographic and/or income information. Improvements to edits related to these topics^15,16 may have also affected the health insurance estimates for such individuals. Results from Tables 1 and 2 nonetheless show that changes in health insurance coverage across the 2017 CPS ASEC Research File and production file are largely concentrated among cases that were fully imputed.

The changes described here mark an improvement to CPS ASEC data quality going forward. The introduction of the updated processing system, which includes updated imputation strategies, marked the second (and final) stage of a 2-stage improvement of CPS ASEC health insurance data that began with the introduction of a redesigned questionnaire in 2014.^9,10 Other research has established the validity of the redesigned questionnaire,^23,24 and the updated processing system builds on the improved set of questions. The updated processing system has replaced the legacy processing system beginning with the 2019 CPS ASEC.

Findings from this article suggest that the health insurance coverage of fully imputed cases closely matches reported cases in these files and that processing changes improve data quality. However, additional research is needed to evaluate the performance of the imputation over time and to isolate effects of imputation net of other changes to processing.