Using enrollment records to evaluate self‐reports of monthly coverage in the redesigned current population survey health insurance module

Abstract

Objective

To evaluate the veracity of self‐reports of month‐level health insurance coverage in the Current Population Survey Annual Social and Economic Supplement (CPS).

Data Sources and Study Setting

The CHIME (Comparing Health Insurance Measurement Error) study used health insurance enrollment records from a large regional Midwest insurer as sample for primary data collection in spring 2015.

Study Design

A sample of individuals enrolled in a range of public and private coverage types (including Medicaid and marketplace) was administered the CPS health insurance module, which included questions about month‐level coverage, by type, over a 17–18‐month time span. Survey data was then matched to enrollment records covering that same time frame, and concordance between the records and self‐reports was assessed.

Data Collection/Extraction Methods

Sample was drawn by the insurer's informatics specialists and Census Bureau interviewers conducted the survey. Following data collection, updated enrollment records were matched to the survey data to produce a person‐level file of coverage by type at the month‐level.

Principal Findings

For 91% of the overall sample, coverage status and type were reported accurately for at least 75% of observed months. Results varied somewhat by stability of coverage. Among those who were continuously covered throughout the 17–18 month observation period (which comprised 64% of the overall sample), that level of reporting accuracy was observed for 94% of the sample; for those who had censored spells (34% of the overall sample), the figure was 87%; and among those with gaps and/or changes according to the records (2% of the overall sample), for 82% of the group at least 75% of months were reported accurately.

Conclusions

Findings suggest that reporting accuracy of month‐level coverage in the CPS is high and that the survey could become a valuable new data source for studying the dynamics of coverage, including the Medicaid unwinding.

What is known on this topic

• Month‐level health insurance coverage data is critical for studying the dynamics of coverage over time associated with changes in life circumstances, the economy, and policy environments.

• Data sources on month‐level health insurance coverage are limited; comprehensive enrollment records are only available for public coverage and survey data are compromised by small sample sizes and measurement error.

• In 2014 the redesigned Current Population Survey (CPS) was implemented and collects month‐level health insurance variables that have not yet been validated using an outside truth source.

What this study adds

• This study links self‐reports from the CPS health insurance module to enrollment records and finds that for 91% of the sample, at least 75% of months were reported accurately.

• If made public, the CPS month‐level variables would serve as a new data source with a large sample and state‐level indicators for the study of transitions in coverage over a 14–16‐month time span.

• The CPS includes multiple covariates (e.g., employment, income, health status) which position it to be a valuable data source for studying the dynamics of coverage, including the Medicaid unwinding.

1. INTRODUCTION

Month‐level health insurance coverage data are critical for studying the dynamics of coverage over time associated with changes in life circumstances, the economy, and policy environments. The need for these data is especially salient right now as the COVID‐19 public health emergency has ended and Medicaid is currently disenrolling millions of individuals who were allowed to maintain their Medicaid coverage without redetermination through the Families First Coronavirus Response Act. ¹ The transition period—known as the “unwinding”—will raise important questions about what happens to those disenrolled and how efficient and equitable the unwinding process is across states. ² , ³ States play a major role in structuring the Medicaid program through policies such as the frequency with which enrollees need to redetermine their eligibility throughout the year and the administrative procedures used for redetermination. ⁴ , ⁵ , ⁶ , ⁷ For example, some states allow as few as 10 days for enrollees to respond to notifications before being disenrolled. ⁶ Studies of Medicaid “churn”—enrollees losing and regaining coverage over a short period of time ⁸ —suggest that the unwinding will have considerable negative consequences. For the individual enrollee, these include reduced access to both needed and preventive care, worse quality of care, and worse health status and well‐being. ⁷ , ⁸ , ⁹ , ¹⁰ , ¹¹ , ¹² , ¹³ At the state‐level, consequences of churn include increased healthcare spending, administrative burden, and costs, and reduced capacity to promote health equity, measure the quality of care, and hold health plans accountable for improved health outcomes. ⁸ , ⁹ , ¹⁴ , ¹⁵ , ¹⁶

The unwinding is only one example of the need to better understand the dynamics of coverage. Another is the uninsured rate, which is estimated as a residual of those who report some kind of coverage. The uninsured can be measured in any number of ways, such as a snapshot (uninsured at a specific point‐in‐time), annual (uninsured throughout the calendar year), or uninsured spells lasting anywhere from a month to some number of years. The point‐in‐time uninsured estimate in the U.S. has come down over the past decade—from 14.6% in 2008 to 8.7% in 2020 according to the American Community Survey (ACS). ¹⁷ However, for many individuals, health coverage is not static or stable, and studies suggest that focusing on the point‐in‐time uninsured rate obscures access, quality, and cost issues. ¹¹ , ¹⁸ Both short and long gaps in coverage can have negative consequences at the individual and community levels, and these gaps cannot be measured with a point‐in‐time estimate. Furthermore, the point‐in‐time uninsured estimate necessarily represents a mix of individuals in various states of uninsurance. Some, for example, are chronically uninsured, some are continuously cycling on and off coverage, and some are experiencing a rare, short spell of uninsurance as they transition, say, from one job to another. The policy response to each of these situations would obviously be different, and understanding the prevalence of each, and the characteristics of individuals in each pool, would aid in decision‐making.

To study the dynamics of coverage over time, researchers typically turn to data sources with indicators of coverage at the month‐level. These include the Transformed Medicaid Statistical Information System (T‐MSIS), ¹⁹ the Survey of Income and Program Participation (SIPP), and the Medical Expenditure Panel Survey (MEPS). Each of these data sources has their advantages and drawbacks. Medicaid enrollment records enable national‐ and state‐level analysis of churn in the program, but there is no indication of what happens when individuals are disenrolled—specifically, whether they obtain coverage from another source or go uninsured. Both SIPP and MEPS indicate coverage status at the month‐level, but sample sizes are too small for state‐level analysis. Both are also panel surveys and prone to “seam bias” which compromises month‐level analysis. In a panel survey, multiple waves of data are collected and then data are concatenated to produce a long timeline. The point where one wave meets the next is called the “seam” and “seam bias” refers to a “tendency for estimates of change measured across the ‘seam’ between two successive survey administrations to far exceed change estimates measured within a single interview—often by a factor of 10 or more.” ²⁰ In 2014, a cross‐sectional survey—the Current Population Survey Annual Social and Economic Supplement (CPS ASEC; simply CPS hereafter)—was redesigned and now captures person‐month‐level data for a range of public and private coverage types over a 14–16‐month time span. Though public release of the new monthly coverage variables is pending further data quality review, the raw data now contain monthly coverage type variables at the person level. The goal of this study is to evaluate the veracity of these self‐reported variables in the redesigned CPS questionnaire.

The primary objective of the CPS health insurance redesign was to reduce under‐reporting of past calendar year coverage. Basic research and development began in 1999, final tests demonstrated improved calendar year uninsured estimates ²¹ , ²² and the redesign was first implemented in 2014. ²³ In that same year, an experimental study was launched to examine the reporting accuracy of more detailed features of coverage, including coverage type and the timing and duration of coverage. The study, called CHIME (Comparing Health Insurance Measurement Error), is a reverse record check study where enrollment records from a large regional insurer in the Midwest were used as sample for a telephone survey that included the redesigned health insurance module from the CPS. The sample was randomly selected from multiple strata of both public and private coverage types, including Medicaid, marketplace, employer‐sponsored insurance (ESI), and individuals moving between public and private coverage in the prior year. The CPS health insurance module was embedded verbatim into the CHIME survey. After data collection, survey data were matched back to an updated file of enrollment records (which covered the 17‐to‐18‐month time frame asked about in the survey). We were then able to examine the extent to which the details about coverage reported by the respondent matched what was indicated in the records.

Thus far, the matched dataset from the CHIME study has been used to examine reporting accuracy on features and type of coverage in the redesigned CPS, ²⁴ point‐in‐time coverage relative to the administrative records, ²⁵ and demographic and other characteristics predicting reporting accuracy. ²⁶ In this analysis we use the same dataset to examine a range of metrics on month‐level reporting accuracy, including monthly reports of status (covered vs. uninsured) and type of coverage (ESI, Medicaid, etc.). In the next section, we provide a brief overview of the redesign of the CPS health insurance questionnaire. Next, we describe the CHIME study design and data collection methods, and the analytic approach. We follow with results and a discussion of the accuracy of person‐month‐level CPS insurance reports.

2. THE REDESIGNED CPS ASEC

The CPS ASEC is a household survey conducted in late February through early April. Prior to 2014, one respondent was asked about coverage “at any time” in the past calendar year for each individual in the household. Accordingly, its estimates of the insured (and thus the uninsured) indicated coverage anywhere from 1 day to a full year. For years the CPS was criticized because its uninsured estimate tracked higher than other major federal surveys, ²⁷ , ²⁸ , ²⁹ leading to speculation that respondents were under‐reporting past year coverage. Studies also found that the CPS uninsured estimate was very similar to point‐in‐time estimates from other surveys, ²⁹ , ³⁰ raising questions about what the CPS estimate really represented—calendar year or point‐in‐time uninsured, or something in between. ³¹ , ³² , ³³

This motivated a survey redesign to improve the CPS estimate of past year coverage. ²² The CPS ASEC has a mandate to produce a measure of calendar year coverage, so it was not an option to simply drop questions about past year coverage. Based on research on recall error, in the redesign, respondents would first be asked about each household member's current coverage, which would serve as an anchor to then ask whether that coverage started before January 1 of the prior calendar year, thereby framing the 14–16‐month time period in scope. Follow‐up questions would determine monthly coverage between January and the interview date. See Figure 1 for a simplified version of the question series on timing of coverage. Thus, while the questionnaire was redesigned primarily in the service of improved reporting of past year coverage, the design change rendered useful month‐level data by person and by type of coverage.

FIGURE 1.

Basic structure of Current Population Survey Annual Social and Economic Supplement questions on month‐level health coverage status.

The redesigned questionnaire was implemented in 2014, ²³ but a “processing system” (edits, imputations, etc.) tailored to the new questionnaire was not available until a few years later. In 2017, a “Research File” was released which incorporated the new processing system, ²³ , ³⁴ and in 2019, further refinements ³⁵ , ³⁶ allowed for person‐month‐plan level analysis. These additional variables have not been released in the public use files, but restricted‐use internal files were used for an initial analysis. ³⁷ Metrics examined included the percent of the population that transitioned from uninsured to insured status, and vice versa, and mean length (in terms of months) of insured and uninsured spells. The analysis compared these metrics from the redesigned CPS to estimates from the MEPS and SIPP for 2017 and 2018. Results showed the CPS and SIPP metrics tracked rather closely together, and that the MEPS produced higher rates of transitions and proportionately more short spells of uninsurance (less than 3 months) relative to longer spells (12 months) than both other surveys. ³⁷ What is unknown, however, is what the “real” number of monthly transitions is, and how estimates across surveys compare to that truth source. To date, the only studies we are aware of that examined the concordance between month‐level survey estimates and administrative records involve Medicaid and the SIPP. ³⁸ , ³⁹ These studies are all pre‐ACA, they predate substantial redesigns of both the SIPP and CPS, and none of them examines private coverage. Thus, we believe this is the first study of its kind to examine person‐month‐level reporting of specific coverage types in the CPS redesign post‐ACA by comparing enrollment records and survey reports.

3. METHODS

3.1. Data collection

In December 2014 the initial sample of enrollment records was drawn by an informatics team on behalf of a large regional health insurance company in the Midwest. Specifically, phone numbers of policyholders under the age of 65 were randomly drawn from five different strata of insurance types: ESI, non‐group, Marketplace, Medicaid, and MinnesotaCare (a state‐based public program with a sliding scale premium cost designed to cover those with incomes too high to qualify for Medicaid but below 200% of the federal poverty level). In addition, because we wanted to examine the dynamics of coverage, we included a “transition” stratum of individuals whose coverage type, according to the records, changed from ESI to public (or vice versa) sometime in 2014. These phone numbers were used as sample for Census Bureau interviewers to conduct a telephone survey in May and June of 2015. To set the context for the health insurance module consistent with the CPS, the CHIME survey began with demographic questions (e.g., age, sex, race), followed by modules on labor force and program participation. The health insurance module asked about coverage from January 2014 up through the interview day for each household member, covering a 17‐month time span for those interviewed in May and 18 months for those interviewed in June. The household response rate was 22%. We used enrollment record data on non‐respondents to conduct a non‐response analysis and created individual‐level non‐response weights to adjust for the observed differences between respondent and non‐respondent households. A base weight was also created so that the sample would reflect the overall population of the insurer. All weighted estimates incorporate both non‐response and base weights.

A few months after data collection ended, a second file of enrollment records including monthly coverage indicators matching the entire 17–18 month reference period was sent by the insurer. We used a computer‐match algorithm to match the survey person‐record to the enrollment person‐record using variables in both datasets: phone number, name, sex, date of birth, and address. Clerical matching was conducted on a small set of individuals where the algorithm produced ambiguous matches. We were able to match at least one person in 87% of surveyed households. Our analysis is restricted to this matched sample. Further details on CHIME methodology can be found in Fertig et al. ⁴⁰

3.2. Matched survey‐records analysis file

The analysis file included about 2000 matched individuals. For a handful of individuals (0.5%), no months of coverage were found in the insurance records during our observation window, so they were dropped from analysis. These individuals may have been in‐sample because they were covered by the insurer sometime before January 2014 and lived in a household with someone who had coverage at the time of the sample draw in December 2014. Table 1 displays demographic characteristics of the covered individual(s) in the household and the adult household respondent of the final CHIME sample file alongside the same characteristics for the full national CPS sample from 2015. Respondents in the CHIME sample are somewhat younger, more female, White, educated and employed, and in better health compared to the national sample. We discuss below the relationship between these characteristics and reporting accuracy.

TABLE 1.

Demographic characteristics in CHIME (Comparing Health Insurance Measurement Error) versus current population survey sample.

	CHIME sample	2015 CPS sample
Covered individual	N ~ 2000	N = 153,890
Age
Less than 18	32%	29%
18–25	11%	12%
26–44	29%	27%
45–64	29%	31%
Fair/poor health status	5%	8%
Respondent	N ~ 1000	N = 50,711
Age
18–25	8%	8%
26–44	48%	40%
45–64	43%	51%
Female	52%	49%
Race/ethnicity
White non‐Hispanic	86%	66%
Black non‐Hispanic	6%	13%
Hispanic	7%	13%
Other non‐Hispanic (e.g., Asian, Native American, Pacific Islander)	1%	8%
Education
High school or less, D/R	23%	33%
Some college or associate's degree	31%	30%
Bachelor's degree or higher	46%	37%
Employed part‐time, part‐year or more	91%	79%
Family income as a percent of FPL
<138%	19%	19%
139%–199%	12%	10%
200%–399%	29%	28%
400%+	39%	43%
Coverage types held at any time between January 2014–May/Jun 2015
Public	30%
Medicaid	26%
MinnesotaCare	5%
Private	70%
Employer sponsored insurance	67%
Non‐Group	3%
Marketplace	0.2%

Note: Weighted percentages are reported. Unweighted sample sizes have been rounded according to disclosure avoidance guidelines. Coverage types sum to more than 100% because individuals may have more than one type of coverage over the observation period. CHIME includes a subset of demographic and socio‐economic questions asked in the CPS Annual Social and Economic Supplement (ASEC) prior to the health insurance module. The CPS ASEC is a household survey with one adult respondent per household who reports about the insurance status of each individual in the household.

Abbreviations: CPS, Current Population Survey; D/R, don't know/refused; FPL, federal poverty level.

Source: CHIME and the 2015 CPS accessed at IPUMS.org.

3.3. Analytic groups by patterns of coverage

Most interviews (78%) were conducted in June and the remainder in May. We examined patterns of coverage indicated in the records for that 17–18 month window, and then categorized individuals into three analytic groups for comparisons to the survey reports. The first and most straightforward were individuals whose records indicated continuous coverage by the same coverage type throughout all 17–18 months (January 2014 up through interview month of May or June 2015). Sixty‐four percent of individuals fell into this category, which we label “Solid.”

Next were individuals whose records indicated continuous coverage by the same coverage type, BUT the spell of coverage began or ended within our 17–18 month observation window, which is known as a “censored” spell. For example, if records indicated no coverage from January through March 2014, and then a spell of Medicaid from April through to the following May 2015, that individual would be left‐censored. We do not have the universe of all insurers, so we do not know whether this individual had Medicaid (or another type of coverage) with a different insurer from January through March of 2014, or whether they were uninsured. Thus, we made the analytic decision to essentially ignore the months before and after censored spells indicated in the records, meaning that we curtailed the observation window of individuals with censored spells somewhat.

Overall, 34% of individuals fell into this group, which we label “Censored.” Most (27%) had left‐censored spells (i.e., they enrolled in a plan with the insurer after January 2014, thus we observed the onset of the coverage) and a little less than 8% had right‐censored spells (i.e., they had known coverage at the beginning of the observation window in January 2014, but disenrolled at some point during the 17–18 month observation window, and thus we observed the termination of the coverage). Among the Censored group, on average we observed about 11 months of coverage; almost 31% were observed for 13+ months and had at least 1 month of missing data, 42% were observed for 7–12 months and almost 28% of individuals were observed for 3–6 months.

The last analytic group, comprising 2% of the overall sample, represents individuals who, according to the records, had a coverage gap of at least 1 month (a gap is a period of no coverage surrounded by coverage before and after observed in the enrollment records) and/or changed coverage type within the observed window. A small number of individuals were both censored and had a gap, and we made the analytic decision to include these in this last group, which we label “Gaps and Changes.”

These three groups generally represent degrees of stability in coverage over time. The Solid group is the most stable, given that they remained enrolled with the same insurance company and maintained the same coverage type for the full 17–18 months. The Censored group represents individuals who maintained consistent coverage for a spell, but they either changed insurance companies or were uninsured for a spell at the beginning or the end of the 17–18 month window. The Gaps and Changes group represents individuals who were observed in the records to have had an actual change within the 17–18 month window—either dropping coverage with our insurer for 1 month or more or changing coverage type. Analysis for all three groups was done using weighted data.

Among the Gaps and Changes group, about half (50 unweighted matched individuals) transitioned between ESI and public coverage, enabling us to examine the reporting accuracy of change. If all 50 individuals were observed in all 18 months, this would have rendered 900 person‐months for analysis. Due to censoring and the 17‐month reference period for some individuals, however, the analysis file rendered 814 person‐months for analysis. While the small sample size limits the kinds of questions we can examine, we conduct analysis on this interesting sub‐sample.

3.4. Analysis plan

The first key metric we examined across groups is “sensitivity” (aka under‐reporting)—that is, if the records indicate Coverage Type X in Month Y, does the respondent report Coverage Type X in Month Y? Because we did not observe the same number of months across groups, we present these findings as the percent of observed months where coverage type was accurately reported, allowing readers to assess relative levels of accuracy across groups in a standardized way.

We then parse out the types of reporting error(s) made by respondents. We first separate out the individuals who were 100% accurate (i.e., for all months, the coverage type indicated in the records matched the coverage type reported). We categorized all other individuals into one of the following types of error categories:

3.4.1. Coverage type error only

For all months of coverage indicated in the records, coverage was reported in the survey, but for at least 1 month, the records indicated Coverage Type X, but the survey report indicated Coverage Type Y.

3.4.2. Under‐reporting only

For at least 1 month, the records indicated coverage, but no coverage was reported in the survey. For all months where coverage status matched, the type of coverage also matched.

3.4.3. Over‐reporting only

For at least 1 month, the records indicated no coverage, but coverage was reported. This type of error is only possible in the Gaps and Changes group, and it is our conjecture that this is over‐reporting because the respondent could have had coverage with a different insurer during these gaps. We categorize this as a reporting error because we believe it is unlikely for an individual to be covered with our insurer, disenroll and pick up coverage with a different insurer, then re‐enroll with our insurer all within a 17–18 month window. For this reason, we presume that an observed gap in coverage in the records likely represents a true spell of uninsurance. We acknowledge this is inconsistent with our treatment of the Censored group, where we ignore spells of no coverage in the records. However, we suggest that in Censored cases it is quite possible that respondents simply transitioned from one insurer to another, and the lack of coverage in our records does not represent a spell of uninsurance.

3.4.4. Combinations of the above types of errors, split into two groups

First are instances of at least 1 month of under‐reporting and at least 1 month of over‐reporting, but no coverage type errors. Second are instances of at least 1 month of under‐reporting and/or at least 1 month of over‐reporting, and also a coverage type error. Note that over‐reporting can only be measured for the Gaps and Changes group.

We note that records (and also our linkage method), of course, have their own potential error. However, we worked closely with informatics staff who were at the site of production of the enrollment records and carefully examined duplicates, missing data, and other anomalies. We cannot rule out the possibility that the records and/or match is in error, but on balance, we have reason to believe the records are “more correct” than the survey data.

4. RESULTS

4.1. Sensitivity across groups

First, we examine sensitivity across the three analytic groups and for the pooled sample overall. Figure 2 presents the distribution of months of accurate reporting in terms of quartiles; for example, for individuals in the 1st quartile, 0%–25% of months were reported accurately, and for those in the 4th quartile, 75.1%–100% of months were reported accurately. Results show that 94%, 87%, and 82% of individuals were in the 4th quartile for the Solid, Censored, and Gaps and Changes groups, respectively, and that 91% in the Pooled sample overall were in this quartile. If we raise the bar to 100% of months reported accurately (see Figure 3), the Solid and Censored groups are still quite high, with 92% and 85% of individuals, respectively. For the Gaps and Changes group, only 33% of individuals had all months of coverage reported accurately. Figure 2 also shows that while the 4th quartile dominated by far, the 1st quartile was the next most prevalent among the Solid and Censored groups, while for the Gaps and Changes group, the distribution was more even across the 1st, 2nd and 3rd quartiles. See Appendix 1 for a more detailed breakdown of sensitivity.

FIGURE 2.

Quartiles of accurately reported months as a percent of all observed months, by group. Groups: (i) Solid = records indicated continuous coverage by the same coverage type throughout all 17–18 months. (ii) Censored = records indicated continuous coverage by the same coverage type, but the spell of coverage began or ended within our 17–18 month observation window. (iii) Gaps and Changes = records indicated a coverage gap of at least 1 month and/or coverage type changed within the observed window. Weighted percentages are shown. Source: CHIME.

FIGURE 3.

Accurate and erroneous reporting, by type of error, by group. Groups: (i) Solid = records indicated continuous coverage by the same coverage type throughout all 17–18 months. (ii) Censored = records indicated continuous coverage by the same coverage type, but the spell of coverage began or ended within our 17–18 month observation window. (iii) Gaps and Changes = records indicated a coverage gap of at least 1 month and/or coverage type changed within the observed window. Coverage type is defined as: (i) Employer‐sponsored insurance (ESI); (ii) non‐group; (iii) marketplace; (iv) public (Medicaid or MinnesotaCare). Type of reporting error: (i) Accurate = coverage status (insured or uninsured) and coverage type were accurately reported for all months. (ii) Coverage type mismatch = for at least 1 month, coverage was reported but the type does not match the records. (iii) Under‐reporting = for at least 1 month, coverage was indicated by the records but was not reported in the survey. (iv) Over‐reporting = for at least 1 month, coverage was not indicated in the records but was reported in the survey. (v) ^aOver‐reporting is only measured for the Gaps and Changes group; see text for details. Weighted percentages are shown. Source: CHIME.

4.2. Reporting error by type across groups

Figure 3 shows individual‐level accuracy and error rates. The light gray portions of the bars indicate individuals where all months were reported accurately, and the other portions of the bars indicate type of reporting error. For the Solid group, under‐reporting (dark gray) is about twice as high as coverage type errors (medium gray) at 5% and 2%, respectively, while for 1% of individuals, there was both under‐reporting and coverage type error (black). In the Censored group, rates of under‐reporting and coverage type errors are similar (7% and 6%, respectively) and 2% had both error types. For the Gaps and Changes group, the most prevalent error was our presumed over‐reporting (crosshatch) at 24%, followed by coverage type error at 18% and then under‐reporting at 13%. The remainder (a total of 11% of individuals) had multiple types of reporting errors (black and gray stripe) (see Appendix 2 for exact figures).

4.3. Changes in coverage over time: A closer look

Finally, we turn to the subsample of 50 matched individuals who, according to the records, transitioned between ESI and public coverage sometime in 2014. In total these individuals yielded 814 person‐months. Figure 4 displays these 814 person‐months grouped by the type of coverage indicated in the records. Half the person‐months were those where the records indicated both public and ESI coverage; one quarter were those where records indicated only ESI coverage; and the other quarter were those where records indicated only public coverage (Medicaid or MinnesotaCare).

FIGURE 4.

Person‐month reporting among those who transitioned to/from employer‐sponsored insurance and public coverage, by group. Base = 814 person‐months representing 50 individuals. Groups: (i) Public and ESI = person‐months where records indicated both public and ESI coverage in the same month. (ii) Employer‐sponsored insurance (ESI) = person‐months where records indicated only ESI coverage. (iii) Public = person‐months where records indicated only public coverage (Medicaid or MinnesotaCare). Unweighted percentages are shown. Source: CHIME.

In the months with Public and ESI, 91% were accurately reported; among the ESI‐only months, 74% were reported accurately, and among the Public only months, the figure was 67%. In terms of type of reporting error, for the Public and ESI months, under‐reporting was more common than coverage type mismatches (6% vs. 3%). For the ESI months, there was very little under‐reporting relative to coverage type errors (2% vs. 24%). For the Public months, 13% were under‐reported and 20% had a coverage type error (see Appendix 3 for exact figures).

5. DISCUSSION

The redesigned CPS collects the same health insurance metric produced since the 1980s—coverage “at any time” during the prior calendar year. It also collects monthly coverage type for each person in the household over a 14‐16‐month time span. The aim of this study was to assess the veracity of these monthly coverage variables using a dataset that matched survey reports with data from administrative records from a health insurer providing a range of public and private insurance plans.

Results showed that reporting accuracy for month‐level coverage is high. Both coverage status and type were correctly reported in all observed months for 92% of the sample who were continuously covered, and for 85% of the sample who had censored spells. Even among those with known gaps and/or changes, at least 75% of month‐level reports on both status and coverage type were accurate for 82% of the sample.

Moreover, among the least stable groups (those with gaps and/or changes and those known to move between ESI and public coverage), findings suggest that levels of misreporting would have little impact on the uninsured point estimate. Among those who transitioned between ESI and public coverage, only 7% of months were under‐reported. For the Gaps and Changes group, 18% of individuals had only coverage type error, and over‐ and under‐reporting rates about cancel each other out (although the levels of misreporting in both directions are a concern for individual‐level analysis). It is also worth noting that the least stable groups make up a very small portion of the overall sample.

While this study provided a unique opportunity to examine reporting accuracy across several types of public and private coverage over time using administrative records as a validation source, it was not without limitations. First, because the CHIME sampling frame was insured individuals and the administrative records were from one insurance company, we were not able to examine the reporting accuracy of uninsured individuals or know the coverage status or type among those who changed insurers in our observation window. Second, we mimic portions of the CPS questionnaire in the CHIME survey for both content and context, but because of our experimental design, the CPS health insurance module did not exactly match production conditions. Third, the CHIME sample differed from the national CPS sample on certain characteristics noted above, which may limit generalizability. However, only some of those characteristics were found to be tied to reporting accuracy, and all were in the direction of the CHIME sample composition having lower reporting accuracy. Specifically, Whites, those with more education and higher employment levels are less accurate reporters of public coverage ²⁶ and the national sample has lower rates of all these characteristics than the CHIME sample. On the private side, younger adults are less accurate reporters, ²⁶ and the national sample is very slightly older than the CHIME sample. This suggests that the CHIME results represent a lower bound of reporting accuracy compared to the nation as a whole. Furthermore, given that accuracy levels are quite robust, even if there were undetected differences in the demographic profile between the CHIME sample and the nation, reporting accuracy differences tied to those demographic characteristics would need to be extreme to produce a meaningful shift in reporting accuracy levels.

6. CONCLUSION

Results bode well for the prospects of the person‐month‐level variables now collected in the redesigned CPS to be used for studying the dynamics of health insurance, including the Medicaid unwinding. Though raw data have been collected since 2014, until now the veracity of these variables has not been carefully examined or documented. The Census Bureau is now reviewing these data, and it is considering releasing them as an experimental data product. This would enable the study of a vast range of policy‐relevant research questions on churn, transitions across coverage types, number and duration of coverage spells, and coverage dynamics associated with other life events measured in the CPS.

FUNDING INFORMATION

This work was supported by U.S. Census Bureau, State Health Access Data Assistance Center (SHADAC), Medica Research Institute, U.S. Department of Health and Human Services Office of the Assistant Secretary for Planning and Evaluation (ASPE), and Robert Wood Johnson Foundation.

APPENDIX A.

A.1.

Detailed data for Figure 2: “Quartiles of accurately reported months as a percent of all observed months, by group.”

% of Months accurately reported	Group
	Solid	Censored	Gaps and Changes	Pooled
0–5	4.06	6.85	0.31	4.90
5.1–10	0.43	0.58	3.13	0.54
10.1–15	0.08	0.25	0.52	0.14
15.1–20	0	0.78	0	0.26
20.1–25	0.01	0.10	0.31	0.05
25.1–30	0.17	0.19	0.13	0.17
30.1–35	0.73	0.43	0.62	0.63
35.1–40	0.11	0.02	3.30	0.15
40.1–45	0	0.01	0.88	0.02
45.1–50	0.13	2.00	2.85	0.81
50.1–55	0	0.05	0.69	0.03
55.1–60	0.01	0.28	0	0.10
60.1–65	0.14	0.70	0.13	0.32
65.1–70	0.44	0.44	4.43	0.53
70.1–75	0.10	0.48	0.50	0.23
75.1–80	0.15	0.43	3.40	0.32
80.1–85	0.13	0.21	12.17	0.18
85.1–90	0.61	0.59	15.67	1.21
90.1–95	1.05	0.63	17.62	1.29
95.1–100	91.69	85.03	33.35	88.13

Note: The percent of individuals whose coverage is reported accurately in 95.1%–100% of observed months is significantly lower (p < 0.05) in both the Censored group (85.03%) and the Gaps and Changes group (33.35%) compared to the Solid group (91.69%).

APPENDIX B.

B.1.

Detailed data for Figure 3: “Accurate and erroneous reporting, by type of error, by group.”

Reporting outcome	Group
	Solid	Censored	Gaps and Changes	Pooled
Accurate	91.69	85.03	33.35	88.13
Coverage type mismatch	2.42	6.13	18.23	4.02
Under‐reporting only	4.78	7.01	13.23	5.72
Both over/under reporting and type mismatch	1.11	1.83	4.57	1.43
Over‐reporting coverage only			24.32	0.56
Over‐ and under‐reporting			6.3	0.15

Note: The percentage of individuals whose monthly coverage is reported as listed on the left.

APPENDIX C.

C.1.

Detailed data for Figure 4: “Person‐month reporting among those who transitioned to/from employer‐sponsored insurance and public coverage, by group.”

Reporting outcome	Months
	Public + ESI (n = 406, 50%)	ESI (n = 206, 25%)	Public (n = 202, 25%)	Pooled (n = 814)
Accurate	91.13	73.79	67.33	80.84
Coverage type Mismatch only	2.71	23.79	19.80	12.29
Under‐reporting coverage only	6.16	2.43	12.87	6.88

Note: The percentage of person‐months whose monthly coverage is reported as listed on the left.

Abbreviation: ESI, employer‐sponsored insurance.

Pascale J, Fertig AR, Call KT. Using enrollment records to evaluate self‐reports of monthly coverage in the redesigned current population survey health insurance module. Health Serv Res. 2024;59(2):e14285. doi: 10.1111/1475-6773.14285