The American Community Survey and Health Insurance Coverage Estimates: Possibilities and Challenges for Health Policy Researchers

Abstract

Objective

To introduce the American Community Survey (ACS) and its measure of health insurance coverage to researchers and policy makers.

Data Sources/Study Setting

We compare the survey designs for the ACS and Current Population Survey (CPS) that measure insurance coverage.

Study Design

We describe the ACS and how it will be useful to health policy researchers.

Principal Findings

Relative to the CPS, the ACS will provide more precise state and substate estimates of health insurance coverage at a point-in-time. Yet the ACS lacks the historical data and detailed state-specific coverage categories seen in the CPS.

Conclusions

The ACS will be a critical new resource for researchers. To use the new data to the best advantage, careful research will be needed to understand its strengths and weaknesses.

The American Community Survey (ACS) is poised to evolve from an unknown entity among the health service research community to one of the most critical data sources for state and local health insurance coverage policy researchers over the next few years. The ACS will be an important new resource for health policy research, with significant advantages over the Current Population Survey (CPS), the major U.S. survey currently used for this purpose (Blewett et al. 2004). However, there are significant challenges associated with using the ACS for health policy analysis that will need to be addressed.

The U.S. Census Bureau has been rolling out the new ACS over the past decade. The ACS was designed to replace the information that was traditionally gathered every 10 years as part of the decennial census “long form” and to improve intercensal estimates. The ACS is conducted every year and includes a wider range of content than the long form, and began including health insurance coverage in 2008 with the first health insurance estimates to be released in 2009. In contrast to most national surveys, the ACS will provide estimates at not only the state level, but also at substate levels of geography such as cities, counties, and even census tract levels (U.S. Census Bureau 2006a). In this paper, we highlight the strengths and weaknesses of the ACS relative to the CPS for use in health policy research.

Overview of the ACS

The ACS is an annual mixed-mode household survey that draws a sample of three million U.S. addresses (including households and institutional group quarters) per year (U.S. Census Bureau 2006a). This is roughly 30 times larger than the sample size of the Annual Social and Economic Supplement to the CPS, which also collects health insurance data (U.S. Census Bureau 2002). The ACS was designed by the Census Bureau to replace the Decennial Census “long form” that was given to a sample of 16 million U.S. households in Census 2000. The ACS collects its sample in all 3,141 counties (or county equivalents) in the United States every year. Starting in 2006, the ACS began collecting data from people living in institutionalized group quarters as well. The ACS became fully operational for residential addresses in 2005 and group quarters in 2006 (U.S. Census Bureau 2006a).

The ACS is conducted as a sequential mixed-mode survey. Sampled households are first sent an ACS questionnaire by mail and nonrespondents to the mail survey are contacted via telephone to complete the survey. Finally, a sample of remaining nonrespondents is selected for an in-person survey component. This sequential contact protocol results in a very high response rate of 96 percent for the ACS (versus 84 percent in the CPS). At the end of each year, all the data collected from the preceding year are processed into a single data file for each year. The annual ACS data products are therefore a rolling average of the preceding calendar year (U.S. Census Bureau 2006a, b).

ACS Survey Content and the ACS Health Insurance

The ACS contains most of the items that were included on the decennial census long form, including income (and the derived measure of poverty), disability status, marital status, education, occupation, home ownership, value of household, amount of mortgage on housing unit for owners or amount of rent paid for renters, and travel to work. See Table 1 for more details on the ACS content.

Table 1.

American Community Survey Questionnaire Content

Social characteristics	School enrollment, educational attainment, marital status, fertility, grandparents caring for children, veteran status, residence 1 year ago, place of birth, U.S. citizenship status, year of entry, country of birth, language spoken at home, relationship, households by type, and ancestry
Health characteristics	Disability status items (items include information on: blindness, deafness, severe vision or hearing impairment, any condition that substantially limits activities such as walking, climbing stairs, reaching, lifting or carrying, learning, remembering or concentrating, dressing bathing or getting around inside the home), and current health insurance coverage
Economic characteristics	Employment status, commuting to work, occupation, industry, class of worker, income and benefits, and poverty status
Housing characteristics	Housing occupancy, units in structure, year structure built, number of rooms, number of bedrooms, housing tenure, year householder moved into unit, vehicles available, house heating fuel, utility costs, occupants per room, housing value, mortgage status and costs, and gross rent
Demographic characteristics	Sex, marital status, age, race, and Hispanic origin

Source: U.S. Census Bureau's American Community Survey's website. Available at http://www.census.gov/acs/www/SBasics/SQuest/SQuest1.htm

The ACS health insurance question will be asked about each person in the household separately. The question appears on the mail questionnaire as shown in Figure 1.1 The health insurance question was developed by the Census Bureau after assessing several different possibilities in an attempt to maximize reliability while minimizing under-reporting and item nonresponse (Nelson and Ericson 2007) and preserving administrative simplicity. The ACS question asks about insurance at the time of the survey.

Figure 1.

American Community Survey Health Insurance Question

Source: 2008 ACS Questionnaire, English. Available at http://www.census.gov/acs/www/Downloads/SQuest08.pdf

ACS Data Products and Release Schedule

The ACS will produce two main types of data products that will be released in late summer of each year: a set of predefined summary tables that make use of the information collected from all the respondents, and a 1 percent public use microdata sample (PUMS) (U.S. Census Bureau 2006a). The predefined summary tables will generally cross three or more variables (e.g., insurance status by race by state). These predefined tables can either be downloaded in their entirety or accessed one (or more) at a time through the Census Bureau American Factfinder data system.

The ACS will release single-year and multi-year summary tables every year (multi-year data products will be released later in the calendar year). These tables will be released annually for state geographic areas with populations >65,000. For areas with smaller populations, summary data tables will be released as rolling multi-year averages. Three-year averages will be released for geographic areas with populations of at least 20,000 persons but <65,000. Five-year average summary tables will be made available for all census geographic areas under 20,000 in population down to the level of the census tract, and the Census Bureau also plans to release data for block groups (U.S. Census Bureau 2006a).

The ACS will also release a 1 percent PUMS each year for researchers who prefer to work with microdata in order to define their own tables for specific research purposes. In 2006, the ACS public use sample contained almost 2.9 million people from 1.3 million households and 81,151 people living in group quarters (U.S. Census Bureau 2006a). The PUMS data represent a little over one-third of all the ACS sampled records.2 The lowest level of geographic detail available in these files is a public use microdata area (PUMA). These are substate geographic areas that are completely contained within state boundaries and generally drawn to have population sizes in excess of 100,000 people.3

Strengths and Limitations of the New ACS Health Insurance Data for Policy Research

Perhaps the most relevant way to assess the strengths and weaknesses of the new ACS health insurance question for health services research and subsequent policy applications is to compare it with the CPS. The CPS is the survey currently used to provide official government estimates of the uninsured. It is also used by the Congressional Budget Office to score legislation and by the Centers for Medicare and Medicaid Services (CMS) to allocate federal funds for the State Children's Health Insurance Program (SCHIP). The CPS is also the most frequently used source of data for state health policy work on the uninsured (Blewett et al. 2004).

The ACS will offer policy researchers many important advantages over the CPS for measuring the uninsured. Table 2 summarizes the key points of comparison between the two surveys. The most noteworthy strength of the ACS is its large sample size, which is 30 times larger than the CPS, and that it draws its sample from all 3,141 counties (or equivalents) in the United States (U.S. Census Bureau 2006a). This allows the ACS to provide more precise estimates of the uninsured at the state and local levels (and for small population subgroups at the national level). Given that so much health reform is occurring at the state level, having more precise state-level estimates of the uninsured is critical to health policy (Kenney, Holahan, and Nichols 2006). The ACS estimates could be made even better for very small areas through the use of small area estimation models, combining ACS data with other sources (O'Hara 2008). These models could be used to correct for known biases in the survey responses.

Table 2.

Comparison of Census Bureau's Current Population Survey (CPS) to the New American Community Survey (ACS)

Survey Features	CPS	ACS
Health insurance items
Reference period for health insurance coverage	Any coverage during the last calendar year	Coverage at the point in time of the interview
Uninsured measure	All year uninsured	Point in time
Specific state names for SCHIP and Medicaid	Yes	No
Specific state program names	Yes	No
Health insurance verification question	Yes	No
Number of survey items devoted to health insurance coverage	8	1
Information collected for everyone in a household	Yes	Yes
Other key content
Demographics (e.g., age, sex, marital)	Yes	Yes
Income (including poverty level)	Yes	Yes
Public program participation (e.g., TANF/SSI)	Yes	Yes
Employment	Yes	Yes
Activities of daily living	No	Yes
Self-reported health status	Yes	No
Survey methods
Annual sample size (number of households sampled)	100,000	3,000,000
Number of people included in the public use microdata sample in 2006	208,562	2,969,741
Population studied	Noninstitutionalized	U.S. population (including institutionalized)
Data collected	February–April	All year
Mode of data collection	Phone and face to face	Mail, phone, and face to face
Response rate	84%	96%
Uninsured estimates
2006 Uninsured estimate	15.8	13.9*
2006 Uninsured estimate in millions	44.8	40.7

^*Taken from 2006 ACS content test (for additional comparison the National Health Interview Surveys 2006 point-in-time estimate is 14.8% [43.3 million]).

For researchers who work with the PUMS microdata sample, however, the ACS PUMS sample size will be only 13 times larger than the CPS (not 30 times larger). Furthermore, the sample distribution across states will vary by survey. The CPS greatly oversamples people living in small population states (e.g., District of Columbia) relative to large populations states such as California. Therefore, the sample size gains for using the ACS for state-level research will be much greater in California (19 times the size of CPS) than in Washington, DC (two times the size of CPS). See Table 3 for the sample size differences in selected states ranging from lowest to highest in ACS sample size.

Table 3.

Household Sample Sizes by Selected States for the 2006 Current Population Survey's Annual Social and Economic Supplement (CPS), the 2006 ACS, and the 2006 ACS PUMS

Selected States	CPS-ASEC	ACS	ACS PUMS	ACS PUMS Sample Increase as a Percent of CPS-ASEC
District of Columbia	1,186	3,672	2,612	120
Wyoming	932	3,877	2,175	133
Alaska	1,013	5,835	2,237	121
Vermont	1,052	8,076	2,522	140
North Dakota	958	8,258	2,788	191
Connecticut	1,656	21,357	13,673	726
Oregon	1,020	23,785	15,001	1,371
Kentucky	1,059	28,658	17,486	1,551
Oklahoma	983	29,492	14,461	1,371
Minnesota	1,692	57,762	20,659	1,121
Florida	3,453	103,089	76,842	2,125
New York	3,309	121,011	72,476	2,090
Texas	3,959	129,186	84,474	2,034
California	6,519	178,666	125,071	1,819

This table excludes institutionalized group quarters.

The ACS has two strengths associated with the measurement of health insurance coverage relative to the CPS. First, it clearly measures uninsurance at a point in time. The CPS questions have the unfortunate problem of asking about coverage at any time during the past year but delivering an estimate that resembles other surveys‘ point-in-time estimates and is often interpreted as such (Swartz 1986; Congressional Budget Office 2003). Thus, the ACS will provide a clear point-in-time measure that can be used for policy development, policy simulations, and cost estimates at the state level. The second advantage of the ACS is that it asks about insurance coverage of each person in the household separately whereas the CPS asks a generic household-level question, “… did anyone in the household have Medicaid ….” The person-level approach has been shown to result in more accurate reports of health insurance coverage (Hess et al. 2002).

The ACS also has additional content of interest to health researchers, including survey items on activities of daily living, housing values, rent payments, mortgage payments, household ownership, and other noncash program participation such as food stamps (see Table 1 for more information). This provides important information on the array of living costs that can be related to income when discussing affordability of health care coverage.

Finally, the ACS sample includes populations not covered by the CPS, including Puerto Rico and institutionalized group quarters such as certified nursing facilities, dormitories, and prisons. Those living in institutional quarters represent important issues related to health status, health insurance coverage, and costs.

There are, however, some areas where the CPS will still be more valuable than the ACS. The CPS has been measuring health insurance coverage since 1980 with a reasonable level of consistency available back to 1988 (with the use of careful adjustments for changes in calendar years 1994, 1999, and 2004). This allows for important state-level comparisons and trend information on the changing characteristics of the uninsured over time.

Another strength of the CPS is that it has been widely used by health policy researchers over the past 30 years and its strengths and limitations for this type of research are widely understood and well researched. The same cannot be said for the ACS. As the ACS begins to receive wide use and attention from the health policy research community, problems are almost certainly going to emerge and at this time the limitations are not clearly understood.

Other CPS strengths relative to the ACS pertain to shortcomings in the ACS content. First, the ACS does not have the self-reported health status question that is often seen as a critical control variable in health policy research. The ACS does, however, have a functional limitation question that may compensate for this omission to an extent. Second, the CPS has a much more detailed health insurance measurement section that includes specific questions on many different types of coverage (e.g., with separate questions on SCHIP, Medicaid, and other state programs), using state-specific program names for Medicaid and SCHIP. The absence of state-specific names for Medicaid, SCHIP, and other state programs in the ACS mean that the reporting error problems in some states could be much worse on the ACS than on the CPS. For example, a report from Maryland showed that simply adding the term Medical Assistance (the name of Maryland's Medicaid program) to the CPS question on Medicaid increased the proportion of people on Medicaid who answered the question correctly (Eberly, Pohl, and Davis 2005). This measurement issue is likely to affect estimates for each state differently, resulting in state-by-state variation in the “Medicaid undercount” that is likely to be different from that observed on the CPS. The Medicaid undercount is phenomenon of survey results underestimating Medicaid enrollment compared with administrative data total, which artificially increases survey-based estimates of the uninsured and of those with private coverage (Eberly, Pohl, and Davis 2005; Davern, Klerman, and Ziegenfuss 2007).

Third, the CPS includes a verification question to confirm that persons who appear to have no health insurance coverage are indeed “uninsured,” whereas this degree of specificity is currently not included in the ACS. Past studies have indicated the importance of adding a direct question that specifically asks a respondent if they are indeed uninsured instead of assuming they are uninsured if they do not check any of the types they are asked about. This is likely not to impact the ACS as much as the CPS, as point-in-time measurement has been shown to be impacted less by the addition of a verification item (Nelson and Mills 2001; Rajan, Zuckerman, and Brennan 2002).

And finally, the ACS is conducted first through the mail, and then nonrespondents are contacted by telephone and a randomly selected subsample is contacted in person. Because of this procedure, the ACS could have additional mode effects in their estimates that are not present in the CPS, which does the vast majority of its interviews over the phone with a minority of interviews done in person. In order to mitigate the possible impact the ACS employs a “mode bias noninterview factor” in their weighting procedure. According to the National Research Council (2007, p. 193) the impact of this adjustment is minimal, “… given the high response rates achieved in the ACS, the nonresponse adjustments have mostly a minor impact. However, for areas with lower response rates, the adjustments may be significant.” The Census Bureau plans to monitor potential mode effects by conducting mode experiments to understand how they can impact the estimates, which may lead to further refinements in these adjustment factors (National Research Council 2007).

Policy Implications

The ACS will be an important new resource for health policy researchers. In order to use the new data to the best advantage, careful and ongoing research will be needed to understand its strengths and weaknesses. The new health insurance questions present new opportunities, but also create challenges for users.

The first and most important opportunity is that the ACS will allow policy makers to have more precise estimates of the size of the uninsured population in each state, where the uninsured are located within a state, and who is eligible for public programs such as Medicaid and SCHIP. This will be critically important in helping decision makers effectively develop and target programs to increase access to coverage. For example, more precise substate estimates could provide information on the location of eligible uninsured children within a state, allowing states to effectively target enrollment and outreach activities.

It is also possible that the ACS could be used in place of the CPS in the formula that allocates federal funds for SCHIP across states. The 2007 SCHIP legislation that was passed by Congress included the requirement that the Department of Commerce assess the validity and reliability of the ACS coverage estimates relative to the CPS and report on whether they should be used in place of or in combination with the CPS to estimate the number of low-income uninsured children.

Additionally, the ACS could be used to monitor states‘ progress toward reducing the number of low-income uninsured children, an explicit requirement of the SCHIP legislation. States are required to report annually to CMS on the increase in coverage rates for low-income uninsured children. Many states have used estimates from the CPS as the basis for this assessment even though, for many states, the sample size for low-income uninsured children is very small and the estimates are imprecise (Blewett and Davern 2006b; Kenney, Holahan, and Nichols 2006).

Finally, the ACS will also greatly improve the accuracy and timeliness of evaluation efforts, such as whether or not state and local health reforms have been successful in reducing the number of uninsured. Using the ACS data, it will be possible to simulate eligibility for Medicaid and SCHIP based on the information available on income and household structure in the ACS, but it will be important to benchmark those estimates to those that have been developed for more established surveys such as the CPS, the National Health Interview Survey, or the Medical Expenditure Panel Survey. The estimates from ACS could also be used by state budget analysts and the CBO to “score” legislative proposals at the state and federal level.

The introduction of the health insurance question on the ACS will not be without its challenges. Comparisons between the health insurance coverage estimates from the CPS, the ACS, other federal surveys, and state household survey data will likely complicate health care debates (Blewett and Davern 2006a). This in turn may move the debate away from important discussions on the most effective approaches for improving access and coverage to a political debate about the best way to measure and monitor coverage over time.

In addition, as indicated above, the CPS is currently used in funding formulae for programs such as SCHIP. If the state-level ACS estimates of low-income uninsured children are dramatically different from the CPS estimates, it could create political battles with winners and losers among the states in the federal allocation of funds, even though the ACS estimates would be much more precise than the CPS estimates. Policy tools such as the use of thresholds, hold harmless clauses, and a phase-in of the new estimates over time could be implemented to mitigate the movement of dollars between states (Davern et al. 2003; Blewett and Davern 2006a).

The ACS represents a promising new development for health policy researchers and decision makers at the state and national levels. However, as with all surveys, it will have its shortcomings. It will be important for data users to understand its strengths and weaknesses before using it. This new source of data on the uninsured has the potential to lead to a better understanding of the changing nature of insurance coverage over time.

Supporting Information

Additional supporting information may be found in the online version of this article:

Appendix SA1: Author Matrix.

Please note: Wiley-Blackwell is not responsible for the content or functionality of any supporting materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.