Can We Trust Parental Reports of Child Care Subsidy Receipt?

Abstract

In recent years, research examining determinants and consequences of the means-tested child care subsidy program (the Child Care and Development Fund [CCDF]) has grown dramatically. To measure subsidy utilization, existing studies typically rely on parent-reported measures of subsidy receipt drawn from large surveys. As the research literature on child care subsidies has grown, however, so have concerns about the trustworthiness of parent-reported subsidy use. One way to assess the quality of parent-reported subsidy use is to examine its overlap with another subsidy receipt measure, drawn from a different source. The current paper uses the Fragile Families and Child Well-Being Study (FFCWS), the only existing survey data source that contains an alternate measure of subsidy receipt – based on child care provider report – which permits a comparison to parent-reported measures. We find evidence that increases our confidence in the trustworthiness of parents as accurate reporters of subsidy receipt. In recognition that neither data source reflects “true” subsidy receipt, however, we conclude with a discussion of limitations and steps for future research.

I. Introduction

In recent years, increased attention has been paid to analyzing the determinants and consequences of means-tested child care subsidies funded through the Child Care and Development Fund (CCDF), a program created alongside the passage of welfare reform in 1996 to facilitate the transition of low-income mothers into the workforce. Much of this research capitalizes on rich, national or multi-state survey data to describe the characteristics of subsidy recipients (e.g., Herbst, 2008; Johnson, Martin, & Brooks-Gunn, 2011; Lee et al., 2004; Tekin, 2005; 2007), examine the relationship between subsidy receipt and child care choices (e.g., Crosby, Gennetian, & Huston, 2005; Johnson, Ryan, & Brooks-Gunn, 2012; Ryan, Johnson, Rigby, & Brooks-Gunn, 2011), and understand the implications of subsidy use for child and family well-being (e.g., Hawkinson, Griffen, Dong, & Maynard, 2013; Herbst & Tekin, 2010a; 2010b; 2011; 2012; Johnson, Martin, & Brooks-Gunn, in press).

As the body of literature on child care subsidies has grown, however, so have concerns about the measurement of subsidy receipt in survey data. While surveys offer a wealth of information on family background characteristics, features of child care and early education settings, and indicators of child development and family wellbeing that cannot be captured in administrative data, questions have been raised regarding the trustworthiness of parental reports of subsidy receipt in survey data sets². Nearly all existing subsidy studies using survey data make use of a similar question or set of questions asked of parents about whether the family receives assistance paying for child care from a government source. For instance, in the nationally representative Early Childhood Longitudinal Study–Kindergarten Cohort (ECLS-K), parents were asked: “Did any of the following people or organizations help to pay for…this provider to care for [CHILD]…?”; parents who responded affirmatively that “a social service agency or welfare office” helped to pay for their child’s care were considered subsidy recipients. The ECLS-K has been used in multiple published studies linking subsidy receipt to child and family outcomes (e.g., Herbst & Tekin, 2010a; 2010b). Similar questions appear in other nationally representative studies or multi-state studies, including the Early Childhood Longitudinal Study–Birth Cohort (ECLS-B), the Fragile Families and Child Wellbeing Study (FFCWS), the National Survey of America’s Families (NSAF), the Survey of Income and Program Participation (SIPP), and the National Household Education Survey (NHES), (see Appendix A for exact wording of questions across these data sets; Blau & Tekin, 2007; Forry, 2009; Hawkinson et al., 2013; Ha & Johnson, 2012; Herbst, 2008; Herbst & Tekin, 2010a; 2010b; 2011; 2012; Johnson, Martin, & Brooks-Gunn, in press; Tekin, 2007). In the context of this large and growing body of subsidy research, it is important to investigate the degree to which parental reports of subsidy might overlap with measures of subsidy receipt gathered from alternate sources, such as child care providers.

Motivation

Three primary reasons have been proposed to explain why survey respondents might misreport receipt of a given public benefit: memory effects; benefit confusion; and stigma or selective underreporting due to the perceived sensitive nature of the question (Giannarelli, Adelman, & Schmidt, 2003; Terracol, 2002; Tourangeau & Yan, 2007). In the case of child care subsidies, parents are often asked retrospectively about child care assistance they received in the prior year or since the last wave of data collection. For example, parents in the ECLS-K are asked in the fall of kindergarten whether their child received a subsidy sometime in the 12 months prior to kindergarten entry (e.g., Herbst & Tekin, 2010a). It is possible that, when survey questions about subsidy utilization are temporally far-removed from when families actually receive assistance, the responding parent does not accurately recall whether a specific care arrangement was partially or fully financed via a subsidy.

With respect to benefit confusion, parents may be unclear about which funding stream (e.g. CCDF, Head Start, states’ pre-kindergarten programs, or the Child and Dependent Care Tax Credit) is actually paying for the child’s care. Recent evidence suggests that, for example, some parents confuse a tax credit for a subsidy (Bowman, Datta, & Yan, 2010). Relatedly, confusion can arise among parents paying a co-payment for their child’s subsidized care when the remaining cost of the care not covered by the co-payment is subsidized through a contract with the provider (Giannarelli et al., 2003). Although most subsidies are administered via vouchers given to parents, some states use contracts where the provider is paid directly (Holod, Johnson, Martin, Gardner, & Brooks-Gunn, 2012). Parents whose children receive a subsidized slot in a contracted setting may believe that their co-payment represents the full cost of care and thus that they are not receiving any assistance. A comparable concern pertains to parents whose full child care costs are covered by a subsidy when the subsidy payment mechanism is a contract. Under this scenario, the full cost of care is covered ‘behind the scenes’, because the local subsidy administration agency pays the provider, and thus the care may appear to the parent to be free. Although the provider would know the child’s care was subsidized, the parent may not.

Finally, parents who experience or worry about stigma associated with receiving public benefits may purposefully not report subsidy receipt. Studies examining take-up rates of public programs including but not limited to child care subsidies have found one explanation for low participation to be social stigma (Rosenberg, Nagatoshi, & Roper, 2003; Shlay, Weinraub, Harmon, & Tran, 2004). Specifically, parents have reported concerns about others’ perceptions of them or negative feelings about their own self-worth if they were to accept public assistance (Shlay et al., 2004). It stands to reason, then, that if stigma deters families from applying for subsidies, it may also make them more likely to deny receipt of a subsidy if they do participate in the program. Indeed, one study examining selective non-response to perceived sensitive questions found the highest rates of non-response to be related to questions about income and receipt of public assistance, though not subsidies specifically (Tourangeau & Yan, 2007).

Regardless of the reason for misreporting, concerns around measurement error may have implications for researchers’ interpretation of emerging findings from subsidy studies. In particular, under certain conditions, measurement error can cause ordinary least squares (OLS) regression estimates to be inconsistent and biased (Wooldridge, 2009). Given that the potential consequences of measurement error differ according to whether a variable is expressed as a dependent or independent variable, it is useful to consider both scenarios in the case of subsidy receipt. Analyses that use subsidy receipt as the dependent variable assume that the measurement error is unobserved to the researcher and thus is embedded in the error term (usually represented by “ε” in a regression framework). As long as the measurement error is not systematically related to one or more of the independent variables, the only consequence is larger error variances (standard errors) of the OLS estimates. However, if the measurement error is non-random with respect to the independent variables, there is a possibility that the OLS estimates will be biased. On the other hand, measurement error in subsidy receipt when it is used as an independent variable is thought to be more serious. By definition, the error in subsidy receipt (which is unobserved and included in “ε”) must be correlated with the observed measure, thereby leading to a problem known as classical errors-in-variables (CEV). The key implication of CEV is that it renders inconsistent and biased OLS estimates of the impact of subsidy receipt. In particular, estimates of subsidy receipt will be biased downward—or toward zero—increasing the likelihood of failing to reject the null hypothesis of no subsidy effect when in fact there is one.

In light of this discussion, two questions exist around (1) the trustworthiness of parental report of subsidy receipt, when compared to provider-reported subsidy receipt, and (2) the extent to which the estimated predictors of subsidy receipt are consistent across both sources of information. Using rich, multi-city survey data from the Child Care Supplement (CCS) to the larger Fragile Families and Child Well-Being Study (FFCWS), the current paper seeks to address these important questions in several ways. First, we examine the degree to which parental reports of child care subsidy receipt overlap with an alternate utilization measure drawn from child care providers. Although a provider-based report of subsidy receipt is itself an imperfect measure, greater reporting agreement across parents and providers increases our confidence in the accuracy of both measures. We then predict subsidy receipt from a standard set of family background characteristics used in the literature to study determinants of subsidy receipt. We compare estimates from models predicting provider-reported subsidy receipt to parent-reported subsidy receipt in an effort to understand whether there are systematic or structural differences in the determinants of subsidy receipt across the two approaches to measuring subsidy receipt. Finally, we examine the extent to which family background characteristics are systematically related to whether parents and providers disagree with the subsidy receipt status of a given child. Assuming that disagreement between parents and providers reflects a source of reporting error, our goal is to determine whether this error is random or non-random with respect to a standard set of family background characteristics.

The remainder of this paper proceeds in the following manner. Section II provides background information on the CCDF, and summarizes the small body of work assessing the validity of parent reports of the receipt of public benefits, including child care subsidies. In section III, we introduce the FFCWS and its CCS and discuss the analytic procedures used to test the overlap between parent and child care provider reports of subsidy receipt, while sections IV and V provide the results from our simple tests. We conclude in section VI by discussing the limitations of our work, as well as by offering suggestions for future directions and policy implications.

II. Background

Overview of the Child Care and Development Fund (CCDF)

As stated above, the CCDF was created alongside the passage of the 1996 Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) to support low-income parents’ transition from welfare to work. Specifically, the CCDF program was created to help parents meet the strict work requirements of welfare reform by providing subsidies that were designed to be highly flexible, allowing parents to use their subsidy with almost any provider of their choice (in centers, licensed family child care homes, or with family members, neighbors, or friends). The actual subsidies are administered to eligible families through vouchers and, to a lesser extent, through direct contracts with providers. In 2010, 89 percent of subsidy recipients’ care was subsidized with a voucher, 9 percent of recipients’ care was subsidized through a contract, and the remaining 2 percent of subsidized care was funded with cash (Office of Child Care, 2011).

Since its inception in 1996, when previously separate funding streams for child care were consolidated into the CCDF, funding for child care subsidies has increased substantially. In 1997, for example, the subsidy program cost approximately $3 billion (Committee on Ways and Means, 2012). In 2010, the most recent year for which data are available, the federal government spent just over $7 billion on the CCDF program, providing subsidies to nearly 2 million children per month (Office of Child Care, 2010). To offer some comparison, the Head Start program served 904, 118 children in 2010, with expenditures totaling approximately $7.2 billion (Office of Head Start, 2010), while state-funded pre-kindergarten programs enrolled approximately 1.3 children at a cost of $5.4 billion (NIEER, 2010). Thus, public expenditures on the CCDF child care program rival other publicly-funded early care and education programs, while the number of children served through CCDF is nearly double that of Head Start and public pre-kindergarten.

To be eligible for a child care subsidy, families must meet basic federal requirements which include having at least one child between the ages of zero and 12, having an income below 85 percent of the state median income, and being employed or participating in a state-defined work-related activity (e.g., education, job search, or job training). Beyond these basic rules, though, states have great flexibility in setting eligibility standards. For example, in 2011 nearly all states established income eligibility limits below 85% of SMI (Schulman & Blank, 2011). States also decide which employment and work-related activities count towards subsidy-eligibility, and specify the minimum number of weekly hours that parents with spouses or partners are required to perform.

There is also great between-state variability on family co-payment rates and provider benefit reimbursement rates (Herbst, 2008). Most states require subsidy recipients to cover a portion of their child care costs through a sliding scale fee that varies with income. Some families may be exempt from this requirement, for example, if family income is below the federal poverty line. However, recent data suggest that a number of states set their parent co-payment levels at greater than the recommended 10% of family income, which imposes a significant financial burden on low-income families (Schulman & Blank, 2011). Reimbursement rates – or the maximum amount states will pay a given child care provider – also vary substantially between and within states, and across urban–rural areas, the age and number of children served within a family, and the type of provider used by the family. In theory, states are encouraged to regularly conduct market rate surveys and to set subsidy reimbursement rates accordingly at 75 percent of the local child care price distribution (Greenberg et al., 2000). In practice, however, many states use outdated market data about provider charges or simply set their reimbursement levels below the 75 percent cap (Adams & Rohacek, 2002).

Another key feature of the CCDF is that, unlike some of its predecessor programs, it does not administer benefits as an entitlement, but rather as close-ended block grant.³ The block grant nature of the new system implies that the supply of subsidy benefits does not change in order to meet changes in demand. Therefore, it is common for states to use eligibility rules and other administrative functions to ration benefits according to need or specific household characteristics. For example, some states adjust the income eligibility threshold according to the volume of demand for subsidies. Other states may prioritize certain groups to receive a subsidy, such as current welfare receipts or families residing below the federal poverty line. Rationing is expected to be intense when states cannot meet the demand for subsidies, and these fiscal constraints may prompt the start of wait lists or frozen intake. Currently, 21 states implement one or both of these policies (Schulman & Blank, 2011).

Prior Research

As the first study of its kind, no prior literature exists specifically examining the trustworthiness of parent-reported subsidy receipt measures drawn from multi-state survey data by checking overlap with an alternative measure of subsidy receipt drawn from another source. Therefore, we look to two related bodies of research to inform our analysis: first, studies examining general survey misreporting/underreporting of participation in other public programs; and second, cognitive testing of subsidy-specific survey questions for the recent National Study of Early Care and Education (NSECE).

Researchers have examined the extent of misreporting participation in other social service programs, such as Medicaid, cash welfare, the Special Supplemental Nutrition Program for Woman, Infants, and Children (WIC), and food stamps (now known as the Supplemental Nutrition Assistance Program, or SNAP), using data from large survey datasets that are frequently employed for analyzing determinants and consequences of public benefit use, such as the American Community Survey (ACS), the Current Population Survey (CPS), the Panel Study of Income Dynamics (PSID), the SIPP, and the Consumer Expenditure Survey (CE Survey).

A large segment of work in this area has focused on addressing the so-called “Medicaid undercount”, or the well-validated concern that estimates of Medicaid participation drawn from survey data sources are consistently lower than participation rates drawn from administrative data records (Call, Davidson, Davern, Blewett, & Nyman, 2008; Call, Davern, Klerman, & Lynch, 2012; Davern, Call, Ziegenfuss, Davidson, Beebe, & Blewett, 2008; Davern, Klerman, Baugh, Call, & Greenberg, 2009; Klerman, Ringel, & Roth, 2005). Studies have tended to use either an experimental approach, in which a random sample of survey respondents is drawn from administrative records and then survey respondents’ reports of program take-up are cross-checked with the administrative data, or a matching approach, in which administrative data records are identified and linked with respondents drawn from existing survey data sources and overlap between the two sources is examined (Call et al., 2008; Davern et al., 2008; 2009). National studies have found substantial underreporting of Medicaid take-up, such that approximately 42% of respondents identified in administrative data to be Medicaid recipients self-identify as non-recipients in national survey reports (e.g., Davern et al., 2009). Similarly, a study of Medicaid take-up in California revealed that Medicaid enrollment estimates increase by nearly 40 percent when underreporting is corrected (Klerman et al., 2005). This study also found that underreporting is more severe among families with household incomes above the poverty line, increasing with income, suggesting that stigma may contribute to inaccurate reports of receipt of means-tested benefits.

A handful of studies have also examined misreporting of food stamps, cash assistance, and WIC receipt. One recent study compared survey respondent reports of food stamp receipt drawn from the ACS and CPS to administrative records and found that between 35 and 50% of “true” (identified from administrative data) recipient households underreported food stamp take-up (Meyer & Goerge, 2011). These findings have been echoed in other studies examining misreporting of food stamp use (e.g., Meyer & Sullivan, 2007; Meyer & Goerge, 2010). Similarly, only approximately two-thirds of recipients of other programs such as cash assistance and WIC actually report receiving them (Meyer, Mok, & Sullivan, 2009).

Studies examining the reliability of survey respondents’ reports of public benefit use are informative, but do not address the question of how trustworthy parent reports of child care subsidy receipt are. Thus, the current paper builds on the small body of work looking specifically at how parents interpret questions around child care assistance that are used to construct measures of subsidy use in survey data contexts. In advance of data collection for the NSECE, which recently concluded, researchers conducted cognitive testing of items designed to capture information on subsidy receipt from parent interviews (Bowman et al., 2010).⁴ Researchers found that the series of questions designed to measure subsidy receipt —which are similar to questions in existing national or multi-state surveys (i.e. “does another person or organization help pay…to care for child?”, for which “government agency” was one possible response category) were generally interpreted correctly by respondents during survey piloting. Bowman and colleagues also experimented with asking pre-test respondents if, in the past year, they had received a “child care subsidy from the government”, and found that this question, with the inclusion of the word “subsidy” was also interpreted mostly as intended. Some confusion did exist, though, around the source of the subsidy. For instance, higher-income families (who were likely not eligible to receive subsidies) were less familiar with the terms associated with, and sources of, child care assistance. In addition, several subsidy recipients expressed confusion over the source of assistance when the provider was paid directly, and between tax credits and subsidies, for example.

For a small subsample of pre-test respondents, researchers also interviewed child care providers to obtain information on the family’s subsidy status, and then cross-checked provider reports with parent responses (Bowman, Connelly, Datta, Guiltinan, & Yan, 2010). Only 43 parents participated in this qualitative segment of the NSECE’s design phase; based on this small sample drawn for survey development purposes, these results should be interpreted with caution. Nonetheless, findings support the notion that parents generally are able to correctly identify whether their child’s care is funded via a subsidy or not. The percent agreement between parents and child care providers on the child’s subsidy status was approximately 75%, with one-quarter of parents misreporting their child’s subsidy status. In an overwhelming majority of disagreement cases, the parent reported not receiving a subsidy when the child care provider did; in all of these cases, the parent believed the child was not receiving subsidized care either because the family paid a co-payment that they believed to cover the full cost of care, or because the child received free care (e.g., public pre-kindergarten). The study’s authors conclude that these descriptive results suggest, overall, that parents appear to know whether their child’s care is subsidized.

While informative, given the small sample size and qualitative nature of the study, findings cannot be generalized to large national or multi-site studies. Thus, empirically we still know little about the validity of parent report of subsidy receipt as it is captured in existing survey data sets. Because the FFCWS CCS includes both parent- and child care provider-reported measures of subsidy use, collected via questions asked highly similarly across most large survey data sets, it offers the best and only opportunity to test the accuracy of parent-reported subsidy receipt.

III. Method

Data Source and Sample Creation

Data for this paper were drawn from the Fragile Families and Child Wellbeing Study (FFCWS) Child Care Supplement (CCS). The FFCWS CCS is uniquely suited for this analysis because, as mentioned earlier, the wording of the questions used to construct a parent-reported measure of subsidy receipt closely mirrors the wording of questions used to construct survey-based measures of subsidy receipt from other national or multi-state survey data sets. However, it is the only large, multi-state survey dataset that collected information on child care subsidy receipt from both parents and child care providers, permitting a test of how much parent-reported subsidy use overlaps with child care provider-reported subsidy use.

The FFCWS is a longitudinal birth cohort study designed to examine associations between non-marital childbearing and child and family outcomes in 20 U.S. cities (see Reichman, Teitler, Garfinkel, & McLanahan, 2001 for a detailed description of the larger FFCWS design). Data for the CCS were collected in 2002 and 2003, in 14 of the 20 FFCWS cities. As part of the FFCWS, participants in all 20 cities were visited in their homes when focal children were approximately one years old, and then again when children were approximately three years old. At both home visits, parents were interviewed and children were directly assessed. During the three year visit, families in 14 of the 20 cities were asked if they used child care for 10 hours or more per week; of those families visited at home in the 14 cities (N = 2,650), 1,150 families were eligible for the CCS because they typically used some form of non-parental care for 10 or more hours per week. The CCS, conducted for both home- and center-based care providers, included an interview with the child’s care provider as well as an observational assessment of the quality of the child care setting. For children who received their care in a center-based setting, whenever possible the center director was interviewed; however, when the center director was not available, the child’s teacher was interviewed instead⁵. For simplicity and consistency, we use the term “provider” across both center- and home-based settings, recognizing that for most of the children in center-based care, the “provider” report was given by the center director. For children in a home-based setting, the family child care provider or informal care provider was interviewed.

Almost one-half of the 1,150 eligible families did not have associated provider interviews and observations due to families’ refusal to allow their child’s care provider to participate, provider refusal to participate, or changes in the child care arrangement. Thus, interviews and child care observations were conducted in the child care settings of 777 children. For the purposes of this paper, we reduced our analytic sample to cases with non-missing data for both the provider and parent report of child care subsidies. Thus, our analytic sample included the 604 cases that had data from both sources.

A comparison between our analytic sample and families who were eligible for the CCS but did not participate revealed some statistically significant differences on demographic characteristics. Specifically, participants were more likely to be married and less likely to be cohabiting at the time of the focal child’s birth, although the two groups of mothers were equally likely to be single. Participants also had, on average, higher household incomes at baseline than non-participants. Finally, participants were more likely to have attended or graduated from college. No other statistically significant differences emerged between the two groups on other key family background characteristics such as maternal race, immigrant status, age, household welfare receipt, number of children in the home, and child gender.

Measures

Subsidy receipt

Child care provider report of subsidy receipt offers a comparison for the often-used parent reported measure. Thus, an analysis assessing the overlap of provider and parent report of subsidy receipt corroborates provider report as much as it does the parent report. In effect, the greater the overlap between the two measures, the greater our confidence is in both measures.

Regarding the parent-based measure, mothers were asked during the three-year interview: “Does any person or agency give you money, a voucher, or a scholarship to help pay for child care?” Those who responded affirmatively were then asked “who or what agency gives you money or the voucher or scholarship?” Children whose mothers who responded that their child care assistance came from “a government agency” or “a child care center” were considered to be subsidy recipients (coded “1”). Children whose mothers reported no assistance paying for child care, and those who reported that their assistance came from an alternate source (e.g., a relative or an employer), were considered to be subsidy non-recipients (coded “0”). As mentioned earlier, this method of identifying subsidy recipients from parent report is similar to that used in other large-scale national surveys including the Early Childhood Longitudinal Study-Kindergarten cohort (ECLS-K), Early Childhood Longitudinal Survey-Birth cohort (ECLS-B), National Survey of America’s Families (NSAF), Survey of Income and Program Participation (SIPP), and National Household Education Survey (NHES). Of the 777 cases with child care provider data, 604 families had data on parent report of subsidy receipt⁶; of the 604 families with data on parent-reported subsidy receipt, the subsidy utilization rate according to parent report was approximately 17 percent, a figure that is within the range of reported take-up rates from other studies using parent-reported survey data (see Blau & Tekin, 2007; Hawkinson et al., 2013; Herbst, 2008; Herbst & Tekin, 2010a).

To permit a comparison with parent-reported subsidy receipt, we constructed an equivalent measure based on the provider report. During the child care provider interview, providers were first asked “is any part of [focal child’s name] care paid for by government support?”⁷ and those that responded affirmatively were then asked “what local, state, or federal programs provide these funds?” Children whose providers responded that the focal child did not receive government support were coded “0”, as were children whose providers reported that the child’s care was funded through a board of education, a state pre-kindergarten program, Head Start or Early Head Start, a private non-profit foundation, and those who reported sponsorship by or affiliation with a Head Start or public school. All remaining children in the subsidy recipient group (coded “1”) had providers who either explicitly reported that the child received a subsidy funded by the Child Care and Development Fund (CCDF) or that the child received government support from a state or local social service agency likely to receive CCDF funding. Of the 777 children with child care provider information, approximately 26 percent received subsidies according to provider report.⁸

Covariates

In multivariate models, we included a standard set of family background characteristics used in other subsidy studies (e.g., Herbst, 2008; Johnson et al., 2011). All covariates (except when noted) were drawn from the one-year parent interview, when children were approximately 1 year old, either because they are time-invariant or, if time-varying, to reduce the threat of simultaneity bias. For time-varying covariates such as maternal education, for instance, it is possible (though unlikely) that mothers with higher levels of education are not more likely to seek subsidies but that the experience of having a subsidy induces these mothers to obtain higher levels of education. To be confident in the directionality of the association between our covariates and key independent variable, we thus drew our covariates from a wave earlier than when subsidy receipt was measured.

The covariates were: maternal race (three dummy variables for black, Hispanic, and other race, with white omitted), maternal education (three dummy variables for less than high school, high school diploma/GED, some college, with BA or more omitted), an indicator variable for maternal marital status (1=single), an indicator for maternal immigrant status (1=U.S. born), maternal employment (two dummy variables for employed full-time and part-time, with unemployed omitted) and education status (an indicator for whether the mother was in school [1=in school]), household structure (one dummy variable for whether mother has more children in addition to the focal child [=0] versus the focal child only [=1], and one dummy variable for whether there are other adults in the household [=1]), indicator variables for whether the mother received welfare, WIC, or food stamps, household income-to-poverty ratio, an indicator for whether the child is the mother’s firstborn, maternal age at focal child’s birth (in years), and the focal child’s age at the 3-year in-home interview (in months).

IV. Results

Sample Characteristics

Table 1 presents summary statistics for all covariates in the analytic sample. Given that the FFCWS oversampled families in large cities, as well as unwed births, it is not surprising that the sample appears to be more disadvantaged compared to other large survey data sets⁹. Additionally, as families with younger children are more likely to be disadvantaged, and the FFCWS CCS only included families with 3-year-old children, we might expect this sample to be less advantaged than a national sample. Indeed, in the current sample most mothers are non-White, more than half the analytic sample has a high school education or less, and nearly half the mothers are unmarried. In addition, approximately one-quarter of mothers received welfare and one-third received food stamps in the previous year, while approximately three-quarters of sample mothers received WIC.

Table 1.

Summary Statistics for the FFCW CCS Analysis Sample

Variable	Mean (SD) /Percentage	Range
Mother is white (omitted)	0.175	0–1
Mother is black	0.621	0–1
Mother is Hispanic	0.179	0–1
Mother is other race	0.250	0–1
Mother has < HS education	0.224	0–1
Mother has HS/GED	0.338	0–1
Mother has some college	0.295	0–1
Mother has BA+ (omitted)	0.143	0–1
Mother is single	0.435	0–1
Mother was born in US	0.920	0–1
Mother works full time	0.530	0–1
Mother works part time	0.192	0–1
Mother is unemployed (omitted)	0.278	0–1
Mother is attending school	0.189	0–1
Mother has 1 child	0.315	0–1
Mother lives with any other adult	0.773	0–1
Mother received welfare last year	0.238	0–1
Mother received WIC last year	0.769	0–1
Mother received food stamps last year	0.359	0–1
Household income-to-poverty ratio	2.5 (2.4)	0–12.3
Child is firstborn	0.404	0–1
Mother’s age (in years)	25 (5.9)	15–44
Child’s age (in months)	35.3 (2.0)	31–45

Notes: N= 604

Three Tests of the Quality of Parent Report of Subsidy Receipt

In the first test, we examined the extent to which there is agreement between provider-and parent-based reports of child care subsidy receipt through simple bivariate cross-tabulations. Table 2 displays the overlap of parent and provider reports. In particular, each cell represents the cross-classification of each child’s subsidy utilization status according to the parent and provider reports. The upper-left-hand and lower-right-hand cells indicate the number of children for whom there is agreement between parents and providers, while the upper-right-hand and lower-left-hand cells indicate the number of children for whom there is disagreement.

Table 2.

Overlap of Parent and Provider Reports of Child Care Subsidy Receipt

	Parent Report (no. of cases)
	Subsidy	No Subsidy	Total
Provider Report (no. of cases)
Subsidy	104	70	174
No Subsidy	62	368	430
Total	166	438	604
Percent agree – subsidy	62.7%	--
Percent agree – no subsidy	--	84.0%
Overall Agreement Rate: 78.1 Percent

Generally speaking, our results suggest that there is a moderate degree of overlap between provider and parental reports of child care subsidy receipt. For example, of the 166 children who received subsidies according to the parent report of subsidy receipt, 104 received subsidies according to provider report (agreement rate of 62.7 percent). Of the 438 children who did not receive subsidies according to parent report, 368 did not receive subsidies according to provider report (agreement rate of 84 percent). These figures represent an overall agreement rate between parents and child care providers of approximately 78 percent.

For the second test, we explored whether the predictors of subsidy receipt differ depending on whether subsidy receipt was reported by the provider or the parent. Table 3 presents a comparison of estimates (odds ratios) from logistic regression models predicting subsidy receipt from all covariates. The first set of results comes from the parent report subsidy receipt model, and the second set of results comes from the provider report model. Looking first at the parent model, we find that mothers who dropped out of high school, as well as mothers who completed high school and those who attended some college are all more likely to use subsidies than are mothers who completed college. Additionally, mothers who received other public benefits (e.g., food stamps) are more likely to use a child care subsidy. The presence of another adult in the home and increased maternal age are associated with reduced likelihood of subsidy use. Such patterns are similar to those emerging from other studies of subsidy utilization (e.g., Herbst, 2008; Johnson et al., 2011; Shlay et al., 2010).

Table 3.

Logistic Regression Predicting Subsidy Receipt from Family Background Characteristics

	Parent Report		ProviderReport
	Odds Ratio	Robust SE	Odds Ratio	Robust SE
Mother is black	1.47	0.56	1.6	0.53
Mother is Hispanic	1.18	0.52	1.13	0.45
Mother is other race	3.00	2.84	1.18	0.95
Mother has < HS education	7.34	5.37 *	1.69	0.87
Mother has HS/GED	9.09	6.40 **	1.75	0.79
Mother has some college	6.22	4.42 *	1.76	0.77
Mother is single	1.45	0.35	1.43	0.34
Mother was born in US	2.63	1.57	0.95	0.42
Mother works full time	1.34	0.36	0.82	0.21
Mother works part time	0.98	0.32	1.02	0.31
Mother is attending school	1.09	0.3	0.9	0.24
Mother has 1 child	0.75	0.23	0.65	0.19
Mother lives with other adult	0.58	0.16 +	0.44	0.12 **
Mother received welfare	1.32	0.37	1.04	0.29
Mother received WIC	1.4	0.47	0.72	0.2
Mother received food stamps	1.87	0.51 *	1.42	0.37
HH income-to-poverty ratio	1.02	0.07	0.99	0.06
Child is firstborn	0.68	0.22	0.78	0.23
Mother’s age	0.94	0.02 *	0.94	0.02 *
Child’s age	1.04	0.05	0.97	0.05

Notes: N=578

⁺p<0.10;

^*p<0.05;

^**p<0.01

When the parent and provider estimates are compared, we find few systematic differences between the two models. In general, the sign, size, and significance of coefficients across the two models is the same, with the exception of maternal education and food stamp receipt, which are only predictive of parent-reported subsidy receipt. To confirm this intuition, we conducted a formal specification test of the null hypothesis of equal coefficients (on the family background characteristics) across the provider and parent report models. Results from these post-estimation Wald tests confirm that only maternal education differentially predicts subsidy receipt across both measures. In fact, of the 20 variables in the model, coefficients on only three (the maternal education dummy variables) were statistically significantly different across the two models. Specifically, in the model predicting parent-reported subsidy use, mothers with lower levels of education were more likely to report subsidy receipt than were mothers with a college degree; the same pattern emerged in the model predicting provider-reported subsidy use, although the coefficients in the provider-reported model did not achieve statistical significance.

Finally, we investigated whether disagreement between the provider- and parent-reported measures of subsidy receipt is associated with the family background characteristics. We suspect that cases in which parents and providers disagree about a child’s subsidy receipt status represents a potential manifestation of the measurement error in subsidy receipt. If the observable family characteristics are uncorrelated with the measurement error, as conceptualized by disagreement between the two reports, this would bolster our confidence in both measures.

To implement this test, we created a binary indicator that equals one if the parent and provider reports of subsidy receipt did not correspond and a zero if they did. We then estimated a logistic regression of this binary indicator on the full set of family background characteristics. We also estimated a multinomial logistic regression model in which we compared both directions of disagreement between parents and providers (parent reports subsidy receipt but provider does not; provider reports subsidy but parent does not) to a base category of agreement. Results from the logistic regression predicting any disagreement did not differ from results produced by the multinomial model, so for ease of interpretation only the results from the former are presented here.

Table 4 presents the results from the logistic regression model predicting disagreement between provider and parent reports of child care subsidy receipt. Overall, it appears that disagreement is relatively random with respect to families’ observable characteristics. In fact, only two of the 20 family characteristics have a statistically significant association with disagreement in the subsidy measures. Specifically, disagreement between provider and parent report of subsidy receipt was less likely among mothers with only one child (the focal child) as well as those who live with another adult. It is noteworthy that disagreement between parents and providers was not systematically related to parents’ educational attainment, in light of the results from Table 3 indicating that lower-educated mothers were more likely to self-report subsidy receipt, whereas there was no difference between low- and higher-educated mothers on provider-reported subsidy receipt. In particular, it might be expected that parents with low levels of education are more prone to misreporting their child’s subsidy receipt status (as was seen in Table 3). However, our results suggest that low- and high-educated mothers are equally likely to disagree with their child care provider about whether the child receives a subsidy. A potential explanation for this result is that provider reports of subsidy receipt are based in part on information gleaned from parents. Therefore, misreporting by low-educated parents might be reflected in both measures, producing less disagreement by level of education.

Table 4.

Logistic Regression Predicting Disagreement Between Providers and Parents

	Odds Ratio	Robust SE
Mother is black	1.60	0.58
Mother is Hispanic	0.80	0.36
Mother is other race	0.38	0.42
Mother has < HS education	1.77	0.89
Mother has HS/GED	1.12	0.51
Mother has some college	1.05	0.47
Mother is single	0.93	0.24
Mother was born in US	0.57	0.25
Mother works full time	0.86	0.23
Mother works part time	1.01	0.33
Mother is attending school	1.41	0.39
Mother has 1 child	0.48	0.16 *
Mother lives with any other adult	0.36	0.11 **
Mother received welfare	1.15	0.35
Mother received WIC	1.00	0.32
Mother received food stamps	0.83	0.24
Household income-to-poverty ratio	0.90	0.06
Child is firstborn	1.55	0.49
Mother’s age	1.00	0.02
Child’s age	1.01	0.06

Notes: N=578

⁺p<.10;

^*p<.05;

^**p<.01

IV. Supplementary Analyses

We conducted several additional analyses to check the sensitivity of our results to alternative model specifications. First, we experimented with a more inclusive definition of subsidy by recoding children who were reported by their child care provider to be enrolled in Early Head Start, Head Start, or public pre-kindergarten as subsidy recipients. When we recalculated overlap between parents and providers, the agreement rate decreased from 78 percent to 64 percent, suggesting that parents generally do not consider other sources of publicly-funded care to be a “subsidy” per se and thereby bolstering our confidence in the quality of the subsidy receipt measure.

We then considered the possibility that disagreement between parents and providers could arise if the child’s teacher, rather than the center director, was interviewed for children in center-based care. If the director handles payments for a given center but the teacher was interviewed, for example, the teacher may not have sufficient knowledge of the source of child care assistance the focal child was receiving to accurately report on subsidy receipt. To address this, we re-estimated our statistical models with the inclusion of a dummy variable indicating whether the center director or teacher was the respondent for the center-based provider interview. This indicator was never significant, and its inclusion in the regression models did not change our results. Additionally, the bivariate estimate of agreement was unchanged when center-based teachers were excluded.

Another source of disagreement between parent and provider reports could be the amount of time that elapsed in between the parent and provider interviews. As mentioned above, parents were interviewed first, answering questions about child care assistance as well as giving consent for the child care provider to be interviewed later. Based on information collected in this parent interview, child care providers were then contacted and, when possible, interviewed. In most cases, these two interviews occurred within six months of each other. However, in some cases, the distance between the parent interview and the provider interview was up to, or (rarely) slightly over, one year. The questions asked of the parent and of the provider both referred to current subsidy receipt, but, because spells of subsidy receipt are short, it is plausible that a child who was receiving a subsidy at the time of the parent interview was no longer a subsidy recipient at the time of the provider interview. To account for this, we included a series of dummy variables in our models representing distance in months between the parent and child care provider reports of subsidy receipt (within 3 months, 3–6 months, 7–12 months, with more than a year omitted). None of these variables were ever significantly associated with subsidy receipt or disagreement.

Next, we re-estimated the models predicting subsidy receipt with the inclusion of an observed measure of child care quality. To account for the possibility that directors of higher-quality arrangements may be more likely to know whether the focal child’s care was funded with a subsidy, we added a continuous measure of child care quality as a control to our regression models. Observed quality was assessed in center-based care settings using the Early Childhood Environment Rating Scale – Revised (ECERS-R; Harms et al., 1998) and in home-based care using the Family Day Care Rating Scale (FDCRS; Harms & Clifford, 1989). Scores on ECERS-R and FDCRS items range from 1 to 7, where 1 indicates inadequate quality care and 7 indicates excellent care. Quality was significantly associated with subsidy receipt as reported by both parents and providers. Specifically, children in higher-quality child care settings were more likely to have parents (OR=1.16, SE=0.08, p<0.05) and providers (OR=1.17, SE=0.08, p<0.05) report that they received a subsidy¹⁰. Curiously, child care quality also predicted disagreement between parents and providers such that children in higher-quality care were more likely to have parents and providers who disagreed on whether that care was subsidized (OR=1.21, SE=0.09, p<0.01). It could be that parents who used higher quality arrangements felt greater stigma about receiving a subsidy, and were thus less likely to report subsidy receipt. Or, perhaps parents with children in higher quality care settings are slightly more advantaged and are not aware that some or all of their child’s care is subsidized. Indeed, as mentioned above, Bowman and colleagues (Bowman, Datta, & Yan, 2010) found that higher income families were less likely to be familiar with terms associated with sources of assistance in paying for child care. Regardless of the explanation, though, all of these results should be interpreted cautiously, as child care quality is likely to be endogenous in a model of subsidy receipt.

To address the possibility that differences in the child care landscape across states (such as early care and education funding, teacher training, or supply of child care workers) could influence our estimates, we re-ran our models with state fixed effects. In the parent and provider subsidy receipt equations, the sign and size of the coefficients were unchanged with the inclusion of dummy variables for the state in which the parent interview occurred. However, in the model predicting parent report of subsidy receipt, whether the mother was born in the U.S. became statistically significant (p<0.05) while prior receipt of food stamps was reduced to borderline significance. In the model predicting provider report of subsidy receipt, whether the mother is black (versus white) became significant (p<0.05). The pattern of significant results did not change in the model predicting disagreement between parents and providers.

Finally, we tested overlap between parent and provider reports of whether the child was enrolled in Head Start, another publicly-funded early childhood program. Head Start may be more easily identified by both parents and providers: it has been in existence since 1965 and is a more nationally recognizable program auspice than subsidies. Additionally, many of the parents of children in our sample may have themselves attended the program. We reasoned that overlap between parents and providers on whether the child was attending a Head Start center could provide a benchmark for expected agreement on subsidy receipt. Given that children in our sample were 3 years old, few actually attended Head Start (the majority of children in Head Start programs are age 4 and older; Office of Head Start, 2010). Nevertheless, we cross-classified parents and providers on responses to questions about whether the child attended a Head Start or Early Head Start program and achieved an overall agreement rate of 75 percent. This rate is actually slightly below that for the overlap between parent and provider reports of child care subsidy receipt.

V. Summary, Limitations, and Implications

This paper aimed to empirically evaluate the trustworthiness of parent report of subsidy receipt by comparing it to provider report of subsidy receipt. Consistent with the only other prior study, we found a moderate degree of overlap (78% agreement) between child care providers and parents in their report of whether the child received a subsidy. Furthermore, multivariate models predicting subsidy receipt from provider- versus parent-report revealed few statistically significant differences in family background characteristics associated with subsidy receipt, and thus do not point to important structural differences between the two modes of measuring subsidy receipt. Finally, only two characteristics were systematically related to disagreement between providers and parents: number of children and the presence of other adults in the household. Taken together, our results suggest that parental report of subsidy receipt does not differ dramatically from provider report and that whatever disagreement that does exist appears to be random and therefore is unlikely to bias OLS estimates of the predictors or effects of subsidy receipt.

Although the percent agreement of subsidy reporting between providers and parents was moderately high, it is worth considering what explains the 22 percent of cases that did not agree. As discussed earlier, this disagreement may represent underreporting. Involuntary underreporting may stem, at least in part, from the fact that interviewers administering survey questions are not trained explicitly to extract information on receipt of the public benefit of interest (Klerman et al., 2005). This possibility is also reflected in the results from the cognitive testing phase of the NSECE in which the authors concluded that using the subsidy program-specific name might increase the number of respondents who recognize and report subsidy receipt (Bowman et al., 2010).

Underreporting in the present study could also be voluntary, stemming from stigma. Indeed, two prior studies found that underreporting of public benefit receipt was most severe among families with incomes above the poverty line (Bowman et al., 2010; Klerman et al., 2005). This suggests that families with incomes above the poverty line who actually do receive subsidies may be misclassified as non-recipients if they are embarrassed to respond honestly; likewise, to the extent that non-poor families do not understand questions about subsidies or sources of child care help and incorrectly respond affirmatively to questions of benefit receipt, true non-recipients could be misclassified as recipients. Of course, stigma about receipt of means-tested public benefits will continue to exist and is responsible for some amount of chronic underreporting. Regardless of whether parental underreporting of subsidy receipt is voluntary or involuntary, however, researchers have acknowledged its existence and have thus suggested that estimates of subsidy receipt drawn from survey data best be thought of as minimum estimates of participation (Giannarelli et al., 2003).

Beyond parental underreporting, it may be that disagreement between parents and child care providers is indicative of limitations in provider report. Just as we assume that the extent of overlap between parents and providers offers corroborative evidence for the strength of both measures, discrepancies between the two may suggest problems with one or both sources of information. We have discussed reasons that parents may misreport subsidy receipt; however, child care providers could also be misreporting. For instance, it is possible that providers who care for multiple children confuse one child’s funding source for another. It may also be that providers who receive blended funding report a subsidy when in fact the majority of the child’s care is funded through another, non-subsidy source. And, because neither parents nor providers were asked about CCDF subsidies specifically, it is possible that children receiving child care assistance through other sources were mistakenly classified as subsidy recipients. For instance, a parent who receives help paying for child care through his or her local YMCA may have reported receiving assistance directly from his or her child’s child care center, and thus could have been mistakenly categorized as receiving a subsidy.

Caution should be taken in generalizing the findings in this paper to other survey data sets. Although the wording of the parent-reported subsidy questions in the FFCWS closely approximates the wording of subsidy-related questions used to construct measures of subsidy receipt in other large survey data sets (see Appendix A), as mentioned earlier the FFCWS only sampled families in large cities and dramatically oversampled unwed births. Although these characteristics may be more representative of the subsidy-eligible population than characteristics of other samples, our findings are nonetheless only generalizable to the population represented by the FFCWS. Additionally, the CCS only sampled families with 3-year-old children who used some type of non-parental care, excluding parents who use no non-parental care (and, most probably, parents who are not in the labor force). We cannot know how sample selection may have affected our results; therefore, future research should aim to collect data on subsidy receipt from parents and child care providers so that tests similar to the one conducted in the present analysis can be replicated on larger, nationally representative data sets.

In addition to collecting data from child care providers as well as from parents, future studies should build on the work of Bowman and colleagues (2010) and consider including the word “subsidy” in surveys, and defining and differentiating subsidies from other sources of assistance like tax credits and flexible spending plans. The authors of that study also recommend using the state-specific subsidy program name (e.g., Illinois Action for Children) or the name of the local subsidy administration office (e.g., New York City ACS), which subsidy recipients may be more likely to recognize, to reduce inaccuracies in self-report of subsidy receipt. However, this is complicated in large national or multi-state surveys because of logistical difficulties associated with incorporating local agency- and program-specific names into the survey. Additionally, the NSECE only recently concluded data collection and the cognitive testing occurred in 2010, after the large national surveys that have been recently used to study subsidies (i.e. the ECLS-K, ECLS-B, FFCWS, NSAF). Therefore, the recommendations of Bowman and colleagues were not incorporated into the way subsidy was measured in these other surveys. However, these recommendations will be important in future surveys that include subsidyrelevant questions.

Another step for future research will be to merge administrative data with rich survey data that contains subsidy receipt reported by both parents and providers. Indeed, as discussed earlier a number of studies investigating the Medicaid undercount have successfully used either an experimental or matching approach to capitalize on both administrative and survey data (e.g., Call et al., 2012). As part of the state subsidy administrative tracking system, states routinely collect a range of information on the sources and amounts of child care assistance a family receives. These administrative data sets represent a source of information on subsidy receipt that is arguably more accurate than either parent or child care provider report, although neither are administrative records perfect given possible time-lags between benefit receipt and data entry. Nevertheless, administrative data records to offer an alternative basis of information on subsidy receipt that could be cross-checked with parent-reported survey measures, using either an experimental or matching approach to combine administrative and survey records. Unfortunately, it was not possible in the current study to link state administrative data with the FFCWS data set. Thus, this represents both an important next step for researchers as well as a limitation of the current study.

A final important future direction will be for researchers to attempt to quantify the amount of error generated in estimates of the impact of subsidy receipt when it is measured via parent-reported survey data, and to detail the implications of that error for studies using subsidy receipt as a predictor. As the first paper of its kind, the current analysis sought primarily to identify the existence of potential error in parental reports of subsidy receipt using several basic tests, and then to check whether that error might be correlated with family demographic characteristics. Given the limitations of the data – the inability to link to administrative records and the cross-sectional nature of the FFCWS CCS – we look to future research to use the administrative data as a benchmark for “true” subsidy receipt to generate an estimate of the magnitude of likely bias.

In spite of its limitations, results from this study offer suggestive evidence that increases our confidence in parent-reported measures of subsidy receipt as they are collected in other large survey data sets with similarly worded questions about subsidy use. The moderately high degree of overlap between parent and provider report found in the current study suggests that these two sources of information on subsidy receipt are satisfactorily consistent with each other. However, researchers are encouraged to interpret results of subsidy studies that draw their data from surveys alone with caution, as parent or provider reported subsidy receipt, without a check with administrative data, may result in an undercount of subsidy use. The ability to check parent (and provider) report of subsidy receipt against administrative records in future studies is a crucial next step for the subsidy research community. This is especially important given that prior work that has found significant underreporting of public benefit receipt in survey data sets when compared to administrative records.

Research Highlights.

• Most subsidy research uses parent-reported subsidy use, drawn from surveys

• We explore overlap between parent- and child care provider-reported subsidy use

• Using several simple tests, we find a modest degree of overlap

• Findings can inform the conduct of subsidy research, a fast-growing research area

Appendix A

Wording of parent-reported child care assistance questions and associated response categories, across survey datasets

Data source	Question(s)
ECLS-B11	“Do any of the following people or organizations help to pay for [CHILD’S PRIMARY CARE ARRANGEMENT]? How about…”
	• A social service or welfare agency
ECLS-K12	“Did any of the following people or organizations help to pay for this…provider to care for [CHILD]…”
	• A social service agency or welfare office
FFCWS13	“Do you receive money, a voucher, or a scholarship to help pay for [CHILD’S] care?”
	“Who or what agency gives you money or the voucher or scholarship?”
	• Government agency
NSAF14	“Did any person or agency help pay for part of [CHILD’S] child care?”
	“What person or agencies paid for or provided child care for [CHILD] so that you didn’t have to pay for it?”
	• Welfare agency, income maintenance, social service agency, family service, or child care resource and referral agency
SIPP	“Did anyone help you pay for all or part of the cost of any child care arrangements for [CHILD]? By this I mean a government agency, an employer, or a friend.”
	“Who or what agency helped pay for this arrangement?”
	• Government (Federal, state, or local government agency, or welfare office)
NHES ECPP	“Did you receive help from any of the following sources to pay for child care?”
	• TANF agency
	• Social service, welfare, or child care agency

Note. Data sources with footnotes indicate that the data source and associated subsidy question(s) has been used in existing, published research. Data sources without footnotes are those on which no published studies exist, but are included in this list to demonstrate continuity across datasets.