The Effect of New Insurance Coverage on the Health Status of Low-Income Children in Santa Clara County

Abstract

Objective

To examine whether providing health insurance coverage to undocumented children affects the health of those children.

Data Sources/Study Setting

The data come from a survey of 1235 parents of enrollees in the new insurance program (“Healthy Kids”) in Santa Clara County, California. The survey was conducted from August 2003 to July 2004.

Study Design

Cross-sectional study using a group of children insured for one year as the study group (N = 626) and a group of newly insured children as the comparison group (N = 609). Regression analysis is used to adjust for differences in the groups according to a range of characteristics.

Data Collection

Parents were interviewed by telephone in either English or Spanish (most responded in Spanish). The response rate was 89 percent.

Principal Findings

The study group—who were children continuously insured by Healthy Kids for one year—were significantly less likely to be in fair/poor health and to have functional impairments than the comparison group of newly insured children (15.9 percent versus 28.5 percent and 4.5 percent versus 8.4 percent, respectively). Impacts were largest among children who enrolled for a specific medical reason (such as an illness or injury); indeed, the impact on functional limitations was evident only for this subgroup. The study group also had fewer missed school days than the comparison group, but the difference was significant only among children who did not enroll for a medical reason.

Conclusions

Health insurance coverage of undocumented children in Santa Clara County was associated with significant improvements in children's health status. The size of this association could be overstated, since the comparison sample included some children who enrolled because of an illness or other temporary health problem that would have improved even without insurance coverage. However, even after limiting the study sample to children who did not enroll for a medical reason, a significant association remained between children's reported health and their health coverage. We thus cautiously conclude that Healthy Kids had a favorable impact on children's health.

The health insurance expansions for children, both for Medicaid and the newer State Children's Health Insurance Program (SCHIP), have increased the rate of public health insurance for children nationwide and improved access to care (Cohen and Coriaty-Nelson 2003; Cunningham 2003; Brown and Lavarreda 2004; Dick et al. 2004; Kenney and Chang 2004; Szilagyi et al. 2004). In spite of these gains, more than eight million U.S. children (11 percent) were without health insurance in 2003. Among certain groups such as Hispanic children, the proportion of uninsured children exceeded 20 percent (DeNavas-Walt, Proctor, and Mills 2004). Consequently, some policymakers and advocates at the national, state, and local levels continue to focus on extending coverage to these children.

Few would disagree that the overall purpose of providing new health insurance to children is to improve and maintain their health, yet measuring the impact of new health insurance on child health status has been very difficult. While intuitively “self-evident” that health insurance affects health, some studies show an impact of having insurance and some show no impact. These equivocal findings on such an important policy issue result from the methodological challenges associated with studying health status, where random assignment to obtain unbiased comparison groups is extremely rare and measurement issues abound. Children are generally healthy, and much of their health care is oriented towards preventing future illness. Thus, developing measures of their health that are sensitive to changes in health insurance status is very difficult, and a long time period may be needed (indeed the full life span) to know the full benefit of such care.

This paper provides new evidence on the effects of insurance coverage on children's health by looking at how an innovative, local coverage program known as Healthy Kids affects the health of the low income, undocumented children it serves. The paper is part of a larger evaluation of the Santa Clara Children's Health Initiative (CHI), which seeks to provide coverage to all children in the county whose family incomes are below 300 percent of the federal poverty level. For more information on the program, see http://www.chikids.org. Those children who are not eligible for other government programs (undocumented children below 250 percent of poverty and uninsured children in families with incomes between 250 and 300 percent of the federal poverty level) are enrolled in a new insurance product called “Healthy Kids.” A previous paper examining the impact of Healthy Kids showed dramatic improvements in enrollees' health care access and use of services, suggesting the potential for gains in health status as well. More detailed results from the larger evaluation, along with more information on the CHI and its efforts, are available in Trenholm et al. (2005).

IMPACT OF INSURANCE ON CHILD HEALTH

Several recent comprehensive literature reviews have summarized the conflicting findings regarding the impact of health insurance coverage on health status (Levy and Meltzer 2001; Hadley 2003; Hughes and Ng 2003). Natural experiments and the RAND health insurance experiment provide some evidence of the impact of insurance coverage on adults, and results are mixed with some studies showing positive effects and some showing no effect (Brook et al. 1983; Lurie et al. 1984,1986; Keeler et al. 1985; Baker et al. 2001; Kunitz and Pesis-Katz 2005). Most of the studies reviewed have focused on either infant health or adult health, with very few studies of child health.

Measuring the effect of health insurance for children is even more challenging, as children are more often healthy making it difficult to detect effects. In the 1970s, the RAND health insurance experiment, the only study of the issue using random assignment, found that in general the health status of children receiving free care did not differ from those whose parents paid for a portion of their care, although there were some exceptions such as for hemocrit levels (Newhouse and The Insurance Experiment Group 1993). However, the experiment did not study uninsured children.

Table 1 summarizes the results from 8 more recent studies that have attempted to address this issue using a variety of methods. Three of the studies (Currie and Gruber 1996; Kaestner, Joyce, and Racine 2001; Racine et al. 2001) use national level data sets—the Current Population Survey, a national sample of hospital discharges, or the National Health Interview Survey—for the time period surrounding the major expansions for Medicaid for children, and compare health outcomes before and after the expansions. These national level studies show evidence of reductions in child mortality and hospital admissions for ambulatory care sensitive conditions for the youngest poor children.

Table 1.

Overview of Studies of Effect of Insurance Expansions on Child Health

Author	Study Date	Place of Study	Data Source	Methods	Conclusions	Issues
Currie and Gruber (1996)	1985–1993	United States	Current Population Survey	Examine trends statewide in mortality for children according to percent of children covered by Medicaid in the state	Child mortality rates fell more rapidly in states with higher increase in Medicaid coverage for children	No individual-level data on mortality. No subgroup analysis
Kaestner, Joyce, and Racine (2001)	1988–1992	United States	Nationwide sample of hospital discharges (Health Cost and Utilization Project)	Rates of admission for Ambulatory Care Sensitive Conditions for children in low income and high income areas compared before and after Medicaid expansions	ACS admissions reduced only for children ages 2–6 in lowest income areas	No individual-level data on insurance status or income
Racine et al. (2001)	1989 and 1995	United States	National Health Interview Survey	Compared changes in restricted activity days before and after Medicaid expansions for poor and nonpoor children	No significant difference in trends in restricted activity days between poor and nonpoor children	Unclear whether rates were adjusted for age
Holl et al. (2000)	1991–1993	Six western New York counties	Telephone survey of parents of children enrolled in new insurance (CH Plus)	Parents of children age 0–5 continuously enrolled for 9 months were asked whether their child's health improved as a result of the program	25% of parents felt their child's health had improved after enrollment	No comparison group
Skarr et al. (2002)	2001–2002	California	Mailed survey to parents of children enrolled in Healthy Families (SCHIP)	Studied changes to health related quality of life from 2001 to 2002, for children in bottom quartile of health related quality of life in 2001	Health related quality of life was higher after 1 year of enrollment	No comparison group. Only 34% of parents responded to both waves of the survey
Damiano et al. (2003)	1999–2000	Iowa	Mail survey (with telephone follow-up) of parents of children enrolled in hawk-i (SCHIP)	Compared perceived health status, school days missed, and functional limitations at time of enrollment in program to year after enrollment	Children's health status improved according to all three measures, after enrollment in hawk-i	No comparison group. Only 21% of parents responded to both waves of the survey
Fox et al. (2003)	1999–2000	Kansas	Mail survey of parents of children enrolled continuously for 1 year in Health Wave (SCHIP), at enrollment and 1 year after enrollment	Compared perceived health status, perception of health improvement, and school days missed, at enrollment and 1 year after enrollment	After 1 year of enrollment children's health status improved and fewer school days were missed due to health. More parents perceived an improvement in their child's health in the past year	No comparison group. Only 48% parents responded to both waves of the survey
Flores et al. (2003)	1997–1998	Boston Medical Center	In-person interview with parents of children admitted with one of 16 conditions. In-person or telephone interview with child's primary care and inpatient physicians	Interviews examined whether problems with access to care led to hospitalization	Uninsured children were almost three times as likely as privately insured children to have an avoidable hospitalization, according to all three respondents	Small sample size: of 554 children in the study, only 16% (88 children) were uninsured

Another four studies (Holl et al. 2000; Skarr et al. 2002; Damiano et al. 2003; and Fox et al. 2003) examine changes in one or more health status measures for children who enrolled in the SCHIP programs of three states (California, Iowa, and Kansas) or a precursor child health insurance program in New York State. These studies all used pre–post designs without comparison groups, and all but one had a very low response rate ( < 50 percent). With these caveats in mind, all four studies found improvements in child health status after a year of enrollment in insurance.

The final study (Flores et al. 2003) used parent and physician descriptions of the circumstances leading up to hospitalization for a selected list of conditions (those amenable to the prevention of hospitalization through primary care access) to determine which were truly preventable. They found that uninsured children were more likely to have a preventable hospitalization. However, the sample size of uninsured children was very small, and the study was limited to one medical center.

In summary, seven of the studies show some improvements and one shows no impact of insurance on health status. Only one study included more than one measure of health status, and half of them studied outcomes that are very rare in children (hospitalization or death). Consequently, as some researchers noted in their conclusions, there is a great need for further research on this important topic.

MEASURING CHILD HEALTH STATUS

Children are generally healthy, making it difficult to find measures that are sensitive to changes in a child's health. A great deal of attention has been paid to identifying better child health status measures, including the Functional Outcomes Project of the American Academy of Pediatrics (2005) and several recent comprehensive reviews of what is known about measuring child health (Drotar 2004; Olson, Lara, and Frintner 2004; Topolski, Edwards, and Patrick 2004). The following outcomes have been used in multiple studies, including several recent national surveys of child health status.

Perceived Health Status (sometimes called “self-rated health,” “respondent-assessed health,” or “general health status”) is the most common measure of health, both for adults and children, and is used in studies of health worldwide. Generally the question is a simple one, some version of the following: “Compared to other people your age, would you say your health is…?” with several response categories, ranging from Excellent to Poor. In the case of child health status, parents generally respond for a child. A comprehensive review of 27 studies of older adults from around the world found that poor perceived health status was highly correlated with mortality (Idler and Benyamini 1997), and another study found the measure to be comparable with other longer instruments in predicting mortality and service use (DeSalvo et al. 2005). RAND Health Insurance Experiment researchers also found the measure to be generally as reliable as other more complex health status measures, and correlated with them (Davies and Ware 1981). While children were not included in any of these studies, it seems reasonable to assume that parents are aware of their children's health, and that this is also a good measure for children. One study found high agreement between parents and medical records on occurrence of certain conditions (accidents, asthma, and bronchitis) but less on others (otitis) (Pless and Pless 1995). On the other hand, of particular relevance to the study results presented below, three separate studies have shown that Hispanics are more likely to rate their health as fair or poor than non-Hispanics and this tendency is greater for those who are less acculturated to the United States (Angel and Guarnaccia 1989; Osmond et al. 1996; Shetterly et al. 1996). Consequently it is important to control for any “acculturation bias” that leads to reporting differences among the groups being compared.

Measuring functional limitations,chronic conditions, and disability has received much attention in recent years, driven in part by concern that the most vulnerable children might not receive sufficient health care under managed care programs. In contrast to the relative comparability of questions used across surveys to measure perceived health status, there is a wide range of instruments to measure functional limitations, chronic conditions, and disability (McPherson et al. 1998; Westbrook, Silver, and Stein 1998; Stein, Silver, and Bauman 2001; Msall et al. 2003; Varni et al. 2003; Wells and Hogan 2003). From the relatively simple question “Is your child limited by a health condition in any activities that most children of the same age can do?” (called “activity limitations”) has evolved a variety of more complex instruments, with batteries of questions that address the functional areas where the child is limited and the degree of limitation (e.g., physical activity, school work). The Children with Special Health Care Needs (CSHCN) screener, used to identify children for a recent nationwide study of CSHCN (Van Dyck et al. 2002), also incorporates whether children heavily use certain types of health services. This approach evolved from the recognition that receiving health services (e.g., asthma care and medication) might alleviate symptoms, but the child might still require continuous care. However, the incorporation of health service use into the definition of CSHCN makes this measure less useful for studying the impact of health insurance, as children are less likely to be identified as CSHCN if they do not have insurance and ready access to care.

An objective measure that may be sensitive to improvements in health care for both short-term (acute) conditions and chronic conditions is the number of schools days missed because of health. However, this measure could be subject to substantial recall bias when parents are asked to recall this over a long period of time. This is a particularly important measure to study, because policy makers often discuss improved school performance as a reason to improve health insurance coverage.

Finally, parents may be asked simply whether their child has selected conditions, such as asthma. The problem with individual conditions is that most do not occur with a high frequency, and—as with the CSHCN screener discussed above—are not good outcomes for studies of the impact of insurance, as they will be underreported in groups that have poor access to care.

METHODS

We studied children who were enrolled in the Healthy Kids program, a health plan operated by the Santa Clara Family Health Plan.1 While the Healthy Kids program covers some higher income children, this analysis focuses on children whose household income is below 250 percent of the federal poverty level. These children account for more than 85 percent of Healthy Kids enrollees in Santa Clara County, and are distinct from children in the 250–300 percent of poverty category, because their program eligibility for Healthy Kids is based on their undocumented immigration status.

Sample Design

The sample for each group was drawn randomly over a 9-month period, August 2003–April 2004, from an enrollment file provided by the health plan. Two strata were formed. The first stratum included children enrolled in Healthy Kids in the sampled month but not in the prior two months. The second stratum included children that enrolled in Healthy Kids 1 year earlier and successfully completed the 1-year renewal process. Within these two strata, a two-stage selection process sampled first families with proportion to size (i.e., number of children), and then one child randomly within each sampled family.2 Sample weights restored the distribution of the sample to that of the study population in each month.

The total eligible sample included 1,389 children, of which 703 were children who had been enrolled for over a year (the study group) and 686 were newly enrolled children (the comparison group). At the time of the interview, the study group had been enrolled for an average of 14.8 months (range 14.2–17.2) and the comparison group had been enrolled for an average of 1.6 months (range 0.9–4.2). From this sample, we completed 1,235 interviews with parents (or guardians), 626 for the study group and 609 for the comparison group. This corresponds to a survey response rate of 89 percent, overall and within both strata.3 This high response rate was due to the use of a $35 incentive for families who completed the interview and high quality contact data provided by the health plan.

Survey

Survey questions were drawn from the National State Children's Health Insurance Program survey (Kenny et al. 2005), the National Health Interview Survey (http://www.cdc.gov/nchs/nhis.htm), and the California Health Interview Survey (http://www.chis.ucla.edu/). Questions were modified to address the expected low literacy of many families served by Healthy Kids.

The survey was translated into Spanish using a professional translator. Researchers, survey methodologists, and community activists in Santa Clara County who work directly with Healthy Kids families reviewed the questionnaire. As a final step in the review process, the Spanish and English versions were pretested, and respondents provided feedback on question wording and clarity.

Administration of the survey took place by telephone using bilingual interviewers, each of whom completed a 2-day training session. At the start of each interview, the respondent was told the purpose of the study, the incentive payment, and the length of the questionnaire and asked whether they would be willing to participate. This oral consent procedure was reviewed and approved by an Institutional Review Board. The interview took 26 minutes on average to complete, and over 85 percent of interviews were conducted in Spanish.

The following questions examined health status:

• Perceived health status: “In general, would you say (CHILD)'s health is excellent, very good, good, fair, or poor?”

• Activity limitations: “Does (he/she) currently have any physical, behavioral, or mental conditions that limit or prevent (his/her) ability to do childhood activities usual for (his/her) age?”

• School days missed due to health: “How many days of school did (he/she) miss because (he/she) was sick during the last four weeks of school?” Responses were none, 1–2, 3–4, 5–10, or 10+.

One child characteristic deserves special attention, a measure of baseline health status. Baseline health status was measured using the following question: “I am going to read you some reasons why you and your family might have decided to enroll (CHILD) in Healthy Kids. Please tell me if any of these reasons are true.” If the parent responded that the child was enrolled because: (1) he/she was sick or injured and needed medical care; (2) he/she needed a prescription medication that the family could not afford; (3) he/she needed to visit a doctor; or, more generally, (4) he/she had a medical condition, then the child was considered to have enrolled because of a medical reason.

Statistical Approach

We used a quasi-experimental design that compared the outcome measures of the two sampled groups: established enrolled children, who had been continuously enrolled in Healthy Kids for about a year (the study group), and newly enrolled children who had only recently enrolled for the first time (the comparison group).

We used multiple weighted linear regression to estimate the effect of the Healthy Kids program. A dummy variable identified each child as being in the study group (in contrast to being in the comparison group). The estimated coefficient of this variable measured the effect of the program on the given health status outcome. When the coefficient was negative and statistically significant, it meant that the children enrolled in Healthy Kids for over a year were significantly less likely to be in poor health. The regression model included controls for characteristics of the children, as well as their zip code of residence (with the possibility that community characteristics—such as environment or crime—affect health) and their month of program enrollment (in order to control for seasonal patterns in health). A similar statistical approach has been used in other recent studies examining the effect of children's health insurance on access and use of services, although those studies did not examine health status impacts (Lave et al. 1998; Szilagyi et al. 2000; Kenny et al. 2005).

This design has several benefits over alternatives. First, because the design requires only one wave of data collection, it avoids sample attrition, which was very high in the previous studies of health status with longitudinal designs described above. Also, the study and comparison groups are similar in most of the important ways that could affect health, including where they live and their parents' choice to enroll them in health insurance.

Sensitivity Analysis

While this study improves on other research because it uses a well-matched comparison group, it is subject to the bias inherent in quasi-experimental designs, as we cannot measure all the ways that the study and comparison groups differ. We investigated two of these sources of bias with sensitivity analyses.

The first, and potentially the most serious, bias is related to the concept of “regression to the mean.” It is possible that children are in worse health right around the time that they enroll in health insurance, as their parents may be more motivated to enroll them while they are sick. Many of these illnesses might have gone away over time, even if the child did not enroll.

To explore the extent of any bias due to regression of the mean, we identified those children who enrolled in Healthy Kids because of a health care need and those who did not (according to their parents). To investigate whether this is a source of serious bias for this study, we performed separate regression analyses for those two groups.

The other source of potential bias for which we did sensitivity analyses relates to subtle differences in acculturation not controlled for in the regressions. To investigate this potential problem, we re-ran all of the regressions including only children who had lived in Santa Clara County for 3 or more years.

FINDINGS

Study and Comparison Groups

Table 2 shows differences between the study and comparison groups according to some of the factors that might affect the child's health status. While the children were not randomly assigned to the two groups, they are similar with a few exceptions. The most notable difference between the two groups is in the length of time the children have resided in Santa Clara County (an artifact of the design). Almost none of the study group had recently arrived in the county, while about a third of the comparison group has been in the county for less than a year. Other significant differences between the two groups were their age (the study group was slightly older), their ethnicity (the study group was slightly less likely to be Hispanic/Spanish speaking), and their income (the study group had slightly higher income).

Table 2.

Demographic Characteristics Healthy Kids Enrollees, Santa Clara County, 2004

	Comparison Group (Recently Insured)		Study Group (Insured 1 Year)
	N	% (Weighted)	N	% (Weighted)
Age (years)*
0–5	167	26.9	130	21.1
6–12	260	42.5	279	44.2
13–18	182	30.6	217	34.7
Gender
M	342	56.9	328	52.0
F	267	43.1	298	48.0
Income (% of poverty)*
< 100	363	59.2	327	52.7
100–149	148	24.3	159	24.5
150–200	61	10.4	91	15.0
200–249	37	6.1	49	7.8
Length of time in Santa Clara county**
< 1 year	205	34.0	7	1.1
1	51	9.2	112	17.9
2	52	8.4	109	17.5
3	45	7.3	79	12.2
4+	252	40.7	318	51.1
Number of parents at home
One	145	23.2	155	24.7
Two	458	76.8	468	75.3
Highest education of either parent
< 5	57	8.9	74	11.8
6–11	297	48.8	289	44.8
12+	117	19.5	136	22.2
Some college	138	22.8	124	20.8
Missing	0	0.0	3	0.5
Ethnicity/language spoken at home**
Hispanic, Spanish	471	76.6	454	70.6
Hispanic, English	81	13.8	86	13.3
Non-Hispanic, English	22	3.7	43	7.9
Non-Hispanic, non-English	34	5.9	42	8.2
Household structure, employment
One parent, does not work	42	6.6	34	5.4
One parent, works	103	16.7	121	19.3
Two parents, neither works	35	5.6	18	2.9
Two parents, one works	327	54.7	338	53.8
Two parents, both work	96	16.5	112	18.6
Number of children at home
One	127	19.9	132	21.0
Two	188	31.3	190	31.0
Three or more	291	48.8	297	48.0
Parent prefers home remedies
Yes	273	46.7	303	50.2
No	314	53.3	303	49.8
Enrolled for medical reasons
Yes	324	53.8	312	50.4
No	285	46.2	312	49.6
Total sample size	609	100.0	626	100.0

Notes: Subtotals do not always add to the total sample size, due to missing values

Study and comparison groups are significantly different according to a χ² test

^*p < 0.05

^**p < 0.01

On the other hand, there are no differences between the groups of children in their family structures or parental education and employment. Of particular importance for this analysis, parents of about the same percentage of children in both groups (about half) reported that they enrolled their children in Healthy Kids for a medical reason.

Previous research has shown an additional important difference between the two groups, their access to and use of health care during 6 months before the survey. Trenholm et al. (2005) found that, for example, use of well-child care and other ambulatory care was about twice as common among those who had been enrolled for a year, than among new enrollees.

Descriptive Health Status Outcomes

Table 3 shows descriptive differences in the health status of 609 recent and 626 established enrollees. Despite many similarities between the two groups in demographic characteristics and reported baseline medical needs, the study and comparison groups are significantly different in the health status reported by their parents at the time of the survey, according to two health status measures, perceived health status and functional limitations. For example, 15.9 percent of the study group's parents reported their child to be in fair/poor health, in contrast to 28.5 percent of the comparison group (in both cases much higher then the general population of U.S. children or U.S. poor children). Children in the comparison group were also more often reported to have activity limitations. However, the school days missed distributions for the two groups are not significantly different.

Table 3.

Health Status of Healthy Kids Enrollees, Santa Clara County, 2004

	Comparison Group (Recently Insured)		Study Group (Insured 1 Year)
	N	% (Weighted)	N	% (Weighted)
Child's perceived health**
Excellent/very good	215	35.8	282	44.8
Good	217	35.8	239	39.3
Fair/poor	176	28.5	103	15.9
Child has activity limitation**
Yes	49	8.4	29	4.5
No	558	91.6	592	95.5
School days missed last month (ages 5–18)
None	311	66.6	364	68.2
1–2	103	21.5	125	23.1
3–4	39	8.7	35	6.9
5–10	11	2.5	9	1.6
10+	4	0.8	1	0.2
Total sample size	609	100.0	626	100.0

Notes: Subtotals do not always add to the total sample size, due to missing values and missing data for school days for the youngest children

Study and comparison groups are significantly different according to a χ² test

^**p < 0.01

Multivariate Analysis

To examine these differences further,Table 4 shows the results of regression analyses predicting the three health status outcomes after controlling for child characteristics. The table presents the coefficient for a dummy variable indicating that the child was in the study group (with the comparison group as the reference group). This coefficient is interpreted as an estimate of the effect of the Healthy Kids program on poor health status. As shown, the impact estimate is negative and significant for two of the health status measures. In other words, Healthy Kids enrollment for 1 year was associated with a reduced likelihood that a parent reported their child to be in fair/poor health or to have activity limitations. There was no significant effect on missed school days, paralleling the descriptive results in Table 3. Still, although not statistically significant, the sign for this coefficient is negative, consistent with the hypothesis of improved health. (Note that missed school days are measured only for children who are 5 years of age or above, reducing the sample size for the analysis and making it more difficult—along with the small percentage of children who miss more than 3 days—to detect significant effects.)

Table 4.

Ordinary Least Squares Regressions Predicting Health Status Outcomes, Santa Clara County, 2004

	Effect of Being Continuously Enrolled in Healthy Kids for a Year
Outcome	Coefficient	Standard Error	t	p-Value	N
Child's perceived health status is fair/poor
All enrollees	−0.130	0.026	−4.920	.000	1190
Child has an activity limitation
All enrollees	−0.057	0.018	−3.220	.001	1189
Child missed more than three school days last month due to health (ages 5–18)
All enrollees	−0.024	0.022	−1.120	.262	965

Notes: Regressions control for differences in: age, gender, income, length of time in Santa Clara County, number of parents, highest education of either parent, ethnicity/language, household structure/employment, number of children at home, parent preference for home remedies, zip code of residence, and month of enrollment in Healthy Kids

Bold p-values indicate coefficients that are significantly different from zero

Sensitivity Analysis

The first sensitivity analysis investigated regression to the mean, by re-analyzing the data separately for children who were enrolled by their parents for a medical reason and those who were not. These results are shown in Table 5.

Table 5.

Sensitivity Analysis: Ordinary Least Squares Regressions Predicting Health Status Outcomes, Santa Clara County, 2004

	Effect of Being Continuously Enrolled in Healthy Kids for a Year
Outcome	Coefficient	Standard Error	t	p-Value	N
Child's perceived health status is fair/poor
Enrolled due to medical problem	−0.187	0.042	−4.510	.000	609
Not Enrolled due to medical problem	−0.065	0.032	−2.040	.041	581
Child has an activity limitation
Enrolled due to medical problem	−0.101	0.028	−3.540	.000	610
Not enrolled due to medical problem	−0.007	0.019	−0.390	.697	579
Child missed more than three school days last month due to health (ages 5–18)
Enrolled due to medical problem	−0.006	0.031	−0.180	.857	508
Not enrolled due to medical problem	−0.058	0.029	−2.000	.045	457

Notes: Regressions control for differences in: age, gender, income, length of time in Santa Clara County, number of parents, highest education of either parent, ethnicity/language, household structure/employment, number of children at home, parent preference for home remedies, zip code of residence, and month of enrollment in Healthy Kids

Bold p-values indicate coefficients that are significantly different from zero

Findings regarding perceived health status parallel those for all enrollees, although the effects are more pronounced for children who were enrolled for medical reasons (where regression to the mean might be a problem). However, findings for the other two measures (activity limitations and missed school days) do not directly parallel those for the full sample. While the signs for the coefficients remain negative for both groups and all health status measures, the significance levels vary according to whether or not the child was enrolled for medical reasons.

The study group (children enrolled for at least a year) is significantly less likely to have activity limitations when their parent report enrolling them for medical reasons. Table 6 shows some of the reasons parents reported when asked the reason why their child's activity was limited. While it seems surprising that the types of serious conditions that may lead to activity limitations could be affected within 1 year, these qualitative data collected as part of the survey show problems such as “Headaches” and “Digestive Disorder” that could be affected by health care.

Table 6.

Reasons for Activity Limitations Reported by Parents

Recent Enrollees	Established Enrollees
Digestive disorder	Allergies
Does not eat well	Arthritis
Fatigued, also gets depressed	Brain damage
Feet hurt	Cannot walk
Feet are not straight	Deformed hand
Fever and nose bleeds	Down syndrome
Headaches	Fatigue
His leg	Flat feet
Knee problems and obesity	Heart problems
Language problem	Kidney problems
Learning disability	Obesity
Obesity	Scoliosis
Orthopedic problem
Previous surgery on hips
Problem with breast
Stomach pain
Very slow
Very thin

The second sensitivity analysis controlled for acculturation bias by restricting the analysis to children whose families had resided in Santa Clara County for 3 or more years. The findings were almost identical, with very similar significance levels for the statistical tests (data not shown).

DISCUSSION

This study adds to a small but growing body of evidence concerning the impact of health insurance on the health of children. Before closing it is important to reiterate some of the limitations of this research.

First, all quasi-experimental designs such as the one used here are subject to potential bias because the study and comparison groups differ in unmeasured ways. For example, the comparison group includes children who will drop out in the first year and the study group does not. This bias is minimized, however, as a large percentage of children re-enroll each year (about 80 percent). In addition, “acculturation bias” in reported health status is possible because the children have been in the United States for different amounts of time. We controlled for this in the regression analyses and sensitivity analysis.

Of greatest importance for this study, the two groups of children who are being compared may have different underlying health status (“health endowment”). We controlled for this using a baseline health status measure. However, the measure was reported by parents and not validated by a physician's assessment, and it could be subject to recall bias. Parents whose children had serious health problems when they enrolled might remember that they enrolled the child because of a health problem, while minor health problems that did not persist might be forgotten. To the extent that there are recall differences, parents of the study group (children enrolled for a year) may underreport enrolling their child for a medical reason, as they have a longer time frame to remember the reason. Consequently, we believe that the two groups are either comparable or that the study group was in somewhat worse health when they enrolled, weakening the association between health status and health insurance and providing “conservative” findings regarding the effect of health insurance on health status in this population.

Keeping the caveats outlined above in mind, we cautiously conclude that providing health insurance to very disadvantaged children has improved the health status of undocumented children in Santa Clara County. The direction of findings was consistent across all three health measures, although only the relationships and with perceived health status and activity limitations were statistically significant for the full population of children. According to the sensitivity analyses, differences in activity limitations were significant only among children who enrolled for medical reasons. Conversely, children were significantly less likely to have 3 or more days of school lost due to health only when they did not enroll for medical reasons.

These somewhat puzzling findings from the sensitivity analysis showing differences in the effect of health insurance according to whether a child enrolled for medical reasons deserve some special discussion. The apparent finding of a effect on activity limitations may partially be explained by regression to the mean, as the relationship was only statistically significant among those who enrolled for a medical reason. However, because many of the conditions parents gave as reasons for activity limitations are amenable to treatment (Table 6), some of the 10 percentage point reduction in activity limitations for children enrolled for medical reasons is also likely due to new health services the children received.

The finding of an impact on days missed from school among children who did not enroll for medical reasons is also very important, because of its potential relationship to school performance. In addition, of all the findings from the study, this is potentially the most robust because regression to the mean is less of a problem. Finally, missed school days may be the most reliable measure of the three health status measures examined, because of its shorter recall period (1 month).

We do not know whether these findings for undocumented children in Santa Clara County, California would be replicated in other settings or for different populations. For example, other places may not have a health care delivery system that is able to provide ready access to health services once children become insured. Alternatively, uninsured children in other places may have had better health care before they became insured. The undocumented children in this study were not born in the United States, and many had inadequate access to health services in their early lives. Because this population was particularly disadvantaged before enrolling, it is possible that the improvements in health shown here would not be replicated in other groups of uninsured children who had better access to care before enrolling.

Since the time that Santa Clara County implemented Healthy Kids in early 2001, 17 other counties have adopted similar programs. Together, these programs currently insure more than 80,000 children across the state. This figure is expected to grow significantly, as another 12 counties are developing their own Healthy Kids programs. A recent poll of likely voters in California suggests that this momentum will continue, as 78 percent support efforts to provide health insurance coverage to every child in the state.

We hope that these study findings provide a stimulus for further research in this very important area. New studies should build on improvements in child health status measurement and use improved designs, in order to avoid some of the biases that have plagued previous research. For example, some of the California counties that have Children's Health Initiatives, including Santa Clara County, have limited enrollment in Healthy Kids due to a lack of available financing. As these programs have waiting lists for obtaining health insurance coverage, it would be possible to use either random assignment or other approaches that use children on the waiting list as a comparison group.

In conclusion, the current study adds to limited evidence that continued investments in health insurance for children for the most vulnerable children will likely improve their health. Future research is needed to confirm these findings for other groups and in other settings.