Effects of Medicaid Accountable Care Organizations on children's access to and utilization of health services

Abstract

Objective

To evaluate the effects of Medicaid Accountable Care Organizations (ACOs) on children's access to and utilization of health services.

Study Setting and Design

This study employs difference‐in‐differences models comparing ACO and non‐ACO states from 2018 through 2021. Access measures are indicators for preventive and sick care sources, unmet healthcare needs, and having a personal doctor or nurse. Utilization measures are preventive and dental care, mental healthcare, specialist visits, emergency department visits, and hospital admissions.

Data Sources and Analytic Sample

Secondary, de‐identified data come from the 2016–2021 National Survey of Children's Health. The sample includes children with public insurance and ranges between 21,452 and 37,177 depending on the outcome.

Principal Findings

Medicaid ACO implementation was associated with an increase in children's likelihood of having a personal doctor or nurse by about 4 percentage‐points concentrated among states that implemented ACOs in 2018. Medicaid ACOs were also associated with an increase in specialist care use and decline in emergency visits by about 5 percentage‐points (the latter being concentrated among states that implemented ACOs in 2020). There were no discernable or robust associations with other pediatric outcomes.

Conclusions

There is mixed evidence on the associations of Medicaid ACOs with pediatric access and utilization outcomes. Examining effects over longer periods post‐ACO implementation is important.

What is known on this topic

• Prior evidence points to improvement in some children's healthcare access and utilization measures with Medicaid Accountable Care Organizations (ACOs).

• Previous studies were descriptive, employed comparisons within ACO states, focused on one or included few states with ACOs, and covered short post‐ACO periods.

What this study adds

• Findings suggest an increase in having a personal doctor/nurse and in specialist care use and decline in emergency department use, but no discernable or robust associations with other outcomes.

• There is some evidence of heterogeneity by ACO implementation year, highlighting the importance of future work to understand differences in ACO designs and implementation between states.

• Optimizing Medicaid ACO performance and outcomes might require customized, evaluative approaches and integrative care models to ameliorate children's outcomes under state‐specific policies.

1. INTRODUCTION

Accountable Care Organizations (ACOs) are increasingly implemented within Medicaid programs nationwide. Over the past decade, 14 states have developed ACOs in their Medicaid programs. Although these Medicaid ACOs vary in design and scope, ¹ , ² , ³ they share basic principles of providing higher‐quality and coordinated care to their patients while keeping expenses below benchmarks and receiving financial incentives (shared savings) for doing so.

Most Medicaid ACOs have pay‐for‐performance quality measures such as well‐child and adolescent well‐care visits, weight assessment and counseling, screening for depression and documentation of follow‐up plans, follow‐up care for children prescribed Attention‐Deficit/Hyperactivity Disorder (ADHD) medication, getting an appointment for routine care/illness/injury/condition as soon as the child needed, and having dental sealants on permanent molars for children. ⁴ However, the specific indicators might vary between ACOs (further discussed below). Childhood is a fundamental period for development and well‐being, ⁵ , ⁶ and the healthcare system plays an important role by supporting children holistically and supplying crucial interventions like immunizations and prompt care during illness. ⁷ , ⁸ , ⁹ Thus, Medicaid ACOs may have a potential to improve access and health outcomes of children covered by Medicaid given their goals for improving care coordination and quality, including for specific child‐related quality performance measures.

The extant evidence on Medicaid ACOs suggests improvements in some access, utilization, quality and cost measures related to maternal and child healthcare. ¹⁰ For example, there is some evidence of an association between Medicaid ACO implementation and increases in timely prenatal care initiation, ¹¹ annual developmental screening, and preventive care use among infants. ¹² Research has also found an increase in preventive care and well health visits among children and adolescents ¹³ , ¹⁴ , ¹⁵ and in use of pharmaceuticals among children associated with Medicaid ACOs. ¹⁶ Other reported care improvements are declines in cesarean section rates, ¹⁷ neonatal intensive care unit (NICU) days, ¹³ and inpatient days among children. ¹⁶ At the same time, there have been reports of increased Emergency Department (ED) visits, ¹⁶ decrease in use of ADHD medications, ¹⁵ and worse performance on children's diabetes short‐term admission and hematoma rates. ¹³

Given that Medicaid ACO implementation is relatively recent, previous studies of their effects on children have largely focused on one ACO (e.g., Partners for Kids in Ohio, ¹³ , ¹⁵ , ¹⁸ , ¹⁹ or Children's Hospitals and Clinics of Minnesota in Minnesota) ¹⁶ , ²⁰ or included few states with ACOs. ¹⁴ , ²¹ Studies have also been descriptive, ¹³ , ¹⁶ , ¹⁹ , ²⁰ employed comparisons within ACO states, ¹⁴ and/or included relatively short post‐ACO periods. ¹⁴ , ¹⁵ , ¹⁸ , ²¹ Most studies only included data through 2016, with one descriptive study including data through 2019. ¹⁹

This study examines the effects of Medicaid ACOs on healthcare access and utilization of children in Medicaid. The study employs nationally representative data over 2016–2021 and includes most states with Medicaid ACOs as well as states without Medicaid ACOs. The study also evaluates a range of children's healthcare access and utilization outcomes.

2. METHODS

2.1. Data source and sample

Data come from the 2016–2021 National Survey of Children's Health (NSCH). ²² The NSCH is an annual cross‐sectional survey that collects detailed data on health, well‐being, and healthcare access and utilization for a nationally representative sample of children. Before 2016, the survey was conducted every 4 years but became annual in 2016. Also in 2016, the sampling design and several questions were changed. Therefore, this study focuses on the annual survey data for consistency. Given the secondary analysis of de‐identified, publicly available data, informed consent or Institutional Review Board review were not required for this study.

Because the study focuses on Medicaid ACOs, the analytical sample is restricted to children aged 0 to 17 years with public health insurance; in the NSCH, this group includes Medicaid, Medical Assistance, or any kind of government assistance plan for those with low incomes or a disability, although we expect the majority to be children in Medicaid. ²³ Children who have private in addition to public insurance are excluded from this sample. For a placebo check, an alternative sample of children covered by private health insurance only is used.

Since the NSCH data used for the study begins in 2016, the treatment group includes states that started Medicaid ACOs or expanded existing pilot programs in 2018 or later to have at least two data years before ACO implementation. Therefore, states that implemented Medicaid ACOs before 2018 are excluded. Idaho is also excluded because it added an accountable care program in 2020 but the program did not go live until 2022. We only include ACO states where the program is statewide with full or near‐full attribution of Medicaid beneficiaries. States that never implemented Medicaid ACOs are included as control states. Appendix Table S1 lists states included in this study that implemented ACOs, years of implementation, and a description of the programs. Appendix Table S2 lists included states by treatment and control groups and excluded states. The final sample includes 42 states and Washington, DC. Of those, ACOs were implemented in 2018 in Colorado, Massachusetts, Minnesota, and Rhode Island; in 2020 in New Jersey and Oregon; and in 2021 in Delaware.

2.2. Outcomes

Access measures include four binary indicators for having usual source for preventive care, usual source for sick care, any unmet healthcare needs, and having a personal doctor or nurse. Utilization measures include six binary indicators for using preventive medical care, preventive dental care, mental healthcare, specialist care, any ED visits, and any hospital admissions, all measured over the past 12 months. Hospital admissions were first collected in 2018, so were only evaluated for states adopting ACOs in 2020 and 2021 to have at least two pre‐ACO years. The analysis for preventive dental care use is limited to children aged 6 years and older given the high missing rate for younger children (~5% for age 5 increasing to ~20% for age 1; question was also not asked until recently for age 0). Similarly, the analysis for mental healthcare use is limited to children aged 3 years and older (~72% missing for younger children).

2.3. Statistical analysis

Because of the staggered ACO adoption over the study period, we begin with a two‐way fixed effects (TWFE) specification. We consider this a reasonable start considering the short post‐ACO period of the study (2–4 years depending on ACO start year) with treatment effects less likely to meaningfully change over time to bias the aggregate estimate, although we evaluate this issue in subsequent models. This regression is specified as follows:

$$Y_{\mathit{ist}}={\beta }_0+\left({{\beta }_1}^{*}{\text{MedicaidACO}}_{\mathit{st}}\right)+{\mathit{X}}_{\mathit{ist}}+{\mathit{Z}}_{\mathit{st}}+{\mathit{\text{𝜏}}}_t+{\mathit{\sigma }}_{\mathit{s}}+{\epsilon }_{\mathit{ist}}.$$ (1)

Y _ist is one of the outcomes for child i in state s in year t. MedicaidACO _st is a time‐varying, state‐level binary (0/1) indicator switched to 1 when the state had an active ACO program. β ₁ is the estimate of the Medicaid ACO effect. The model includes year (𝜏) fixed effects to capture outcome trends shared between states, and state (σ) fixed effects to account for time‐invariant differences between states. The model adjusts in X for conceptually relevant sociodemographic variables including child's age, sex, race/ethnicity, primary caregiver(s)’ highest education level, whether English is the primary household language, whether household is two‐parent headed, number of children, and average family income as percent of the Federal Poverty Level (FPL). The model also adjusts in Z for the following state‐level time‐varying covariates: whether state has Earned Income Tax Credit (EITC) program, ²⁴ the EITC rate for states that refund the credit (0 otherwise), ²⁴ the state effective minimum wage, ²⁴ unemployment rate, ²⁵ Medicaid MCO enrollment rates, ²⁶ and duration of Medicaid expansion under the Affordable Care Act (ACA). ²⁷ These variables capture the overall state economic and financial conditions, which are conceptually relevant for Medicaid budgets, cuts, and policy change in addition to capturing population socioeconomic differences in healthcare access and utilization over time and between states. The minimum wage and EITC have also been associated with children's health, ²⁸ , ²⁹ and the ACA Medicaid expansion has been associated with increased Medicaid coverage among children. ³⁰ The model is estimated using linear least squares weighted by the NSCH sampling weights to reflect the demographic composition of noninstitutionalized children in each state. Standard errors are clustered at the state‐level. Statistical significance is considered at 5% type‐1 error rate. Analyses are done using STATA versions 15.1 and 17.0. ³¹

TWFE uses earlier treated states as controls for later treated states, which could bias the average treatment effect estimate if effects change over time across treatment states. ³² To address this concern, we also employ a staggered difference‐in‐differences model developed by Callaway and Sant'Anna (C&S, 2021) that compares each cohort of treatment states (based on ACO adoption year in this case) to states that never adopted ACOs, then aggregates the cohort‐specific effect estimates into one treatment effect estimate. ³³ In addition to the aggregate effect estimate combining ACO cohorts and years, we estimate a C&S event‐study with year‐by‐year post‐ and pre‐ACO estimates relative to the reference year before ACO implementation. The model is best suited for time‐invariant covariates, so we include the individual‐level sociodemographic covariates but exclude the state‐level time varying covariates. We employ the doubly robust estimator and estimate standard errors clustered at the state level using 1000 wild bootstrap replications.

Finally, to evaluate potential effect heterogeneity by implementation year, we estimate a classical difference‐in‐differences model based on ACO start year separately for the four states implementing ACOs in 2018 and for the two states implementing ACOs in 2020, compared to non‐ACO states. Because the NSCH supports state‐specific estimates combining 2 or 3 years of data, and Delaware has only one year of ACO implementation in the dataset (2021), we do not estimate this model specifically for Delaware. The classical difference‐in‐differences model based on ACO implementation year is specified as follows:

$$Y_{\mathit{ist}}={\beta }_0+\left({{\beta }_1}^{*}{{\text{MedicaidACO}}_s}^{*}\text{POST}\right)+{\mathit{X}}_{\mathit{ist}}+{\mathit{Z}}_{\mathit{st}}+{\mathit{\text{𝜏}}}_t+{\mathit{\sigma }}_s+{\epsilon }_{\mathit{ist}}.$$ (2)

MedicaidACO _s is a (0/1) indicator for states that adopted Medicaid ACOs (treatment) in a given year (2018 or 2020) versus those that never adopted ACOs (control). For each ACO state cohort, the other ACO state cohort is excluded from the regression (Delaware is also excluded from these models). POST is a (0/1) indicator for post‐ACO period (2018–2021 or 2020–2021). The difference‐in‐differences parameter is β₁, which represents the outcome change over time in ACO versus non‐ACO states.

We also estimate an event‐study specification of Equation (2) to examine pre‐trends and year‐by‐year post‐ACO effects for each of the 2018 and 2020 ACO‐adopting state cohorts as follows:

$$Y_{\mathit{ist}}={\beta }_0+\mathrm{\sum }\left({{\beta }_t}^{*}{{\text{MedicaidACO}}_s}^{*}{\text{Year}}_t\right)+{\mathit{X}}_{\mathit{ist}}+{\mathit{Z}}_{\mathit{st}}+{\mathit{\text{𝜏}}}_t+{\mathit{\sigma }}_s+{\epsilon }_{\mathit{ist}}.$$ (3)

Year includes binary (0/1) indicators for each year from 2016 through 2021. For states implementing ACOs in 2018, the reference year is 2017, and post‐ACO effect parameters are β ₂₀₁₈₋ β ₂₀₂₁, representing changes in outcomes over time from 2018 through 2021 relative to 2017 for ACO versus non‐ACO states; β ₂₀₁₆ represents the difference in pre‐trends (2016–2017) between ACO and non‐ACO states. For states implementing ACOs in 2020, the reference year is 2019, and post‐ACO effect parameters are β ₂₀₂₀ and β ₂₀₂₁ while β ₂₀₁₆–β ₂₀₁₈ capture pre‐trend differences. Equations (2) and (3) are estimated with weighted least squares similar to Equation (1).

3. RESULTS

The overall sample includes 38,082 observations over the study period including 1806 and 4002 observations in ACO states in the pre‐ and post‐periods, respectively. Depending on the outcome and model, the sample size ranges between 21,452 and 37,177 observations. Table 1 shows descriptive sample characteristics. Mean age was 8.3 years; 51.4% were males; 40.0% were Hispanic; English was the primary household language for 73.7%; 57.0% were in two‐parent‐headed families; 53.7% had high school or less educated adults in households; and mean family income was 138.9% of the FPL.

TABLE 1.

Descriptive characteristics of the study sample (weighted).

Outcomes of interest	Frequency (%) or Mean (SD a ) b
Access measures
Child has usual source of preventive care	17,154 (89.0%)
Child has usual source of sick care	13,303 (69.8%)
Child had any unmet care needs (foregone healthcare)	734 (3.8%)
Child has a personal doctor or nurse	12,740 (66.1%)
Utilization measures
Child used preventive medical care	14,614 (76.1%)
Child used preventive dental care	13,686 (75.4%)
Child used mental health care	2014 (12.6%)
Child used specialist care	2337 (12.2%)
Child had emergency department visit	5166 (26.7%)
Child was admitted to hospital	519 (4.5%)
Covariates
Child's age a	8.3 (5.1)
Child's sex
Male	9974 (51.4%)
Female	9412 (48.6%)
Child's race/ethnicity
Hispanic	7754 (40.0%)
Non‐Hispanic Black	3138 (16.2%)
Non‐Hispanic White	6367 (32.8%)
Non‐Hispanic persons with other or multiple races	2127 (11.0%)
Highest educational level of adults in the household
High school or less	10,410 (53.7%)
Some college or associate degree	5464 (28.2%)
College degree or higher	3512 (18.1%)
Household with English as the primary language	14,292 (73.7%)
Two‐parent‐headed household	11,048 (57.0%)
Number of children in the household
Only child	4707 (24.3%)
Two children	6507 (33.6%)
Three children	4832 (24.9%)
At least four children	3341 (17.2%)
Family income as percent of the Federal Poverty Level a	138.9 (85.8)

^a SD, standard deviation.

^b Descriptive statistics are based on the total sample of children covered by public health insurance and with no missing data on any of the covariates and are weighted using the NSCH sampling weights.

3.1. Access outcomes

Table 2 (row 1) presents the TWFE (Equation 1) estimates of the association between implementing Medicaid ACOs and access measures. There was no association with having a usual source for preventive and sick care and with unmet care needs. There was an increase in the likelihood of having a personal doctor or nurse by 3.7 percentage‐points.

TABLE 2.

Estimated effects of Medicaid Accountable Care Organizations on children's access to health services.

		Usual source for preventive care a	Usual source for sick care a	Any unmet care needs a	Child has personal doctor or nurse a
		(1)	(2)	(3)	(4)
TWFE b	(1)	0.023	0.011	−0.007	0.037*
		(0.018)	(0.017)	(0.010)	(0.015)
Staggered difference‐in‐differences (C&S) c	(2)	0.045	0.022	0.018*	0.047
		(0.018)	(0.029)	(0.009)	(0.023)
Classical difference‐in‐differences, Group 1 d	(3)	0.016	0.012	−0.000	0.044**
		(0.024)	(0.026)	(0.011)	(0.014)
Classical difference‐in‐differences, Group 2 e	(4)	0.035*	0.003	−0.023	0.035
		(0.014)	(0.015)	(0.011)	(0.030)

^a Standard errors clustered at the state level are in parentheses.

^b TWFE—two‐way fixed effects.

^c C&S—Callaway and Sant'Anna.

^d States implementing ACOs in 2018: Colorado; Massachusetts; Minnesota; Rhode Island.

^e States implementing ACOs in 2020: New Jersey; Oregon.

^* p < 0.05;

^** p < 0.01.

The aggregate C&S estimates for the same access outcomes are also in Table 2 (row 2). Estimates for usual source for preventive and sick care and for having a personal doctor or nurse were overall comparable to the TWFE but not statistically significant. In contrast to the TWFE, there was an increase in the likelihood of unmet care needs by 1.8 percentage‐points.

Figure 1 shows the C&S event‐study estimates for the access outcomes. None of the individual pre‐ or post‐year coefficients were statistically significant, and there were overall no apparent systematic pre‐trends. Similarly, there were no discernable differences in effects over time since implementation. However, most event‐study estimates were imprecise with wide confidence intervals and should be cautiously interpreted.

FIGURE 1.

Callaway and Sant'Anna event‐study estimates of effects of Medicaid Accountable Care Organizations on children's access to health services. (A) Panel A exhibits regression coefficients from the main event‐study analysis presenting effects on the child's likelihood of having a usual source of preventive care. (B) Panel B exhibits regression coefficients from the main event‐study analysis presenting effects on the child's likelihood of having a usual source of sick care. (C) Panel C exhibits regression coefficients from the main event‐study analysis presenting effects on the child's likelihood of having any unmet care needs. (D) Panel D exhibits regression coefficients from the main event‐study analysis presenting effects on the child's likelihood of having a personal doctor or nurse. (E) ATT—average treatment effect. (F) The red line marks the reference period.

Table 2 (rows 3 and 4) also show the classical difference‐in‐differences (Equation 2) estimates for access separately for states adopting ACOs in 2018 and 2020. For states implementing ACOs in 2018 (Colorado, Massachusetts, Minnesota, and Rhode Island), there was an increase in the likelihood of having a personal doctor or nurse by 4.4 percentage‐points but no discernable effects on the other access outcomes. For the two states implementing ACOs in 2020 (New Jersey and Oregon), there was an increase in the likelihood of having a usual source for preventive care by 3.5 percentage‐points but no significant effects on the other access outcomes.

Appendix Figures S1 and S2 plot the classical difference‐in‐differences event‐study (Equation 3) estimates by year of ACO implementation. Overall, most pre‐trend estimates were not statistically significant and showed no systematic pre‐trends that would explain the observed associations with ACO implementation. One exception was that for states implementing ACOs in 2020, there was an increase in having a usual source for preventive care from 2018 to 2019 in the same direction as the increase post‐ACO implementation. There was overall no discernable difference in effects over time since implementation.

3.2. Utilization outcomes

Table 3 (row 1) presents the TWFE (Equation 1) estimates of the association between implementing Medicaid ACOs and the health services utilization measures. There were no associations with use of preventive medical, preventive dental, mental health, specialist care, or ED. There was a decrease in the likelihood of hospital admission by 1.8 percentage‐points.

TABLE 3.

Estimated effects of Medicaid Accountable Care Organizations on children's utilization of health services.

		Used preventive medical care a	Used preventive dental care a	Used mental healthcare services a	Used specialist care services a	Emergency Department use a	Hospital admission a
TWFE b	(1)	−0.021	0.027	−0.011	0.002	−0.023	−0.018*
		(0.015)	(0.021)	(0.013)	(0.009)	(0.015)	(0.007)
Staggered difference‐in‐differences (C&S) c , d	(2)	−0.050*, g	−0.013	−0.041	0.048*, h	−0.039	−0.007
		(0.021)	(0.022)	(0.025)	(0.017)	(0.027)	(0.005)
Classical difference‐in‐differences, Group 1 e	(3)	−0.026	0.001	−0.012	0.008	−0.008	N/A i
		(0.025)	(0.025)	(0.020)	(0.013)	(0.021)
Classical difference‐in‐differences, Group 2 f	(4)	−0.024*	0.063*	−0.008	−0.008	−0.051**	−0.022*
		(0.011)	(0.027)	(0.012)	(0.006)	(0.007)	(0.008)

^a Standard errors clustered at the state level are in parentheses.

^b TWFE—two‐way fixed effects.

^c C&S—Callaway and Sant'Anna.

^d Standard errors of the C&S estimates are obtained using wild bootstrap, and inference of statistical significance of these estimates is based on 95% uniform confidence intervals.

^e States implementing ACOs in 2018: Colorado; Massachusetts; Minnesota; Rhode Island.

^f States implementing ACOs in 2020: New Jersey; Oregon.

^g The average of the pre‐trend C&S estimates in the event study is statistically significant and is in the same direction as the post‐trend average (decline in the pre‐period and in the post‐period over time relative to the reference year before ACO implementation). The estimate should be interpreted with caution.

^h The average of the pre‐trend C&S estimates in the event study is statistically significant and is in the opposite direction as the post‐trend average (decline in the pre‐period and increase in the post‐period relative to the reference year before ACO implementation). The estimate should be interpreted with caution.

ⁱ Data on hospital admission was first collected in the NSCH in 2018 so this outcome is not evaluated for group 1.

^* p < 0.05;

^** p < 0.01.

Table 3 (row 2) shows the aggregate C&S estimates for the same utilization outcomes. These estimates showed no associations with preventive dental care, ED use, or hospital admission (the latter estimate was nearly half that of the TWFE estimate). There was a decline in preventive medical care use by 5 percentage‐points and an increase in specialist care use by 4.8 percentage‐points.

Figure 2 shows the C&S event‐study estimates for these outcomes. For preventive medical care use, there was a statistically significant pre‐trend (when averaged across the entire pre‐trend) in the same direction as the aggregate post‐ACO estimate, which might bias this estimate (in direction and magnitude). For specialist care use, there was also a statistically significant decline on average before ACO implementation; this pre‐trend was in the opposite direction to the post‐ACO aggregate estimate, which might bias its magnitude. Lastly, for hospital admission, there was a decline in the second year following ACO implementation but also a potential opposite pre‐trend. However, most event‐study estimates were imprecise with overlapping confidence intervals and should be cautiously interpreted.

FIGURE 2.

Callaway and Sant'Anna event‐study estimates of effects of Medicaid Accountable Care Organizations on children's utilization of health services. (A) Panel A exhibits regression coefficients from the main event‐study analysis presenting effects on the child's likelihood of utilizing preventive medical care. (B) Panel B exhibits regression coefficients from the main event‐study analysis presenting effects on the child's likelihood of utilizing preventive dental care. (C) Panel C exhibits regression coefficients from the main event‐study analysis presenting effects on the child's likelihood of utilizing mental care. (D) Panel D exhibits regression coefficients from the main event‐study analysis presenting effects on the child's likelihood of utilizing specialist care. (E) Panel E exhibits regression coefficients from the main event‐study analysis presenting effects on the child's likelihood of emergency department visit. (F) Panel F exhibits regression coefficients from the main event‐study analysis presenting effects on the child's likelihood of hospital admission. (G) ATT—average treatment effect. (H) The red line marks the reference period.

Table 3 (rows 3 and 4) shows the classical difference‐in‐differences (Equation 2) estimates for the utilization measures by ACO implementation year. There were no associations for states implementing ACOs in 2018 (as previously mentioned, hospitalizations were added to the survey in 2018 so this outcome was not included for those states). For states implementing ACOs in 2020, there was a decrease in preventive medical care use by 2.4 percentage‐points and an increase in preventive dental care use by 6.3 percentage‐points. There was also a decrease in likelihood of ED use and hospital admission by 5.1 and 2.2 percentage‐points, respectively.

Appendix Figures S3 and S4 show the classical difference‐in‐differences event‐study (Equation 3) estimates for utilization outcomes by year of ACO implementation. For states implementing ACOs in 2018 (Appendix Figure S3), there were opposite changes from 2016 to 2017 in preventive medical care, mental healthcare, and specialist care utilization compared to post‐ACO year estimates. For states implementing ACOs in 2020 (Appendix Figure S4), there were some pre‐ACO declines relative to the reference year in preventive medical and dental care and mental health and specialist care. Overall, these event‐study estimates were imprecise with overlapping confidence intervals between pre‐ and post‐estimates and should be cautiously interpreted.

3.3. Placebo checks

Appendix Tables S3 and S4 show the results of the placebo check analyses among children covered only by private health insurance, who are not expected to be directly affected by Medicaid ACOs. Most of the associations observed for publicly insured children were not observed for privately covered children. An exception is the increase in unmet care needs (observed in the C&S model for publicly insured children and the TWFE model for privately insured children). Another is the decline in hospitalizations (observed in the TWFE model for publicly insured children and the C&S model for privately insured children). Most estimates were also noticeably smaller in magnitude and close to the null among privately covered children. In the latter group, there was a decline in having a usual source for sick care in the C&S model, an increase in usual source for preventive care with ACO implementation in 2018, and a decline in preventive dental care use with ACO implementation in 2020 (an opposite association was observed for publicly insured children). Overall, the placebo check analyses do not indicate bias in estimates for having a personal doctor or nurse, specialist use, and ED use for publicly covered children. However, they warrant caution in interpreting the estimates, especially for outcomes showing associations in this placebo analysis.

4. DISCUSSION

This study examines effects of implementing Medicaid ACOs on children's access to and utilization of health services from 2018 through 2021. Overall, we found mixed evidence on the association between state Medicaid ACO implementation and children's healthcare access and utilization indicators. Specifically, we find some evidence for an increase in having a personal doctor or nurse that is concentrated among states that implemented ACOs in 2018. There is suggestive evidence for increase in specialist care use and decline in ED services, with the latter observed for the two states that implemented ACOs in 2020. However, these associations should be cautiously interpreted as event‐study estimates were generally imprecise to clearly support these events. For the other outcomes, there were either no discernable associations or associations that were more likely to be biased by pre‐trends or also observed in placebo checks (where they are unexpected).

Having a personal doctor or nurse is an indicator of care continuity and a key factor in establishing a medical home. ³⁴ Such interpersonal continuity of care is pivotal to most patients, especially those from vulnerable groups, as they value the relationship with their healthcare provider, their provider's knowledge about them, and the ability to communicate their concerns. ³⁵ With time, continuity of care through regular interactions with a consistent provider enhances person‐centered care ³⁶ and trust and confidence in medical decision‐making. ³⁵ , ³⁶ Less primary care continuity is associated with higher risk of ED utilization and hospitalization among children. ³⁷ The suggested increase in likelihood of having a personal doctor or nurse of nearly 4 percentage‐points in this study might seem small in magnitude but is a potentially meaningful change of nearly 5% relative to the baseline outcome rate (70.8%) and considering that this is one of many factors that influence care continuity.

Even though the study is among the first to evaluate Medicaid ACO implementation and children's access and utilization measures using national survey data, its findings are broadly within range of previous studies on individual or few ACO states. Previous studies have reported some improvement in access including in number of well‐child visits, ¹³ , ¹⁴ , ¹⁸ , ¹⁹ adolescent preventive services, ¹⁴ , ¹⁵ and in‐office visits, ¹⁶ although as noted above, we find no discernable or robust effects on most access measures. The suggestive decline in ED visits is consistent with findings of most studies ¹⁴ , ¹⁹ , ²¹ but not all. ¹⁶ Prior studies reported declines in hospital admissions ¹⁴ and in inpatient ¹⁶ and NICU ¹³ days, but our finding on hospital admission decline is not robust.

All ACO states included in this study except Rhode Island have pediatric‐specific quality performance indicators aligning with financial incentives; Appendix Table S5 details the key performance indicators (KPIs) of Medicaid ACOs in these states. There is variation across ACOs in number and domains of KPIs and whether they are mandatory or voluntary for financial incentives (Appendix Table S5). Such variation could be associated with different levels of effort to ameliorate children's care, an important question for future research.

Another potential source of heterogeneity is the presence and relationships with managed care organizations (MCOs). While some goals might seem to be shared between ACOs and MCOs, these are two conceptually different systems overall that address different issues. ACOs are adopted in multiple states with high MCO rates; in our study, average MCO penetration rates were comparable between ACO and non‐ACO states (~68% and ~60%, respectively). Future research that evaluates differences between ACOs and MCOs including potential synergies or overlaps and potential heterogeneity in ACO effects by MCO presence is important.

Conceptually, Medicaid ACOs have the potential to enhance children's health outcomes by effectively aligning quality and financial incentives to foster efficient, well‐coordinated care including for Medicaid, which insures over a third of US children. However, opportunities for improvement may remain untapped, and some potential impacts are still not fully realized. Pediatric ACOs face a myriad of challenges as they strive to tailor care to the distinctive needs of children, a generally healthy population presenting fewer opportunities for immediate cost savings compared to adults. Managing ACOs requires a nuanced approach beyond one‐size‐fits‐all solutions, focusing on long‐term investments in pediatric health. Considering upstream interventions such as tackling social determinants of health that traditional payment models overlook, while leaning into multigenerational care strategies to ensure holistic outcomes for children, might be important opportunities to consider. Additionally, pediatric ACOs grapple with forging robust provider networks, incentivizing providers despite varied payment structures, and developing quality metrics that meaningfully capture pediatric care outcomes. Resistance to quality improvement projects due to resource constraints and unfamiliar methodologies, compounded by data‐sharing challenges and the need for more sophisticated data and IT infrastructure might be additional challenges for successful pediatric ACOs. ³⁸ , ³⁹ , ⁴⁰ , ⁴¹ , ⁴² , ⁴³ Through strategic adaptation of their pediatric ACO programs to address these challenges and seize opportunities, state Medicaid programs have the potential to enhance the delivery and outcomes of pediatric healthcare for their children. This study offers some impetus for policy and organizational considerations on Medicaid ACOs. These programs may benefit from refined, state‐specific designs aligned with the healthcare needs of local Medicaid populations. Continuing to assess the long‐term impact of Medicaid ACOs is also crucial for ensuring ongoing improvements in pediatric care quality and cost‐efficiency, while considering and prioritizing strategies that bolster pediatric care and optimize the sustainability and effectiveness of ACOs.

Healthcare providers within ACOs may also benefit from targeted supports to improve care management and coordination, potentially leading to more favorable care utilization patterns. Education programs for patients and providers could reduce unnecessary ED visits and hospitalizations, while a focus on integrating mental health, specialty, and preventive care within the ACO structure could strengthen care coordination and improve health outcomes.

The study findings should be considered with some caveats. First, there is potential error in some survey outcome responses. However, it is unlikely that any possible inaccuracy is systemically related to Medicaid ACO implementation; thus, effect estimates should not be biased, although this could reduce estimate precision (i.e., increase standard errors). Second, the post‐ACO period is still potentially short for meaningful changes in provider care processes to affect the studied outcomes. Therefore, future work with longer post‐ACO time frame would be useful. Third, various types of health insurance coverage are grouped into a single public insurance category in the NSCH data; nonetheless, most children reported in the public insurance group would have been covered by Medicaid and were therefore potentially affected by Medicaid ACOs. Lastly, most ACO years in the study were during the COVID‐19 pandemic, which disrupted children's access to and use of medical and dental services, ⁴⁴ , ⁴⁵ , ⁴⁶ , ⁴⁷ , ⁴⁸ , ⁴⁹ , ⁵⁰ and in turn, might have modified the Medicaid ACO effects. The pandemic potentially muted the effects of ACOs on children's access to and utilization of health services by diverting healthcare resources and shifting priorities toward the pandemic response. ⁵¹ , ⁵² , ⁵³ , ⁵⁴ , ⁵⁵ Routine and nonurgent services were often delayed or foregone to reduce exposure risk and allocate medical supplies and personnel to COVID‐19 care. ⁵⁶ Additionally, families may have chosen to avoid healthcare settings to minimize their own risk of infection, leading to decreased utilization of services that ACOs would typically manage or improve. Future work reexamining Medicaid ACO effects over longer periods post‐pandemic would be important.

To summarize, we found mixed evidence on effects of implementing Medicaid ACOs on children's access to and utilization of health services. There is some evidence for an increase in having a personal doctor or nurse and in specialist care use, as well as a decline in ED use. There are, however, no discernable effects on other examined outcomes. There is also some evidence for effect heterogeneity by ACO implementation year. Important directions for future research include examining effects over a longer period after ACO implementation and impacts on children's health outcomes. Understanding facilitators and barriers for implementing high‐performing ACOs and sources of effect heterogeneity across ACOs is also relevant.

Supporting information

Data S1. Supporting Information.

Constantin J, Wehby GL. Effects of Medicaid Accountable Care Organizations on children's access to and utilization of health services. Health Serv Res. 2024;59(5):e14370. doi: 10.1111/1475-6773.14370