Importance of medical home domains on emergency visits using a cross-sectional national survey of US children

Abstract

Background and objectives

Receiving care at patient-centred medical homes (PCMH) is associated with reduced emergency department (ED) visits among children. Adverse social determinants of health (SDoH), such as lower socioeconomic status and household poverty, are associated with increased ED visits in children. The objective of this study is to use machine learning techniques to understand the relative importance of each PCMH component among different populations with adverse SDoH on the outcome of ED visits.

Methods: design, setting and participants

This study used the 2018–2019 pooled data from the National Survey of Children’s Health (NSCH), an annual survey of parents and caregivers of US children from birth to 17 years. PCMH components were operationalised by classifying parent/caregiver responses into five domains: care coordination (CC), having a personal doctor or nurse, having a usual source of care, family-centred care and ease of getting referrals. SDoH included five categories: (1) social and community context, (2) economic stability, (3) education access and quality, (4) healthcare access and quality and (5) neighbourhood and built environment.

Primary outcome measure

We used a split-improvement variable importance measure based on random forests to determine the importance of PCMH domains on ED visits overall and stratified by SDoH.

Results

Overall, between 3% and 28% experienced one or more gaps in PCMH domains. Models show that problems with referrals (rank, 2; Gini, 83.5) and gaps in CC (rank, 3; Gini, 81.0) were the two most important domains of PCMH associated with ED visits in children. This result was consistent among black and Hispanic children and among children with lower socioeconomic status.

Conclusions

Our study findings underscore the importance of poor CC and referrals on ED visits for all children and those from disadvantaged populations. Initiatives for expanding the reach of PCMH should consider prioritising these two domains, especially in areas with significant minority populations.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• The study is based on nationally representative data of US children from birth to 17 years old.

• It uses ‘random forests’, a machine learning technique and an associated importance measure to rank the domains of patient-centred medical homes (PCMH).

• It explores the interplay between five domains of PCMH and five adverse social determinants of health to understand the use of the emergency department (ED) among children.

• The study is cross-sectional, and the temporal relationship between care and ED visits cannot be ascertained.

• Since this survey is based on parent or caregiver self-report, the answers to questions about PCMH may not reflect the actual PCMH certification of providers.

Background

Children in the USA have approximately 34 million emergency department (ED) visits per year with one study estimating the median cost per visit in 2016 to be around $1300.¹ Visits to the ED by children have been steady for many years,² with recent upticks in visits for mental health-related concerns.³ While there will always be necessary ED visits among children, effective ambulatory care has been shown to reduce ED visits and costs in children.^{4 5}

Over the last decade in the USA, efforts to improve the quality of ambulatory care for children have focused on delivering care through a patient-centred medical home (PCMH) after it was introduced as a goal in Healthy People 2010.^{6 7} PCMH is a set of guidelines and processes for the delivery of ambulatory care. The American Academy of Pediatrics specifies seven qualities essential to care from a PCMH: accessible, family-centred, continuous, comprehensive, coordinated, compassionate and culturally effective care.^{6 8} In the USA, a practice can be accredited as PCMH by organisations like the National Committee for Quality Assurance, which will evaluate and ensure compliance with the PCMH criteria. Several practices within the USA are certified as PCMHs; however, many are not. PCMH evolved from a simple desire to better coordinate care for children in the 1960s, with a focus on children with chronic conditions or special needs, to the recommended way to provide care for all children in general paediatric practices today. In the early 2000s, most children received appropriate care in one or more domains, but few received all the domains of PCMH care.⁷

Studies have confirmed that effective ambulatory care via PCMHs is also associated with reduced ED visits among all children as well as those with chronic conditions and special medical needs (children with special healthcare needs (CSHCN)).59,12 One recent paper from this team showed that gaps in care coordination (CC) are associated with increased ED visits among all children, not just those with chronic conditions or special medical needs.¹³

There have also been studies on the association of each specific domain of PCMH on ED visits, such as coordination of care13,16 and family-centred care.^{17 18} The main concern with the PCMH is that its uptake among children has stalled after some early successes.¹⁹ In fact, a US national survey of children in 2016 reported that 48.6% of children had access to a PCMH and 46.6% had access in 2021, indicating no recent improvement.²⁰

In contrast, adverse social determinants of health (SDoH), such as lower socioeconomic status and household poverty, are associated with increased ED visits in children.^{21 22} Studies have shown that the different domains of the PCMH may affect ED use differently depending on the patient’s SDoH.²³ Further, PCMH has failed to reach all children equitably. In 2020–2021, 55.6% of white children reported receiving care from a PCMH compared with 34.7% of Hispanic and 37.1% of black children.²⁰ One recent paper from this team showed that children with adverse SDoH, particularly those with multiple adverse SDoH, are more likely to experience gaps in CC.²⁴ Hence, understanding which domains of the PCMH are most important in populations experiencing adverse SDoH is essential to finding ways to increase the uptake of PCMH and eventually reduce ED visits.¹³ However, there have been no studies that rank the relative importance of PCMH among children overall or among those with SDoH with the specific outcome of ED visits. Two other factors that are known to affect ED use among children are CSHCN status⁵ and age of the child.² Understanding how these factors interact with PCMH and SDoH is critical when studying ED visits. By including several domains of the PCMH, multiple adverse SDoH and other attributes (age, gender and CSHCN), one can determine their relative importance on ED visits. This approach helps disentangle their interrelationships and provides guidance for providers and policymakers on how to prioritise domains of PCMH based on the social circumstances, age and CSHCN status, to provide ambulatory care that best fits the patients’ needs and reduces unnecessary ED visits.

Hence, the primary objective of this study is to understand the relative importance of each PCMH component among different populations with adverse SDoH on the outcome of ED visits. Secondarily, we look at the inter-relationships among the domains of PCMH, adverse SDoH, CSHCN and age categories (infants, young children and teenagers) to determine the relative strengths of their associations with ED visits among children. We use a machine learning technique to understand the relative importance of these different elements after considering their association with each other and to the outcome of ED visits.

Methods

This study uses the National Survey of Children’s Health (NSCH), an annual survey of parents and caregivers of children in the USA from birth to 17 years of age. These data are available for public use, and this project was deemed exempt by the Weill Cornell Medicine Institutional Review Board.

Data source and population

The survey is managed by the Maternal and Child Health Bureau which is a division of Health and Human Services, and it conducts this survey with the US Census Bureau.²⁵ The survey is administered annually in all 50 states and DC in English and Spanish. Data collection typically starts in June, and the complete dataset is released in October of the following year. NSCH also releases pooled data that combines responses from multiple years to enhance the sample sizes and reliability of the estimates. Our study used the pooled data from 2 years: 2018 and 2019.

The NSCH has a complex design as it oversamples households with children between birth and 5 years old and, separately, children with special needs. The surveys are completed by an adult, usually a parent, who can respond by mail or online and who has knowledge of the child’s health.^{25 26} There are age-specific surveys with targeted questions for children 0–5 years, 6–11 years and 12–17 years. More details of the survey are published elsewhere.^{13 24} Overall response rates were around 40% and hot-deck imputation techniques were employed for sex, race and ethnicity variables in the survey.²⁷

All children are screened using five criteria to classify them as CSHCN or not.²⁸ Screening criteria include regular use of medications; routine need for medical, mental health or educational services; use of specialised services (speech, occupational or physical therapy); inability to do things like their peers; or receiving treatment for an emotional, behavioural or developmental problem. CSHCN are defined as having at least one of the above criteria that is expected to last at least 12 months.²⁹ Children who do not meet any of the criteria are classified as non-CSHCN.

Patient-centred medical Home

The PCMH components were operationalised in the survey by classifying parent/caregiver responses into five domains: CC, having a personal doctor or nurse, having a usual source of care, family-centred care and ease of getting referrals. Online supplemental table 1 details the questions and the definitions as well as the responses used to define each domain.

Social determinants of health

We used the WHO conceptual model²¹ and the Healthy People 2030 implementation guidelines to select adverse SDoH.³⁰ SDoH includes five distinct domains: (1) social and community context, (2) economic stability, (3) education access and quality, (4) healthcare access and quality and (5) neighbourhood and built environment. Online supplemental table 2 describes the categories and the specific questions used to operationalise them.

Adverse SDoH often cluster together within the same individual. For example, an individual with low education may also have low income and live in a poor neighbourhood. As such, we generated a variable that reflected the number of SDoH for each individual and calculated a score ranging from 0 to 5. We further classified the score to indicate none, 1, 2 and 3 or more adverse SDoH. The rationale behind this SDoH score is to capture the cumulative burden of adverse SDoH for each child.³¹

The primary outcome measure was ED visits dichotomised from the NSCH survey question: ‘During the past 12 months, how many times did this child visit a hospital emergency room?’ Those with one or more visits will be coded as 1 and those with none as 0.

Statistical analysis

We reported the unweighted number and the weighted percentages for each domain after removing missing values. After a visual analysis of the importance of PCMH domains by SDoH populations, we used a split-improvement variable importance measure based on random forests (RFs) to obtain the importance of each explanatory variable. RFs are a modern ensemble method which combine many estimated decision trees into an overall model for the endpoint given the explanatory variables.^{32 33} This method has several advantages. First, RFs are a flexible machine learning method which can detect nonlinear variable effects and interactions instead of merely linear effects such as in a generalised linear model such as a linear or logistic regression.³³ Second, the aggregate of trees which defines RFs results in estimates having lower variance compared with estimates based on just one tree.³² Third, RFs can take into account observation weights such as the sampling weights in survey response data.³⁴ The variable importance measure we used is based on ‘Gini impurity’, which directly measures the degree of improvement in prediction attributed to each explanatory variable.³² The Gini impurity is a well-established measure for decision trees.³⁵ We used a modern variable importance measure which updates the Gini impurity to appropriately compare continuous and categorical variables.³⁵ Further, there are well-developed computational packages which support RFs and associated variable importance measures; we used the R package ‘ranger’ for the analyses.³⁴ We report the importance rank that reflects importance of each variable after adjustment of all others in the model and the corrected Gini impurity score that shows how beneficial it is to split on a variable. The higher the value of the score, the more beneficial it is. If the score is negative, it indicates that the variable is detrimental to split.

Candidate variables for stratified analysis

The main objective was to rank the importance of the five domains of PCMH overall and stratified by SDoH and demographics. However, to determine the ‘best’ candidates for stratification, we first ran a model that ranked the five domains of PCMH, the five SDoH variables and three potential confounders (age, gender of the child and CSHCN status) on the endpoint of ED visits for the total population. Next, we selected the highest-ranked co-variates and stratified the population by these variables. Third, we built separate models for each of the strata selected in the prior step where we ranked the importance of PCMH domains. Finally, we ran additional models using the measures of overall burden of SDoH.

The National Survey of Children’s Health 2018–2019 data can be accessed at the data resource centre http://www.childhealthdata.org. All analyses were performed using SAS V.9.4 (Cary, NC) or R V.4.2.2.

Results

Sample characteristics

There were 59 993 children in the survey, and about 33% were in each age group (infants, from birth to 5 years; young children, 6–11 years; and teenagers, 12–17 years). Half (51%) were male, and 19% were classified as CSHCN. Overall, between 3% and 28% experienced some gap in the PCMH domains, with 15% experiencing gaps in the domain of CC, 11% experiencing poor family-centred care, 3.4% having problems with referrals, 28.2% having no personal doctor or nurse and 24% having no usual source of care (table 1).

Table 1. Weighted proportion of US children who report PCMH domains from the National Survey of Children’s Health 2018–2019—overall and by SDoH category.

SDoH	Total	Domains of PCMH†
	Unwtd N	Wtd % with gaps in CC	P	Wtd % with poor family-centred care	P	Wtd % with problems with referrals	P	Wtd % with no personal doctor/nurse	P	Wtd % with no usual sources for sick care	P
All children	59 963	14.8		11.0		3.43		28.2		24.3
Social and community context					*		*		*		*
Black or Hispanic	11 018	15.2		14.2		4.1		36.0		34.0
All others	48 945	14.6		8.98		3.0		23.2		18.2
Economic stability			*		*		*		*		*
FPL<400% or any benefit	35 809	15.3		12.8		3.9		32.7		29.3
All others	24 154	13.7		6.7		2.0		17.7		13.0
Education access and quality					*				*		*
High school or less	9371	13.7		14.1		3.6		41.3		40.9
Above high school	50 592	15.3		9.7		3.3		22.9		17.8
Healthcare access and quality			*		*		*				*
No or inadequate insurance	16 881	24.1		17.1		5.7		26.9		21.7
Adequate insurance	40 002	11.5		8.5		2.7		28.6		25.2
Neighbourhood and built environment			*		*		*		*		*
Experience to violence or fewer than two amenities	6432	20.2		18.5		7.0		33.3		26.8
All others	52 100	14.1		9.8		2.9		27.4		24.0
Count of SDoH			*		*		*		*		*
0	13 191	9.3		4.0		1.6		16.3		12.0
1	20 500	13.0		7.4		2.6		21.4		15.3
2	14 986	16.0		11.3		3.4		29.0		25.6
3 or more	7054	18.4		18.3		5.4		41.2		39.6

*P value<0.05 using Rao-Scott χ2 test.

* –* – using Rao Scott ChiSquare test†Weighted percentage calculated on the N’s for each domain after removing missing values.

CCcare coordinationFPLfederal poverty limitPCMHpatient-centred medical homeSDoHsocial determinants of healthUnwtd Nunweighted number in the total survey samplewtd %weighted percentage

In terms of SDoH, 37% met criteria for adverse social and community context (being Hispanic or black), 68.1% for poor economic stability, 26.1% for lower education access and quality, 26.9% for inadequate healthcare access and quality and 13% for poor neighbourhood and built environment. The proportion of children experiencing gaps in PCMH components was almost always higher among children with any adverse SDoH compared with those without any SDoH. Among children with poor neighbourhood and built environment, 7% reported problems with referrals compared with 3% who do not experience this adversity (table 1). Further, the additional burden of SDoH was significantly associated with more gaps in care in all domains of PCMH. For example, 40% of children with three or more adverse SDoH had no usual source of care compared with 12% for children with no adverse SDoH. Most strikingly, of children with inadequate healthcare access and quality, 24% reported a gap in CC compared with 12% of those who had adequate access.

Candidate variables for stratification

We chose variables for stratification based on their importance ranking in the overall model on all children. As such, age category, which was ranked as the most important predictor (rank, 1; Gini, 131.2), CSHCN status (rank, 5; Gini, 56.6), lower education access and quality (highest education is high school or less (rank, 4; Gini, 58.4)) and adverse social and community context (being black or Hispanic (rank, 2; Gini, 50.7) were chosen for subgroup analysis (table 2).

Table 2. Ranking of the Gini importance score from random forest models on emergency department visits—overall, by age category and by CSHCN status.

			Stratified by CSHCN				Stratified by age category
	All children		CSHCN		Non-CSHCN		Ages 0 to 5		Ages 6 to 11		Ages 12 to 17
	Importance rank	Gini	Importance rank	Gini	Importance rank	Gini	Importance rank	Gini	Importance rank	Gini	Importance rank	Gini
Medical home domains
Gaps in care coordination	3	81.0	7	13.0	3	37.9	9	5.3	3	22.1	3	22.1
Poor family centred care	7	38.9	9	6.0	4	34.1	4	15.9	5	16.4	5	16.0
Problems with referrals	2	83.5	3	26.5	2	46.1	5	15.5	1	35.3	1	34.5
No personal doctor or nurse	10	21.6	8	11.3	9	13.3	8	5.6	8	8.1	8	7.7
No usual sources for sick care	9	21.7	6	16.7	8	13.8	6	11.7	7	11.1	7	12.1
Social determinants of health
Social and community context	6	50.7	4	24.9	6	28.3	3	25.7	4	20.5	4	21.0
Economic stability	13	−81.8	12	−9.4	12	−71.8	13	−30.4	13	−23.5	13	−24.0
Education access and quality	4	58.4	2	36.7	5	31.9	2	25.7	2	−26.6	2	26.7
Healthcare access and quality	11	−24.9	11	0.1	10	−19.2	11	−7.9	11	−4.7	11	−4.7
Neighbourhood and built environment	8	25.3	5	16.7	7	17.3	7	11.2	9	5.7	9	5.7
Demographics and health
Child gender	12	−34.4	10	1.5	11	−25.0	12	−14.3	12	−10.4	12	−10.6
Age of child	1	131.2	1	74.6	1	96.0	10	−0.1	10	0.1	10	0.5
CSHCN status	5	56.6					1	33.2	6	15.3	6	15.4

CSHCN – Children with special healthcare needs; Shading implies the top 2 ranked in each categoryMedical Home Domains; Nnegative scores imply no importance.

CSHCNchildren with special healthcare needsGini, impurity corrected value

Relative importance ranking of PCMH domains

Figure 1 shows a heatmap of the relative importance of the five PCMH domains overall and for specific subgroups. It shows that problems with referrals and gaps in CC are among the two most important domains of PCMH associated with ED visits in children overall, with some slight variations (figure 1, table 2). In age-stratified analysis, among children aged birth to 5 years, poor family-centred care and problems with referrals were the two most important predictors of ED visits. Among the subgroup of CSHCN, problems with referrals and having no usual sources for sick care were the most important predictors. Among older children and non-CSHCN, the results were consistent with the overall population (figure 1, table 2). Among those with adverse education access and quality, gaps in CC and problems with referrals were the two most important domains of the PCMH to predict ED visits (figure 1, table 3). Among those with adverse social and community contexts, problems with referrals and gaps in CC were also the most important domains of the PCMH to predict ED visits (figure 1, table 3).

Figure 1. Importance ranking of patient-centred medical home (PCMH) domains for all children and by subgroups. Each box represents the relative importance of each PCMH domain associated with emergency department visits, determined by the random forest models and Gini scores. Rank 1 indicates that domain was ranked most important among the five domains, and rank 5 indicates that it was least important. Rankings for ‘All Children’ are derived from a model that includes age, gender, adverse social determinants of health, children with special healthcare needs (CSHCN) status and PCMH domains. Other models include all attributes except the one that determines the subgroup.

Table 3. Ranking of the Gini importance of patient-centred medical homes domains from random forest models on emergency department visits—stratified by adverse social determinants of health.

	Education access and quality highest level in HH: high school or less		Education access and quality highest level in HH: more than high school		Social and community context hispanic ethnicity or black race		Social & community context all other ethnicity/ races
	Importance rank	Gini	Importance rank	Gini	Importance rank	Gini	Importance rank	Gini
Medical home domains
Gaps in care coordination	2	30.8	4	45.0	3	16.3	2	56.8
Poor family centred care	4	7.7	5	20.8	7	6.7	7	13.9
Problems with referrals	3	21.9	2	55.6	2	22.8	3	45.4
No personal doctor or nurse	9	−0.9	6	16.7	9	1.2	9	5.4
No usual sources for sick care	7	5.3	9	7.8	6	7.1	6	21.6

Shading implies the top 2 ranked in Medical Home Domains; negative scores imply no importance.

Results by burden of SDoH

Finally, the model that included the burden of SDoH identified this variable as the single most important predictor of ED visits (rank, 1; Gini, 83.5) (online supplemental table 3).

Patient and public involvement

Patients were not involved in the aims of this study or the analysis of this data.

Discussion

This study uses data from a US nationally representative survey and a novel methodological approach to highlight the most important aspects of the PCMH that predict ED visits among children. It shows that problems with referrals and gaps in CC are the two most important elements of PCMH for ED use among children, after adjusting for age, CSHCN status and SDoH. Our study findings suggest that reducing gaps in CC and referrals may lower ED use, especially among children aged 6 to 17 years. We also observed that family-centred care needs should be prioritised for the youngest children (birth–5 years) and a usual place of care for children with special needs.

This study builds on prior work by this team that showed how adverse SDoH can increase gaps in CC²⁴ and how these gaps can be associated with more ED use in all children, not just those with special healthcare needs.¹³ This study includes the five PCMH domains, five adverse social determinants (including a burden of SDoH) and other important factors such as the age of the child and CSHCN status and uses a supervised classification algorithm to determine their relative importance for ED use. Our study adds to a growing area of research that looks at the individual components of the PCMH to better inform how this model of care can be improved for all children.^{36 37} The advantages of PCMH use on the reduction of ED visits are clearly documented59,13; however, the recent inability to increase the proportion of children getting PCMH care has given impetus for this type of research.¹⁹

Consistent with prior work, our study emphasises the importance of reducing gaps in CC and problems with referrals for minority populations (black and Hispanic) and for those children living in households with lower socioeconomic status (less than high school).²⁴ This is particularly interesting as the adoption of PCMH has been much slower among communities of colour,³⁸ and such prioritisation can help policymakers reach these communities better.

Our findings are supported by two studies that used earlier versions of the NSCH survey.^{36 37} One study sought to understand the contribution of each PCMH domain on ED visits using multivariable logistic regression analysis. It showed that those receiving CC when they needed it, those with higher education and white children had lower odds of ED visits; these results are consistent with the findings from this study.³⁶ However, our study uses an RF method of ranking the relative importance of each domain and uses more recent survey results.

The second study³⁷ uses the NSCH to describe ethnic disparities in access to the different components of the PCMH. Our study adds to this by highlighting the importance of gaps in CC and referrals for these communities to improve care and lower ED visits.

Finally, our study finds that the burden of SDoH is the single most important predictor of ED visits, confirming the need for keeping SDoH top of mind when designing interventions to improve care for children. It shows that the interplay of PCMH, a set of enabling components to improve care, and SDoH, a set of predisposing characteristics that can detract from care, are important to detangle to address the current inequities in children’s care. Further research with other data sources and analytic methods are necessary to make the PCMH more adaptable to fit the prioritised needs of different, particularly disadvantaged, communities.

Limitations

This study has some limitations. First, the survey is cross-sectional, and we cannot discern the temporal relationship between ambulatory care and ED visits. Second, unmeasured confounding may exist. Third, it is because the survey did not ask about the reasons for the ED visits, and we are unable to determine if these visits could have been prevented with improved ambulatory care. Fourth, since this survey is based on parent or caregiver reports, the answers to questions about PCMH may not align with how the practices see themselves or what PCMH certification they may have received. Fifth, we also acknowledge that these relationships may have changed during the COVID-19 pandemic. Finally, statistical comparisons are not possible in importance analysis with RF in surveys as standard errors in this setting have not been developed.

Conclusion

This study underscores the important role that addressing problems with CC, and referrals may play to reduce ED visits for children, especially those with adverse SDoH. Future studies should identify the mechanisms by which better CC and improved referral processes may reduce ED visits. Strategies to expand the reach of PCMH should consider prioritising these two domains, especially in geographic regions that serve a greater proportion of underserved populations.

Data availability statement

Data are available upon reasonable request.