A Secret Shopper Study of Language Accessibility of Community-based Behavioral Health Services for Children in Families Who Speak Spanish and English

Abstract

Objective

To compare outpatient behavioral health scheduling for children in Spanish-speaking families in Pennsylvania with that for children in families who speak English.

Study design

We made paired English and Spanish telephone calls to outpatient behavioral health facilities using a standardized script, describing a simulated, stable, Medicaid-insured child. Facilities were identified using the Pennsylvania Department of Human Services Online Provider Directory for Mental Health and Substance Abuse Services, which had 288 outpatient facilities with non-duplicate telephone numbers. An English-language caller following a script made up to two call attempts per facility from December 2019 through February 2020. The 126 facilities that did not answer the phone, accept Medicaid, or see children were removed. A Spanish-language caller then made up to two scripted call attempts to the 162 remaining facilities. The primary outcome was whether the facility tried to schedule an appointment for the simulated adolescent.

Results

125 facilities answered both English- and Spanish-language calls. For the English-language caller, 71% of facilities attempted to schedule an appointment and 100% communicated in the caller’s preferred language. For the Spanish-language caller, 24% attempted to schedule an appointment (P<0.001) and 25% communicated in the caller’s preferred language (P<0.001).

Conclusions

Among outpatient behavioral health facilities for Medicaid-insured children in Pennsylvania, there were inequities in access to appointments for families who speak Spanish compared with English. This a modifiable barrier to care. Community-based behavioral health care for children should strengthen language access training, contracting, and oversight.

Approximately one in six US children has a behavioral health condition.^1,2 There are significant disparities in access to health services for both adult and pediatric populations.^3,4,5,6,7,8 Behavioral health care may be particularly difficult to navigate for individuals with limited English proficiency (LEP), meaning they self-assess their ability to speak English as less than “very well.” However, the extent to which access to behavioral health services differs based on English proficiency remains poorly documented.^9,10

Nationally, approximately 16% of children have at least one LEP parent, including approximately 44% of Latino and Asian American children.¹¹ Among older (16–17 year old) adolescents—who can seek behavioral health services independently in some states—nearly 4% are themselves LEP.¹¹ Federally-funded health entities are responsible for ensuring their programs and services are accessible to LEP individuals under Title VI of the 1964 Civil Rights Act and the Affordable Care Act, but prior studies raise concerns regarding how language access policies are implemented.^12,13,2 Notably, these challenges exist despite the majority of children in immigrant and LOE families having health insurance.¹⁵

Most research examining the root causes of poor access to outpatient behavioral health services for LEP individuals has focused on adults, on cultural factors (eg, stigma or normative beliefs about child behavior), or on selected subgroups of children (eg, those with autism).^2,16 There has been less research on the degree to which the child behavioral health system itself accommodates or fails to accommodate the language needs of families who speak languages other than English.¹⁷ Notable exceptions include a 2010 survey on language services use by members of the American Academy of Pediatrics—many of whom are pediatricians who manage behavioral health conditions such as ADHD in primary care—which showed that 57% of participating pediatricians continued to use bilingual family members as non-professional interpreters.¹⁸ A 2013 telephone survey of Washington state services for individuals with traumatic brain injury found that only 9% of 210 children’s mental health services offered accommodation for language needs.¹⁹ More recently, a 2016 secret shopper study found that 31% of developmental pediatrics programs affiliated with US children’s hospitals were unable to accommodate the language needs of a Spanish-language caller.²⁰

This evidence points to an ongoing need for improved language access in pediatric behavioral health services. Additional research on language accommodation by individuals who serve as the first point of contact for parents seeking pediatric behavioral health services, such as schedulers and front desk staff, is also necessary, as these individuals represent a critical access point into the system. To address this gap in our understanding of access to care, we examined community-based pediatric behavioral health scheduling for simulated Spanish-language callers and compared scheduling experiences with those of otherwise comparable English-language callers.

Methods

Paired English and Spanish-language calls in which a simulated parent called to schedule care for a simulated, stable adolescent patient were made to outpatient behavioral health facilities located in Pennsylvania (PA). Spanish was selected because, after English, it is the second most widely spoken language in PA, where it is spoken by 5.6% of the population ages 5 years and older. Facilities were identified using the publicly available PA Department of Human Services (DHS) Human Services Provider Directory (version date August 1, 2019) using the “Outpatient” Service Code and the “Office of Mental Health and Substance Abuse Services” Program Office, which generated an initial list of 2,737 facilities.²¹ As illustrated in Figure 1, duplicate entries (defined as facilities with the same phone number or address) were then removed from this list, and calls were made to the remaining 288 facilities.

Figure 1: Development of the Analytic Sample.

*PA Directory: Service providers were identified in the Pennsylvania Department of Human Services Human Services Provider Directory (version date August 1, 2019) using the “Outpatient” Service Code and the “Office of Mental Health and Substance Abuse Services” Program Office

**Exclusion criteria included not seeing children/adolescents, not accepting Medicaid, or not answering the English-language caller after up to two attempts.

Study Sample

An English-language caller made two call attempts to each of the 288 outpatient behavioral health facilities, of which 24 did not have an accurate phone number, 34 were ineligible (did not see children and/or did not accept Medicaid), and 68 did not answer the phone. Up to two Spanish-language call attempts were then made to each the 162 eligible facilities with completed English-language calls. Of these facilities, 35 did not answer the phone after two attempts and 2 withdrew from the study after receipt of a debriefing letter (which invited sites to opt out of the study if they preferred), leaving a final sample of 125 facilities (Figure 1).

Study Design

Using a protocol and interview script, two study team members made phone calls to each facility. They followed a script in which the mother of a Medicaid-insured, stable adolescent sought to establish behavioral health care for her child. The script was adapted from a prior secret shopper study on access to mental health care for children in New York state, and it was refined with input from the project’s community advisory board. The latter comprised four immigrant community leaders with experience supporting LEP families seeking access to behavioral health care.²² Apart from primary language and caller/patient names, English and Spanish-language callers followed identical scripts. The scripts were translated by one bilingual, bicultural study team member with personal, community and academic experience working with Spanish-speaking immigrant families in the United States. The same individual made each Spanish-language phone call. The translated script was reviewed by multiple other study team members with professional experience delivering health or social services in Spanish and English. The scripts included a simulated caller name, simulated patient name and date of birth, simulated local address, and brief description of the adolescent’s chronic, non-acute mental health concern. Because behavioral health care for Medicaid-insured children in PA is covered by county-specific behavioral health managed care organizations (MCOs), the name of the county-specific behavioral health MCO was the scripted response to any question about insurance coverage. Scripts did not include fictitious Medicaid ID numbers. Nor did they include fictitious Social Security Numbers. Each script ended with the caller declining to schedule an appointment if one was offered.

Data Collection

Phone calls were made between December 2019 and February 2020 during business hours (9:00 am to 5:00 pm, excluding holidays and weekends). Up to two call attempts per language were made to each facility. The primary outcome was whether any attempt was made to assist with scheduling an appointment of any kind (intake, evaluation or other) for the simulated patient, regardless of whether this scheduling attempt ended with the caller being offered a specific appointment date and time. This benchmark was selected because many facilities cannot offer a specific appointment date and time without first verifying insurance information. Facilities that asked callers to leave a message or call back another time, routed calls to voicemail, or placed the caller on hold for longer than 20 minutes were categorized as not having attempted to assist with scheduling. These stipulations were included because they reflect reasonable expectations for a real parent with time and resource constraints, eg, inability to engage in personal calls while at work. Facilities that simply stated they did not speak Spanish and then waited in silence for the simulated Spanish-language caller to hang up were categorized as not having attempted to assist with scheduling.

The secondary outcome was whether communication took place in the caller’s preferred language. If the Spanish-language call was answered in English and then connected to a bilingual scheduler or interpreter, or if the Spanish-language call was answered in Spanish, this was counted as appropriate use of the caller’s preferred language. If the Spanish-language caller was told in English to hang up and call a different number, transferred to someone who did not speak Spanish, or transferred to the voicemail box of someone who reportedly spoke Spanish, this was not counted as appropriate use of the caller’s preferred language.

For callers for whom an attempt was made to schedule an appointment, we also recorded whether this attempt was successful, meaning whether an appointment of any kind (intake, evaluation, or other) was ultimately offered to the caller. If the facility provided the caller with walk-in intake hours, this was also categorized as a successful scheduling attempt. For unsuccessful scheduling attempts, we recorded the reason if one was offered by the scheduler or self-evident to the caller, eg, no available providers or not accepting new patients. For all calls, we recorded the overall call length and total time spent on hold in minutes. Callers maintained detailed field notes. Data were entered directly into REDCap, a secure web platform for managing online databases.^23,24

Statistical Analysis

Descriptive analyses were completed with R statistical software within R studio.^25,26 Using McNemar’s chi-square test, we examined primary and secondary outcomes for the overall study sample. We also compared outcomes for rural/urban counties, behavioral health MCO service areas, and counties with >5% LEP residents. Rural counties were categorized according to the PA State Legislature (Center for Rural Pennsylvania) definition of less than 284 persons per square mile.²⁷ PA has five behavioral MCO service areas, which are based on county-level contracts for the provision of behavioral health services for Medicaid-insured children. The proportion of LEP residents in each county was obtained from the Census Bureau (American Community Survey 2017 5-year estimates).²⁸ These analyses were specified a priori.

In addition, we performed subgroup analyses in which we separately examined outcomes for English and Spanish-language calls. For each language, we used bivariate and multivariable logistic regression to examine the relationship between outcomes and the county-level characteristics described above (rural/urban, behavioral health MCO, LEP%).

This study, including secret shopper methodology, was approved by the University of Pennsylvania Institutional Review Board.²⁹ After completing the phone calls, we sent each facility a letter describing the study and offering them the opportunity to withdraw. Two facilities withdrew from the study.

Results

Table I compares the 288 unique facilities identified in the PA directory with the final study sample (N=125). Facilities included in the final sample were somewhat more likely to be rural, have ≤5% LEP residents, and contract with behavioral health MCO #5 when compared with facilities that were not included. These differences were not statistically significant (P>0.05). Facilities included in the final sample were distributed across 32 of 67 PA counties and all five behavioral health MCO service areas.

Table 1.

Description of the Sample

	Statewide Sampling Frame a	Final Sample b
N	288	125
Behavioral Health MCO Service Area, % c
BH-MCO 1	20	18
BH-MCO 2	34	38
BH-MCO 3	22	15
BH-MCO 4	12	13
BH-MCO 5	10	16
Rural County, % d	25	32
County with >5% LEP Residents, % e	31	25

^aCommunity-based behavioral health facilities were identified in the Pennsylvania Department of Human Services Human Services Provider Directory (version date August 1, 2019) using the “Outpatient” Service Code and the “Office of Mental Health and Substance Abuse Services” Program Office, which generated an initial list of 2,737 facilities. Duplicate entries (defined as facilities with the same phone number or address) were then removed from this list, and calls were made to the remaining 288 unique facilities.

^bEligible outpatient behavioral health facilities for Medicaid-insured children with completed, paired English and Spanish language calls and that did not opt to withdraw from the study. Differences between facilities included and not included in the sample were not statistically significant (P>0.05).

^cLocated within a county served by this Medicaid behavioral health managed care organization (MCO)

^dLocated in a rural county, as defined by the PA State Legislature as a county where the number of persons per square mile is less than 284

^eLocated in a county with >5% LEP residents, as per the Census Bureau’s 2015 American Community Survey 5-year Estimates

Figure 2 compares attempts to assist the English- and Spanish-language caller with scheduling an appointment for the simulated pediatric patient without being asked to leave a message, call back another time, or wait on hold for longer than 20 minutes. Nearly three-quarters (71%) of facilities attempted to schedule an appointment for the English-language caller. One-quarter (24%) attempted to schedule an appointment for the Spanish-language caller (P<0.001). As shown in Figure 3, every facility communicated with the English-language caller in English, and one-quarter (25%) of facilities communicated with the Spanish-language caller in Spanish (P<0.001). Broadly, field notes indicate staff answering the phone were polite and attempted to be helpful to the Spanish-language caller, but staff were also uncertain how to communicate with someone speaking Spanish.

Figure 2: Outcomes for 125 paired Spanish- and English-language calls seeking outpatient, behavioral health care for a simulated, Medicaid-insured adolescent in Pennsylvania.

¹ We determined whether an attempt was made to assist with scheduling an appointment of any kind, e.g., intake, evaluation, walk in, or other (“Answered, assisted”), for a simulated, stable, Medicaid-insured adolescent as a new patient at 125 community-based behavioral health facilities in Pennsylvania. We then compared outcomes for paired Spanish- and English-language calls (p<0.001).

Figure 3: Preferred language use during 125 paired Spanish- and English-language calls seeking outpatient, behavioral health care for a simulated, Medicaid-insured adolescent in Pennsylvania.

¹ We determined whether communication took place in each caller’s preferred language. If the Spanish-language caller was answered in English and then transferred to a bilingual scheduler or interpreter—or if the caller was answered in Spanish—this counted as appropriate use of the caller’s preferred language. If the Spanish-language caller was told in English to hang up and call a different number, transferred to someone who did not speak Spanish, or transferred to the voicemail box of someone who reportedly spoke Spanish, this was not counted as appropriate use of the caller’s preferred language. We compared outcomes for paired Spanish- and English-language calls (p<0.001).

We also examined more detailed scheduling outcomes for the subset of 30 Spanish-language and 98 English-language calls in which an attempt was made to assist the caller in their preferred language: 42% of 98 facilities scheduled an appointment or offered walk-in hours to the English-language caller, whereas only 13% of 30 facilities scheduled an appointment or offered walk-in hours to the Spanish-language caller (data not shown). We did not ask why clinics were unable to schedule appointments. However, field notes maintained by study staff suggest that common reasons included lack of Medicaid ID, lack of Social Security Number (for the Spanish-language caller), and facilities being unwilling to schedule Spanish-speaking patients with clinicians who do not speak Spanish.

For English-language calls, as shown in Table II, neither county urban-rural status, % LEP residents, or behavioral health MCO were associated with substantive variation in the primary outcome (P>0.05). In contrast, scheduling attempts and preferred-language communication were more common when Spanish-language calls were made to facilities in urban counties compared with non-urban counties (P<0.05) or counties with >5% LEP residents compared with counties with fewer LEP residents (P<0.001). In models adjusted for other county-level characteristics, Spanish-language calls were more likely to be successful when facilities were in counties with >5% LEP residents (AOR 1.60, 95% CI 1.21, 2.10). Similarly, Spanish-language callers were more likely to experience communication in their preferred language when facilities were in counties with >5% LEP residents (AOR 1.66, 95% CI 1.24, 2.21). Urban-rural status was not associated with either outcome in models adjusted for other county-level characteristics.

Table 2:

Outcomes for 125 paired Spanish- and English-language calls seeking outpatient, behavioral health care for a simulated, Medicaid-insured adolescent in Pennsylvania: Subgroup analyses by County-level characteristics and Behavioral Health Managed Care Organization

	Attempt to Schedule (%)		Communication in Caller’s Preferred Language (%)
	Spanish	English	Spanish	English
County-level Rural-Urban Status:
Urban (N=85)	31	73	31	100
Rural (N=40)	10	68	15	100
County-level % LEP Residents:
Greater than 5% (N=31)	55	77	55	100
5% or less (N=94)	14	70	16	100
Behavioral Health MCO:
1 (N=23)	48	70	48	100
2 (N=47)	23	77	19	100
3 (N=19)	16	63	26	100
4 (N=16)	25	75	31	100
5 (N=20)	5	65	10	100

We determined whether an attempt was made to assist with scheduling an appointment of any kind—e.g., intake, evaluation, or other—for a simulated pediatric patient (“Attempt to Schedule”). We also determined whether communication took place in each caller’s preferred language (“Preferred Language”).

LEP: Limited English proficiency

MCO: Managed care organization

Discussion

In this study, Spanish-language callers rarely received immediate assistance scheduling an appointment for a simulated pediatric patient, in contrast to English-language callers. Further, the behavioral health facilities in this sample communicated in the preferred language of a simulated Spanish-language caller for only 25% of calls. The differences in the experiences of English and Spanish-language callers point to a clear and modifiable inequity in access to children’s behavioral health services.

Title VI of the 1964 Civil Rights Act protects individuals with LEP from discrimination when seeking or receiving services from a federally-funded entity.³⁰ Those entities include hospitals and ambulatory care facilities, which are required to make reasonable accommodations for LEP individuals, eg, by contracting with telephonic and other interpreters.¹² However, our findings suggest that outpatient behavioral health clinics serving Medicaid-insured children in Pennsylvania may not consistently meet these standards. This is consistent with prior, national research on access to developmental-behavioral pediatrics scheduling for a simulated Spanish-language caller.²⁰ Given the pervasiveness of difficulty scheduling appointments for simulated Spanish-language callers, these disparities point to systems-level gaps and suggest systems-level solutions. For example, Medicaid agencies or managed care organizations could identify and offer additional training to clinical sites that are struggling to serve patients who speak LOE.

These studies show there is also an urgent need to improve language access-related workflows, training, and access to technology (eg, phone interpreting) for staff who schedule outpatient pediatric behavioral health care. Concrete actions include adequate contracting with language services and reprogramming “phone trees” to ensure accessibility for LOE callers Staff onboarding at all levels should include mandatory training on accessing language services paired with an overview of best practices for interpreter-mediated communication (eg, orient the interpreter to the purpose of the call; speak directly to the caller). In regions with a high percentage of LOE residents, experienced human resources professionals can strengthen recruitment of bilingual staff. Taking these steps is likely to improve staff morale and retention by ensuring scheduling staff are well-prepared to offer high quality services to all families.

This study is subject to certain limitations. Our sampling frame was restricted to a single state and focused only on programs for Medicaid-insured children. We collected data prior to the COVID-19 pandemic, but we have not seen more recent publications suggesting language access has markedly improved. However, we hope there has been progress and welcome more recent studies for which this can serve as a benchmark. We could not reach every facility in our sampling frame. For this reason, it is possible that our results represent the “best case” scenario, as clinics that are easier to reach may also have more robust scheduling options. We only included a simulated caller for one language other than English (Spanish). Because Spanish is the language most spoken by LEP individuals in the state of Pennsylvania, we anticipate that health facilities are more likely to have Spanish-language staff relative to non-Spanish languages. As such, our results are likely to overestimate access to behavioral health services for families who speak languages other than English and Spanish. Additionally, we categorized clinics that asked the caller to leave a message or call back as not having offered scheduling assistance. We made this decision because we could not verify that leaving a message or calling back would, in fact, lead to scheduling assistance. However, for some clinics this system may be a successful strategy for scheduling appointments.

These findings illustrate the difficulty people with limited English proficiency encounter when seeking outpatient behavioral health services and highlight a profound yet modifiable disparity in service delivery for this population. These data highlight the need to strengthen language access training, contracting, and oversight. Doing so is a critical next step to ensure that all children and adolescent can access necessary behavioral health services.^{31, 32}

Abbreviations:

AOR

Adjusted Odds Ratio

MCO

Behavioral Health Managed Care Organization

PA DHS defines Behavioral Health MCOs as “An entity directly operated by the county government or licensed by the Commonwealth as a Health Maintenance Organization or risk-assuming Preferred Provider Organization which manages the purchase and provision of behavioral health services.”

CI

Confidence Interval

DHS

Department of Human Services

LEP

Limited English Proficiency

An individual who speaks English less than “very well” (self-assessed)

LOE

Language Other than English

PA

Pennsylvania