Assessing Access to Trauma-Informed Outpatient Mental Health Services for Adolescents: A Mystery Shopper Study

Abstract

Objective:

The authors aimed to examine how access to trauma-informed mental health services in safety-net health centers varies by insurance type and race-ethnicity of the care seeker.

Methods:

In this mystery shopper study, three women (White, Latina, and Black voice actresses) called community mental health centers (CMHCs) and federally qualified health centers (FQHCs) (N=229) in Cook County, Illinois, posing as mothers requesting a mental health appointment for their traumatized adolescent child. Each health center was called twice—once in the spring and once in the summer of 2021—with alternating insurance types reported (Medicaid or private insurance). Ability to schedule an appointment, barriers to access, wait times, and availability of trauma-specific treatment were assessed.

Results:

Callers could schedule an appointment in only 17% (N=78 of 451) of contacts. Reasons for appointment denial varied by organization type: the primary reasons for denial were capacity constraints (67%) at CMHCs and administrative requirements to switch to in-network primary care providers (62%) at FQHCs. Insurance and organization type did not predict successful appointment scheduling. Non-White callers were significantly less likely (incidence rate ratio=1.18) to be offered an appointment than the White caller (p=0.019). The average wait time was 12 days; CMHCs had significantly shorter wait times than FQHCs (p=0.019). Only 38% of schedulers reported that their health center offered trauma-informed therapy.

Conclusions:

Fewer than one in five contacts resulted in a mental health appointment, and an apparent bias against non-White callers raises concern that racial discrimination may occur during scheduling. For equitable access to care, antidiscrimination policies should be implemented.

In fall 2021, the American Academy of Pediatrics, the American Academy of Child and Adolescent Psychiatrists, and the Children’s Hospital Association declared a national emergency in child and adolescent mental health. The organizations noted “soaring rates of mental health challenges among children, adolescents, and their families over the course of the COVID-19 pandemic, exacerbating the situation that existed before the pandemic” (1). Adolescence is a time of heightened stress during which most psychiatric disorders arise (2). Of note, early intervention lowers the risk for adverse outcomes (3).

Despite the benefits of early intervention, fewer than half of youths with a psychiatric disorder receive such intervention (4, 5). This gap disproportionately affects youths from low-income families and racial-ethnic minority groups, who have higher unmet mental health needs (6, 7). These unmet needs have been attributed to various barriers, including difficulty navigating the health care system, a shortage of specialized providers, lack of insurance acceptance, and long wait times (8). Furthermore, research indicates that discrimination may occur during scheduling, creating additional barriers for families of color (9–11). Given the United States’ youth mental health crisis, which has worsened during the pandemic, substantial work is needed to identify and reduce the treatment barriers families face when seeking care.

SAFETY-NET HEALTH CENTERS

Youths tend to access mental health services through schools, primary care, and community-based outpatient health centers, such as community mental health centers (CMHCs) and federally qualified health centers (FQHCs) (12). FQHCs are federally funded community-based centers that offer general medical and mental health services in medically underserved areas. CMHCs specialize in both short- and long-term treatments of psychiatric disorders. Providers at these health centers offer a range of evidence-based psychosocial treatments and are critical access points, or safety nets, for youths from low-income families (13) because unlike mental health providers in private practice, most of these centers accept Medicaid (14, 15). In fact, FQHCs provide the highest percentage of mental health services to Medicaid recipients in Cook County, Illinois, the second-most-populous county in the United States and the location of the present study (16, 17). Given that Medicaid insures more than one in three youths in the United States (18), it is imperative to analyze the availability and accessibility of mental health services at each of these safety-net health centers.

TRAUMA-INFORMED CARE

The COVID-19 pandemic has increased the potential for exposure to trauma for youths through media coverage of traumatic events, increased exposure to maltreatment and family violence, and the loss of a family member or loved one due to COVID-19 (19, 20). Understanding whether youths who have experienced traumatic events have access to evidence-based, trauma-informed services could address serious public health concerns, because most youths are exposed to at least one and often multiple traumatic experiences by age 17 (21).

GOALS OF THIS STUDY

Using a mystery shopper method, we asked the following questions: Is insurance type (Medicaid vs. private insurance) associated with the likelihood of scheduling an appointment and with the length of wait times? Is the race-ethnicity of the care seeker associated with appointment access and wait times? Is organizational type (FQHC vs. CMHC) associated with appointment access and wait times? and What percentage of health centers offer trauma-informed treatment for adolescents and what types of treatment do they offer?

METHODS

Study Design

We conducted a mystery shopper study of publicly listed FQHCs and CMHCs to examine adolescents’ access to outpatient mental health services from a family’s perspective. Mystery shopper studies, also known as secret shopper, audit, or simulated client studies, are methodologically robust, innovative, and policy relevant, because they capture the experiences of families seeking care (22–24). This mystery shopper study employed a deceptive design in which one of three trained voice actresses used a script to call each health center twice to attempt to schedule an appointment, such that the calls varied only in terms of predictor variables (insurance type and race-ethnicity of the care seeker). This approach reduces limitations that are common in traditional surveys, such as social desirability, nonrepresentative samples, recall bias, and nonresponse bias (25). The Crown Family School of Social Work, Policy, and Practice and Chapin Hall Institutional Review Board at the University of Chicago approved the study with a waiver for informed consent.

Study Population and Sample Selection

The 2020 Substance Abuse and Mental Health Services Administration (SAMHSA) treatment locator (https://findtreatment.gov), a publicly available mental health search tool, was used to define the sampling frame. The SAMHSA treatment locator is populated annually by the National Mental Health Services Survey (26), which collects information from all known facilities in the United States that provide services to people with psychiatric disorders. As described elsewhere (27), the principal investigator (D.R.A.) undertook a multistage process to ensure the completeness and accuracy of the sample. First, she validated the initial sample (N=357) against three additional national- and local-level lists of mental health centers (17, 28). Then she called all health centers between March and April 2021 and administered a 5-minute survey to ensure eligibility. Health centers were considered eligible if they offered outpatient psychosocial therapy to adolescents (14–18 years old), accepted at least one Medicaid managed care plan, and were located in Cook County, Illinois. (More information on excluded health centers and reasons for exclusion is available in the online supplement to this article.) After removing ineligible and closed health centers, the final sample consisted of 229 health centers, including 119 FQHCs and 110 CMHCs.

Data Collection

Self-identified White, Latina, and Black voice actresses (“callers” hereafter) posed as mothers. The callers participated in a 4-hour training session before study initiation. During the training, they piloted scripts and selected pseudonyms aligned with their own racial and ethnic backgrounds (see the online supplement). The callers piloted the scripts with nine health centers (three per caller) that were not in the sample. Scripts were iteratively adjusted after pilot call completion to ensure that questions and responses were as close to real life as possible.

Calls were made in two waves of data collection during the COVID-19 pandemic. Wave 1 occurred from mid-May to mid-June 2021 (spring); wave 2 occurred from mid-June to mid-July 2021 (summer). Callers used a standardized script to simulate a mother seeking therapy for her adolescent child who had recently witnessed a traumatic event and was experiencing symptoms of posttraumatic stress disorder (PTSD). PTSD warrants an urgent appointment with a mental health specialist, and practice parameters recommend evidence-based psychosocial interventions (29).

Two different scripts simulated prospective clients with identical clinical histories but different insurance types (Medicaid or private insurance) and varying only in racially aligned pseudonyms selected by the voice actress (scripts are available in the online supplement). The scripts included minor variations in the client’s addresses and dates of birth to ensure that callers were not recognized. A series of standardized “workarounds” to the script was developed to address potential questions that the caller could not answer (e.g., insurance identification number; see the online supplement).

Callers made up to five separate call attempts to their assigned health center to schedule an appointment after some initial contact with the center (e.g., the caller spoke to a receptionist but was waiting to hear back from the scheduler). If the caller could not talk with anyone at the health center (e.g., the call went straight to an answering machine), only three separate call attempts were made. The callers contacted the health centers in a random order, with 4 weeks of separation between wave 1 and wave 2 calls. Appointments were made and then canceled at the end of the call, enabling a comparison of appointment rates and wait times by using paired calls (i.e., spring vs. summer, Medicaid vs. private insurance, CMHC vs. FQHC) from the same caller to the assigned health center.

Study Variables

The primary outcomes were the ability to schedule a mental health appointment (no, yes) and wait time for an initial appointment (in days). Secondary outcomes included the availability of trauma-informed care (no, yes, or do not know) and type of trauma-specific treatments offered by health centers (as reported by schedulers). The primary independent variables were insurance type (Medicaid or private insurance) and the caller’s race-ethnicity (White, Latina, or Black). CountyCare was reported as the Medicaid managed care organization, and Blue Cross–Blue Shield was the private insurance type, because these plans hold the largest market share for their insurance type in Cook County (30, 31). Race-ethnicity was represented by the callers’ own tone of voice and pseudonym. This approach is the standard for measuring the effect of race-ethnicity in mystery shopper studies that use telephone or e-mail to request services (9, 11). If an appointment was offered, the caller documented the wait time in days from the date of the call. If an appointment was not offered, the caller recorded the reason. Offers to be put on an indeterminate waitlist were defined as “no appointment offered.” When a health center required additional steps before the center staff would schedule an appointment (e.g., referral required), callers recorded these occurrences as “no appointment offered,” because it was unclear how many families would follow through and accomplish the required administrative tasks.

When attempting to schedule an appointment, callers requested “trauma treatment,” noting that their child’s school social worker had recommended this treatment. If schedulers affirmed that the health center offered trauma treatment, the caller asked what types of treatment were offered.

Statistical Analysis

Analyses were conducted with Stata, version 17.0. Descriptive results were calculated for appointment availability and wait time, stratified by the insurance and race-ethnicity of the caller, as well as the availability of trauma-specific treatment. Chi-square and t tests were conducted to determine the relationship between various independent and dependent variables, including the ability to schedule an appointment and length of wait times. A generalized linear model was used to analyze how specific factors (i.e., insurance type, season when call was made, organization type, and race-ethnicity of the caller) affected access outcomes (appointment availability and wait times).

RESULTS

A cohort flow diagram is available in the online supplement. From the initial sampling frame of 229 safety-net health centers, seven (3%) were excluded from wave 2, after it was discovered in wave 1 that these health centers no longer offered outpatient mental health services to youths. After three call attempts in each wave, callers could not reach anyone at 32 health centers (14%) in the sampling time frame. Of the remaining health centers (N=190), callers could reach 110 (48% of the initial sample) in both waves and 80 (35% of the initial sample) in only one of the waves (Table 1). On average, reaching an appointment scheduler took a mean±SD of 1.8±1.2 calls.

TABLE 1.

Outcomes of attempted contacts with 229 safety-net health centers, by center and client characteristics^a

	Excluded from wave 2 sample (N=7, 3%)		No successful contact (N=32, 14%)		One completed contact (N=80, 35%)		Two completed contacts (N=110, 48%)
Characteristic	N	%	N	%	N	%	N	%
Center type
Federally qualified health center (N=119, 52%)	4	3	16	13	40	34	59	50
Community mental health center (N=110, 48%)	3	3	16	15	40	36	51	46
Race-ethnicity of callerb
White	5	7	1	1	9	12	61	80
Black	0	—	13	17	39	51	25	33
Latina	2	3	18	24	32	42	24	32

^aPercentages are row percentages.

^bThe White caller attempted contact with 76 (33%), the Black caller with 77 (34%), and the Latina caller with 76 (33%) health centers.

Appointment Availability and Wait Times

Across two waves of data collection, 451 contacts were attempted (i.e., calls to 229 centers in wave 1 and 222 in wave 2). Regardless of the insurance type or race of the caller, an appointment was offered in fewer than one out of five contacts (N=78, 17%). For almost half of the attempted contacts (N=222, 49%), appointments were not offered. In about one-third of attempted contacts (N=144, 32%), the caller could not speak to the scheduler after at least three call attempts. Importantly, the number of completed contacts (contacts where the scheduler called the family back to schedule an appointment or explain why an appointment could not be scheduled) varied by race-ethnicity of the caller. For the White caller, 131 of 147 (89%) attempted contacts resulted in a completed contact, compared with only 89 of 154 attempted contacts (58%) for the Black caller and 80 of 150 (53%) for the Latina caller (Table 1).

After removing the incomplete contacts (N=144) and ineligible health centers (N=7), we assessed appointment access by insurance type. An appointment was offered in 44 of 160 (28%) completed contacts from Medicaid callers and 34 of 140 (24%) from private insurance callers (Table 2). Among the 110 health centers reached by both the Medicaid and privately insured callers (i.e., paired calls), 9% (N=10) offered appointments to both, 59% (N=65) to neither, 15% (N=17) to the Medicaid-insured but not privately insured caller, and 16% (N=18) to the privately insured but not Medicaid-insured caller.

TABLE 2.

Characteristics of completed contacts to health centers, by insurance type

	Medicaid (N=160, 53%)		Private insurance (N=140, 47%)		All completed contacts (N=300)
Characteristic	N	%	N	%	N	%	p
Organization type							.449
Federally qualified health center	81	51	77	55	158	53
Community mental health center	79	50	63	45	142	47
Season							.361
Spring	83	52	80	57	163	54
Summer	77	49	60	43	137	46
Race-ethnicity of caller							.253
White	66	41	65	46	131	44
Black	54	34	35	25	89	30
Latina	40	25	40	29	80	26
Appointment status							.527
Appointment not offered	116	72	106	76	222	74
Appointment offered	44	28	34	24	78	26
Wait time (M±SD days)	13.6±12.2		9.5±8.2		11.8±10.8		.092

For offered appointments (N=78), the mean wait time, regardless of insurance type, was 11.8±10.8 days (Table 2). On average, Medicaid-insured callers had to wait 13.6 days for an appointment, whereas privately insured adolescents had to wait 9.5 days, a difference that was not statistically significant (Table 2). We observed a statistically significant difference in wait times between organization types, with a mean wait time of 14.2 days for FQHCs and 8.4 days for CMHCs (t=2.40, df=38, p=0.019). On average, the White caller had to wait 11.0 days for an appointment, whereas wait times were 9.9 days for the Black caller and 16.3 days for the Latina caller, a difference that was not statistically significant.

The most frequent reasons appointments were not offered were the health center requiring the family to switch their primary care provider (PCP) into their network (32%), having to put the family on a waitlist (22%), or closure of the health center’s waitlist (17%) (Table 3). We found a significant association between reasons for appointment denials and organization type (χ²=109.06, df=5, p<0.001). FQHCs required that families switch their PCP into their network (62%) and rarely put families on waitlists (5%). In contrast, a greater proportion of CMHCs put families on waitlists (39%) or reported their waitlists as closed (28%). Furthermore, we noted a statistically significant association between reasons for appointment denials and the insurance type of the caller (χ²=13.64, df=5, p<0.05), with a greater proportion of Medicaid callers (38%) required to switch their PCP, compared with privately insured callers (26%). We found a significant association between reasons for appointment denials and the race-ethnicity of the caller (χ²=68.52, df=10, p<0.001) (Table 4).

TABLE 3.

Reasons provided by scheduler for not offering an appointment (N=222)^a

	Organization type				Insurance type				Race-ethnicity of caller
	CMHC (N=110, 50%)		FQHC (N=112, 50%)		Medicaid (N=116, 52%)		Private (N=106, 48%)		White (N=90, 41%)		Black (N=69, 31%)		Latina (N=63, 28%)		Total (N=222)
Primary reason	N	%	N	%	N	%	N	%	N	%	N	%	N	%	N	%
In-network primary care physician needed	2	2	69	62	44	38	27	25	28	31	30	43	13	21	71	32
Put on waitlist	43	39	6	5	28	24	21	20	27	30	6	9	16	25	49	22
Waitlist closed	31	28	7	6	18	16	20	19	16	18	4	6	18	29	38	17
Insurance plan number needed	13	12	14	13	14	12	13	12	0	—	23	33	4	6	27	12
Insurance type not accepted	11	10	4	4	2	2	13	12	9	10	1	1	5	8	15	7
Need referral	4	4	10	9	5	4	9	8	9	10	1	1	4	6	14	6
Other (e.g., SSN needed)	6	5	2	2	5	4	3	3	1	1	4	6	3	5	8	4

^aCMHC, community mental health center; FQHC, federally qualified health center; SSN, Social Security number.

TABLE 4.

Association of caller and health center characteristics with likelihood of appointment denial, all completed contacts (N=300)^a

Characteristic	IRR	SE	95% CI	p
Medicaid (reference: private insurance)	.93	.063	.82–1.07	.311
Wave 2 (reference: wave 1)	1.19	.080	1.04–1.36	.010
FQHC (reference: CMHC)	1.11	.074	.97–1.26	.136
Black or Latina caller (reference: White)	1.18	.085	1.03–1.36	.019

^aCMHC, community mental health center; FQHC, federally qualified health center; IRR, incidence rate ratio.

Table 4 shows results from the generalized linear model, indicating that neither insurance nor organization type significantly predicted callers’ ability to schedule an appointment. Calls made in the summer were less likely to result in an appointment than calls made in the spring (incidence rate ratio [IRR]=1.19, p=0.010). The Black and Latina callers were less likely to be offered an appointment than the caller who identified as White (IRR=1.18, p=0.019).

Availability of Trauma-Specific Treatment

Regardless of whether an appointment was offered, the caller asked the scheduler whether trauma-informed care was available and, if so, what trauma treatments were offered. In 38% of completed contacts (N=115 of 300), schedulers reported that their health center offered trauma-informed care; the same proportion (N=115, 38%) reported that they were unsure whether their health center offered such care. In approximately 10% of calls (N=31), schedulers reported that their health center did not offer trauma-informed care, and in 13% of calls (N=39), the schedulers could not be asked whether they offered trauma treatment because the call ended before the question could be asked. When schedulers noted that they were unsure about whether trauma treatment was offered, callers requested that someone else at the health center call them back with the information. Only five of 115 requests (4%) resulted in a callback to provide additional information on the trauma-specific services offered.

Among the completed contacts during which schedulers reported that their health center offered trauma-informed care, in 39% contacts (N=45, 15% of all completed contacts) the scheduler could name a specific trauma treatment. Most commonly, schedulers reported that cognitive-behavioral therapy (CBT) was available (N=24), followed by trauma-focused CBT (TF-CBT; N=6), eye movement desensitization and reprocessing (EMDR; N=5), dialectical behavior therapy (N=4), and play therapy (N=2).

DISCUSSION

Using a mystery shopper method, we found that a mental health appointment for an adolescent who needed trauma-informed care was offered in only 17% of contacts with CMHCs and FQHCs. When callers could schedule an appointment, the average wait time was about 12 days, regardless of the caller’s insurance type. Our results highlight barriers to care access in the mental health system, even when a family is insured. Contrary to previous research that found disparities in access between individuals enrolled in Medicaid and private insurance (32, 33), in our study, insurance type did not predict access to mental health care at safety-net health centers.

In our study, reasons for appointment denials varied by organization type. Most FQHCs did not offer appointments because of a requirement that the family switch their PCP to their network before scheduling. A similar finding was reported in a mystery shopper study of medical clinics in Illinois conducted in 2019 as a part of an external quality review (34). The report noted that requiring families to designate their PCP with the FQHC through insurance before scheduling an appointment acted as a “barrier to all Medicaid enrollees trying to schedule appointments” (34). Although this task is likely meant to integrate general medical and mental health care, it could pose an unintended barrier for families seeking mental health treatment. More research is needed to understand why FQHCs implement this requirement before scheduling.

For CMHCs, waitlists or closed waitlists, indicating a lack of capacity to take on new clients, were the primary reasons for not offering appointments. Compared with FQHCs, CMHCs may have less capacity to hire the staff needed to fill demand, given two important organizational differences. First, FQHCs receive an enhanced rate for accepting Medicaid clients, potentially increasing their operating budget (35). Second, one report by the consulting firm Milliman found that PCPs were reimbursed 23.8% higher than mental health professionals (36), perhaps leading to higher operating budgets for FQHCs because these centers are reimbursed for providing more primary care services than are provided by CMHCs, which provide primarily specialty mental health services, resulting in lower reimbursement rates for CMHCs.

Callers were less likely to be offered an appointment in the summer versus the spring. A mystery shopper study in New York State reported similar results, where “the ability to schedule an intake for a psychiatry appointment was 4.1 times higher in the spring than summer” (37). Safety-net health centers often depend on student therapists who end their practicums when summer begins, potentially affecting the centers’ capacity to see new clients.

Of concern, the caller’s race-ethnicity significantly predicted the ability to schedule an appointment. Whereas the White caller had a rate of 89% completed contacts with health centers, the corresponding rates for Black and Latina callers were only 58% and 53%, respectively, indicating that schedulers may be less likely to return calls to individuals whose tone of voice or name suggests a racial-ethnic minority group. Furthermore, the Black and Latina callers were significantly less likely to be offered an appointment than the White caller. These results add to the growing literature indicating that discrimination may occur at mental health care scheduling, with schedulers having a bias toward being more likely to call back and schedule individuals with a White-sounding name and voice (9, 10, 38). Discrimination at the point of scheduling is likely an important factor for significant racial-ethnic disparities in mental health access outcomes (6). It is vital that safety-net health centers have an antidiscrimination policy in place and regularly audit whether employees are offering equitable services to everyone.

Finally, this study uncovered deficiencies in the availability of trauma-informed services. During 300 completed contact calls, only 38% of schedulers reported that their health center offered trauma-informed care; however, of those, only about 40% could name a specific type of trauma treatment that their health center offered. For the specific treatments named, only 11 schedulers reported that their health center offered evidence-based treatments for trauma (i.e., TF-CBT and EMDR [29, 39]). When a scheduler was unsure whether their health center offered trauma therapy, the caller requested a return call. Yet, only five of 115 requests received a callback providing more information. These findings indicate that schedulers have gaps in knowledge of the mental health services offered by their health center and that health centers lack the resources to return calls. Both are issues for parents seeking to be informed consumers of mental health services.

The limitations of this study suggest areas for future research. We could not make conclusions about the impact of race-ethnicity on access to mental health care because only one caller each represented one race-ethnicity. Future studies may employ multiple callers of each race-ethnicity to control for individual callers’ abilities in navigating calls. Notably, demand for mental health services increased during the COVID-19 pandemic when this study was conducted (40). Future research is needed to determine how generalizable the findings of this study are postpandemic. Finally, this study could not determine the effectiveness of mental health services offered by health centers because we did not measure treatment fidelity.

CONCLUSIONS

Fewer than one in five contacts with safety-net health centers resulted in a scheduled mental health appointment, regardless of the caller’s insurance type. We identified significant care capacity and administrative barriers associated with reduced access to urgently needed youth mental health services. Our results also raise the concern that racial-ethnic discrimination may occur during scheduling. Of note, it is concerning that fewer than half of health centers reported offering trauma-informed care and that in only 15% of completed calls, schedulers could name a specific type of trauma treatment that their center offered. Our results add to research calling for greater financial investments in mental health resources, particularly in safety-net systems of care (41, 42).

HIGHLIGHTS.

• Callers who contacted safety-net mental health centers posing as mothers of an adolescent with trauma symptoms could schedule a mental health appointment in fewer than one in five contacts.

• The primary reasons for appointment denial were capacity constraints (i.e., waitlisting or waitlists closed) at community mental health centers (67%) and administrative requirements to switch to in-network primary care providers at federally qualified health centers (62%).

• The results raise concern that discrimination on the basis of race or ethnicity may be occurring at scheduling, because Black and Latina callers were significantly less likely than a White caller to be offered an appointment.