Short Report: A Quantitative Methodological Review of Participant Characteristics in The Literature Testing Mental Health Interventions for Youth with ASD

Abstract

There have been significant efforts to develop, adapt, and test interventions targeting the co-occurring mental health conditions common among autistic youth, with several extant narrative and systematic reviews demonstrating the efficacy of these interventions. Yet, a methodological analysis of the literature testing these interventions, particularly the characteristics of the participant samples comprising this evidence base, is needed to contextualize our current understanding of treatment effects and highlight current gaps in the current evidence base to inform future research. This systematic quantitative methodology review characterized participants included in the mental health treatment research for autistic youth. One hundred and thirty-one articles testing mental health interventions for ASD were coded for youth, caregiver, and provider participant demographics. Findings indicated limited representation of females, transition age youth, racial/ethnic minority groups, and youth diverse in cognitive functioning and co-occurring mental health symptoms or conditions. Limited inclusion of providers representative of the clinical workforce was also noted. These results illuminate several critical gaps in the current evidence-base for mental health interventions for autistic youth and provide recommendations for future research directions.

Autistic children and adolescents (henceforth referred to as youth) experience high rates of co-occurring mental health conditions, with most experiencing multiple co-occurring conditions (Lai et al., 2019). The presence of co-occurring mental health conditions confers vulnerability to greater functional impairment and worse outcomes across the lifespan (Cadman et al., 2012; Factor et al., 2017). There has been a prolific growth in research developing, adapting, and testing mental health interventions for ASD over the past two decades, resulting in numerous evidence-based interventions (EBIs) targeting co-occurring mental health problems in autistic youth (Dickson et al., 2021). Several recent narrative and systematic reviews denote the beneficial effects of these EBIs (Keefer et al., 2018; Weston et al., 2016). However, a significant gap in this evidence base is a review of the nature of the literature comprising the evidence base referred to as a quantitative methodological review (Weisz et al., 2005).

Quantitative methodological reviews examine features of the broader literature such as the characteristics and clinical representativeness of study samples in order to complement and contextualize our understanding of treatment effects (Weisz et al., 2005). Such reviews are necessary given the limited reporting of sample characteristics and clinical representativeness of study participants (i.e., extent to which trial participants reflect those receiving or providing services in clinical settings) common in intervention trials; this includes the underrepresentation of participants from diverse backgrounds (DeCarlo Santiago & Miranda, 2014; Weisz et al., 2005). This limited representation is immensely concerning given the implications for the generalizability and external validity of these interventions and role in perpetuating care disparities for vulnerable populations. Given the documented health disparities in ASD (Bishop-Fitzpatrick et al., 2017) as well as increasing prioritization and research efforts to improve implementation of mental health EBIs for autistic youth (Interagency Autism Coordinating Committee [IACC], 2019), evaluation of the evidence base for mental health interventions for ASD is needed. The aim of this systematic quantitative methodology review is to characterize participants included in the literature testing mental health interventions for autistic youth in service of identifying gaps in the current evidence base and informing future directions for research.

Methods

The current project was part of a larger systematic review characterizing interventions tested for co-occurring mental health conditions in autistic youth, including necessary adaptations for use with this population (see Dickson et al., 2021). No community is involved in this study.

Search Strategy

We conducted a systematic search, completed on 30 June 2020, and an additional search, completed 28 July 2021, of the PubMed, PsycINFO, Scopus, CINAHL Plus, and Web of Science electronic databases. See Table 1 for specific search strategy used. Backward (reference list) and forward (citation) searches were also conducted with recent relevant reviews and seminal articles in this area. Relevant articles identified were included in data collection and extraction.

Table 1.

Search Strategy

Search String

1. Autis* OR Asperger* AND 2. “Mental health” OR internaliz* OR externaliz* OR “behavior problems” OR anxiety OR depression OR psychosis OR ADHD OR “Attention deficit” OR “oppositional defiant disorder” OR Conduct disorder OR “disruptive behavior disorder” OR Trauma OR PTSD OR “Post-traumatic stress” AND 3. Intervention OR psychotherap* OR “parent training” OR “Cognitive behavior*” or Mindfulness or “Parent Child Interaction Therapy” OR “Parent-Child Interaction Therapy” OR PCIT or “Positive Parenting Program” OR “Triple P” OR “Stepping Stones” OR “Incredible Years” OR “AIM HI”

Inclusion and Exclusion Criteria

Articles were screened based on the following inclusion criteria: (1) a psychosocial intervention tested via experimental design; (2) studies with a sample size greater than 10 participants (to exclude single subjects or case study designs); (3) an intervention with a primary focus on mental health disorders or symptoms and/or primary presenting problems for mental health services¹; (4) an intervention delivered to autistic youth and co-occurring mental health conditions or symptoms and/or their families; (5) youth participants aged birth to 25 years old. We excluded studies if they included: (1) a non-experimental design and/or (2) focused on interventions targeting non-mental health symptoms (e.g., social skills, sensory sensitivities, core symptoms of ASD) as the primary intervention target or outcome.

Data Extraction and Synthesis

Five trained master’s or bachelor’s level reviewers completed initial title and abstract screening, with two reviewers independently assessing each article abstract. Discrepancies were resolved by a third reviewer (first author). We applied a codebook to extract data pertaining to participant (child, caregiver, provider) characteristics. Two trained coders independently reviewed and extracted data from each article, with discrepancies resolved via consensus meetings.

Results

Our search identified 10,579 records (Figure. 1). After removal of duplicates, 7,476 articles were screened of which 457 initially met the inclusion criteria. Multiple articles reporting on participants from within the same larger project or dataset were identified and removed so that we only included the primary or initial article reporting on outcomes, resulting in a total of 131 unique research projects (see Additional file 1). Full participant characteristics are included in Table 2.

Figure 1.

PRISMA Diagram

Table 2.

Participant Characteristics

Child Factors	N (%)
Race/Ethnicity (Indicated)	71 (54.20)
Caucasian	65 (91.55)
Hispanic/Latinx	31 (43.66)
Asian/Pacific Islander	35 (49.30)
African American/Black	35 (49.30)
Native American	7 (9.86)
Multiethnic/Multiracial	21 (29.58)
Other	26 (36.62)
% Caucasian (Mean)	74.85%
Gender (Indicated)	125 (95.42)
Included ONLY male	6 (4.58)
Included male and female	119 (90.84)
% Female (Mean)	21.12%
Age Groups Included (Indicated; years:months)	128 (97.71)
<2:11	15 (11.72)
3–5:11	52 (40.63)
6–11:11	109 (85.16)
12–13:11	83 (64.84)
14–18:11	55 (42.97)
19–25	11 (8.59)
More than one age group	118 (92.19)
Co-Occurring MH Condition (Indicated)	93 (70.99)
Internalizing	61 (65.59)
Externalizing	51 (54.84)
Trauma	6 (6.45)
ADHD	38 (40.86)
Psychosis	3 (3.23)
OCD	14 (15.05)
Other	10 (10.75)
Presence of +1 co-occurring mental health condition	37 (39.78)
Cognitive Abilities (Indicated)	92 (70.23)
High Cognitive Abilities Required	64 (69.57)
ASD Severity (Indicated)	37 (28.24)
Parent/Caregiver Factors	N (%)
Race/Ethnicity (Indicated)	14 (10.69)
Caucasian	12 (85.71)
Hispanic/Latinx	3 (21.43)
Asian/Pacific Islander	6 (42.86)
African American	8 (57.14)
Native American	3 (21.43)
Multiethnic/Multiracial	3 (21.43)
Other	2 (14.29)
% Caucasian (Mean)	72.09%
Gender (Indicated)	64 (48.85)
Included ONLY female	12 (18.75)
% Female (Mean)	78.34%
Provider Factors	N (%)
Discipline (Indicated)	107 (81.68)
Psychology	87 (81.31)
Social Work	12 (11.21)
Marriage/Family Therapy	3 (2.80)
Psychiatry	9 (8.41)
Counseling	10 (9.35)
Education	17 (15.89)
Paraprofessionals	1 (.93)
Other	30 (28.04)
Education (Indicated)	85 (64.89)
Graduate school/Trainees	48 (56.47)
PhD/MD	57 (67.06)
Master’s level degree	30 (35.29)
Bachelor’s degree	8 (9.41)

Note: Total Articles = 131

Child Demographic Characteristics

Most studies (95.4%) indicated the gender of participants, including both male and female participants. Across these studies, an average of 21.1% of participants were female. No studies specified other gender identities. Almost all studies (97.7%) indicated the age group of participating youth. Most studies included more than one age group, mainly those of middle childhood or early adolescent ages; few (8.6%) included transition age participants (i.e., ages 19–25). Half of the studies (54.2%) specified the race/ethnicity of participating youth. Of these, half (49.3%), or 26.7% overall, included participants identified as non-Caucasian. While multiple racial/ethnic groups were represented, the average percentage of participants identifying as Caucasian across studies was 74.9% (range=22%−100%).

Child Clinical Characteristics

Most studies (71.0 %) indicated whether participants had an identified co-occurring mental health problem or symptoms, most frequently internalizing (e.g., anxiety, depression) followed by externalizing and/or attentional (e.g., disruptive behavior, attention-deficit, hyperactivity) problems. Half of these studies (48.9%) specified having an inclusionary or exclusionary criterion related to the presence of multiple co-occurring mental health conditions or problems; of those, 3.1% excluded youth with more than one co-occurring condition from participation and 65.6% excluded youth with specific problems or conditions (e.g., psychosis, aggression). The remaining third (31.3%) indicated including youth participants with more than one co-occurring condition. Most studies (70.5%) specified the cognitive abilities of participating youth. Of these, a third (32.6%) included individuals with lower cognitive abilities (e.g., IQ scores below 70). Importantly, most studies specifying cognitive abilities (71.8%) reported participant inclusion and/or exclusion criteria related to child cognitive abilities, with most of these studies (68.1%) requiring participants to have higher cognitive abilities. Finally, a quarter (28.2%) of studies indicated the ASD symptom severity level of participating youth. Examination of these studies indicated immense variability in specific measures used as indicators of ASD symptom severity, thereby limiting comparison across studies.

Caregiver Demographic Characteristics

Approximately half of studies (48.9%) included demographic information for caregivers, mainly caregiver gender. While most studies (81.3%) included both male and female caregivers, the overall average percentage of female caregivers was 78.3% (range=14%−100%). Only 10.7% of studies indicated the race/ethnicity of caregivers. Similar to our child-level findings, the average percentage of caregivers identifying as Caucasian was 72.1% (range=24%−96%).

Provider Professional Characteristics

Most studies (81.8%) provided background information of providers, namely the discipline and education of providers. Most providers were in the field of Psychology, with few from Social Work, Counseling, and Marriage/Family Therapy disciplines. In terms of education level, most studies (67.1%) indicated that providers had a PhD or MD, half included graduate school students/trainees (56.5%), and less than half included providers with a terminal master’s or bachelor’s degree (35.3%).

Discussion

We conducted a systematic methodological review of participant characteristics in the literature testing mental health interventions to treat co-occurring mental health problems in autistic youth. Results from this review identified several key gaps in the current evidence base regarding mental health interventions for ASD and indicate a need for further efforts to expand the clinical representativeness of participants included in future work.

In terms of child demographic characteristics, our results indicated that in contrast to school-aged or middle school-aged youth, few studies tested mental health interventions for transition aged autistic youth. This is consistent with the broader gaps surrounding treatment and services research focused on transition aged autistic youth and highlights the need for further work in this area, especially given the high, and often increasing in the case of some conditions, rates of mental health problems among this age group (Baker-Ericzén et al., 2018; Hume et al., 2021; Lai et al., 2019). Additionally, while the majority of studies included at least some female participants, inclusion of female participants was limited overall. Despite fitting the gender demographics of ASD, this underscores gaps related to our understanding of how mental health interventions are utilized and their effect with youth who identify as female. This may be especially important in light of work noting the unique care experiences of autistic females as compared to autistic males, including higher rates of unmet mental health service needs and barriers to care (Tint & Weiss, 2018).

Our results indicating strikingly low representation of racial or ethnic minorities are also concerning, especially in light of data suggesting individuals from racial/ethnic minority groups represent as much as 60% of autistic youth receiving community-based mental health services (Brookman-Frazee et al., 2017). These results are unfortunately not surprising given similar underrepresentation of minorities in study samples within the intervention literature examining EBIs specific for ASD as well as mental health EBIs more broadly (DeCarlo Santiago & Miranda, 2014; Hume et al., 2021; Weisz et al., 2005). However, these results underscore a critical gap in our current evidence base regarding the effect of these EBIs for youth from racial or ethnically diverse backgrounds.

Turning to child clinical factors, results suggested that few studies included participants with lower cognitive abilities. Despite the wide range of cognitive abilities of autistic individuals, those with lower cognitive abilities have often been excluded from interventions targeting mental health concerns. While many studies noted lower cognitive abilities as a barrier to participation in mental health treatment (e.g., cognitive behavioral therapy), many of these individuals likely require as well as present for mental health services, underscoring the need to better understand the effect of mental health EBIs for these individuals. Results also highlight gaps regarding the characteristics of sample participants, with many studies excluding youth with specific behaviors or mental health symptoms (e.g., aggression) and/or multiple co-occurring conditions. Similar to the methodological review of mental health intervention trials conducted by Weisz and colleagues (2005), these clinical characteristics raise concerns regarding the representativeness of the youth included within trial samples. Specifically, these youth likely do not represent the majority of autistic youth with co-occurring mental health conditions presenting for community mental health services, who largely present with behaviors that challenge (e.g., aggression, noncompliance) and meet criteria for more than one co-occurring condition (Brookman-Frazee et al., 2012, 2017).

Our results indicating limited inclusion of non-female caregivers and those from racial/ethnic minority groups also raise concerns regarding the validity and effects of these interventions for the broader diverse sample of caregivers typical in community mental health settings. Given the critical role of caregivers in autism services and youth mental health services more generally, future work examining the effects of these interventions among a diverse range of caregivers is critical.

Finally, our results pertaining to provider professional characteristics indicated a high representation of graduate students or trainees, those with doctoral degrees, and individuals from the field of psychology. When compared to the typical demographics of practicing clinicians, there are discrepancies with provider participants in these trials and the characteristics of the mental health workforce in community settings, who are largely master’s level clinicians or trainees from non-psychology backgrounds. Further, many participant trainees received clinical supervision from experts and/or intervention developers as part of these trials, which is not typical in routine care. Again, these discrepancies parallel findings from Weisz and colleagues (2005).

Overall, our results illuminate several critical gaps in our current evidence base regarding interventions targeting co-occurring mental health conditions for autistic youth, including the paucity of research focused on transition age youth, autistic females, and limited clinical representativeness of provider, youth, and caregiver participants. The results of this review also highlight several key considerations for future research aiming to expand the current evidence base regarding mental health interventions for autistic youth. This includes a targeted focus on examining the effects of these interventions with diverse and clinically representative youth, especially transition aged and female youth, when delivered by providers typical of the clinical workforce providing community-based mental health services. Addressing these gaps represents a key next step towards improving the delivery of effective, equitable mental healthcare for autistic youth.