A comparative effectiveness trial of functional behavioral assessment methods

Abstract

Clinicians report primarily using functional behavioral assessment (FBA) methods that do not include functional analyses. However, studies examining the correspondence between functional analyses and other types of FBAs have produced inconsistent results. In addition, although functional analyses are considered the gold standard, their contribution toward successful treatment compared with other FBA methods remains unclear. This comparative effectiveness study, conducted with 57 young children with autism spectrum disorder, evaluated the results of FBAs that did (n = 26) and did not (n = 31) include a functional analysis. Results of FBAs with and without functional analyses showed modest correspondence. All participants who completed functional communication training achieved successful outcomes regardless of the type of FBA conducted.

Behavioral interventions have the potential to significantly reduce challenging behavior exhibited by some neurodivergent individuals (Call et al., 2015, 2017; Kurtz et al., 2015; Rooker et al., 2013). However, they are much more likely to do so when the components of those interventions are selected based on the function of the individual’s challenging behavior (Campbell, 2003; Heyvaert et al., 2010, 2014; Hurl et al., 2016). Functional analysis (Iwata et al., 1982/1994) is well established as the gold-standard methodology for identifying the function of challenging behavior (Beavers et al., 2013; Hanley et al., 2003; Saini et al., 2020; Schlinger & Normand, 2013; Tiger & Effertz, 2021). Among several reasons why functional analyses are the benchmark against which other functional behavioral assessments (FBAs) are compared is that it is the only method that can demonstrate a causal relation between challenging behavior and reinforcing consequences.

Despite evidence that a treatment’s efficacy improves when based on the results of a functional analysis (e.g., Herzinger & Campbell, 2007), several barriers hinder consumers’ access to this optimal format of FBA. For example, the need to ensure the safety of therapists and clients can preclude the possibility of conducting functional analyses of certain dangerous topographies of behavior because they involve intentionally evoking it (Iwata & Dozier, 2008). It also might not be possible to conduct functional analyses in settings that do not allow for controlling critical variables. For instance, when conducting an FBA in a client’s home, it is not always feasible to limit the presence of other individuals or the availability of certain items, which can confound the results of a functional analysis (Lerman et al., 2020).

Given these barriers, it is unsurprising that a minority of Board Certified Behavior Analysts (BCBAs) report using functional analyses regularly, with the majority instead relying upon less rigorous FBA methods (Ellingson et al., 1999; Oliver et al., 2015; Petursdottir et al., 2010; Roscoe et al., 2015; Saini & Cox, 2020). Oliver et al. (2015) found that 75.2% of BCBAs reported always or almost always using indirect assessments to identify the function of challenging behavior and an even larger majority (94.7%) reported relying on descriptive assessments. In contrast, only 34.8% of BCBAs reported always or almost always using functional analyses. Yet, FBA methods that do not include a functional analysis have notable limitations. For example, indirect assessments indicate possible functions without the clinician directly observing the targeted challenging behavior, typically by gathering information from caregivers. However, the few systematic evaluations of the psychometric properties of indirect assessments have generally shown only modest reliability and validity (e.g., Iwata et al., 2013; Paclawskyj et al., 2001). Although descriptive assessments involve direct observation of challenging behavior and the environmental events that naturally co-occur, relevant antecedents or consequences may not occur during the observation. Descriptive assessments are also limited to demonstrating only correlations between challenging behavior and antecedents or consequences.

Comparisons of results from different FBA methods have produced inconsistent findings regarding correspondence. Although a few studies have found that results from indirect and descriptive assessments correspond with those from functional analyses (e.g., Arndorfer et al., 1994; Borrero et al., 2016; Newcomer & Lewis, 2004; Wasano et al., 2009), others have reported more equivocal findings or even a complete lack of correspondence (e.g., Alter et al., 2008; Contreras et al., 2023; Hall, 2005; Martens et al., 2019; Pence et al., 2009; Romani et al., 2023; Virues-Ortega et al., 2011). In addition, most studies evaluating the correspondence between the results of functional analyses and other forms of FBAs have included only a few participants (e.g., Arndorfer et al., 1994; Cunningham & O’Neill, 2000; Hall, 2005; Lerman & Iwata; 1993; Martens et al., 2019; Newcomer & Lewis, 2004; Pence et al., 2009). Many of those showing poor agreement also evaluated the correspondence between functional analyses and either indirect or descriptive assessments but not both (e.g., Contreras et al., 2023; Lerman & Iwata; 1993; Martens et al., 2019; Pence et al., 2009; Sasso et al., 1992). However, it is common practice for BCBAs to conduct both descriptive and indirect assessments for the same individual’s challenging behavior (Oliver et al., 2015; Petursdottir et al., 2010). Such an approach is consistent with some practice guidelines (e.g., Council of Autism Service Providers, 2020) and suggestions that combining indirect assessments and descriptive assessments may be more accurate than either alone (Floyd et al., 2005). Finally, most comparisons of functional analyses and other FBAs have evaluated outcomes of different assessment approaches in terms of whether they identified the same function for a given individual’s challenging behavior. Although agreement with the gold standard is one metric of the validity of other FBA methods, a more socially significant measure of an FBA’s clinical utility is the outcome of an intervention based on its results (Tiger & Effertz, 2021).

These lingering questions are perhaps best suited to examination via a comparative effectiveness study. This type of research seeks to improve care delivery or outcomes by comparing the benefits (or harms) of alternative methods for the prevention, diagnosis, treatment, or monitoring of a clinical condition (Sox, 2010). In contrast to efficacy studies, which often employ single-subject methods or group designs with waitlist or placebo controls, the clinical methods evaluated in comparative effectiveness studies have usually demonstrated at least preliminary efficacy and are often already being implemented in clinical practice.

This study sought to compare the effectiveness of the type of FBA that appears to be commonly implemented by providers in the field, comprising a combination of indirect and descriptive assessments, with those that included a functional analysis. Effectiveness was gauged in two ways: first, via correspondence between FBAs that differed only by the inclusion of a functional analysis and, second, by comparing the effectiveness of functional communication training (FCT; Carr & Durand, 1985; Tiger et al., 2008) based on the results of an FBA that did not include a functional analysis versus one that did. This randomized clinical trial included substantially more participants (n = 57) than have been previously included in prospective comparisons of FBA formats.

METHOD

Participants, setting, and materials

This study analyzed data from a subset of participants from a larger funded host project (R01MH104363) on telehealth-based delivery of FBAs and function-based treatments for socially mediated challenging behavior exhibited by young children diagnosed with autism spectrum disorder (ASD). Select data from the host project have been previously published in articles on matters encountered when delivering behavioral interventions via clinic-to-home telehealth (Lerman et al., 2020) and global applications of telehealth-based delivery of function-based treatments (Schieltz et al., 2022). The host project included participants between 18 and 83 months of age

1. with an ASD diagnosis based on the results of the Autism Diagnostic Observation Schedule (ADOS), Second Edition (Lord et al., 2013); a cognitive assessment (Differential Ability Scales, Second Edition [Elliot et al., 1990] or Mullen Scales of Early Learning [Mullen, 1995]); the Autism Diagnostic Interview–Revised (Rutter et al., 2003); and the Vineland-II assessment of adaptive behavior (Sparrow et al., 2005), each conducted and interpreted by a licensed psychologist;

2. whose caregivers expressed concerns regarding their child’s challenging behavior, as indicated by a score of 13 or greater on the Aberrant Behavior Checklist–Irritability subscale (ABC-I; Aman et al., 1985);

3. whose caregiver reported the absence of complicating medical concerns (e.g., uncontrolled seizures); and

4. for whom results of an initial, indirect screening assessment (described below) suggested that the challenging behavior was not automatically reinforced.

A false-negative outcome on the screener for automatic reinforcement (i.e., the screener indicated that target behavior was not maintained by automatic reinforcement, but subsequent functional assessment suggested it was) would have resulted in the dismissal of the participant from the study. This never occurred for any of the participants included in this data set. Participants with challenging behavior maintained by automatic reinforcement were excluded because challenging behavior maintained by social reinforcement can often be treated by the same general treatment (e.g., FCT), whereas challenging behavior maintained by automatic reinforcement often requires a different approach and produces different outcomes (LeBlanc et al., 2000; Verriden & Roscoe, 2019).

The current study included data for all participants who completed the procedures described below up to and including the start of treatment. As a result, any participants who dropped out of the study prior to beginning treatment were not included in this data set. Fifty-seven participants from the larger host project met these criteria. All were between 2 and 6.5 years old at the start of their participation. Most were male (73.7%), White (56.1%), and of non-Hispanic ethnicity (79.6%). Participants included in the current sample were somewhat younger than those in the larger host project (r_pb = −.19, p = .04). Participants in the current sample were otherwise comparable to those in the larger host project sample in the distribution of reported sex, χ²(1, N = 145) = 2.34, p = .13, phi = .13; race, χ²(1, N = 154) = .64, p = .42, phi = .06; ethnicity, χ²(1, N = 137) = 1.60, p = .45, phi = .11; and ADOS scores, r_pb = .14, n = 122, p = .12. A substantial majority of the primary caregivers who implemented study procedures were participants’ mothers (94.7%), most of whom were White (55.1%) and of non-Hispanic ethnicity (72.0%). See Table 1 for complete participant and caregiver demographics and Table 2 for participant age and standardized assessment score outcomes.

TABLE 1.

Demographics across groups for participants and caregivers in the current experiment

	Participant				Caregiver
	Non-FA group		FA group		Non-FA group		FA group
	n	%	n	%	n	%	n	%
Sex
Female	5	16.1	10	38.5	29	93.6	25	96.1
Male	26	83.9	16	61.6	2	6.5	1	3.9
Race 1
American Indian/ Alaska Native	0	0	1	4	0	0	1	5.3
Asian	1	3.3	2	8	0	0	1	5.3
Black	4	13.3	2	8	5	19.2	1	5.3
Hispanic	6	20	2	8	5	19.2	2	10.5
Native Hawaiian/ Pacific Islander	0	0	0	0	0	0	0	0
White	16	53.3	16	64	14	53.9	13	68.4
More than one race	3	10	2	8	2	7.7	1	5.3
Ethnicity 2
Hispanic	7	25	3	12	6	23.1	4	20
Not Hispanic	21	75	22	88	20	77	16	80

Note. Of the 57 participants who met all inclusion criteria for the current experiment, 31 were in the non-FA analysis group and 26 were in the FA group. Percentages are out of the total number of participants per group unless noted otherwise. Post-randomization analyses indicated no systematic association between group and race, χ²(1, N = 55) = .64, p = .43, phi = .11, or ethnicity, χ²(1, N = 53) = 1.46, p = .23, phi = .17. There were slightly more females in the FA (10 of 26, 38.5%) relative to the non-FA group (5 of 31, 16.1%), χ²(1, N = 57) = 3.64, p = .06, phi = .25. Non-FA = no functional analysis; FA = functional analysis.

¹Caregivers for 30 of 31 and 25 of 26 participants across the non-FA and FA groups, respectively, reported their child’s race. Only 26 of 31 and 19 of 26 caregivers across the non-FA and FA groups, respectively, reported their own race.

²Caregivers for 28 of 31 and 25 of 26 participants across the non-FA and FA groups, respectively, reported their child’s ethnicity. Only 26 of 31 and 20 of 26 caregivers across the non-FA and FA groups, respectively, reported their own ethnicity.

TABLE 2.

Age and standardized assessment scores across groups for participants in the current experiment

	Non-FA						FA
	n	%	M	Min	Max	SD	n	%	M	Min	Max	SD
Age (months) 1
0–47.9	13	54.2					10	43.5
48–59.9	4	16.7					7	30.4
60–84	7	29.2					6	26.1
Total			49.2	31	71	11.7			49.4	23	78	16.1
ADOS	24	77.4	16	6	28	5.3	20	76.9	17.2	8	25	3.9

Note. Of the 57 participants that met all inclusion criteria for the current experiment, 31 were in the non-FA group and 26 were in the FA group. Postrandomization analyses indicated no systematic association between experimental group and age, r_pb = .01, n = 50, p = .96, or ADOS score, r_pb = .14, n = 40, p = .41. ADOS = Autism Diagnostic Observation Schedule; Non-FA = no functional analysis; FA = functional analysis.

¹Only 24 of 31 and 23 of 26 caregivers across the non-FA and FA groups, respectively, reported their child’s age.

Participants were recruited from research sites in the state where they lived, with one site each located in the midwestern, southeastern, and southwestern regions of the United States. Lead investigators at each site ensured adherence to the procedures described below and met approximately biweekly via a conference call to coordinate consistent implementation across sites. Caregivers conducted all experimental procedures with coaching from master’s- or doctoral-level therapists via telehealth. Due to differences in each institution’s technology and regulatory policies, each site used different hardware and software platforms to conduct appointments. Generally, caregivers and therapists used an electronic device (e.g., desktop, laptop, or smartphone) and other accompanying materials (e.g., headphones, earbuds, speakers, microphones, wired or wireless Internet connection) to communicate via HIPAA-compliant videoconferencing software. This software was available for free to caregivers and accessible through a direct link sent by the therapist to the caregiver’s personal email.

Sessions took place in one or more locations in the participant’s home (e.g., the living room, a bedroom) containing the furniture and household items typically present in that setting. Appointments occurred weekly for about 60 min. Absenteeism or other scheduling accommodations occasionally resulted in some variability in both the duration and frequency of appointments.

Response measurement, interobserver agreement, and procedural fidelity

Caregivers and therapists worked together to operationally define each topography of the participant’s challenging behavior. For the 55 participants for whom topography-specifying information was available, almost all (87.3%, n = 48) engaged in multiple topographies of challenging behavior. For all participants who engaged in multiple topographies, the caregivers identified which topographies to target. The most common topography participants displayed was physical aggression (85.5%, n = 47), followed by disruption (58.2%, n = 32), tantrums (e.g., crying, whining; 50.9%, n = 28), and self-injurious behavior (38.2%, n = 21).

Therapists or trained data collectors scored video recordings of each session using specialized data-collection software. Data collectors scored target behavior as either a frequency or a duration measure, depending on the topography. A second, independent observer collected data on all dependent variables from the same video recording for at least 15% of sessions per condition, with a mean of 58.1, 46.6, and 44.5 of sessions with interobserver agreement data for the structured descriptive assessment (SDA), functional analysis, and FCT conditions, respectively. We calculated interobserver agreement for 54 of the 57 (94.7%), 25 of the 26 (96.2%), and 54 of the 57 (94.7%) participants across the SDA, functional analysis, and FCT conditions, respectively.¹ Total count interobserver agreement was computed for each participant by dividing the lower observed frequency or duration by the higher observed frequency or duration of target behavior per session and then multiplying by 100. Mean interobserver agreement was 95.9, 93.8, and 95% for the SDA, functional analysis, and FCT conditions, respectively (see Table 3).

TABLE 3.

Interobserver agreement and procedural fidelity across participants for the structured descriptive assessment, functional analysis, and functional communication training

	n	%	M (%)	Range (%)	SD
Interobserver agreement
SDA	54	94.7
Sessions with IOA			58.1	29–100	24.1
IOA			95.9	79–100	6.28
FA	25	96.2
Sessions with IOA			46.6	27–100	22.9
IOA			93.8	75–100	6.7
FCT	54	94.7
Sessions with IOA			44.5	17–100	24.7
IOA			95	75–100	6.5
Procedural fidelity
SDA	36	63.2
Sessions with fidelity			45.7	21–100	20.8
Fidelity			96.7	80–100	6.5
FA	36	63.2
Sessions with fidelity			39.9	20–100	23.4
Fidelity			92.1	70–100	8.1
FCT	36	63.2
Sessions with fidelity			29.3	20–63	9.7
Fidelity			96.4	70–100	6.6

Note. Mean, range, and SD are representative of the total number of participants for which data were available for each condition. For IOA, that was 54, 25, and 54 participants for the SDA, FA, and FCT conditions, respectively. For procedural fidelity, that was 36 participants for all conditions. The percentage of participants for whom we collected interobserver agreement and procedural-fidelity data are out of the 57 total participants. We calculated total-count IOA. SDA = structured descriptive assessment; FA = functional analysis; FCT = functional communication training; IOA = interobserver agreement.

To measure caregiver accuracy when implementing the procedures as coached by the therapists, the same data collectors who scored interobserver agreement scored procedural fidelity. For the functional analysis and FCT, procedural fidelity was measured using the same task-analysis scoring systems as were used in Wacker et al. (2013) for the functional analysis and Suess et al. (2020) for FCT. For the SDA, a novel task-analysis scoring system was developed (see Supporting Information A for all procedural-fidelity task analyses). Procedural fidelity was calculated by dividing the total number of steps completed accurately by the total number of steps in the task analysis and multiplying by 100. Fidelity data were available for 36 of the 57 participants (63.2%). Mean procedural fidelity for these participants was 96.7, 92.1, and 96.4% for the SDA, functional analysis, and FCT, respectively (see Table 3).

Experimental design

This study included random assignment of participants to either the no-functional-analysis (non-FA) or functional-analysis (FA) group. Procedures for these groups differed only in terms of the inclusion of a functional analysis in the FBA conducted with participants in that group. Group assignment was stratified by study site and participant sex, age (< 4 years or ≥ 4 years), and cognitive ability (IQ < 70 or ≥ 70) to ensure the two groups were as similar as possible on these variables. Of the 57 participants who met all inclusion criteria for the current experiment, 31 were assigned to the non-FA group and 26 were assigned to the FA group. Postrandomization analyses indicated no systematic association between experimental group and age, r_pb = .01, n = 50, p = .96; race, χ²(1, N = 55) = .64, p = .43, phi = .11; ethnicity, χ²(1, N = 53) = 1.46, p = .23, phi = .17; or ADOS score, r_pb = .14, n = 40, p = .41. However, females were somewhat overrepresented in the FA group (10 of 26, 38.5%) relative to the non-FA group (5 of 31, 16.1%), χ²(1, N = 57) = 3.64, p = .06, phi = .25, though this difference was statistically nonsignificant.

The study design allowed for an examination of the correspondence between the results of three increasingly rigorous FBA methods. All participants received a combination of an indirect assessment and evaluation of unstructured descriptive data. The indirect assessment consisted of a semistructured caregiver interview conducted by the therapist. The unstructured descriptive data consisted of caregiver-collected frequency or duration data, a topographical description of the target challenging behavior, and information on the antecedents that preceded each occurrence as well as the consequences that followed (i.e., ABC data; Bijou et al., 1968). We considered this Indirect+ABC assessment to represent the least rigorous form of FBA that still meets minimum standards described in some practice guidelines (e.g., Council of Autism Service Providers, 2020). Based on the literature, this type of FBA also appears to be frequently employed by practitioners in the field (Oliver et al., 2015; Petursdottir et al., 2010).

Therapists also conducted an SDA (Anderson & Long, 2002; Freeman et al., 2000) with participants in both groups. For participants randomly assigned to the non-FA group, the SDA was the terminal FBA method. The SDA was designed to ensure that participants contacted putative establishing operations for all three of the classes of social reinforcers typically included in a functional analysis (i.e., attention, tangible, and escape). However, consequences for any target behavior that occurred varied, as therapists instructed caregivers to respond “as they normally would.” This form of descriptive assessment is likely more rigorous than is typically conducted in the community, representing an approximation of what could be considered a best-case functional assessment that stops short of including a functional analysis. Thus, we considered an agreement between the results of the SDA and those of a functional analysis to represent an informative upper limit on correspondence between these two classes of functional assessment.

In addition to the procedures described above, participants randomly assigned to the FA group also had their challenging behavior exposed to a functional analysis. Findings from Oliver et al. (2015) suggest that when clinicians conduct a functional analysis, they frequently also conduct at least one additional type of FBA. Thus, we considered the sequence of assessments conducted with participants in the FA group to represent optimal clinical practice. Conducting both the SDA and functional analysis with participants in the FA group also made it possible to compare the correspondence between the results of these two FBA methods for participants in this group.

Following the terminal FBA format for their group assignment (i.e., SDA or functional analysis) and before receiving treatment, participants in both groups experienced a brief series of extinction baseline sessions with no programmed consequences for target behavior. The purposes of this series of extinction sessions were to (a) demonstrate that a brief period of extinction alone was insufficient to address target behavior, indicating that a more intensive treatment (e.g., FCT) was warranted (Suess et al., 2020), and (b) establish a more consistent history of reinforcement across groups before introducing treatment (Wacker et al., 2011). Participants who received a functional analysis of their target behavior had a recent history of dense reinforcement (fixed-ratio 1) for target behavior. In contrast, the target behavior of participants in the SDA may or may not have contacted reinforcement. Controlling for this pretreatment history of reinforcement was warranted because a purpose of the study was to compare treatment effectiveness based on how accurately the different FBA methods identified the reinforcer(s) maintaining target behavior and not due to potential artifacts of the contingencies associated with the FBA methods themselves. Rather than comparing levels of challenging behavior during treatment with those in the relevant test condition of the FBA, as is sometimes done (Scheithauer et al., 2020), a separate baseline condition was also necessary because the terminal FBA for the non-FA group (i.e., SDA) often produced too few data points to constitute a valid baseline. Following the completion of the extinction baseline, participants in both groups received treatment in the form of FCT.

Therapists, training, and supervision

Therapists included five doctoral-level behavior analysts (BCBA-Ds) or licensed clinical psychologists with extensive behavior-analytic training, two masters-level BCBAs, and two graduate students near the completion of their doctoral training in school or clinical psychology. The therapists had 1–20 years of experience conducting FBAs (including functional analyses) and coaching caregivers to implement behavior-analytic interventions. Therapists with a master’s degree met at least monthly with a doctoral-level principal investigator for their site to receive supervision and sent data following each appointment to the supervising investigator for review and discussion.

Procedures

Preliminary procedures and indirect+ABC assessments

Following informed consenting and randomization, the first appointment began with a general overview of telehealth and behavioral strategies. The therapist coached the caregiver on, for example, the use of telehealth and related materials (e.g., electronic devices and software), how to set up the room to minimize distractions for the participant, and how to handle the presence of other people (see Lee et al., 2015; Lerman et al., 2020). The therapist also explained the purposes and process of an FBA and described common functions of challenging behavior. Caregivers and therapists then completed the indirect assessment, which consisted of the therapist reading a set of questions (see Supporting Information B for the interview guide) and documenting the caregiver’s responses. The therapist attempted to address all caregiver questions. Caregiver responses during the indirect assessment were used to determine which topographies of challenging behavior to target for intervention, operationally define each target behavior, and develop hypotheses regarding the function of each target behavior. The appointment concluded with the therapist training the caregiver to collect ABC data and providing them with a corresponding data sheet. This didactic training consisted of the therapist providing vocal and written examples of how to complete the data sheet and answering any caregiver questions.

The second appointment began with a review of the content covered in the first appointment and the ABC data collected by the caregiver since that appointment. The therapist and caregiver discussed episodes of challenging behavior recorded by the caregiver (e.g., antecedents, topographies, and consequences) and whether it met the existing operational definitions. The therapist and caregiver adjusted the operational definitions if needed to best reflect the behavior of greatest caregiver concern. If this occurred, data collected by the caregiver between the first and second appointments were not incorporated into the interpretation of the ABC assessment. Next, the therapist worked with the caregiver to identify instructional situations and preferred items for inclusion in the SDA. Finally, with coaching from the therapist, the caregiver conducted consecutive 5-min toy-play sessions. The toy-play condition included noncontingent reinforcement in the form of free access to preferred leisure activities (identified via the indirect assessment), no instructions, and neutral attention from the caregiver. The purposes of these sessions were (a) to familiarize the caregiver with receiving instruction from the therapist, (b) for the participant to habituate to their caregiver speaking with the therapist via the videoconferencing technology, and (c) to screen for automatically reinforced target behavior. Thus, these sessions continued until any participant reactivity (e.g., attending to the device used to conduct the telehealth visit) ceased or until the appointment ended, whichever occurred first. If target behavior persisted during these sessions, suggesting a potential automatic function, we would have dismissed the participant from the study, but this never occurred. Therapists and caregivers then collaborated to develop hypotheses regarding a hierarchy of likely social functions (i.e., attention, tangible, and escape) based on the information gathered to that point via the indirect assessment and ABC data. The amount of ABC data collected varied across participants because (a) caregivers did not adhere perfectly to the request to collect data each day; (b) ABC data were event-based, and participants engaged in various levels of target behavior; and (c) the duration of ABC data collection varied across participants, but it always ended at the commencement of the SDA, which typically occurred in the third appointment. Although, in theory, it would have been possible for a caregiver to collect no ABC data, all caregivers collected at least some.

Structured descriptive assessment

The therapist coached the caregiver to conduct a series of 5-min sessions that each included one potential establishing operation in the form of restricted attention, restricted preferred items, or instructions. Therapists did not give instructions regarding specific consequences to deliver contingent on target behavior. Rather, they instructed the caregiver to respond to target behavior as they normally would. As a result, consequences varied across (and often within) sessions and participants. The order of the SDA conditions corresponded to the hypothesized likelihood of the target behavior serving each function generated by the therapist and caregiver in the prior appointment. Caregivers eventually conducted sessions for all three conditions even if they hypothesized that the target behavior did not serve the corresponding function. The SDA was completed in a single appointment.

During the attention condition, the therapist coached the caregiver to provide the child with approximately 2 min of presession access to high-quality attention and moderate- or low-preferred items. The session began when the caregiver restricted their attention and diverted it to another activity, such as reading a book or talking on the phone. During the tangible condition, the therapist coached the caregiver to provide their child with approximately 2 min of presession access to a highly preferred item. The session began when the caregiver restricted access to the item. During the escape condition, the therapist coached the caregiver to deliver repeated instructions to their child using a three-step progressive prompting procedure (i.e., vocal, model, physical guidance) and deliver praise for compliance. For all these conditions, the therapist prompted the caregiver to reinstate the establishing operation if at any point they discontinued it for greater than approximately 30 s. A toy-play session typically followed any session that resulted in the occurrence of target behavior to reestablish a baseline level of behavior. Therapists determined the function of the target behavior from the results of the SDA by using visual inspection to compare the levels of target behavior in each test condition with those observed during the toy-play condition, including the previous series of consecutive toy-play sessions. A potential function was identified if the level of target behavior was elevated (i.e., no overlap) in a test condition relative to the toy-play control.

If target behavior did not occur in any of the SDA conditions, the therapist repeated all test conditions. For some participants, the therapist modified the specific establishing operation included in one or more of the conditions for this second presentation (e.g., selected a different instruction or preferred item). Participants enrolled in the non-FA group were dismissed from the study if the SDA did not evoke target behavior, although treatment was generally still conducted outside of study participation. For those enrolled in the FA group, the therapist proceeded to conduct the functional analysis even if no target behavior occurred in the SDA (this occurred for only one participant). When the SDA indicated that target behavior served more than one function, the therapist consulted with the caregiver to select a single function to inform treatment. Table 4 summarizes the percentage of participants for whom each function was selected for treatment by group.

TABLE 4.

Functions identified by each functional behavioral assessment method and targeted in treatment across groups

	Total		Non-FA group		FA group
FBA format and outcomes	n	%	n	%	n	%
Indirect+ABC
Attention	15	26.3	9	29	6	23.1
Escape	12	21.1	7	22.6	5	19.2
Tangible	30	52.6	15	48.4	15	57.7
SDA
Function identified
Attention	26	45.6	15	48.4	11	42.3
Escape	39	68.4	23	74.2	16	61.5
Tangible	44	77.2	24	77.4	20	76.9
Number of functions identified
0	1	1.8	0	0	1	3.8
1	15	26.3	8	25.8	7	26.9
2	29	50.9	15	48.4	14	53.8
3	12	21.1	8	25.8	4	15.4
FA
Function identified
Attention					3	11.5
Escape					17	65.4
Tangible					20	76.9
Number of functions identified
0					0	0
1					14	53.8
2					10	38.5
3					2	7.7
Function treated
Attention	4	7	3	9.7	1	3.8
Escape	25	43.9	14	45.2	11	42.3
Tangible	28	49.1	14	45.2	14	53.8

Note. All participants (N = 57) received Indirect+ABC assessments and an SDA. Only those in the FA group (n = 26) subsequently received an FA. Only action was identified by the Indirect+ABC assessment. The SDA and FA could identify multiple functions. Because the SDA and FA could identify multiple functions, the total number of participants for whom these FBAs identified particular functions exceeded sample sizes for groups that received those FBA methods. FBA = functional behavioral assessment; SDA = structured descriptive assessment; FA = functional analysis; Non-FA = no functional analysis.

Functional analysis

All functional analysis sessions followed a format similar to that for the SDA except that the functional analysis employed a multielement design and the therapist coached the caregiver to deliver the relevant consequences contingent on target behavior in test conditions. That is, in the attention condition, 3–5 s of caregiver attention (e.g., positive soothing) followed target behavior. In the tangible condition, 30 s of access to the previously restricted preferred item followed target behavior. In the escape condition, either the removal of instructions and related materials or caregiver redirection of the participant away from the work area (e.g., pushed their chair out from the table and vocally indicated the participant could have a break) followed target behavior. In the toy-play condition, no programmed consequences followed target behavior.

The functional analysis continued until a function was identified based on the clinician’s (i.e., therapist or the supervising site principal investigator) judgement using the objective criteria described by Roane et al. (2013). If target behavior did not occur in any of the functional analysis conditions or no function was discernable the therapist increased the session duration from 5 min to 10 min and continued the functional analysis. As described above, only participants who initiated treatment were included in this data set. Thus, participants for whom no target behavior occurred during the functional analysis or for whom no function could be identified after these steps and after repeated series were conducted were dismissed from further participation. We followed the same procedures as we did for the SDA if the functional analysis identified more than one function of target behavior.

Extinction baseline

Extinction baseline sessions involved the same procedures for introducing the establishing operation from the condition identified as containing the functional reinforcer via the terminal FBA method (i.e., the SDA for participants in the non-FA group and the functional analysis for participants in the FA group) but without programmed consequences for target behavior. For most participants, this series consisted of three sessions. However, if the level of target behavior was on a decreasing trend, the therapist extended the baseline until they observed elevated levels of target behavior. This occurred within eight or fewer sessions for all participants.

Treatment

Treatment consisted of FCT based on the function identified in the terminal FBA method (i.e., SDA or functional analysis) for each participant. We treated functions other than the one targeted in treatment following study participation if the terminal FBA method identified target behavior to be multiply controlled. Table 4 summarizes the percentage of participants for whom each function was selected for treatment by group.

Functional communication training sessions were similar to extinction baseline sessions (e.g., duration, location, materials), except the therapist coached the caregiver to deliver the functional reinforcer contingent on the functional communication response (FCR). The therapist and caregiver, together, identified the most appropriate FCR topography (e.g., vocal mand, card exchange, augmentative communication device) to be taught during FCT based on the participant’s existing communicative repertoire. Although therapists worked with caregivers to thin the schedule of reinforcement or bring it under stimulus control, such as using a multiple schedule (Greer et al., 2016), we included data only for the initial phase of treatment in this data set. Therapists conducted the initial treatment phase until (a) challenging behavior was reduced by at least 90% relative to the extinction baseline for three consecutive sessions, (b) the therapist determined via visual inspection of the data that the treatment was ineffective (e.g., an extended stable or increasing trend in target responding), or (c) the participant dropped out of the study.

Data analysis

Correspondence between FBAs

For all FBA methods, results were dichotomized (i.e., function identified/function not identified) for each potential function. As described above, therapists interpreted the results of the indirect and ABC data by rank ordering potential functions in terms of the likelihood of maintaining the target behavior. However, therapists were not asked to hypothesize whether the target behavior served one or more functions based on these data. Therefore, we restricted the interpretation of the combined Indirect+ABC assessments to only the function deemed most likely to maintain target behavior. Thus, we calculated partial agreement between the Indirect+ABC assessment and the two other FBA methods. An agreement consisted of the SDA or functional analysis identifying at least the function identified by the Indirect+ABC assessment for these comparisons.

It was only possible to calculate agreement between the SDA and functional analysis for participants in the FA group because only they underwent both FBA methods. We defined an exact agreement between the SDA and functional analysis as instances in which both methods identified all the same functions and nonfunctions (e.g., both the SDA and functional analysis identified attention and tangible functions, and neither identified an escape function). We defined partial agreement as instances in which the SDA and functional analysis identified some but not all of the same functions and nonfunctions (e.g., the SDA identified only attention and tangible functions, whereas the functional analysis identified only an attention function, and neither identified an escape function). We differentiated between partial agreements with one third (33%) or two thirds (66%) agreements. Finally, we defined instances of no agreement as those in which the SDA and functional analysis exclusively identified different functions and nonfunctions (e.g., the SDA identified only attention and tangible functions, and the functional analysis identified only an escape function).

When the SDA and functional analysis did not result in an exact agreement, the SDA either identified one or more functions not identified by the functional analysis (or vice versa) or failed to identify one or more functions identified by the functional analysis (or vice versa). Because these scenarios are the inverse of one another and because the functional analysis is considered the more rigorous FBA method, we report the first of these types of disagreements in terms of the SDA identifying an uncorroborated function and the second as a potentially missed function. We analyzed the prevalence of these outcomes by calculating the overall percentage of accurate, uncorroborated, and potentially missed functions by dividing the number of occasions of each outcome by the sum of accurate, uncorroborated, and potentially missed functions, and then multiplying by 100.

Finally, we calculated total agreement between the SDA and functional analysis, a measure of overall agreement between these FBA methods across participants, by dividing all agreements (whether for an identified or nonidentified function) by the number of all potential functions and multiplying the quotient by 100. Although potentially an overestimate of agreement, this measure is aligned with how some other studies have evaluated the agreement between FBA methods (e.g., Martens et al., 2019).

Treatment outcomes

We evaluated the outcomes of treatment using both the per-protocol and intent-to-treat approaches. Within the clinical trials literature, per protocol refers to the inclusion of results from only those participants who completed the course of treatment under investigation. By excluding participants who dropped out of the study, per protocol examines the effects of having received the treatment as designed (Tripepi et al., 2020). In contrast, intent to treat refers to the inclusion of results for all participants regardless of whether they dropped out. This approach is widely adopted within efficacy and comparative effectiveness trials (Hollis & Campbell, 1999; Lewis & Machin, 1993) and is mandated by certain government agencies that fund clinical trials (Food and Drug Administration, 1997). With intent to treat, dropouts are considered cases of unsuccessful treatment because the desired outcome was not achieved regardless of the reason. This approach takes an agnostic posture as to the reason a participant dropped out because self or caregiver report may not accurately reflect whether a participant did so because treatment was insufficiently effective or for reasons unrelated to treatment effectiveness. Instead, the intent-to-treat approach relies on random assignment to assume that variables that may lead to a participant dropping out for reasons unrelated to treatment outcome are distributed equally across participants in both experimental groups. The percentage of participants from each group for whom treatment achieved the predefined success criteria (≥ 90% reduction from baseline across three consecutive sessions) served as the primary treatment outcome measure for both the per-protocol and intent-to-treat analyses. We conducted a nonparametric analysis (chi-square test of independence) using IBM Statistics (version 26) to determine if the percentage of those achieving success criteria in each group was greater than chance (p < .05). Cramer’s phi statistic was also calculated with this analysis as the indicator of effect size, interpreted as .10 = small effect, .30 = medium effect, ≥.50 = large effect (Cohen, 1988). These analyses were completed both as per protocol and intent to treat.

As a secondary outcome for the intent-to-treat analysis, we compared the duration of treatment across groups. In addition to providing a comparison of treatment efficiency, this outcome served as a check against bias in therapists’ persistence. That is, potential for bias existed because the treatment phase continued until the participant met the success criterion, the participant dropped out, or the therapist determined that treatment was ineffective. Thus, therapists’ persistence with treatment may have systematically favored participants in one of the two groups. Comparing treatment duration served as a check against this bias because any tendency for therapists to persist with treatment for participants in either of the groups would be expected to result in longer treatment durations for participants in that group. We used a nonparametric analysis (Mann–Whitney U) and an r effect size statistic (r = Z / √N; .10 = small, .30 = medium, ≥.50 = large) to evaluate whether session and appointment attendance systematically differed between the FA and non-FA groups.

Expected rate of treatment success for the non-FA group

This study’s design made it possible to examine how well treatments for the non-FA group performed compared with a theoretical expected rate (Tiger & Effertz, 2021) of treatment success based on some common assumptions about FBA methods. Specific assumptions that we attempted to evaluate included (a) results of functional analyses are the closest available approximation to a theoretical “ground truth” regarding the function of an individual’s challenging behavior and (b) treatments based upon false-positive FBA results are ineffective. We calculated the expected rate of treatment success for the non-FA group by extrapolating from the findings of the FA group. First, we used the results of the functional analysis for this group as the ground truth regarding the function of participants’ target behavior. We then calculated the positive predictive value of the SDA conducted with the FA group relative to that ground truth. We calculated the positive predictive value by dividing the number of instances in which the SDA and functional analysis both identified the presence of a function, divided by the combined total number of these true positives and uncorroborated functions. This positive predictive value represents the probability that any function identified by the SDA was a true positive as determined by the functional analysis. Finally, we multiplied the positive predictive value by the percentage of participants in the FA group who met the predetermined definition of treatment success (using both per-protocol and intent-to-treat approaches) based on an assumption of similar probability of treatment success when FCT is based on a true-positive result. Negative predictive value was not calculated because it was not necessary for this analysis of expected rate of treatment success and because to do so would have required implementing treatments for uncorroborated functions, which was beyond the scope of this study.

RESULTS

Table 4 depicts the outcomes of the different FBA methods by group. For the 31 participants in the non-FA group, the Indirect+ABC assessments resulted in an attention, escape, or tangible function for nine (29.0%), seven (22.6%), and 15 (48.4%) participants, respectively. The SDA identified attention, escape, and tangible functions for 15 (48.4%), 23 (74.2%), and 24 (77.4%) participants, respectively. The SDA identified only one function for eight (25.8%) participants, two functions for 15 (48.4%) participants, and three functions for eight (25.8%) participants.

For the 26 participants in the FA group, the Indirect+ABC assessments resulted in an attention, escape, or tangible function for six (23.1%), five (19.2%), and 15 (57.7%) participants, respectively. The SDA identified attention, escape, and tangible functions for 11 (42.3%), 16 (61.5%), and 20 (76.9%) participants, respectively. The SDA identified only one function for seven (26.9%) participants, two functions for 14 (53.8%) participants, and three functions for four (15.4%) participants, and it failed to identify any function for one (3.8%) participant. Thus, except for a greater propensity for the SDA to identify an escape function or three functions in the non-FA group, the two groups were generally similar with respect to the results of their Indirect+ABC assessment and SDA.

The functional analysis identified an attention, escape, or tangible function for three (11.5%), 17 (65.4%), and 20 (76.9%) participants in the FA group, respectively. It also identified only one function for 14 (53.8%) participants, two functions for 10 (38.5%) participants, and three functions for two (7.7%) participants.

Correspondence between FBA methods

Figure 1 depicts the comparison of results from the different FBA methods: Indirect+ABC to SDA, Indirect+ABC to functional analysis, SDA to functional analysis (separated by exact, partial, and no agreement), and total agreement between the SDA and functional analysis. The Indirect+ABC assessment identified a function indicated by the SDA for 40 participants (70.2%). The Indirect+ABC assessment identified a function identified by the functional analysis for 17 participants (65.4%). The SDA and functional analysis had no agreement for two participants (7.7%), 33% partial agreement for five participants (19.2%), 66% partial agreement for nine participants (34.6%), and exact agreement for 10 participants (38.5%). Total agreements between the functional analysis and SDA occurred for 53 (67.9%) potential functions across all participants in the FA group. Note that for only five of the 26 participants in the FA group (19.2%) did the SDA and functional analysis both identify a single function. For only one of those participants (20% of these five participants; 3.9% of all participants in the functional FA group) did that single function (tangible) match across the Indirect+ABC assessment, SDA, and functional analysis.

FIGURE 1.

Correspondence between the indirect+ABC, structured descriptive assessment, and functional analysis. The therapist and caregiver only identified one function following the Indirect+ABC. The SDA and FA could have identified multiple functions. We scored an agreement if the terminal FBA format identified at least the function identified in the Indirect+ABC. We scored an agreement across terminal FBA formats if both identified the same function and nonfunction for each possible function. FA = functional analysis; SDA = structured descriptive assessment.

Figure 2 depicts the accurate, uncorroborated, and potentially missed outcomes identified when comparing results of the SDA with those of the functional analysis. There were 78 opportunities for comparing SDA and functional-analysis outcomes overall (i.e., three potential functions for each of the 26 participants). The SDA identified a function that was corroborated by the functional analysis for 53 (67.9%) comparisons but identified an uncorroborated function for 16 (20.5%) and potentially missed a function for nine (11.5%) comparisons. In other words, the SDA was almost twice as likely to identify a function that was not identified by the functional analysis than not identify a function that was identified by the functional analysis. The SDA identified an attention function that was corroborated by the functional analysis for 18 (69.2%) participants but identified an uncorroborated attention function for eight (30.8%) participants. The SDA had no potentially missed attention functions. The SDA identified an escape function that was corroborated by the functional analysis for 15 (57.7%) participants but identified an uncorroborated escape function for five (19.2%) participants and potentially missed an escape function for six (23.1%) participants. The SDA identified a tangible function that was corroborated by the functional analysis for 20 (76.9%) participants but identified an uncorroborated tangible function for three (11.5%) participants and potentially missed a tangible function for three (11.5%) participants.

FIGURE 2.

Accurate, uncorroborated, and potentially missed outcomes of the structured descriptive assessment. Overall outcomes were out of the total possible functions that the SDA could have identified (n = 78). Outcomes for the three specific functions were out of the total number of participants in the FA group (n = 26) because they received both the SDA and the FA—the latter serving to validate the former. FA = functional analysis; SDA = structured descriptive assessment.

Treatment outcomes

For the per-protocol analysis of treatment outcomes, all participants (n = 48) who completed treatment, regardless of group assignment, met the predetermined criterion for treatment success (≥ 90% reduction across three consecutive sessions). Treatment for the FA and non-FA group decreased challenging behavior relative to the extinction baseline by a mean of 98.7% (range: 91.6%–100%, SD = 2.3) and 99.6% (range: 94.8%–100%, SD = 1.3), respectively. The association between experimental group and treatment success for the per-protocol sample was nonsignificant and small in effect, χ²(1, N = 48) = 1.11, p = .29, phi = .15.

Overall, nine participants (15.8%) dropped out of the study during treatment: six (66.7%) from the non-FA group and three (33.3%) from the FA group. Because all participants who completed treatment met the success criterion, the percentage of participants with successful outcomes using the intent-to-treat analysis equaled the percentage of participants from each group who completed treatment. Specifically, the non-FA group had a success rate of 25 of 31 participants (80.6%), whereas the FA group had a success rate of 22 of 26 participants (84.6%). The association between experimental group and treatment success for the intent-to-treat sample was also nonsignificant and small in effect, χ²(1, N = 57) = .15, p = .69, phi = .05.

We considered analyzing outcomes by comparing the level of target behavior during treatment to that observed during the relevant condition of the terminal functional assessment (i.e., SDA for the non-FA and functional analysis for the FA group) rather than during the extinction baseline. However, due to its programmed brief procedures, the SDA often resulted in less than three sessions per test condition—an inadequate number to serve as a baseline. In addition, Scheithauer et al. (2020) recently reported little difference between comparisons of responding during treatment to that observed in the relevant condition of a functional analysis versus a “stand-alone” baseline (extinction baseline in this study).

Figure 3 depicts the duration of all assessment procedures for all participants and for treatment procedures for participants who completed that phase of the study. Although both groups required a similar number of sessions to complete assessment procedures up to and including the SDA (M = 6.4, range: 3–11, SD = 2 for the non-FA group; M = 5.2, range: 3–7, SD = 1.3 for the FA group), participants in the FA group required an additional 22 sessions on average to complete the functional analysis (range: 10–57, SD = 8.5). Allocating time to conduct a functional analysis may still be more efficient if doing so produces a commensurate or greater reduction in the duration of treatment. However, treatment also required slightly more sessions for participants in the FA group who completed treatment (M = 32.8, range: 3–72, SD = 18.1) relative to the non-FA group (M = 26.7, range: 3–60, SD = 16.6). Notably, this group difference in the number of sessions was not statistically significant, U (N_non-FA = 31, N_FA = 26) = 326.5, z = −1.23, p = .22, r =.16. In total, participants in the FA group required a mean of 27 more sessions to complete all assessment and treatment procedures, which equated to about five additional appointments (M = 14.1, range: 6–22, SD = 4.4) relative to those in the non-FA group (M = 9, range: 3–15, SD = 3.6). The group difference in the number of appointments completed was statistically significant and large in effect, U (N_non-FA = 31, N_FA = 26) = 155.0, z = −3.98, p < .001, r =.53.

FIGURE 3.

Duration of the experiment as the distribution of sessions across groups for the structured descriptive assessment, the functional analysis, and functional communication training and as the distribution of overall appointments across groups. The total height of the bar denotes the mean number of sessions or appointments across all conditions. The height of each condition bar for each group denotes the mean number of sessions per condition. SDA = structured descriptive assessment; FA = functional analysis; Appts = Appointments.

Expected rate of treatment success for the non-FA group

The positive predictive value of the SDA conducted with the FA group relative to the functional analysis conducted with the same group was .68. Again, this value represents the probability that any function identified by the SDA was a true positive as determined by the functional analysis. Multiplying this positive predictive value by the percentage of participants in the FA group who met the predetermined definition of treatment success (100% for the per-protocol analysis; 88.5% for the intent-to-treat analysis) generated an expected rate of treatment success of 68% and 60.1%. In other words, given our assumptions that a functional analysis is the closest approximation of ground truth and that false-positive FBA results are ineffective, one could have expected treatments for 68% of participants in the non-FA group who completed treatment (per protocol) to produce a 90% or greater reduction in target behavior. Additionally, one could have expected a 90% or greater reduction in target behavior for 60.1% of all participants in the non-FA group, when including dropouts (intent to treat).

DISCUSSION

This study replicates and extends the literature on correspondence between and outcomes of different FBA methods of varying levels of rigor. Specifically, an FBA consisting of indirect and descriptive assessments showed only modest correspondence (38.5% exact agreement; 68% total agreement) with one that also included a functional analysis. This result replicates previous findings showing poor correspondence between functional analyses and less rigorous FBA approaches (Alter et al., 2008; Hall, 2005; Pence et al., 2009; Virues-Ortega et al., 2011), even though the FBA conducted with participants in the non-FA group is probably more rigorous than the type of FBA typically conducted in the community. Thus, this poor correspondence can be considered a likely upper limit on agreement between results of FBAs that do and do not include a functional analysis. However, this finding extends prior research because it was demonstrated across substantially more participants (N = 57) than have previously been included in a single prospectively collected sample.

Tiger and Effertz (2021) suggested that the ultimate measure of the clinical utility of an FBA is the degree to which it increases the likelihood of selecting an effective treatment. To our knowledge, this is the first large-scale prospective comparative effectiveness study to assess this type of clinical utility across two FBA methods by juxtaposing the outcomes of treatments based on each approach. Within the per-protocol analysis, all participants, regardless of which FBA method they received, achieved the predetermined criterion for treatment success of at least a 90% reduction from baseline, with a mean reduction in challenging behavior of 99.2% across groups. This result demonstrates that FCT was effective for participants who completed the entire course of treatment. However, treatment persisted until this success criterion was achieved, the therapist determined it was unsuccessful, or the caregiver dropped out. Thus, participant dropouts may have been an indicator of treatment ineffectiveness. The intent-to-treat analysis, which considered dropouts to be instances of treatment failure, showed that, as has been suggested but not empirically demonstrated before via direct comparison, treatments based on an FBA that included a functional analysis were more successful than those based on an FBA that did not. Specifically, the percentage of participants in the FA group meeting the predetermined standard for treatment success (84.6%) was slightly higher than that for the non-FA group (80.6%).

Although a slightly greater percentage of participants from the FA group met the criteria for successful treatment than from the non-FA group, this difference was statistically no greater than chance. In short, treatments based on the non-FA FBA substantially outperformed the expected rate of treatment success (68%) regardless of whether dropouts were included (per protocol = 100%) or not (intent to treat = 80.6%). In other words, the non-FA FBA resulted in successful treatment more often than what would have been expected given the assumptions that functional analyses approximate ground truth and that treatments based on false-positive findings are ineffective. There are several potential explanations for this discrepancy. Specifically, there may have been systematic differences between the participants in the non-FA and FA groups. However, to minimize this possibility, participants in this study were randomly assigned to experimental groups, with stratification according to demographic variables hypothesized as potential sources of such bias (i.e., study site, participant sex, age, and cognitive ability). Further, statistical analyses evaluating potential group differences in known demographics and ADOS scores suggested that the groups were relatively comparable, although females were somewhat overrepresented in the FA group. Thus, although there may have been other unmeasured variables that systematically varied as a function of experimental group, the possibility of systematic group differences was controlled to the extent possible.

Another explanation for the discrepancy between the expected and obtained rate of treatment success for the non-FA group is that one or both assumptions on which the expected rate was based may not be entirely correct. With respect to the first assumption, although this result raises the possibility that results of functional analyses may not approximate ground truth as well as is sometimes assumed, the consensus remains that functional analyses are the most thorough FBA format and the process includes properties that are deemed important for scientific and clinical rigor (e.g., demonstration of a causal relation between environmental events and target behavior). Thus, until these results can be replicated and expanded on, it remains our recommendation that when accuracy is deemed of utmost importance, behavior analysts should continue to rely on functional analyses.

With respect to the second assumption, treatments based on an FBA that did not include a functional analysis produced better outcomes than would be expected if treatments based on false-positive findings do not produce successful outcomes. This result highlights how it is possible for some treatments to effectively reduce challenging behavior even when they are based on an inaccurate conclusion about its function. For example, providing preferred leisure or edible items can reduce challenging behavior, even when they are not functional reinforcers (e.g., Carter, 2010; Lalli et al., 1999; Payne & Dozier, 2013; Piazza et al., 1997). Given this example, it is notable that a high percentage of the treatments in both groups were based on the conclusion that challenging behavior served a tangible function (n = 28, 49.1%; see Table 4). Thus, these findings support previous suggestions of the importance of further study of the circumstances under which non-FA-based treatments may be effective (e.g., Newman et al., 2021).

The time and resources required to conduct each assessment is another important aspect of effectiveness (Tiger & Effertz, 2021). The functional analysis added a mean of about 22 sessions during which treatment was implemented with the participants in the non-FA group. Furthermore, the inclusion of the functional analysis did not increase treatment efficiency, with treatment for participants in the FA group requiring slightly more sessions on average (M = 32.8) than those in the non-FA group (M = 26.7), although this difference was statistically nonsignificant.

The fact that treatments were successful regardless of the FBA method that informed them may be in part due to therapist discretion to persist with treatment until it was either successful or they determined that it was ineffective. However, therapists never discontinued treatment for any participant due to ineffectiveness, indicating that they were not more likely to persist with treatments for participants in one group over the other. Furthermore, persistence is likely to be correlated with treatment duration and treatments were longer for participants in the FA group. Thus, if any such bias existed it was likely in favor of the FA group.

Another reason treatments were effective even when informed only by the SDA may be that the brief extinction baseline might have produced an extinction burst, which would have increased the value from which we determined the 90% reduction criterion for treatment success. Such an extinction burst might be particularly likely for participants in the FA group given the abrupt transition from the dense schedule of reinforcement programmed within a functional analysis to extinction (see Fisher et al., 2023). Data from all participants at one of the experimental sites (n = 13) showed that an extinction burst, defined using the criteria from Lerman and Iwata (1995), occurred for 3 of 5 (60%) and 5 of 8 (62.5%) participants across the non-FA and FA groups, respectively. This analysis suggests that the probability of an extinction burst was nearly equal across groups, supporting the efficacy of treatment regardless of the FBA format that informed it.

Care should be taken when extending these findings to other populations, settings, or treatment approaches. For example, we specifically sampled the population of young children with ASD and socially mediated challenging behavior. It is possible that results may not be replicated with individuals with longer histories of challenging behavior or those whose challenging behavior is maintained by automatic reinforcement. Inclusion criteria also required treatment to address a single function. However, it is not unusual for FCT to be applied to multiple functions simultaneously (Falcomata et al., 2012). Also, because it was deemed important to control for the specific type of treatment, FCT was implemented with all participants, regardless of function. However, other function-based treatments have also been shown to be as effective and, in some cases, hold advantages over FCT.

Caution should be exercised when inferring the degree to which these methods and findings represent community practice or outcomes. Therapists in this study were experienced in conducting FBAs, including functional analyses, which is not representative of many community providers. Thus, these findings might not be replicated when less experienced therapists conduct FBAs or treatment. Specific elements of the FBAs included in this study also undoubtedly differ from those conducted in many settings. We selected the non-FA FBA methods to represent a best-case assessment. As a result, it is inappropriate to speculate on how less rigorous FBA methods may fare in similar comparisons. However, the modest correspondence between the results of these two FBA formats is consistent with findings from previous comparisons, including those that compared other forms of indirect and descriptive assessments with functional analyses (Alter et al., 2008; Contreras et al., 2023; Hall, 2005; Pence et al., 2009; Romani et al., 2023; Virues-Ortega et al., 2011). The treatment findings did not appear to be affected by this modest degree of correspondence.

For participants in the FA group, the functional analysis always came after the other FBA methods, which may have influenced the degree to which results of the two FBA methods corresponded. We elected to always conduct the indirect and descriptive assessments first because the inclusion of consequences can produce learning during assessment (Irwin Helvey & Van Camp, 2022; Retzlaff et al., 2020). However, conducting the functional analysis last may have also introduced bias for the therapists because the results of the Indirect+ABC and SDA were known to them while they conducted the functional analysis.

The finding that FBAs that did not include a functional analysis were nearly as likely to produce successful treatments as an FBA that did may give insights into the persistent finding that most clinicians report greater reliance on non-FA FBA methods (e.g., Oliver et al., 2015; Saini & Cox, 2020). Some of this literature has intimated that when clinicians do so, they are in error. However, the results of this study suggest that reliance on FBAs that do not include a functional analysis is likely to result in treatment success for young children with ASD as frequently as when a functional analysis is included. It is notable that most of the clinicians who have reported greater reliance on non-FA FBAs in those prior studies served populations similar to the one evaluated in this study. Given the barriers to conducting functional analyses, this rate of treatment success appears sufficient to maintain clinicians’ selection of such FBA methods.

Finally, this study allowed for variability across participants in several ways such as the number of instances of target behavior recorded using ABC methods, therapists’ clinical judgment while interpreting data from indirect assessments or ABC data, and the ways caregivers implemented consequences during the SDA. Although behavior-analytic research more often controls for these kinds of potentially confounding variables experimentally, there are certain research questions relevant to applied behavior analysis that may be best addressed through other means. In this study, the sample size and group design provided some control over the degree to which these sources of variability may have influenced its findings because random assignment of many participants can be expected to distribute such variability across groups randomly, making systematic bias unlikely. One limitation of group designs often described in the behavior-analytic literature is that averaged or aggregated results may not represent the outcomes for individual participants (Walker & Carr, 2021). This limitation may be offset by other advantages of group designs for certain research questions, such as the one posed in this study. Furthermore, some audiences rarely contact the behavior-analytic literature, in part because of biases against single-subject methods. Adopting group designs when appropriate can increase the probability of disseminating behavior-analytic research to such constituencies. Thus, in addition to attempting to address this specific research question, it is our hope that this study may serve as a model for the use of group designs to address research questions of importance to applied behavior analysts.

DATA AVAILABILITY STATEMENT

Supporting information includes materials used to guide therapist interviews as part of the indirect assessment and data sheets for the ABC assessment.