Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2025 Jul 31.
Published in final edited form as: J Psychopathol Behav Assess. 2023 Nov 16;46(1):25–34. doi: 10.1007/s10862-023-10104-1

Reliability and Validity of the Therapy Attitude Inventory in Caregivers Receiving Internet-Delivered Parent Child Interaction Therapy for Young Children with Developmental Delay

Keara J Neuman 1, Timothy Hayes 1, Jonathan S Comer 1, Daniel M Bagner 1
PMCID: PMC12311981  NIHMSID: NIHMS2036552  PMID: 40746441

Abstract

Despite common use of the Therapy Attitude Inventory (TAI) to measure satisfaction with parenting interventions, psychometric support has only been demonstrated in predominantly White caregivers of typically developing children following clinic-based treatments. Additionally, the reliability and validity of the Spanish version of the TAI have not been assessed. The present study examined the reliability and validity of the English and Spanish versions of the TAI among mostly Hispanic/Latinx caregivers of children with developmental delay (DD; n = 65) following Internet-delivered Parent-Child Interaction Therapy (iPCIT). Internal consistency was excellent for both versions. Convergent validity was demonstrated by a strong correlation with the Client Satisfaction Questionnaire-8. Satisfaction was predicted by improvements in child behavior and parenting skills, as well as reductions in caregivers’ negative feelings about parenting, following treatment. These findings support use of the TAI with Spanish-speaking families, families from minoritized backgrounds and with children with DD, and following telehealth interventions.

Keywords: Satisfaction, Reliability, Validity, Behavioral Parenting Interventions, Developmental Delay

Client satisfaction, referring to the degree to which individuals value therapy received and feel the treatment has met their needs and goals for services (Brestan et al., 1999), is an important process variable and outcome of mental health services. Measures of client satisfaction typically assess how much clients liked the treatment format, the therapist, the techniques used, the skills learned, and the treatment outcomes achieved (Brestan et al., 1999). Client satisfaction is associated with greater treatment participation and reductions in symptoms (Attkisson & Zwick, 1982), suggesting the importance of evaluating satisfaction as an indicator of treatment engagement and effectiveness. Satisfaction may be particularly important to assess when expanding patient-centered care and the use of evidence-based treatments with populations who have been underrepresented in clinical research, as well as with different treatment modalities, such as telehealth. In this study, we assessed the psychometric properties of the Therapy Attitude Inventory (TAI; Eyberg, 1993) among caregivers of young children with developmental delay (DD) from predominantly minoritized, low-income, and Spanish-speaking backgrounds and who received remote treatment for their child’s behavioral challenges.

Research with adult populations suggests that there is a myriad of individual- and treatment-level factors associated with client satisfaction with therapy. For instance, satisfaction with treatment has been shown to be predicted by the amount of time spent with the therapist during sessions, the amount of advice or direction provided in therapy, and change in symptoms following treatment (Stewart et al., 2009; Moore & Kenning, 1996; Attkisson & Zwick, 1982). Individual factors, such as client race/ethnicity, presenting diagnosis, and comorbid diagnoses may also relate to satisfaction with therapeutic services (Lebow, 1982; Hasler et al., 2004; Schulte et al., 2011). Compared to adult populations, less research has been conducted on satisfaction with child mental health treatment. Findings similarly indicate that improvement in symptoms often predicts satisfaction with child mental health services (O’Reilly et al., 2021; Rey et al., 2002), although results suggest that there may not be significant differences in satisfaction with child mental health services based on race or ethnicity (Shapiro et al., 1997; Copeland et al., 2004).

When it comes to child mental health treatment, caregiver satisfaction is particularly important to assess, especially when treating young children who are not old enough to provide valid self-reports of satisfaction or when treating children with DD who may have difficulties expressing their perspectives on treatment. Additionally, externalizing behavioral challenges, which are common in young children with DD (Neece et al., 2012), create difficulties for the child’s caregiver(s), who often initiate treatment for their children. Moreover, caregivers are typically targeted in treatment and thus seen as co-clients and important reporters of treatment satisfaction. For example, in widely used and well-established behavioral parenting interventions for child behavioral challenges (Comer et al., 2013; Kaminski & Claussen, 2017), caregivers are directly involved in treatment and form a relationship with the therapist, providing them sufficient insight to report on satisfaction with a wide variety of aspects of the treatment.

One commonly used and supported measure of client satisfaction with parenting interventions is the TAI. The TAI was developed to reflect the specific goals of parenting interventions (Foote et al., 1998), including to reduce child behavior problems, increase positive parenting skills and limit-setting, and improve the quality of the caregiver-child relationship. Thus, the TAI specifically assesses satisfaction with parenting skills learned, changes in child behavior and compliance, and overall satisfaction with the treatment. In a psychometric evaluation of the TAI in caregivers participating in parenting intervention services for behavioral challenges in their typically developing children (Brestan et al., 1999), the TAI demonstrated excellent internal consistency (a = 0.91) and high test-retest reliability (r = .85). Furthermore, TAI scores were found to be predicted by improvements across treatment in caregiver parenting practices and child behavioral challenges (Brestan et al., 1999), providing support for the validity of the measure.

While the TAI has demonstrated favorable psychometric properties, to date, research has only evaluated the psychometric properties of the TAI in predominantly White and English-speaking samples of caregivers of typically developing children. The TAI has previously been used with families of children with DD receiving behavioral parenting interventions (e.g., Bagner, et al., 2023; Bagner & Eyberg, 2007; Bagner et al., 2010), but the psychometric properties of the scale have never been assessed in this important clinical population. Children with DD demonstrate significantly higher levels of behavioral challenges than their typically developing peers (Neece et al., 2012), and increasing evidence suggests that behavioral parenting interventions are efficacious in the treatment of behavioral challenges in children with DD (Petrenko, 2013). Given that parenting interventions are a recommended first-line treatment for behavioral challenges in young children with DD, there is a critical need to establish the reliability and validity of consumer satisfaction measures, such as the TAI, specifically with this clinical population.

Additionally, examining the reliability and validity of the TAI with families from low-income and racial/ethnic minoritized backgrounds is particularly important given that such families have been underrepresented in research assessing satisfaction with parenting interventions and have been underserved in clinic settings. Importantly, children from low-income and racial/ethnic minoritized backgrounds are at increased risk for developmental and behavioral challenges (Qi & Kaiser, 2003) and thus may benefit from improved access to behavioral parenting interventions. To optimally inform patient-centered care for low-income and minoritized families, it is important to have measures of satisfaction with parenting interventions that have been psychometrically evaluated specifically in families from similar backgrounds. Further, in response to the growing Spanish-speaking population within the United States (Ryan, 2013) and the associated need for Spanish language mental health services, many supported parenting interventions for child behavioral challenges, such as Parent-Child Interaction Therapy (PCIT) and the Incredible Years program, have been translated into Spanish and implemented with Spanish-speaking families (Borrego et al., 2006; McCabe et al., 2012; Webster-Stratton, 2001). While research supports the overall efficacy of such treatments for Spanish-speaking families (e.g., Matos et al., 2006; McCabe et al., 2012), treatment outcomes and findings can differ across English-versus Spanish-speaking families (Ramos et al., 2018), underscoring the importance of examining satisfaction with Spanish-speaking families. The TAI has been translated into Spanish and used with Spanish-speaking families (Matos et al., 2006). However, to our knowledge, the psychometric properties of the scale have never been assessed in a Spanish-speaking population.

Despite the need for effective and acceptable treatments for underrepresented families, there are disparities in access to care for children with DD and from low-income and racial/ethnic minoritized backgrounds (Rosenberg et al., 2008; Morgan et al., 2012). To reduce barriers to care, such as lack of transportation, limited childcare, work conflicts, and stigma of going to a mental health clinic, video-based telehealth interventions may be a useful alternative to in-person services delivered at a clinic (Comer et al., 2017, 2021). Recent research supports the efficacy and acceptability of telehealth programs for treating behavioral challenges in young children with DD from underrepresented minoritized backgrounds (Bagner et al., 2023). However, the psychometric properties of the TAI have yet to be evaluated with a telehealth parenting intervention for young children. Given the rise in telehealth treatment throughout the COVID-19 pandemic (Cantor et al., 2022; Comer, 2021; Gurwitch et al., 2020), it is necessary to assess the reliability and validity of measures of satisfaction with telehealth interventions.

The current study addressed these significant gaps in the literature by examining the psychometric properties of the TAI in a sample of caregivers of preschoolers with DD from predominantly low-income and racial/ethnic minoritized backgrounds who participated in a randomized controlled trial examining Internet-delivered PCIT (iPCIT; Comer et al., 2015). Furthermore, the current study is the first, to our knowledge, to examine the reliability and validity of the Spanish language translation of the TAI. For reliability, we examined the internal consistency of the scale. Additionally, we assessed the convergent validity of the TAI in this sample by examining associations between the TAI and the Client Satisfaction Questionnaire-8 (CSQ-8; Nguyen et al., 1983), a commonly used generic measure of consumer satisfaction with therapy that is not specific to parenting intervention services, but has demonstrated validity in racial/ethnic minoritized and Spanish-speaking populations (Roberts et al., 1984; Roberts & Attkisson, 1983). Finally, to further examine validity, we measured the extent to which TAI scores were predicted by changes across treatment in caregiver parenting practices and child behavior.

Methods

Participants

Participants were primary caregivers and their children (mean age = 36.3 months) participating in a randomized controlled trial (Bagner et al., 2023) evaluating iPCIT for the treatment of behavioral challenges in young children with DD (see details below). Only families in the iPCIT (treatment) condition of the study were included in these analyses. Of the 75 families randomized in the trial to the iPCIT condition, 65 caregivers completed the TAI at the post-treatment assessment and were included in the present analyses.

Families met inclusion for the larger study if they had: (a) a 3-year-old child exiting Part C (of the Individuals With Disabilities Education Act) Early Intervention services for a developmental delay; (b) a clinically significant score (i.e., T -Score ≥ 60) on the Externalizing Problems scale of the Child Behavior Checklist (CBCL; Achenbach et al., 2001); and (c) the primary caregiver spoke English or Spanish. Exclusion criteria for the study were: (a) the child was receiving medication for problem behaviors; (b) the child was deaf, blind, or displayed severe physical impairments; (c) the child displayed severe social communication deficits (i.e., score > 75 on the Social Responsiveness Scale; Constantino & Gruber, 2012), although children with ASD were included; or (d) the primary caregiver scored a standard score of < 4 on the vocabulary subtest of the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999) for English-speaking families or La Escala de Inteligencia Wechsler Para Adultos—Third Edition (EIWA-III; Pons et al., 2008) for Spanish-speaking families. While allowing families of caregivers with very low scores on verbal intelligence testing to participate may have increased generaliz-ability, such families were excluded in this study to ensure that variability in outcomes was not due to differences across caregivers in cognitive functioning. In addition, the assessments for this research study required families to be able to read and comprehend questionnaires in either English or Spanish, and thus this exclusion criterion ensured all participating families could engage with the research demands of the study. Only 10 families, out of 683 assessed for eligibility, were excluded due to low caregiver scores on the WASI or EIWA-III. For further information about study exclusion procedures, see the Consort diagram in Bagner et al. (2023).

Demographic characteristics of the sample are presented in Table 1. Among these families, 95% of primary caregivers reported their gender as female, and 69% (n = 45) reported their ethnicity as Hispanic/Latinx. Of the caregivers who reported their ethnicity as Hispanic/Latinx, 95.6% (n = 43) reported their race as White, 2.2% (n = 1) reported their race as Black or African American, and 2.2% (n = 1) did not report an additional race. Of the 31% (n = 20) of caregivers who reported their ethnicity as non-Hispanic/Latinx, 60% (n = 12) reported their race as Black or African American, 35% (n = 7) reported their race as White, and 5% (n = 1) reported their race as Asian. Additionally, the median family income was $30,000, with 62% of the sample reporting a low or impoverished income as defined by income-need ratios (a family’s household income divided by the Federal Poverty Threshold for a household of their size (United States Department of Health and Human Services)).

Table 1.

Demographic Characteristics

Variable n (%) Relation to TAI scores
Caregiver Race/Ethnicity - -
 Hispanic/Latinx 45 (69) − 0.31*
 Non-Hispanic, non-Latinx White 7 (11) − 0.24
 Black or African American 13 (20) − 0.12
 Asian 1 (1.5) − 0.06
 Not reported 1 (1.5) 0.08
Caregiver Gender - − 0.05
 Female 62 (95) -
 Male 3 (5) -
Caregiver Primary Language - 0.13
 English 36 (55) -
 Spanish 29 (45) -
Family Income-Need Ratios - − 0.09
 Extreme Poverty 3 (5) -
 Poor 16 (25) -
 Low Income 21 (32) -
 Adequate 11 (17) -
 Affluent 10 (15) -
 Missing 4 (6) -
Open in a new tab
*

correlation significant at the p = .05 level;

**

correlation significant at the p = .01 level

a.

Note: Categories for race/ethnicity equal to > 100% because categories are not mutually exclusive

Measures

Therapy Attitude Inventory

The TAI (Eyberg, 1993; Eyberg & Johnson, 1974) is a 10-item caregiver-report measure assessing satisfaction with parenting interventions. Items assess caregiver beliefs about the impact of treatment on their parenting behaviors and their child’s behavioral challenges, as well as their general satisfaction with treatment. For each item, caregivers rate their level of satisfaction on a scale from 1 (dissatisfied with treatment, or problems worsened) to 5 (maximum satisfaction with treatment, or problems improved). Item responses are summed to yield a total score (range: 10 to 50). The TAI has demonstrated excellent psychometric properties with caregivers of typically developing youth (Brestan et al., 1999).

Client Satisfaction Questionnaire – 8

The CSQ-8 (Nguyen et al., 1983) is a well-established, brief (8-item) generic measure of satisfaction with services of any kind. The CSQ-8 has demonstrated strong psychometric properties, including high internal consistency and test-retest reliability among racially diverse populations and in both English and Spanish (Roberts et al., 1983, 1984). The CSQ-8 demonstrated excellent internal consistency (α = 0.93) in the current sample.

Eyberg Child Behavior Inventory

The Eyberg Child Behavior Inventory (ECBI; Eyberg & Pincus, 1999) is a 36-item caregiver report measure of child behavior challenges. For each item on the Intensity scale, which we used in this study, caregivers rate the frequency with which their child exhibits a given behavior on a scale from 1 (never) to 7 (always) to measure the frequency of disruptive behaviors. The ECBI Intensity scale has demonstrated excellent reliability and validity in prior research (e.g., Butler et al., 2008) and in the current sample (α = 0.92).

Dyadic Parent-Child Interaction Coding System

The Dyadic Parent-Child Interaction Coding System-Fourth Edition (DPICS; Eyberg et al., 2013) was used to code caregiver and child behaviors during a 5-minute child-led play and clean-up interactions between the child and primary caregiver at all assessments. The DPICS has shown well documented reliability and validity (Eyberg et al., 2004). Positive parenting behaviors were assessed by the frequency of parenting “do” skills (i.e., labeled praises, behavior descriptions, and reflections) during the 5-minute child-led play interaction, whereas controlling/critical parenting behaviors were assessed by the frequency of “don’t” skills (commands, questions, and criticisms) during the same interaction. Child compliance (beta) was measured by calculating the number of caregiver commands with which the child complied during the 5-minute clean-up task divided by the total number of commands from the caregiver (including commands to which the child complied, commands to which the child did not comply, and commands to which there was not an opportunity to comply). Graduate student coders, masked to treatment condition, were trained to 80% agreement with criterion recordings. A random 20% of the video recordings of participants in the present trial were coded a second time to assess inter-coder reliability (mean Kappa across codes = 0.77).

Parenting Practices Inventory

The Parenting Practices Inventory (PPI; Webster-Stratton et al., 2001) is a caregiver self-report measure of parenting strategies and styles. Caregivers are provided with hypothetical situations (e.g., if your child hit another child…) and asked to rate the likelihood that they would respond in various ways. Additionally, caregivers rate how often they respond with certain discipline and parenting strategies and rate their agreement with statements about the use of parenting strategies such as praise, rewards, and rule-setting. Scores on this measure are divided into subscales for different categories of parenting strategies. In this study, we used the Harsh and Inconsistent Discipline subscale (α = 0.86).

Family Impact Questionnaire

The Family Impact Questionnaire (FIQ; Donenberg & Baker, 1993) assesses the extent to which children’s behavior challenges impact various domains of family functioning and has demonstrated excellent reliability. In this study we included the subscale measuring negative impact of child behavioral challenges on feelings towards parenting. Higher scores indicate higher levels of child impact on various aspects of family life. The scale demonstrated acceptable internal consistency in this sample (α = 0.73).

Procedures

All procedures were approved by the university Institutional Review Board. After screening into the study, families completed an initial baseline assessment consisting of a series of caregiver questionnaires, including the ECBI, PPI, and FIQ, and structured home-based observations of caregiver-child interactions using the DPICS. After completing the baseline assessment, families were randomized to receive either iPCIT (n = 75) or referrals as usual (RAU; n = 75). In the RAU control condition, families continued receiving usual services and referrals from their Early Intervention exit interview but did not receive iPCIT. Families in the treatment group received iPCIT, a telehealth adaptation of PCIT that uses a video teleconferencing platform to remotely deliver care to the home (Comer et al., 2017). PCIT consists of two stages of treatment: Child-Directed Interaction (CDI) and Parent-Directed Interaction (PDI). In CDI, the first stage of PCIT, parents are coached to increase warmth and positivity in the parent-child relationship through their use “PRIDE” skills (praising the child, reflecting child statements, imitating the child’s play, describing the child’s actions, and showing enjoyment). In PDI, the second stage of PCIT, parents learn effective discipline strategies and time out procedures. In iPCIT, live feedback and coaching is delivered via a caregiver-worn Bluetooth earpiece (Comer et al., 2015).

In the iPCIT group, families had 20 weeks to complete as many sessions of iPCIT as possible/needed. On average, families completed 10.20 sessions of iPCIT. After 20 weeks, all families were invited to complete a post-treatment assessment that also consisted of the caregiver questionnaires and structured home-based assessment of caregiver-child interactions administered at baseline. At this post-treatment assessment, caregivers also completed the TAI and CSQ-8, which were not administered at baseline.

Data Analysis Plan

Prior to analysis, missingness was assessed. Fewer families completed the DPICS assessment than the online surveys at the post-treatment assessment (n = 57). Thus, for analyses involving DPICS codes, listwise deletion was used for families with missing DPICS data. Additionally, demographic data and relations to the TAI were assessed, and an ANCOVA was conducted to examine differences in overall satisfaction between families who completed the TAI in Spanish (n = 29) versus English (n = 36).

To assess internal consistency, the Cronbach’s α coefficient was calculated with the full sample, as well as separately for caregivers who completed the TAI in English vs. Spanish. To assess convergent validity, a regression coefficient was computed for the relation between the TAI and the CSQ-8. Language was assessed as a moderator of this relation to test whether TAI convergent validity varied between groups. Additionally, given the relation between satisfaction and treatment engagement (Attkisson & Zwick, 1982), the association between session attendance (i.e., number of sessions attended) and TAI scores in this sample was assessed. To further examine the validity of the TAI, linear regressions were conducted assessing the extent to which TAI scores were predicted by changes in relevant treatment outcomes from baseline to post-treatment. Specifically, TAI scores were regressed on post-treatment caregiver report of child behavior challenges (measured via ECBI) and observed child compliance (measured via DPICS compliance score), controlling for pre-treatment scores. We similarly assessed relations between TAI scores and changes across treatment in observed and self-reported parenting behaviors (measured via DPICS do and don’t skills and the PPI Harsh and Inconsistent Discipline subscale). For analyses with DPICS scores, regression models further controlled for total number of caregiver verbalizations during the 5-minute play task to account for differences in overall talkativeness across the sample. Finally, we similarly assessed the association between TAI scores and changes in caregiver reported strain and attitudes towards parenting as measured by the FIQ negative feelings towards parenting subscales. All analyses were conducted in SPSS version 28 (IBM Corp., 2021).

Results

Preliminary Findings

Overall, caregivers reported high levels of treatment satisfaction on the TAI (M = 44.45 out of total possible 50). TAI scores did not vary significantly based on primary caregiver gender, language, or race (see Table 1). However, there was a significant relation between TAI scores and caregiver ethnicity, such that caregivers who reported their ethnicity as Hispanic/Latinx reported higher scores on the TAI, r = .31, p < .05. Thus, caregiver ethnicity was included as a covariate in all subsequent analyses. An ANCOVA comparing iPCIT treatment satisfaction reported by caregivers completing the English-language TAI versus the Spanish-language TAI (controlling for caregiver ethnicity) found no significant differences, F(1,62) = 0.04, p > .05.

Reliability

TAI internal consistency was excellent in the full sample (α = 0.92). Furthermore, internal consistency was similar for the English-language TAI (α = 0.92) and for the Spanish-language TAI (α = 0.90).

Validity

Results for all validity analyses are presented in Table 2. Convergent validity was examined by assessing the relation between TAI and CSQ scores at post-treatment. A strong positive relation was observed in the full sample, b = 0.80, t(63) = 3.92, p < .001, and this relation was not moderated by the language in which caregivers completed the questionnaires, b = − 0.17, t(63) = −1.12, p > .05.

Table 2.

Validity of the TAI

Measure Mean (SD) Baseline Mean (SD) Post-Treatment Relation to TAI
Convergent Validity
 Client Satisfaction Questionnaire - 28.74 (4.23) 0.80***
 Number of sessions attended - 10.20 (5.46) 0.23
Predictive Validity
 ECBI Intensity T-Score 58.12 (9.38) 48.98 (9.44) − 0.27*
 DPICS Compliance 0.25 (0.25) 0.16 (0.26) 10.9†
 DPICS Do Skills 4.16 (4.06) 7.85 (7.40) 0.22*
 DPICS Don’t Skills 31.02 (24.64) 21.25 (16.34) − 0.13*
 PPI Harsh & Inconsistent Discipline 3.04 (0.93) 2.52 (0.68) − 3.02†
 FIQ Negative Feelings Towards Parenting 11.98 (4.43) 9.25 (4.30) − 0.47**
Open in a new tab
*

correlation significant at the p = .05 level;

**

correlation significant at the p = .01 level;

***

correlation significant at the p = .001 level,

†

regression coefficient significant at the p = .01 level

Given the similarity in associations across caregivers completing forms in English versus Spanish, further validity tests were assessed with the combined sample (see Table 2). To assess whether TAI scores were associated with known correlates of treatment satisfaction, we first assessed whether number of sessions attended predicted TAI-reported treatment satisfaction. The positive relation between number of sessions attended and TAI scores was not statistically significant (p = .07). To examine whether TAI scores were predicted by treatment-related changes in child behavioral challenges, TAI scores were regressed on post-treatment ECBI Intensity scores, controlling for baseline ECBI scores. Lower post-treatment ECBI scores significantly predicted higher TAI scores, b = − 0.27, t(64) = −2.03, p < .05. We also examined improvements in observed child compliance following treatment as predictors of TAI scores. TAI scores were regressed on post-treatment DPICS observer ratings of child compliance, controlling for baseline ratings of compliance. TAI scores were positively related to post-treatment DPICS compliance ratings, controlling for baseline compliance ratings, b = 10.90, t(57) = 2.92, p < .01.

Associations between TAI scores and changes in parenting skills were also evaluated by assessing relations between TAI scores and DPICS do and don’t skills and the PPI Harsh and Inconsistent Discipline subscale. First, TAI scores were regressed on DPICS do skills at post-treatment, controlling for baseline do skills, as well as total verbalizations at post-treatment. TAI scores were significantly predicted by improvements in observer-rated do skills at post-treatment, b = 0.22, t(57) = 2.02, p < .05. TAI scores were also regressed on DPICS don’t skills at post-treatment, controlling for baseline don’t skills, as well as total verbalizations at post-treatment. TAI scores were significantly predicted by reductions in observer rated don’t skills at post-treatment, b = − 0.13, t(57) = −2.24, p < .05. Lastly, TAI scores were regressed on PPI Harsh and Inconsistent Discipline scores at post-treatment, controlling for baseline Harsh and Inconsistent Discipline scores. Greater reductions in caregiver report of harsh and inconsistent discipline predicted higher TAI scores, b = −3.02, t(63) = −2.82, p < .01.

The final validity test examined the extent to which TAI scores were predicted by reductions across treatment in caregiver strain. TAI scores were regressed on post-treatment FIQ scores on the negative feelings towards parenting subscale, controlling for scores at baseline. TAI scores were significantly predicted by reductions in caregiver-reported negative feelings towards parenting, b = − 0.47, t(64) = −2.66, p = .01.

Discussion

The current study examined the reliability and validity of the TAI, a measure of satisfaction with parenting interventions, in caregivers of children with DD from predominantly racial, ethnic, and language minoritized backgrounds participating in iPCIT. Results build on initial psychometric studies of the TAI (Brestan et al., 1999) and provide broadened evidence for the reliability and validity of the TAI in both English and Spanish languages. Specifically, the internal consistency of the scale was excellent, and nearly identical for both English and Spanish versions. Additionally, the TAI demonstrated strong convergent validity with the CSQ-8, a well-established generic measure of client satisfaction with any form of treatment that has demonstrated strong psychometric properties in Spanish-speaking and racial/ethnic minoritized populations (Roberts et al., 1983, 1984). Importantly, this relation was not moderated by language, suggesting that both the Spanish and English versions of the TAI demonstrate strong convergent validity with the CSQ. Given the excellent reliability and convergent validity demonstrated for the Spanish version of the TAI, along with the fact that no significant differences in satisfaction scores were observed between families who completed the survey in English versus Spanish, this study demonstrates the first support for the reliability and validity of the Spanish version of the TAI. Collectively, these study findings extend support for the TAI as a tool with which to assess caregiver satisfaction with parenting interventions, and meaningfully broaden the range of families with which the measure can be used with confidence. Further support for the validity of the TAI when used across diverse families was underscored by the extent to which TAI scores were predicted by a variety of treatment process variables and treatment outcomes. Higher TAI scores were significantly related to caregiver-reported and observed reductions across treatment in child behavioral challenges, harsh and inconsistent parenting behaviors, and caregiver strain. Additionally, higher TAI scores were predicted by observed improvements across treatment in positive parenting behaviors and child compliance to commands. These results are consistent with prior TAI research conducted in more homogenous samples of typically developing youth (Brestan et al., 1999) in finding that, as predicted, positive treatment engagement and clinical improvements are linked with higher treatment satisfaction as measured by the TAI. Overall, these findings support the TAI for broadened use with families of children with DD, families from racial/ethnic minoritized backgrounds, families that speak Spanish, and families participating in telehealth parenting interventions such as iPCIT.

Limitations

The results of this study should be interpreted in the context of some limitations. First, the sample size was relatively small, and thus continued work is needed to replicate the presently observed associations in samples providing greater power. Relatedly, the sample size was not sufficiently powered to conduct a confirmatory factor analysis (CFA), given that 5 to 20 cases per parameter estimate are suggested for CFA (Suhr, 2006). Thus, further research is needed to confirm the factor structure of the scale, particularly for the Spanish version. Additionally, we did not find a significant relation between session attendance and TAI scores despite prior work suggesting that session attendance is typically associated with greater treatment satisfaction (e.g., Attkisson & Zwick, 1982). There was a low positive relation between TAI scores and sessions attended which approached significance (p = .07). Thus, a larger sample may also have been more adequately powered to assess this relation.

Second, the TAI was only administered at the post-treatment assessment in the RCT from which the present data were drawn, and thus it was not possible to assess the test-retest reliability of the measure. Although the present study was not designed to evaluate the reliability and validity of the TAI as a mid-treatment measure of satisfaction, in many clinical contexts, it is also important to assess treatment satisfaction as an ongoing factor throughout the course of treatment. Assessing treatment satisfaction earlier in treatment may allow for the identification of families at risk for prematurely discontinuing treatment due to low satisfaction. It will be important for future research to include the TAI at multiple time points to determine the utility of the TAI in predicting treatment retention/dropout.

Next, this study was the first psychometric examination of the TAI in the unique context of a telehealth-based treatment. Continued research in this area should consider whether treatment satisfaction, as measured by the TAI, is linked with key factors specifically relevant to telehealth care, such as technology-based treatment disruptions and overall caregiver technological literacy (e.g., see Harkey et al., 2020). Additionally, future research should examine the extent to which post-treatment TAI scores predict the subsequent long-term maintenance of gains and across time. Lastly, one aspect of treatment satisfaction not assessed by the TAI is satisfaction with the therapist and/or strength of the therapeutic relationship. This limitation is underscored by prior research with samples of predominantly Latinx families receiving services for child DD or disability, who, when interviewed, often discussed their relationship with the therapist and the competence of the therapist as important indicators of their satisfaction with services (Bailey Jr et al., 1999).

Additionally, there are limitations related to the generality of the sample which should be considered. First, all children in the present study were recruited around the time of their third birthday, as part of eligibility criteria for the larger trial from which the present data were drawn. Behavioral parenting interventions are recommended for children and adolescents across a wide age range (see, Bagner et al., 2016; Comer et al., 2013; Griffith, 2010). It will be important for continued studies of the TAI to evaluate the psychometric properties of the scale in samples with wider age ranges. Second, we did not examine psychometric differences in satisfaction based on type of DD. Most children in this sample received early intervention for a language delay, but some had co-occurring delays and/or diagnoses (e.g., autism spectrum disorder). It is possible that the reliability or validity of the TAI is not uniform across families of children with different presenting challenges, which could be assessed in future psychometric work on the TAI. Additionally, while the sample included in this study was diverse and representative of the population of the region in which the study was conducted, the numbers of participants from certain ethnic/racial groups (e.g., Asian, Native American) were low, suggesting that further research is needed to assess the psychometric properties of this scale in these groups. Finally, given that there is significant variation among individuals within ethnic/racial groups, it may be important for future research to assess additional individual factors, such as acculturation, which may be associated with psychometric differences on the TAI.

Conclusions

Despite these limitations, the current study makes several contributions to the literature regarding the assessment of satisfaction with behavioral parenting interventions for young children. First, the current study was the first to examine the reliability and validity of the TAI in a sample of caregivers of children with DD, as well as in caregivers from predominantly minoritized Hispanic/Latinx backgrounds. Additionally, this study was the first to provide evidence for the reliability and validity of the Spanish version of the TAI. Finally, the current study was the first to assess the psychometric properties of the TAI in the context of telehealth care. Given expanding use of behavioral parenting interventions for child behavioral challenges for children with DD and recent adaptations of such treatments for individuals from minoritized, and Spanish-speaking, backgrounds, it is important to have reliable and valid tools with which to assess satisfaction for these populations and inform more patient-centered care. Furthermore, with recent shifts to telehealth modalities (Comer et al., 2021; Sullivan et al., 2021), it is also important to ensure that measures of satisfaction adequately assess satisfaction with telehealth treatments for child behavioral challenges. The current study offers initial support for the use of the TAI with caregivers of children with DD from racial/ethnic minoritized backgrounds and following telehealth treatment. Furthermore, this study offers initial support for the reliability and validity of the Spanish language version of the TAI.

Acknowledgements

This work was supported by a research project grant from the National Institute of Child Health and Human Development (NICHD) to the third and fourth authors (R01HD084497). We thank all the families who participated in this research.

The data presented in this manuscript come from a research project grant (R01HD084497) entitled “Advancing Child Competencies by Extending Supported Services (ACCESS) for Families Program” which was provided to Drs. Bagner and Comer by the National Institute of Child Health and Human Development (NICHD). The data presented here represent a secondary data analysis from this project. Materials and analysis code for this study are not available. However, further information about the larger project can be found at the ClinicalTrials.gov Identifier: NCT03260816, where the study is registered. Portions of the data in this manuscript have been accepted for presentation at the Association for Behavioral and Cognitive Therapies Annual Convention in November 2023.

Competing Interests

Dr. Bagner reported receiving grants from the National Institutes of Health and personal fees from PCIT International to train clinicians in PCIT supported by a grant from the Florida Department of Children and Families outside the submitted work. Dr. Comer reported receiving grants from the National Institutes of Health; personal fees from Macmillan Learning for textbook royalties; and personal fees from Association for Behavioral and Cognitive Therapies for an editorial stipend outside the submitted work. No other disclosures were reported.

Footnotes

Ethics Approval and Consent to Participate This study was performed in line with the principles of the Declaration of Helsinki. Approval was obtained from the Institutional Review Board (IRB) of Florida International University.

Consent to Participate Informed consent, which was approved by the IRB, was obtained from all caregivers prior to screening.

Data Availability

The data presented here represent a secondary data analysis from a larger project led by Drs. Bagner and Comer (R01HD084497). Materials and analysis code for this study are not available. However, further information about the larger project can be found at the ClinicalTrials.gov Identifier: NCT03260816, where the study is registered.

References

  1. Achenbach TM, Dumenci L, & Rescorla LA (2001). Ratings of relations between DSM-IV diagnostic categories and items of the CBCL/6–18, TRF, and YSR (pp. 1–9). University of Vermont. [Google Scholar]
  2. Attkisson CC, & Zwick R (1982). The client satisfaction questionnaire: Psychometric properties and correlations with service utilization and psychotherapy outcome. Evaluation and Program Planning, 5(3), 233–237. [DOI] [PubMed] [Google Scholar]
  3. Bagner DM, & Eyberg SM (2007). Parent–child interaction therapy for disruptive behavior in children with mental retardation: A randomized controlled trial. Journal of Clinical Child and Adolescent Psychology, 36(3), 418–429. [DOI] [PubMed] [Google Scholar]
  4. Bagner DM, Sheinkopf SJ, Vohr BR, & Lester BM (2010). Parenting intervention for externalizing behavior problems in children born premature: An initial examination. Journal of Developmental and Behavioral Pediatrics: JDBP, 31(3), 209. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bagner DM, Coxe S, Hungerford GM, Garcia D, Barroso NE, Hernandez J, & Rosa-Olivares J (2016). Behavioral parent training in infancy: A window of opportunity for high-risk families. Journal of Abnormal Child Psychology, 44, 901–912. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bagner DM, Berkovits MD, Coxe S, Frech N, Garcia D, Golik A, & Comer JS (2023). Telehealth Treatment of Behavior Problems in Young Children With Developmental Delay: A Randomized Clinical Trial. JAMA pediatrics. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bailey DB Jr., Skinner D, Rodriguez P, Gut D, & Correa V (1999). Awareness, use, and satisfaction with services for latino parents of young children with disabilities. Exceptional Children, 65(3), 367–381. [Google Scholar]
  8. Borrego J Jr., Anhalt K, Terao SY, Vargas EC, & Urquiza AJ (2006). Parent-child interaction therapy with a spanish-speaking family. Cognitive and Behavioral Practice, 13(2), 121–133. [Google Scholar]
  9. Brestan EV, Jacobs JR, Rayfield AD, & Eyberg SM (1999). A consumer satisfaction measure for parent-child treatments and its relation to measures of child behavior change. Behavior Therapy, 30(1), 17–30. [Google Scholar]
  10. Butler AM, Brestan EV, & Eyberg SM (2008). Examination of the Eyberg child behavior inventory discrepancy hypothesis. Child & Family Behavior Therapy, 30(3), 257–262. [Google Scholar]
  11. Cantor J, McBain RK, Kofner A, Hanson R, Stein BD, & Yu H (2022). Telehealth adoption by mental health and substance use disorder treatment facilities in the COVID-19 pandemic. Psychiatric Services, 73(4), 411–417. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Comer JS (2021). Rebooting mental health care delivery for the COVID-19 pandemic (and beyond): Guiding cautions as telehealth enters the clinical mainstream. Cognitive and Behavioral Practice, 28(4), 743–748. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Comer JS, Chow C, Chan PT, Cooper-Vince C, & Wilson LA (2013). Psychosocial treatment efficacy for disruptive behavior problems in very young children: A meta-analytic examination. Journal of the American Academy of Child & Adolescent Psychiatry, 52(1), 26–36. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Comer JS, Furr JM, Cooper-Vince C, Madigan RJ, Chow C, Chan PT, & Eyberg SM (2015). Rationale and considerations for the internet-based delivery of parent–child interaction therapy. Cognitive and Behavioral Practice, 22(3), 302–316. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Comer JS, Furr JM, Miguel EM, Cooper-Vince CE, Carpenter AL, Elkins RM, & Chase R (2017). Remotely delivering real-time parent training to the home: An initial randomized trial of internet-delivered parent–child interaction therapy (I-PCIT). Journal of Consulting and Clinical Psychology, 85(9), 909. [DOI] [PubMed] [Google Scholar]
  16. Comer JS, Furr JM, Busto D, Silva C, Hong K, Poznanski N, & Puliafico B,A (2021). Therapist-led, internet-delivered treatment for early child social anxiety: A waitlist-controlled evaluation of the iCALM telehealth program. Behavior Therapy, 52(5), 1171–1187. [DOI] [PubMed] [Google Scholar]
  17. Constantino JN, & Gruber CP (2012). Social responsiveness scale: SRS-2. Western Psychological Services Torrance. [Google Scholar]
  18. Copeland VC, Koeske G, & Greeno CG (2004). Child and mother client satisfaction questionnaire scores regarding mental health services: Race, age, and gender correlates. Research on Social Work Practice, 14(6), 434–442. [Google Scholar]
  19. Donenberg G, & Baker BL (1993). The impact of young children with externalizing behaviors on their families. Journal of Abnormal Child Psychology, 21, 179–198. [DOI] [PubMed] [Google Scholar]
  20. Eyberg SM (1993). Consumer satisfaction measures for assessing parent training programs. In VandeCreek L, Knapp S, & Jackson TL (Eds.), Innovations in clinical practice: A source book (Vol. 12). Professional Resource Press. [Google Scholar]
  21. Eyberg SM, & Johnson SM (1974). Multiple assessment of behavior modification with families: Effects of contingency contracting and order of treated problems. Journal of Consulting and Clinical Psychology, 42, 594–606. [DOI] [PubMed] [Google Scholar]
  22. Eyberg SM, & Pincus D (1999). Eyberg Child Behavior Inventory and Sutter Eyberg Student Behavior Inventory: Professional manual. Psychological Assessment Resources. [Google Scholar]
  23. Eyberg SM, Nelson MM, Duke M, & Boggs SR (2004). Manual for the dyadic parent-child interaction coding system third edition.
  24. Eyberg SM, Nelson MM, Ginn NC, Bhuiyan N, & Boggs SR (2013). Dyadic parent–child Interaction Coding System (DPICS) Comprehensive manual for research and training (4.). PCIT International. Gainesville, FL. [Google Scholar]
  25. Foote R, Eyberg S, & Schuhmann E (1998). Parent-child interaction approaches to the treatment of child behavior problems. Advances in Clinical Child Psychology: Volume, 20, 125–151. [Google Scholar]
  26. Griffith AK (2010). The Use of a behavioral parent training program for parents of adolescents. Journal of At-Risk Issues, 15(2), 1–8. [Google Scholar]
  27. Gurwitch RH, Salem H, Nelson MM, & Comer JS (2020). Leveraging parent–child interaction therapy and telehealth capacities to address the unique needs of young children during the COVID-19 public health crisis. Psychological Trauma: Theory Research Practice and Policy, 12(S1), S82. [DOI] [PubMed] [Google Scholar]
  28. Harkey LC, Jung SM, Newton ER, & Patterson A (2020). Patient satisfaction with telehealth in rural settings: A systematic review. International Journal of Telerehabilitation, 12(2), 53. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Hasler G, Moergeli H, Bachmann R, Lambreva E, Buddeberg C, & Schnyder U (2004). Client satisfaction with outclient psychiatric treatment: The role of diagnosis, pharmacotherapy, and perceived therapeutic change. The Canadian Journal of Psychiatry, 49(5), 315–321. [DOI] [PubMed] [Google Scholar]
  30. IBM Corp. Released 2021. IBM SPSS statistics for Windows, Version 28.0. Armonk, NY: IBM Corp. [Google Scholar]
  31. Kaminski JW, & Claussen AH (2017). Evidence base update for psychosocial treatments for disruptive behaviors in children. Journal of Clinical Child & Adolescent Psychology, 46(4), 477–499. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Lebow J (1982). Consumer satisfaction with mental health treatment. Psychological Bulletin, 91(2), 244. [PubMed] [Google Scholar]
  33. Matos M, Torres R, Santiago R, Jurado M, & Rodríguez IXA (2006). Adaptation of parent–child interaction therapy for Puerto Rican families: A preliminary study. Family Process, 45(2), 205–222. [DOI] [PubMed] [Google Scholar]
  34. McCabe K, Yeh M, Lau A, & Argote CB (2012). Parent-child interaction therapy for Mexican americans: Results of a pilot randomized clinical trial at follow-up. Behavior Therapy, 43(3), 606–618. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Moore KE, & Kenning M (1996). Assessing client satisfaction in a psychology training clinic. The Journal of Mental Health Administration, 23, 180–189. [DOI] [PubMed] [Google Scholar]
  36. Morgan PL, Farkas G, Hillemeier MM, & Maczuga S (2012). Are minoritized children disproportionately represented in early intervention and early childhood special education? Educational Researcher, 41(9), 339–351. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Neece CL, Green SA, & Baker BL (2012). Parenting stress and child behavior problems: A transactional relationship across time. American Journal on Intellectual and Developmental Disabilities, 117(1), 48–66. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Nguyen TD, Attkisson CC, & Stegner BL (1983). Assessment of client satisfaction: Development and refinement of a service evaluation questionnaire. Evaluation and Program Planning, 6(3–4), 299–313. [DOI] [PubMed] [Google Scholar]
  39. O’Reilly A, Donnelly A, Rogers J, Maloney O, O’Brien G, & Doyle E (2021). Measuring parent satisfaction in youth mental health services. Mental Health Review Journal, 26(3), 213–225. [Google Scholar]
  40. Petrenko CL (2013). A review of intervention programs to prevent and treat behavioral problems in young children with developmental disabilities. Journal of Developmental and Physical Disabilities, 25, 651–679. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Pons JI, Matías-Carrelo L, Rodríguez M, Rodríguez JM, Herrans LL, Jiménez ME, & Yang J (2008). Estudios De validez de la Escala De Inteligencia Wechsler para Adultos Versión III, Puerto Rico (EIWA-III). Revista Puertorriqueña De Psicología, 19, 75–111. [Google Scholar]
  42. Qi CH, & Kaiser AP (2003). Behavior problems of preschool children from low income families: Review of the literature. Topics in Early Childhood Special Education, 23. [Google Scholar]
  43. Ramos G, Blizzard AM, Barroso NE, & Bagner DM (2018). Parent training and skill acquisition and utilization among spanish-and English-speaking latino families. Journal of Child and Family Studies, 27, 268–279. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Rey JM, O’brien M, & Walter G (2002). Is the satisfied customer one who also ‘does well’? The relationship between outcome and parent satisfaction in a child and adolescent mental health service. Australasian Psychiatry, 10(3), 1–4. [Google Scholar]
  45. Roberts RE, & Attkisson CC (1983). Assessing client satisfaction among hispanics. Evaluation and Program Planning, 6(3–4), 401–413. [DOI] [PubMed] [Google Scholar]
  46. Roberts RE, Attkisson CC, & Stegner BL (1983). A client satisfaction scale suitable for use with hispanics? Hispanic Journal of Behavioral Sciences, 5(4), 461–475. [Google Scholar]
  47. Roberts RE, Atrkisson CC, & Mendias RM (1984). Assessing the client satisfaction questionnaire in English and Spanish. Hispanic Journal of Behavioral Sciences, 6(4), 385–396. [Google Scholar]
  48. Rosenberg SA, Zhang D, & Robinson CC (2008). Prevalence of developmental delays and participation in early intervention services for young children. Pediatrics, 121(6). [DOI] [PubMed] [Google Scholar]
  49. Ryan CL (2013). Language use in the United States: 2011. [Google Scholar]
  50. Schulte SJ, Meier PS, & Stirling J (2011). Dual diagnosis clients’ treatment satisfaction-a systematic review. BMC Psychiatry, 11(1), 1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Shapiro JP, Welker CJ, & Jacobson BJ (1997). The youth client satisfaction questionnaire: Development, construct validation, and factor structure. Journal of Clinical Child Psychology, 26(1), 87–98. [DOI] [PubMed] [Google Scholar]
  52. Stewart CD, Quinn A, Plever S, & Emmerson B (2009). Comparing cognitive behavior therapy, problem solving therapy, and treatment as usual in a high risk population. Suicide and Life-Threatening Behavior, 39(5), 538–547. [DOI] [PubMed] [Google Scholar]
  53. Suhr DD (2006). Exploratory or confirmatory factor analysis?.
  54. Sullivan AD, Forehand R, Acosta J, Parent J, Comer JS, Loiselle R, & Jones DJ (2021). COVID-19 and the acceleration of behavioral parent training telehealth: Current status and future directions. Cognitive and Behavioral Practice, 28(4), 618–629. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. United States Department of Health and Human Services. Poverty guidelines computations.
  56. Webster-Stratton C (2001). The incredible years: Parents, teachers, and children training series. Residential Treatment for Children & Youth, 18(3), 31–45. [Google Scholar]
  57. Webster-Stratton C, Reid MJ, & Hammond M (2001). Preventing conduct problems, promoting social competence: A parent and teacher training partnership in Head Start. Journal of Clinical Child Psychology, 30(3), 283–302. [DOI] [PubMed] [Google Scholar]
  58. Wechsler D (1999). Wechsler Abbreviated Scale of Intelligence (WASI). Psychological Corp. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

The data presented here represent a secondary data analysis from a larger project led by Drs. Bagner and Comer (R01HD084497). Materials and analysis code for this study are not available. However, further information about the larger project can be found at the ClinicalTrials.gov Identifier: NCT03260816, where the study is registered.

RESOURCES