Project ImPACT for Toddlers: Pilot Outcomes of Community Adaptation of an Intervention for Autism Risk

Abstract

This study reports child and family outcomes from a community-based, quasi-experimental pilot trial of Project ImPACT for Toddlers (PI^T). PI^T is a parent-mediated, naturalistic, developmental behavioral intervention for children with or at-risk for autism spectrum disorder developed through a research community partnership. Community early interventionists delivered either PI^T (n=10) or Usual Care (UC; n=9) to families based on Part C assigned provider. Twenty-five families participated, with children averaging 22.76 months old (SD=5.06). Family and child measures were collected at intake, after three months of service and after a three-month follow-up. Results indicate significantly greater improvements in positive parent-child interactions for PI^T than UC families, as well as large, but non-significant, effect sizes for PI^T families in children’s social and communication skills.

Lay Abstract

This study reports how well children and families did after participating in a community program as part of an early study of a intervention called Project ImPACT for Toddlers (PI^T). PI^T is a parent-led intervention with research support that can be done during regular activities. A group of community providers, parents and researchers worked together to help the intervention fit a community program for children with or at-risk for autism spectrum disorder. Community early interventionists delivered either PI^T (n=10) or did what they usually (Usual Care; UC; n=9) with families based on their training. Twenty-five families participated, with children averaging 22.76 months old. Parents and children completed some tests and play activities when they started the program, after three months of service and after a three-month follow-up. Results show greater improvements in positive parent-child interactions for PI^T than UC families, somewhat more change for PI^T families in children’s social and communication skills.

Childhood developmental, social and communicative challenges often indicate possibility of future identification of neurodevelopmental challenges such as Autism Spectrum Disorder (ASD¹; Landa, Holman, & Garrett-Mayer, 2007; Zwaigenbaum et al., 2005). These issues represent a significant public health challenge, with 1 in 4 children estimated to be at risk for developmental, social or behavioral delays (Altevogt, Hanson, & Leshner, 2008) which are associated with ongoing linguistic, educational and social difficulties (Briggs-Gowan & Carter, 2008; Stothard, Snowling, Bishop, Chipchase, & Kaplan, 1998). These toddlers are increasingly likely to receive an ASD diagnosis, which is rising at dramatic rates, with a current incidence of 1 in 59 children (Baio et al., 2018).

As ASD identification improves, communities are struggling to implement appropriate, effective programs for toddlers and their families. Although evidence-based practices for ASD exist (Wong et al., 2014), practices specifically designed for toddlers have not been implemented in community settings where children receive routine care. Additionally, in many early intervention programs, children later diagnosed with ASD receive services due to speech-language concerns exclusively, and many agencies wait until age three to establish a diagnosis (Yeates & Stahmer, 2017). This means that ASD-specific intervention may not be introduced until later in development than optimal, potentially due to concerns about identifying ASD incorrectly at a very young age (Rossi, Newschaffer, & Yudell, 2013). Further, the effectiveness of most interventions for children with early signs that may be indicative of ASD, but not a confirmed diagnosis, has not been established. On local and national levels, there are calls for capacity building for this population of young children with social communication challenges (Altevogt et al., 2008; Cordero et al., 2006; Zero to Three, 2005).

Addressing issues related to ASD concerns in early invention has the potential to produce positive downstream effects. In fact, intervening at the first signs of ASD, prior to a formal diagnosis, may be a powerful option and may reduce social communication challenges and make learning from social interaction easier for some children (Dawson, 2008; Lavelle et al., 2014). A focus on moving high quality, evidence-based intervention into community intervention settings where it can be delivered as early as possible may increase provider capacity to serve toddlers with early signs of ASD, improve outcomes for children and families, and reduce overall costs of supporting this population (Cidav et al., 2017).

Need for Evidence-based Practices in the Community

To achieve these goals, evidence-based practices (EBPs) specifically designed to meet the needs of toddlers with social communication delays or ASD and that are feasible to deliver within existing Individuals with Disabilities Education Act (IDEA) Part C early intervention services are required. Part C programs include publicly funded services provided to children under age 3 in the USA who have identified developmental concerns. Eligibility, identification of risk for ASD, and service provision vary widely by state and local agency. Fortunately, during the last decade we have learned a great deal about EBPs for toddlers with ASD. A review by leading experts in the field led to recommendations for best practices for young children with ASD or early ASD signs. These recommendations include: (1) a combination of developmental and behavioral strategies, delivered as early as possible; (2) active involvement of caregivers as part of the intervention; (3) a focus on social communication needs; and (4) individualization based on family needs and culture in clinical care (Zwaigenbaum et al., 2015). These recommendations are consistent with community-focused recommendations for the use of family centered help-giving practices (Dunst, Trivette, & Hamby, 2007) and fit with the mission of Part C services broadly, which is to enhance the development of infants and toddlers with disabilities and the capacity of families to meet the needs of their young children with disabilities (IDEA, 2004). In fact, including the caregiver is key to the mission of Part C because expected outcomes of early intervention include: (1) families understand their child’s strengths, abilities, and special needs; (2) families help their child develop and learn and (3) improvements in family quality of life (Bailey et al., 2006).

Despite wide agreement on how to best support toddlers with ASD, however, meeting these recommendations in practice remains challenging. Family involvement is not widely implemented in community settings, despite being a value and mandate of publicly implemented early intervention systems. For example, data indicate that a majority of Part C early intervention sessions involve the parent playing a passive rather than active role in services (Campbell & Sawyer, 2007). Current training for early intervention providers equips them to work directly with children presenting with general developmental delays, but not to build parent capacity, coach parents effectively (Fleming, Sawyer, & Campbell, 2011) or address learning needs of children with ASD specifically (Ingersoll, Meyer, Bonter, & Jelinek, 2012). Recently, the field of early intervention has begun to focus on how to optimally coach parents (Friedman, Woods, & Salisbury, 2012) as well as how to support children with early signs of ASD in the context of early intervention (Stahmer et al., 2018), but considerable work is needed in these areas to effectively move these shared values to routine community practice.

Current Study: Project ImPACT for Toddlers

The purpose of the current pilot study is to test the impact of community early intervention providers delivering a community adapted evidence-based, parent-coaching intervention. Specifically, the intervention, Project ImPACT for Toddlers (PI^T), represents an adaptation of Project ImPACT (Improving Parents as Communication Teachers; Ingersoll & Dvortcsak, 2010). Project ImPACT is an evidence-based, naturalistic developmental behavioral intervention (Ingersoll & Wainer, 2013; Schreibman et al., 2015) that focuses on equipping parents to support their child’s social and communication development in children 18 months to 8 years old, and is soundly supported in the scientific literature (Wong et al., 2014). PI^T was collaboratively adapted through a community-academic partnership consisting of a multidisciplinary group of providers, funding agency representatives, parents, and researchers, known as the BRIDGE Collaborative (BRIDGE = Bond, Regulate, Interact, Develop, Guide, Engage; (see Brookman-Frazee, Stahmer, Lews, Feder & Reed, 2012), to meet the needs of all stakeholders serving the youngest children with ASD and their families. A systematic, mixed-methods approach (Creswell & Creswell, 2017) was used to collect data to inform the adaptation of Project ImPACT for the community in a way that would also maintain the integrity of the intervention (see Rieth, Stahmer, & Brookman-Frazee, 2018 for details of the adaptation process). This type of intervention tailoring is recommended as a way to extend the reach of EBP into a community (Curran, Mukherjee, Allee, & Owen, 2008). Specific adaptations were made to address early intervention principles and practice recommendations, match community values, and to facilitate provider training in active parent coaching (See Table 1). The BRIDGE Collaborative, in coordination with the original Project ImPACT developers, created PI^T and associated intervention and training materials specifically for early intervention providers and families in the context of Part C services. See Table 2 for a description of the components of PI^T .

Table 1.

Project ImPACT for Toddlers Adaptations

Area Adapted	Description
Parent Manual	Editing of language and examples to be developmentally appropriate for toddlers
	Re-formatting and re-ordering strategies to increase ease of use for parents
	Development of examples and tools to support caregiver use the strategies during daily routines
	Removal of the word autism in all materials due to the young age of the children and common provisional or “at risk” diagnoses
Provider Training	Enhancement of training in how to effectively coach parents
	Increased attention to physiological regulation and individual sensory and motor differences in children
	Embedding principles of reflective practice at all levels of the program
Parent Engagement Strategy Additions	Adding content promoting therapist use of evidence-informed engagement strategies to build Alliance, Collaboration, and Empowerment (e.g., seeking and incorporating parent input, using partnership language, giving suggestions rather than directions; Haine-Schlagel, Martinez, Roesch, Bustos, & Janicki, 2016).

Table 2.

PI^T Fidelity Checklist: Domains, Items, and Descriptions

Items	Description
Get Ready
Limits distractions	The adult manages potential distractions and adjusts the environment accordingly (e.g., creates a clear space to interact, adjusts noise and lighting levels, clears extraneous toys).
Adjusts animation	The adult varies the gestures, vocal quality, and facial expressions according to the child’s arousal level (e.g., uses big, exaggerated gestures and an energetic tone of voice for a child with low energy/involvement)
Focus on Your Child
Lets child choose	The adult follows the child’s lead by allowing them to choose the toys/object/activity, location, length and manner of interaction
Stays face to face	The adult positions and moves him/herself to be face to face and at eye level with the child as much as possible
Makes comments	The uses statements and labels to describe the child’s actions, and the environment, and avoids asking questions
Imitates child	The adult consistently imitates child’s spontaneous behavior (including gestures, vocalizations, and facial expressions)
Models communication and play	The adult provides labels, description, and/or narration of the environment around the child’s focus of attention; the adult models simple play actions related to the child’s interests
Adjusts communication and play	The adult provides communication and play levels that are at or just above the child’s developmental level
Create Opportunities
Helps child anticipate interruptions	If necessary, the adult uses a consistent anticipatory phrase or gesture to indicate playful blocking or interruption of the child’s actions
Uses communication temptations	The adult creates opportunities for the child to respond through playful blocking, controlling access, creating silly situations, partner games, requiring the child to ask for help or protest, or other natural means
Wait
Waits	The adult waits for the child to react after providing an opportunity to respond, approximately 5–15 seconds, or until the child responds
Respond and Expand
Responds to spontaneous behavior	The adult interprets and acknowledges the majority of the child’s independent actions, verbalizations, and other behaviors as if they were purposeful and related to the environment/interaction
Expands communication and play	The adult consistently and naturally adds additional content to the child’s communication (additional words, sounds, gestures) and play (new objects, actions)
Help Your Child Succeed - Prompt
Provides prompts	The adult provides support for a more complex response in approximately 1/3 of opportunities (score drops for either too few or too many prompts provided)
Provides related prompts	The adult provides prompts that are naturally related to the on-going interaction and the child’s focus of interest
Provides clear prompts	The adult provides prompts that are at or just above the child’s developmental level; it is clear how the child is expected to respond
Adjusts prompt level	The adult varies the type of prompts provided based on the child’s response, increasing or decreasing level of support as needed
Respond and Expand
Provides natural and immediate rewards	The adult rewards the child’s appropriate behavior immediately, and the rewards are logically related to the on-going interaction
Only rewards appropriate behavior	The adult consistently follows through with requiring the child to respond before providing a reward; rewards are not provided for inappropriate behavior

The primary aim of this study (Aim 1) was to examine intervention effects of PI^T in community settings on parent and child factors from pre- to post- intervention. A secondary aim (Aim 2) was to examine the relationship between providers’ use of PI^T Parent Coaching strategies and parent fidelity of the intervention. A final, exploratory aim (Aim 3) was to examine the sustainment of intervention effects by examining changes in parent and child outcomes three months following the end of intervention. We hypothesized that parents in the PI^T group would be more likely to use PI^T strategies after receiving the intervention than parents in usual care, and that children in the PI^T group would make greater progress on social communication assessments than children in usual care. We expected that provider use of PI^T parent coaching strategies would be associated with parent fidelity and that the intervention changes for both children and parents would sustain after a three month follow up.

Methods

The study is a quasi-experimental pilot test of PI^T in community settings. Families enrolled in publicly funded (Part C) early intervention services in a large urban county received either PI^T or typically available early intervention services (usual care, UC). Part C funded agencies (n=8) enrolled in the study and provider (n=19) and family participants (n=28) were recruited from within enrolled agencies. Families were assigned to therapists per standard practice in their agencies (i.e. based on provider availability/caseload and language match). Family referral to agencies was determined by the Part C administrator, based on the usual considerations of availability, location, language preference, and other family needs. Intervention group was determined based on training status of the providers. This research was approved by the Institutional Review Board at the University of California, San Diego and relied upon by all other participating institutions.

Participants

Agencies and providers.

Agencies were recruited to enroll in the study based on existing relationships with the investigators, agency-initiated contact with the research team to express interest, and/or prior participation in earlier research projects related to early intervention with the same research group. All eight eligible agencies approached by the research team agreed to participate. Agency eligibility criteria included: 1) provides Part C-funded early intervention services to children identified as at-risk for ASD under 30 months; and 2) employs three or more providers working with parent/child dyads. Agency types included school-based early intervention (N=2), infant and early childhood services (N=1), ABA-based services (N=3), a Children’s Hospital ASD-specific clinic (N=1), and a federally qualified health center (N=1). All agencies provided Part C services through California Early Start.

In a partner project (Stahmer, Rieth, Dickson, Searcy, Feder & Brookman-Frazee, 2018), groups of community providers at participating agencies self-selected to receive training in PI^T from experienced clinical psychologists who were part of the PI^T adaptation team. Detailed information about the training process for PI^T as well as the adaptation is available in Reith, Stahmer & Brookman-Frazee, 2018. In brief, PI^T training for providers involved weekly, two-hour meetings across a period of 12 weeks. The content of the meetings alternated between didactic presentation of intervention content (including direct instruction to therapists, role-play activities among providers, and example videos) and hands-on practice of strategies with feedback from the trainer (i.e., volunteer children/parents attending sessions and providers taking turns in using the PI^T strategies with the child, explaining strategies to the parent, and/or coaching the parent directly). Agencies participating in the training project supported their providers time to attend the training, and training was provided free of charge. In the current study, the only requirement for providers was employment by a participating agency, agreement to have sessions filmed and willingness to refer new, eligible families to the research study as appropriate. Once enrolled, both the providers trained in PI^T (n=10 providers) and providers who had not received training (n=9 providers; UC) were asked to refer potentially eligible families for whom they were providing services. Agencies that included both trained and untrained providers were eligible to refer families for both the PI^T and UC groups (n=5 agencies, 62.5% of all participating agencies); however, only one agency provided family referrals to both groups. Three agencies referred to UC only (i.e., no PI^T trained providers at those agencies). The mean number of families completing treatment with each provider was 2.2 (range 1–4). As self-reported on a demographics questionnaire, providers’ primary discipline was behavior specialist (PI^T=4; UC=0), child development (PI^T=1; UC=1), early childhood education (PI^T=0; UC=4), psychology (PI^T=2; UC=2) and social work (PI^T=1; UC=2). Providers’ highest level of education was either a bachelor’s degree (PI^T=2; UC=4) or master’s degree and higher (PI^T=6; UC=3). PI^T and UC providers were similar in the number of years experience in early intervention (PI^T = 5.46, UC = 3.77) and number of years experience coaching parents (PI^T = 4.92, UC = 7.27); PI^T providers reported more experience working with individuals with ASD (PI^T = 6.88, UC =4.08, t(22)=−2.12, p=.05). When considering provider age, ethnicity, and education, no significant differences were found between the PI^T and UC groups

Parent-child dyads.

Forty-four potential family participants were referred from PI^T providers (n=19) and UC providers (n=25; see Figure 1). Inclusion criteria for parent/child dyads were: (1) child age 12–30 months at time of enrollment, (2) clinician has a current concern regarding ASD risk as defined by their Part C provider, (3) Score in the ‘concern’ range on one or more scales of the Communication and Symbolic Behavior Scales – Infant Toddler Checklist (4) English or Spanish speaking, and (5) parent consents to regularly participate in sessions with child. Exclusion criteria were (1) parent had not previously received PI^T, and (2) child was receiving 6 or more hours per week of early intervention services overall. This final criterion regarding early intervention service amount was included to keep service intensity similar across the intervention and control groups (i.e., avoid comparing children receiving low intensity parent-mediated intervention in PI^T [3 hours per week] with children receiving high intensity direct service from a clinician [e.g., 15–20 hours per week]).

Figure 1.

Enrollment Flow Diagram.

Of the referred families, 16 children were not enrolled in the study: five in the PI^T group and eleven in the UC Group. Four of these families did not consent to video recording (PI^T=1, UC=3), two families did not respond to multiple attempts to contact them (UC=2) and five families reported they did not have time to participate (PI^T=3, UC=2). Additionally, four families (UC=4) were receiving more than 6 hours of intervention service per week and one family (PI^T=1) had started receiving PI^T services prior to referral. Enrolled dyads included 28 children and their parents. Three enrolled families did not complete the study: Two families in PI^T ended participation early (one moved out of the area and the other initiated intensive intervention services, thus making them no longer eligible for the study). One family in the UC group ended participation early when their provider changed jobs. Overall, 12 child/parent dyads completed participation in the PI^T Group, and 13 completed participation in the UC Group. Six of these families who participated from intake to exit of treatment (PI^T=2; UC=4) did not complete measures at follow-up due being non-responsive or not interested in continuing participation.

Each parent-child dyad was either enrolled in the PI^T group (n=12) or the UC group (n=13) based on their providers’ training history. The mean age of participating children was 22.76 months (SD=5.06) and 68.0% were male. Children were ethnically diverse, with 48.0% identifying as Hispanic/Latinx per parent report. The average Early Learning Composite on the Mullen Scales was 72.14 (SD=16.97). Children received an average of 3.7 hours per week of intervention during the study. The biological mother was the respondent for 96% of the sample. Table 3 characterizes the sample across groups. There were no significant differences across groups, except that the PI^T Group included significantly more boys (n=11) than the UC group (n=6; [X² (1, N=25) = 5.94, p=.01]).

Table 3.

Characteristics of the Sample

	Project ImPACT for Toddlers (n = 12)	Part C Usual Care (n = 13)	Total Sample (n = 25)
	M (SD)	M (SD)	M (SD)
Child Age (months)	23.08 (4.31)	22.46 (5.82)	22.76 (5.06)
Child Sex male	11 (91.7)	6 (46.2)	17 (68.0)*
ADOS-T Overall Score	15.00 (5.06)	10.83 (8.90)	12.96 (7.56)
Social Affect Score	13.17 (4.95)	9.33 (7.25)	11.26 (6.52)
Restrictive/Repetitive Behavior	1.83 (1.64)	1.50 (1.93)	1.70 (1.80)
Mullen ELC	71.00 (13.83)	73.27 (20.26)	72.14 (16.97)
MB-CDI Total Score	34.09 (13.40)	36.23 (19.96)	35.25 (16.94)
Vineland ABC	89.18 (8.34)	88.75 (16.98)	88.96 (13.25)
Communication	81.36 (11.26)	86.42 (13.42)	84.00 (12.42)
Social	90.91 (9.49)	91.75 (15.39)	91.35 (12.63)
	n (%)	n (%)	n (%)
Parent Race/Ethnicity (May Select more than 1)
Caucasian/White	4 (33.3)	7 (53.8)	11 (44.0)
Hispanic/Latinx	6 (50.0)	6 (46.2)	12 (48.0)
Asian/Pacific Islander	3 (25.0)	0 (0.0)	3 (12.0)
Filipino American	0 (0)	1 (7.7)	1 (4.0)
African American	0 (0.0)	0 (0.0)	0 (0.0)
Multiracial	1 (8.3)	1 (7.7)	2 (8.0)
Other	4 (33.3)	6 (46.2)	10 (40.0)
Maternal Education
High School/GED	3 (25.0)	2 (15.4)	5 (20.0)
Associate’s Degree	1 (8.3)	2 (15.4)	3 (12.0)
Bachelor’s Degree	3 (25.0)	5 (38.5)	8 (32.0)
Master’s Degree	3 (25.0)	3 (23.1)	6 (24.0)
Doctoral Degree	1 (8.3)	0 (0.0)	1 (4.0)
Mother Employed Outside Home	7 (58.3)	9 (69.2)	17 (68.0)
Father Employed Outside Home	12 (100.0)	13 (100.0)	25 (100.0)
Service Language
Spanish	2 (16.7)	5 (38.5)	7 (28.0)
English	10 (83.3)	8 (61.5)	18 (72.0)
Family Receives Govt Assist	5 (41.7)	4 (30.8)	9 (36.0)
Number of Services	1.27 (1.10)	2.38 (1.56)	1.88 (1.45)
Service Intensity (hrs/week)	3.61 (4.37)	3.80 (4.23)	3.72 (4.20)

Note: ADOS ASD Severity = ADOS-2 Module T autism severity score; Mullen ELC = Mullen Scales of Early Learning Early Learning Composite Standard Score; MB-CDI= MacArthur Bates Communicative Development Inventory; Vineland ABS = Vineland Adaptive Behavior Scales, Second Edition Adaptive Behavior Composite Standard Score;

^*p<.05

Procedures

Recruitment

Children and families were recruited from eight community Part C agencies. Upon referral, families completed a phone screening. If they met inclusion criteria and the parent consented to participate, the family was enrolled and completed intake assessments (see Assessments). The parent completed a set of standardized questionnaires and video recorded a 10 minute play interaction with his/her child. After intake, the participating provider delivered either usual care or the intervention (see intervention descriptions, below). Video recordings of sessions were collected four times during the intervention period. Post intervention assessments were completed at the conclusion of PI^T or after approximately 3 months of participation (M=3.01; SD=1.38), to keep the timeline consistent across groups. Follow up data were collected approximately 3 months after post data (M=3.01; SD=.58). Each assessment appointment lasted approximately 1–2 hours and parents received $10 for completion of each appointment.

Project ImPACT for Toddlers (PI^T) group.

Children in the PI^T Group received early intervention services from providers who had received PI^T training within their agency. Agencies delivering PI^T had previously worked closely with the BRIDGE Collaborative to prepare one employee as an “Agency Trainer”. The trainer participated in a 12-week training with BRIDGE Collaborative experts on using PI^T with families as well as how to effectively train therapists in the model (training structure described above). Additional details on the training model for PI^T can be found in Rieth et al., 2018. PI^T services are designed to be delivered across 12 weeks and follow the structure of the PI^T parent manual, which is provided to parents at the start of the service. The manual describes each strategy and provides examples for how to use the technique with young children. PI^T services were provided to families through Part C either as general early intervention services (50% of families) or autism-focused services (50% of families). Two families received PI^T in a center-based program (16.67%), and 10 received it in a home-based program (83.33%). The average intensity of PI^T services was 2.5 hours per week (SD=.90).

Community comparison group.

Usual care providers were not trained in PI^T but were employed by participating community agencies. Providers delivered whatever intervention techniques and strategies they provided to families in routine care. Services in UC were provided either as general early intervention services (77% of families), autism-focused services (8% of families), or special education early intervention services (15% of families). All services in usual care were home-based. The average intensity of UC services was 2 hours per week (SD=1.83).

Assessments

Video Coding Procedures

For each measure requiring video coding, including Use of PI^T Parent Coaching Strategies, Parent Interactions with Children: Checklist of Observations Linked to Outcomes (PICCOLO), and Project ImPACT for Toddlers - Parent Intervention Fidelity, research assistants naïve to the training condition scored the measure based on the video recordings. Coders learned one coding system each. All coders met a reliability criterion of 80% agreement with coding keys across two separate video recordings prior to beginning independent coding. Ongoing agreement was evaluated throughout the coding process. Coders with two consecutive videos below 80% agreement received a didactic review of components or elements for which they were having difficulty and were required to code two reliability videos at 80% agreement or above again before further independent coding. Reliability data are reported for each measure below.

Sample characterization measures.

Demographics questionnaire.

Parents completed a demographics questionnaire at pre-intervention providing basic family and child information.

Early Intervention History Interview.

Parents were interviewed at pre- and post- intervention regarding the nature and history of early intervention received. The interviewer obtained information regarding the type of intervention, service provider, setting, duration and frequency of the intervention. This measure has been used by the Autism Centers for Excellence research studies.

Autism Diagnostic Observation Schedule, Second Edition – Toddler Module (ADOS-2; Lord, Luyster, Gotham & Guthrie, 2012).

The ADOS-2, completed at pre-intervention, is a standardized, observational measure of social and communication skills. It has been shown to have high reliability and discriminant validity (Lord, DiLavore, Risi, Gotham, & Bishop, 2012). The ADOS-2 largely operationalizes the process of identifying ASD symptoms. The total score, social affect score, and restricted and repetitive behavior scales were used to characterize the sample.

Mullen Scales of Early Leaning (MSEL; Mullen, 1995).

The MSEL, completed at pre-intervention, measures developmental level across domains. The MSEL can be administered to children from birth to 68 months of age. T scores, percentile ranks, and age equivalents can be computed for five scales separately (Gross Motor, Visual Reception, Fine Motor, Expressive Language, and Receptive Language; each scale has a mean of 50 and a standard deviation of 10). An overall standard score, the Early Learning Composite (ELC) can also be calculated (M=100, SD=15). Construct, concurrent, and criterion validity are all verified by independent studies and the technical manual for the MSEL (Mullen, 1995). The ELC was used to characterize children in the sample at intake.

Intervention content and process measure

Use of PI^T Parent Coaching Strategies was characterized across both groups (PI^T and UC) via the video recorded intervention sessions. Sessions were recorded three to four times per family (n=96 total videos, mean of 3.2 videos per family) either directly by the provider or by a member of the research team. All recorded sessions were behaviorally coded using a 22-item Likert scale measure to characterize the structure of the session and the provider’s approach to parent coaching. This measure was adapted from the Fidelity of Implementation for Coaching Sessions Form in the original Project ImPACT materials (p. 368, Ingersoll & Dvortscak, 2010). Each item was rated on a 1–5 Likert scale, where 1 indicated the provider did not use that strategy/element or did so poorly, and 5 indicated that the provider used that strategy/element competently and at an appropriate intensity. The measure includes items around setting up the coaching environment (e.g., limited distractions; availability of materials; presence of developmentally appropriate toys), use of coaching strategies (e.g., explaining the session content, reviewing prior weeks’ goals, explaining key technique accurately, briefly demonstrating strategies competently, facilitating parent practice of the technique, providing clear and specific parent feedback), use of strategies to promote parent engagement (e.g., inviting comments/questions, acknowledging parent concerns, assessing parent understanding, recognizing parent strengths and efforts) and use of general strategies to partner with caregiver (e.g., responsive interaction style; reflective approach). Complete definitions for the measure are available from the authors.

A total of 32% of all video samples (n = 31) were evaluated by two independent coders. Intraclass correlation coefficients were calculated for each item to assess interrater reliability. ICCs ranged from .69–1.0 across items, with a mean of .89 (SD=.08). Overall, the codes exhibited good to excellent ICCs according to accepted standards (Cicchetti, 1994).

Dependent measures: Parent.

Parent Participatory Engagement Measure (PPEM; Haine-Schlagel et al., 2016).

Parents completed this five-item questionnaire at the initial, middle and final intervention sessions with their provider to examine whether training in parent coaching and engagement strategies in PI^T would increase parent engagement in the intervention. The questions, regarding parent comfort with, as well as engagement and participation in intervention, are rated on a 1 (not at all) through 5 (very much) scale. The PPEM has demonstrated excellent internal consistency (Cronbach’s alpha estimates of .86 to .93 across subsamples; Haine-Schlagel et al., 2016). A total mean score was calculated across time points (initial, middle, and final session) for each parent and utilized in analyses.

Social Support Index (SSI; McCubbin, Patterson, & Glynn, 1982).

Parents completed the SSI at pre- and post- intervention, which measured their perceived support within the community. This tool was developed to measure family social support, as well as the amount of community-based social support families believe exist in the community which we hypothesized would be higher in the PI^T group due to provider training in coaching and engagement The SSI is comprised of 17 questions that ask respondents to indicate their agreement or disagreement with a statement on a five-point Likert scale (1-Strongly Disagree; 5-Strongly Agree). A higher score indicates a higher level of social support (maximum score of 85). The SSI has strong internal consistency, with an alpha across several samples of .82; it is also reported to have a test-retest stability correlation of .83 (Fischer, et al., 2007). Mean total score was calculated at each time point, with pre- and post- intervention scores utilized to examine differences in social support between groups.

Parent Interactions with Children: Checklist of Observations Linked to Outcomes (PICCOLO; Roggman et al., 2013).

The PICCOLO, conducted at pre- and post- intervention and 3- month follow-up, is an observational measure in which a parent child interaction is video recorded and trained observers code specific parenting behaviors known to predict children’s early social, cognitive, and language development. Specifically, the PICCOLO examines four domains of parenting behaviors, including affection, responsiveness, encouragement and teaching. We hoped this measure would provide an objective way to determine whether training in PI^T would change positive parent skills. Each domain is comprised of 7–8 individual items that are rated on a 0–2 scale, where 0 = Absent, 1 = Rarely/Briefly and 2 = Frequently. Higher scores are indicative of more developmentally supportive parent-child interactions. ICCs for individual items ranged from .73 to .82, with an average ICC of .77, and are considered good according to current standards (Cichetti, 1994). The PICCOLO demonstrates strong reliability and both construct and predictive validity. Mean scores for each domain (Affection, Responsiveness, Encouragement, & Teaching) were calculated at each designated time point and utilized in analyses.

Project ImPACT for Toddlers - Parent Intervention Fidelity.

(PI^T Fidelity). PI^T Fidelity definitions were used to evaluate parents’ use of PI^T components during parent-child interactions. Definitions were adapted from the original Project ImPACT Intervention Fidelity checklist (Ingersoll & Dvortcsak, 2010) to reflect PI^T adaptations. The PI^T fidelity measure includes 19 items that comprise seven composites reflective of PI^T strategies (see Table 2 for list of components and definitions). Items are rated on a 1 to 5 Likert scale (1-minimal or no use of the strategy throughout the observation; 5-correct use of the strategy at least 80% of the time throughout the entire observation). ICCs for individual items ranged from .61 to .94, with an average ICC of .80, and are considered good to excellent according to current standards (Cichetti, 1994). Mean scores for each of the seven composites and one overall score were calculated; parent post-intervention scores were utilized as predictor of child outcomes in analyses.

Dependent measures: Child.

Measures examining social communication in toddlers based on independent evaluation and parent report were chosen based on the skills targeted in the intervention.

Communication and Symbolic Behavior Scales - Infant Toddler Checklist (CSBS ITC; Wetherby & Prizant, 2002).

The CSBS ITC was completed at pre-, post-, and three-month follow-up of intervention. The CSBS ITC is a parent questionnaire used to determine risk for developmental disorders, including ASD. The CSBS ITC includes 24 questions reflecting four composites with 3 to 5 choices about developmental milestones of social communication. It is a standardized tool with screening cutoffs and standard scores for children 6 to 24 months based on a normative sample of 2,188 children (Wetherby & Prizant, 2002). It has good reliability and validity as well as high sensitivity and specificity (both 88.9%) for catching toddlers who are later diagnosed with ASD and other developmental delays from a general pediatric sample. Summative scores for the Social, Speech, and Symbolic domains as well as an overall score were calculated for each time point and utilized in analyses.

MacArthur-Bates Communicative Development Inventory (CDI; Fenson, et al., 2006).

Parents completed the CDI at pre-, post-and follow-up time points. The CDI assesses major features of communicative development, including vocabulary, understanding and use of gesture, and emergence and expansion of grammatical features. The CDI/Words and Gestures form is a vocabulary checklist structured so the parent can indicate what words the child understands and what words the child understands and says. This form also assesses what phrases the child understands and what play and other imitation skills s/he exhibits. Raw scores for the Words Understood, Words Produced, Early Gestures, and Late Gestures composites were calculated for each time point and utilized in analyses.

Vineland Adaptive Behavior Scales, Second Edition (VABS-II; Sparrow, Cicchetti, & Balla, 2005).

This assessment, completed at pre-, post- and follow-up, measures personal and social skills. It has been validated with children with developmental disabilities and is applicable to children from birth through 18 years, 11 months. Standardization included national samples of children with and without disabilities. The scales yield normative standard scores (M=100; SD=15) that indicate level of functioning and can be used for comparison across groups. Standard scores for the Communication and Socialization domains were utilized in analyses.

Data Analysis

Data analyses were conducted in Statistical Package for the Social Sciences (SPSS (v.25). Initially, associations between study condition and child and parent demographic and clinical characteristics at baseline were examined to address observed pre-treatment group differences. To compare differences in outcomes by study condition (Aim 1), we conducted linear regressions for end of treatment outcomes with intervention group as the primary predictor. Covariates included pre-treatment scores (for repeated measures) and a variable representing the number of children enrolled for each provider to address issues of nested data (with one or more child nested within provider). The inclusion of this later covariate is recommended to address issues of nesting when sample sizes limit the ability to include these as separate levels (McNeish & Wentzel, 2017). For analyses comparing mean differences in group regardless of time point (e.g., examining group differences in the PPEM), we utilized independent samples t-tests. Aim 1 analyses specifically focused intervention effects whereas Aim 2 examined impact of providers’ use of coaching strategies consistent with PI^T on parent outcomes, namely parent fidelity and parent-child interaction (PICCOLO scores). Lastly, due to smaller sample size and resulting impact on statistical power, at follow-up (PI^T n=10; UC n=9), effect sizes were calculated based on changes from pre-intervention to three-month follow-up for each group (Aim 3). Given the pilot nature of this work and limited statistical power, significant results (p<.05), marginal trends (p<.10) and non-significant effects with moderate to large effect sizes are discussed.

Results

Aim 1 Intervention Effects: Parent Outcomes

See Table 4 for a presentation of all parent outcome results.

Table 4.

Aim 1 Intervention Effects at Baseline (Pre) to End of Treatment (Post): Parent Outcomes

Parent Outcomes	pIT		UC		Regression Coefficients			Effect Sizes
Measure/Subscale	Pre M(SD) N=12	Post M(SD) N=11	Pre M(SD) N=13	Post M(SD) N=12	B	β	p	PIT ES	UC ES
SSI	3.86 (.51)	3.81 (.65)	4.37 (.42)	4.23 (.38)	−.11	−.10	.56	−.08	−.35
PPEM(Overall Mean Score) PICCOLO	4.08 (.66)		3.02 (1.12)		--	--	--	--	--
Affection	1.49(.24)	1.71(.23)	1.51(.18)	1.54(.20)	.21	.47	.03	.97	.14
Response	1.38(.37)	1.83(.27)	1.47(.30)	1.27(.25)	.63	.84	<.01	1.33	−.66
Encourage	1.42(.22)	1.77(.16)	1.51(.22)	1.55(.26)	.28	.59	<.01	1.88	.16
Teach	1.01(.34)	1.25(.28)	1.18(.33)	1.03(.23)	.24	.45	.05	.83	−.48
Fidelity (average)
Get Ready	3.54(.78)	4.14(.78)	3.65(.75)	4.27(.79)	−.03	−.02	.94	.73	.70
Focus	3.08(.51)	3.65(.64)	3.00(.40)	3.30(.37)	.28	.27	.24	.94	.72
Create	2.96(.94)	3.23(.72)	2.69(.85)	2.86(1.16)	.32	.17	.47	.30	.16
Wait	3.25(.75)	4.00(1.10)	3.15(.90)	3.55(1.13)	.44	.21	.37	.77	.36
Respond	2.96(.78)	3.72(.98)	3.15(.24)	3.55(.79)	.27	.16	.51	.83	.65
Prompt	2.56(1.02)	3.72(.83)	2.73(.96)	3.04(.65)	.68	.44	.05	1.19	.35
Reward	2.96(1.39)	3.95(.79)	2.92(1.04)	4.10(.77)	−.14	−.09	.70	.83	1.18
Overall	2.94(.60)	3.73(.67)	2.97(.42)	3.45(.38)	.25	.23	.31	1.18	1.10

Abbreviations: PI^T = Project ImPACT for Toddlers; UC=Usual Care; PPEM=Parent Participatory Engagement Measure; SSI= Social Support Index; PICCOLO=Parent Interactions with Children: Checklist of Observations Linked to Outcomes.

Supportive Parenting Behaviors.

PICCOLO scores indicated a significant effect of condition for all four parent domains: affection (β=.47, t(19)=2.28 p=.03), responsiveness (β=.84, t(19)=6.46 p<.01), encouragement (β=.59, t(19)=2.93 p=.01), and teaching (β=.45, t(19)=2.10 p=.05), with larger increases in positive parent behaviors for the PI^T group. Large effect sizes were observed for the PI^T group across the four domains (Cohen’s d Range=.83–1.88) compared to the small effects observed for the UC group (Cohen’s d Range=−.66-.16).

Parent PI^T Intervention Fidelity.

Analyses indicate a marginally significant effect of condition on caregiver use of the Prompt strategy (β=.44, t(19)=2.09 p=.05), with caregivers in the PI^T group demonstrating higher fidelity in use of Prompt strategies compared to UC group. A large effect size was observed for the Prompt strategy (Cohen’s d=1.19) for the PI^T group compared to a small effect size (Cohen’s d= .35) for the UC group. Similarly, caregivers in the PI^T group demonstrated higher, albeit non-significant (β=.21, t(19)=.69 p=.37), fidelity using the Wait strategy compared to those in the UC group, with a large effect size (Cohen’s d=.77) for PI^T compared the moderate effect for UC (Cohen’s d=.36). Results were non-significant and had similar effect sizes across groups for all other PI^T strategies (t’s<1.52, p’s>.15).

Parent Intervention Engagement and Social Support.

Independent samples t-test analyses indicated significant differences in overall average parent participatory engagement in the intervention (PPEM average across time points; t(21)=−2.66, p<.02), with higher overall engagement among families receiving PI^T (M=4.08, SD=.66) compared to UC (M=3.02, SD=1.12). Linear regressions analyses indicated no significant differences from intake to end of treatment in perceived social support by condition (t(16)=−.62, p=.55).

Aim 1 Intervention Effects: Child Outcomes

See Table 5 for full child outcomes results.

Table 5.

Aim 1 Intervention Effects at Baseline (Pre), End of Treatment (Post), and Follow Up (F): Child Outcomes

	PIT			UC			Regression Coefficients			Effect Sizes (Pre to Post) (Pre to Follow-Up)
Measure/Subscale	Pre M(SD) N=12	Post M(SD) N=11	F M(SD) N=10	Pre M(SD) N=13	Post M(SD) N=12	F M(SD) N=9	B	β	P	PIT ES	UC ES	PIT ES	UC ES
CSBS
Social	17.83 (5.52)	20.55 (3.64)	23.40 (3.03)	19.08 (6.14)	19.36 (5.39)	20.00 (6.72)	1.13	.13	.28	.55	.05	1.19	.15
Speech	6.58 (3.87)	8.73 (3.66)	10.50 (3.38)	8.08 (2.90)	9.82 (3.25)	10.63 (4.07)	−.50	−.07	.66	.54	.54	1.04	.75
Symbolic	10.00 (2.86)	13.82 (1.94)	14.40 (2.01)	12.23 (5.12)	13.55 (4.10)	14.38 (3.96)	1.23	.20	.07	1.47	.25	1.71	.45
Total	34.42 (10.70)	43.09 (6.85)	47.30 (6.27)	39.38 (13.25)	42.73 (12.10)	45.00 (14.13)	1.89	.10	.29	.91	.24	1.40	.41
CDI
Words Understood	108.6 (116.3)	162.2 (103.0)	225.2 (98.62)	167.8 (143.6)	171.6 (113.3)	181.7 (116.2)	−16.5	−.08	.49	.48	.03	1.08	.11
Words Produced	41.18 (100.7)	61.00 (104)	89.56 (133.9)	39.46 (54.82)	67.70 (97.67)	93.29 (102.5)	−35.4	−.18	.09	.19	.35	.43	.75
Early Gestures	11.36 (4.23)	13.27 (2.41)	15.00 (2.06)	13.15 (5.19)	13.50 (3.69)	13.14 (4.63)	−.52	.10	.65	.54	.07	1.07	<−.01
Late Gestures	22.73 (10.55)	28.91 (7.56)	31.50 (7.47)	23.08 (15.22)	22.80 (12.34)	26.29 (14.89)	2.06	.10	.46	.66	−.02	.95	.22
Vineland
Communication SS	81.36 (11.26)	85.50 (10.59)	87.33 (14.06)	86.42 (13.42)	84.44 (32.40)	86.38 (14.76)	12.76	.28	.18	.38	−.08	.49	<−.01
Socialization SS	90.91 (9.49)	92.80 (13.22)	91.22 (12.65)	91.75 (15.39)	96.44 (14.95)	94.88 (16.29)	.44	.02	.95	.17	.30	.03	.20
ABC Composite SS	89.18 (8.33)	90.70 (8.10)	91.65 (8.37)	88.75 (16.98)	92.56 (14.25)	91.75 (20.53)	.38	.02	.92	.19	.24	.29	.17

Abbreviations: PI^T = Project ImPACT for Toddlers; UC=Usual Care; CSBS=Communication and Symbolic Behavior Scales-Infant Toddler Checklist; CDI=MacArthur Bates Communicative Development Inventory; Vineland= Vineland Adaptive Behavior Scales, Second Edition; ES = Effect size; SS=Standard score

Child Communication and Social Communication.

Analyses revealed a marginal trend for a significant impact of condition on CDI Words Produced (β=−.18, t(16)=−1.76 p=.09), with small effect sizes for both conditions (UC Cohen’s d=.35; PI^T Cohen’s d=.19). A marginal trend was also observed for CSBS Symbolic Composite (β=.20, t(18)=1.90 p=.07), with a large effect size observed for the PI^T condition (Cohen’s d=1.47) compared to a small effect size for the UC condition (Cohen’s d=.25). Although further analyses did not reveal any statistically significant condition effects, analyses revealed medium to large effect sizes for the PI^T children compared to the small effect sizes for the UC children on several measures, including CSBS Social Composite (t(18)=1.11, p=.28; PI^T Cohen’s d = .55, UC Cohen’s d = .05), CSBS Total Score (t(18)=1.08, p=.29; PI^T Cohen’s d = .91, UC Cohen’s d = .24), CDI Words Understood (t(16)=−.71, p=.49; PI^T Cohen’s d = .48, UC Cohen’s d = .03), CDI Early Gestures (t(16)=−.46, p=.66; PI^T Cohen’s d = .54, UC Cohen’s d = .07), and CDI Late Gestures (t(16)=.76, p=.46; PI^T Cohen’s d = .66, UC Cohen’s d = −.02). No group differences were observed for CSBS Speech Composite (t(18)=−.45, p=.66).

Child Adaptive Skills.

Analyses revealed no significant condition effect for the Communication (t(14)=1.42, p=.18) or Socialization (t(14)=.07, p=.95). Composites on the Vineland from pre- to post-intervention. Effect sizes were small and similar across groups.

Aim 2: Associations Between Provider use of PI^T Coaching Strategies and Parent Fidelity

Providers who delivered PI^T to families had significantly higher scores on the Use of PI^T Parent Coaching Strategies measure than UC providers (PI^T M=3.68, SD=.66; UC M=2.17, SD=1.01; F (1,87) =72.77, p<.01), indicating a difference between groups in the structure and approach of services delivered to families.

We examined whether providers use of PI^T parent coaching strategies predicted overall parent fidelity and/or positive parent-child interaction. Controlling for the number of children enrolled for each provider, analyses indicated that providers’ use of parent coaching strategies was significantly associated with parent fidelity (β=.47, t(20)=2.08 p=.05) as well as higher parental Teaching behaviors as measured by the PICCOLO (β=.49, t(20)=13 p<.05). Additionally, there was a marginally significant association between parent fidelity and higher Responsiveness behaviors from parents as measured by the PICCOLO (β=.40, t(20)=1.70 p=.10). Providers’ use of coaching strategies was not significantly associated with parental Affection (t(20)=.88 p=.39) and Encouragement (t(20)=.39 p=.70) and effect sizes were similar across groups.

Aim 3: Exploratory Analyses Examining Intervention Effects at Follow-up

Parent outcomes at follow-up.

See Table 6 for Parent follow-up results. As mentioned, effect sizes were examined based on small sample sizes and limited statistical power.

Table 6.

Aim 3 Intervention Effects from Baseline (Pre) to Follow-up (F): Parent Outcomes

	PIT		UC		Regression Coefficients			Effect Sizes
Measure/Subscale	Pre M(SD) (N=12)	F M(SD) (N=10)	Pre M(SD) (N=13)	F M(SD) (N=9)	B	β	P	PIT ES	UC ES
PICCOLO
Affection	1.49(.24)	1.60(.21)	1.51(.18)	1.29(.26)	.33	.62	.01	.53	−1.07
Response	1.38(.37)	1.83(.20)	1.47(.30)	1.55(.27)	.28	.55	.03	1.44	.28
Encouragement	1.42 (.22)	1.69 (.20)	1.51 (.22)	1.59 (.17)	.12	.32	.25	1.38	.43
Teaching	1.01 (.34)	1.18 (.24)	1.18 (.33)	1.20 (.25)	<.01	.01	.98	.64	05

Abbreviations: PI^T = Project ImPACT for Toddlers; UC=Usual Care; PICCOLO=Parent Interactions with Children: Checklist of Observations Linked to Outcomes; F=Follow-Up; ES=Effect Size

Supportive Parenting Behaviors.

Moderate to large effect sizes were observed for the PI^T group across the four domains of the PICCOLO (Cohen’s d Range=.53–1.44) compared to the small effects observed for the UC group (Cohen’s d Range=−1.07-.43), indicating larger changes in Affection, Responsiveness, Encouragement, and Teaching parenting behaviors from pre-intervention to three-month follow-up for parents who received PI^T. Differences between the groups were not statistically significant.

Child outcomes at follow-up.

See Table 5 for full child outcomes results.

Child Communication and Social Communication.

Analyses revealed large effect sizes for the PI^T children compared to the small effect sizes for the UC for several measures, including the CSBS Social Composite (β=.17, t(15)=1.20 p=.25; PI^T Cohen’s d = 1.19, UC Cohen’s d = .15), CSBS Symbolic Composite (β=.13, t(15)=1.10 p=.29; PI^T Cohen’s d = 1.71, UC Cohen’s d = .45), CSBS Total Composite (β=.08, t(15)=.85 p=.41 PI^T Cohen’s d = 1.40, UC Cohen’s d = .41), CDI Words Understood (β=−.02, t(13)=−.10 p=.92; PI^T Cohen’s d = 1.08, UC Cohen’s d = .11), CDI Early Gestures (β=.16, t(13)=.96 p=.35; PI^T Cohen’s d = 1.07, UC Cohen’s d < −.01), and CDI Late Gestures (β=.05, t(13)=.34 p=.74; PI^T Cohen’s d = .95, UC Cohen’s d = .22) from pre-intervention to three month follow-up.

Child Adaptive Skills.

Analyses revealed a medium effect sizes on the Vineland Communication Scale for the PI^T children (Cohen’s d = .49) compared to the small effect sizes for the UC children (Cohen’s d <−.01; β=.14, t(14)=1.01, p=.33).

Discussion

This study represents one of the first demonstrations of effectiveness for an adapted evidence-based, naturalistic developmental behavioral intervention for toddlers in community early intervention programs. These promising results provide preliminary evidence for the potential benefit of systematically fitting evidence-based interventions into service and community contexts through research-community partnership. Because the intervention was adapted with local providers and funders, implementation of the strategies with fidelity was more likely (see Rieth et al., 2018 for further details regarding training and fidelity) and the sustainment of strategies over time provided the opportunity for this quasi-experimental examination of child and family outcomes of the adapted intervention.

Our data indicate that this intervention shows promise for improving social communication outcomes for young children with early signs of ASD, even if a diagnosis has not yet been made. Although the design was quasi experimental, the groups were generally equivalent at intake to the intervention. Encouragingly, though all children made progress in early intervention and results were not statistically significant in many areas, there were some promising differences favoring the PI^T group. For example, children receiving PI^T had strong effect sizes related to progress in symbolic communication and play on a standardized behavioral measure. Parents in that group also reported greater effect sizes for words understood and gesture use after only 3 months of a low intensity intervention. As may be potentially expected to happen with parent-implemented interventions, even greater differences in effect sizes were evident between the groups after the follow up period. The reason for the increased differences is posited to be the changes in parents’ use of strategies after completion of the training, thereby increasing intervention intensity and duration for the child.

Parents in the PI^T group made greater changes in the way they interacted with their children after receiving intervention. This is important because parents of children with social communication challenges and/or ASD, although just as responsive as parents of typically developing children (Siller & Sigman, 2002), have fewer opportunities to respond because of differences in the way their children learn from their environment (Leezenbaurm, Campbell, Butler, & Iverson, 2014). Results are similar to outcomes in community trials of a developmental, relationship-based parent-mediated intervention with older children that has many similar principles to PI^T (e.g., Solomon et al., 2014). Parent sensitivity and responsivity predicts language outcomes in typical development as well as ASD (Siller & Sigman, 2002). PI^T coaching from providers resulted in increased responsivity of parents in all areas including affection, response to child’s cues, encouragement and teaching opportunities. These changes may subsequently support child development and have sustained, long-term impacts on child outcomes (Pickles et al., 2016).

Interestingly, although having a provider trained in PI^T led to strong differences in more distal parent child interaction, providers were not effective at increasing parents’ use of specific intervention skills measured on the PI^T fidelity rating scale. This was the case even though use of the parent coaching strategies by the provider was associated with overall parent PI^T fidelity. This may be related to the fidelity measurement tool itself. Fidelity to an intervention is a complex, multi-dimensional construct that may encompass several areas, including: 1) content-the steps delivered; 2) competence or quality-the skill and judgment used during delivery; 3) quantity-how much of the intervention was delivered; 4) adherence-the degree to which prescribed and not proscribed procedures are utilized; and 5) differentiation-features unique to the intervention (Schoenwald, Garland, Chapman, Frazier, & Sheidow, 2011). The PI^T fidelity rating scale may be good at capturing content and adherence but less accurate when examining quality and clinical appropriateness of skills use. The relationship between coaching strategies and parent use of the strategies overall may be based on increases in specific skills (e.g., prompting) and not consistent changes in all of the strategies in the intervention. Alternatively, parents may be learning about responsivity through general coaching and the specific skills may or may not be the mechanism of action for the intervention. This is an important limitation for community research seeking to understand the key ingredients of intervention. Ideally, we would see differences between the groups in their use of the specific strategies taught in the intervention over time (in addition to impacts on more distal measures). More research with larger samples examining the link between coaching in specific strategies and changes in parent behavior over time will help reveal if these types of fidelity measures are capturing what we intend and which strategies truly lead to change in parent and child behavior.

Importantly, this community sample included a diverse group of families who represent typical consumers of early intervention in an urban area. A total of 50% of families receiving PI^T were Hispanic/Latinx, and they represented a range of maternal education levels. Although the purpose and design of this study were not to examine differential impact of an intervention across racial and ethnic groups, the large effect sizes for PI^T within this representative sample are encouraging as to the cultural appropriateness of the practice. The adaptation process for PI^T included many members of the Latinx community, both during broad community feedback phases and as members of the BRIDGE Collaborative. The impact of the intervention in the current trial with a diverse sample is evidence of the utility of involving stakeholders in the early phases of intervention development.

Results should be considered preliminary given several limitations. First, children were not randomly assigned to groups by the research team, but rather were assigned to providers by case workers at the local Part C funder as publicly provided service. While the examination of PI^T within the context of community care is strength of the current study, this lack of random assignment is a limitation, as it led to apparent, albeit not statistically significant, differences in ASD symptom severity in the PI^T group compared to the UC groups. This difference may limit the conclusions that can be drawn from comparison between the two groups of children. Additionally, statistical power was limited in all analyses due to the small sample size and pilot nature of the project. Limited power lead us to compare effect sizes across groups as an indicator of the intervention’s impact, but the larger effect sizes on some measures for the PI^T group are not equivalent to statistically significant differences in more rigorous analyses. Future work should examine the impact of PI^T with a larger group of children in a rigorous experimental design in order to better understand any differential impact of the PI^T intervention.

This project was made possible through the development of a research-community partnership that adapted a specific evidence-based practice to fit the community context. Community agencies who had received training in PI^T through previous interaction with the research team continued to use PI^T and to receive funding through Part C to provide this intervention in the community. Participating parents and children obtained the service through public agencies vendored with Part C. Greater change was seen in both parent and child behavior when services were delivered from a PI^T trained provider. This pilot project supports the use of research-community partnerships to increase access to care and to support broader research in child and family outcomes for this population. Next steps will include a better understanding of how to measure fidelity of the intervention and conducting a larger, controlled trial of PI^T effectiveness in the community.