Generating Attention, Inhibition, and Memory: A Pilot Randomized Trial for Preschoolers With Executive Functioning Deficits

Abstract

This goal of this study was to assess the initial feasibility and efficacy of a play-based intervention targeting executive functions (EF) and parent–child relationships in preschoolers compared with an active control group. Preschoolers with EF deficits (M age = 3.7 ± 0.47, predominantly White boys) and their parents were randomized to intervention (n = 36) or active control (n = 32) conditions. Child performance on EF tasks, parent and masked teacher ratings of EF and behavior, and masked clinician ratings of severity were collected at baseline and at 3 and 6 months postbaseline. Partial eta-squared effect sizes at .02 or higher comparing performance across the two groups was considered evidence of meaningful, albeit small, intervention effects. Intervention effects were observed for parent ratings of inattention, hyperactivity/impulsivity, and number/severity of problems experienced in various home situations, teacher ratings of severity of problems in various school situations, parent and teacher ratings of overall impairment, and clinician ratings of impairment. Intervention effects for functional improvements were maintained at the 6-month follow-up. No effect of the intervention was observed on the objective EF measures, although parent ratings of emotional control were improved for children in the intervention group. An intervention utilizing play-based activities targeting EF, when administered in a structured way by parents, is a promising approach for improving behavior in preschoolers with self-regulation deficits. More work is needed to investigate potential impact on EF and to disentangle mechanisms of action. It may be that the intervention's focus on the structure and quality of parent–child interactions is a mediator of outcomes, rather than improved EFs.

Introduction

Young children at risk for attention deficit/hyperactivity disorder (ADHD) have difficulties with executive functioning (EF) including self-regulation, attention, working memory, cognitive flexibility, and behavioral inhibition (American Psychiatric Association, 2013; Byrne, DeWolfe, & Bawden, 1998; Hughes, White, Sharpen, & Dunn, 2000; Mariani & Barkley, 1997). The failure of children to develop effective EF has been proposed as a putative cause of ADHD (Brocki, Nyberg, Thorell, & Bohlin, 2007; Halperin, 2011; Nigg, Willcutt, Doyle, & Sonuga-Barke, 2005; Sonuga-Barke, 2005). Further, EF deficits independently contribute to poorer outcomes in ADHD (Halperin, 2011; Wåhlstedt, Thorell, & Bohlin, 2008; Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005), and have been linked directly to academic (Biederman et al., 2004; Clark, Pritchard, & Woodward, 2010; Raggi & Chronis, 2006; Rimm-Kaufman, Curby, Grimm, Nathanson, & Brock, 2009) and social functioning deficits (Clark, Prior, & Kinsella, 2002; Diamantopoulou, Rydell, Thorell, & Bohlin, 2007; Levin, Hanten, & Li, 2009; Nigg, Quamma, Greenberg, & Kusche, 1999). Poor inhibitory capacity is associated with persistent ADHD (Brocki et al., 2007) and later noncooperative behavior (Ciairano, Visu-Petra, & Settanni, 2007), whereas intact behavioral inhibition has been linked to social and school competence, positive school attitudes, and fewer academic problems (Chen, Chen, Li, & Wang, 2009). Thus, EF is an excellent potential treatment target.

Early intervention targeting EF is necessary because these functions are developing and changing rapidly during the preschool and early school years. Neurodevelopment is more than an unfolding of the genetic blueprint; it is highly malleable with a child's early experiences influencing the developing brain (Courchesne, Townsend, & Chase, 1995). Neuropsychological studies suggest extensive development of attention and executive control functions between the ages of 3 and 7, which correlate with developments in brain structure and function (Halperin & Healey, 2011). Studies of typically developing preschoolers have attributed age-related changes in EF to a variety of separable abilities (Davidson, Amso, Anderson, & Diamond, 2006), including representational flexibility and rule use (DeLoache, Sugarman, & Brown, 1985; Jacques, Zelazo, Kirkham, & Semcesen, 1999), inhibition and response control (Barkley, 1997; Kanaka et al., 2008; Reed, Pien, & Rothbart, 1984), working memory (Case, Kurland, & Goldberg, 1982), planning and depth of search (Klahr & Robinson, 1981), and dimensional abstraction and representation (Jacques & Zelazo, 2001). There are also developmental changes in metacognition with increasing conscious control over strategic processes including awareness of attention and EF, increasing understanding of task demands, and increasing capacity to relate abilities to task demands (Jarman, Vavrik, & Walton, 1995). There is preliminary evidence that practice and training can alter brain mechanisms implicated in EF deficits (Dopfner, Rothenberger, & Sonuga-Barke, 2004; Fan, McCandliss, Sommer, Raz, & Posner, 2002; Halperin & Healey, 2011; Mateer & Mapou, 1996; Olesen, Westerberg, & Klingberg, 2004), directed play activities can enhance neural development (Halperin & Healey, 2011), and parenting can mold brain structure and function (Belsky & De Haan, 2011). It is also hypothesized that the decline of ADHD symptoms with age is accounted for by the degree to which later development of higher cortical circuitry and functions can compensate through “top down” regulatory control (Halperin & Schulz, 2006). Taken together with findings that ADHD is associated with a delay in cortical maturation rather than fixed morphological differences (P. Shaw et al., 2007), early intervention targeting EF has the potential to remediate ongoing attention problems and reduce the severity of ADHD symptoms and impairments (Halperin & Schulz, 2006), and it may have a long-term impact on the functional development of these systems (Halperin & Healey, 2011; Rueda, Rothbart, McCandliss, Saccomanno, & Posner, 2005). Thus, interventions that facilitate development of EFs in “at-risk” preschool children could potentially contribute to later school success and impact the trajectory of ADHD, inhibiting the emergence of a full-blown diagnosis and associated health care costs in later childhood (Halperin, Bedard, & Curchack-Lichtin, 2012; Halperin & Healey, 2011; Sonuga-Barke & Halperin, 2010).

Recently, there have been several investigations examining whether EF can be trained in young children at risk for ADHD using a play-based approach (for reviews, see Diamond & Lee, 2011; Diamond & Ling, 2016). These interventions are based on strong evidence for the impact of play on the cognitive, physical, social, and emotional well-being of children (Ginsburg, 2007) and take advantage of “teachable moments” during the early years of learning and development (Halperin et al., 2012). Such interventions use games and activities that are designed to target EF skills such as inhibitory control, working memory, and motor coordination. Involving parents in the intervention process is critical given the key role they play in the child's development of self-regulation (Chang, Olson, Sameroff, & Sexton, 2011; Kochanska & Knaack, 2003; Kochanska, Murray, & Harlan, 2000). Parenting behaviors that favor the development of children's self-regulation and effortful control include maternal sensitivity (appropriate and consistent responses), scaffolding (offering age-appropriate problem-solving strategies), and mind-mindedness (using “mental” terms while talking to the child offers verbal tools that can facilitate the transition from external regulation to self-regulation; Bernier, Carlson, & Whipple, 2010; Carlson, 2003). These behaviors have been termed proactive, positive parenting (Chang, Shaw, Dishion, Gardner, & Wilson, 2015). Thus, play-based interventions, such as the Generating Attention, Inhibition, and Memory (GAIM) intervention¹ (Tamm & Nakonezny, 2015; Tamm, Nakonezny, & Hughes, 2012), have focused not only on exposing children to the activities themselves, but also on training the parent to employ strategies such as use of common and simple language regarding what it means to pay attention (e.g., eyes looking, ears listening, brain thinking), scaffolding activities (e.g., moving from simpler to more complex activities as the child achieves mastery), and focusing on the positive (e.g., reinforcing positive attending skills vs. successful activity completion). Having the parent implement an intervention within the home setting dramatically increases the dose of intervention.

It should be noted that these play-based approaches targeting EF differ from traditional behavioral parent training approaches, which typically focus on teaching parents strategies targeting children's maladaptive behaviors (particularly conduct problems, noncompliance, and oppositionality) such as limit setting, time-out, daily report cards, when-then strategies, effective commands, and homework management (Barkley, 1987; Tamm et al., 2005; Webster-Stratton, Reid, & Beauchaine, 2011); some approaches include a “special playtime” prescription but not necessarily as a primary or sole focus of the intervention (Barrett & Ollendick, 2004). However, the utilization of behavioral reinforcement principles is common in both approaches. These interventions also differ from traditional play therapy approaches in that the activities are preselected, structured, and directive. The play is not utilized to elicit a child's emotional issues, nor is the focus on using play to help a child change the way they feel toward, think about, and resolve their concerns; rather, the activities elicit EF skills that can be practiced and reinforced.

Preliminary studies with play-based EF interventions have demonstrated an impact on parent-reported attention and hyperactivity symptoms (Halperin et al., 2013; Healey & Halperin, 2015; Tamm & Nakonezny, 2015; Tamm et al., 2012), parent EF ratings (Tamm & Nakonezny, 2015; Tamm et al., 2012), functional impairment (Halperin et al., 2013; Tamm & Nakonezny, 2015), and child performance on visual/auditory attention, and cognitive flexibility (Tamm & Nakonezny, 2015; Tamm et al., 2012); working memory (Healey & Halperin, 2015; Tamm et al., 2012); and visuomotor precision tasks (Healey & Halperin, 2015). Moreover, interventions targeting EF in young children have been shown to promote neural and social development (Healey, Marks, & Halperin, 2011; Rueda et al., 2005). Yet none of these studies included an active control group, which is necessary to rule out potential nonspecific effects of parental attention, practice and expectancy, and the natural course of development. Furthermore, only one study (Halperin et al. 2013) examined whether treatment-related gains generalized across settings (e.g., school). Thus, additional research with an active control group and inclusion of other raters is critical to understand the specific and nonspecific effects of these interventions.

We conducted a pilot randomized trial of the GAIM training program (Tamm & Nakonezny, 2015; Tamm et al., 2012) in preschoolers with EF deficits utilizing an active control group (parent education) adapted from Kern et al. (2007) to control for therapist attention. It was hypothesized that GAIM would yield improvements in parent EF ratings and ratings of ADHD compared to an active control group. We also anticipated that performance on EF tasks of attention would be improved by GAIM.

Methods

The study was approved by the hospital Institutional Review Board, and informed consent was obtained from all participants prior to initiating any procedures.

Participants

Families were recruited from local preschools and the community and through flyers and mailings at a children's hospital. Recruitment materials referenced preschool children with “self-control” difficulties. At baseline, participants ranged in age from 3 to 4 years (M = 3.7, SD = .47), were all in a structured educational setting, were predominantly White boys, and represented a broad range of family incomes. Table 1 reports the demographic characteristics of the sample. All children had a T score at or greater than 60 as rated by the parent or teacher on the Emergent Metacognition Index (EMI) of the Behavior Rating Inventory of Executive Functioning-Preschool Version (BRIEF-P; Gioia, Isquith, Guy, & Kenworthy, 2000). This criterion was selected to acquire a sample at risk for ADHD. Validity studies with the BRIEF-P suggest that inattention is most highly correlated with the EMI index (Gioia et al., 2000) and ADHD inattention symptoms are uniquely related to EF even after controlling for hyperactivity/impulsivity (Nigg & Casey, 2005; Sonuga-Barke, 2002). Further, validity studies show strong correlations in the expected direction between BRIEF-P subscales and general behavior ratings (e.g., Behavioral Assessment System for Children; Gioia et al., 2000). Exclusion criteria included an estimated IQ of 85 or lower; currently taking psychotropic medications or receiving psychosocial interventions; diagnosed with a pervasive developmental disorder; a Clinical Global Impressions (CGI; Leon et al., 1993) score of 6 or greater (severely ill), indicating a need for immediate intensive treatment; or non-English speaking. Of the 84 children who participated in a baseline evaluation, 68 met eligibility criteria and were randomized to the intervention (n = 36) or the control group (n = 32; see Figure 1).

Table 1. Participant Demographics.

Baseline Variable	Controla	Interventionb	Statistical Test
Age in years	M = 3.7 (.46)	M = 3.6 (.49)	t(66) = .70, p = .49
Gender	75% boys	69.4% boys	χ2(1) = .26, p = .61
Ethnicity	0% Hispanic	3% Hispanic	χ2(1) = .90, p = .34
Race	3% Asian	0% Asian	χ2(3) = 6.3, p = .10
	22% Black	6% Black
	72% White	83% White
	3% Biracial	11% Biracial
IQ	104.9 (13.7)	104.8 (13.4)	t(65) = .06, p = .96
Family Income	16% < $25K	6% < $25K	χ2(4) = 4.4, p = .35
	19% $25–49K	8% $25–49K
	13% $50–75K	19% $50–75K
	25% $75–100K	25% $75–100K
	28% > $100K	42% > $100K
Emergent Metacognition Index (Parent)	T = 74.3 (11.9)	T = 73.3 (15.0)	t(66) = .33, p = .74
Emergent Metacognition Index (Teacher)	T = 72.8 (14.3)	T = 67.5 (13.4)	t(66) = 1.56, p = .12
CGI Severity at Baseline	3.35 (.80)	3.33 (.59)	t(65) = .13, p = .90
Average Attendance of 8 Group Sessions	4.72 (3.2)	6.17 (2.2)	t(66) = -2.2, p = .005**
Attended at Least 1 Session	78%	94%	χ2(1) = 3.9, p = .047*
Met ADHD Criteria at Baseline	72%	59%	χ2(1) = 1.1, p = .30

Note: CGI = Clinical Global Impressions; ADHD = attention-deficit/hyperactivity disorder.

^an = 32

^bn = 36.

^*p < .05.

^**p < .01.

Figure 1.

CONSORT diagram. Note: CGI = Clinical Global Impressions.

Measures

Electronic Preschool Age Psychiatric Assessment

The Electronic Preschool Age Psychiatric Assessment (ePAPA; Egger & Angold, 2004) is a computerized diagnostic interview assessing psychiatric disorders in children ages 2 to 5. The ePAPA covers a comprehensive set of symptoms from the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; American Psychiatric Association, 1994), as well as their effects on the child's relationships and functioning. It uses a semistructured format and an interviewer-based approach. The interviewer must adhere to the protocol and ask all required questions but must also confirm the parents' understanding of the question, elicit examples of relevant behaviors, and apply a priori guidelines for rating symptoms using a glossary. Diagnostic and Statistical Manual of Mental Disorders (4th ed.) diagnoses are derived using algorithms created by the instrument's developers. Parents were administered the ePAPA by master's-level graduate students, postdoctoral fellows, or clinical psychologists to establish baseline severity and ADHD status. The ePAPA has adequate psychometric properties with preschoolers in various settings (Egger et al., 2006).

Differential Ability Scales–II

The Differential Ability Scales–II (DAS-II; Elliott, 2007) assesses a range of cognitive abilities (e.g., verbal, nonverbal, spatial), which together are used to obtain a General Conceptual Ability composite score, an analog for IQ, in children ages 2 and older. IQ was estimated at baseline using the DAS-II. The DAS-II has excellent psychometric properties including high reliability and good evidence of concurrent and construct validity (Gordon & Elliot, 2001).

Services Assessment for Children and Adolescents (SACA)

The Services Assessment for Children and Adolescents (SACA; Horwitz et al., 2001) is a parent interview assessing the types of mental health services children use and the mental health treatments they receive within service settings. The SACA gathers data on a child's use of 30 service settings grouped into inpatient, outpatient, and school. Parents were interviewed at the 3- and 6-month follow-up time points with the SACA. The SACA has excellent test–retest reliability (≥ .80; Horwitz et al., 2001).

Parent and Teacher Ratings

Behavior Rating Inventory of Executive Function–Preschool Version

Parents and teachers completed the BRIEF-P (Gioia et al., 2000) 63-item rating scale yielding T scores on Inhibit, Shift, Emotion Regulation, Working Memory, and Planning/ Organization subscales. The EMI is composed of the Working Memory and Plan/Organize scales. Studies investigating psychometric properties of the BRIEF-P report good convergent and discriminant validity with other behavioral rating systems (e.g., Child Behavior Checklist), test–rest reliability ranging from .79 to .88, and internal consistency ranging from .80 to .98 (Gioia et al., 2000). We utilized the five subscale T scores.

Disruptive Behavior Disorders Rating Scale

Parents and teachers rated each symptom of ADHD, oppositional defiant disorder, and conduct disorder on a 4-point Likert scale: 0 (not at all), 1 (just a little), 2 (pretty much), and 3 (very much). The Disruptive Behavior Disorders Rating Scale (DBDRS; Pelham, Gnagy, Greenslade, & Milich, 1992) has adequate psychometric properties with at-risk preschool children (Pelletier, Collett, Gimpel, & Crowley, 2006). We used the Inattention (parent α = .89, teacher α = .91) and Hyperactivity/Impulsivity (parent α = .86, teacher α = .92) subscales from the ADHD symptom ratings.

Home and School Situations Questionnaires

The Home Situations Questionnaire (HSQ; Barkley & Edlebrock, 1987) consists of 16 situations in which parents commonly observe and manage child behaviors (e.g., mealtimes). Parents rate whether problem behaviors occur in these situations, and if so, the severity on a 1 (mild) to 9 (severe) scale. The School Situations Questionnaire (SSQ; Barkley & Edlebrock, 1987) is the teacher counterpart and includes 12 situations in which teachers commonly observe and manage child behaviors (e.g., recess). The HSQ and SSQ each yield the following: number of problems (possible range = 0–16 parent, 0–12 teacher) and severity of problems in each setting (possible range = 1–9, with higher scores indicating greater severity). Strong internal consistency and test–retest reliabilities have been documented for both the HSQ and the SSQ (Altepeter & Breen, 1989; Breen & Altepeter, 1991; Pelletier et al., 2006). In addition, good interrater agreement and criterion validity have been reported for the HSQ (Altepeter & Breen, 1989; Breen & Altepeter, 1991). In the current sample, internal consistency was good for the HSQ (number of problems α = .79, severity α = .90) and SSQ (number of problems α = .83, severity α = .89)

Impairment Rating Scale

We used the overall impairment rating of the Impairment Rating Scale (IRS; Fabiano et al., 2006) rating (0 = no impairment; 6 = extreme impairment) completed by parents and teachers. The psychometric properties of the IRS are well-established in children as young as 3 years old, with excellent temporal stability and evidence of convergent and discriminant validity (Fabiano et al., 2006).

Swanson, Kotkin, Agler, M-Flynn, and Pelham Scale

The Swanson, Kotkin, Agler, M-Flynn, and Pelham Scale (McBurnett, Swanson, Pfiffner, & Tamm, 1997) assesses context-bound behaviors critical to success in school. Teachers rate how well 10 behaviors describe the child on a 4-point scale: 0 (not at all), 1 (just a little), 2 (pretty much), and 3 (very much). Two subscales are derived (Attention and Deportment) by averaging items in each domain. Internal consistency is high (McBurnett et al., 1997; Murray et al., 2009), and there is evidence of divergent (McBurnett et al., 1997) and predictive (Murray et al., 2009) validity. In our sample, internal consistency was good (Attention α = .87, Deportment α = .85).

Treatment Acceptability Questionnaire

For the Treatment Acceptability Questionnaire (TAQ; Hunsley, 1992), parents rated how acceptable (1 = very unacceptable; 7 = very acceptable) and effective (1 = very ineffective; 7 = very effective) the treatment was and whether it caused any negative side effects (1 = very unlikely; 7 = very likely) on a 7-point Likert scale.

Child EF Measures

The child EF measures were selected because previous studies have shown an effect of the GAIM intervention on these tasks (Tamm & Nakonezny, 2015; Tamm et al., 2012).

NEPSY-Visual Attention (Korkman, Kirk, & Kemp, 1998)

The Visual Attention subtest assesses the speed and accuracy with which a child can scan an array and locate a target and involves inhibition, vigilance, scanning, and impulse control. The age-based scaled score served as a measure of selective and sustained attention. Reliabilities for this measure at ages 3 through 5 range from .68 to .76, stability coefficient was .62, and validity was established with continuous performance tests and scanning tasks (Korkman et al., 1998).

Clinical Evaluation of Language Fundamentals, Preschool Second Edition—Concepts and Following Directions

The Concepts and Following Directions subtest (Wiig, Secord, & Semel, 2004) required children to interpret, recall, and execute oral commands of increasing length and complexity. This subtest yields age-based scaled scores and served as a measure of auditory and visual attention. Across 3- and 4-year-old age groups, internal consistency reliability coefficients ranged from .84 to .85, and validity was established with the Preschool Language Scale, Fourth Edition (Semel, Wiig, & Secord, 2003).

Clinician Ratings

Clinical Global Impressions

Clinicians (master's-level graduate students, postdoctoral fellows, or clinical psychologists) interviewed the parent regarding severity of the child's impairment on a 7-point scale from 1 (not at all ill) to 7 (very severely ill). The scale has adequate reliability and validity (Leon et al., 1993). Clinicians were masked and families were advised not to reveal to the clinician which group they had attended. All raters (n = 6) scored a few cases together in the beginning to establish interrater reliability. Scoring of each CGI occurred after the final assessment visit, and improvement was rated compared to baseline. CGI scores were computed by two raters, and in cases of disagreement, a third rater weighed in. Scores were informed by rating scales completed by parents and teachers.

Design and Procedure

Parents responded to an initial telephone screen. Those families who met initial eligibility criteria completed the BRIEF-P and the child's teacher completed the BRIEF-P. If a child was rated at 60 or higher on the EMI of the BRIEF-P by either parent or teacher, the family participated in a baseline evaluation, which included the ePAPA, several parent rating scales, and the child completing neurocognitive measures. Child assessments were administered and scored by trained undergraduate, postbaccalaureate, and graduate research assistants over one or two testing sessions (approximately 4 hr in total). Teachers also completed rating scales. Participants were randomly assigned to the intervention or control group. Parents were masked to the condition to which they were assigned, that is, they were told only which night their group was being held and that both groups contained useful information for preschool children. Participants, parents, and teachers who were masked to the child's assignment completed the same measures 3 and 6 months after the baseline visit. Participants were compensated for their time.

Generating Attention, Inhibition, and Memory Intervention

As described by Tamm and colleagues (Tamm & Nakonezny, 2015; Tamm et al., 2012), the intervention involved eight weekly 60-min sessions (the first session lasted 2 hr). Parents and children participated in separate concurrent groups.

Children met in small groups (four to six children per group) with two interventionists. Children participated in several activities designed to practice different aspects of EF and related skills (Diamond & Lee, 2011; O'Neill, Rajendran, & Halperin, 2012), including attention, inhibition, memory, hand-eye coordination, balance, sensory awareness, listening skills, and visual focusing. Interventionists reinforced attending skills including “eyes looking, ears listening, mouth quiet, body still, brain thinking” during completion of the activities. The activities stand alone as enjoyable learning experiences but are also crucial for bridging the metacognitive strategies emphasized across all activities in the program, thus nurturing generalization of the attention skills. Principles of behavior modification (e.g., preventing behavior before it occurs, reinforcement for following group rules, ignoring, and timeout) were also implemented.

While children were participating in small groups, parents met in a group (four to 12 parents, depending if both parents attended) with a psychologist who explained and modeled how to implement the activities at home with emphases on well-timed attention, inhibition, and memory skill building and on how to use specific and labeled praise. For example, to increase fine motor control, parents practiced the game Jenga at home and reinforced the child for demonstrating body control; to increase attention to details, parents played Highlights Search with their child and reinforced the child using their eyes to look and brain to think. To promote generalization, parents were asked to implement the metacognitive strategies during group prior to completing each activity at home, to brainstorm different activities that activate desired EF skills, and to identify different situations in which these skills are necessary. For example, parents generalized from a memory game to having children help remember grocery lists. Between sessions, parents were asked to practice at least one of the activities with their child three or more times.

Active Control

A didactic parent education group, which included topics of interest to parents of preschoolers (e.g., self-esteem, school readiness) but did not include a focus on the parent-child relationship or directly address EF and behavioral reinforcement, was desirable because it would be less likely to impact child EF while still providing relevant information to parents and controlling for therapist attention. The parent group content was derived from sessions included in another study's Parent Education condition (Kern et al., 2007), which was selected because it offered a benefit for child behavior but did not target child EF.

The general procedures for the active control group were identical to the intervention group with the exception that the control group was not assigned tasks to complete between sessions. The control condition involved eight weekly 60-min sessions (the first session lasted 2 hr), and parents and children participated in separate groups offered concurrently.

Children met in small groups (four to six children per group) with two interventionists and were exposed to the same activities as children in the intervention group. However, there was no reference to or reinforcement of attention, EF, metacognition, and so on. Principles of behavior modification were implemented.

While children were participating in the small groups, parents met in a large group with a psychologist or master's-level clinician who delivered didactic presentations on a variety of content areas. Session content included topics such as child health and nutrition, parent self-care, building self-esteem, and understanding children's emotional and social development. No parenting strategies were provided.

Training and Fidelity

Interventionists for the child groups were trained undergraduate, postbaccalaureate, or graduate research assistants who were supervised weekly. Training involved a review of each activity and materials, observation of videotaped recordings, and live observation and supervision at each session. Video-recordings of the group were made, and research assistants coded the interventionists' adherence to the treatment manuals. Interrater reliability was established by having multiple coders rate the same videotape with a goal of 90% or above reliability. Fidelity for GAIM was 95% and for the active control group was 92%.

Statistical Analysis

Independent sample t-tests (for continuous variables) and chi-square tests (for categorical variables) were used to compare the two treatment groups on demographic and baseline clinical characteristics, as well as the TAQ and SACA. Chi-square tests were also computed comparing individuals who did and did not attend at least one group session on demographics and severity.

Missing data were observed in 57% of the outcome variables examined across all time points. Missing data ranged between 1.7% and 10.2% and was handled via multiple imputation with M = 100 imputed data sets keeping two considerations in mind. First, acknowledging that our research question bears on group differences over time (Group × Time interaction), and consistent with accepted practice (Enders & Gottschall, 2011), missing data were imputed separately for the treatment and control groups. Second, consistent with research that stresses the importance of including auxiliary correlate data to better meet the assumption of missing at random (Enders, 2010; Graham, 2012), 69 additional variables were added to the imputation model.

General linear modeling analyses examining group, time (baseline vs. 3 months; baseline vs. 6 months), and the Group × Time interaction were computed for the various response measures. Treatment effects were assessed via the statistical tests of the Group × Time interactions. These analyses were conducted for participants who attended at least one session (n = 59; 25 in the control group and 34 in the intervention group). We included only those individuals who started the groups, as attrition immediately following randomization was greater in the control group for no appreciable reason (parents dropped even though they were not informed of the condition to which they had been randomized). Because this was a preliminary study and not powered as an efficacy trial, we focused on clinical significance using effect sizes. Partial eta-squared (Cohen, 1998) effect sizes were calculated for the Group × Time interaction. Generally, effect sizes of .01 or greater are considered small, .06 or greater are moderate, and .14 or greater are large (Cohen, 1998). To avoid overinterpreting findings from an underpowered study, however, we utilized a more conservative effect size estimate of partial eta-squared (η_p²) at or greater than .02 (equivalent to a Cohen's d of .30) as reflecting meaningful benefits of the intervention, based on a meta-analytic data showing the average unweighted effect sizes for behavioral interventions on child behavior and functioning were in this range for between-group designs such as ours (Fabiano et al., 2009).

Results

Preliminary Analyses

As shown in Table 1 , children in the intervention and control conditions did not differ demographically, indicating that randomization worked as intended. There was a significant difference in attendance, whereby families in the intervention group attended more sessions than controls, and more families in the intervention group attended more than one session than controls. Individuals who did not attend more than one session did not significantly differ from those retained for analysis on age, gender, race/ethnicity, IQ, or severity (all ps ≥ .10); however, participants who attended fewer than two sessions were more likely than other participants to earn a lower family income, χ²(4) = 12.9, p = .01, and be African American, χ²(3) = 12.1, p = .007. It should be noted that parents in the intervention group were compliant with the request that they practice the activities three times per week at home between sessions, and every family reported practicing activities between sessions (M = 8.71 per week, range = 4–30), and attempting to generalize GAIM strategies to other activities (M = 5.16 per week, range = 1–22) each week.

Satisfaction and Mental Health Services Received

Results examining group differences on the TAQ favored the intervention group. Parents rated the intervention as more acceptable, GAIM M = 5.4 ± 1.1 versus control M = 4.4 ± 1.4, t(50) = 2.8, p < .001, and effective, GAIMM = 4.7 ± 1.2 versus control M = 3.6 ± 1.2, t(50) = 3.3, p < .001, than the control group. Neither treatment was perceived as having negative side effects, and the group ratings were not significantly different from one another, GAIM M = 1.2 ± 0.4 versus control M = 1.4 ± 1.1, t(50) = 1.1, p = .29.

Results examining group differences on the SACA showed that very few children (one in each group at the 3- and 6-month assessment points) started medication with no group differences at the 3-month, χ²(1) = .02, p = .88, or 6-month, χ²(1) = .07, p = .79, time points. Similarly, there were no group differences with regards to the number of children receiving outpatient services from a mental health professional at the 3-month time point (intervention n = 6, control n = 5), χ²(1) = .00, p = .95. However, between the 3- and 6-month time points, more children in the control group reported receiving outpatient services from a mental health professional (intervention n = 3, control n = 8), χ²(1) = 5.4, p = .02.

Primary Analyses

Although our primary focus was on effect sizes, we did find a few statistically significant results favoring the intervention group. Parents rated children in the intervention group as having fewer problems in the home setting (HSQ) than those in the control group at both the 3- and 6-month time points, and as having less severe problems (HSQ) than those in the control group at the 3-month time point (Table 2).

Table 2. Results of General Linear Models for Parent, Teacher, and Clinician Ratings of Behavior and Impairment.

	Baseline		3 Month		6 Month		G × T		ηp2
	Controla	Intvnb	Control	Intvn	Control	Intvn
	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)	3 Month	6 Month	3 Month	6 Month
Parent Ratings
DBDRS Inattention	1.8 (0.7)	1.6 (0.7)	1.63 (0.67)	1.28 (0.69)	1.52 (0.74)	1.26 (0.69)	F(57) = 2.82	F(57) = 0.67	0.05	0.01
DBDRS Hyp/Imp	2.1 (0.6)	2.0 (0.6)	1.88 (0.51)	1.50 (0.59)	1.77 (0.66)	1.46 (0.70)	F(57) = 2.49	F(57) = 0.72	0.04	0.01
HSQ: No. Problems	11.3 (2.9)	11.0 (2.9)	11.3 (3.40)	9.2 (3.40)	11.1 (3.40)	8.8 (3.50)	F(57) = 4.26*	F(57) = 6.33*	0.07	0.10
HSQ: M Severity	3.2 (1.5)	2.9 (1.4)	3.31 (1.50)	1.96 (1.20)	2.91 (1.64)	1.97 (1.22)	F(57) = 8.20**	F(57) = 3.30***	0.13	0.06
IRS Impairment	3.5 (1.4)	2.9 (1.0)	3.22 (1.70)	2.13 (1.34)	2.36 (1.47)	1.94 (1.39)	F(57) = 2.45	F(57) = 0.18	0.04	0.00
Teacher Ratings
DBDRS Inattention	1.7 (0.7)	1.4 (0.8)	1.63 (0.83)	1.25 (0.66)	1.43 (0.74)	1.09 (0.75)	F(57) = 0.43	F(57) = 0.15	0.01	0.00
DBDRS Hyp/Imp	1.9 (0.7)	1.5 (0.8)	1.77 (0.69)	1.31 (0.76)	1.50 (0.72)	1.27 (0.84)	F(57) = 0.79	F(57) = 0.44	0.01	0.01
SSQ: No. Problems	6.9 (2.3)	5.7 (3.8)	6.60 (2.53)	4.88 (3.15)	5.17 (3.33)	4.42 (3.20)	F(57) = 0.57	F(57) = 0.34	0.01	0.01
SSQ: M Severity	2.5 (1.7)	2.3 (2.0)	2.26 (1.51)	1.90 (1.70)	1.41 (1.51)	F(57) = 2.35	F(57) = 0.62	0.04	0.01
IRS Impairment	3.3 (1.7)	3.4 (2.2)	2.84 (1.89)	2.50 (1.71)	2.88 (1.73)	2.55 (1.86)	F(57) = 0.93	F(57) = 0.76	0.02	0.01
SKAMP Attention	1.6 (0.6)	1.4 (0.9)	1.43 (0.61)	1.14 (0.63)	1.41 (0.75)	1.04 (0.70)	F(57) = 0.17	F(57) = 0.63	0.00	0.01
SKAMP Deport.	1.6 (0.8)	1.3 (0.8)	1.35 (0.81)	1.03 (0.70)	1.22 (0.73)	1.08 (0.83)	F (57) = 0.02	F(57) = 0.77	0.00	0.01
Clinician Ratings
CGI Severity Score	3.3 (0.8)	3.3 (0.6)	3.13 (0.82)	2.81 (0.70)	3.14 (0.85)	2.91 (0.78)	F(57) = 3.69***	F(57) = 1.40	0.06	0.02

Note: Intvn = intervention group; G × T = Group by Time; DBDRS = Disruptive Behavior Disorders Rating Scale average scores; Hyp/ Imp = hyperactivity/impulsivity; HSQ = Home Situations; IRS = Impairment Rating Scale; SSQ = School Situations; SKAMP = Swanson, Kotkin, Adler, M-Flynn, and Pelham average scores; CGI = Clinical Global Impressions.

^an = 25.

^bn = 34.

^*p < .05

^**p < .01

^***p < .10

Effect Sizes at the 3-Month Time Point

A review of effect sizes at or greater than .02 comparing the intervention and control groups revealed positive effects of the intervention. For behavioral ratings (Table 2), these included improvements for parent ratings of inattentive (η_p² = 0.05) and hyperactive/impulsive symptoms (η_p² = 0.04), reduced and less severe problems in the home (η_p² = 0.07 and η_p² = 0.13 respectively), and less overall impairment (η_p² = .04). In addition, teachers rated the child as having less severe problems at school (η_p² = 0.04) and as being less impaired overall (η_p² = 0.02). Clinicians rated the intervention group as less severe (η_p² = 0.06). Parents rated children in the intervention group as having fewer challenges with emotion regulation on the BRIEF-P (η_p² = 0.02) compared to controls (Table 3).

Table 3. Results of General Linear Models for Parent and Teacher Ratings of Executive Functioning and Child Neurocognitive Measures.

	Baseline		3 Month		6 Month		G × T		ηp2
	Controla	Intvnb	Control	Intvn	Control	Intvn
	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)	3 Month	6 Month	3 Month	6 Month
Parent BRIEF-P T
Scores
Inhibit	76.7 (9.0)	71.4 (11.6)	69.7 (11.2)	64.6 (12.3)	66.4 (11.9)	63.4 (12.3)	F(57) = 0.93	F(57) = 0.37	0.00	0.01
Shift	60.2 (14.6)	57.0 (14.1)	57.4 (11.8)	53.3 (10.2)	55.9 (10.0)	52.3 (10.9)	F(57) = 0.78	F(57) = 0.90	0.00	0.00
Emotion Regulation	71.8 (12.7)	67.2 (10.5)	63.5 (13.9)	62.3 (11.7)	60.7 (13.3)	57.1 (9.5)	F(57) = 0.32	F(57) = 0.73	0.02	0.00
Working Memory	75.4 (11.7)	73.5 (15.4)	68.4 (12.7)	66.7 (13.3)	65.4 (12.7)	65.2 (14.3)	F(57) = 0.95	F(57) = 0.62	0.00	0.00
Plan/Organize	70.4 (10.0)	69.9 (13.2)	64.4 (13.8)	64.0 (12.1)	63.8 (13.6)	61.7 (16.0)	F(57) = 0.94	F(57) = 0.67	0.00	0.00
Teacher BRIEF-P T
Scores
Inhibit	73.9 (14.6)	67.6 (14.8)	70.2 (11.4)	64.1 (13.1)	66.4 (12.1)	62.6 (13.2)	F(57) = 0.94	F(57) = 0.47	0.00	0.01
Shift	60.0 (12.6)	58.1 (13.7)	58.8 (12.8)	56.1 (11.4)	56.1 (12.8)	54.4 (11.1)	F(57) = 0.80	F(57) = 0.93	0.00	0.00
Emotion Regulation	64.2 (16.0)	63.2 (16.4)	62.6 (13.2)	61.3 (13.4)	62.3 (14.1)	60.0 (13.4)	F(57) = 0.92	F(57) = 0.74	0.00	0.00
Working Memory	71.1 (13.8)	67.4 (13.1)	69.8 (11.4)	62.9 (11.2)	68.2 (12.2)	61.8 (13.9)	F(57) = 0.37	F(57) = 0.37	0.01	0.01
Plan/Organize	71.0 (12.5)	66.0 (14.7)	68.1 (11.3)	64 (11.7)	65.8 (11.1)	60.4 (14.1)	F(57) = 0.81	F(57) = 0.92	0.00	0.00
Neurocognitive Tasks—
Subscale Scores
CELF CFD	8.8 (3.7)	9.0 (3.5)	9.6 (3.6)	9.6 (3.5)	9.8 (3.1)	8.8 (3.5)	F(57) = 0.08	F(57) = 1.89	0.00	0.03
NEPSY Visual Attention	10.8 (2.4)	10.6 (2.9)	10.1 (3.2)	10.3 (2.8)	11.0 (2.6)	10.4 (2.8)	F(57) = 0.24	F(57) = 0.12	0.00	0.00

Note: Intvn = intervention group; G × T = Group by Time; BRIEF-P = Behavior Rating Inventory of Executive Function—Preschool T Scores; CELF CFD = Clinical Evaluation of Language Fundamentals Concepts and Following Directions.

^an = 25.

^bn = 34.

Effect Sizes at the 6-Month Time Point

For behavioral ratings, gains were maintained for fewer (η_p² = 0.10) and less severe (η_p² = 0.06) problems in the home setting, and clinician ratings of severity (η_p² = .02). However, gains were not maintained for parent ratings of ADHD symptoms, overall impairment ratings by parent and teacher, or teacher SSQ severity ratings. We did not observe positive effects for the intervention group on the EF measures at the 6-month time point (Table 3). Children in the control group demonstrated improved performance on the Concepts and Following Directions subtest (η_p² = .03).

Discussion

The current study adds to a growing body of research indicating that the use of play-based interventions, such as the GAIM program, is well received by parents and can positively and meaningfully impact functional behavioral outcomes in preschoolers. However, the evidence is less clear that improving EF, at least as measured in the current study, was the mechanism for GAIM-related intervention effects. Rather, it is possible that GAIM-related improvements in behavioral functioning may have resulted from the focus on the parent–child relationship and the emphasis on positive reinforcement of behaviors related to self-regulation in addition to or rather than the enhancement of EF, per se. Nonetheless, improvements were observed for parent ratings of ADHD symptoms, parent and teacher ratings of impairment, and number (home) and severity (home/school) of problems observed in home and school settings, and clinician ratings of impairment. Notably, both teachers and clinicians were masked to the child's treatment condition. Behavioral ratings of EF revealed an effect of intervention at the 3-month time-point on emotional control as rated by parents. Although these effect size findings should be interpreted with caution, it is nevertheless encouraging that the direction of findings for the behavioral outcomes is consistent with previous trials reporting on the same intervention (Tamm & Nakonezny, 2015; Tamm et al., 2012), even when controlling for the effects of therapist attention, expectancy, and natural developmental changes through use of an active control group, which has been shown to impact parent and child outcomes (Kern et al., 2007). Although our effect sizes were generally modest, the case has been made that small changes to the trajectory of ADHD during the preschool years have the potential to yield very substantial benefits later in development (Halperin et al., 2012; Sonuga-Barke & Halperin, 2010). Perhaps the fact that fewer families in the intervention group reported receiving outpatient services from a mental health professional at the 6-month time point than in the control group reflects this impact.

Of interest, among the largest effect sizes favoring the GAIM intervention that was observed at both the 3- and 6-month assessment points were those for parent ratings on the HSQ, with a significant group difference emerging for problem severity. Teachers also reported less severe problems on the SSQ for children in the GAIM versus active control group. The HSQ and SSQ assess situational variability or pervasiveness of behavior problems rather than specific types of behaviors (Barkley & Edlebrock, 1987). Taken together with parent, teacher, and clinician ratings of less overall child impairment, our findings indicate that parents and teachers were observing meaningful reductions in the number and severity of behavior problems the child exhibited after participating in GAIM (i.e., less functional impairment). Further, consistent with previous work (Halperin & Healey, 2011; Halperin et al., 2013; Tamm & Nakonezny, 2015; Tamm et al., 2013), we observed reduced parent ratings of ADHD symptoms for children assigned to the GAIM intervention as compared to the active control group. Given that teachers, presumably masked to treatment condition, also reported reduced inattention and hyperactivity/impulsivity for children in the GAIM condition (although these effect sizes were small; η_p² = .01), it appears that our approach has the potential to alter the trajectory of ADHD symptomatology. Of course, longitudinal studies will be needed to evaluate this possibility.

The overall lack of findings on the EF measures was surprising given the intervention's focus on EF. The BRIEF-P captures everyday behaviors associated with EF and is thought to provide a picture of idiographic cognitive functioning of an individual (McCandless & O'Laughlin, 2007). BRIEF ratings have also been shown to predict later diagnosis with ADHD (Gioia et al., 2000; McCandless & O' Laughlin, 2007), and previous studies with this intervention have shown moderate to large effects on the BRIEF subscales (Kern et al., 2007; Tamm & Nakonezny, 2015). Thus, we expected that we would observe more improvements on this measure. Instead we found reported improvements for the intervention group only on emotional control. The Emotional Control scale measures the impact of EF difficulties on emotional expression and assesses a child's ability to modulate or control his emotional responses, which is an important component of self-regulation. It is not clear why we did not observe greater effect sizes for parent ratings on the other BRIEF-P subscales. One hypothesis is that the GAIM intervention did not in fact impact EF, other than self-regulation skills, which were directly reinforced by parents during the activities. Alternatively, parental bias may have inflated ratings in the active control group, as parents in both groups expected an intervention targeting self-control and EF deficits. The impact on the active control group may have been more powerful than anticipated. Even though the metacognitive and reinforcement strategies utilized in GAIM were not implemented with the control group, the activities used in GAIM were selected to elicit and practice EF skills, and it is possible that the control group experienced some gains simply as a result of being exposed to the EF activities. In fact, simpler, less intense interventions delivered early may be sufficient to address some child behavior problems (D. S. Shaw, Dishion, Supplee, Gardner, & Arnds, 2006; Sonuga-Barke, Thompson, Abikoff, Klein, & Brotman, 2006; Webster-Stratton, Reid, & Hammond, 2004), which may be a particularly relevant point given our focus on “at-risk” preschoolers.

Although EF deficits are frequently observed in children with ADHD, heterogeneity is the norm rather than the exception (Willcutt et al., 2005), and recent work suggests that EF deficits observed in ADHD may be nested within the variability observed in typically developing children (Fair, Bathula, Nikolas, & Nigg, 2012). To examine these possibilities, future work may include children with EF deficits defined by performance on objective neurocognitive tasks. Although we used EF deficits as an inclusion criterion, it was based on parent/teacher ratings; the Clinical Evaluation of Language Fundamentals and NEPSY baseline subscale scores were in the average range for both groups, and we did not have a reliable measure of inhibition, which may be the most impaired EF in preschoolers (Sonuga-Barke, Dalen, Daley, & Remington, 2002). The children most likely to benefit from EF interventions are those with the poorest functioning initially (Diamond & Lee, 2011). Thus, participant characteristics may have limited our ability to detect treatment effects on EF.

Effect sizes were generally larger for behavioral than EF measures, which suggests a few things. First, these data point to the possibility that the focus on parent–child interaction in GAIM is a key component of the intervention. Not only are the activities important to practice, but the delivery of the activities in a positive, reinforcing context, with the parent utilizing sensitive, scaffolding, mind-mindedness, and proactive parenting strategies, is critically important. It is likely that generalizing these parenting skills to other non-GAIM activities (parents reported on average attempting 5.16 generalization attempts per week) also positively impacted child behavior partly driving reductions in HSQ, IRS, and DBDRS ratings. Further, given that the control group did not have a play-based home activity, it may be that spending time with the parent in a positive context is driving the behavioral change in GAIM participants. In fact, it has been shown that parents just spending time with their child can impact behavior (Gardner, Ward, Burton, & Wilson, 2003). It has been argued that the strong link between quality of parent–child relationships and child cognitive development is due to the interplay between genetic factors and the social environment in shaping early brain development (Halperin et al., 2012; Schore, 1996). It would have been ideal to conduct observations of the parents interacting with their children on the GAIM activities to further investigate the quality and content of their interactions.

Studies comparing the efficacy of GAIM relative to parent training programs with a stronger focus on maladaptive behaviors, and those with an interactive component, may be useful to further investigate mechanisms of action. The New Forest Parenting Programme (NFPP; Thompson et al., 2009) combines traditional behavioral management techniques with some sessions focused on training parents to target EFs and on the parent–child interaction, and thus represents a combined behavioral parent training/interaction approach. Studies of the NFPP report reduced ADHD symptoms and conduct problems, and more on-task behaviors than active (parent counseling and support; Sonuga-Barke, Daley, Thompson, Laver-Bradbury, & Weeks, 2001) and waitlist control groups (Abikoff et al., 2015; Sonuga-Barke et al., 2001), but no particular benefit of NFPP compared to a traditional behavioral parent training approach targeting noncompliance with no focus on EF (Abikoff et al., 2015). Strong positive effects for behavioral outcomes are also observed with another combined approach, Parent–Child Interaction Therapy (PCIT), which was developed for use in children with oppositional defiant disorder and conduct disorder, yet has been suggested as potentially useful for children with ADHD (Wagner & McNeil, 2008). PCIT focuses on relationship-building skills (praise, reflection, imitation, description, and enthusiasm), followed by a positive discipline program including effective delivery of commands, responding to child noncompliance, and increasing compliance. However, both the NFPP and PCIT approaches integrate behavioral management techniques targeting maladaptive behaviors in the child, which differs from the current approach. Thus, additional work is needed to investigate the mechanism of action of the GAIM intervention.

Limitations

Perhaps the most important limitation of this study is the inability of the data to inform the mechanism of treatment effects. GAIM may lead to improvements through changes in EF and/or through improving the parent–child relationship and encouraging parents to practice effective strategies (e.g., labeled praise). The active control condition did not include a structured weekly playtime, as was prescribed in GAIM, which makes it difficult to disentangle whether the mechanism of treatment was the focus on EF or the quality of the parent–child interaction or both. Because the control group received active intervention, it may be that both groups benefitted. This hypothesis gains some traction, as previous work comparing GAIM to a waitlist control group showed larger effects than the current study on EF and behavioral measures (Tamm & Nakonezny, 2015), and Kern et al. (2007) reported positive effects of the parent education condition on children's behavior. Also, the study was underpowered, resulting in a focus on effect sizes. A larger randomized clinical trial is necessary to statistically assess the efficacy of GAIM, as well as to directly test treatment mechanisms and moderators. Children were generally rated in the mildly impaired range on the CGI, which may affect generalizability of these findings to more clinically impaired samples. We relied on parental self-report regarding the number of activities practiced per week and attempts at generalization rather than a more objective assessment. Further, we did not collect information from families assigned to GAIM regarding how much they continued practice of the GAIM activities and strategies after completing intervention. We also did not include a comprehensive battery of EF tasks capturing attention, inhibition, and memory, and participants were not generally poor performers on the objective EF neurocognitive tasks. Individuals who dropped out prior to the start of the intervention were more likely to be from a lower socioeconomic background and to be African American; thus, our findings may not generalize to these populations.

Conclusion

The results of this trial and of previous studies using similar interventions (Halperin et al., 2013; Healey & Halperin, 2015; Tamm & Nakonezny, 2015; Tamm et al., 2012) suggest the potential of developmentally appropriate interventions to improve behavior in at-risk preschoolers. Specifically, play-based activities targeting aspects of self-regulation and the parent–child relationship, when administered in a structured way by parents, is a promising approach for improving functional outcomes and reducing impairment in preschoolers with self-regulation deficits. Although promising, the intervention cannot yet be classified as “evidence-based” nor should it be considered a first-line treatment for preschoolers with EF deficits. Additional work is needed to investigate the mechanism of action in children with objectively measured EF deficits related to ADHD (sustained attention, working memory, inhibition, etc.). It may be that the focus on EF in GAIM is not necessary and that the intervention's focus on the structure and quality of parent-child interactions is in fact a mediator of outcomes rather than the converse. Alternatively, it may be that both the focus on EF and the parent-child interaction is warranted for children at risk for ADHD, which is likely to result from multiple determinants. Further work is also needed to assess the relative efficacy of such interventions compared to behavioral parent training, as well as integrative approaches.