Abstract
Objective:
Treatment for childhood anxiety disorders is insufficient in many cases. Parent involvement has been examined as an augment to child-based cognitive behavioral therapy (CBT), but no studies have compared the efficaciousness of stand-alone parent-based treatment to CBT. Research implicates family accommodation in the maintenance and course of childhood anxiety. Supportive Parenting for Anxious Childhood Emotions (SPACE) is a parent-based treatment that reduces accommodation of childhood anxiety. This study compared SPACE to CBT in a noninferiority trial.
Method:
Participants were children with primary anxiety disorders (N=124; 7–14 years old; 53% female participants; 83% white), randomly assigned to either SPACE (N=64) with no direct child-therapist contact, or CBT (N=60) with no parent treatment. Ninety-seven (78%) of participants completed all treatment sessions and assessments. Attrition did not differ significantly between groups. Primary anxiety outcomes included diagnostic interviews and clinician-rated scales. Secondary outcomes included parent and child ratings of anxiety severity, family accommodation, and parenting stress. Noninferiority margins were determined based on statistical and clinical considerations. Change in family accommodation and parenting stress were examined using mixed models analyses.
Results:
SPACE was noninferior, relative to CBT, on primary and secondary anxiety outcomes, and based on ratings provided by independent evaluators, parents, and children. Family accommodation and parenting stress were significantly reduced in both treatments, with significantly greater reduction in family accommodation following SPACE, compared to CBT. Treatment credibility and satisfaction were high.
Conclusion:
SPACE is an acceptable and efficacious treatment for childhood anxiety disorders, noninferior to CBT, and provides an alternative strategy for treating anxiety in children.
Keywords: anxiety disorders, parent-based treatment, cognitive behavioral therapy clinical trials, family accommodation
INTRODUCTION
Childhood anxiety disorders are common, chronic, and impairing, and confer major short-term and long-term risks to physical and mental health when not treated successfully.1,2 Efficacious treatments include cognitive behavioral therapy (CBT) and medications, but are insufficient in up to half of cases in clinical trials3 indicating the need for additional treatment options.
Decades of research, tying parent and family variables to the etiology and course of childhood anxiety disorders, led to repeated efforts to improve outcomes by involving parents in treatment. Early outcomes suggested a benefit to child-and-parent treatment over child-only treatment.4 Subsequent research however has failed to support this conclusion and reviews and meta-analyses have concluded that outcomes of child-only treatment are comparable to those of child-and-parent treatment,5,6 although the latter may be superior when parent anxiety is high.7 These well-documented findings underscore the importance of identifying alternative treatment targets if parent-work is to significantly enhance outcomes.
In contrast to numerous randomized studies examining whether parent-involvement enhances child-based treatment,4,8–12 it is unknown whether parent-based treatment alone, without child-based therapy, is efficacious. Only two randomized trials tested parent-only interventions for childhood anxiety, and neither included a comparison with CBT. One study, specifically aimed at young children below age nine, compared a parent-only group intervention to a waitlist condition.13 Another compared two versions of parent-guided CBT to waitlist.14 Both studies showed promising results, as have a number of open trials,15–17 suggesting parent-based treatment may present an efficacious alternative to child-based therapy. Parent-led and family-based interventions have also been developed for anxiety-related problems including OCD18 and PTSD19. But whether parent-only treatment can be as efficacious as CBT for child anxiety remains unanswered.
Another critical question is what should be the focus and objectives of efficacious parent-based interventions for childhood anxiety? Shifting the focus of treatment from child to parents opens the door to a meaningful change in treatment conceptualization, enabling development of novel approaches grounded in theoretical and empirical research on the unique role of parents for child anxiety. In human and non-human mammals offspring respond to anxiety with parent-oriented attachment behaviors, and parental proximity exerts anxiolytic effects on offspring.20–22 Human parents reduce child anxiety through physical contact,23 and more complex behaviors such as verbal reassurance. Burgeoning research underscores the importance of considering these parental responses to child anxiety, in particular the high levels of family accommodation consistently reported by parents of anxious children.24–29
Family accommodation refers to the myriad changes in parental behaviors and routines intended to help a child avoid or alleviate anxiety-related distress. Despite being well-intentioned, family accommodation is linked to more severe child anxiety and greater functional impairment, and may predict poorer response to CBT.24,30–34 Examples of family accommodation include sleeping next to a child with separation anxiety, speaking for a child with social phobia, or repeatedly reassuring a child with generalized anxiety. From a theoretical perspective, family accommodation may maintain child anxiety by promoting avoidance and maintaining the child’s reliance on parents rather than developing independent coping skills. Family accommodation can also reduce child motivation for treatment, by providing the means to avoid otherwise anxiety-provoking situations.
Translating research on these patterns of familial interactions into novel clinical applications enables the shift from child-work to parent-work to be a meaningful change in the principles and components of treatment, rather than a change in the modality of treatment delivery alone. Parent-based interventions for child anxiety to date have derived primarily from traditional CBT, with parents trained as lay CBT therapists. As such, the interventions have focused on child’s behavior and cognitions with comparatively little emphasis on family accommodation.5,10,13,14,16,35
The empirical and theoretical rationale for parent-based treatment focused on reducing family accommodation, and the critical need for alternatives to currently available treatments, led to the development of SPACE (Supportive Parenting for Anxious Childhood Emotions).36 SPACE is unique in making the reduction of parental accommodation the centerpiece of the intervention. Rather than training parents as lay CBT therapists, SPACE focuses on systematically identifying and monitoring family accommodation, developing and implementing detailed plans for reducing accommodation, and equipping parents with strategies for coping with children’s distressed and/or aggressive responses to reduced accommodation. Because SPACE focuses entirely on parent-change, parents can implement SPACE even when a child is not amenable to treatment. A pilot trial of SPACE provided initial support for its feasibility, acceptability, and potential efficacy.17 Parents of ten children (ages 9–13) participated in weekly SPACE sessions, with no direct child-therapy. All participants completed treatment and client satisfaction was high. Child anxiety was significantly reduced following treatment, as were ratings of family accommodation. Another small pilot of SPACE, with parents of children with OCD, also showed significant improvement and high satisfaction.37
The present investigation was a randomized controlled noninferiority trial to determine whether SPACE is as efficacious as CBT, the best-established strongest evidence-based treatment for childhood anxiety disorders. Noninferiority methodology was selected rather than the more commonly reported superiority testing because failure to show superiority of one treatment over another is insufficient evidence of treatment equivalence (see Data Analysis). Participants were randomly assigned to one treatment or the other, with those assigned to SPACE receiving no direct child-based treatment, and those assigned to CBT receiving no parent-based treatment. Primary outcomes were clinician-rated measures of child anxiety. Secondary outcomes included child and parent ratings of child anxiety and of family accommodation, as well as a parent-rated measure of parenting stress, to investigate the impact of SPACE on parenting stress associated with childhood anxiety. We hypothesized that: 1) SPACE would not be inferior to CBT on primary outcomes of child anxiety; 2) SPACE would not be inferior to CBT on secondary outcomes of child and parent rated child anxiety symptoms, and parenting stress; 3) SPACE would be associated with greater reduction in family accommodation of child anxiety symptoms, compared to CBT; 4) SPACE would not be inferior to CBT on treatment credibility and client satisfaction.
METHOD
Study Design
The study was approved by the university’s Institutional Review Board and registered on www.clinicaltrials.gov. A total of 124 children were assigned to either SPACE or CBT in a 1:1 ratio using a computerized randomization algorithm (Figure 1). Parents of children assigned to SPACE received 12 parent-only sessions, following the manualized SPACE treatment protocol,17,36 with no direct child-therapist contact. Children assigned to CBT received 12 sessions of exposure-based CBT, following an established manualized treatment protocol used in previous child anxiety trials.38 Parents of children assigned to CBT received no parent-treatment sessions but met with the child’s therapist at the start, middle, and end of treatment, for approximately 20 minutes each time, to keep them informed about their child’s therapy. Independent evaluators (IEs) masked to study arm completed assessments with parents and children at baseline, mid-treatment, and post-treatment.
Figure 1: CONSORT Diagram of Study Enrollment and Retention.
Note: Response indicates Clinical Global Impression Scale (CGI) improvement rating of ‘Very Much Improved’ or ‘Much Improved’. Remission indicates a CGI Severity rating of ‘Not At All Ill’ or ‘Borderline Ill’. CBT = individual cognitive behavioral therapy; SPACE = Supportive Parenting for Anxious Childhood Emotions.
Participants
Participants were 124 children aged 7–14 years (Mean age: 9.6 years, SD=2.45; 53% female participants), randomly assigned to SPACE (n=64) or CBT (n=60). Parents self-referred or were referred by providers including mental health providers within secondary and tertiary care settings, primary care general practitioners, and school personnel, between 2013 and 2018. The sample was predominantly white (83%) and non-Hispanic (88%), with a minority being black (6%) or of more than one race (11%). Most parents were married or in domestic partnerships (92%; 4% single; 4% divorced). Parents’ modal educational attainment was Master’s level (40%; 28% Bachelor’s; 12% some college; 9% professional/technical degree; 6% Associate’s; 3% high school; 2% PhD). Most parents (76%) were employed during the study; modal family income was >$125,000 (49%; 19% $100,000-$124,999; 10% $81,000-$99,999; 9% $61,000-$80,999; 7% $41,000-$60,999; 4% $21,000-$40,999; 2% $0-$20,999).
Primary anxiety diagnoses were: generalized anxiety disorder (35.2%), social phobia (34.8%), separation anxiety disorder (18.2%), specific phobia (11.8%). Comorbidity was common with 75% having at least two anxiety disorders, and 48.4% having at least one nonanxiety diagnosis. Child medications included antidepressants (11%) and stimulants (7%). Table 1 summarizes baseline characteristics for the sample overall and by treatment arm.
Table 1.
Baseline Characteristics Overall and by Treatment Arm
| Full Sample (N=124) | SPACE (n=64) | CBT (n=60) | t / χ2 | p | |
|---|---|---|---|---|---|
|
|
|||||
| Child Age, M (SD) | 9.4 (2.41) | 9.1 (2.2) | 9.9 (2.54) | .8 | .35 |
| Child Sex, % female | 53 | 47 | 61 | 2.5 | .11 |
| Parent Age, M (SD) | 42.3 (5.9) | 41.6 (6.5) | 43.1 (5.2) | 1.2 | .21 |
| PARS, M (SD) | 19.3 (4.3) | 19.8 (4.21) | 18.8 (4.64) | 1.2 | .24 |
| CGI-S, M (SD) | 4.98 (.83) | 4.9 (.82) | 5.1 (.84) | 1.1 | .28 |
| SCARED parent report, M (SD) | 31.4 (11.26) | 31.5 (12.1) | 31.2 (10.38) | .21 | .83 |
| SCARED child report, M (SD) | 32.7 (14.49) | 31 (13.85) | 34.6 (15.06) | 1.37 | .17 |
| FASA, M (SD) | |||||
| Total | 15.7 (8.09) | 16.7 (7.97) | 14.6 (8.13) | 1.4 | .14 |
| Participation | 10.7 (4.63) | 11.3 (4.52) | 10.1 (4.71) | 1.5 | .14 |
| Modification | 5 (4.23) | 5.4 (4.18) | 4.5 (4.27) | 1.5 | .14 |
| Distress | 1.51 (0.94) | 1.65 (0.97) | 1.36 (0.89) | 1.7 | .09 |
| Consequences | 5.25 (3.16) | 5.63 (2.98) | 4.84 (3.32) | 1.3 | .17 |
| FASA-CR, M (SD) | |||||
| Total | 13.2 (6.5) | 13.2 (6.7) | 13.2 (6.3) | .03 | .97 |
| Participation | 9.1 (4.0) | 8.9 (4.1) | 9.3 (3.9) | .47 | .64 |
| Modification | 4.1 (3.5) | 4.2 (3.6) | 3.9 (3.4) | .47 | .64 |
| Distress | 1.35 (1.26) | 1.41 (1.34) | 1.29 (1.18) | .52 | .61 |
| Consequences | 5.93 (3.17) | 6.18 (3.41) | 5.65 (2.89) | .92 | .36 |
| PSI | 133.6 (20.6) | 131.49 (19.42) | 135.9 (21.68) | 1.02 | .31 |
| Comorbid diagnosesAny comorbid anxiety, % | 75 | 80 | 70 | 1.8 | .17 |
| Depression, % | 10 | 8 | 12 | .62 | .43 |
| ADHD, % | 18 | 16 | 20 | .53 | .47 |
| OCD, % | 14 | 11 | 17 | 1.1 | .31 |
| ODD, % | 13 | 14 | 12 | .1 | .74 |
| MedicationsSSRI | 14 | 8 | 6 | .19 | .66 |
| Stimulants | 9 | 4 | 5 | .2 | .65 |
Note: ADHD = attention-deficit/hyperactivity disorder; CGI-S = Clinical Global Impressions – Severity; FASA = Family Accommodation Scale– Anxiety; FASA-CR = Family Accommodation Scale – Anxiety Child Report; OCD = obsessive-compulsive disorder; ODD = oppositional defiant disorder; PARS = Pediatric Anxiety Rating Scale; PSI = Parenting Stress Index; SCARED = Screen for Childhood Anxiety Related Emotional Disorders; SPACE = Supportive Parenting for Anxious Childhood Emotions; SSRI = Selective Serotonin Reuptake Inhibitors.
Inclusion criteria were: primary DSM-5 anxiety disorder diagnosis; aged 7–14 years; residing with mother at least 50% of the time; fluent in English; medication-free or on a stable dose of antidepressant or stimulant when child and parent, upon consultation with the prescriber, agreed to refrain from changes during the study period; parental informed permission and consent; child assent.
Exclusion criteria were: drug or alcohol abuse; psychotic symptoms; autism spectrum disorder; any comorbid disorder more impairing than the most severe anxiety disorder; concurrent psychotherapy or medication, apart from stable dose of antidepressant or stimulant; serious suicidal intent or risk.
Mothers were the identified participating parents, had to be present in all SPACE sessions, and completed all parent evaluations and assessments. Fathers could choose to attend sessions and were present in 12% of sessions, attending at least once in 25% of cases. Father attendance was not found to be related to any baseline or outcome variables.
Randomization success was confirmed using χ2 and t-tests. There were no significant differences between the treatment groups in anxiety diagnoses or on demographic or study variables (see Table 1).
Procedures
Following initial telephone screening, families were invited to the baseline evaluation, and after providing informed consent and assent were administered separate diagnostic interviews and a standardized assessment battery. Parents and children returned to the clinic one week later, received clinical feedback, reviewed the study protocol, and were subsequently randomized to SPACE or CBT. Therapists were crossed across treatment arms to reduce therapist variance. Following the 6th session a mid-treatment evaluation was conducted. Following the 12th and final treatment session a post-treatment evaluation was conducted including diagnostic interviews and primary and secondary outcomes.
Treatment Arms
CBT.
CBT Was the comparator arm given it is the best-established treatment for childhood anxiety with the strongest evidence base, and the current standard of care.39 The study utilized a prototypical CBT manual utilized in previous clinical trials.38 Children met alone with their therapist for 12 weekly 60-minute sessions. The first sessions included discussion of the presenting problem and psychoeducation about anxiety and the treatment rationale. Then an exposure hierarchy was devised, and therapy focused on in-session and out-of-session exposures. Cognitive work included identification of faulty cognitions, generating alternative cognitions and self-statements, and practicing cognitive restructuring in-session and out-of-session. Termination included review of gains and remaining problems, and relapse prevention. Therapists met with parents to provide information on the child’s therapy and elicit information to inform exposure hierarchies, but were trained not to provide parent-guidance or suggest modifications to parental behavior. Parents who asked for guidance were told to encourage their child to utilize skills learned in therapy.
SPACE.
Parents of children assigned to SPACE participated in 12 weekly 60-minute sessions. The study utilized the published SPACE manual.17,36 The first sessions included discussion of the child’s presenting problem, and the rationale for SPACE including addressing any concerns about parent-based treatment. Parents were then taught supportive responses to child anxiety that acknowledge the child’s experience while also conveying confidence in the child’s ability to cope. Family accommodations were carefully and comprehensively mapped out and a target accommodation was selected for modification. A detailed plan for changes to the accommodation was constructed, and parents were instructed in how to communicate the plan to the child. Treatment then focused on implementation and trouble-shooting of the accommodation reduction plan and parents monitored their accommodation between sessions. When the accommodation was successfully reduced, a second target was selected and addressed in similar manner. SPACE includes modules for problem-solving common difficulties relating to child responses to reduced accommodation, including distress, anger, and aggression.
Therapist Training and Treatment Integrity and Fidelity.
Therapists were doctoral and postdoctoral level psychology students who received extensive training in both treatments. Training included didactic learning, viewing of treatment sessions, and leading a case prior to independently treating study cases. Weekly supervision to all study therapists was co-led by the primary investigator and another clinician with decades of experience supervising CBT. The two treatments are highly distinct: SPACE is parent-only and not focused on child behavior; CBT is child-only and focused entirely on child thoughts and behaviors. Fidelity and lack of carry-over were confirmed through fidelity checklists completed by clinicians after each session and at the end of treatment, and by independent raters. All treatment sessions were video-taped and 25% of sessions were randomly selected for review by the independent raters using the fidelity checklists used by the clinicians. Weekly supervision was used to review treatment delivery and promptly address any drift in fidelity.
Measures
Anxiety Disorders Interview Schedule: Child and Parent Versions.
The Anxiety Disorders Interview Schedule: hild and Parent Versions (ADIS C/P)40 is a semi-structured diagnostic interview with excellent psychometric properties considered the gold-standard in establishing childhood anxiety diagnoses. The interview was administered separately to children and parents. Final diagnoses were determined by integrating information from both, and agreed upon by expert consensus, including one of the interview’s authors. Remission on the ADIS was defined conservatively as loss of primary and all other anxiety disorders post-treatment.
Pediatric Anxiety Rating Scale.
The Pediatric Anxiety Rating Scale (PARS)41 is a clinician-administered child anxiety severity measure with established psychometric properties that has been used in major clinical trials. The PARS consists of a 50-item symptom checklist followed by global items that rate severity of identified symptoms on a 6-point scale. The PARS was administered to children and parents together and six global items were summed to produce a total score from 0 to 30.42 Inter-rater reliability was established for IEs (r=0.9).
Clinical Global Impressions.
The Clinical Global Impressions (CGI) scales are widely used in clinical trials and provide clinician ratings of overall severity of psychopathology (CGI-S) and overall improvement following treatment (CGI-I). CGI-S scores range from 1 (‘no illness’) to 7 (‘severely ill’); CGI-I scores range from 1 (‘very much improved’) to 7 (‘very much worse’). Remission on CGI-S was defined as a post-treatment rating of ‘not at all ill’ (1) or ‘borderline ill’ (2);43 treatment response was defined as a post-treatment rating of ‘very much improved’ (1) or ‘much improved’ (2).44
Screen for Childhood Anxiety Related Emotional Disorders.
The Screen for Childhood Anxiety Related Emotional Disorders45 (SCARED) is a 41-item rating scale of childhood anxiety symptoms. Parallel parent and child versions were administered. The SCARED has established psychometric properties.45,46 Internal consistency was excellent (α=.89 for the parent-version and α=.91 for the child-version).
Family Accommodation Scale – Anxiety.
The Family Accommodation Scale – Anxiety is a rating scale for assessing family accommodation of childhood anxiety. Parallel parent24 (FASA) and child25 (FASA-CR) versions were administered. A total accommodation score is calculated from 9-items that rate frequency of accommodations on a 5-point scale. Two subscores are calculated from items pertaining to active participation in symptoms and modification of family routines and schedules. Additional items query parental distress stemming from accommodation, and short-term negative child responses to not being accommodated. FASA and FASA-CR are the most widely used measures of family accommodation of childhood anxiety with established psychometric properties. Internal consistency was excellent for FASA (α=.9) and good for FASA-CR (α=0.8).
Parenting Stress Index.
The Parenting Stress Index47 (PSI) is a 36-item index of parenting-related stress, scored on a 5-point scale. The PSI has been widely used and has established psychometric properties. Internal consistency was excellent (α=.91).
Client Credibility Questionnaire.
The Client Credibility Questionnaire48 (CCQ) is a 4-item questionnaire that assesses perceptions of the rationale for psychotherapeutic interventions, and expectancies of treatment outcomes. Parallel parent and child versions were administered after subjects were randomized and treatment rationale was explained.
Client Satisfaction Questionnaire.
The Client Satisfaction Questionnaire (CSQ-8) is an 8-item questionnaire that assesses satisfaction with treatment services. Parallel versions were administered post-treatment.
Independent Evaluator Training and Reliability
IEs were master’s and doctoral level clinicians, trained according to procedures established by instrument developers including didactic learning, observation, comparison with ratings by expert clinicians, and live weekly supervision. IEs were masked to treatment assignment. Inter-rater reliability for pre-treatment and post-treatment ratings was excellent (ICC>0.9 for all comparisons).
Data Analytic Strategy
Noninferiority methodologies differ from superiority methodologies that test whether a treatment is superior to a comparator against a null hypothesis that both treatments are the same. Failing to reject this null hypothesis, in superiority testing, does not necessarily indicate treatment equivalence. Rather, it indicates that any differences detected are insufficient to confidently reject the possibility that the treatments do not differ. Any number of factors could contribute to the null hypothesis not being rejected in superiority analysis (e.g., lack of power), and thus equivalence can only be established when tested against a null hypothesis that the treatments do in fact differ. This is the goal of noninferiority testing. In noninferiority testing the null hypothesis posits that the comparator condition is superior to the treatment being tested, and is rejected only when CIs around the mean differences between treatment arms do not exceed a predetermined noninferiority margin. The noninferiority margin is selected to represent the amount of ‘acceptable difference’, or the maximum difference in outcomes that is permitted for both treatments to still be considered equivalent. Because the noninferiority test is essentially one-tailed some researchers advocate using 97.5% CIs; this conservative approach was implemented in the current study. Furthermore, because intent-to-treat analyses (ITT) can artificially increase the perception of noninferiority through narrower CIs, testing focused on treatment completers for whom post-treatment data was available (per FDA guidelines for noninferiority trials).
Establishing noninferiority margins.
For the primary outcome (PARS) the noninferiority margin was set at 6-points. Thus, the null hypothesis that SPACE is inferior would be rejected if the upper limit of the 97.5% CI around the mean posttreatment PARS score for children who received SPACE was no more than 6-points higher than the mean posttreatment PARS score for CBT. The 6-point margin was selected based on statistical and clinical considerations. As noninferiority margins were not previously established for PARS we first calculated the reliable change index (RCI) for PARS. The RCI is a statistic that determines the magnitude of change necessary to identify reliable change on a given self-report measure and is calculated as: RCI = √(2 × (SE)2 where SE is the standard error of measurement. The RCI for PARS, based on previously published data, including results from the childhood anxiety multimodal treatment study (CAMS), is 8-points.44 Clinically, an 8-point noninferiority margin may be considered overly lenient. We therefore further reduced the margin by 25% to 6-points. This margin was further supported by research indicating that the average reduction in PARS score that optimally predicts treatment response is 6-points.44
Noninferiority margins for child and parent SCARED were 11 and 13 points, respectively, and were established in similar manner. The RCI for child and parent SCARED are 21 and 17-points, respectively. These were reduced by 25% to 15 and 13-points. Because the 15-point margin for the child-rated SCARED is still greater than the average reduction in child SCARED scores that optimally predicted treatment response in CAMS49 (i.e., 11 points) the noninferiority margin was further reduced to 11.
Mean differences between treatment arms on primary and secondary outcomes, with 97.5% CIs, were compared using the t-tests function in SAS 9.4.
Improvement on secondary outcomes of family accommodation (FASA; FASA-CR) and parenting stress (PSI) were examined using separate mixed model effects with Group (SPACE, CBT), Time (Pre- Mid- Post-Treatment), and a Group X Time interaction covarying for baseline anxiety severity, using unstructured covariance matrices to account for within-subject correlation across measurement times, and fit via restricted maximum likelihood. Estimated marginal means were examined to characterize longitudinal patterns in significant interactions.
Power Analysis.
Power calculation using PASS-16 with alpha set at 0.025 indicated a sample of 41 children in each treatment arm would provide ≥90% power for noninferiority margins of primary and secondary outcomes. Actual recruitment was larger to allow for expected attrition up to 30%.
RESULTS
Treatment Retention
Ninety-seven participants (78%) completed the post-treatment assessment (Figure 1). Retention did not differ significantly between SPACE (n=48) and CBT (n=49) (χ2 = .8, p=.34). Treatment completers and non-completers did not differ on clinical or sociodemographic variables. In CBT only, non-completers had higher baseline parent-rated child anxiety than completers (t=2.3, p<.05). No other significant differences emerged for either treatment. Before conducting further analyses, we assessed for missing data bias, outliers, and statistical violations, which were found to be inconsequential.
Primary Outcomes
PARS.
Table 2 summarizes descriptive statistics for all outcomes at baseline, mid-treatment and post-treatment, for study completers. Figure 2A presents the 97.5% CIs for the mean difference between SPACE and CBT on the primary outcome of PARS. The 97.5% CI lay entirely below the 6-point noninferiority margin, indicating that SPACE was not inferior to CBT (p<.001).
Table 2.
Outcome Measures for Treatment Completers at Baseline, Mid-Treatment, and Post-Treatment
| Baseline | Mid-Treatment | Post-Treatment | ||||
|---|---|---|---|---|---|---|
|
|
||||||
| SPACE | CBT | SPACE | CBT | SPACE | CBT | |
|
|
|
|
||||
| CGI-I | 1.61 (.58) | 1.65 (.75) | ||||
| CGI-S | 4.92 (.84) | 4.97 (.84) | 2.17 (1.16) | 2.35 (1.25) | ||
| PARS | 19.89 (4.36) | 18.65 (4.43) | 7.88 (3.79) | 8.98 (4.69) | ||
| SCARED parent report | 32.35 (12.11) | 29.78 (10.27) | 28.9 (13.6) | 27.1 (9.9) | 22.04 (13.72) | 17.29 (12.43) |
| SCARED child report | 30.43 (14.52) | 33.43 (14.91) | 25.4 (14.9) | 28.7 (14.9) | 22.12 (13.9) | 19.63 (14.05) |
| FASA | 16.7 (7.89) | 14.2 (8.39) | 12.7 (7.9) | 13.1 (9.3) | 8.52 (6.8) | 7.68 (6.11) |
| FASA-CR | 14.1 (6.7) | 12.9 (6.5) | 8.9 (7.5) | 8.1 (6.6) | 7.2 (5.51) | 6.63 (5.35) |
| PSI | 131.32 (19.53) | 137.63 (20.96) | 79.8 (12.9) | 81.9 (14.2) | 82.69 (13.14) | 83.7 (11.73) |
Note: CGI-I = Clinical Global Impressions – Improvement; CGI-S = Clinical Global Impressions – Severity; FASA = Family Accommodation Scale – Anxiety; FASA-CR = Family Accommodation Scale – Anxiety Child Report; PARS =Pediatric Anxiety Rating Scale; PSI = Parenting Stress Index; SCARED = Screen for Childhood Anxiety Related Emotional Disorders; SPACE = Supportive Parenting for Anxious Childhood Emotions.
Figure 2: Mean Differences and Confidence Intervals for Child Anxiety Outcomes.
Note: Panels A, B, and C, show mean differences in PARS (Pediatric Anxiety Rating Scale), parent-rated SCARED (Screen for Childhood Anxiety Related Disorders), and child-rated SCARED scores, respectively. The Diamond indicates the actual differences and bars indicate the 97.5% CI around the mean. The dotted lines indicate the noninferiority margin for each measure. Mean differences below 0 indicate lower anxiety following SPACE while mean differences above 0 indicate lower anxiety following CBT.
Response and Remission.
There were no significant group differences in the proportions of participants classified as treatment responders on CGI-I (SPACE: 87.5%, CBT: 75.5%; χ2=3.2, p=0.7). Likewise, there were no significant group differences in the proportions of participants classified as remitters on CGI-S (SPACE: 58.3%, CBT: 59.2%; χ2=.02, p=.88) or ADIS C/P (SPACE: 68.8%, CBT: 63.3%; χ2=.32, p=.57).
Secondary Outcomes
SCARED.
For both child and parent SCARED, the 97.5% CIs for the difference between treatments lay entirely below the noninferiority margin, indicating that SPACE was not inferior to CBT (P<.01 and p<.01 respectively). Figure 2-B,C presents the 97.5% CIs for SCARED.
FASA/FASA-CR.
Mixed models analysis indicated parent-rated family accommodation was reduced significantly in both treatments (FTIME = 3.42, p<.05; FTREATMENT = .964, p=0.3). A significant interaction emerged between treatment arm and timepoint, indicating greater reduction in family accommodation following SPACE, compared with CBT (FINTERACTION = 3.51, p<.01). Figure 3 presents longitudinal estimated marginal means data from the mixed models analysis for change in parent-rated family accommodation. Also apparent in Figure 3, the rate of accommodation reduction was linear for SPACE with 48% of reduction occurring in the first half of treatment, whereas in CBT accommodation reduction occurred mostly (65%) in the second half of treatment. Child-rated family accommodation was likewise reduced significantly in both treatments, but no significant interaction emerged between time and treatment arm.
Figure 3. Parent-Rated Family Accommodation by Treatment Condition From Baseline to PostTreatment.
Note: Data presented for estimated marginal means from mixed models analysis covarying for baseline child anxiety. CBT = Cognitive Behavioral Therapy; SPACE = Supportive Parenting for Anxious Childhood Emotions.
PSI.
Mixed models analysis indicated parenting stress was significantly reduced in both SPACE and CBT (FTIME = 1196, p<.001; FTREATMENT = .27, p=.6), with no significant interaction between time and treatment arm (FINTERACTION = .98, p=.32).
Treatment Credibility and Satisfaction
CCQ.
Treatment credibility was high, and not significantly different for SPACE and CBT. Child-rated credibility averaged 2.4 (SD=.33) for SPACE and 2.5 (SD=.41) for CBT, from a maximum of 3 (t=1.7, p=.09). Parent-rated credibility averaged 6.8 (SD=1.3) for SPACE and 7.3 (SD=1.2) for CBT, from a maximum of 9 (t=1.6, p=.11).
Satisfaction.
Children and parents reported high levels of satisfaction with both treatments, with no significant differences between groups. Average child-rated satisfaction was 27 (SD=4.9) for SPACE and 28 (SD=3.3) for CBT (t=1.9, p=.06.) Average parent-rated satisfaction was 28 (SD=3.2) for SPACE and 28 (SD=4.5) for CBT (t=.34, p=.74).
DISCUSSION
SPACE, a novel completely parent-based treatment for childhood anxiety disorders, was as efficacious as CBT, the established treatment for childhood anxiety with the strongest evidence base. Noninferiority of SPACE was established for both primary and secondary outcomes, and based on ratings by IEs, parents, and children. In contrast to numerous studies of parental involvement in child-based therapy, entirely parent-based treatment protocols are exceedingly rare in childhood anxiety research.13,14 This is the first randomized clinical trial to compare parent-based treatment to child-based treatment.
Theoretical and empirical research supports a unique role for parents in childhood anxiety, stemming from children’s natural reliance on caregivers for protection and reassurance. Chronic activation of this interpersonal parent-oriented anxiety response entangles parents in childhood anxiety symptoms through high levels of family accommodation.24,26,32 Family accommodation, which causes significant distress to parents and maintains child anxiety over time,26,29–31,34 provides a target for novel interventions. SPACE stems directly from this interpersonal formulation of child anxiety.
The finding that SPACE is as efficacious as CBT has direct clinical implications. For clinicians, efficacious parent-based treatment provides an alternative approach to be deployed alongside or instead of CBT. Parent-based treatment may be particularly useful when child-based treatment is not a viable option, such as when severe developmental or communication problems preclude individual or cognitive interventions. The finding that baseline severity predicted attrition in CBT but not in SPACE may point to its usefulness for severe anxiety cases. But the high consent rate and the absence of significant differences in attrition, credibility or satisfaction, between SPACE and CBT support the broad acceptability and feasibility of SPACE for a wide variety of cases.
More research is required to replicate these findings, and address questions pertaining to optimal selection and sequencing of parent-based and child-based therapies. Examination of differential response patterns to the two treatments, based on child, parent, or family variables, may enhance clinicians’ ability to personalize treatment selection. Mounting evidence for the efficaciousness of parent-based treatments through replication studies will also have important but challenging policy implications. One such challenge is ensuring reimbursement for parent-based treatments matches reimbursement for child-based therapy.
Research is also required to investigate the respective mechanisms of action responsible for clinical improvement in SPACE and CBT. SPACE represents a natural next step in translating the rapidly expanding research on family accommodation into a treatment for childhood anxiety.24,30–34 The finding that parent-rated family accommodation was significantly more reduced following SPACE, compared with CBT, supports the premise of SPACE that reducing accommodation will improve child anxiety. This hypothesis is also in line with the finding that reduction in family accommodation occurred equally across treatment in SPACE, whereas in CBT the reduction in accommodation occurred mostly in the latter half of treatment. It may be that reduction in family accommodation in SPACE preceded, and led to, reduction in anxiety symptoms, whereas in CBT reduced accommodation resulted from lower anxiety levels as treatment progressed. Research on directionality of change in child anxiety treatment is sparse and needed to advance understanding of mechanisms of change.38
Paternal involvement in treatment did not significantly impact outcomes in the current trial. Data on family accommodation by fathers are lacking and may inform understanding of the importance of involving fathers in parent-based treatment.
The current study must be considered in light of certain limitations. The study included two active and potent treatment arms and, as is common in noninferiority trials, did not include an inactive or sham treatment arm. It was necessary to restrict parental involvement in CBT to ensure treatment differentiation and enable the comparison of child- and parent-based treatments, but this differs from the greater parental involvement often employed in CBT. It would be useful to compare SPACE and CBT to a parent-and-child treatment, combining CBT with parent-work focused on reducing accommodation. The sample was mostly white and of medium-to-high socioeconomic status, making it important to establish whether findings generalize to heterogenous populations. Research on CBT has indicated low socioeconomic status may predict poorer treatment response.50
Despite these limitations the study is important and novel, being the first clinical trial to establish the efficacy of a parent-based intervention relative to the standard of care CBT, and the first randomized trial of SPACE, a treatment focused exclusively on shaping parental behavior. Findings indicate that SPACE is as efficacious as CBT for childhood anxiety disorders and help to establish SPACE as a useful treatment option for anxious children.
ACKNOWLEDGEMENTS:
The authors wish to thank Dr. James Jaccard Ph.D. of the New York University School of Social Work for statistical consultation for this study.
The authors wish to thank Dr. James Leckman, MD, Ph.D., of the Yale School of Medicine Child Study Center and Dr. Ruth Feldman, Ph.D., of Interdisciplinary Center of Herzilya who provided mentorship and consultation for this project.
FINANCIAL DISCLOSURES:
Dr. Lebowitz reports royalties form John Wiley and Sons and funding from the National Institute of Mental Health.
Dr. Marin has no financial disclosures to report.
Dr. Shimshoni has no financial disclosures to report.
Ms. Martino has no financial disclosures to report.
Dr. Silverman reports royalties from Oxford Press and funding from the National Institute of Mental Health.
FUNDING FOR THIS MANUSCRIPT:
Dr. Lebowitz’ work on this manuscript was funded by the National Institute of Mental Health (K23MH103555) and the Center for the Advancement of Translational Science (KL2TR000140).
Dr. Marin did not receive funding for this work.
Dr. Shimshoni did not receive funding for this work.
Ms. Martino did not receive funding for this work.
Dr. Silverman did not receive funding for this work.
Dr. Lebowitz’s work on this manuscript was funded by the National Institute of Mental Health (K23MH103555) and the Center for the Advancement of Translational Science (KL2TR000140).
Footnotes
Disclosure: Dr. Lebowitz has received royalties from John Wiley and Sons and funding from the National Institute of Mental Health. Dr. Silverman has received royalties from Oxford University Press and funding from the National Institute of Mental Health. Drs. Marin and Shimshoni and Ms. Martino report no biomedical financial interests or potential conflicts of interest.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Costello EJ, Egger HL, Copeland W, Erkanli A, Angold A. The developmental epidemiology of anxiety disorders: Phenomenology, prevalence, and comorbidity. Anxiety disorders in children and adolescents: Research, assessment and intervention. 2011:56–75. [Google Scholar]
- 2.Greenberg PE, Sisitsky T, Kessler RC, et al. The economic burden of anxiety disorders in the 1990s. J. Clin. Psychiatry 1999;60(7):427–435. [DOI] [PubMed] [Google Scholar]
- 3.Silverman WK, Pina AA, Viswesvaran C. Evidence-based psychosocial treatments for phobic and anxiety disorders in children and adolescents. J. Clin. Child Adolesc. Psychol 2008;37(1):105–130. [DOI] [PubMed] [Google Scholar]
- 4.Barrett PM, Dadds MR, Rapee RM. Family treatment of childhood anxiety: A controlled trial. J. Consult. Clin. Psychol 1996;64(2):333–342. [DOI] [PubMed] [Google Scholar]
- 5.Barmish AJ, Kendall PC. Should parents be co-clients in cognitive-behavioral therapy for anxious youth? J. Clin. Child Adolesc. Psychol 2005;34(3):569–581. [DOI] [PubMed] [Google Scholar]
- 6.Reynolds S, Wilson C, Austin J, Hooper L. Effects of psychotherapy for anxiety in children and adolescents: a meta-analytic review. Clin. Psychol. Rev 2012;32(4):251–262. [DOI] [PubMed] [Google Scholar]
- 7.Cobham VE, Dadds MR, Spence SH. The role of parental anxiety in the treatment of childhood anxiety. J. Consult. Clin. Psychol 1998;66(6):893–905. [DOI] [PubMed] [Google Scholar]
- 8.Bodden DH, Bogels SM, Nauta MH, et al. Child versus family cognitive-behavioral therapy in clinically anxious youth: an efficacy and partial effectiveness study. J. Am. Acad. Child Adolesc. Psychiatry 2008;47(12):1384–1394. [DOI] [PubMed] [Google Scholar]
- 9.Mendlowitz SL, Manassis K, Bradley S, Scapillato D, Miezitis S, Shaw BF. Cognitive-behavioral group treatments in childhood anxiety disorders: The role of parental involvement. J. Am. Acad. Child Adolesc. Psychiatry 1999;38(10):1223–1229. [DOI] [PubMed] [Google Scholar]
- 10.Nauta MH, Scholing A, Emmelkamp PMG, Minderaa RB. Cognitive-behavioral therapy for children with anxiety disorders in a clinical setting: No additional effect of a cognitive parent training. J. Am. Acad. Child Adolesc. Psychiatry 2003;42(11):1270–1278. [DOI] [PubMed] [Google Scholar]
- 11.Spence SH, Donovan C, Brechman-Toussaint M. The treatment of childhood social phobia: The effectiveness of a social skills training-based, cognitive-behavioural intervention, with and without parental involvement. J Child Psychol Psyc. 2000;.41(6):pp. [PubMed] [Google Scholar]
- 12.Wood JJ, Piacentini JC, Southam-Gerow M, Chu BC, Sigman M. Family Cognitive Behavioral Therapy for Child Anxiety Disorders. J. Am. Acad. Child Adolesc. Psychiatry 2006;45(3):314–321. [DOI] [PubMed] [Google Scholar]
- 13.Cartwright-Hatton S, McNally D, Field AP, et al. A new parenting-based group intervention for young anxious children: results of a randomized controlled trial. J. Am. Acad. Child Adolesc. Psychiatry 2011;50(3):242–251 e246. [DOI] [PubMed] [Google Scholar]
- 14.Thirlwall K, Cooper PJ, Karalus J, Voysey M, Willetts L, Creswell C. Treatment of child anxiety disorders via guided parent-delivered cognitive-behavioural therapy: randomised controlled trial. Br. J. Psychiatry 2013;203(6):436–444. [DOI] [PubMed] [Google Scholar]
- 15.Cartwright-Hatton S, McNally D, White C. A New Cognitive Behavioural Parenting Intervention for Families of Young Anxious Children: A Pilot Study. Behav. Cogn. Psychother 2005;33(2):243–247. [Google Scholar]
- 16.Thienemann M, Moore P, Tompkins K. A parent-only group intervention for children with anxiety disorders: pilot study. J. Am. Acad. Child Adolesc. Psychiatry 2006;45(1):37–46. [DOI] [PubMed] [Google Scholar]
- 17.Lebowitz ER, Omer H, Hermes H, Scahill L. Parent Training for Childhood Anxiety Disorders: The SPACE Program. Cognitive and Behavioral Practice. 2014;21(4):456–469. [Google Scholar]
- 18.Freeman J, Sapyta J, Garcia A, et al. Family-Based Treatment of Early Childhood OCD: The Pediatric OCD Treatment Study Junior (POTS Jr.) Randomized Controlled Trial. JAMA psychiatry. 2014;71(6):689. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Salloum A, Swaidan VR, Torres AC, Murphy TK, Storch EA. Parents’ perception of stepped care and standard care trauma-focused cognitive behavioral therapy for young children. Journal of Child and Family Studies. 2016;.25(1):pp. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Moriceau S, Sullivan RM. Maternal presence serves as a switch between learning fear and attraction in infancy. Nat. Neurosci 2006;9:1004–1006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Callaghan BL, Tottenham N. The Neuro-Environmental Loop of Plasticity: A Cross-Species Analysis of Parental Effects on Emotion Circuitry Development Following Typical and Adverse Caregiving. Neuropsychopharmacology. 2016;41:163–176. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Tottenham N, Hare TA, Millner A, Gilhooly T, Zevin J, Casey BJ. Elevated Amygdala Response to Faces Following Early Deprivation. Dev Sci. 2011;14:190–204. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Ollendick TH, Lewis KM, Cowart MJ, Davis T 3rd. Prediction of child performance on a parent-child behavioral approach test with animal phobic children. Behav. Modif 2012;36(4):509–524. [DOI] [PubMed] [Google Scholar]
- 24.Lebowitz ER, Woolston J, Bar-Haim Y, et al. Family accommodation in pediatric anxiety disorders. Depress. Anxiety 2013;30(1):47–54. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Lebowitz ER, Scharfstein L, Jones J. Child-Report of Family Accommodation in Pediatric Anxiety Disorders: Comparison and Integration with Mother-Report. Child Psychiatry Hum. Dev 2015;46(4):501–511. [DOI] [PubMed] [Google Scholar]
- 26.Storch EA, Salloum A, Johnco C, et al. Phenomenology and clinical correlates of family accommodation in pediatric anxiety disorders. J. Anxiety Disord 2015;35:75–81. [DOI] [PubMed] [Google Scholar]
- 27.Lebowitz ER, Panza KE, Bloch MH. Family accommodation in obsessive-compulsive and anxiety disorders: a five-year update. Expert Rev. Neurother 2016;16(1):45–53. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Kerns CE, Pincus DB, McLaughlin KA, Comer JS. Maternal emotion regulation during child distress, child anxiety accommodation, and links between maternal and child anxiety. J. Anxiety Disord 2017;50:52–59. [DOI] [PubMed] [Google Scholar]
- 29.Reuman L, Abramowitz JS. Predictors of accommodation among families affected by fear-based disorders. Child Psychiatry Hum. Dev. 2017. [DOI] [PubMed] [Google Scholar]
- 30.Norman KR, Silverman WK, Lebowitz ER. Family Accommodation of Child and Adolescent Anxiety: Mechanisms, Assessment, and Treatment. J. Child Adolesc. Psychiatr. Nurs 2015;28(3):131–140. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Settipani CA. The effect of child distress on maternal accommodation of anxiety: Relations with mother and child factors. Dissertation Abstracts International: Section B: The Sciences and Engineering. 2015;76(2-B(E)). [Google Scholar]
- 32.Kagan ER, Peterman JS, Carper MM, Kendall PC. Accommodation and Treatment of Anxious Youth. Depress. Anxiety. 2016. [DOI] [PubMed] [Google Scholar]
- 33.Peterman JS, Carper MM, Elkins RM, Comer JS, Pincus DB, Kendall PC. The effects of cognitive-behavioral therapy for youth anxiety on sleep problems. J. Anxiety Disord. 2016;37. [DOI] [PubMed] [Google Scholar]
- 34.Salloum A, Andel R, Lewin AB, Johnco C, McBride NM, Storch EA. Family Accommodation as a Predictor of Cognitive-Behavioral Treatment Outcome for Childhood Anxiety. Families in Society. 2018;99(1):45–55. [Google Scholar]
- 35.Manassis K, Lee TC, Bennett K, et al. Types of parental involvement in CBT with anxious youth: a preliminary meta-analysis. J. Consult. Clin. Psychol 2014;82(6):1163–1172. [DOI] [PubMed] [Google Scholar]
- 36.Lebowitz ER, Omer H. Treating Childhood and Adolescent Anxiety: A Guide for Caregivers. NJ: Wiley; 2013. [Google Scholar]
- 37.Lebowitz ER. Parent-based treatment for childhood and adolescent OCD. Journal of Obsessive-Compulsive and Related Disorders. 2013;2(4):425–431. [Google Scholar]
- 38.Silverman WK, Kurtines WM, Jaccard J, Pina AA. Directionality of change in youth anxiety treatment involving parents: an initial examination. J. Consult. Clin. Psychol 2009;77(3):474–485. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Silverman WK, Pina AA. Psychosocial treatments for phobic and anxiety disorders in youth. In: Roberts MC, ed: Springer Science + Business Media: New York; 2008:65–82. [Google Scholar]
- 40.Silverman WK, Albano AM. Anxiety Disorders Interview Schedule (ADIS-IV) Parent Interview Schedule. New York: Oxford University Press; 1996. [Google Scholar]
- 41.RUPP Anxiety Study Group. The Pediatric Anxiety Rating Scale (PARS): development and psychometric properties. J. Am. Acad. Child Adolesc. Psychiatry 2002;41(9):1061–1069. [DOI] [PubMed] [Google Scholar]
- 42.Caporino NE, Brodman DM, Kendall PC, et al. Defining treatment response and remission in child anxiety: signal detection analysis using the pediatric anxiety rating scale. J. Am. Acad. Child Adolesc. Psychiatry 2013;52(1):57–67. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Ginsburg GS, Kendall PC, Sakolsky D, et al. Remission after acute treatment in children and adolescents with anxiety disorders: findings from the CAMS. J. Consult. Clin. Psychol 2011;79(6):806–813. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Walkup JT, Albano AM, Piacentini J, et al. Cognitive behavioral therapy, sertraline, or a combination in childhood anxiety. N. Engl. J. Med 2008;359(26):2753–2766. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Birmaher B, Khetarpal S, Brent D, et al. The Screen for Child Anxiety Related Emotional Disorders (SCARED): scale construction and psychometric characteristics. J. Am. Acad. Child Adolesc. Psychiatry 1997;36(4):545–553. [DOI] [PubMed] [Google Scholar]
- 46.Birmaher B, Brent DA, Chiappetta L, Bridge J, Monga S, Baugher M. Psychometric properties of the Screen for Child Anxiety Related Emotional Disorders (SCARED): a replication study. J. Am. Acad. Child Adolesc. Psychiatry 1999;38(10):1230–1236. [DOI] [PubMed] [Google Scholar]
- 47.Abidin RR. Parenting Stress Index. 3rd. ed. Odessa, FL: Psychological Assessment Resources; 1995:1 professional manual (viii, 124 p. [Google Scholar]
- 48.Borkovec TD, Nau SD. Credibility of analogue therapy rationales. J. Behav. Ther. Exp. Psychiatry 1972;3(4):257–260. [Google Scholar]
- 49.Caporino NE, Sakolsky D, Brodman DM, et al. Establishing clinical cutoffs for response and remission on the Screen for Child Anxiety Related Emotional Disorders (SCARED). J. Am. Acad. Child Adolesc. Psychiatry 2017;.56(8):pp. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Taylor JH, Lebowitz ER, Jakubovski E, Coughlin CG, Silverman WK, Bloch MH. Monotherapy insufficient in severe anxiety? Predictors and moderators in the child/adolescent anxiety multimodal study. J. Clin. Child Adolesc. Psychol 2018;47(2). [DOI] [PMC free article] [PubMed] [Google Scholar]



