Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Aug 1.
Published in final edited form as: J Am Acad Child Adolesc Psychiatry. 2017 Jun 6;56(8):696–702. doi: 10.1016/j.jaac.2017.05.018

Establishing Clinical Cutoffs for Response and Remission on the Screen for Child Anxiety Related Emotional Disorders (SCARED)

Nicole E Caporino 1,*, Dara Sakolsky 2,*, Douglas M Brodman 3, Joseph F McGuire 4, John Piacentini 5, Tara S Peris 6, Golda S Ginsburg 7, John T Walkup 8, Satish Iyengar 9, Philip C Kendall 10, Boris Birmaher 11
PMCID: PMC5546231  NIHMSID: NIHMS882566  PMID: 28735699

Abstract

Objective

To determine optimal percent reduction and raw score cutoffs on the parent- and child-report Screen for Child Anxiety Related Emotional Disorders (SCARED) for predicting treatment response and remission among youth with anxiety disorders.

Method

Data were from youth (N = 438; ages 7–17 years) who completed treatment in the Child/Adolescent Anxiety Multimodal Study, a multi-site, randomized clinical trial that examined the relative efficacy of medication (sertraline), cognitive-behavioral therapy (Coping Cat), their combination, and pill placebo for the treatment of separation anxiety disorder, generalized anxiety disorder, and social phobia. The parent- and youth-report SCARED were administered at pre- and posttreatment. Quality receiver operating characteristic methods evaluated the performance of various SCARED percent reductions and absolute cutoff scores in predicting treatment response and remission, as defined by posttreatment ratings on the Clinical Global Impression scales and the Anxiety Disorders Interview Schedule.

Results

Reductions of 55% on the SCARED-Parent and 50% on the SCARED-Youth optimally predicted treatment response. Posttreatment absolute raw scores of 10 (SCARED-Parent) and 12 (SCARED-Youth) optimally predicted remission in the total sample, though separate SCARED-Parent cutoffs for children (12–13) and adolescents (9) showed greatest quality of efficiency. Each cutoff significantly predicted response and remission at 6-month follow-up.

Conclusion

Results serve as guidelines for operationalizing treatment response and remission on the SCARED, which may help clinicians systematically monitor treatment outcomes of youth with anxiety disorders in a cost- and time-efficient manner.

Clinical trial registration information

Child and Adolescent Anxiety Disorders (CAMS); http://clinicaltrials.gov/; NCT00052078.

Keywords: anxiety, treatment, measurement

INTRODUCTION

Anxiety disorders are the most common class of psychiatric disorders in youth, with point prevalence of 15% and twelve-month prevalence of 25% in adolescents.1 These disorders can lead to meaningful distress and functional impairment in social, academic, and family functioning.2,3 Treatment of youth with anxiety with cognitive-behavioral therapy (CBT), selective serotonin reuptake inhibitors (SSRIs), or their combination often leads to response (meaningful improvement in symptoms) or remission (absence or near absence of symptoms). In the Child/Adolescent Anxiety Multimodal Treatment Study (CAMS), 12-week response rates4 were 81% for combination treatment, 60% for CBT, 55% for sertraline, and 24% for placebo, while remission rates5 ranged from 46–68% for combination, 20–46% for CBT, 34–46% for sertraline, and 15–27% for placebo based on outcome measured.

Response and remission in childhood anxiety disorders have typically been measured using the Anxiety Disorders Interview Schedule for DSM-IV–Child and Parent Versions (ADIS-IV-C/P),6 the Pediatric Anxiety Rating Scale (PARS),7 and the Clinical Global Impressions Scales (CGI)8 in research studies. Although these measures are well regarded for defining outcomes, they are infrequently used in clinical practice because some are time-consuming to administer and all are completed as part of the clinical interview. Cutoffs have not yet been derived for parent- or child-report symptom measures, which may be completed prior to the clinical interview and help to inform the assessment process.

The Screen for Child Anxiety Related Emotional Disorders (SCARED)9 is a youth- and parent-report measure that was developed to screen youth for anxiety disorders. The SCARED has been evaluated in numerous settings across the world,10 including outpatient mental health clinics,11 primary care offices,12 schools,13 and community-based populations.14 While several versions of the SCARED have been developed, the most commonly used and best studied version consists of 41 questions. Responses are rated on a 0–2 scale and yield a total score as well as five subscale scores: generalized anxiety disorder (GAD), social phobia (SoP), separation anxiety disorder (SAD), somatic symptoms/panic disorder, and school phobia. A total score of 25 on the SCARED results in the optimal cutoff point maximizing both sensitivity and specificity when discriminating between anxiety and non-anxiety disorders in clinical populations.11

Studies in adult depression,15,16 for example, suggest that systematic use of patient-rated symptom monitoring in routine clinical care settings is not only feasible and acceptable, but can lead to meaningful change in how clinicians approach treatment. Youth- and parent-rated measures of anxiety symptoms could be similarly used to guide treatment decisions if the definition of response and remission were established for these instruments. To this end, we used CAMS data to identify the optimal parent- and youth-reported SCARED percent reduction and absolute cutoff scores for treatment response (CGI-Improvement rating of 1 or 2) and remission (CGI-Severity rating of 1 or 2; loss of all targeted anxiety diagnoses on the ADIS-IV-C/P). We also evaluated SCARED-defined response and remission as predictors of 6-month follow-up outcomes.

METHOD

Participants

Participants were a subset of youth (n = 438; 51% female) enrolled in CAMS (N = 488)4 who were administered all measures at pre- and posttreatment. They ranged in age from 7 to 17 years (M = 10.72, SD = 2.80) and had a principal diagnosis of GAD (48%), SoP (49%), and/or SAD (31%). Approximately 36% of youth in the current sample met criteria for all three, 43% met criteria for two, and 21% met criteria for one of these targeted anxiety disorders. Participants were each recruited with a parent (89% mothers) from one of six outpatient specialty clinics. Parent-report data were available for 99% (n = 434) of the 438 participants. Youth were randomly assigned to four treatment conditions: sertraline (SRT; n = 115), cognitive-behavioral therapy (CBT; n = 132), their combination (COMB n = 127), and pill placebo (n = 64), representing 86% of SRT, 95% of CBT, 91% of COMB, and 84% of pill placebo conditions in CAMS. Between posttreatment and 6-month follow-up assessments, acute-phase responders received maintenance care involving monthly visits, placebo non-responders were offered the active treatment of their choice, and all other non-responders were referred to community providers.17 Secondary analyses used 6-month follow-up data (n = 374). The current sample did not differ significantly from the CAMS sample in terms of demographic or diagnostic characteristics.4,18

Procedures

All recruitment and data collection procedures were approved by the institutional review board at each study site. Clinician-rated measures were completed by independent evaluators (IEs) blind to treatment condition. All IEs had prior experience administering the measures and were trained to a predetermined reliability criterion.4 Drift was prevented through twice-monthly cross-site conference calls.

Measures

Screen for Child Anxiety Related Emotional Disorders (SCARED).9,11

The SCARED is a 41-item questionnaire that assesses anxiety symptoms in youth. There are parallel parent- and youth-report versions. Internal consistency and retest reliability have been established and the SCARED has shown strong associations with other measures of childhood anxiety symptoms.19,20 The original versions of the SCARED ask about anxiety symptoms over the past 3 months; however, the version used in CAMS inquired about anxiety symptoms within the past 2 weeks, since this measure was administered at multiple time points.

Clinical Global Impressions (CGI).8

The CGI scales are commonly used in clinical trials and provide global ratings of the severity of psychopathology (CGI-Severity) and overall improvement (CGI-Improvement). The CGI-Severity scale ranges from 1 (no illness) to 7 (extremely severe) while the CGI-Improvement scale ranges from 1 (very much improved) to 7 (very much worse). Consistent with Walkup et al.,4 a CGI-Improvement score of 1 (very much improved) or 2 (much improved) was used to designate treatment response. Consistent with Ginsburg et al.,5 a CGI-Severity score of 1 (not at all ill) or 2 (borderline ill) was used to designate remission.

Anxiety Disorders Interview Schedule for DSM-IV—Child and Parent Versions (ADIS-IV-C/P).6

The ADIS-IV-C/P is a clinician-administered semi-structured diagnostic interview that assesses major DSM-IV anxiety disorders and associated psychopathology (e.g., mood disorders) in school-aged children and adolescents. Composite diagnoses are determined based on parent and youth interviews, and require the presence of core symptoms and a clinical severity rating ≥ 4 (on a 0–8 scale). The ADIS-IV-C/P has excellent psychometric properties.21,22 Consistent with Ginsburg et al.,5 the loss of all targeted anxiety diagnoses (i.e., SAD, GAD, and SP) was used as an alternative, albeit highly rigorous, definition of remission.

Data Analytic Plan

As in previous studies,23–25 receiver operating characteristic (ROC) methods26–28 were used to assess the performance of various SCARED percent reduction and absolute cutoff scores at measuring response (i.e., CGI-Improvement of 1 or 2) and remission (i.e., CGI-Severity of 1 or 2; loss of all targeted diagnoses on ADIS-IV-C/P) according to “gold standard” criteria used in CAMS. SCARED scores were divided into cutoffs based on percent reduction (in 5% intervals) and raw scores. For each SCARED (i.e., test) cutoff, analyses were conducted to examine the sensitivity (probability that youth meeting the gold standard criterion will exceed the test cutoff), specificity (probability that youth not meeting the gold standard criterion will not exceed the test cutoff), positive predictive value (probability that youth exceeding the test cutoff will meet the gold standard criterion), negative predictive value (probability that youth not exceeding the test cutoff will not meet the gold standard criterion), and efficiency (probability that the test and the gold standard agree). Youden’s J (J = sensitivity + specificity − 1)29 is reported as a measure of optimal tradeoff between sensitivity and specificity at a single cutoff. Youden’s J ranges from −1 (no discrimination) to +1 (perfect discrimination).

Quality Receiver Operating Characteristic (QROC) method30,31 were also employed. QROC methods were developed in response to concerns about the application of traditional ROC analyses to behavioral research given error in measurement of the criterion variable (“gold standard”). Relying on the efficiency statistic alone would be problematic in that it does not adjust for the rate of positive identification (e.g., percentage of participants in the sample who were non-responders) and the level of test (i.e., the percentage of participants who had SCARED percent reductions at or above the specified cutoff). QROC methods involve using weighted kappa statistics32 to recalibrate or assess the quality of various ROC statistics.30,31 This study utilized the κ(0.0), κ(0.5), and κ(1.0) statistics, which measure the quality of specificity, efficiency, and sensitivity, respectively. For each of these statistics, a value of 0.00 indicates that the assessment of responder/remitter status is no different from chance, while a value of 1.00 indicates perfect assessment of responder/remitter status. For the current study, priority was given to maximizing efficiency or accuracy of classification. Thus, optimal cutoffs were identified using κ(0.5) to minimize false positives and false negatives simultaneously.

Using the total sample (N = 438), analyses were conducted separately to compare the equality of SCARED-Parent and SCARED-Youth reported κ(0.5); as recommended for dependent samples,31 κ-overall confidence intervals and chi-square equality tests were utilized. Analyses were repeated using the child (7–12 years old; n = 325) and adolescent (13–17 years old; n = 113) independent subsamples, and z-scores were calculated to compare κ(0.5). All standard errors (SEs) of κ(0.5) were calculated using conventional methodology.33

Missing Data

Data were analyzed for CAMS participants who were administered both pre- and posttreatment measures. For these analyses, there were no data missing for the CGI scales, the ADIS-IV-C/P, or the SCARED. Secondary analyses used 6-month follow-up data for 374 individuals; missing data were handled using multiple imputation, with pooled estimates reported.

RESULTS

Descriptive Statistics

For participants who completed a week-12 assessment, the average pretreatment SCARED-Parent score was 32.05 (SD = 12.35), and the average posttreatment SCARED-Parent score was 13.46 (SD = 11.58). A dependent samples t-test for SCARED-Parent score reduction was significant (t[433] = 27.84, p < .001, Cohen’s d = 1.34). The average percent reduction in SCARED-Parent scores was 58%.

The average pretreatment SCARED-Youth score was 23.52 (SD = 15.14), and the average posttreatment SCARED-Youth score was 10.99 (SD = 11.68). A dependent samples t-test for SCARED-Youth score reduction was significant (t[437] = 18.03, p < .001, Cohen’s d = 0.86). The average percent reduction in SCARED-Youth scores was 43%.

The average pretreatment CGI-Severity rating for the youth sample (n = 438) was 5.03 (SD = 0.73), and the average posttreatment rating was 2.94 (SD = 1.47). A dependent samples t-test for overall CGI-Severity reduction was significant (t[437] = 30.84, p < .001, Cohen’s d = 1.81). The average percent reduction in CGI-Severity scores was 58% (SD = 28%). At posttreatment, approximately 65% of the total sample met criteria for treatment response based on CGI-Improvement ratings, 46% met criteria for remission based on CGI-Severity ratings, and 53% met criteria for remission based on ADIS-IV-C/P diagnostic status. Descriptive statistics were nearly identical for the youth of parents with complete data (n = 434).

Neither baseline SCARED scores nor baseline CGI-Severity scores were correlated with SCARED percent reductions (suggesting that the present findings apply across baseline severity levels). SCARED percent reductions were significantly associated with treatment response based on dichotomized CGI-Improvement ratings (rpb = .57, p < .001 for SCARED-Parent and rpb = .25, p < .001 for SCARED-Youth) and with remission based on dichotomized CGI-Severity ratings (rpb = .51, p < .001 for SCARED-Parent and rpb = .28, p < .001 for SCARED-Youth). The correlation between SCARED percent reductions and ADIS-IV-C/P diagnostic status was also significant (rpb = .56, p < .001 for SCARED-Parent and rpb = .24, p < .001 for SCARED-Youth).

Clinical Cutoffs for Treatment Response With SCARED Percent Reduction

Table 1 presents the series of SCARED-Parent percent reduction cutoffs used to describe response based on dichotomized CGI-Improvement ratings, with ROC and QROC statistics for each cutoff. Maximum efficiency (.83) for response was found at a cutoff of 55%. Quality of efficiency at this cutoff (κ[0.5] = .63) is considered large.34 When age groups were compared, the optimal SCARED-Parent percent reduction cutoff was similar for children (=60%) and adolescents (=55%); κ(0.5) was greater for parents of adolescents (z = 3.17, p < .01).

Table 1.

Treatment Response Using Clinical Global Impression-Improvement (CGI-I) at Varying Screen for Child Anxiety-Related Emotional Disorders (SCARED-Parent) Percent Reduction Cutoffs (N = 438)

SCARED-
Parent
Reduction (%)		 Sensitivitya Specificityb Positive
Predictive Valuec 		Negative
Predictive Valued 		Efficiencye κ(.5)f κ(0)g κ(1)h Ji
≥5 0.98 0.20 0.69 0.86 0.71 0.22 0.79 0.13 0.18
≥10 0.98 0.24 0.70 0.88 0.72 0.27 0.82 0.16 0.22
≥15 0.97 0.29 0.72 0.83 0.73 0.31 0.74 0.19 0.26
≥20 0.96 0.39 0.74 0.84 0.76 0.40 0.76 0.27 0.35
≥25 0.95 0.44 0.76 0.83 0.77 0.44 0.74 0.32 0.39
≥30 0.92 0.53 0.78 0.79 0.78 0.49 0.67 0.38 0.45
≥35 0.92 0.59 0.81 0.80 0.80 0.54 0.69 0.45 0.51
≥40 0.90 0.65 0.82 0.78 0.81 0.57 0.66 0.50 0.55
≥45 0.89 0.69 0.84 0.78 0.82 0.60 0.66 0.55 0.59
≥50 0.88 0.73 0.85 0.76 0.82 0.61 0.63 0.59 0.60
≥55 0.84 0.81 0.89 0.73 0.83 0.63 0.58 0.69 0.65
≥60 0.77 0.88 0.92 0.68 0.81 0.61 0.50 0.78 0.65
≥65 0.69 0.92 0.94 0.62 0.77 0.55 0.41 0.84 0.62
≥70 0.59 0.94 0.95 0.56 0.72 0.46 0.32 0.85 0.54
≥75 0.52 0.95 0.95 0.52 0.67 0.39 0.25 0.85 0.46
≥80 0.46 0.96 0.96 0.49 0.64 0.35 0.22 0.87 0.42
≥85 0.36 0.97 0.95 0.45 0.57 0.26 0.15 0.87 0.33
≥90 0.26 0.99 0.97 0.42 0.52 0.19 0.11 0.93 0.25
≥95 0.16 0.99 0.98 0.39 0.45 0.11 0.06 0.94 0.15
Open in a new tab

Note: Statistics for the cutoff with the highest quality of efficiency are boldfaced.

a

Probability of exceeding the SCARED-Parent cutoff among youths meeting criterion for treatment response (CGI-Improvement of 1 or 2).

b

Probability of not exceeding the SCARED-Parent cutoff among those youths not meeting criterion for treatment response (CGI-Improvement of 1 or 2).

c

Probability of meeting criterion for treatment response (CGI-Improvement of 1 or 2) for those youths who exceed the SCARED-Parent cutoff.

d

Probability of not meeting criterion for treatment response (CGI-Improvement of 1 or 2) for those youths who do not exceed the SCARED-Parent cutoff.

e

Probability that the SCARED-Parent cutoff and the dichotomous CGI-Improvement rating (CGI-Improvement of 1–2 vs. CGI-Improvement of 3–7) agree.

f

Weighted kappa statistic measuring quality of efficiency.

g

Weighted kappa statistic measuring quality of specificity.

h

Weighted kappa statistic measuring quality of sensitivity.

i

Youden’s J measuring optimal tradeoff of sensitivity and specificity at any single cutoff point.

When analyses for the total sample were repeated using the SCARED-Youth, maximum efficiency (.70) for response was found at a cutoff of 50%. The quality of efficiency (κ[0.5] = .37) was medium. The SCARED-Parent showed greater quality of efficiency for predicting treatment response than the SCARED-Youth (κ-overall = .52, 95% CI [.47, .58]; χ2 = 19.04, p < .001). Analyses for the child and adolescent subsamples yielded a higher SCARED-Youth percent reduction cutoff for adolescents (=65%) than for children (=50%); κ(0.5) did not differ significantly (z = .64, p = .52).

Clinical Cutoff for Remission With SCARED Percent Reduction

SCARED-Parent percent reduction cutoffs were used to determine remission based on dichotomized CGI-Severity ratings. Maximum efficiency (efficiency = .78, κ[0.5] = .56) was found at a cutoff of 60%. Results were consistent when analyses were repeated with clinical remission defined as a loss of all targeted diagnoses (i.e., GAD, SoP, and/or SAD) on the ADIS-IV-C/P; maximum efficiency (efficiency = .82, κ[0.5] = .63) was again found at a cutoff of 60% reduction. There were no significant differences between the child and adolescent subsamples in optimal SCARED-Parent percent reduction cutoffs or κ(0.5) for CGI-defined remission (z = 1.95, p = .05) or ADIS-defined remission (z = 1.24, p = .22).

SCARED-Youth percent reduction cutoffs were also used to assess CGI-defined remission. Maximum efficiency (.69–.70) and quality of efficiency (κ[0.5] = .39) was highest at 55%, 60%, and 65% reduction cutoffs. The SCARED-Parent showed greater quality of efficiency for predicting CGI-defined remission than the SCARED-Youth (κ-overall = .48, 95% CI [.43, .54]; χ2 = 8.42, p < .01). When remission was defined as loss of all targeted diagnoses (i.e., GAD, SoP, and/or SAD) on the ADIS-IV-C/P), the 55% and 60% reduction SCARED-Youth cutoffs were identified as optimal (maximum efficiency = .69), with moderate quality of efficiency (κ[0.5] = .38). The SCARED-Parent showed greater quality of efficiency for predicting ADIS-defined remission than the SCARED-Youth (κ-overall = .53, 95% CI [.47, .58]; χ2 = 18.59, p < .001). The optimal SCARED-Youth cutoffs were higher for adolescents (70% for CGI and 65% for ADIS) than for children (55% for CGI and 50% for ADIS); there were no significant differences between the child and adolescent subsamples in κ(0.5) for CGI-defined remission (z = .72, p = .47) or ADIS-defined remission (z = .75, p = .45).

Clinical Cutoffs for Remission With SCARED Absolute Scores

When remission was determined using dichotomized CGI-Severity ratings, maximal efficiency (efficiency = .77, κ[0.5] = .55) was found for a SCARED-Parent raw score cutoff of 10, with moderate to strong sensitivity (.80), specificity (.75), positive predictive value (.73), and negative predictive value (.82). When analyses were repeated with clinical remission defined as loss of all targeted diagnoses (i.e., GAD, SoP, and/or SAD) on the ADIS-IV-C/P, maximal efficiency was again found for a raw score cutoff of 10. When analyses were repeated for child and adolescent subsamples, the optimal SCARED-Parent raw score cutoffs were higher for children (=12 for CGI-defined and 13 for ADIS-defined remission) than for adolescents (=9 for both CGI- and ADIS-defined remission). ROC statistics for SCARED-Parent absolute cutoffs used to describe loss of ADIS-IV-C/P diagnoses in children (7–12 years old) are displayed in Table 2. The κ(0.5) for parents of adolescents was greater than κ(0.5) for parents of children, both for CGI-defined remission (z = 2.44, p = .01) and for ADIS-defined remission (z = 2.58, p < .01).

Table 2.

Remission Using Anxiety Disorders Interview Schedule for DSM-IV Child and Parent Versions (ADIS-IV-C/P) at Varying Screen for Child and Related Emotional Disorders-Parent Version (SCARED-Parent) Absolute Cutoff Scores in Children (n = 321)

SCARED-
Parent Raw
Score		 Sensitivitya Specificityb Positive
Predictive Valuec 		Negative
Predictive Valued 		Efficiencye κ(.5)f κ(0)g κ(1)h Ji
5 0.51 0.93 0.90 0.58 0.69 0.40 0.27 0.77 0.43
6 0.57 0.93 0.91 0.61 0.72 0.46 0.33 0.79 0.49
7 0.61 0.91 0.90 0.63 0.74 0.49 0.36 0.77 0.52
8 0.65 0.88 0.88 0.65 0.75 0.51 0.40 0.72 0.54
9 0.70 0.83 0.85 0.67 0.75 0.51 0.43 0.64 0.53
10 0.76 0.80 0.84 0.71 0.78 0.55 0.49 0.62 0.56
11 0.77 0.78 0.83 0.72 0.78 0.55 0.51 0.59 0.55
12 0.82 0.74 0.81 0.75 0.78 0.55 0.56 0.55 0.55
13 0.86 0.69 0.79 0.79 0.79 0.56 0.63 0.51 0.55
14 0.88 0.64 0.76 0.80 0.78 0.53 0.65 0.45 0.52
15 0.90 0.61 0.75 0.81 0.77 0.52 0.68 0.42 0.50
16 0.93 0.58 0.75 0.86 0.78 0.53 0.75 0.41 0.51
17 0.93 0.53 0.73 0.85 0.76 0.49 0.74 0.36 0.46
18 0.95 0.50 0.72 0.87 0.75 0.47 0.78 0.33 0.44
19 0.96 0.46 0.70 0.89 0.74 0.44 0.80 0.31 0.42
20 0.96 0.45 0.70 0.90 0.74 0.44 0.82 0.30 0.41
Open in a new tab

Note: Statistics for the cutoff with the highest quality of efficiency are boldfaced. Children = 7–12 years old.

a

Probability of exceeding the SCARED-Parent cutoff among those children meeting ADIS-IV-C/P criterion for clinical remission.

b

Probability of not exceeding the SCARED-Parent cutoff among those children not meeting ADIS-IV-C/P criterion for clinical remission.

c

Probability of meeting ADIS-IV-C/P criterion for clinical remission for those children who exceed the SCARED-Parent cutoff.

d

Probability of not meeting ADIS-IV-C/P criterion for clinical remission for those children who do not exceed the SCARED-Parent cutoff.

e

Probability that the SCARED-Parent cutoff and the ADIS-IV-C/P agree.

f

Weighted kappa statistic measuring quality of efficiency.

g

Weighted kappa statistic measuring quality of specificity.

h

Weighted kappa statistic measuring quality of sensitivity.

i

Youden’s J measuring optimal tradeoff of sensitivity and specificity at any single cutoff point.

For the SCARED-Youth, maximal efficiency was found at a raw score cutoff of 12, both when remission was defined using the CGI-Severity (efficiency = .68, κ[0.5] = .38) and when remission was defined as loss of all targeted ADIS-IV-C/P diagnoses (efficiency = .70, κ[0.5] = .39). In both cases, sensitivity was higher than specificity, and negative predictive value was higher than positive predictive value. The SCARED-Parent showed greater quality of efficiency for predicting remission than the SCARED-Youth (CGI-defined remission: κ-overall = .47, 95% CI [.41, .52]; χ2 = 8.89, p < .01; ADIS-defined remission: κ-overall = .51, 95% CI [.45, .56]; χ2 = 13.2, p < .001). For ADIS-defined remission, there were no significant differences between the child and adolescent subsamples in the optimal SCARED-Youth raw score cutoff (= 12) or κ(0.5) (z = .77, p = .44). For CGI-defined remission, the optimal SCARED-Youth raw cutoff score was 5 for children and 7 for adolescents; κ(0.5) did not differ significantly (z = .91, p = .36)

Follow-Up Outcomes

Posttreatment response based on the SCARED-Parent 55% reduction cutoff significantly predicted CGI-defined responder status at 6-month follow-up (OR = 5.57, p < .001). The same was true for the SCARED-Youth 50% reduction cutoff (OR = 3.04, p < .001). Posttreatment remission based on the SCARED-Parent 60% reduction cutoff significantly predicted CGI-defined remission 6 months out (OR = 4.84, p < .001), as did posttreatment remission based on the SCARED-Youth 60% reduction cutoff (OR = 3.11, p < .001). CGI-defined remission at 6-month follow-up was also significantly predicted by posttreatment remission based on the SCARED-Parent absolute cutoff of 10 (OR = 6.07, p < .001) and the SCARED-Youth absolute cutoff of 12 (OR = 3.93, p < .001).

DISCUSSION

Signal detection analysis of SCARED-Parent revealed that a 55% reduction in total score from pre- to posttreatment best predicted treatment response and a 60% reduction in SCARED-Parent scores best predicted remission (defined using either the ADIS or the CGI) for the total sample. Signal detection analysis of the SCARED-Youth showed that 50% reduction in total score best predicted treatment response, and a 60% reduction best predicted remission for the total sample. Posttreatment absolute cutoffs of 10 for total SCARED-Parent score and 12 for total SCARED-Youth score were most strongly associated with measures of remission in the total sample; however, SCARED-Parent quality of efficiency is maximized by using separate cutoffs for children (total score of 12–13) and adolescents (total score of 9). Optimal percent reductions from pre- to posttreatment and absolute cutoffs at posttreatment predicted response and remission at 6-month follow-up.

Optimal percent reductions and absolute cutoffs for the SCARED-Parent consistently demonstrated significantly greater quality of efficiency compared to the SCARED-Youth. Because “gold standard” measures of response and remission are based upon a composite of parent and youth report by independent evaluators (IEs), it appears that IEs generally placed a greater emphasis on parent report than youth report in their final evaluation. Whether parent report is more accurate than youth report or discrepancies reflect contextual variation in anxiety symptoms is unclear; personalized assessment with data from multiple informants is an important area for future study.35

When analyses were repeated for child (ages 7–12 years) and adolescent (ages 13–17 years) subsamples, a few differences emerged. The optimal SCARED-Parent absolute cutoffs for predicting CGI- and ADIS-defined remission showed significantly greater quality of efficiency for adolescents than for children, suggesting that IE ratings and parent report were more closely aligned for adolescents and that there is some benefit to using the distinct cutoffs: 12–13 for children and 9 for adolescents. Although the optimal SCARED-Parent percent reduction cutoff for predicting treatment response also showed significantly greater quality of efficiency in the adolescent subsample than in the child subsample, the 55% cutoff calculated using the total sample performed well in both age groups. Across all SCARED-Youth analyses, quality of efficiency was not related to age group and was moderate for the total sample.

For the SCARED-Parent in particular, the percent reduction cutoffs that best predicted treatment response and remission were similar and may reflect the 3-point scale used to rate each SCARED item. Although similar cutoffs were expected to some degree because a subset of treatment responders are remitters by definition, the clinician-rated PARS percent reduction cutoffs23 may be preferred for differentiating response and remission. For the SCARED, the absolute cutoff is most closely linked to the definition of remission as the near absence of symptoms (regardless of pretreatment symptom severity) and should be used in individual cases, with the idea that residual symptoms may predict relapse. Indeed, remission defined using SCARED absolute cutoffs more strongly predicted remitter status at 6-month follow-up than did remission defined using SCARED percent reduction cutoffs. Nevertheless, percent reduction cutoffs for determining remission may facilitate cross-study comparisons and provide another way for practitioners to compare their performance to that of CAMS therapists (e.g., calling attention to possible differences between their practice and the CAMS with regard to severity of cases).

Strengths of this study are that it sampled youth who received treatment that was evaluated and found efficacious,4 assessments were conducted by highly trained and reliable IEs, and multiple operational definitions of remission were used. However, potential limitations merit consideration. First, CAMS inclusion and exclusion criteria may limit the generalizability of these findings to some youth (e.g., with co-principal depression). However, comparisons between youth presenting for anxiety treatment at university-based clinics and in community settings have shown minimal differences on measures of both anxiety and mood symptoms.36 Also, the CAMS sample was predominantly non-Hispanic White; future studies should replicate findings with other racial/ethnic groups.

This study has implications for routine clinical practice. Since the SCARED can be used free of charge and with minimal time burden on clinicians and families, SCARED cutoffs can assist clinicians in assessing the progress of youth with anxiety in treatment against the outcomes reported in CAMS and similar clinical trials. Specifically, the use of SCARED cutoffs can inform decisions to pursue changes in treatment. For example, if a youth receives an adequate trial of CBT without exhibiting a 55% reduction in the SCARED-Parent score, his/her clinician may recommend medication to maximize the likelihood of a positive response to treatment. Given that SCARED cutoffs agreed with “gold standard” outcome measures in no more than 88% of cases, there is some benefit to administering a diagnostic interview such as the ADIS or a clinician-rated measure such as PARS. At a minimum, the SCARED cutoffs should be used in combination with a clinical interview. For example, clinicians may ask parents and youth to complete the SCARED prior to administering the PARS or conducting an unstructured clinical interview. Responses on SCARED can be used to guide the clinician’s inquiry of anxiety symptoms and make the clinical interview process more efficient. As such, SCARED cutoffs have the potential to enhance treatment planning by alerting clinicians to treatment failure that might otherwise go undetected.37,38

Acknowledgments

There are no acknowledgements.

Funding

This research was supported by grants (U01 MH064089 to Dr. Walkup; U01 MH64092 to Dr. Albano; U01 MH64003 to Dr. Birmaher; U01 MH63747 to Dr. Kendall; U01 MH64107 to Dr. March; U01 MH64088 to Dr. Piacentini; and U01 MH064003 to Dr. Compton) from the National Institute of Mental Health (NIMH). Sertraline and matching placebo were supplied free of charge by Pfizer.

Dr. Caporino has received an honorarium from NICHD.

Dr. Sakolsky has received research support from NIMH. She serves as an editorial board member of Child & Adolescent Psychopharmacology News and specialty consultant for the Prescriber’s Letter. She has received consulting fee from LEK consulting in 2015.

Dr. McGuire has received support from the Tourette Association of America, American Academy of Neurology, the Brain Research Foundation, and NIMH. He has received royalties from Elsevier.

Dr. Piacentini has received grant or research support from NIMH, Pfizer Pharmaceuticals through the Duke University Clinical Research Institute CAPTN Network, Psyadon Pharmaceuticals, and the Tourette Association of America. He has received financial support from the Petit Family Foundation and the Tourette Syndrome Association Center of Excellence Gift Fund. He is a co-author of the Child OCD Impact Scale-Revised (COIS-R), the Child Anxiety Impact Scale (CAIS), the Parent Tic Questionnaire (PTQ), and the Premonitory Urge for Tics Scale (PUTS) assessment tools, all of which are in the public domain therefore no royalties are received. He has received royalties from Guilford Press and Oxford University Press. He has served on the speakers’ bureau of the Tourette Association of America, the International Obsessive Compulsive Disorder Foundation, and the Trichotillomania Learning Center.

Dr. Peris has received research funding from NIMH and the Society of Clinical Child and Adolescent Psychology.

Dr. Ginsburg has received support from NIMH.

Dr. Walkup has received grant or research support from the Hartwell Foundation and the Tourette Syndrome Association. He is an unpaid advisor to the Anxiety and Depression Association of America, the Trichotillomania Learning Center, and the American Foundation for Suicide Prevention. He has received royalties for books from Guilford Press and Oxford University Press and educational materials from WoltersKluwer. He has served as a paid speaker for the Tourette Syndrome - Center for Disease Control and Prevention outreach educational programs, the American Academy of Child and Adolescent Psychiatry, and the American Psychiatric Association. He and his collaborators received free drug/placebo from Pfizer (2007) to support the NIMH funded Child Adolescent Anxiety Multimodal study.

Dr. Kendall has received research support from NIMH. He has received honoraria from professional societies for speaking at conventions. He has received royalties from Guildford Press, Ericsson, Workbook Publishing (his spouse’s employment), and Oxford University Press.

Dr. Birmaher has received research support from NIMH. He has or will receive royalties from Random House, Inc., Lippincott Willaims & Wilkins, and UpToDate.

Drs. Caporino and Brodman served as the statistical experts for this research; results were reviewed by Dr. Iyengar.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Disclosures

Drs. Brodman and Iyengar report no biomedical financial interests or potential conflicts of interest.

Contributor Information

Dr. Nicole E. Caporino, American University, Washington, DC.

Dr. Dara Sakolsky, University of Pittsburgh School of Medicine, Pittsburgh, PA.

Dr. Douglas M. Brodman, New York University Langone Medical Center, New York.

Dr. Joseph F. McGuire, University of California Los Angeles School of Medicine, Los Angeles.

Dr. John Piacentini, University of California Los Angeles School of Medicine, Los Angeles.

Dr. Tara S. Peris, University of California Los Angeles School of Medicine, Los Angeles.

Dr. Golda S. Ginsburg, University of Connecticut Health Center, Farmington, CT.

Dr. John T. Walkup, New York-Presbyterian Weill Cornell Medical Center, New York.

Dr. Satish Iyengar, University of Pittsburgh.

Dr. Philip C. Kendall, Temple University, Philadelphia.

Dr. Boris Birmaher, University of Pittsburgh School of Medicine, Pittsburgh, PA.

References

  • 1.Kessler RC, Avenevoli S, Costello EJ, et al. Prevalence, persistence, and sociodemographic correlates of DSM-IV disorders in the National Comorbidity Survey Replication Adolescent Supplement. Arch Gen Psychiatry. 2012;69:372. doi: 10.1001/archgenpsychiatry.2011.160. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Langley AK, Falk A, Peris T, et al. The Child Anxiety Impact Scale: examining parent- and child-reported impairment in child anxiety disorders. J Clin Child Adolesc Psychol. 2014;43:579–91. doi: 10.1080/15374416.2013.817311. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Swan A, Kendall PC. Fear and missing out: youth anxiety and functional outcomes. Clin Psychol Sci Pract. 2016;23:417–435. [Google Scholar]
  • 4.Walkup JT, Albano AM, Piacentini J, et al. Cognitive behavioral therapy, sertraline, or a combination in childhood anxiety. N Engl J Med. 2008;359:2753–2766. doi: 10.1056/NEJMoa0804633. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Ginsburg GS, Kendall PC, Sakolsky D, et al. Remission after acute treatment in children and adolescents with anxiety disorders: findings from the CAMS. J Consult Clin Psychol. 2011;79:806–813. doi: 10.1037/a0025933. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Silverman W, Albano A. The Anxiety Disorders Interview Schedule for Children for DSM-IV (Child and Parent versions) San Antonio, TX: Psychological Corporation; 1996. [Google Scholar]
  • 7.The Research Units on Pediatric Psychopharmacology Anxiety Study Group. The Pediatric Anxiety Rating Scale (PARS): development and psychometric properties. J Am Acad Child Adolesc Psychiatry. 2002;41:1061–1069. doi: 10.1097/00004583-200209000-00006. [DOI] [PubMed] [Google Scholar]
  • 8.Guy W. ECDEU Assessment Manual for Psychopharmacology. Rockville, MD: National Institute of Mental Health; 1976. [Google Scholar]
  • 9.Birmaher B, Khetarpal S, Brent D, et al. The Screen for Child Anxiety Related Emotional Disorders (SCARED): scale construction and psychometric characteristics. J Am Acad Child Adolesc Psychiatry. 1997;36:545–553. doi: 10.1097/00004583-199704000-00018. [DOI] [PubMed] [Google Scholar]
  • 10.Hale WW, Crocetti E, Raaijmakers QAW, Meeus WHJ. A meta-analysis of the cross-cultural psychometric properties of the Screen for Child Anxiety Related Emotional Disorders (SCARED) J Child Psychol Psychiatry Allied Discip. 2011;52:80–90. doi: 10.1111/j.1469-7610.2010.02285.x. [DOI] [PubMed] [Google Scholar]
  • 11.Birmaher B, Brent DA, Chiappetta L, Bridge J, Monga S, Baugher M. Psychometric properties of the Screen for Child Anxiety Related Emotional Disorders (SCARED): a replication study. J Am Acad Child Adolesc Psychiatry. 1999;38:1230–1236. doi: 10.1097/00004583-199910000-00011. [DOI] [PubMed] [Google Scholar]
  • 12.Wren FJ, Bridge JA, Birmaher B. Screening for childhood anxiety symptoms in primary care: integrating child and parent reports. J Am Acad Child Adolesc Psychiatry. 2004;43:1364–71. doi: 10.1097/01.chi.0000138350.60487.d3. [DOI] [PubMed] [Google Scholar]
  • 13.Hale WW, Raaijmakers Q, Muris P, Meeus W. Psychometric properties of the Screen for Child Anxiety Related Emotional Disorders (SCARED) in the general adolescent population. J Am Acad Child Adolesc Psychiatry. 2005;44:283–290. doi: 10.1097/00004583-200503000-00013. [DOI] [PubMed] [Google Scholar]
  • 14.Desousa DA, Salum GA, Isolan LR, Manfro GG. Sensitivity and specificity of the Screen for Child Anxiety Related Emotional Disorders (SCARED): a community-based study. Child Psychiatry Hum Dev. 2013;44:391–399. doi: 10.1007/s10578-012-0333-y. [DOI] [PubMed] [Google Scholar]
  • 15.Duffy FF, Chung H, Trivedi M, Rae DS, Regier DA, Katzelnick DJ. Systematic use of patient-rated depression severity monitoring: is it helpful and feasible in clinical psychiatry? Psychiatr Serv. 2008;59:1148–1154. doi: 10.1176/ps.2008.59.10.1148. [DOI] [PubMed] [Google Scholar]
  • 16.Trivedi MH, Rush AJ, Wisniewski SR, et al. Evaluation of outcomes with citalopram for depression using measurement-based care in STAR*D: implications for clinical practice. Am J Psychiatry. 2006;163:28–40. doi: 10.1176/appi.ajp.163.1.28. [DOI] [PubMed] [Google Scholar]
  • 17.Piacentini J, Bennett S, Compton SN, et al. 24- and 36-week outcomes for the Child/Adolescent Anxiety Multimodal Study (CAMS) J Am Acad Child Adolesc Psychiatry. 2014;53:297–310. doi: 10.1016/j.jaac.2013.11.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Kendall PC, Compton SN, Walkup JT, et al. Clinical characteristics of anxiety disordered youth. J Anxiety Disord. 2010;24(3):360–365. doi: 10.1016/j.janxdis.2010.01.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Muris P, Gadet B, Moulaert V, Merckelbach H. Correlations between two multidimensional anxiety scales for children. Percept Mot Skills. 1998;87(1):269–270. doi: 10.2466/pms.1998.87.1.269. [DOI] [PubMed] [Google Scholar]
  • 20.Muris P, Merckelbach H, Mayer B, et al. The Screen for Child Anxiety Related Emotional Disorders (SCARED) and traditional childhood anxiety measures. J Behav Ther Exp Psychiatry. 1998;29(4):327–339. doi: 10.1016/s0005-7916(98)00023-8. [DOI] [PubMed] [Google Scholar]
  • 21.Silverman WK, Saavedra LM, Pina AA. Test-retest reliability of anxiety symptoms and diagnoses with the anxiety disorders interview schedule for DSM-IV: child and parent versions. J Am Acad Child Adolesc Psychiatry. 2001;40(8):937–944. doi: 10.1097/00004583-200108000-00016. [DOI] [PubMed] [Google Scholar]
  • 22.Wood JJ, Piacentini JC, Bergman RL, McCracken J, Barrios V. Concurrent validity of the anxiety disorders section of the Anxiety Disorders Interview Schedule for DSM-IV: Child and Parent versions. J Clin Child Adolesc Psychol. 2002;31(3):335–342. doi: 10.1207/S15374424JCCP3103_05. [DOI] [PubMed] [Google Scholar]
  • 23.Caporino NE, Brodman DM, Kendall PC, et al. Defining treatment response and remission in child anxiety: signal detection analysis using the Pediatric Anxiety Rating Scale. J Am Acad Child Adolesc Psychiatry. 2013;52(1):57–67. doi: 10.1016/j.jaac.2012.10.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Storch EA, Lewin AB, De Nadai AS, Murphy TK. Defining treatment response and remission in obsessive-compulsive disorder: a signal detection analysis of the Children’s Yale-Brown Obsessive Compulsive Scale. J Am Acad Child Adolesc Psychiatry. 2010;49:708–17. doi: 10.1016/j.jaac.2010.04.005. [DOI] [PubMed] [Google Scholar]
  • 25.Tolin DF, Abramowitz JS, Diefenbach GJ. Defining response in clinical trials for obsessive-compulsive disorder: a signal detection analysis of the Yale-Brown Obsessive Compulsive Scale. J Clin Psychiatry. 2005;66(12):1549–1557. doi: 10.4088/jcp.v66n1209. [DOI] [PubMed] [Google Scholar]
  • 26.Swets JA, Dawes RM, Monahan J. Psychological science can improve diagnostic decisions. Psychol Sci Public Interes. 2000;1(1):1–26. doi: 10.1111/1529-1006.001. [DOI] [PubMed] [Google Scholar]
  • 27.Swets JA. Measuring the accuracy of diagnostic systems. Science. 1988;240:1285–1293. doi: 10.1126/science.3287615. [DOI] [PubMed] [Google Scholar]
  • 28.Swets JA, Pickett RM. Evaluation of Diagnostic Systems : Methods from Signal Detection Theory. New York: Academic Press; 1982. [Google Scholar]
  • 29.Youden WJ. Index for rating diagnostic tests. Cancer. 1950;3(1):32–35. doi: 10.1002/1097-0142(1950)3:1<32::aid-cncr2820030106>3.0.co;2-3. [DOI] [PubMed] [Google Scholar]
  • 30.Kraemer HC. Evaluating Medical Tests: Objective and Quantitative Guidelines. Thousand Oaks, CA: Sage Publications; 1992. [Google Scholar]
  • 31.Kraemer HC, Periyakoil VS, Noda A. Kappa coefficients in medical research. Stat Med. 2002;21(14):2109–2129. doi: 10.1002/sim.1180. [DOI] [PubMed] [Google Scholar]
  • 32.Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull. 1968;70(4):213–220. doi: 10.1037/h0026256. [DOI] [PubMed] [Google Scholar]
  • 33.Fleiss JL, Levin B, Paik MC. Statistical Methods for Rates and Proportions. Hoboken, NJ: J. Wiley; 2003. [Google Scholar]
  • 34.Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2. Hillsdale, NJ: L. Erlbaum Associates; 1988. [Google Scholar]
  • 35.De Los Reyes A, Augenstein TM, Wang M, et al. The validity of the multi-informant approach to assessing child and adolescent mental health. Psychol Bull. 2015;141:858–900. doi: 10.1037/a0038498. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Southam-Gerow MA, Weisz JR, Kendall PC. Youth with anxiety disorders in research and service clinics: examining client differences and similarities. J Clin Child Adolesc Psychol. 2003;32:375–385. doi: 10.1207/S15374424JCCP3203_06. [DOI] [PubMed] [Google Scholar]
  • 37.Yalom ID, Lieberman MA. A study of encounter group casualties. Arch Gen Psychiatry. 1971;25:16. doi: 10.1001/archpsyc.1971.01750130018002. [DOI] [PubMed] [Google Scholar]
  • 38.Hannan C, Lambert MJ, Harmon C, et al. A lab test and algorithms for identifying clients at risk for treatment failure. J Clin Psychol. 2005;61:155–163. doi: 10.1002/jclp.20108. [DOI] [PubMed] [Google Scholar]

RESOURCES