Abstract
Objective: The purpose of this study was to determine guidelines for delineating treatment response and symptom remission for children with anxiety disorder based on the five item and Pediatric Anxiety Rating Scale (PARS5), and replicate guidelines using the six item PARS (PARS6).
Methods: Participants were 73 children 7–13 years of age with a primary anxiety disorder who received computer-assisted cognitive behavioral therapy for anxiety. Signal detection analyses utilizing receiver operating curve procedures were used to determine optimal guidelines for defining treatment response and symptom remission for youth with anxiety disorders on the PARS5 and PARS6. The percent reduction in anxiety severity was used to predict treatment responder status. The percent reduction in symptoms and posttreatment raw score were used to predict remission status.
Results: Optimal prediction of treatment response based on gold standard criteria was achieved at 15–20% reduction in symptoms on the PARS5 (with 20% reduction achieving marginally higher accuracy), and 20% reduction on the PARS6. A 25% reduction in symptoms on the PARS5 or a posttreatment raw score cutoff of 9 optimally predicted remission status. For the PARS6, a cutoff of 35% reduction or a posttreatment score of 11, was considered optimal for determining remission in clinical settings, whereas a 30% reduction or score of 12 was considered optimal for research settings.
Conclusions: With different scoring options available for the PARS, these results provide guidelines for determining response and remission based on the PARS5 and PARS6 scores. Guidelines have implications for use in clinical trials, as well as for assessment of change in clinical practice.
Introduction
Accurate measurement of symptoms, treatment progress, and end states are important for clinicians and researchers alike. The Pediatric Anxiety Rating Scale (PARS) (RUPP 2002) has become an increasingly popular clinician-administered measure of anxiety severity for use with children and adolescents following use in several large-scale treatment studies (e.g., RUPP 2001, 2002; Walkup et al. 2008). In comparison with diagnostic measures that are lengthy to administer and provide measures of anxiety severity only within individual disorder categories (for example, the Anxiety Disorder Interview Schedule [ADIS-IV]) (Silverman and Albano 1996), the PARS provides a continuous measure of anxiety severity across all anxiety disorders and has relatively brief administration time (∼30 minutes). These features make the PARS a potentially viable measure for use in clinical practice, and a promising measure for clinical trials that, typically, target youth with heterogeneous anxiety disorders. Psychometrically validated guidelines for classifying treatment response and remission using standardized measures, such as the PARS, are important for use in clinical treatment trials, as well as for benchmarking in clinical practice.
The PARS was originally developed as a clinician-rated measure to evaluate change in anxiety severity across disorders during a trial of fluvoxetine in youth with separation, social, and generalized anxiety (RUPP 2001). The PARS consists of seven clinician-rated items, although the five item total score has been recommended for use in clinical trials. The five item PARS (PARS5) excludes the “Number of Symptoms” item and the “Physical Symptoms” item, given concerns that the former may not be a valid index of anxiety severity, and that the latter may be confounded by medication side effects, especially those from serotonin reuptake inhibitors (SRI) (RUPP 2002). More recently, a six item version of the PARS (PARS6) has been used in the seminal Child/Adolescent Anxiety Multimodal Treatment Study (CAMS) (Walkup et al. 2008). This six item version excludes only the symptom count item, but retains the physical symptoms item, despite SRI medication treatment being used in two treatment arms. The PARS5 and PARS6 may have different utility and relevance in different contexts. For example, the PARS5 may be most useful for assessing anxiety severity in contexts in which medication side effects may be confounded with physical symptoms, whereas the PARS6 may be most useful in nonpharmacological contexts in which physical symptoms are less likely to be attributable to medication side effects.
Psychometric information has been reported for the PARS5, PARS6, and the seven item total score (PARS7) in a number of studies. Internal consistency was 0.64 for the PARS5 in a clinical sample of anxious youth (RUPP 2002), which has been suggested to reflect that the items are related, but not redundant. Higher internal consistency has been found in youth “at risk” for anxiety (having a parent with an anxiety disorder; α = 0.75 for the PARS5 and α = 0.81 for the PARS7) (Ginsburg et al. 2011), and in nonclinical samples (α = 0.90 for the PARS5 and α = 0.91 for the PARS7) (Ginsburg et al. 2011). Good interrater reliability has been noted for the PARS5 (intraclass correlation coefficient [ICC] = 0.87) (Storch et al. 2015), PARS6 (r = 0.85) (Caporino et al. 2013), and PARS7 (ICC = 0.97) (Ginsburg et al. 2011) as well as strong convergent validity with self-report measures of anxiety (Ginsburg et al. 2011; RUPP 2002). As a screening instrument, a cutoff of 11.5 on the PARS5 and a cutoff of 17.5 on the PARS7 most accurately differentiated youth with an anxiety disorder from those without (Ginsburg et al. 2011). Importantly, the PARS has shown good sensitivity to changes over treatment in typically developing youth (RUPP 2001, RUPP 2002; Walkup et al. 2008; Ginsburg et al. 2011) and youth with autism spectrum disorders (Storch et al. 2013; White et al. 2013; Wood et al. 2015). The PARS shows utility and promising psychometric properties for continued use in pediatric anxiety studies, necessitating further psychometric examination of the PARS5 and PARS6 versions.
Treatment efficacy studies often examine rates of treatment response and symptom remission in addition to measures of linear change. Typically, treatment response is a measure of how effective the treatment has been in reducing symptoms, although variable levels of symptoms may still be present (Bandelow 2006). In comparison, remission is typically a measure of diagnostic change, whereby those who are classified as in remission no longer meet diagnostic criteria for the specified disorder (Bandelow 2006; Steele et al. 2006). The metric used to define response and remission include percent reduction in symptoms on an outcome measure, cutoffs scores on outcome measures, as well as specific measures of improvement (e.g., Clinical Global Impressions -Improvement [CGI-I]) (Guy 1976). For comparability among studies, it is important to develop empirically derived guidelines to define treatment response and symptom remission on relevant outcome measures. Empirically derived guidelines also need to bear some utility for clinical practitioners. Research findings of effect sizes or group differences bear no utility for benchmarking an individual client's treatment progress. Therefore, empirical guidelines the utilize metrics, such as percent reduction in symptoms and raw score cutoffs, are also important to promote dissemination of research findings into clinical practice.
To date, two studies have examined optimal guidelines for defining treatment response and symptom remission based on the PARS; one utilizing the PARS6 in typically developing children (Caporino et al. 2013), and one utilizing the PARS5 in children on the autism spectrum (Johnco et al. 2015). Caporino et al. (2013) reported the results of a signal detection study that examined optimal cutoffs for defining response and remission on the PARS6 in a large sample of typically developing children with a primary diagnosis of separation, social, or generalized anxiety. Using the PARS6, a 35% reduction in symptoms on the PARS optimally predicted treatment response (κ = 0.75) whereas a 50% reduction in symptoms, or a cutoff score of 8–10, most reliably predicted symptom remission (κ = 0.69 and κ = 0.74–0.77 respectively) (Caporino et al. 2013). The second signal detection study provided guidelines for defining response and remission after cognitive behavioral therapy for anxiety in youth with autism spectrum disorders using the recommended PARS5 score (Johnco et al. 2015). Results suggested that a 15% reduction in symptoms optimally predicted treatment response (κ = 0.57), whereas a 40% reduction in symptoms (or a raw score ≤10), optimally predicted symptom remission (κ = 0.50 and 0.60 respectively).
Replicating signal detection analyses using the PARS6 is important to confirm the consistency of guidelines findings. However, extension of signal detection analyses to provide guidelines for defining response and remission based on the PARS5 are clearly warranted, given that the majority of clinical and pharmacological studies utilize the PARS5 score (RUPP 2001, RUPP 2002; Geller et al. 2007; Rynn et al. 2007; Ginsburg et al. 2011; Storch et al. 2013, 2015). Given that the PARS5 and PARS6 may have different utility and relevance for clinicians and researchers in different contexts, the availability and comparison of guidelines across these scores are important for user flexibility. Extending current guidelines to define response and remission using the PARS5 would also facilitate comparisons between the guidelines developed for typically developing youth and youth with autism spectrum disorder to inform clinical expectations about treatment effects.
Based on methods used in numerous signal detection studies (e.g., Tolin et al. 2005; Storch et al. 2010; Lewin et al. 2011; Storch et al. 2011; Caporino et al. 2013; Farris et al. 2013), this study aimed to use receiver operating characteristic (ROC) procedures to examine optimal guidelines for defining treatment response and remission of anxiety symptoms based on the PARS5 and to replicate existing guidelines using the PARS6. We hypothesized that existing recommendations (Caporino et al. 2013) for defining response and remission on the PARS6 would be replicated. We expected slightly smaller percent reductions to define response and remission based on the PARS5 given reduced variability in scores.
Method
Participants
Participants were 73 children with anxiety 7–13 years of age (mean = 9.67, SD = 1.84, 50.7% male) and their parents who completed treatment as part of a randomized controlled trial comparing a computerized cognitive behavioral intervention for anxiety (Camp Cope-A-Lot [CCAL]) (Kendall and Khanna 2008) with a treatment as usual condition. See Storch et al. (2015) for a full description of the study protocol. Participants were recruited from three community mental health centers in North, Central, and South Florida. The race of participants was primarily white (82.2%), with 11.0% identifying as African American, 4.1% identifying as Asian, and 2.7% identifying as another race. In reference to ethnicity, 9.6% were Hispanic or Latino. Participants were included if they met Diagnostic and Statistical Manual of Mental Disorders, 4th ed., Text Revision (DSM-IV-TR) criteria (American Psychiatric Association 2000) for a primary diagnosis of generalized anxiety, social anxiety, separation anxiety, and specific phobia. Because of the specificity of the treatment protocol, youth with a primary obsessive-compulsive disorder or posttraumatic stress disorder diagnoses were not included, although these disorders could be comorbid. The most common primary diagnoses were generalized anxiety (39.7%), social phobia (27.4%), and separation anxiety (23.3%). Participants were excluded if they did not meet full diagnostic criteria for a primary anxiety disorder at entry into active treatment, had not stabilized on psychotropic mediations, or were diagnosed with psychosis or bipolar disorder.
Treatment
The CCAL treatment program (Kendall and Khanna 2008) is a 12 session cognitive behavioral treatment for children 7–13 years of age, who have anxiety. The first six sessions are completed primarily on the computer, with the remaining sessions being therapist led. During the computer-led session, therapists were present to ensure proper completion of in-session tasks, as well as the homework tasks. Treatment components include psychoeducation about anxiety, identifying and dealing with anxious cognitions, problem solving, and graded exposure. This program has demonstrated efficacy for pediatric anxiety, with studies reporting remission rates of 53–81% using the CCAL treatment, and durability of treatment gains at 1 and 3 month follow-up (Khanna and Kendall 2010; Storch et al. 2015).
Measures
PARS
The PARS is a clinician-rated measure of anxiety symptom presence and anxiety severity over the past week. Questions pertain to the presence of 50 anxiety symptoms, as well as to seven global severity items that assess anxiety symptom presence, frequency, severity, avoidance, interference, and physiological symptom severity. Global items are rated on a 0–5 scale where 0 = no symptoms and 5 = extreme symptoms. The PARS5 reflects the total of five of the seven global items, excluding the item assessing symptom count and the physiological symptom severity item given potential overlap with side effects from SRI medication in children (RUPP 2002). The PARS6 reflects the total of six of the global items, excluding only the symptom count item, given concerns about the use of number of symptoms as a proxy for severity. Interviews are conducted with parents and children separately, with clinician ratings based on the combined report.
ADIS-IV
The ADIS-IV is the gold standard clinical interview to diagnose anxiety and related disorders based on DSM-IV-TR criteria (American Psychiatric Association 2000). Diagnoses are determined based on semistructured interviews with parents and children separately. Each diagnosis is given a clinician severity rating (CSR) ranging from 0 to 8, with scores ≥4 indicating clinical caseness.
CGI-I
The CGI-I is a single item clinician-rated measure of treatment-related improvement in symptoms. This measure is rated on a seven point scale, with higher scores reflecting greater improvement.
Procedure
The study was approved by the relevant institutional review board. Parents provided written consent, and assent was obtained from children. All participants completed assessments before and after CCAL treatment. Participants who were randomized to the treatment as usual condition were eligible to complete the CCAL treatment after 12 weeks, and were included in this study if they continued to meet inclusion criteria at the start of CCAL treatment. Clinical assessments were conducted via webcam and secure Internet platform by experienced assessors under the supervision of a board certified clinical child and adolescent psychologist. All assessors completed standardized training programs, including didactic training, observation, and in vivo supervision.
Data analysis
Replicating previous methodology (e.g., Tolin et al. 2005; Storch et al. 2010; Lewin et al. 2011; Storch et al. 2011; Caporino et al. 2013; Farris et al. 2013; Jeon et al. 2013; Johnco et al. 2015), we conducted signal detection analyses using ROC procedures to identify optimal cutoffs for identifying treatment response and remission on the PARS5 and PARS6. Consistent with existing definitions (e.g., Tolin et al. 2005; Lewin et al. 2011; Storch et al. 2011; Caporino et al. 2013; Farris et al. 2013; Jeon et al. 2013; Johnco et al. 2015), participants were classified as treatment responders if they scored “much” or “very much” improved on the CGI-I at posttreatment. Consistent with others (e.g., Caporino et al. 2013; Hudson et al. 2013, 2015; Johnco et al. 2015), remission was defined as loss of primary diagnosis on the ADIS-IV (CSR≤3) at posttreatment. For each proposed PARS cutoff value (percent reduction of symptoms in 5% intervals, and posttreatment raw scores), we examined the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), efficiency (percentage agreement with the gold standard test), and κ statistics to determine optimal cutoff values. Sensitivity refers to the true positive rate; the proportion of participants classified as responders/remitters by the test cutoff who meet response/remission on gold standard tests (i.e., how many responders/remitters are accurately classified by the test cutoff). Specificity refers to the true negative rate; the proportion of participants who are not responders/remitters based on gold standard measures that are accurately classified by the test as nonresponders/not remitted. The PPV is a measure of the proportion of participants classified by PARS test scores as responders/remitters who are true positives. The NPV reflects the proportion of participants classified as nonresponders/not remitted by the PARS test score, who are true negatives. Efficiency is one measure of accuracy, and reflects the percentage agreement between the PARS test score and the gold standard test. Cohen's κ is a measure of accuracy between classifications on the PARS test score and the gold standard test that also accounts for chance agreement.
The first criterion for identifying the optimal cutoffs was based on maximizing accuracy with gold standard measures. Indices of simple percentage agreement between two tests are negatively affected when there is error in the gold standard test (Kraemer et al. 2012). As such, we used methods based on quality receiver operating statistics (QROC) (Kraemer et al. 2002) that account for chance agreement between the two tests by utilizing Cohen's κ as the primary criteria to determine accuracy between the PARS cutoffs and gold standard tests. Secondary criteria were used to assist with identifying optimal cutoffs in the case of similar κ statistics, and were particularly focused on implications for use of these criteria in clinical practice. In the case of treatment response, a false negative (failing to classify a treatment responder as such) would have the potential implications of clinicians augmenting treatment (e.g., adding pharmacotherapy) and unnecessarily increasing treatment burden. As such, sensitivity was chosen as the secondary criterion to ensure maximum utility to classify actual treatment responders correctly. In the case of remission, a positive classification is likely to initiate treatment cessation (i.e., the client no longer meets criteria for an anxiety disorder; therefore, no longer needs to continue treatment for their anxiety disorder). In the case of a false positive test result, treatment may be prematurely discontinued, increasing the risk of relapse. In this context, specificity was selected as the secondary criterion for determining optimal cutoff values.
Results
Descriptive statistics
Scores on the PARS5 ranged from 7 to 20 (mean = 13.97, SD = 2.93) at pretreatment and from 0 to 21 (M = 9.58, SD = 4.61) at posttreatment. Scores on the PARS6 ranged from 9 to 24 at pretreatment (mean = 16.19, SD = 3.34) and from 0 to 23 at posttreatment (mean = 11.15, SD = 5.26). Paired-sample t tests indicated significant reduction in symptoms over treatment on the PARS5 and PARS6 (t[72] = 7.82, p < 0.001 and t[72] = 7.65, p < 0.001, respectively). The internal consistency was poor at pretreatment for the PARS5 (α = 0.54) and PARS6 (α = 0.59) but good at posttreatment (α = 0.83 for the PARS5 and α = 0.87 for the PARS6).
Prediction of treatment response using percent symptom reduction on the PARS
Based on CGI-I, 67.1% (n = 49) of the sample met criteria for treatment response. Figure 1 shows the ROC curves for the percent reduction in symptoms on the PARS5 and PARS6 to predict treatment response. The accuracy of the percent reduction in symptoms on the PARS to predict response is estimated using the area under the ROC curve (AUC), where a value of 1 indicates perfect prediction and a value of 0.5 suggests prediction that is no better than chance. The AUC statistics suggested good predictive utility for the PARS5 (AUC = 0.90, SE = 0.04, p < 0.001, 95% CI = 0.83–0.97) and PARS6 (AUC = 0.92, SE = 0.03, p < 0.001, 95% CI = 0.86–0.98). Psychometric indices used to determine optimal percent reduction cutoff on the PARS5 are reported in Table 1. The highest κ value (indicative of accuracy with the CGI-I) was achieved at a cutoff of 20% (κ = 0.66). At this cutoff, sensitivity = 0.88, specificity = 0.79, PPV = 0.90, NPV = 0.85, and simple agreement (efficiency) = 0.85. When considering the secondary criterion of sensitivity, slightly higher sensitivity achieved at 15% reduction (0.90 compared with 0.88 at 20% reduction), suggested that 90% and 88% (respectively) of responders (based on the criterion variable/CGI-I) would be classified as responders based on the PARS5 score. However, the 15% reduction cutoff resulted in a 4% loss in accuracy (κ = 0.62 compared with 0.66 at 20% reduction). In addition, the PPV value was slightly higher for a cutoff of 20% (0.90 compared with 0.86 for a 15% cutoff) indicating that 90% of individuals classified as responders on the PARS were likely to be responders based on the criterion variable (CGI-I). Although either a 15% or a 20% reduction on the PARS5 may be suitable to use as a cutoff value, a 20% reduction was considered a slightly more optimal cutoff for identifying response on the PARS5.
FIG. 1.
Receiver operative characteristic (ROC) curve for percent reduction in symptoms on the Pediatric Anxiety Rating Scale five item (PARS5) and six item (PARS6) total scores to predict treatment response (per Clinical Global Impressions–Improvement).
Table 1.
Prediction of Clinical Response Based on Clinical Global Impressions – Improvement (CGI-I) Scale Ratings (Much Improved or Very Much Improved) Using Percent Reduction of Symptoms on the Five Item Pediatric Anxiety Rating Scale (PARS5)
| PARS reduction (%) | Sensitivity | Specificity | PPV | NPV | Efficiency | k |
|---|---|---|---|---|---|---|
| ≥5 | 0.94 | 0.50 | 0.79 | 0.80 | 0.8 | 0.49 |
| ≥10 | 0.90 | 0.67 | 0.85 | 0.76 | 0.82 | 0.58 |
| ≥15 | 0.90 | 0.71 | 0.86 | 0.77 | 0.84 | 0.62 |
| ≥20 | 0.88 | 0.79 | 0.90 | 0.76 | 0.85 | 0.66 |
| ≥25 | 0.80 | 0.83 | 0.91 | 0.67 | 0.81 | 0.59 |
| ≥30 | 0.69 | 0.96 | 0.97 | 0.61 | 0.78 | 0.57 |
| ≥35 | 0.63 | 0.96 | 0.97 | 0.56 | 0.74 | 0.5 |
| ≥40 | 0.61 | 1.00 | 1.00 | 0.56 | 0.74 | 0.51 |
| ≥45 | 0.53 | 1.00 | 1.00 | 0.51 | 0.69 | 0.43 |
| ≥50 | 0.47 | 1.00 | 1.00 | 0.48 | 0.64 | 0.37 |
| ≥55 | 0.35 | 1.00 | 1.00 | 0.43 | 0.56 | 0.26 |
| ≥60 | 0.31 | 1.00 | 1.00 | 0.41 | 0.53 | 0.23 |
| ≥65 | 0.20 | 1.00 | 1.00 | 0.38 | 0.47 | 0.14 |
| ≥70 | 0.20 | 1.00 | 1.00 | 0.38 | 0.47 | 0.14 |
PPV, positive predictive value; NPV, negative predictive value.
Indices used to examine cutoffs on the PARS6 are reported in Table 2. A cutoff of 20% reduction yielded the highest κ value (κ = 0.69). At this cutoff, sensitivity = 0.90, specificity = 0.79, PPV = 0.90, NPV = 0.79, and efficiency = 0.86. Although a reduction of 25% yielded a similar accuracy (κ = 0.68), there was an 8% loss in sensitivity (0.82), suggesting that a 20% reduction was a more optimal cutoff for predicting response on the PARS6.
Table 2.
Prediction of Clinical Response Based on Clinical Global Impressions – Improvement (CGI-I) Scale Ratings (Much Improved or Very Much Improved) Using Percent Reduction of Symptoms on the Six Item Pediatric Anxiety Rating Scale (PARS6)
| PARS reduction (%) | Sensitivity | Specificity | PPV | NPV | Efficiency | k |
|---|---|---|---|---|---|---|
| ≥5 | 0.92 | 0.54 | 0.80 | 0.76 | 0.80 | 0.50 |
| ≥10 | 0.90 | 0.58 | 0.81 | 0.74 | 0.80 | 0.51 |
| ≥15 | 0.90 | 0.71 | 0.86 | 0.77 | 0.84 | 0.62 |
| ≥20 | 0.90 | 0.79 | 0.90 | 0.79 | 0.86 | 0.69 |
| ≥25 | 0.82 | 0.92 | 0.95 | 0.71 | 0.85 | 0.68 |
| ≥30 | 0.76 | 0.96 | 0.97 | 0.66 | 0.82 | 0.64 |
| ≥35 | 0.65 | 0.96 | 0.97 | 0.58 | 0.75 | 0.52 |
| ≥40 | 0.55 | 1.00 | 1.00 | 0.52 | 0.70 | 0.45 |
| ≥45 | 0.49 | 1.00 | 1.00 | 0.49 | 0.66 | 0.39 |
| ≥50 | 0.43 | 1.00 | 1.00 | 0.46 | 0.62 | 0.33 |
| ≥55 | 0.33 | 1.00 | 1.00 | 0.42 | 0.55 | 0.24 |
| ≥60 | 0.29 | 1.00 | 1.00 | 0.41 | 0.52 | 0.21 |
| ≥65 | 0.20 | 1.00 | 1.00 | 0.38 | 0.47 | 0.14 |
| ≥70 | 0.18 | 1.00 | 1.00 | 0.38 | 0.45 | 0.13 |
PPV, positive predictive value; NPV, negative predictive value.
Prediction of symptom remission using the PARS
To maximize clinical utility of results for research and clinical trials, we examined two outcome scores on the PARS for predicting remission status: The percent reduction in symptoms on the PARS5 and PARS6, and posttreatment raw scores on the PARS5 and PARS6. Based on ADIS-IV criteria, 54.8% (n = 40) of the sample met criteria for remission. Figure 2 shows two ROC curves examining the predictive utility of percent reduction in symptoms on the PARS5 and PARS6 for predicting remission, and Figure 3 shows two ROC curves that examine the utility of posttreatment scores on the PARS5 and PARS6 to predict remission. ROC analyses require higher test scores to be reflective of higher probability of a positive gold standard test result. In the case of posttreatment raw scores, lower scores reflect an increased chance of being classified as in remission. To preserve the direction of the relationship for the ROC analyses, posttreatment raw scores were reversed to run the ROC analyses. Normally coded scores were used for the calculation of other fit indices. The AUC statistic suggested good predictive utility for the percent reduction scores on the PARS5 (AUC = 0.90, SE = 0.04, p < 0.001, 95% CI = 0.83–0.97) and PARS6 (AUC = 0.91, SE = 0.03, p < 0.001, 95% CI = 0.85–0.97), as well as the posttreatment raw scores on the PARS5 (AUC = 0.92, SE = 0.03, p < 0.001, 95% CI = 0.86–0.98) and PARS6 (AUC = 0.93, SE = 0.03, p < 0.001, 95% CI = 0.87–0.98).
FIG. 2.
Receiver operative characteristic (ROC) curve for percent reduction in symptoms on the Pediatric Anxiety Rating Scale five item (PARS5) and six item (PARS6) total scores to predict remission (per loss of primary diagnosis on the Anxiety Disorders Interview Schedule).
FIG. 3.
Receiver operative characteristic (ROC) curve for posttreatment raw scores on Pediatric Anxiety Rating Scale five item (PARS5) and six item (PARS6) total score to predict remission (per loss of primary diagnosis on the Anxiety Disorders Interview Schedule).
Table 3 reports on the indices used to examine prediction of response based on percent reduction in symptoms on the PARS5. A cutoff of 25% reduction resulted in a considerably higher κ value (κ = 0.69) than other cutoffs. Any improvements in specificity were at the expense of accuracy; therefore, 25% was determined to be the optimal cutoff on the PARS5. At this cutoff, sensitivity = 0.90, specificity = 0.79, PPV = 0.84, NPV = 0.87, and efficiency = 0.85.
Table 3.
Prediction of Clinical Remission Based on Loss of Primary Diagnosis on the Anxiety Disorders Interview Schedule Using Percent Reduction of Symptoms on the Five Item Pediatric Anxiety Rating Scale (PARS5)
| PARS reduction (%) | Sensitivity | Specificity | PPV | NPV | Efficiency | k |
|---|---|---|---|---|---|---|
| ≥5 | 0.95 | 0.39 | 0.66 | 0.87 | 0.7 | 0.36 |
| ≥10 | 0.93 | 0.55 | 0.71 | 0.86 | 0.75 | 0.49 |
| ≥15 | 0.93 | 0.58 | 0.73 | 0.86 | 0.77 | 0.52 |
| ≥20 | 0.90 | 0.63 | 0.75 | 0.84 | 0.78 | 0.55 |
| ≥25 | 0.90 | 0.79 | 0.84 | 0.87 | 0.85 | 0.69 |
| ≥30 | 0.75 | 0.85 | 0.86 | 0.74 | 0.8 | 0.59 |
| ≥35 | 0.70 | 0.88 | 0.88 | 0.71 | 0.78 | 0.57 |
| ≥40 | 0.68 | 0.91 | 0.90 | 0.70 | 0.78 | 0.57 |
| ≥45 | 0.63 | 0.97 | 0.96 | 0.68 | 0.78 | 0.57 |
| ≥50 | 0.58 | 1.00 | 1.00 | 0.66 | 0.77 | 0.55 |
| ≥55 | 0.43 | 1.00 | 1.00 | 0.59 | 0.69 | 0.40 |
| ≥60 | 0.38 | 1.00 | 1.00 | 0.57 | 0.66 | 0.35 |
| ≥65 | 0.25 | 1.00 | 1.00 | 0.52 | 0.59 | 0.23 |
| ≥70 | 0.25 | 1.00 | 1.00 | 0.52 | 0.59 | 0.23 |
PPV, positive predictive value; NPV, negative predictive value.
For the PARS6, the highest accuracy was seen at 30% reduction (κ = 0.67), although it was similar at a 35% reduction (κ = 0.65; see Table 4). In the context of a 6% improvement in specificity at a cutoff of 35% (0.91 compared with 0.85 at 30% reduction), we considered 35% to be a slightly more optimal cutoff for identifying remission on the PARS6 for clinical practice. At this cutoff, sensitivity = 0.75, specificity = 0.91, PPV = 0.91, NPV = 0.75, and efficiency = 0.82. However a cutoff of 30% may still be useful in some contexts in which remission is used as an end-point, given that there was more similarity between the sensitivity and specificity indices. At a cutoff of 30%, sensitivity = 0.83, PPV = 0.87, NPV = 0.75, and efficiency = 0.84.
Table 4.
Prediction of Clinical Remission Based on Loss of Primary Diagnosis on the Anxiety Disorders Interview Schedule Using Percent Reduction of Symptoms on the Six Item Pediatric Anxiety Rating Scale (PARS6)
| PARS reduction (%) | Sensitivity | Specificity | PPV | NPV | Efficiency | k |
|---|---|---|---|---|---|---|
| ≥5 | 0.95 | 0.45 | 0.68 | 0.88 | 0.73 | 0.42 |
| ≥10 | 0.95 | 0.52 | 0.70 | 0.89 | 0.75 | 0.48 |
| ≥15 | 0.93 | 0.58 | 0.73 | 0.86 | 0.77 | 0.52 |
| ≥20 | 0.90 | 0.61 | 0.73 | 0.83 | 0.77 | 0.52 |
| ≥25 | 0.85 | 0.76 | 0.81 | 0.81 | 0.81 | 0.61 |
| ≥30 | 0.83 | 0.85 | 0.87 | 0.80 | 0.84 | 0.67 |
| ≥35 | 0.75 | 0.91 | 0.91 | 0.75 | 0.82 | 0.65 |
| ≥40 | 0.65 | 0.97 | 0.96 | 0.70 | 0.80 | 0.60 |
| ≥45 | 0.60 | 1.00 | 1.00 | 0.67 | 0.78 | 0.58 |
| ≥50 | 0.53 | 1.00 | 1.00 | 0.63 | 0.74 | 0.50 |
| ≥55 | 0.40 | 1.00 | 1.00 | 0.58 | 0.67 | 0.38 |
| ≥60 | 0.35 | 1.00 | 1.00 | 0.56 | 0.64 | 0.33 |
| ≥65 | 0.25 | 1.00 | 1.00 | 0.52 | 0.59 | 0.23 |
| ≥70 | 0.23 | 1.00 | 1.00 | 0.52 | 0.58 | 0.21 |
PPV, positive predictive value; NPV, negative predictive value.
Next, we examined optimal raw score cutoffs on the PARS5 and PARS6 to predict remission. Table 5 reports on fit indices for predicting remission based on posttreatment raw scores on the PARS5. The highest accuracy was achieved at a cutoff score of 9 (κ = 0.76), and specificity was high (0.97). At this cutoff, sensitivity = 0.80, PPV = 0.97, NPV = 0.80, and efficiency = 0.88. No other cutoff provided comparable predictive validity.
Table 5.
Prediction of Clinical Remission Based on Loss of Primary Diagnosis on the Anxiety Disorders Interview Schedule Using Posttreatment Cutoff Scores on the Five Item Pediatric Anxiety Rating Scale (PARS5)
| PARS cutoff score | Sensitivity | Specificity | PPV | NPV | Efficiency | k |
|---|---|---|---|---|---|---|
| ≤1 | 0.08 | 1.00 | 1.00 | 0.47 | 0.49 | 0.07 |
| ≤2 | 0.15 | 1.00 | 1.00 | 0.49 | 0.53 | 0.14 |
| ≤3 | 0.25 | 1.00 | 1.00 | 0.52 | 0.59 | 0.23 |
| ≤4 | 0.28 | 1.00 | 1.00 | 0.53 | 0.60 | 0.26 |
| ≤5 | 0.33 | 1.00 | 1.00 | 0.55 | 0.63 | 0.30 |
| ≤6 | 0.55 | 1.00 | 1.00 | 0.65 | 0.75 | 0.53 |
| ≤7 | 0.60 | 1.00 | 1.00 | 0.67 | 0.78 | 0.58 |
| ≤8 | 0.73 | 1.00 | 1.00 | 0.75 | 0.85 | 0.70 |
| ≤9 | 0.80 | 0.97 | 0.97 | 0.80 | 0.88 | 0.76 |
| ≤10 | 0.80 | 0.76 | 0.80 | 0.76 | 0.78 | 0.56 |
| ≤11 | 0.88 | 0.64 | 0.74 | 0.81 | 0.77 | 0.52 |
| ≤12 | 0.95 | 0.55 | 0.72 | 0.90 | 0.77 | 0.51 |
| ≤13 | 1.00 | 0.50 | 0.71 | 1.00 | 0.77 | 0.51 |
| ≤14 | 1.00 | 0.36 | 0.66 | 1.00 | 0.71 | 0.39 |
| ≤15 | 1.00 | 0.18 | 0.60 | 1.00 | 0.63 | 0.20 |
| ≤16 | 1.00 | 0.09 | 0.57 | 1.00 | 0.59 | 0.10 |
| ≤17 | 1.00 | 0.06 | 0.56 | 1.00 | 0.58 | 0.07 |
| ≤18 | 1.00 | 0.03 | 0.56 | 1.00 | 0.56 | 0.03 |
PPV, positive predictive value; NPV, negative predictive value.
For the PARS6, the highest accuracy was reached at a cutoff score of 12 (κ = 0.72), although there was similar accuracy at a cutoff score of 11 (κ = 0.70; see Table 6). Given the 9% improvement in specificity at a score of 11 (0.94) in comparison with 12 (0.85), this lower cutoff was considered to be slightly more optimal for clinical practice. At this cutoff, sensitivity = 0.78, PPV = 0.94, NPV = 0.78, and efficiency = 0.85. However, a cutoff of 12 may also be useful in cases in which a more even trade-off between sensitivity and specificity is needed (e.g., research contexts), given that at a cutoff of 12 resulted in a 10% improvement in sensitivity and a 9% reduction in specificity in comparison with a score of 11, sensitivity = 0.88, PPV = 0.88, NPV = 0.85, and efficiency = 0.86. Given that guidelines for defining remission in research trials are typically used to evaluate treatment efficacy, rather than to inform treatment discontinuation decisions, there is less emphasis to preference higher specificity for guidelines used in this context.
Table 6.
Prediction of Clinical Remission Based on Loss of Primary Diagnosis on the Anxiety Disorders Interview Schedule Using Posttreatment Cutoff Scores on the Six Item Pediatric Anxiety Rating Scale (PARS6)
| PARS cutoff score | Sensitivity | Specificity | PPV | NPV | Efficiency | k |
|---|---|---|---|---|---|---|
| ≤1 | 0.05 | 1.00 | 1.00 | 0.46 | 0.48 | 0.05 |
| ≤2 | 0.15 | 1.00 | 1.00 | 0.49 | 0.53 | 0.14 |
| ≤3 | 0.23 | 1.00 | 1.00 | 0.52 | 0.58 | 0.21 |
| ≤4 | 0.23 | 1.00 | 1.00 | 0.52 | 0.58 | 0.21 |
| ≤5 | 0.28 | 1.00 | 1.00 | 0.53 | 0.60 | 0.26 |
| ≤6 | 0.40 | 1.00 | 1.00 | 0.58 | 0.67 | 0.38 |
| ≤7 | 0.50 | 1.00 | 1.00 | 0.62 | 0.73 | 0.48 |
| ≤8 | 0.58 | 1.00 | 1.00 | 0.66 | 0.77 | 0.55 |
| ≤9 | 0.65 | 1.00 | 1.00 | 0.70 | 0.81 | 0.63 |
| ≤10 | 0.73 | 0.97 | 0.97 | 0.74 | 0.84 | 0.68 |
| ≤11 | 0.78 | 0.94 | 0.94 | 0.78 | 0.85 | 0.70 |
| ≤12 | 0.88 | 0.85 | 0.88 | 0.85 | 0.86 | 0.72 |
| ≤13 | 0.88 | 0.61 | 0.73 | 0.80 | 0.75 | 0.49 |
| ≤14 | 0.95 | 0.55 | 0.72 | 0.90 | 0.77 | 0.51 |
| ≤15 | 0.98 | 0.48 | 0.70 | 0.94 | 0.75 | 0.48 |
| ≤16 | 1.00 | 0.39 | 0.67 | 1.00 | 0.73 | 0.42 |
| ≤17 | 1.00 | 0.21 | 0.61 | 1.00 | 0.64 | 0.23 |
| ≤18 | 1.00 | 0.09 | 0.57 | 1.00 | 0.59 | 0.10 |
| ≤19 | 1.00 | 0.06 | 0.56 | 1.00 | 0.58 | 0.07 |
| ≤20 | 1.00 | 0.06 | 0.56 | 1.00 | 0.58 | 0.07 |
PPV, positive predictive value; NPV, negative predictive value.
Discussion
The PARS has good utility for providing a clinician-rated measure of anxiety severity across anxiety disorders, and empirically derived guidelines for defining treatment response and symptom remission are important to facilitate comparability among trials, and for dissemination into clinical practice. With two different scoring methods being used for the PARS, it is important to have guidelines available for both, to improve the potential for usage. Recent guidelines were developed for defining response and remission based on the PARS6 (Caporino et al. 2013); however, these findings warrant replication, as well as extension and comparison with guidelines using the PARS5 score. Most studies have utilized the five item total score (e.g., RUPP 2001, 2002; Geller et al. 2007; Rynn et al. 2007; Ginsburg et al. 2011; Storch et al. 2013, 2015), given that this score is recommended for use in clinical trials on the basis that inclusion of an item assessing physiological symptoms is confounded in the context of pharmacotherapy side effects (RUPP 2002). As such, we replicated signal detection studies to provide alternative guidelines for defining treatment response and remission based on the PARS5, and replicated analyses using the PARS6. There are different contexts in which reduction in symptoms, or raw score cutoffs, are preferable metrics, and each has strengths and limitations. Percentage reduction in symptoms serves as a useful metric to measure intraindividual change, and can be relevant for measuring whether a particular treatment is having an impact on symptoms. In the same manner as raw score cutoffs may be useful in context of identifying when participants are likely to meet diagnostic caseness, raw score cut-offs are also useful in circumstances in which it is necessary to predict at what score an individual is no longer likely to be considered a clinical “case” or to no longer have a diagnosis. Similar to previous studies (e.g., Storch et al. 2010; Lewin et al. 2011; Storch et al. 2011; Caporino et al. 2013, Farris et al. 2013), we developed guidelines based on both metrics to maximize clinical and empirical utility.
Based on our findings, a 15–20% reduction in symptoms on the PARS5 optimally predicted treatment response, with slightly higher accuracy at a cutoff of 20%. These findings are mostly consistent with guidelines for anxious youth with autism spectrum disorders that suggested optimal prediction of treatment response at 15% reduction on the PARS (Johnco et al. 2015). We attempted to replicate guidelines for predicting treatment response using the PARS6, with our results suggesting an optimal cutoff of 20% reduction, which is slightly lower than previous guidelines for typically developing youth using the PARS6 (35%) (Caporino et al. 2013). It is possible that the low internal consistency of the PARS at pretreatment, commonly found in clinical samples (e.g., RUPP 2002), increases the variability in the percent reduction metric across studies.
Results suggested that a 25% reduction in symptoms on the PARS5 optimally predicted remission status, which is lower than guidelines for the PARS5 in youth with autism spectrum disorders (40%) (Johnco et al. 2015). For the PARS6, a 30% or 35% reduction in symptoms was considered optimal to predict remission, with 30% likely to be most useful in research contexts, and 35% likely to be most useful in clinical contexts. The more conservative criterion for clinical settings is based on higher specificity to reduce false positive rates, given that this has implications for premature discontinuation of treatment. The slightly less conservative rates recommended for research settings are based on maximizing accuracy, with equal weighting given to specificity and sensitivity, given that remitter classification is used more to evaluate treatment efficacy than to change treatment course. These cutoffs are slightly lower than previous guidelines using the PARS6, in which a 50% reduction was recommended (Caporino et al. 2013), and as discussed, it is possible that the internal consistency at pretreatment influences variability in the score between studies. However, the optimal raw score cutoffs to define remission were similar among the studies, with a posttreatment raw score cutoff of 9 best predicting remission status in typically developing youth on the PARS5 compared with a score of 10 for youth with autism spectrum disorders (Johnco et al. 2015). A posttreatment score of 11 or 12 on the PARS6 optimally predicted remission, with 12 being more useful for research contexts and 11 being more useful for clinical contexts. This is similar to previous guidelines that suggested slightly more conservative cutoffs of 8–10 on the PARS6 (Caporino et al. 2013). Given the better internal consistency of the PARS at posttreatment, a raw score cutoff for assessing remission may be preferable to percent reduction scores.
There are several methodological issues to consider when interpreting results. First, the gold standard measure of improvement was administered by the same clinician who administered the PARS, and it is possible that ratings may be somewhat influenced by each other. However, any rating of improvement will require the gathering of contextual information, and information obtained during administration of other measures is consistent with information that would also be gathered during the PARS assessment. Second, unlike guidelines developed using the PARS6 that included change averaged across both psychological and pharmacological treatments (Caporino et al. 2013), data utilized in this study reflect changes resulting from psychological treatment only. It is possible that the differences in treatments received may explain some of the differences in definitions for the PARS6 between our findings and those of Caporino et al. (2013). Although there is currently no evidence to suggest that the PARS measures change differently depending on the mechanism of symptom change, it is possible that our less conservative cutoffs on the PARS6 are reflective of lower levels of symptom change needed to indicate clinically meaningful response to psychological treatments. Third, results were analyzed using data from those who completed treatment and had posttreatment data available, and limitations of completer analyses are acknowledged. Finally, these guidelines are based on, and, therefore, are applicable for, children 7–13 years of age with a subset of primary anxiety disorders (social, separation, generalized anxiety, and specific phobias), including those with internalizing and externalizing comorbidities. These guidelines do not generalize to youth with obsessive compulsive disorder or posttraumatic disorder, and replication of signal detection analyses are needed to extend findings to adolescents.
Conclusions
This study extends current guidelines for defining treatment response and symptom remission to the PARS5, as well as replicating results using the PARS6. Optimal guidelines tended to be more apparent using PARS5 scores, whereas scores on the PARS6 often had two potential cutoffs available, with recommendations being made based on fit metrics and the goal of the test usage. Overall, our findings suggest slightly less conservative guidelines for defining treatment response and remission in anxious children using the PARS6, in comparison with previous guidelines. There appear to be some psychometric advantages to using raw score cutoffs at posttreatment to define remission status.
Clinical Significance
The PARS continues to show usefulness as a measure of anxiety severity across disorders. This study provides psychometrically validated guidelines for identifying treatment response and remission based on the widely used PARS5, and updates existing guidelines based on the PARS6. Given the inconsistencies in definitions for defining response and remission, these guidelines provide two metrics to increase utility and ease of use: percent reduction in symptoms and raw score cutoffs. These guidelines have implications for standardizing criteria used during research trials, as well as for providing empirically based guidelines for treatment decision making in clinical practice.
Acknowledgments
We acknowledge the contributions of Tyne Pierce, Amanda Krucke, Christin Cooper, Wendy Kubar, Stephanie Dobbs, and April Lott at Directions for Living in Largo, Florida; James Zenel, in Clearwater, FL; Ashley Holden, Elise Ward, Sonya Hernandez, Bhagirat Sahas, and Pamela Galan at Henderson Behavioral Health in Fort Lauderdale, Florida; Tanya White, Lori Olsen, Shannon Massingale, Ruqayyah Gaber, John Bilbrey, Carol Clark, Shaun Dahle, Ed Mobley, and Larry Williams, at Lakeview Center Inc. in Pensacola, Florida; Ross Andel, Michael Sulkowski, Morgan A. King, Elysse Arnold, Alessandro De Nadai, Joshua Nadeau, Anna Jones, Brittany Kugler, Joseph McGuire, Erika A. Crawford, Nicole M. McBride Danielle Ung, Jennifer Park, Benjamin Chang, Stella Polycarpou, Marie McPherson, and Robert Constantine, at the University of South Florida; Nick Dewan, of BayCare Health System; and Muniya Khannaat the University of Pennsylvania.
Disclosures
Dr. Carly Johnco reports no conflicts of interest. Dr. Alison Salloum has grant support from the National Institute of Mental Health. Dr. Adam Lewin (in the past 3 years) has research support from All Children's Hospital and the International OCD Foundation; travel support from the American Psychological Association, the National Institute for Mental Health, Rogers Memorial Hospital, and the Tourette Syndrome Association; and honoraria from the Children's Tumor Foundation, Elsevier, Oxford Press, and Springer Publishing. He has served as an educational consultant for Prophase, LLC. Dr. Lewin is a member of the scientific and clinical advisory board for the International OCD Foundation and is on the executive board for the Society for Clinical Child and Adolescent Psychology. Dr. Eric Storch has received grant funding from the Agency for Healthcare Research and Quality (AHRQ), All Children's Hospital Research Foundation, the Centers for Disease Control (CDC), the Foundation for Research on Prader–Willi Syndrome, the International OCD Foundation, Janssen, the National Alliance for Research on Schizophrenia and Affective Disorders (NARSAD), the National Institutes of Health (NIH), and the Tourette Syndrome Association (TSA). He receives honoraria from the American Psychological Association, Lawrence Erlbaum, and Springer Publishing. He has served as an educational consultant for CroNos, Prophase, and Rogers Memorial Hospital. He has served on the speakers' bureau and scientific advisory board for the International OCD Foundation. He has received research support from the All Children's Hospital Guild Endowed Chair.
References
- American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders, 4th ed., Text Revision. Washington, DC: American Psychiatric Association; 2000 [Google Scholar]
- Bandelow B: Defining response and remission in anxiety disorders: Toward an integrated approach. CNS Spectr 11:21–28, 2006 [DOI] [PubMed] [Google Scholar]
- Caporino NE, Brodman DM, Kendall PC, Albano AM, Sherrill J, Piacentini J, Sakolsky D, Birmaher B, Compton SN, Ginsburg G, Rynn M, McCracken J, Gosch E, Keeton C, March J. and Walkup JT: Defining treatment response and remission in child anxiety: Signal detection analysis using the Pediatric Anxiety Rating Scale. J Am Acad Child Adolesc Psychiatry 52:57–67, 2013 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Farris SG, McLean CP, Van Meter PE, Simpson HB, Foa EB: Treatment response, symptom remission, and wellness in obsessive-compulsive disorder. J Clin Psychiatry 74:685–690, 2013 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Geller D, Donnelly C, Lopez F, Rubin R, Newcorn J, Sutton V, Bakken R, Paczkowski M, Kelsey D, Sumner C: Atomoxetine treatment for pediatric patients with attention-deficit/hyperactivity disorder with comorbid anxiety disorder. J Am Acad Child Adolesc Psychiatry 46:1119–1127, 2007 [DOI] [PubMed] [Google Scholar]
- Ginsburg GS, Keeton CP, Drazdowski TK, Riddle MA: The utility of clinicians ratings of anxiety using the Pediatric Anxiety Rating Scale (PARS). Child Youth Care Forum 40:93–105, 2011 [Google Scholar]
- Guy W: Clinical Global Impressions ECDEU Assessment Manual for Psychopharmacology. Rockville, MD: National Institute for Mental Health; 1976 [Google Scholar]
- Hudson JL, Keers R, Roberts S, Coleman JRI, Breen G, Arendt K, Bogels S, Cooper P, Creswell C, Hartman C, Heiervang E, Hötzel K, In-Albon T, Lavallee K, Lyneham HJ, Marin CE, McKinnon A, Meiser–Stedman R, Morris T, Nauta M, Rapee RM, Schneider S, Schneider S, Silverman WK, Thastum M, Thirlwall K, Waite P, Wergeland GJ, Lester KJ, Eley TC: Clinical predictors of response to cognitive-behavioral therapy in pediatric anxiety disorders: The Genes for Treatment (GxT) Study. J Am Acad Child Adolesc Psychiatry 54:454–463, 2015 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hudson JL, Lester KJ, Lewis CM, Tropeano M, Creswell C, Collier DA, Cooper P, Lyneham HJ, Morris T, Rapee RM, Roberts S, Donald JA. and Eley TC: Predicting outcomes following cognitive behaviour therapy in child anxiety disorders: The influence of genetic, demographic and clinical information. J Child Psychol Psychiatry 54:1086–1094, 2013 [DOI] [PubMed] [Google Scholar]
- Jeon S, Walkup JT, Woods DW, Peterson A, Piacentini J, Wilhelm S, Katsovich L, McGuire JF, Dziura J, Scahill L: Detecting a clinically meaningful change in tic severity in Tourette syndrome: A comparison of three methods. Contemp Clin Trials 36:414–420, 2013 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Johnco CJ, De Nadai AS, Lewin AB, Ehrenreich–May J, Wood JJ, Storch EA: Defining treatment response and symptom remission for anxiety disorders in pediatric autism spectrum disorders using the Pediatric Anxiety Rating Scale. J Autism Dev Disord 45:3232–3242, 2015 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kendall PC, Khanna MS: Camp Cope-A-Lot: The Coping Cat DVD [DVD]. Ardmore, PA: Workbook, 2008 [Google Scholar]
- Khanna MS, Kendall PC: Computer- assisted cognitive behavioral therapy for child anxiety: Results of a randomized clinical trial. Journal of consulting and clinical psychology. 78:737–745, 2010 [DOI] [PubMed] [Google Scholar]
- Kraemer HC, Kupfer DJ, Clarke DE, Narrow WE, Regier DA: DSM-5: How reliable is reliable enough? Am J Psychiatry 169:13–15, 2012 [DOI] [PubMed] [Google Scholar]
- Kraemer HC, Periyakoil VS, Noda A: Kappa coefficients in medical research. Stat Med 21:2109–2129, 2002 [DOI] [PubMed] [Google Scholar]
- Lewin AB, Denadai AS, Park JM, Goodman WK, Murphy TK, Storch EA: Refining clinical judgment of treatment outcome in obsessive–compulsive disorder. Psychiatry Res 185:394–401, 2011 [DOI] [PubMed] [Google Scholar]
- RUPP: Fluvoxamine for the treatment of anxiety disorders in children and adolescents. New Engl J Med 344:1279–1285, 2001 [DOI] [PubMed] [Google Scholar]
- RUPP: The Pediatric Anxiety Rating Scale (PARS): Development and psychometric properties. J Am Acad Child Adolesc Psychiatry 41:1061–1069, 2002 [DOI] [PubMed] [Google Scholar]
- Rynn MA, Riddle MA, Yeung PP, Kunz NR: Efficacy and safety of extended-release venlafaxine in the treatment of generalized anxiety disorder in children and adolescents: Two placebo-controlled trials. Am J Psychiatry 164:290–300, 2007 [DOI] [PubMed] [Google Scholar]
- Silverman WK, Albano AM: The anxiety disorders interview schedule for DSM-IV–child and parent versions. London: Oxford University Press; 1996 [Google Scholar]
- Steele M, Jensen PS, Quinn DM: Remission versus response as the goal of therapy in ADHD: A new standard for the field? Clin Ther 28:1892–1908, 2006 [DOI] [PubMed] [Google Scholar]
- Storch EA, Arnold EB, Lewin AB, Nadeau JM, Jones AM, De Nadai AS, Mutch JP, Selles RR, Ung D, Murphy TK: The effect of cognitive-behavioral therapy versus treatment as usual for anxiety in children with autism spectrum disorders: A randomized, controlled trial. J Am Acad Child Adolesc Psychiatry 52:132–142.e132, 2013 [DOI] [PubMed] [Google Scholar]
- Storch EA, De Nadai AS, Lewin AB, McGuire JF, Jones AM, Mutch PJ, Shytle RD, Murphy TK: Defining treatment response in pediatric tic disorders: A signal detection analysis of the Yale Global Tic Severity Scale. J Child Adolesc Psychopharmacol 21:621–627, 2011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Storch EA, Lewin AB, De Nadai AS, Murphy TK: Defining treatment response and remission in obsessive-compulsive disorder: A signal detection analysis of the Children's Yale-Brown Obsessive Compulsive Scale. J Am Acad Child Adolesc Psychiatry 49:708–717, 2010 [DOI] [PubMed] [Google Scholar]
- Storch EA, Salloum A, King MA, Crawford EA, Andel R, McBride NM, Lewin AB: A randomized controlled trial in community mental health centers of computer-assisted cognitive behavioral therapy versus treatment as usual for children with anxiety. Depress Anxiety 32:843–852, 2015 [DOI] [PubMed] [Google Scholar]
- Tolin DF, Abramowitz JS, Diefenbach GJ: Defining response in clinical trials for obsessive-compulsive disorder: A signal detection analysis of the Yale-Brown obsessive compulsive scale. J Clin Psychiatry 66:1549–1557, 2005 [DOI] [PubMed] [Google Scholar]
- Walkup JT, Albano AM, Piacentini J, Birmaher B, Compton SN, Sherrill JT, Ginsburg GS, Rynn MA, McCracken J, Waslick B, Iyengar S, March JS, Kendall PC: Cognitive behavioral therapy, sertraline, or a combination in childhood anxiety. New Engl J Med 359:2753–2766, 2008 [DOI] [PMC free article] [PubMed] [Google Scholar]
- White SW, Ollendick T, Albano AM, Oswald D, Johnson C, Southam–Gerow MA, Kim I, Scahill L. Randomized controlled trial: Multimodal anxiety and social skill intervention for adolescents with autism spectrum disorder. J Autism Dev Disord. 43:382–394, 2013 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wood JJ, Ehrenreich–May J, Alessandri M, Fujii C, Renno P, Laugeson E, Piacentini JC, De Nadai AS, Arnold EB, Lewin AB, Murphy TK, Storch EA: Cognitive behavioral therapy for early adolescents with autism spectrum disorders and clinical anxiety: A randomized, controlled trial. Behav Ther 46:7–19, 2015 [DOI] [PMC free article] [PubMed] [Google Scholar]