Parent and Child Agreement on Anxiety Disorder Symptoms Using the DISC Predictive Scales

Abstract

Growing recognition of the negative impact of anxiety disorders in the lives of youth has made their identification an important clinical task. Multiple perspective assessment (e.g., parents, children) is generally considered a preferred method in the assessment of anxiety disorder symptoms although it has been generally thought that disagreement between parent and youth ratings of the child’s emotions is common. This study examined parent and child reports of the child’s anxiety disorder symptoms using the Diagnostic Interview Schedule for Children-Predictive Scales (DISC-PS) in a clinic referred sample of substance using adolescents. Parents and adolescents (N = 480) who were referred for substance abuse treatment were screened for anxiety disorder symptoms using the DISC-PS at pre treatment. Results suggest similar (low) levels of agreement between the parent report and child report versions as found with other anxiety symptom and anxiety disorder measures. Findings provide data on multi-informant agreement and highlight issues in the use of the (DISC-PS).

Anxiety disorders in childhood and adolescence are prevalent, predict substance use problems later in life (Kendall, Safford, Flannery-Schroeder, & Webb, 2004; Pine, Cohen, Gurley, Brook, & Ma, 1998) and are associated with a number of negative cognitive, neuro-developmental, and hormonal outcomes (Carrión, Weems, & Reiss, 2007; De Bellis et al., 1999). Multiple perspective assessment (e.g., parents, children) is generally considered a preferred method in the identification and assessment of anxiety disorders (Weems & Stickle, 2005). However, disagreement between parent and youth ratings of the child’s emotional problems is generally thought to be common (e.g., Achenbach, McConaughty, & Howell, 1987; De Los Reyes & Kazdin, 2005; Edelbrock, Costello, Dulcan, Conover, & Kala, 1986). This is particularly true for anxiety problems because although anxiety symptoms do have behavioral referents that parents can observe they also have internal symptoms that may be most salient to the child. Reviews of the literature on parent child agreement for childhood anxiety disorders suggest that, on average, kappas are in the fair (k > .4 but < .59) to poor (k < .40) range (Grills & Ollendick, 2002).

The purpose of this paper was to examine parent and child agreement on the child’s anxiety disorder symptoms using the Diagnostic Interview Schedule for Children-Predictive Scales (DISC-PS; Lucas et al., 2001) in a clinic referred sample of substance using adolescents. The DISC-PS is a commonly used measure to screen for mental health problems; however, we are aware of no data on the cross informant associations for the child anxiety symptoms of this instrument. Data on the full DISC interview suggests parent-child kappa’s in the poor range (k < .2) for anxiety disorders and correlation between parent and child ratings of symptoms in the r = .1 to .36 range (See Grills & Ollendick, 2002). Examining agreement on the DISC-PS anxiety symptoms is both clinically and theoretically important. Theoretically, unlike exact parallel forms of many self reported anxiety measures, where the stems are simply changed from “I” to “my child”, the DISC-PS uses a reporter targeted way of evaluating anxiety symptoms. Specifically, the DISC-PS uses items from the DISC that are most salient/predictive from the standpoint of the reporter (i.e., more behavioral indicators are used on the parent report versus the child report). For example, on social phobia the question focuses on the avoidance of social situations for the parent version and more on the internal feelings of fear and distress of social situations for the child. However, no studies have examined the level of parent child agreement this format produces (i.e., better or worse).

Clinically, there is also a need to examine agreement in samples where empirical knowledge of agreement may have direct clinical implications as studies suggest lower agreement on secondary anxiety problems (Reuterskiold, Ost, & Ollendick, 2008). For example, while anxiety problems are common in substance abuse referred cases (Couwenbergh et al., 2006; Riggs, 2003) a discrepancy in reporting anxious symptoms in such samples might result in less attention paid to the anxiety problem in therapy if agreement is expected by a clinician. Identifying the level of agreement and distribution of discrepancies in reporting may help clinicians make better informed decisions about targets for treatment in the context of disagreement. We are aware of no studies that have examined anxiety disorder symptoms agreement in a substance use referred sample.

Based on the extant literature, we expected a relatively small (r in the .1 to .36 range) but significant association between parent report and child report on the DISC-PS anxiety symptoms. The pattern of agreement (kappa) between parent and child reported diagnoses based on the criteria for probable diagnoses reported in Lucas et al. (2001) was also examined and we expected kappas in the .1 to .3 range.

Method

Participants and Procedures

The data for this study represents secondary data analysis of a community based effectiveness study for the treatment of youth substance problems (Details can be found in Horigian et al., 2009 and Robbins et al., 2009). This study used the baseline pre treatment assessment data. Four hundred and eighty adolescents¹ and their families referred for the treatment of drug abuse comprised the sample. The study was approved by the Institutional Review Board of the University of Miami. The sample was aged 12–17 years (mean age 15.5 years) and was 78.5% male. Ethnic composition was as 44% Hispanic (n = 213), 31% White (n = 148), 23% African-American (n = 110), and 2% (n = 9) youth were of other ethnicities. “Parent” reporters included 374 mothers (78%), 67 fathers (14%), 20 grandmothers (4%), 3 grandfathers (1%) and 16 other (3%).

Measure

The Diagnostic Interview Schedule for Children-Predictive Scales (DISC-PS; Lucas et al., 2001) was used to measure anxiety symptoms. Items assess each of the anxiety disorders (i.e., simple phobia, social anxiety disorder, agoraphobia, panic disorder, generalized anxiety disorder, and obsessive-compulsive disorder symptoms) except post-traumatic stress. This instrument has demonstrated good sensitivity and specificity compared to the full Diagnostic Interview Schedule for Children (Lucas et al., 2001). In addition to probable diagnosis (i.e., yes or no diagnoses based on the symptoms criteria reported in Lucas et al. and exclusionary criteria in the instrument) two continuous scales were created from the symptom items [i.e., anxiety symptoms reported by the child (34 items, in this sample coefficient alpha was .85) and the parent (36 items, in this sample coefficient alpha was .86). To examine agreement we computed the difference in z scores and the absolute value of the DIZ scores. For consistency, we will use the term “difference in z” scores and the acronym (DIZ) and the term “absolute value of the difference in z” scores (AVDIZ) when referring to these metrics in the remainder of this paper. DIZ score was computed as the child’s reported DISC anxiety standardized using the child report mean and standard deviation (z score) minus the parents’ report of the child’s anxiety standardized using the parent report mean and standard deviation (z score).

Results

The correlation between parent and child reports on the DISC-PS anxiety scale was (r = .29, p < .001, n = 478). We next examined the distribution of DIZ scores. As shown in Figure 1, the distribution of DIZ scores was normally distributed and this illustration helps to show that there are a relatively few parent child pairs at either end of the distribution who are showing very wide disagreement in reporting the child’s anxiety levels. Kappa coefficients for agreement between parent and child reported diagnoses were: Social Anxiety Disorder (k = .11), Generalized Anxiety Disorder (k = .00), Separation Anxiety Disorder (k = .15), Specific Phobia (k = .20), Panic (k = .18), Obsessive Compulsive Disorder (k = −.01), and Agoraphobia (k = .16).² For comparison, Kappa’s for Conduct disorder, Attention Deficit Disorder and Oppositional Defiant Disorder were .11, .05 and .07, respectively.

Figure 1.

Distribution of Difference in Z (DIZ) Scores

We next examined if the low correlations and kappas ware primarily a function of the small number of parent-child dyads who are showing large discrepancies in reporting (Figure 1). Drawing from the literature on robust statistical methods (e.g., Erceg-Hern & Mirosevich, 2008 argues cogently for the use of trimmed means) we next examined the correlation between the parent and child report trimming DIZ score outliers. Specifically, if the most extreme DIZ score cases are eliminated using a 10% trimmed mean of discrepancy scores, the correlation between the child and parent report on the DISC-PS is r = .66, p < .001 (n = 384). Kappa coefficients for agreement between parent and child reported diagnoses in this trimmed subsample were: Social Anxiety Disorder (k = .16), Generalized Anxiety Disorder (k = .02), Separation Anxiety Disorder (k = .23), Specific Phobia (k = .23), Panic (k = .26), Obsessive Compulsive Disorder (could not be calculated no child reported OCD in this subsample), and Agoraphobia (k = .40)³.

Discussion

This study adds to the existing research on the assessment of anxiety disorder symptoms in both clinically and theoretically important ways. Theoretically, the DISC-PS uses items from the DISC that are most salient/predictive from the standpoint of the reporter. Results suggested similar levels of agreement to measures that use parallel reports. Thus, while the association wasn’t larger than previous studies the association was similar to studies that used the exact same items to index anxiety. Moreover, the parent child kappa for the diagnoses and the symptom correlation were similar to findings on the full version of the DISC (Grills & Ollendick, 2002). Clinically, and consistent with previous research we found that the correlation between the child-parent reports for the DISC-PS anxiety symptoms was fairly low (Weems et al. 1999; 2010). Moreover, the agreement (i.e., kappas) between parents and children in terms of probable diagnosis of the various anxiety disorders was low and so an important take home message for the assessment of anxiety disorders with the DISC-PS in clinic samples of substance using adolescents is that clinicians might reasonably expect parents and children to “disagree” but that the level of disagreement on the DISC-PS is similar to the full version. One practical suggestion in using the DISC-PS in clinical settings might be to expect a possible anxiety disorder (and conduct further assessment) if only one reporter meets the cut off for a probable diagnosis.

The findings at first glance are consistent with a large body of literature (e.g., Achenbach et al., 1987) showing fairly low correspondence. However, this low agreement was largely a function of a relatively small proportion of the sample. DIZ scores were normally distributed and there are a relatively few parent child pairs at either end of the distribution who are showing very wide disagreement in reporting the child’s anxiety levels. The correlation between the child and parent report was r = .66 when these cases are trimmed. Relative agreement may be more the “norm” than previously thought (see also Weems et al., 2010) if the instances of extreme disagreement are removed the estimates of agreement.

Despite the contributions that this investigation makes to the understanding of the assessment of anxiety in youth with the DISC-PS, the study is not without limitations. First and foremost, this study was not designed to examine reporter differences. Future research could include independent measures of anxious arousal. Research suggests that youth can be valid reporters and indeed sometimes better reporters of their own anxiety symptoms than parents given an independent measure of anxious arousal. For example, Weems, Zakem, Costa, Cannon and Watts (2005) found that child reports but not parent reports of the child anxiety disorder symptoms was related to fear provoked physiological responding (heart rate response to a scary stimulus). An examination of congruence between child reports and reports from parents to physiological responding (as in Weems et al., 2005) or to independent observer ratings of anxiety/fear would have been an interesting adjunct to this study and may be an important next step in this area of research. Finally, while we examined age, sex, and ethnicity on reporter discrepancies, these variables are proxies for the underlying mechanisms responsible for agreement. Research is needed which examines the reasons for disagreement among those of different ages, sex, and ethnicity.