Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2009 Nov 9.
Published in final edited form as: Dev Psychopathol. 2009 Summer;21(3):815–838. doi: 10.1017/S0954579409000443

Predictors of withdrawal: Possible precursors of avoidant personality disorder

NATALIE D EGGUM 1, NANCY EISENBERG 1, TRACY L SPINRAD 1, CARLOS VALIENTE 1, ALISON EDWARDS 1, ANNE S KUPFER 1, MARK REISER 1
PMCID: PMC2774890  NIHMSID: NIHMS148717  PMID: 19583885

Abstract

Relations of avoidant personality disorder (AvPD) with shyness and inhibition suggest that a precursor of AvPD is withdrawal. Using a sample of 4.5- to 7-year-olds studied four times, 2 years apart, four and three classes of children differing in trajectories of mother- and teacher-reported withdrawal, respectively, were identified. Mothers and teachers generally did not agree on children’s trajectories but the pattern of findings in the two contexts did not differ markedly. The mother-identified high and declining withdrawal class, in comparison with less withdrawn classes, and the teacher-identified high and declining class compared with low withdrawal classes, were associated with relatively high levels of anger and low levels of attentional control and resiliency. The mother-identified moderate and increasing withdrawal class was distinguished from less problematic withdrawal classes by higher anger, lower resiliency, and sometimes, lower attentional control. The teacher-identified low and increasing withdrawal class was distinguished from less problematic withdrawal classes by lower resiliency and lower attentional control. Findings are discussed in terms of the developmental precursors to social withdrawal and avoidant behavior.


Avoidant personality disorder (AvPD), an adulthood disorder, is defined as involving a pervasive pattern of social inhibition, feelings of inadequacy, and hypersensitivity to negative evaluation, beginning in early adulthood and present in a variety of contexts. The criteria for AvPD in the Diagnostic and Statistical Manual of Mental Disorders—Fourth Edition, Text Revision (DSM-IV-TR; American Psychiatric Association, 2000) include an unwillingness to get involved with people unless certain of being liked; preoccupation with criticism and social rejection; avoidance of occupational activities that involve significant interpersonal context because of fears of criticism, disapproval, or rejection; inhibition in new interpersonal situations because of feelings of inadequacy; unusual reluctance to take personal risks or engage in any new activities because they may prove embarrassing; and perceptions of the self as socially inept, personally unappealing, or inferior to others (Rettew, 2000). Because AvPD is an Axis II disorder in DSM-IV (American Psychiatric Association, 1994), it is not diagnosed until adulthood, although childhood avoidant disorders were included in DSM-III (American Psychiatric Association, 1980). In a randomly selected community sample of adults (Ekselius, Tillfors, Furmark, & Fredrikson, 2001), 6.6% were identified as having AvPD using DSM-IV criteria.

According to numerous studies, AvPD and social phobia (SP; generalized and specific subtype) overlap considerably (Boone et al., 1999; Marteinsdottir, Tillfors, Furmark, Anderberg, & Ekselius, 2003; Rettew, 2000). Indeed, some investigators have questioned whether the distinction between these disorders is justified (e.g., Chambless, Fydrich, & Rodebaugh, 2008). SP (an Axis I disorder) is characterized as “a marked and persistent fear of one or more social or performance situations in which the person is exposed to unfamiliar people or to possible scrutiny by others. The individual fears that he or she will act in a way (or show anxiety symptoms) that will be humiliating orembarrassing” (American Psychiatric Association, 2000, p. 456). Diagnostic criteria also include the presence of anxiety or panic attacks when exposed to feared social situations, avoidance of social or performance situations, and interference with normal functioning or experience of intense distress. For children to be diagnosed with SP, they must be capable of relationships with familiar people, display anxiety with peers (not only with adults), and display symptoms for at least 6 months. Some investigators have argued that AvPD, compared to SP, is the more severe and encompassing disorder (Hummelen, Wilberg, Pedersen, & Karterud, 2007; Widiger, 1992), perhaps involving more anxiety, avoidance, depression, and fear of criticism (Rettew, 2000). Rettew (2000) suggested that AvDP is less likely than SP to involve somatic anxiety (e.g., a panic attack), and that people with AvPD fear being rejected orcriticized no matter what they do, whereas people with SP fear acting in ways that are embarrassing or humiliating. AvPD also differs from SP in that SP can be diagnosed prior to adulthood.

SP and AvPD have been related to each other. Joyce et al. (2003) found that the presence of an early-onset anxiety disorder, which included SP, significantly predicted AvPD. Results from a twin study utilizing Norwegian females have suggested that SP and AvPD may be influenced by the same genetic, but not the same environmental, factors (Reichborn-Kjennerud et al., 2007).

Predictors of AvPD and SP

The developmental psychopathology framework suggests that precursors of AvPD include childhood personality/temperament, that is, enduring dispositional characteristics, including those observed early and with some constitutional bases (Rothbart & Bates, 2006). Shiner and Caspi (2003; Caspi & Shiner, 2006), for example, argued that relations between early temperament/personality and the emergence of psychopathology are to be expected for several reasons: because psychopathology may be an extreme manifestation of personality/temperament or personality/temperament may put children at risk for the development of psychopathology, avert its development in the face of stress, and/or affect the presentation and course of psychopathology (in addition, they suggested that psychopathology may alter personality functioning). Geiger and Crick (2001) discussed similar mechanisms related to vulnerability to personality disorders and suggested, for example, that personality rigidity may be a risk factor for AvPD because individuals do not take personal risks or engage in new activities that might prove embarrassing. They also suggested that hypersensitivity, fearfulness of novelty, and an overly active behavioral inhibition system may confer risk for later disorders, as might poor emotion regulation skills. Thus, shy or inhibited children may experience aversive social interactions, and hence, develop attributions/cognitions that put them at risk for AvPD. Such arguments are consistent with the call for the DSM criteria to be more dimensional and hence to relate better to personality/temperament (Watson & Clark, 2006).

The antecedents of AvPD have seldom been examined with prospective longitudinal research, although there is a modest amount of research on the childhood antecedents of SP. Investigators have argued that AvPD has roots in, or at least is predicted by, childhood temperamental characteristics such as shyness and behavioral inhibition (Geiger & Crick, 2001; Rettew, 2000) and severe shyness also appears to overlap with SP (Heiser, Turner, & Beidel, 2003). Behavioral inhibition in childhood is a pattern of behavior in response to unfamiliar and/or challenging situations, which often includes anxiety, distress, disorganization, or avoidance (Kagan & Fox, 2006; Rubin, Hastings, Stewart, Henderson, & Chen, 1997). Kagan and Fox (2006) noted that “An inhibited temperament assumes that a child can display an avoidant style in any one of a number of contexts” (p. 198). Shyness is defined by withdrawn, timid behavior in social contexts, especially with unfamiliar people. SP has been hypothesized to be related to childhood passive withdrawal, which includes shyness, and behavioral inhibition (Neal & Edelmann, 2003). Rettew (2000) reviewed relevant research and argued that shyness, generalized SP, and AvPD exist along a continuum.

Empirical findings support an association between early shyness or inhibition and later SP or AvPD. For example, Biederman and colleagues (1990) initially did not find that the rate of avoidant disorder (DSM-III) was significantly higher in behaviorally inhibited children versus children who were not classified as inhibited (using two samples combined, one at-risk sample and one including behaviorally inhibited and uninhibited children). However, in a 3-year follow-up, behaviorally inhibited children had significantly higher rates of avoidant disorder when compared to uninhibited children and higher rates of the emergence of avoidant disorder (i.e., the disorder was not present at baseline but was present at follow-up; Biederman et al., 1993). Schwartz, Snidman, and Kagan (1999) utilized some of the same study participants selected for degree of inhibition and a separate community cohort and classified children as inhibited or uninhibited at 2 year of age. At 13 years of age, more previously inhibited than uninhibited children were classified as having generalized social anxiety disorder.

Moreover, Biederman et al. (2001) studied 21-month-olds, 4-year-olds, and 6-year-olds whose parents were diagnosed with panic disorder with or without comorbid major depression, major depression without comorbid panic disorder or agoraphobia, or no disorder. Behaviorally inhibited children ages 5 years or older had a significantly higher rate of “social anxiety disorder” (a category comprising individuals with avoidant disorder [DSM-III-R; American Psychiatric Association, 1987] or SP [DSM-IV]) than children who were not inhibited. This association occurred primarily for children of parents with panic disorder (with or without depression); however, the interaction between parental diagnosis and children’s inhibition class membership was not significant. In a follow-up of that sample, behaviorally inhibited children were more likely than children who were not inhibited to manifest avoidant disorder or SP and to exhibit a new onset of avoidant disorder or of SP at the follow-up (Hirshfeld-Becker et al., 2007; contrast with Caspi, Moffitt, Newman, & Silva, 1996, using a normative sample). Furthermore, in a number of studies, shyness in adulthood has been related to AvPD (Joyce et al., 2003, for depressed patients; Marteinsdottir et al., 2003) or SP (Chavira, Stein, & Malcarne, 2002; Heiser et al., 2003; Marteinsdottir et al., 2003), and retrospectively reported histories of shyness (Stemberger, Turner, Beidel, & Calhoun, 1995) or behavioral inhibition (Mick & Telch, 1998; see Rettew, 2000, for a review) have also been associated with SP. These results suggest that behavioral inhibition and shyness, which often involve withdrawal and avoidance, are developmental precursors to disorders such as AvPD and SP. Moreover, findings with inhibition in children suggest that it is important to look at relatively extreme inhibition to observe prediction across time (e.g., Asendorpf, Denissen, & van Aken, 2008; Kagan, Reznick, & Gibbons, 1989).

Thus, based on both conceptual and empirical reasons, there is good reason to expect childhood avoidance and/or withdrawal in childhood to predict AvPD. Given the role of inhibition, shyness, and negative emotionality in AvPD, it is quite plausible that a persistent pattern of withdrawal as assessed with the withdrawal scale of the Child Behavior Checklist (CBCL; Achenbach, 1991a) across childhood predicts AvPD, albeit not for all individuals. The CBCL withdrawal scale contains items such as “refuses to talk,” “stares blankly,” “unhappy, sad, depressed,” and “withdrawn, doesn’t get involved with others” (Achenbach & Edelbrock, 1981). Thus, adults’ ratings of children on the CBCL Withdrawal Scale are likely to reflect the timid, avoidant behavior, negative emotionality, and social anxiety that are characteristic of AvPD.

A complication in considering withdrawal as a predictor of AvPD is that social withdrawal, which is one aspect of avoidance, can be due to factors other than inhibition or shyness, such as the simple desire to be alone or rejection by peers (Asendorpf, 1990; Coplan & Armer, 2007; Rubin, Bukowski, & Parker, 2006). However, the child who is withdrawn because of rejection in the early school years is likely aggressive rather than shy (Rubin et al., 2006). Withdrawn social behavior does not appear to be consistently associated with peer rejection until about age 10 (Rubin & Mills, 1988; Younger, Gentile, & Burgess, 1993; however, compare with Coplan, Prakash, O’Neil, & Armer, 2004). Moreover, there is some evidence that socially disinterested children (who are low in the desire to play with others) do not seem to have the problems with negative emotionality that are associated with shyness (Coplan et al., 2004) and which are tapped to a limited degree in the CBCL Withdrawal Scale. Those children who are avoidant of social interaction because of high behavioral inhibition and low approach behavior tend to be maladapted and prone to depressive symptoms (Coplan, Wilson, Frohlick, & Zelenski, 2006) and might be relatively prone to AvPD.

Thus, in the present study, we assumed that a persistent pattern of withdrawal in childhood is relatively likely to predict symptoms associated with AvPD, at least for many children. Withdrawal was assessed with teachers’ and mothers’ reports on their respective behavior checklist’s (Achenbach, 1991a) withdrawal subscale, which includes items tapping social withdrawal, as well as items that might be associated with more general withdrawal. We identified patterns of withdrawal trajectories across four assessments in childhood and examined prediction of these patterns from measures of children’s negative emotionality (sadness, fear, and anger), impulsivity, attentional control, and ego resiliency at the first assessment. The logic for our selection of predictors is discussed below. Given the relative dearth of research on AvPD, research on correlates and antecedents of SP as well as AvPD (or sometimes combined) is reviewed.

Trajectories of Withdrawn or Shy Behavior

As a first step in examining the relations of withdrawn behavior to temperament/personality, we examined the trajectories in children’s withdrawn behavior. Two sets of researchers recently have delineated similar trajectories. Booth-Laforce and Oxford (2008) examined trajectories of children’s teacher-reported withdrawal. Three latent classes of teacher-reported withdrawal trajectories from Grade 1 to Grade 6 were identified: a decreasing class (5% of the sample) in which children had initially high but declining withdrawal scores, an increasing class (9% of the sample) in which children had initially low but rising scores, and a normative class (86% of the sample) in which children had the lowest levels of initial withdrawal which remained low. Although they did not identify a stably high withdrawn group in the trajectory modeling analyses, they did identify about 7% of children who were consistently 1 SD above the mean in withdrawn behavior. These children were characterized by poor inhibitory control and demographic risks, but not shyness at the earliest age (likely because many of the children were in the increasing withdrawal group).

Oh, Rubin, Bowker, Booth-Laforce, and Rose-Krasnor (2008) examined trajectories of peer-nominated social withdrawal from fifth to eighth grade. Efforts were made to identify children who were socially withdrawn because of internal (e.g., shyness or social disinterest) but not external (e.g., peer rejection) reasons, and hence, that reflected the type of withdrawal inherent in SP and AvPD (although social disinterest probably would not be involved in SP or AvPD). Three trajectories were identified: a decreasing class (8% of the sample) with initially high but declining social withdrawal scores, an increasing class (7% of the sample) with initially low but rising scores, and a low-stable class (85% of the sample) with the lowest levels of initial social withdrawal that remained low over time. Peer exclusion, friendlessness, and instability of friendships were related to the increasing and/or declining trajectories, suggesting that peer processes may have affected peers’ reports of withdrawal.

In the present study, trajectories for teacher-and mother-reported withdrawal were examined for school-aged children. We hoped to identify a small class of high withdrawn children (who possibly declined or increased somewhat) who would be expected to be particularly vulnerable to developing AvPD. However, based on the two aforementioned studies, we were unsure if such a class would be identified.

Negative Emotionality

There are numerous empirical studies linking SP or AvPD to negative emotionality (Hofmann, Heinrichs, & Moscovitch, 2004). At a global level, in studies with adults, investigators have found associations of personality neuroticism with AvPD (Hummelen et al., 2007; Meyer, 2002) and/or SP (Brown, 2007; Hummelen et al., 2007; Stemberger et al., 1995; van Velzen, Emmelkamp, & Scholing, 2000). Moreover, negative emotion is elevated in shy (Coplan et al., 2004; Eisenberg, Shepard, Fabes, Murphy, & Guthrie, 1998), behaviorally inhibited (Coplan et al., 2006; Kagan, 1998), and socially phobic (Beidel, Turner, & Morris, 1999) children. In addition, researchers sometimes have described individuals with AvPD as having a low tolerance for negative emotion or sensory stimulation (e.g., responding strongly to various stimuli; Meyer, Ajchenbrenner, & Bowles, 2005).

Fear is one of the negative emotions most expected to be associated with AvPD and SP (see Hofmann et al., 2004). In relevant research, the fears examined usually have been specific social fears, such as talking in front of others or social interactions with others (Hofmann et al., 2004). Few investigators have examined AvPD or SP in relation to a broad array of fears, especially the fears that are typical in childhood, although Beidel et al. (1999) found that third through eighth graders with SP were fearful across a range of situations. It has been argued that a large percent of people who suffer from fears and inhibition in many social situations also are timid in other aspects of their lives and, consequently, that individuals with AvPD are likely to be fearful and inhibited more generally (Rettew, 2000; see Holt, Heimberg, & Hope, 1992).

Moreover, in studies with children, fear or timidity has been linked to internalizing problems, including withdrawal (e.g., Eisenberg et al., 2005; Oldehinkel, Hartman, Ferdinand, Verhulst, & Ormel, 2007; Oldehinkel, Hartman, Winter, Veenstra, & Ormel, 2004), shyness (Coplan et al., 2004), and behavioral inhibition (Kagan & Fox, 2006; Rubin et al., 1997; also in early adulthood, Mick & Telch, 1998). These findings suggest that early fearfulness may underlie avoidant behaviors in childhood/adolescence and AvPD in adulthood.

Few investigators have examined the relation of sadness to AvPD, although sadness has been associated with withdrawn behavior in early childhood (e.g., Eisenberg et al., 2001). Moreover, despite the fact that experts tend not to identify depression as a prototypic characteristic of AvPD (Warner et al., 2004), depression or depressive symptoms have been associated with AvPD and SP (Beidel et al., 1999; Boone et al., 1999; Hummelen et al., 2007; Johnson & Lydiard, 1995; Meyer, 2002; van Velzen et al., 2000). It is unclear if depressed affect is a cause of avoidant behavior or an outcome of it, but in either case, one might expect it to be associated over time with an increased probability of avoidance and feelings of inadequacy.

Warner et al. (2004) found that experts also did not report that anger is a prototypic trait of people with AvPD; however, investigators have found positive associations between anger and AvPD or features of AvPD (Meyer, 2002) or SP (Erwin, Heimberg, Schneier, & Liebowitz, 2003; see Hofmann et al., 2004; Moscovitch, McCabe, Antony, Rocca, & Swinson, 2008). In contrast, Hummelen et al. (2007) found that compared to norms for the measure of anger, people with AvPD were not particularly high in self-reported anger. In a review, Hofmann et al. (2004) suggested that some individuals with SP show behaviors that are angry or hostile in social situations, whereas others are submissive. Some investigators have argued that the association between anger and social anxiety may be due to feelings of being shamed, which result in anger (Gilbert, 1998; Tangney, Wagner, & Gramzow, 1992).

Young children (age 4.5–7) with relatively high levels of social withdrawal seem to be only slightly prone to anger (Eisenberg et al., 2001), but the association between anger/frustration and internalizing problems, including withdrawal, seems to be more evident with age in childhood (Eisenberg et al., 2005; see also Lengua, 2006; Oldehinkel et al., 2004, 2007) and in cultures such as China (Eisenberg et al., 2007). The relation between children’s internalizing symptoms and dispositional anger is consistent with an expectation that withdrawn children and adults with severe problems with avoidance may often be prone to anger and hostility, although it is unclear if anger predisposes individuals to avoidant problems or if anger and frustration are an outcome of negative interactions and experiences because of avoidance.

Impulsivity and Attentional Regulation

Experts report that a prototypic trait of people with AvPD is low impulsivity (Warner et al., 2004). Consistent with this idea, investigators have found that people with AvPD are high in harm avoidance (Griego, Stewart, & Coolidge, 1999; Joyce et al., 2003; Marteinsdottir et al., 2003). Moreover, Meyer (2002) found that features of AvPD in college students were associated with low reward responsiveness, a construct reflecting low impulsivity (Pickering & Gray, 1999), as well as high scores on a measure of the behavioral inhibition system (behavioral inhibition) and low fun seeking. Children with social withdrawal, inhibition, or internalizing symptoms also tend to be quite low in impulsivity, especially if they do not also have externalizing problems (Caspi et al., 2003; Eisenberg et al., 2001, 2005, 2007; compare with Lengua, West, & Sandler, 1998). Thus, it is plausible to expect low impulsivity, which is somewhat stable in childhood and into adolescence (Murphy, Eisenberg, Fabes, Shepard, & Guthrie, 1999; Valiente et al., 2003), to be a precursor of AvPD.

The abilities to shift and/or focus attention as needed to regulate emotion and behavior (Rothbart & Bates, 2006) generally have been negatively related to impulsivity (e.g., Eisenberg et al., 2004; Valiente et al., 2003). Moreover, similar to impulsivity, problems in effectively managing attention have been linked to shyness (Coplan et al., 2004; Eisenberg, Fabes, & Murphy, 1995), although the findings sometimes have differed for teachers’ and parents’ reports of children’s shyness (Eisenberg et al., 1998). Self-regulation (including attentional) problems also have been associated with solitary play in preschool (Spinrad et al., 2004). The abilities to move attention from negative thoughts and to focus on affectively neutral or positive thoughts and activities seem to be important in curtailing negative emotion and have been linked to low levels of anger, anxiety, and depression (Derryberry & Reed, 2002; Derryberry & Rothbart, 1988; Eisenberg et al., 2007; Silk, Steinberg, & Morris, 2003). Given that attentional control (e.g., distraction) has been related to effective regulation/coping with negative emotion in social interactions (Eisenberg, Fabes, Nyman, Bernzweig, & Pinuelas, 1994) as well as low levels of maladjustment (e.g., Eisenberg et al., 2004; Kochanska & Knaack, 2003; Lengua et al., 1998), it seems likely that people with AvPD, who tend to have problems with modulating their evaluative concerns and negative emotionality, are low in attentional regulation.

Ego Resiliency

Ego resiliency (i.e., personality resiliency) has been defined by Block and Block (1980) as “the dynamic capacity of an individual to modify his/her modal level of ego-control, in either direction, as a function of the demand characteristics of the environmental context” (p. 48). High ego resilience involves resourceful adaptation to changing circumstances and flexible use of problem-solving strategies; low resilience is characterized by little adaptive flexibility, an inability to respond to changing circumstances, the tendency to perseverate or become disorganized when dealing with change or stress, and difficulty recouping after traumatic or stressful experiences.

By definition, AvPD involves feelings of inadequacy and self-rated AvPD features have been linked to low self-esteem, low optimism, and high levels of pessimism (Meyer, 2002; Meyer & Carver, 2000). A pessimistic rather than optimistic perspective would be expected to hinder recovery from negative emotionality and adaptive coping. Given individuals with AvPD’s proneness to negative emotionality and the likelihood that they have relatively limited access to a broad supportive social network (given their avoidance), one would expect them to be low in ego resiliency relative to peers without symptoms of AvPD. Consistent with this argument, withdrawn children have problems with negative emotionality (e.g., Eisenberg et al., 2001) and children with internalizing problems, including withdrawn behavior, are low in ego resiliency (Eisenberg, Chang, Ma, & Huang, 2009; Eisenberg et al., 2004; Martel et al., 2007). Moreover, as previously mentioned, Geiger and Crick (2001) suggested that personality rigidity would be expected to contribute to children’s reluctance to engage in novel situations/interactions or take personal risks, which would make them vulnerable to AvPD. Children with chronic symptoms of withdrawal, as well as adults with AvPD, likely do not have the social and emotional resources to easily rebound from stress or to deal with the negative consequences of their avoidant behavior.

The Present Study

In the present study, we identified patterns of mother- and teacher-reported child withdrawal (or the lack thereof) across four assessments (Time 1 to Time 4 [T1–T4]), 2 years apart, starting at age 4.5–7 (to about 10.5 to turning 13 years). After identifying trajectories, we examined if these trajectories were related to individual differences in temperament/personality at T1. Based on both the literature regarding AvPD/SP and childhood withdrawal, shyness, and internalizing problems, we expected children who were stably withdrawn or at least the most withdrawn across time (perhaps increasing or declining somewhat), in comparison to less withdrawn children, to be prone to fear, anger, and sadness, as well as low impulsivity, attentional control, and ego resiliency. We expected teacher- and mother-designated withdrawn classes to be similar in regard to their relations with attentional regulation, impulsivity, and ego resiliency (especially as reported in the same context). However, lack of agreement or only modest agreement about children’s negative emotionality and internalizing problems is common (Achenbach, McConaughy, & Howell, 1987; Goldsmith, Rieser-Danner, & Briggs, 1991; Stanger & Lewis, 1993; Verhulst & Akkerhuis, 1989), and it seemed likely that teachers would be less aware than parents of the negative emotions associated with stable withdrawal (because they seem less attuned to internalizing emotions more generally; Verhulst & Akkerhuis, 1989; Youngstrom, Loeber, & Stouthamer-Loeber, 2000). Thus, it was expected that mother-and teacher-reported trajectories of withdrawn behavior would differ somewhat, as might their relations with children’s negative emotionality (and perhaps some other dispositional variables).

Methods

Participants

Participants were involved in a longitudinal study (e.g., Eisenberg et al., 2001); the first four time points (T1, T2, T3, and T4) were used in this study. Families were recruited through schools, newspaper ads, and flyers that were placed at after-school programs and preschools in a large suburban area (mostly schools; Eisenberg et al., 2001). A primary goal of the study was to recruit a sample of children in which problem behavior was well represented (without using a clinical sample). Therefore, the CBCL (Achenbach, 1991a, 1991b) was administered over the phone to 315 parents, most often mothers, who had expressed interest in participation. Children with T scores of 60 or higher, indicating moderate to high risk (Achenbach, 1991a, p. 81, 1991b, p. 63) on either the internalizing or externalizing scale (or both), were selected for participation. Children who had T scores below 60 on both internalizing and externalizing were considered control children. For selection purposes, children with co-occurring adjustment problems (children with scores above 60 for both internalizing and externalizing scales) were considered to belong to the group of the scale on which the child scored higher. Control children, children with internalizing problems, and children with externalizing problems were matched as closely as possible in terms of age, sex, race, and social class (e.g., parental education and occupation).

The selection process resulted in an initial (T1) sample of 214 children (118 boys; M age = 73.48 months, SD = 9.59, range = 55–97 months). The child and a primary caregiver, usually the mother, came into the laboratory. For data regarding internalizing problems (including withdrawal), 209 mothers completed questionnaires and teachers completed questionnaires for 194 children. For data regarding temperament and ego resiliency, primary caregivers (203 mothers, 11 fathers) completed questionnaires and teachers completed questionnaires for 195 children. Most of the children were non-His-panic Caucasians (71.0%). Hispanics (15.9%), Native Americans (4.2%), African Americans (3.7%), Asians (0.9%), and children of mixed origin (4.2%) were represented. Percentages for race/ethnicity slightly differ from previous reports because of obtaining previously missing data at follow-up visits. Mothers’ and fathers’ education was reported by the participating parent (1 = less than high school education, 2 = high school diploma/equivalent, 3 = some college, 4 = 2-year degree/trade school, 5 = college degree, 6 = professional degree). On average, mothers and fathers reported some college education (Ms =3.54 and 3.51, SDs =1.58 and 1.70). Annual family income ranged from $6,000 to $160,000 (Mdn = $35,000).

Approximately 2, 4, and 6 years later (T2, T3, and T4), 193, 185, and 167 children (105, 103, and 91 boys; M ages = 91.92, 121.41, and 133.35 months; SDs = 9.96, 9.47, and 9.43 months) and a primary caregiver participated in follow-up assessments. Children were rated on internalizing (including withdrawal) by mothers (ns = 192, 174, and 165) and teachers (ns = 180, 163, and 145). Primary caregivers (ns = 174, 158, and 158 mothers; 8, 7, and 4 fathers; and 3, 2, and 4 grandmothers) completed questionnaires on temperament and ego resiliency and teachers also completed temperament and ego resiliency questionnaires for 180, 164, and 150 children. On average, mothers and fathers reported some college education (Ms =3.80, 3.86, and 3.93, SDs =1.37, 1.42, and 1.41 for mothers; Ms =3.84, 3.82, and 3.89, SDs =1.59, 1.62, and 1.69 for fathers). T2 and T3 family income ranged from <$20,000 to >$100,000 (Mdn = $40,000–$60,000 at T2 and $60,000–$80,000 at T3 and T4).

Attrition

Twenty-one (9.8%), 29 (13.6%), and 47 (22.0%) participants were lost because of attrition from T1 to T2 (2 years), T1 to T3 (4 years), and T1 to T4 (6 years), respectively. We used t tests and Pearson chi-square tests to examine differences between nonattrited families and families who dropped out of the study by T4. In terms of demographic variables, participants attrited from T1 to T4 differed in race when compared to nonattrited participants, χ2 (5) = 15.88, p < .01. Five cells had expected counts less than 5; thus, validity of the Pearson chi-square results may be compromised and additional details are reported. Approximately 67% (n = 6) of Native American, 50% (n = 4) of African American, 21% (n = 7) of Hispanic, and 18% (n = 28) of non-Hispanic Caucasian, and 22% (n = 2) of mixed-origin children who participated at T1 did not participate at T4. Families attrited from T1 to T4 had significantly lower annual income at T1 in comparison to nonattrited families, t (192) = 2.30, p < .05 (Ms = 32.72 for attrited and 43.03 for nonattrited families). Fathers of families attrited from T1 to T4 had significantly lower levels of education than fathers of nonattrited families, t (204) = 2.77, p < .01 (Ms = 2.91 for attrited and 3.69 for nonattrited families). In terms of study variables, children of families attrited from T1 to T4 were rated by mothers as significantly higher in T1 withdrawal than children of nonattrited families, t (207) =−2.22, p < .05 (Ms = 60.97 for attrited and 57.93 for nonattrited families). No other study variables differed between attrited and nonattrited families.

Procedure

Consent was obtained when the participant and his/her parent arrived at the laboratory or through the mail. The child’s parent completed questionnaires while the child participated in a series of tasks unrelated to the present study. Participants were debriefed, compensated, and asked for permission to collect questionnaire data from the child’s teacher. Toward the end of the semester in which the participant visited the laboratory, questionnaires were sent to the child’s teacher. Questionnaires were sent to families who moved from the area.

Measures

Withdrawal

At each time point, adults reported on children’s withdrawal problems using a sub-scale on the CBCL (mothers) or the Teacher Report Form (TRF; Achenbach, 1991a). Three items from the original scale were dropped because of expert raters determining that the items overlapped with measures of temperament (i.e., Would rather be alone than with others, Shy or timid, Underactive, slow moving or lacks energy; see Eisenberg et al., 2004). Mothers and teachers rated six items (e.g., Withdrawn, doesn’t get involved with others; refuses to talk) as 0 = not true, 1 = sometimes true, and 2 = very true (αs = .64, .63, .60, and .65 for mothers’ reports and .75, .82, .69, and .75 for teachers’ reports at T1, T2, T3, and T4, respectively). Children’s withdrawal scores were computed by averaging responses within reporter.

Anger/frustration

At T1, adults rated children’s anger/frustration (1 = extremely untrue to 7 = extremely true) on the Child Behavior Questionnaire (CBQ; Rothbart, Ahadi, Hershey, & Fisher, 2001) anger/frustration scale. Three items deemed inappropriate for teachers were dropped from the teacher-rated scale (e.g., Gets irritable about having to eat food (s)he doesn’t like). Anger/frustration items (e.g., Gets mad when provoked by others) were averaged (parents’ and teachers’ report αs = .84 [13 items] and .91 [10 items], respectively).

Sadness

At T1, adults rated children’s sadness (1 = extremely untrue to 7 = extremely true) on the CBQ Sadness Scale. Five items were omitted from the teacher-rated version (e.g., Becomes upset when loved relatives or friends are getting ready to leave, Does not usually become tearful when tired [reversed]) as they appeared to be inappropriate for teachers. In addition, three items were removed from the parent and teacher-reported sadness scale because experts rated them as overlapping with internalizing problems assessed by the CBCL/TRF (e.g., Seems to feel depressed when unable to accomplish some task; see Eisenberg et al., 2004). Sadness items (10 for parents and 5 for teachers; e.g., Becomes sad when told to do something s/he does not want to do) were averaged (αs = .75 and .73 for parents’ and teachers’ reports, respectively).

Fearfulness

At T1, parents rated children’s fearfulness (1 = extremely untrue to 7 = extremely true) on the fear scale of the CBQ. One of the original items, Is very frightened by nightmares, was removed from the scale as expert raters judged the item as confounded with internalizing problems assessed by the CBCL/TRF (for details, see Eisenberg et al., 2004). The remaining 12 items (e.g., Is afraid of the dark) were averaged to form composites of parent-reported fear (αs = .75).

Impulsivity

At T1, adults rated (1 = never to 7 = always) children’s impulsivity on the CBQ. One item, When eager to go outside, sometimes rushes out without putting on the right clothes, was dropped from the teacher-reported scale as we felt teachers would not have seen children in this situation. Thirteen (parents’ reports) or 12 (teachers’ reports) items, for example, Usually rushes into an activity without thinking about it, were averaged to form parent- and teacher-reported composites (αs = .81 and .89 for parents’ and teachers’ reports, respectively).

Attentional control

At T1, adults rated (1 = extremely untrue to 7 =extremely true) children’s attentional control on the attention focusing and attention shifting subscales of the CBQ. Eleven items (e.g., Can easily shift from one activity to another) were averaged to form composites of parent- and teacher-reported attention shifting (αs = .82 and .88 for parents’ and teachers’ reports, respectively). One item was dropped from the attention focusing scale as it substantially decreased the alpha (Has difficulty leaving a project (s)he has begun). In addition, one item, When watching TV, is easily distracted by other noises or movements, was not included on the teacher-reported attention focusing scale because teachers often would not have seen the children in the situation. Nine (parents’ reports) or eight (teachers’ reports) items, for example, When practicing an activity, has a hard time keeping her/his mind on it, were averaged to compute parent- and teacher-reported attention focusing (αs = .74 and .85 for parents’ and teachers’ reports, respectively). For both parents’ and teachers’ reports, attention shifting and attention focusing were correlated, rs (206, 193) =.36 and .62, ps < .01. Thus, attention shifting and focusing were averaged (αs for parents’ and teachers’ reports = .83 and .91, respectively).

Ego resiliency

At T1, parents and teachers rated (1 = most undescriptive to 9 = most descriptive) children’s ego resiliency on an adapted questionnaire version of Block’s Resiliency Q-Sort (Block & Block, 1980). Seven items rated by experts as most clearly indicative of resiliency (e.g., Can bounce back or recover after a stressful or bad experience; see Cumberland-Li, Eisenberg, & Reiser, 2004) were averaged within reporter to form composites of parent- and teacher-reported ego resiliency (αs = .68 and .81, respectively).

Results

Descriptive analyses

Data from T1 to T4 were used to compute withdrawal growth trajectories and to determine latent classes of trajectories (see below). Data on predictors of the latent classes were taken from T1 (see Table 1 for means and standard deviations within each withdrawal class).

Table 1.

Withdrawal and predictor means and standard deviations within each withdrawal class

M LD
M HD
M MS
M MI
T LS
T HD
T LI
Variable M SD M SD M SD M SD M SD M SD M SD
M with. T1 0.24 0.22 1.20 0.28 0.43 0.27 0.69 0.31 0.38 0.34 0.52 0.47 0.25 0.23
T with. T1 0.16 0.26 0.50 0.53 0.21 0.33 0.26 0.27 0.11 0.17 1.01 0.31 0.27 0.26
M with. T2 0.15 0.17 0.85 0.40 0.45 0.27 0.65 0.21 0.28 0.29 0.30 0.28 0.44 0.43
T with. T2 0.22 0.32 0.30 0.32 0.27 0.43 0.44 0.67 0.18 0.30 0.54 0.39 0.63 0.57
M with. T3 0.17 0.19 0.63 0.33 0.49 0.26 0.74 0.46 0.31 0.31 0.25 0.22 0.40 0.29
T with. T3 0.22 0.29 0.10 0.20 0.26 0.32 0.52 0.46 0.18 0.23 0.40 0.44 0.61 0.50
M with. T4 0.08 0.11 0.38 0.12 0.49 0.16 1.04 0.22 0.25 0.29 0.40 0.28 0.33 0.28
T with. T4 0.17 0.26 0.15 0.27 0.26 0.34 0.15 0.23 0.11 0.16 0.15 0.30 0.78 0.25
P fear 3.90 0.88 4.18 0.99 4.20 0.96 4.36 0.91 4.00 0.93 4.29 0.52 4.01 1.10
P sad 4.31 0.72 4.83 0.86 4.46 0.75 4.89 0.99 4.39 0.77 4.64 0.49 4.28 0.77
T sad 3.75 1.14 4.56 1.46 3.83 0.95 4.86 0.61 3.75 1.07 4.60 1.06 4.24 1.31
P anger 4.49 0.86 5.15 0.73 4.96 0.94 5.60 0.65 4.71 0.94 4.89 0.55 4.64 0.93
T anger 3.58 1.35 4.55 1.43 3.35 1.14 4.37 0.80 3.43 1.22 4.56 1.46 4.26 1.41
P impuls. 4.63 0.82 4.44 0.92 4.42 0.84 4.34 0.92 4.55 0.85 4.63 0.79 4.47 0.80
T impuls. 4.16 1.12 4.45 1.43 3.97 1.18 3.90 1.23 4.13 1.14 3.65 1.34 4.36 1.15
P att. cont. 4.46 0.69 3.92 0.72 4.25 0.74 3.79 0.37 4.35 0.72 4.00 0.49 4.46 0.82
T att. cont. 4.97 1.01 4.29 0.76 4.75 1.10 3.80 0.91 4.96 1.01 4.03 0.82 4.31 1.14
P resil. 6.43 1.08 5.18 1.92 5.88 1.27 5.18 0.52 6.13 1.27 5.94 0.95 6.42 1.27
T resil. 6.70 1.29 5.77 1.42 6.47 1.45 6.18 1.29 6.79 1.27 5.08 1.18 5.82 1.23

Note: M, mothers’ reports; LD, low and declining; HD, high and declining; MS, moderate and stable; MI, moderate and increasing; T, teachers’ reports; LS, low and stable; LI, low and increasing; with., withdrawal; T1–T4, Time 1–Time 4; P, parents’ reports; impuls., impulsivity; att. cont., attentional control; resil., resiliency.

Relations of measures

T1 measures of temperament and ego resiliency were used as predictors of withdrawal trajectory latent class membership. Correlations were used to examine consistency of reports of predictors across reporter. The correlations between mothers’ and teachers’ reports of children’s sadness, anger, impulsivity, attentional control, and ego resiliency were rs (184, 187, 189, 190, 189) = .10, .25, .47, .42, and .13, ps = ns, < .01, < .01, < .01, and .07, respectively.

At each time point, correlations between mothers’ and teachers’ reports of children’s withdrawal were calculated, rs (187, 177, 152, 136) = .18, .05, .13, and .07, ps = .01, ns, ns, and ns at T1 to T4, respectively. After T1, there was a lack of agreement between mothers and teachers regarding children’s degree of withdrawal within time.

Withdrawal trajectory latent classes

The number of latent classes of trajectories for mother- and for teacher-reported withdrawal was established with modeling in Mplus Version 5.1 (Muthén & Muthén, 1998–2007). Missing data were handled by using full information maximum likelihood (FIML) in which “the discrepancy function maximizes the sum of N casewise contributions to the likelihood function that measure the discrepancy between the observed data and current parameter estimates using all available data for a given case” (Enders & Peugh, 2004, p. 2). FIML assumes data are missing at random (missing values on a measured variable are not related to the underlying values of the variable, but are related to other measured variables). This assumption is less stringent than the missing completely at random assumption (missing values on a variable are independent of values on that variable and on other observed variables; Enders & Peugh, 2004) and was more likely to be met by the data.

The approach to examining withdrawal trajectories in the present study was latent class growth analyses (LCGA).1 Essentially, LCGA allows for latent classes of growth trajectories to be specified. Between-class variation in the trajectory is allowed (i.e., the average intercept and slope may differ across class), but within-class variation (i.e., the intercept variance and slope variance within class) is not estimated. The researcher determines the number of classes by specifying different numbers of latent classes and doing nested model testing with the Bayesian information criterion (BIC), considering class size, considering theory, and comparing other statistics reported in the output (e.g., posterior probabilities, entropy). LCGA models were computed separately for mothers’ reports and teachers’ reports of children’s withdrawal because there was little agreement between their reports. Rather than reporting incremental fit indices, the sample size-adjusted BIC is reported (see Widaman & Thompson, 2003). Models were evaluated primarily through nested model testing.

Linear and quadratic models were tested for one-, two-, three-, and four-class models for both mother-reported withdrawal and teacher-reported withdrawal. Linear models were set up so that the latent intercept factor was indicated by withdrawal at T1, T2, T3, and T4 with each factor loading set at 1.00. The latent linear slope factor was indicated by withdrawal at T1, T2, T3, and T4 with factor loadings set at 0.00, 1.00, 2.00, and 3.00, respectively. In the quadratic model, the setup was the same, but a latent quadratic slope factor was indicated by withdrawal at T1 to T4 with factor loadings set at 0.00, 1.00, 4.00, and 9.00, respectively. For all models, as is typically done, manifest withdrawal means were fixed at 0.00 for model identification, and latent intercept, linear slope, and quadratic slope (if applicable) means were estimated. For the two-, three-, and four-class quadratic models, the quadratic growth latent means were estimated within each class first. If the output suggested that the quadratic mean for a given class was non-significant, it was fixed at zero and the more parsimonious model was run.

Because of space limitations, some of the detail pertaining to models that were not used in later analyses is omitted. A comparison of the sample-size adjusted BICs is given first. Then a more detailed description and comparison of the better fitting models are given. In addition to comparison of BICs and other indices previously mentioned, significance of quadratic growth terms was considered when determining whether the linear or quadratic model was better.

Mother-reported withdrawal LCGAs

For the mother-reported withdrawal LCGAs, the following sample size-adjusted BICs were obtained: one-class linear model =384.42, one-class quadratic model = 386.34, two-class linear model = 245.03, two-class quadratic model = 249.17, three-class linear model = 195.76, three-class quadratic model = 201.37, and four-class linear model = 175.18. The four-class quadratic model could not be reliably estimated (i.e., the best log likelihood value was not replicated), perhaps because of the complexity of the model. Thus, the four-class linear model was considered superior to the four-class quadratic model. Based upon the comparisons of the sample size-adjusted BIC values and other indices, the four-class linear model was considered the best LCGA model for mother-reported withdrawal.

For the mother-reported four-class linear model, classification quality was adequate, as noted by the high entropy value (.80) and the high classification probabilities (.91, .88, .81, and .96 for Class 1, 2, 3, and 4, respectively). Class 1 (57.70% of the sample) included children whose mother-reported withdrawal was initially low (mean withdrawal at T1 =.23) and significantly decreased (mean linear slope = −.05, p < .01). Class 2 (6.80% of the sample) included children whose mother-reported withdrawal was initially high (mean withdrawal at T1 = 1.14) and significantly decreased (mean linear slope = −.26, p < .01). Class 3 (30.29% of the sample) included children whose withdrawal was initially moderate (mean withdrawal at T1 = .40) and remained stable (mean linear slope = −.03, ns). Class 4 (5.21% of the sample) included children whose withdrawal was initially moderate (mean withdrawal at T1 =.57) and significantly increased (mean linear slope = .14, p < .05).

Teacher-reported withdrawal LCGAs

For the teacher-reported withdrawal LCGAs, the following sample size-adjusted BICs were obtained: one-class linear model = 406.31, one-class quadratic model =404.87, two-class linear model = 306.62, two-class quadratic model = 297.00, and three-class linear model = 255.11. The three-class quadratic and four-class linear and quadratic models could not be reliably estimated (i.e., the best log likelihood value was not replicated), perhaps because of the complexity of the models. Although the one- and two-class quadratic models had lower sample size-adjusted BICs than the one- and two-class linear models, respectively, the three-class linear model had a much lower sample-size adjusted BIC than the two-class linear model.

Based upon the comparisons of the sample size-adjusted BIC values and convergence issues with the more complex models, the three-class linear model was considered the best model for teacher-reported withdrawal. Classification quality was adequate for the three-class model, as noted by the high entropy value (.86) and the classification probabilities (.95, .92, and .89 for Class 1, 2, and 3, respectively). Class 1 (79.70% of the sample) included children whose teacher-reported withdrawal was initially low (mean withdrawal at T1 = .13) and remained stable (mean linear slope = .00, ns). Class 2 (8.25% of the sample) included children whose teacher-reported withdrawal was initially high (mean withdrawal at T1 = .97) and significantly declined (mean linear slope = −.28, p < .01). Class 3 (12.04% of the sample) included children whose teacher-reported withdrawal was relatively initially low, but not as low as in Class 1 (mean withdrawal at T1 =.27), and significantly increased (mean linear slope = .17, p <.01).

Final withdrawal models

Once the number of classes and trajectory shapes were determined for mother- and teacher-reported withdrawal, the models were considered “final” and used for subsequent analyses. The final model for mother-reported withdrawal was the four-class linear LCGA. From this point forward the four classes are referred to as the low and declining (LD) class (initially low and significantly declining withdrawal over time), the high and declining (HD) class (initially high and significantly declining withdrawal over time), the moderate and stable (MS) class (initially MS withdrawal over time), and the moderate and increasing (MI) class (initially moderate and significantly increasing withdrawal over time). Estimated means within latent classes are plotted in Figure 1.

Figure 1.

Figure 1

Estimated means within the latent class for mother-reported withdrawal.

The final model for teacher-reported withdrawal was the three-class linear LCGA. From this point forward the three classes are referred to as the low and stable (LS) class (initially LS withdrawal over time), the HD class (initially high and significantly declining withdrawal over time),2 and the low and increasing (LI) class (initially low and significantly increasing over time). Estimated means within teacher-reported latent classes are plotted in Figure 2.

Figure 2.

Figure 2

Estimated means within the latent class for teacher-reported withdrawal.

Chi-square analyses were used to see if mothers’ and teachers’ withdrawal classifications differed by children’s sex. Differences were not significant for mother-reported withdrawal class, χ2 (3) = .30, ns, or for teacher-reported withdrawal class, χ2 (2) = .42, ns.

Prediction of withdrawal trajectory latent class membership

Mplus was used to compute multinomial logistic regressions to examine prediction of belonging to a latent class (relative to another latent class) of mother- or teacher-reported withdrawal from the T1 temperament or ego control predictors.3 Note that FIML normally utilizes participants with some missing data; however, in Mplus, participants are not used in the analyses if they have missing data on observed predictors (Muthén & Shedden, 1999).

If a predictor’s parameter estimate was significant, the odds ratio is reported, which represents the change in the odds that a child belongs to a particular class (relative to another class) corresponding to a one-unit change in the predictor (Cohen, Cohen, West, & Aiken, 2003) and often is used as a measure of effect size. For mother-reported withdrawal, six contrasts (HD vs. LD, MS vs. LD, MI vs. LD, HD vs. MS, MI vs. MS, and HD vs. MI) were computed. All contrast results are reported, but the LD versus MS contrast was of secondary interest. For teacher-reported withdrawal, three contrasts of interest are reported (HD vs. LS, LI vs. LS, and HD vs. LI). The class following each “versus” was set as the reference class within the contrast.

Because initial analyses indicated that the withdrawal trajectories were not significantly predicted by children’s age at T1, age was not considered further. Also in initial analyses, when sex was entered as a covariate in models in which T1 temperament and resiliency were used to predict latent class withdrawal membership, latent class membership was never significantly predicted by children’s sex. Thus, sex was omitted from the models described below.

Results are summarized in Table 2 for mother-reported withdrawal classes and in Table 3 for teacher-reported withdrawal classes (only significant results are presented for clarity and to save space). Within each contrast, for all but fearfulness (which had only one analysis because teachers did not report on fearfulness), there were two analyses per predictor (parent-or teacher-reported). Thus, within each contrast, 11 predictors were examined. Although we examined prediction within and across context (within teachers’ or parents’ reports and across them), most findings were from within-reporter analyses (six analyses for mothers and five for teachers), as might be expected given the lack of agreement across contexts in regard to withdrawal and the modest to low agreement in ratings of some predictors, especially sadness and ego resiliency.

Table 2.

Predicting mother-reported withdrawal class membership from predictors

HD Versus LD
MS Versus LD
MI Versus LD
HD Versus MS
MI Versus MS
Contrast Beta OR Contrast Beta OR Contrast Beta OR Contrast Beta OR Contrast Beta OR
Anger
 T1 PR HD > LD 1.19* 3.29 MS > LD 0.65* 1.92 MI > LD 1.73** 5.64 MI > MS 1.08* 2.94
 T1 TR HD > LD 0.62* 1.86 MI > LD 0.45* 1.57 HD > MS 0.83** 2.29 MI > MS 0.66** 1.93
Fear
 T1 PR MS > LD 0.49* 1.63
Att. control
 T1 PR HD < LD −1.79** 0.17 MS < LD −1.05** 0.35
 T1 TR HD < LD −0.96** 0.38 MI < LD −1.40* 0.25 HD < MS −0.74* 0.48
Resiliency
 T1 PR HD < LD −1.29** 0.28 MI < LD −1.19** 0.30 HD < MS −0.83* 0.44 MI < MS −0.73* 0.48
 T1 TR HD < LD −0.58* 0.56

Note: HD, high and declining; LD, low and declining; MS, moderate and stable; MI, moderate and increasing; OR, odds ratio; T1, Time 1; PR, parent reported; TR, teacher reported; Att., attentional. An OR of >1.0 indicates an increase in the odds of being in the boldface class, and an OR of <1.0 indicates a decrease in the odds. There were no significant predictors for HD versus MI and no significant contrasts for parent- or teacher-reported sadness or impulsivity.

*

p < .05.

**

p < .01.

Table 3.

Predicting teacher-reported withdrawal class membership from predictors

HD Versus LS
LI Versus LS
Contrast Beta OR Contrast Beta OR
Anger
 T1 PR
 T1 TR HD > LS 0.75** 2.12
Sadness
 T1 PR
 T1 TR HD > LS 0.94** 2.56
Att. control
 T1 PR HD < LS −0.81** 0.44
 T1 TR HD < LS −1.06** 0.35 LI < LS −0.87** 0.42
Resiliency
 T1 PR
 T1 TR HD < LS −1.26** 0.28 LI < LS −0.76** 0.47

Note: HD, high and declining; LS, low and stable; LI, low and increasing; OR, odds ratio; T1, Time 1; PR, parent reported; TR, teacher reported; Att., attentional. An OR of >1.0 indicates an increase in the odds of being in the boldface class, and an OR of <1.0 indicates a decrease in the odds. There were no significant predictors for HD versus LI. There were no significant contrasts for fearfulness or impulsivity.

*

p < .05.

**

p < .01.

HD versus LD mother-reported withdrawal

For anger, sadness, fearfulness, impulsivity, attentional control, and ego resiliency, 2, 0, 0, 0, 2, and 2 effects were significant, respectively. An increase in the probability of mother-reported HD (vs. LD) class membership was predicted by higher T1 parent- and teacher-reported anger, whereas a decrease in the probability was predicted by higher T1 parent- and teacher-reported attentional control and T1 parent- and teacher-reported ego resiliency.

MI versus LD mother-reported withdrawal

For anger, sadness, fearfulness, impulsivity, attentional control, and ego resiliency, 2, 0, 0, 0, 1 and 1 effects were significant, respectively. An increase in the probability of mother-reported MI (vs. LD) class membership was predicted by higher T1 parent- and teacher-reported anger, whereas a decrease in the probability was predicted by higher T1 teacher-reported attentional control and T1 parent-reported ego resiliency.

MS versus LD mother-reported withdrawal

For anger, sadness, fearfulness, impulsivity, attentional control, and ego resiliency, 1, 0, 1, 0, 1, and 0 effects were significant, respectively. An increase in the probability of mother-reported MS (vs. LD) class membership was predicted by higher T1 parent-reported anger and fear, whereas a decrease in the probability was predicted by higher T1 parent-reported attentional control.

HD versus MS mother-reported withdrawal

For anger, sadness, fearfulness, impulsivity, attentional control, and ego resiliency, 1, 0, 0, 0, 1 and 1 effects were significant, respectively. An increase in the probability of mother-reported HD (vs. MS) class membership was predicted by higher T1 teacher-reported anger, whereas a decrease in the probability was predicted by higher T1 teacher-reported attentional control and T1 parent-reported ego resiliency.

MI versus MS mother-reported withdrawal

For anger, sadness, fearfulness, impulsivity, attentional control, and ego resiliency, 2, 0, 0, 0, 0 and 1 effects were significant, respectively. An increase in the probability of mother-reported MI (vs. MS) class membership was predicted by higher T1 parent- and teacher-reported anger, whereas a decrease in the probability was predicted by higher T1 parent-reported ego resiliency.

HD versus MI mother-reported withdrawal

There were no significant effects for any of the predictors for the HD versus MI contrast.

HD versus LS teacher-reported withdrawal

For anger, sadness, fearfulness, impulsivity, attentional control, and ego resiliency, 1, 1, 0, 0, 2, and 1 effects were significant, respectively. An increase in the probability of teacher-reported HD (vs. LS) class membership was predicted by higher T1 teacher-reported anger and sadness, whereas a decrease in the probability was predicted by higher T1 parent- and teacher-reported attentional control and T1 teacher-reported ego resiliency.

LI versus LS teacher-reported withdrawal

For anger, sadness, fearfulness, impulsivity, attentional control, and ego resiliency, 0, 0, 0, 0, 1, and 1 effects were significant. A decrease in the probability of teacher-reported LI (vs. LS) membership was predicted by higher T1 teacher-reported attentional control and resiliency.

HD versus LI teacher-reported withdrawal

There were no significant effects for any of the predictors for the HD versus LI contrast.

Discussion

In the present study, we examined the relations of patterns of withdrawn behavior over 6 years (four assessments from childhood into adolescence) with parents’ and teachers’ reports of children’s negative emotionality (anger, sadness, fear), impulsivity, attentional control, and ego resiliency. In this sample in which we overselected children who had internalizing symptoms, mothers identified 6.80% and teachers identified 8.25% of children as highly withdrawn at the first assessment; however, children in both of these classes (mother- and teacher-reported HD classes) had significant declines in their withdrawal over time and eventually ended up with moderate or low withdrawal scores at T4 for mothers’ and teachers’ reports, respectively. In addition, mothers and teachers identified classes of children (5.21 and 12.04% of the children, respectively) whose withdrawal was moderate or low initially, but increased significantly over time (mother-reported MI and teacher-reported LI classes) and had the highest withdrawal scores at T4. These four classes likely include children experiencing problematic withdrawal at some point during childhood, but children in the mother-and teacher-reported HD classes appeared to be overcoming their problems, whereas children in the mother-reported MI and teacher-reported LI classes experienced elevated withdrawal problems over time. It is possible that children in the latter classes might be at risk for developing AvPD, especially if they do not find ways to cope with their withdrawal.

In addition to the aforementioned changing classes, we identified mother-reported LD (58% of the children) and MS classes (30%), as well as a teacher-reported LS class (80%). The overall pattern of trajectories obtained is similar to that found by Booth-Laforce and Oxford (2008) and Oh et al. (2008), who identified a low stable withdrawal group (including 86 and 85% of children, respectively) and smaller decreasing (5 and 8%) and increasing (9 and 7%, respectively) groups. The fact that mothers in this study identified a moderate (but not that high) stable class may have been because we overselected for children with internalizing problems. It is of note that all three studies identified comparatively small groups of children who began relatively high in withdrawal and improved and small groups that worsened with age.

Based on the literature on AvPD, SP, shyness, and inhibition in adulthood, we expected stably withdrawn behavior to be associated with negative affectivity, low attentional regulation, and behavior that was relatively constrained (i.e., low impulsivity) and rigidly maladaptive (low ego resiliency). In general, the probability of being in the mother- or teacher-reported HD withdrawal classes, in comparison to less problematic withdrawal classes (i.e., LD or MS for mother-reported withdrawal, LS for teacher-reported withdrawal), was associated with higher anger, lower attentional control, and lower ego resiliency (and higher sadness within teachers’ reports). Thus, although there was little association between mother- and teacher-reported withdrawal, a similar pattern of relations with the predictors was found for mother- and teacher-reported HD withdrawal in comparisons with less problematic classes. In addition, the probability of belonging to the mother-reported MI class, in comparison to less problematic withdrawal classes, was also predicted by higher anger, lower ego resiliency, and, in one case, lower attentional control. Similarly, the probability of belonging to the teacher-reported LI class in comparison to the LS class was predicted by lower attentional control and lower ego resiliency. In summary, children with either HD or MI/LI patterns of withdrawal exhibited more maladapted patterns of behavior than children with less problematic withdrawal, especially with regard to anger, attentional control, and ego resiliency. This pattern of findings was most evident within a given context (the home or school), but prediction across context was significant in several cases.

Although not of central interest in the present study, there was some evidence that the MS pattern of mother-reported withdrawal in comparison to the LD class was associated with parents’ reports of high anger, fear, and lower attentional control. Thus, a moderate level of mother-reported withdrawal was associated with a somewhat problematic temperament, at least as reported by parents. Because of the moderate level of problems and the high percentage of children in the MS withdrawal class (30%), it is likely that this level of problematic characteristics is not predictive of AvPD or SP.

In general, the pattern of findings suggests that temperament/personality in the early school years predicts trajectories of withdrawal over a 6-year period, as children moved into adolescence. For some children, mother- or teacher-reported withdrawal was initially high and significantly declined to moderate or low levels across this period of time. In contrast, for some children whose withdrawal was initially moderate or low, withdrawal significantly increased to high levels when approaching adolescence. It seems likely that this high level of withdrawal would be sustained into adulthood and would, at least for some children, be predictive of AvPD. Consistent with this argument, Asendorpf et al. (2008) found that the upper 8% in inhibition in early childhood continued to show internalizing problems in early adulthood. Of course, given the correlational nature of the data, causal relations cannot be affirmed with any certainty.

Although higher anger, lower attentional control, and lower ego resiliency in early childhood often distinguished between children who possessed high levels of withdrawal at some point during the study from children with moderate or low levels of withdrawal, these characteristics did not assist in predicting whether children’s withdrawal became worse or better over time. This was evident when examining the probability of membership in the mother-reported HD versus MI class, as well as membership in the teacher-reported HD versus LI class. None of the variables examined significantly predicted the probability of membership in one or the other of these classes. Perhaps inclusion of environmental variables (e.g., peer treatment, success of social overtures, relationships with parents or teachers) would have provided additional predictive utility in this regard. Children with problematic personalities may be relatively likely to improve if they are in environments that buffer their temperamental vulnerabilities and help them to develop adaptive coping and social skills or sociocognitive processes.

In fact, empirical findings suggest an environmental contribution to AvPD. Researchers have highlighted early negative experiences with parents (e.g., maltreatment, separation) or peers (e.g., rejection) as a potential root of AvPD (see Sperry, 2003). For example, self-reported parental neglect has been associated with increased risk of AvPD in adult outpatients with depression (Joyce et al., 2003). Battle et al. (2004) examined retrospective self-reports from treatment-seeking adults with personality disorder(s) (PD; e.g., AvPD, borderline PD). The majority of participants indicated being the victim of some form of childhood abuse (73%; e.g., emotional, verbal, physical, sexual) or childhood neglect (82%). Moreover, Nakash-Eisikovits, Dutra, and Westen (2002) found that secure attachment negatively, and disorganized/unresolved attachment positively, related to AvPD in a 14- to 18-year-old clinical sample, whereas avoidant and anxious/ambivalent attachments were not significantly related to AvPD. Although these findings could partly reflect evocative genetic characteristics of the child, they suggest that environmental factors also play a role.

Although environmental factors likely affect which temperamentally vulnerable children are susceptible to problems of withdrawal, heredity also appears to contribute to risk for AvPD. Johnson et al. (1995) found that 13- to 19-year-old inpatients with AvPD (using the DSM-III-R) had first-degree adult relatives with increased prevalence of AvPD. In a Nor-wegian twin sample, characteristics of AvPD were found to be only moderately influenced by genetics (nonadditive heritability = .28), but the authors speculated that this modest effect reflected a cultural effect in that AvPD is more common in Norway relative to other countries. In addition to heritability, nonshared environmental influences were noted (.72; Torgersen et al., 2000). The relation of relatively high levels of avoidance with temperamental vulnerabilities is consistent with literature indicating that AvPD has a genetic as well as environmental basis. Indeed, it is likely that AVPD is predicted by a vulnerable temperament (which is partly because of heredity) combined with an environment that heightens children’s vulnerabilities in the social domain.

The children in increasing and decreasing withdrawal classes appear to be similar in at least some aspects of their temperament/personality, in particular, those vulnerabilities identified in this study, but their trajectories may diverge as a function of dispositional factors not assessed in this study or their social environment. Because AvPD is characterized by high levels of social anxiety, a sense of inadequacy, and concern about others’ evaluations, social environments that diminish children’s sense of self and heighten their concern about others’ evaluations may, in interaction with a vulnerable temperament, promote symptoms of AvPD. Because anxieties and negative self-perceptions related to social interactions appear to emerge and increase for some children (e.g., withdrawn children) in the early school years (Asendorpf & van Aken, 2004; Rubin & Mills, 1988; Vasey, Crnic, & Carter, 1994), children with a vulnerable temperament exposed to a self-diminishing environment might be expected to increase in their withdrawal during the school years. In contrast, children with high levels of initial withdrawal because of temperamental behavioral inhibition and problems with EC, anger, and resiliency may improve, especially if they have social skills and are relatively intelligent (Asendorpf, 1994) and are in a social environment that helps them develop skills for dealing with their inhibition.

The mother-identified, MI withdrawal class, the one we predicted is most likely to predict AvPD, did differ from the mother-reported moderately stable class in their level of anger and lack of ego resiliency. These findings suggest that dispositional factors might predict which children with moderate withdrawal in the early school years are at risk for increasing levels of withdrawal across childhood. Children both prone to anger and low in ego resiliency may tend to react with anger and have difficulty recouping when they have negative social interactions. Their anger may sometimes result from feelings of shame or humiliation, and in turn, may elicit disapproval from others. Because the MI mother-identified class also had a somewhat higher intercept than the MS class, it would be important to study these children at an earlier age to determine when their upward trajectory begins and early predictors of the increasing trajectory.

There was only one significant effect of parent-reported child fearfulness with withdrawn class membership (recall that teachers did not rate fearfulness). Higher fearfulness was associated with membership in the MS in comparison to the LD mother-reported withdrawal class. This may have been because the fearfulness scale of the CBQ taps a range of childhood fears, most of which are not social and many of which would be expected to decline with age. It is likely that measures of social fears, worrying, or anxiety would show stronger patterns of association with withdrawn behavior.

Higher teacher-reported sadness was related to membership only in the teacher-reported HD versus LS class. Thus, temperamental sadness only weakly predicted withdrawal trajectories. It is possible that more extreme levels of sadness would be predictive of or associated with withdrawal trajectories. As noted in the literature review, depressive symptoms have been associated with AvPD and SP in previous studies (e.g., Beidel et al., 1999; Boone et al., 1999; van Velzen et al., 2000). For children with problems with withdrawal, anger and anxiety may be characteristic negative emotions. It is also possible that adults often are not aware of older children’s sadness.

Perhaps most surprising was that parent- and teacher-reported impulsivity were not related to membership in the withdrawal classes. Low impulsivity may be indicative of inhibited behavior, has been related to withdrawal and general internalizing problems (see Eisenberg, Eggum, Vaughan, & Edwards, in press), and, according to experts, often is associated with AvPD (Warner et al., 2004). Perhaps impulsivity is especially associated with low levels of shyness; recall that the items most indicative of shyness were dropped from our withdrawal scale because experts deemed them to be overlapping with temperament measures.

We had hoped to find a stably high withdrawal class but did not do so when using modeling procedures. However, when, like Booth et al. (2008), we identified children who were 1 SD above the mean (T score ≥ 60) in withdrawal at each time point (or who had this pattern at three assessments and were within three points below that cutoff at just one time point), we found that mothers and teachers reported that 14% and 4%, respectively, of children remained fairly high in withdrawal over the 6 years. As was found by Booth et al. (2008), these children likely included some from the increasing and the declining classes. We found that the children with high stable withdrawal, especially those identified by parents, were prone to not only problems with attentional control, anger, and resiliency but also low impulsivity and high levels of sadness. This was particularly true when compared to children whose withdrawal T score was less than 60 at each time point (or within 3 points above that cutoff at just one of the time points).4 Thus, there were some children who remained at least somewhat high in withdrawal and appeared to be prone to sadness and high levels of inhibition (i.e., low impulsivity). Perhaps over time these children received fairly stable levels of peer rejection and/or negative feedback or lack of support from adults that resulted in heightened sadness and a continuing pattern of relative constraint. It is an empirical question whether children in the increasing trajectory or those who maintained a relatively high level of withdrawal despite some change in level of withdrawal would be more susceptible to AvPD.

There was very little agreement between mothers and teachers in regard to their ratings of children’s withdrawn behavior. Other investigators also have also found scant agreement between parents and teachers in regard to deviant scores on withdrawal on the CBCL and TRF (e.g., Verhulst, Koot, & van der Ende, 1994). Teachers are likely less aware of children’s withdrawn behavior given the demands on their attention and the fact that such behavior is usually not disruptive in the classroom. Yet the pattern of associations between withdrawal class membership and temperament/personality was fairly similar for parents’ and teachers’ reports.

Because of the similarities in the correlates of mother-reported and teacher-reported withdrawn behavior, it is possible that both predict problems in adulthood, but of a different nature. Because of the presence of peers at school and the importance of withdrawn and self-concerned behavior with peers as a symptom of SP, it is possible that school-linked withdrawn behavior is a better predictor of SP and problems with peers than AvPD. However, the low levels of teacher-reported ego resiliency and high teacher-rated sadness for these children suggest that their withdrawal may be associated with a more general pattern of avoidance. It also is possible that mothers’ and teachers’ ratings of withdrawal are based somewhat on different items; for example, parents may attend more to negative emotion and timidity, symptoms that may reflect feelings of inadequacy, inferiority, and rejection, whereas teachers may notice behaviors more indicative of social withdrawal (e.g., withdrawn and constrained behavior). In exploratory analyses in which we compared mothers’ versus teachers’ ratings on each of the items on the withdrawal scale, mothers’ average ratings were significantly higher than teachers’ average ratings at all time points for two of the items, refuses to talk, and, sulks a lot. Mothers’ average ratings on two additional items, secretive, keeps things to self, and, unhappy, sad, depressed, were significantly higher than teachers’ average ratings at T1, T3, and T4 and at T1, respectively. Teachers’ average ratings were higher than mothers’ ratings on one item, withdrawn, doesn’t get involved with others, but only at T2 and T3 (and this item may often reflect peer rejection). Mothers’ and teachers’ average ratings did not differ at any time point for stares blankly. Thus, teachers seemed to be picking up primarily on withdrawn behavior that can occur for a variety of reasons, whereas parents seemed to notice children’s affect and attempts to keep to themselves in terms of talking and sharing information. Given that teachers’ ratings of withdrawal were not very high for any trajectory class, it is unclear if teacher-reported withdrawn behavior predicts AvPD or SP. It would be useful to follow withdrawn children at school and home into adulthood and compare their levels of SP and AvPD.

Conclusion

In summary, because we did not have diagnoses of AvPD for the sample, we can only speculate based on relations between inhibition or shyness and AvPD in the prior literature that many of the children with increasing levels of withdrawal would eventually be high in symptoms of AvPD. In addition to the limitation of not having a diagnosis of AvPD in adulthood, other limitations are the fact that the sample was selected with some specific criteria so the findings may not fully apply to an unselected community sample. Moreover, most of the children in the study were from middle-class, European American families, although there is some diversity in the sample. It is unclear, however, if our findings would apply to samples of economically deprived or minority children. Nonetheless, our findings indicate that children who are relatively highly withdrawn in early childhood or as they move into adolescence are viewed by adults as exhibiting some temperamentally based characteristics and behaviors that probably would be expected to predict AvPD. A small group of children whose withdrawal was initially high did not exhibit high withdrawal when approaching adolescence. The reason for their sharp decline in withdrawal is unclear; it could be due to an age-related decline in behavioral inhibition and/or social influences (e.g., socialization; Eggum et al., in press). In contrast, a relatively small group of the children initially experiencing moderate withdrawal eventually displayed elevated levels of withdrawal. This class of children, who were reported to have problematic temperaments and low ego resiliency, may be especially at risk for problems such as AvPD. Our findings provide clues to dispositional factors in childhood that should be examined in future studies of AvPD. Moreover, given that mother- and teacher-reported withdrawal yielded classes of children exhibiting significant increases or decreases in withdrawal (as also was found by Booth-LaForce & Oxford, 2008; Oh et al., 2008), it is important to identify environmental factors that affect change in withdrawal and moderate the role of constitutional factors in predicting high levels of withdrawal across childhood into adulthood.

Acknowledgments

This research was supported in part by grants from the National Institutes of Mental Health (to N.E., T.L.S., and M.R.) and a Research Scientist Award from the National Institute of Mental Health (to N.E.). We thank our undergraduate and graduate research assistants for their contributions to this study, as well as Craig Enders and Jenn Yun-Tein for statistical advice. We also thank the children, parents, principals, and teachers involved in the study.

Footnotes

1

Growth mixture modeling (GMM) also was attempted in which latent classes of growth trajectories were specified, but between- and within-class variation were estimated (recall that in LCGA only between-class variation is allowed). LCGAs were used to determine an upper-bound estimate of the number of latent classes of withdrawal, obtain an initial idea of the shapes of the average trajectory within each latent class, and provide start values for GMM parameter estimates. Indices used to evaluate the number of latent classes in LCGA also were used in GMM. The shape of the trajectory within each latent class was also evaluated. For mother- and teacher-reported withdrawal, the data often suggested constraining all variances within each latent class to be fixed at zero or fatal estimation errors were encountered that could not be eradicated, likely because of the complexity of the models. Thus, modeling for withdrawal proceeded within the LCGA framework rather than within the GMM framework.

2

Of the mother-reported HD withdrawn children (n = 16), 3 children were also in the teacher-reported HD class (n = 73).

3

At T1, adults rated children’s shyness (1–7) on the CBQ Short Form (Putnam & Rothbart, 2006). After removing an item confounded with negative emotion, five items (e.g., Is shy even around people (s)he has known for a long time) were used to compute parent- and teacher-reported shyness composites (αs = .82 for mothers and .86 for teachers). For mother-reported withdrawal, T1 parent-reported shyness significantly predicted the HD (vs. LD) class membership (b = .73, p < .01) and the MS (vs. LD) class membership (b = .60, p < .01). T1 teacher-reported shyness did not significantly predict mother-reported withdrawal class membership. For teacher-reported withdrawal, T1 teacher-reported shyness positively predicted HD (vs. LS) class membership (b = .94, p < .01). T1 parent-reported shyness did not significantly predict teacher-reported withdrawal class membership.

4

Details are available upon request from the authors.

References

  1. Achenbach TM. Integrative guide for the 1991 CBCL/4–18, YSR, and TRF profiles. Burlington, VT: University of Vermont, Department of Psychiatry; 1991a. [Google Scholar]
  2. Achenbach TM. Manual for the Child Behavior Checklist/4–18 and 1991 profile. Burlington, VT: University of Vermont, Department of Psychiatry; 1991b. [Google Scholar]
  3. Achenbach TM, Edelbrock CS. Behavioral problems and competencies reported by parents of normal and disturbed children aged four through sixteen. Monographs of the Society for Research in Child. Development. 1981;46:1–82. [PubMed] [Google Scholar]
  4. Achenbach TM, McConaughy SH, Howell C. Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations of situational specificity. Psychological Bulletin. 1987;101:213–232. [PubMed] [Google Scholar]
  5. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 3. Washington, DC: Author; 1980. [Google Scholar]
  6. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 3. Washington, DC: Author; 1987. [Google Scholar]
  7. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4. Washington, DC: Author; 1994. [Google Scholar]
  8. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4. Washington, DC: Author; 2000. text revision. [Google Scholar]
  9. Asendorpf JB. Beyond social withdrawal: Shyness, unsociability, and peer avoidance. Human Development. 1990;33:250–259. [Google Scholar]
  10. Asendorpf JB. The malleability of behavioral inhibition: A study of individual developmental functions. Developmental Psychology. 1994;30:912–919. [Google Scholar]
  11. Asendorpf JB, Denissen JJA, van Aken MAG. Inhibited and aggressive preschool children at 23 years of age: Personality and social transitions into adulthood. Developmental Psychology. 2008;44:997–1011. doi: 10.1037/0012-1649.44.4.997. [DOI] [PubMed] [Google Scholar]
  12. Asendorpf JB, van Aken MAG. Traits and relationship status: Stranger versus peer group inhibition and test intelligence versus peer group competence as early predictors of later self-esteem. Child Development. 1994;65:1786–1789. doi: 10.1111/j.1467-8624.1994.tb00849.x. [DOI] [PubMed] [Google Scholar]
  13. Battle CL, Shea MT, Johnson DM, Yen S, Zlotnick C, Zanarini MC, et al. Childhood maltreatment associated with adult personality disorders: Findings from the collaborative longitudinal personality disorders study. Journal of Personality Disorders. 2004;18:193–211. doi: 10.1521/pedi.18.2.193.32777. [DOI] [PubMed] [Google Scholar]
  14. Beidel DC, Turner SM, Morris TL. Psychopathology of childhood social phobia. Journal of the American Academy of Child & Adolescent Psychiatry. 1999;38:643–650. doi: 10.1097/00004583-199906000-00010. [DOI] [PubMed] [Google Scholar]
  15. Biederman J, Hirshfeld-Becker DR, Rosenbaum JF, Hérot C, Friedman D, Snidman N, et al. Further evidence of association between behavioral inhibition and social anxiety in children. American Journal of Psychiatry. 2001;158:1673–1679. doi: 10.1176/appi.ajp.158.10.1673. [DOI] [PubMed] [Google Scholar]
  16. Biederman J, Rosenbaum JF, Bolduc-Murphy EA, Faraone SV, Chaloff J, Hirshfeld DR, et al. A 3-year follow-up of children with and without behavioral inhibition. Journal of the American Academy of Child & Adolescent Psychiatry. 1993;32:814–821. doi: 10.1097/00004583-199307000-00016. [DOI] [PubMed] [Google Scholar]
  17. Biederman J, Rosenbaum JF, Hirshfeld DR, Faraone SV, Bolduc EA, Gersten M, et al. Psychiatric correlates of behavioral inhibition in young children of parents with and without psychiatric disorders. Archives of General Psychiatry. 1990;47:21–26. doi: 10.1001/archpsyc.1990.01810130023004. [DOI] [PubMed] [Google Scholar]
  18. Block JH, Block J. The role of ego-control and ego-resiliency in the organization of behavior. In: Collins WA, editor. Minnesota symposia on child psychology: Vol. 13. Development of cognition, affect, and social relations. Hillsdale, NJ: Erlbaum; 1980. pp. 39–101. [Google Scholar]
  19. Boone ML, McNeil DW, Masia CL, Turk CL, Carter LE, Ries BJ, et al. Multimodal comparisons of social phobia subtypes and avoidant personality disorder. Journal of Anxiety Disorders. 1999;13:271–292. doi: 10.1016/s0887-6185(99)00004-3. [DOI] [PubMed] [Google Scholar]
  20. Booth-LaForce C, Oxford ML. Trajectories of social withdrawal from Grades 1 to 6: Prediction from early parenting, attachment, and temperament. Developmental Psychology. 2008;44:1298–1313. doi: 10.1037/a0012954. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Brown TA. Temporal course and structural relationships among dimensions of temperament and DSM-IV anxiety and mood disorder constructs. Journal of Abnormal Psychology. 2007;116:313–328. doi: 10.1037/0021-843X.116.2.313. [DOI] [PubMed] [Google Scholar]
  22. Caspi A, Harrington H, Milne B, Amell JW, Theodore RF, Moffitt TE. Children’s behavioral styles at age 3 are linked to their adult personality traits at age 26. Journal of Personality. 2003;71:495–513. doi: 10.1111/1467-6494.7104001. [DOI] [PubMed] [Google Scholar]
  23. Caspi A, Moffitt TE, Newman DL, Silva PA. Behavioral observations at age 3 years predict adult psychiatric disorders. Archives of General Psychiatry. 1996;53:1033–1039. doi: 10.1001/archpsyc.1996.01830110071009. [DOI] [PubMed] [Google Scholar]
  24. Caspi A, Shiner RL. Personality development. In: Damon W, Lerner RM, Damon W, Lerner RM, editors. Handbook of child psychology: Vol. 3. Social, emotional, and personality development. 6. New York: Wiley; 2006. pp. 300–365. [Google Scholar]
  25. Chambless DL, Fydrich T, Rodebaugh TL. Generalized social phobia and avoidant personality disorder; meaningful distinction or useless duplication? Depression and Anxiety. 2008;25:8–19. doi: 10.1002/da.20266. [DOI] [PubMed] [Google Scholar]
  26. Chavira DA, Stein MB, Malcarne VL. Scrutinizing the relationship between shyness and social phobia. Journal of Anxiety Disorders. 2002;16:585–598. doi: 10.1016/s0887-6185(02)00124-x. [DOI] [PubMed] [Google Scholar]
  27. Cohen J, Cohen PA, West SG, Aiken LS. Applied multiple regression/correlation analysis for the behavioral sciences. 3. Mahwah, NJ: Erlbaum; 2003. [Google Scholar]
  28. Coplan RJ, Armer M. A “multitude” of solitude: A closer look at social withdrawal and nonsocial play in early childhood. Child Development Perspectives. 2007;1:26–32. [Google Scholar]
  29. Coplan RJ, Prakash K, O’Neil K, Armer M. Do you “want” to play? Distinguishing between conflicted-shyness and social disinterest in early childhood. Developmental Psychology. 2004;40:244–258. doi: 10.1037/0012-1649.40.2.244. [DOI] [PubMed] [Google Scholar]
  30. Coplan RJ, Wilson J, Frohlick SL, Zelenski J. A person-oriented analysis of behavioral inhibition and behavioral activation in childhood. Personality and Individual Differences. 2006;41:917–927. [Google Scholar]
  31. Cumberland-Li A, Eisenberg N, Reiser M. Relations of young children’s agreeableness and resiliency to effortful control and impulsivity. Social Development. 2004;13:193–212. [Google Scholar]
  32. Derryberry D, Reed M. Anxiety-related attentional biases and their regulation by attentional control. Journal of Abnormal Psychology. 2002;111:225–236. doi: 10.1037//0021-843x.111.2.225. [DOI] [PubMed] [Google Scholar]
  33. Derryberry D, Rothbart MK. Arousal, affect, and attention as components of temperament. Journal of Personality and Social Psychology. 1988;55:958–966. doi: 10.1037//0022-3514.55.6.958. [DOI] [PubMed] [Google Scholar]
  34. Eggum ND, Eisenberg N, Spinrad TL, Reiser M, Gaertner BM, Sallquist J, et al. Development of shyness: Relations with children’s fearfulness, sex, and maternal behavior. Infancy. doi: 10.1080/15250000902839971. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Eisenberg N, Chang L, Ma Y, Huang X. Relations of parenting style to Chinese children’s effortful control, ego resilience, and maladjustment. Development and Psychopathology. 2009;21:455–477. doi: 10.1017/S095457940900025X. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Eisenberg N, Cumberland A, Spinrad TL, Fabes RA, Shepard SA, Reiser M, et al. The relations of regulation and emotionality to children’s externalizing and internalizing problem behavior. Child Development. 2001;72:1112–1134. doi: 10.1111/1467-8624.00337. [DOI] [PubMed] [Google Scholar]
  37. Eisenberg N, Eggum ND, Vaughan J, Edwards A. Relations of self-regulatory capacities to maladjustment, social competence, and emotionality. In: Hoyle RH, editor. The handbook of self-regulation and personality. Malden, MA: Blackwell; in press. [Google Scholar]
  38. Eisenberg N, Fabes RA, Murphy B. Relations of shyness and low sociability to regulation and emotionality. Journal of Personality and Social Psychology. 1995;68:505–517. doi: 10.1037//0022-3514.68.3.505. [DOI] [PubMed] [Google Scholar]
  39. Eisenberg N, Fabes RA, Nyman M, Bernzweig J, Pinuelas A. The relations of emotionality and regulation to children’s anger-related reactions. Child Development. 1994;65:109–128. [PubMed] [Google Scholar]
  40. Eisenberg N, Ma Y, Chang L, Zhou Q, West SG, Aiken L. Relations of effortful control, reactive undercontrol, and anger to Chinese children’s adjustment. Development and Psychopathology. 2007;19:385–409. doi: 10.1017/S0954579407070198. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Eisenberg N, Sadovsky A, Spinrad TL, Fabes RA, Losoya SH, Valiente C, et al. The relations of problem behavior class membership to children’s negative emotionality, effortful control, and impulsivity: Concurrent relations and prediction of change. Developmental Psychology. 2005;41:193–211. doi: 10.1037/0012-1649.41.1.193. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Eisenberg N, Shepard SA, Fabes RA, Murphy BC, Guthrie IK. Shyness and children’s emotionality, regulation, and coping: Contemporaneous, longitudinal, and across-context relations. Child Development. 1998;69:767–790. [PubMed] [Google Scholar]
  43. Eisenberg N, Spinrad TL, Fabes RA, Reiser M, Cumberland A, Shepard SA, et al. The relations of effortful control and impulsivity to children’s resiliency and adjustment. Child Development. 2004;75:25–46. doi: 10.1111/j.1467-8624.2004.00652.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Ekselius L, Tillfors M, Furmark T, Fredrikson M. Personality disorders in the general population: DSM-IV and ICD-10 defined prevalence as related to sociodemographic profile. Personality and Individual Differences. 2001;30:311–320. [Google Scholar]
  45. Enders CK, Peugh JL. Using an EM covariance matrix to estimate structural equation models with missing data: Choosing an adjusted sample size to improve the accuracy of inferences. Structural Equation Modeling. 2004;11:1–19. [Google Scholar]
  46. Erwin BA, Heimberg RG, Schneier FR, Liebowitz MR. Anger experience and expression in social anxiety disorder. Behavior Therapy. 2003;34:331–350. [Google Scholar]
  47. Geiger TC, Crick NR. A developmental psychopathology perspective on vulnerability to personality disorders. In: Ingram RE, Price JM, editors. Vulnerability to psychopathology: Risk across the lifespan. New York: Guilford Press; 2001. [Google Scholar]
  48. Gilbert P. What is shame? Some core issues and controversies. In: Gilbert P, Andrews B, editors. Shame: Interpersonal behavior, psychopathology, and culture. Oxford: Oxford University Press; 1998. pp. 5–38. [Google Scholar]
  49. Goldsmith HH, Rieser-Danner LA, Briggs S. Evaluating convergent and discriminate validity of temperament questionnaires for preschoolers, toddlers, and infants. Developmental Psychology. 1991;27:566–580. [Google Scholar]
  50. Griego J, Stewart SE, Coolidge FL. A convergent validity study of Cloninger’s Temperament and Character Inventory with the Coolidge Axis 11 Inventory. Journal of Personality Disorders. 1999;13:256–267. [PubMed] [Google Scholar]
  51. Heiser NA, Turner SM, Beidel DC. Shyness: Relation to social phobia and other disorders. Behaviour Research and Therapy. 2003;41:209–221. doi: 10.1016/s0005-7967(02)00003-7. [DOI] [PubMed] [Google Scholar]
  52. Hirshfeld-Becker DR, Biederman J, Henin A, Faraone SV, Davis S, Harrington K, et al. Behavioral inhibition in preschool children at risk is a specific predictor of middle childhood social anxiety: A five-year follow-up. Journal of Developmental and Behavioral Pediatrics. 2007;28:225–233. doi: 10.1097/01.DBP.0000268559.34463.d0. [DOI] [PubMed] [Google Scholar]
  53. Hofmann SG, Heinrichs N, Moscovitch DA. The nature and expression of social phobia: Toward a new classification. Clinical Psychology Review. 2004;24:769–797. doi: 10.1016/j.cpr.2004.07.004. [DOI] [PubMed] [Google Scholar]
  54. Holt CS, Heimberg RG, Hope DA. Avoidant personality disorder and the generalized subtype of social phobia. Journal of Abnormal Psychology. 1992;101:318–325. doi: 10.1037//0021-843x.101.2.318. [DOI] [PubMed] [Google Scholar]
  55. Hummelen B, Wilberg T, Pedersen G, Karterud S. The relationship between avoidant personality disorder and social phobia. Comprehensive Psychiatry. 2007;48:348–356. doi: 10.1016/j.comppsych.2007.03.004. [DOI] [PubMed] [Google Scholar]
  56. Johnson BA, Brent DA, Connolly J, Bridge F, Matta J, Constantine D, et al. Familial aggregation of adolescent personality disorders. Journal of the American Academy of Child & Adolescent Psychiatry. 1995;34:798–804. doi: 10.1097/00004583-199506000-00021. [DOI] [PubMed] [Google Scholar]
  57. Johnson MR, Lydiard RB. Personality disorders in social phobia. Psychiatric Annals. 1995;25:554–563. [Google Scholar]
  58. Joyce PR, McKenzie JM, Luty SE, Mulder RT, Carter JD, Sullivan PF, et al. Temperament, childhood environment and psychopathology as risk factors for avoidant and borderline personality disorders. Australian and New Zealand Journal of Psychiatry. 2003;37:756–764. doi: 10.1080/j.1440-1614.2003.01263.x. [DOI] [PubMed] [Google Scholar]
  59. Kagan J. Biology and the child. In: Damon W, Eisenberg N, editors. Handbook of child psychology. Vol. 3. Social, emotional, and personality development. 5. Hoboken, NJ: Wiley; 1998. pp. 177–235. [Google Scholar]
  60. Kagan J, Fox NA. Biology, culture, and temperament biases. In: Damon W, Lerner RM, Damon W, Lerne RM, editors. Handbook of child psychology. Vol. 3. Social, emotional, personality development. 6. New York: Wiley; 2006. pp. 167–225. [Google Scholar]
  61. Kagan J, Reznick JS, Gibbons F. Inhibited and uninhibited types of children. Child Development. 1989;60:838–845. [PubMed] [Google Scholar]
  62. Kochanska G, Knaack A. Effortful control as a personality characteristic of young children: Antecedents, correlates, and consequences. Journal of Personality. 2003;71:1087–1112. doi: 10.1111/1467-6494.7106008. [DOI] [PubMed] [Google Scholar]
  63. Lengua LJ. Growth in temperament and parenting as predictors of adjustment during children’s transition to adolescence. Developmental Psychology. 2006;42:819–832. doi: 10.1037/0012-1649.42.5.819. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Lengua LJ, West SG, Sandler IN. Temperament as a predictor of symptomatology in children: Addressing contamination of measures. Child Development. 1998;69:164–181. [PubMed] [Google Scholar]
  65. Marteinsdottir I, Tillfors M, Furmark T, Anderberg UM, Ekselius L. Personality dimensions measured by the Temperament and Character Inventory (TCI) in subjects with social phobia. Nordic Journal of Psychiatry. 2003;57:29–35. doi: 10.1080/08039480310000239. [DOI] [PubMed] [Google Scholar]
  66. Martel MM, Nigg JT, Wong MM, Fitzgerald HE, Jester JM, et al. Childhood and adolescent resiliency, regulation, and executive functioning in relation to adolescent problems and competence in a high-risk sample. Development and Psychopathology. 2007;19:541–563. doi: 10.1017/S0954579407070265. [DOI] [PubMed] [Google Scholar]
  67. Meyer B. Personality and mood correlates of avoidant personality disorder. Journal of Personality Disorders. 2002;16:174–188. doi: 10.1521/pedi.16.2.174.22546. [DOI] [PubMed] [Google Scholar]
  68. Meyer B, Ajchenbrenner M, Bowles DP. Sensory sensitivity, attachment experiences, and rejection responses among adults with borderline and avoidant features. Journal of Personality Disorders. 2005;19:641–658. doi: 10.1521/pedi.2005.19.6.641. [DOI] [PubMed] [Google Scholar]
  69. Meyer B, Carver CS. Negative childhood accounts, sensitivity, and pessimism: A study of avoidant personality disorder features in college students. Journal of Personality Disorders. 2000;14:233–248. doi: 10.1521/pedi.2000.14.3.233. [DOI] [PubMed] [Google Scholar]
  70. Mick MA, Telch MJ. Social anxiety and history of behavioral inhibition in young adults. Journal of Anxiety Disorders. 1998;12:1–20. doi: 10.1016/s0887-6185(97)00046-7. [DOI] [PubMed] [Google Scholar]
  71. Moscovitch DA, McCabe RE, Antony MM, Rocca L, Swinson RP. Anger experience and expression across the anxiety disorders. Depression and Anxiety. 2008;25:107–113. doi: 10.1002/da.20280. [DOI] [PubMed] [Google Scholar]
  72. Murphy BC, Eisenberg N, Fabes RA, Shepard S, Guthrie I. Consistency and change in children’s emotionality and regulation: A longitudinal study. Merrill–Palmer Quarterly. 1999;45:413–444. [Google Scholar]
  73. Muthén LK, Muthén BO. Mplus user’s guide. 5. Los Angeles: Author; 1998–2007. [Google Scholar]
  74. Muthén BO, Shedden K. Finite mixture modeling with mixture. Outcomes using the EM algorithm. Biometrics. 1999;55:463–469. doi: 10.1111/j.0006-341x.1999.00463.x. [DOI] [PubMed] [Google Scholar]
  75. Nakash-Eisikovits O, Dutra L, Westen D. Relationship between attachment patterns and personality pathology in adolescents. Journal of the American Academy of Child & Adolescent Psychiatry. 2002;41:1111–1123. doi: 10.1097/00004583-200209000-00012. [DOI] [PubMed] [Google Scholar]
  76. Neal JA, Edelmann RJ. The etiology of social phobia: Toward a developmental profile. Clinical Psychology Review. 2003;23:761–786. doi: 10.1016/s0272-7358(03)00076-x. [DOI] [PubMed] [Google Scholar]
  77. Oh W, Rubin KH, Bowker JC, Booth-Laforce C, Rose-Krasnor L. Trajectories of social withdrawal from middle childhood to early adolescence. Journal of Abnormal Child Psychology. 2008;36:553–566. doi: 10.1007/s10802-007-9199-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  78. Oldehinkel AJ, Hartman CA, Ferdinand RF, Verhulst FC, Ormel J. Effortful control as modifier of the association between negative emotionality and adolescents’ mental health problems. Development and Psychopathology. 2007;19:523–539. doi: 10.1017/S0954579407070253. [DOI] [PubMed] [Google Scholar]
  79. Oldehinkel AJ, Hartman CA, Winter AFD, Veenstra R, Ormel J. Temperament profiles associated with internalizing and externalizing problems in preadolescence. Development and Psychopathology. 2004;16:421–440. doi: 10.1017/s0954579404044591. [DOI] [PubMed] [Google Scholar]
  80. Pickering AD, Gray JA. The neuroscience of personality. In: Pervin L, John O, editors. Handbook of personality. San Francisco, CA: Guilford Press; 1999. pp. 277–299. [Google Scholar]
  81. Putnam SP, Rothbart MK. Development of the short and very short forms of the Children’s Behavior Questionnaire. Journal of Personality Assessment. 2006;87:103–113. doi: 10.1207/s15327752jpa8701_09. [DOI] [PubMed] [Google Scholar]
  82. Reichborn-Kjennerud T, Czajkowski N, Torgersen S, Neale MC, Ørstavik RE, Tambs K, et al. The relationship between avoidant personality disorder and Social Phobia: A population-based twin study. American Journal of Psychiatry. 2007;164:1722–1728. doi: 10.1176/appi.ajp.2007.06101764. [DOI] [PubMed] [Google Scholar]
  83. Rettew DC. Avoidant personality disorder, generalized social phobia, and shyness: Putting the personality back into personality disorders. Harvard Review of Psychiatry. 2000;8:283–297. [PubMed] [Google Scholar]
  84. Rothbart MK, Ahadi SA, Hershey KL, Fisher P. Investigations of temperament at three to seven years: The Children’s Behavior Questionnaire. Child Development. 2001;72:1394–1408. doi: 10.1111/1467-8624.00355. [DOI] [PubMed] [Google Scholar]
  85. Rothbart MK, Bates JE. Temperament. In: Damon W, Lerner RM, Damon W, Lerner RM, editors. Handbook of child psychology: Vol. 3. Social, emotional, and personality development. 6. New York: Wiley; 2006. pp. 105–176. [Google Scholar]
  86. Rubin KH, Bukowksi WM, Parker JG. Peer interactions, relationships, and groups. In: Damon W, Lerner RM, Eisenberg N, editors. Handbook of child psychology: Vol. 3. Social, emotional, and personality development. 6. New York: Wiley; 2006. pp. 571–645. [Google Scholar]
  87. Rubin KH, Hastings PD, Stewart SL, Henderson HA, Chen X. The consistency and concomitants of inhibition: Some of the children, all of the time. Child Development. 1997;68:467–483. [PubMed] [Google Scholar]
  88. Rubin KH, Mills RSL. The many faces of social isolation in childhood. Journal of Consulting and Clinical Psychology. 1988;56:916–924. doi: 10.1037//0022-006x.56.6.916. [DOI] [PubMed] [Google Scholar]
  89. Schwartz CE, Snidman N, Kagan J. Adolescent social anxiety as an outcome of inhibited temperament in childhood. Journal of the American Academy of Child & Adolescent Psychiatry. 1999;38:1008–1015. doi: 10.1097/00004583-199908000-00017. [DOI] [PubMed] [Google Scholar]
  90. Shiner R, Caspi A. Personality differences in childhood and adolescence: Measurement, development, and consequences. Journal of Child Psychology and Psychiatry. 2003;44:2–32. doi: 10.1111/1469-7610.00101. [DOI] [PubMed] [Google Scholar]
  91. Silk JS, Steinberg L, Morris AS. Adolescents’ emotion regulation in daily life: Links to depressive symptoms and problem behavior. Child Development. 2003;74:1869–1880. doi: 10.1046/j.1467-8624.2003.00643.x. [DOI] [PubMed] [Google Scholar]
  92. Sperry L. Handbook of diagnosis and treatment of DSM-IV-TR personality disorders. 2. New York: Routledge; 2003. [Google Scholar]
  93. Spinrad TL, Eisenberg N, Harris E, Hanish L, Fabes RA, Kupanoff K, et al. The relations of children’s everyday nonsocial peer play behavior to their emotionality, regulation, and social functioning. Developmental Psychology. 2004;40:67–80. doi: 10.1037/0012-1649.40.1.67. [DOI] [PubMed] [Google Scholar]
  94. Stanger C, Lewis M. Agreement among parents, teachers, and children on internalizing and externalizing behavior problems. Journal of Clinical Child Psychology. 1993;22:107–115. [Google Scholar]
  95. Stemberger RT, Turner SM, Beidel DC, Calhoun KS. Social phobia: An analysis of possible developmental factors. Journal of Abnormal Psychology. 1995;104:526–531. doi: 10.1037//0021-843x.104.3.526. [DOI] [PubMed] [Google Scholar]
  96. Tangney JP, Wagner P, Gramzow R. Proneness to shame, proneness to guilt, and psychopathology. Journal of Abnormal Psychology. 1992;101:469–478. doi: 10.1037//0021-843x.101.3.469. [DOI] [PubMed] [Google Scholar]
  97. Torgersen S, Lygren S, Oien PA, Skre I, Onstad S, Edvardsen J, et al. A twin study of personality disorders. Comprehensive Psychiatry. 2000;41:416–425. doi: 10.1053/comp.2000.16560. [DOI] [PubMed] [Google Scholar]
  98. Valiente C, Eisenberg N, Smith CL, Reiser M, Fabes RA, Losoya S, et al. The relations of effortful control and reactive control to children’s externalizing problems: A longitudinal assessment. Journal of Personality. 2003;71:1179–1205. doi: 10.1111/1467-6494.7106011. [DOI] [PubMed] [Google Scholar]
  99. van Velzen CJM, Emmelkamp PMG, Scholing A. Generalized social phobia vs. avoidant personality disorder: Differences in psychopathology, personality traits, and social and occupational functioning. Journal of Anxiety Disorders. 2000;14:395–411. doi: 10.1016/s0887-6185(00)00030-x. [DOI] [PubMed] [Google Scholar]
  100. Vasey MW, Crnic KA, Carter WG. Worry in childhood: A developmental perspective. Cognitive Therapy and Research. 1994;18:529–549. [Google Scholar]
  101. Verhulst FC, Akkerhuis GW. Agreement between parents’ and teachers’ ratings of behavioral/emotional problems of children aged 4–12. Journal of Child Psychology and Psychiatry. 1989;30:123–136. doi: 10.1111/j.1469-7610.1989.tb00772.x. [DOI] [PubMed] [Google Scholar]
  102. Verhulst FC, Koot HM, van der Ende J. Differential predictive value of parents’ and teachers’ reports of children’s problem behaviors: A longitudinal study. Journal of Abnormal Child Psychology. 1994;22:531–546. doi: 10.1007/BF02168936. [DOI] [PubMed] [Google Scholar]
  103. Warner MB, Morey LC, Finch JF, Gunderson JG, Skodol AE, Sanislow CA, et al. The longitudinal relationships of personality traits and disorders. Journal of Abnormal Psychology. 2004;113:217–227. doi: 10.1037/0021-843X.113.2.217. [DOI] [PubMed] [Google Scholar]
  104. Watson D, Clark LA. Clinical diagnosis at the crossroads. Clinical Psychology: Science and Practice. 2006;13:210–215. [Google Scholar]
  105. Widaman KF, Thompson JS. On specifying the null model for incremental fit indices in structural equation modeling. Psychological Methods. 2003;8:16–37. doi: 10.1037/1082-989x.8.1.16. [DOI] [PubMed] [Google Scholar]
  106. Widiger TA. Generalized social phobia vs. avoidant personality disorder: A commentary on three studies. Journal of Abnormal Psychology. 1992;101:340–343. doi: 10.1037//0021-843x.101.2.340. [DOI] [PubMed] [Google Scholar]
  107. Younger AJ, Gentile C, Burgess K. Children’s perceptions of social withdrawal: Changes across age. In: Rubin KH, Asendorpf JB, editors. Social withdrawal, inhibition, and shyness in childhood. Mahwah, NJ: Erlbaum; 1993. pp. 215–235. [Google Scholar]
  108. Youngstrom E, Loeber R, Stouthamer-Loeber M. Patterns and correlates of agreement between parent, teacher, and male adolescent ratings of externalizing and internalizing problems. Journal of Consulting and Clinical Psychology. 2000;68:1038–1050. doi: 10.1037//0022-006x.68.6.1038. [DOI] [PubMed] [Google Scholar]

RESOURCES