Feasibility, Reproducibility, and Clinical Validity of the Pediatric Anxiety Rating Scale—Revised for Fragile X Syndrome

Abstract

Fragile X syndrome (FXS) is the most common inherited cause of intellectual disability and the most common known genetic cause of autism. FXS is associated with psychiatric impairments, including anxiety disorders. There is a paucity of well-developed measures to characterize anxiety in FXS. However, such scales are needed to measure therapeutic responses to interventions. The Pediatric Anxiety Rating Scale—Revised (PARS-R) was evaluated in 49 individuals with FXS. Feasibility, reproducibility, and clinical validity were assessed. High inter-rater, test–retest, and cross-site reliability were achieved. PARS-R scores were correlated with parent-report and physician ratings of anxiety, suggesting good clinical validity. Results were similar within gender and age subgroups. The PARS-R is a promising tool for measuring the efficacy of interventions targeting anxiety in FXS.

Fragile X syndrome (FXS) is an inherited genetic disorder characterized by intellectual deficits and emotional disabilities. It is the most common known genetic cause of intellectual disability and autism spectrum disorder (ASD; Hagerman et al., 2009). FXS affects one in 4,000 males and one in 6,000 females in all ethnic groups. FXS results from expansion of a CGG repeat sequence within the promoter region of the fragile X mental retardation 1 gene (FMR1) on the X chromosome. Whereas normal FMR1 alleles have fewer than 45 CGG repeats, alleles in individuals with FXS contain more than 200 repeats. As a result, the FMR1 promoter is hyper-methylated and silenced. This silencing leads to significantly decreased or absent fragile X mental retardation protein (FMRP, the FMR1 gene product) expression. FMRP is necessary for normal brain development and function. Thus, diminished levels of FMRP lead to intellectual disability, learning disabilities, and/or ASDs (Loesch, Huggins, & Hagerman, 2004; Tassone et al., 1999). The severity of FXS is variable because it is associated with the degree of FMR1 methylation and the resultant level of FMRP expression. Anxiety disorders, perseveration, aggression, sensory hypersensitivity, social dysfunction, and attention deficit/hyperactivity disorder are often present in FXS. Such conditions are found in affected individuals of all levels of cognitive impairment, including both males and females (e.g., Wang, Berry-Kravis, & Hagerman, 2010). However, females with FXS typically have milder intellectual and behavioral deficits than males because of protective effects from the normal gene on their non-fragile X chromosome (Wolff, Gardner, Lappen, Paccia, & Meryash, 1988).

Anxiety in FXS

Marked anxiety is a prominent clinical feature of the FXS phenotype, even though the exact prevalence of anxiety varies among studies (e.g., Cordeiro, Ballinger, Hagerman, & Hessl, 2011). Although anxiety disorder criteria given in the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) may not always be easily applied to individuals with intellectual disability and developmental delays as seen in FXS, adaptations to the diagnostic process, such as the Diagnostic Manual - Intellectual Disability (DM-ID) (Fletcher, Loschen, Stavrakaki, & First, 2007) and the text revision of the fourth edition of the DSM, are appropriate and can lead to more accurate and valid diagnostic decisions. For example, the DSM diagnosis of social phobia requires that the person recognizes that the fear is excessive or unreasonable, and in several diagnostic interviews, symptoms often depend on verbal statements by the patient that they feel “embarrassed” or “afraid that others will laugh or make fun of you.” In children, the recognition of excessive fear may be absent, and in those with intellectual disability (ID), social anxiety may clearly be present without the patient’s ability to verbalize the concept of embarrassment. Cordeiro et al. (2011) carried out DSM-IV diagnostic interviews (American Psychiatric Association, 1994) with the parents of 97 individuals with FXS between the ages of 5 and 33 years, showing that 86.2% of males and 76.9% of females met the criteria for an anxiety disorder, with social phobia (36.5%) and specific phobia (59.6%) being the most commonly diagnosed. When the above-mentioned adaptations were used for the rate increased to 58.3%. Rates of the other DSM-IV anxiety disorders for the sample with FXS are reported in Cordeiro et al. (2011). Other manifestations of anxiety in FXS include difficulty maintaining conversation, selective mutism, avoidance, withdrawal, limited eye contact, and nail-biting behavior (e.g., Freund, Reiss, & Abrams, 1993; Hall, Lightbody, Huffman, Lazzeroni, & Reiss, 2009; Keysor & Mazzocco, 2002; Lesniak-Karpiak, Mazzocco, & Ross, 2003; Levitas, 1996; Mazzocco, Baumgardner, Freund, & Reiss, 1998; Roberts, Weisenfeld, Hatton, Heath, & Kaufmann, 2007; Sullivan, Hooper, & Hatton, 2007; Wang et al., 2010). Elevated levels of anxiety are documented through clinical presentation (e.g., Freund et al., 1993; Mazzocco et al., 1998), parent-report (Lachiewicz & Dawson, 1994), and behavioral observations during social interactions (e.g., Lesniak-Karpiak et al., 2003; Sudhalter & Belser, 2001).

Anxiety Outcome Measures

There is extensive use of medications to manage anxiety in approximately 50% of individuals with FXS (Amaria, Billeisen, & Hagerman, 2001; Berry-Kravis & Potanos, 2004; Cordeiro et al., 2011). Specifically, anxiety symptoms are often treated with stimulants, selective serotonin reuptake inhibitors (SSRIs), or other medications such as antipsychotics (Amaria et al., 2001; Berry-Kravis & Potanos, 2004; Hagerman, Lauterborn, Au, & Berry-Kravis, 2012; Hagerman, Fulton, Leaman, & Riddle, 1994). Despite clinical impressions suggesting that up to 70%–80% of patients show improvements when medicated, there is a lack of controlled trials examining the true efficacy of these treatments (Berry-Kravis, Sumis, Hervey, & Mathur, 2012). This gap is perpetuated by the absence of an anxiety outcome measure that is specifically validated for use in the FXS population.

Without a valid outcome measure for anxiety in FXS, treatment recommendations lack empirical support and rely predominantly on clinical gestalt. Since much of anxiety is expressed only as internalizing symptoms, validity and consistency can be difficult to assess in lower functioning and nonverbal populations (Matson, Smiroldo, Hamilton, & Baglio, 1997). Common methods to assess anxiety include self-report, third-party report by parents and teachers, and clinician observations of patients during role playing activities and interactions with parents (e.g., see review of pediatric anxiety assessments in Greenhill, Pine, March, Birmaher, & Riddle, 1998). Some measures that were previously employed for use with FXS (Table 1) include the Child Behavior Checklist (CBCL; Achenbach, 1991; Lachiewicz, 1992), the Childhood Symptom Inventory-4 (Sullivan et al., 2007), the Multidimensional Anxiety Questionnaire (MAQ; Lachiewicz et al., 2010), the Anxiety Disorders Interview Schedule (ADIS-IV; Cordeiro et al., 2011), the Anxiety, Depression, and Mood Scale (ADAMS; Cordeiro et al., 2011; Esbensen, Rojahn, Aman, & Ruedrich, 2003; Rojahn, Rowe, Kasdan, Moore, & van Ingen, 2011), and the Aberrant Behavior Checklist-Community Edition (ABC-C; Aman, Singh, Stewart, & Field, 1985; Aman, Burrow, & Wolford, 1995; Sansone et al., 2011). Clinicians may also provide a global assessment of anxiety by rating patients on a Clinical Global Impression-Severity (CGI-S) scale. Existing rating scales have both strengths and weaknesses as described in Table 1. These commonly used scales target many problem behaviors in FXS, but many do not fully cover the specific anxiety symptoms that are most common in FXS. For example, the ABC-C is used extensively in clinical trials (e.g., Berry-Kravis et al., 2006; Berry-Kravis et al., 2008; Jacquemont et al., 2011; Paribello et al., 2010), but it lacks a pure anxiety scale. Rather, the ABC-C is usually used to assess behaviors such as irritability. Additional limitations to these measures are a mismatch between the items on the scales and the manifestations of anxiety in a cognitively impaired population, reliance on parent-ratings, or, if clinician-rated, a requirement for extensive clinical training, experience, and extended time to administer the test. In order to properly evaluate the efficacy of standard prescription medications, new targeted pharmaceutical treatments, or behavioral interventions, improved anxiety assessments are needed for use in FXS.

Table 1.

Comparison of the PARS-R to Other Measures Previously Used in FXS

Scale	Format & length	Study	Normative samples	Reliability & validity	Strengths	Weaknesses
Pediatric Anxiety Rating Scale/ Pediatric Anxiety Rating Scale-Revised	Parent/Child Clinician-rated interview(30–45 min.)	Riddle et al.,2002	128 children, ages 2002 6–17 years(DSM-IV diagnosis of SoP, SAD, or GAD)	ICC = .97;α= .64; ICC = .55; r = .61 (CGI-S)	Used with pediatric populations and with developmental disabilities; clinically-sensitive, wide-range of anxiety symptom coverage, continuous distribution of scores; originally intended to be an outcome measure; sensitive to psychopharmacological treatment of anxiety	Not originally intended for populations with developmental disabilities; does not render sub-categorization of anxiety
		Storch et al.,2012	72 children, ages 7–17 years(diagnosis of autism, Asperger’s syndrome, or PDD-NOS & GAD,SAD, SA, or OCD)	ICC = .86; α= .59; ICC = .83
Anxiety, Depression, and Mood Scale	Caregiver-rated questionnaire (15 min.)	Esbensen et al.,2003	265 subjects, ages 10–79 years of age (diagnosisof MR)	ICC = .75–.83; α= .48 (Full scale); α= .37–.62(subscales);ICC = .81 (Full scale); ICC 5.72–.83 (subscales)	Established data in FXS; validated against DSM-IV; normed and developed forintellectual disability	Does not have anestablished track record, limited anxiety symptom coverage, not intended as outcome measure
Anxiety Disorders Interview Schedule-IV	Parent/Child Clinician-rated interview (1 hr.)	Silverman, Saavedra,& Pina,2001	62 children, ages 7–16 years and their parents (exclusion criteria:developmental delays and severe psychopathology)	ICC =.78–.96 (Symptom scales); κ = .80 to .92(Combined diagnoses of these disorders); κ = .63–.80 (child-only interview); κ = .65–.88(parent-only interview); r = .94(parent); r = .92(child)	Used with pediatric populations and with developmental disabilities; used in assessment of DSM-IV anxiety disorders	Assesses anxiety over the past year and therefore not an effective outcome measure; not a continuous measure of severity
Aberrant Behavior Checklist-Community Edition	Caregiver-rated questionnaire (15 min.)	Aman et al.,1995	927 institutionalized, profoundly developmentally delayed adolescents and adults	ICC −.86–.94; α = .96–.99	Used with pediatric populations and with developmental disabilities; used in many FXS clinical trials; offers ratings of behaviors that may be present when anxious(e.g., inappropriate speech); sensitive to treatment effects	Does not have anxiety subscales; many symptoms not relevant for higher functioning patients without ID
Child Behavior Checklist	Caregiver-rated questionnaire (20 min.)	Achenbach,1991	2,368 nationally-representative children, ages 4–18 years without disabilities	ICC = .92–.96; α = .83–.92	Used with pediatric populations; provides broad-band internalizing and externalizing scores and narrow-band scaled scores	Not generally used as outcome measure; many itemsinappropriate for developmental/ intellectual disability; not specifically normed in ID
Clinical Global Impression-Severity Scale	Clinical impression (10 min.)	Forkman et al.,2011	31 inpatient adults with major depressive disorder	ICC = .22–.24	Used with pediatric populations and developmental disabilities; used in controlled treatment studies	No specific interviewer guide available so response format can be ambiguous
Multidimensional Anxiety Questionnaire	Self-report questionnaire (10 min.)	Reynolds,1999	More than 1,800college students,600 community adults, 407 psychiatric outpatients diagnosed with anxiety-related or other disorders	α = .96 (Total scale); α = .88 to .91(subscales); α = .95(Total scale); α =.90–.93 (subscales)	Used with women with pre-mutation FXS; provides global assessment of DSM-IV anxiety symptoms in four domains; provides a cutoff score for clinically relevant anxiety; easy to administer/score screener; clinically valid	Not used with pediatric populations; self-report may not be possible for some individuals with developmental disabilities; onlynormed for 18 and older

Pediatric Anxiety Rating Scale

Experts convened at an NIH Consensus Meeting, “Outcome Measures for Clinical Trials in Children with Fragile X Syndrome Part II” in Bethesda, Maryland, (November 2009) to review a wide range of potential measures of behavior and psychiatric symptoms for fragile X treatment studies. The Pediatric Anxiety Rating Scale-Revised (PARS-R; Riddle, Ginsburg, Palapattu, & Walkup, 2004; Riddle et al., 2002) was identified as a potentially useful tool based on its face validity for FXS and its sensitivity in prior controlled pharmacological trials in pediatric anxiety disorders (Birmaher et al., 2003; Geller et al., 2007; Walkup et al., 2001). The structure of the original and revised PARS is a clinician-rated parent or caregiver interview administered by a clinical or research professional. The interviewer first reviews a list of anxiety symptoms (50 items on the PARS, 61 items on the PARS-R) with the primary caregiver(s) to identify behaviors that occurred within the past week. The PARS-R provides broad coverage of separation anxiety, social phobia, and generalized anxiety. Symptoms are further categorized into Social Interactions or Performance Situations, Separation, Generalized, Specific Phobia, Panic Symptoms/Physical Signs, Obsessive-Compulsive, Health/Illness Concerns, and Other. This range of measurement is relevant to the types of anxiety experienced by individuals with FXS (Cordeiro et al., 2011). The last item on the scale, Other, allows the caregiver to indicate any anxiety symptoms that were not previously mentioned on the symptoms checklist. If the caregiver endorses additional symptoms, these symptoms are later included in the rating of frequency and severity of anxiety on seven dimensions. The interviewer assesses the severity by quantifying the frequency and degree of interference and avoidance in family, school (or work), and community settings.

Each of seven severity items is scored on a scale of 1 to 5, with 5 being the most severe and frequent. The final score is the sum of five or seven of the seven severity items, which comprise a five- or seven-item severity score, respectively. The five-item scale is recommended for use during treatment studies, while use of the seven-item scale is recommended for general assessment per the instructions (Riddle et al., 2004; Riddle et al., 2002). The five-item scale excludes Overall Number of Anxiety Symptoms and Severity of Physical Symptoms of Anxiety. In early studies of the PARS, children with anxiety would typically score high on Overall Number of Anxiety Symptoms both before and during treatment. It was also typically more relevant regarding how the medication affected the severity of anxiety, rather than the number of symptoms present. Since quantity may not be directly related to anxiety severity, it may be omitted. Severity of Physical Symptoms of Anxiety may be skewed because of side effects of medications (e.g., SSRIs) that may be confused with physical symptoms of anxiety. Therefore, this item may also be omitted when assessing treatment outcomes. Once totaled, the five-item severity scale ranges from zero to 25. As guidelines, a score of 10 suggests “mild but clinically meaningful” anxiety, whereas individuals who present with scores of 20 or above are “severely” affected by anxiety (Walkup et al., 2001). The seven-item scale has a maximum score of 35. To date, there are no published guidelines for interpreting the seven-item scale. However, the authors of the PARS-R indicated that a score of 18 represents significant anxiety (Y. Yoon, personal communication, May 5, 2010).

Although the detailed symptoms checklist accounts for symptoms of various types of anxiety, the PARS and PARS-R final scores do not render differential DSM-IV anxiety diagnoses. An advantage of the PARS and PARS-R is the clinician-rated interview format. This structure is preferable over other parent-rating questionnaires because it limits variability that may be introduced by misinterpretation of item intent or caregiver biases. There is still a reliance on subjective parent-report. Through structured questioning, though, a skilled administrator can clarify intent of the items and distill a clinically informed rating of anxiety.

Validation of the PARS

There are currently no controlled validation studies establishing the psychometric properties of the PARS-R. The PARS was assessed for interrater reliability, internal consistency, and test–retest reliability (Riddle et al., 2002). Generally, values .60–.74 are considered good for reliability and consistency, and values ≥ .75 are considered excellent (Cicchetti & Sparrow, 1981; Hermans, van der Pas, & Evenhuis, 2011). However, when internal consistency is very high, the items may be entirely redundant. In a study of children and adolescents (ages 6–17 years) with anxiety disorders, the original PARS scale demonstrated high inter-rater reliability (.97), while test–retest reliability (.55) and internal consistency (.64) were lower, but acceptable. Lower internal consistency was suggested to be due to the presentation of coping strategies during testing that affected the experience of anxiety symptoms. The PARS was also examined for its construct validity by correlating its scores with scores on other anxiety measures (CGI-S scale, Hamilton Anxiety Scale, CBCL). Validity correlations were .27–.55 with these scales when used to assess children with anxiety disorders.

Although the PARS was not originally intended for use in developmental disabilities, it was recently validated for use with children and adolescents (7–17 years old) with ASDs (Storch et al., 2012). As measured by intraclass correlation coefficients (ICC), the PARS exhibited strong inter-rater and test–retest reliability (ICC = .86 and .83, respectively). Internal consistency was adequate (.59). Scores on the PARS also exhibited moderate to strong convergent validity with CGI, the ADIS-IV-C/P (Silverman & Albano, 1996), parent-report on the Multidimensional Anxiety Scale for Children (Langer, Wood, Bergman, & Piacentini, 2010), and internalizing items on the CBCL (r ≥ .35, p < .01, all comparisons). These findings are encouraging for the potential use of the PARS or PARS-R in FXS populations. However, currently, neither the original nor the revised scale is validated in children or adults with FXS.

Differences Between the PARS and the PARS-R

The PARS-R includes new symptoms and anxiety categories and reorganizes others. The Social, Separation, and Specific Phobia categories remain unchanged from the original PARS. Two new sections, Obsessive-Compulsive and Health/Illness Concerns, were added to the 61-item scale. The Generalized section includes a new item about competency. The Acute Physical Signs and Symptoms section was renamed Panic Symptoms and a new item regarding fear of death was added. Several items were relocated to the Other section, while two items from the Other section were moved into the Panic Symptoms section, and a new item about headaches was added. To quantify the severity of symptoms, the PARS-R distinguishes between duration and severity for several items, whereas the PARS did not. Finally, Severity Items 6 and 7 on the PARS-R indicate interference in relationship and performance, whereas the original PARS allowed for one or the other. The PARS-R is currently in use in clinical trials even though there are no psychometric or validation studies of the PARS-R (M. Riddle, personal communication, December 27, 2012).

Use of the PARS-R to Assess Adults With FXS

The PARS-R may have utility in assessing anxiety in adults with FXS, even though it was originally intended for pediatric populations. First, adult self-rated anxiety scales are not useful in the FXS population since adults with intellectual disability often lack insight or the ability to express themselves adequately for reliable reporting of symptoms (Matson et al., 1997). Furthermore, the items on a pediatric scale may be relevant to adults with FXS provided that fully affected individuals with FXS typically have mental ages in the child to adolescent range (Turk, 2011). The types of anxiety experienced in FXS are expected to occur throughout the lifespan and are not known to change over time in individuals with intellectual disability; (Horovitz et al., 2011). Thus, the PARS-R has the potential to apply to both children and adults with FXS, and a caregiver- or clinician-rated form is needed.

Goals of the Study

The goals of this pilot study were to explore the (a) feasibility of administering the PARS-R to parents or caregivers of individuals with FXS, (b) inter-rater reliability, (c) preliminary cross-site reliability, (d) test–retest reproducibility, and (e) clinical validity in a cohort of children and adults with FXS.

Method

The study was approved by the Institutional Review Boards at both universities. For the purposes of this study, the term caregiver refers to either parental or other caregiver respondents. All patients and their caregivers signed informed consent to participate. Subjects with FXS were identified and recruited through the Rush University Medical Center (RUMC) Fragile X Clinic and Research Program (program director E.B.-K.) or through the Fragile X Research and Treatment Center at the UC Davis MIND Institute (clinical psychologist D.H.). Interviews were conducted at the Universities’ main campuses, satellite clinics, or in the families’ homes. All original interviews were video-recorded for later use in inter-rater reliability analyses.

Participants

Forty-three data sets were collected at RUMC, and six data sets were collected at UC Davis. The administration of the PARS-R at two sites allowed for preliminary assessment of cross-site reliability. All participants were required to have FMR1 DNA testing demonstrating the full mutation associated with FXS. The study included 49 total participants with FXS who were between 5 and 35 years of age (M = 16.06, SD = 7.19) and who had varying functional levels. Of the 49 enrolled, 33 were males (n = 21 < 18 years old; n = 12 ≥ 18 years) and 16 were females (n = 8, ≥ 18 years old; n = 8 ≥ 18 years). For descriptive purposes, the most recent IQ scores for subjects were extracted from clinical and research charts at RUMC to characterize the functional level of the cohort. Twenty-seven total IQ scores (21 males, 6 females) were available (group M IQ = 53.26, SD = 13.95; males M = 50.95, SD = 13.68; females M = 61.33, SD = 12.74). There were no significant differences in mean IQ between males and females (U = 33.0, p = .08; Cohen’s d = 0.79). However, it is widely known that, as a group, females with FXS have substantially higher IQ scores than males. The imbalance in number of males versus females in the subset of participants for whom IQ data was available and a broad range of IQ assessments utilized likely contributed to the lack of an expected difference in IQ. The pattern of means in our male and female samples is consistent with patterns found in FXS. Additionally, the significance level and effect size together support that the sample was not likely powered adequately to detect these differences.

Test–retest data were available from 38 participants (25 males, 13 females) from the RUMC site. In most cases, the PARS-R was readministered 3 to 8 weeks after the original interview (M = 8.23 weeks, SD = 5.64 weeks, range 2.9–21.9 weeks). One child’s post-test date was inadvertently not recorded and therefore unavailable. Variability in re-administration time was due to caregiver availability. Participants did not change medications or therapy regimens or initiate new behavioral treatments between the two testing sessions. Only subjects who adhered to this requirement were eligible for retesting. Twelve of the follow-up interviews for test–retest reliability were conducted over the phone in order to accommodate families.

Anxiety Assessments

The same assessment measures were used with both the child and adult populations. Specifically, anxiety was assessed with the (a) PARS-R, (b) ADAMS and ABC-C, and (c) CGI-S.

Clinician-rated PARS-R interview with caregiver.

The PARS-R was administered to caregivers by either a first-year medical student research assistant (JY), a doctorate-level researcher with training in autism and FXS (NRP), or a clinical psychologist (DH). The medical student was trained in the administration of the PARS-R according to instruction guidelines and under the observation of the doctorate-level researcher. As part of the PARS-R training, the medical student reviewed DSM-IV-TR (American Psychiatric Association, 2000) criteria for anxiety disorders and several papers describing anxiety in FXS. The administration and scoring instructions were reviewed and discussed among all of the study authors. Prior to the onset of the study, the doctorate-level researcher administered the PARS-R to two families while the medical student observed the live administrations. Subsequently, the medical student conducted four interviews that she video-recorded. The doctorate-level researcher then reviewed the videos and scored the PARS-R. In each of these circumstances, both the researcher and medical student rated the interviews independently and then compared results. Where discrepancies occurred, they discussed the items and a consensus score prevailed.

Recommendations by the authors of the PARS and PARS-R suggest use of the five-item scale during treatment studies based on anecdotal evidence and use of the seven-item scale for general assessments. Since no studies have explicitly established a rationale for the use of the five-or seven-item severity scales and the scales have not been validated in individuals with FXS, the psychometric properties of Total Number of Items Endorsed and both the five- and seven-item severity scales’ total scores were evaluated.

Caregiver ratings on the ADAMS and ABC-C.

The second means of assessing anxiety was through caregiver ratings on established measures in FXS, the ADAMS and the ABC-C. Both scales have been used historically to assess behavioral improvements in FXS during clinical evaluations (Cordeiro et al., 2011; Sansone et al., 2011) and after treatment and represent a useful benchmark for comparison and validation in this study. For the ADAMS and the ABC-C, caregivers were invited to complete the questionnaires following administration guidelines as published. Sub-scores on specific domains were assessed.

CGI-S.

Thirdly, for participants at the RUMC site, a CGI-S for anxiety was obtained independently from the treating physician (E.B.-K.). For the CGI-S, one score ranging from 1 to 7 was rendered based on clinical impression of the participant’s current functioning in the anxiety domain.

Statistical Analyses

Distribution and severity of scores on the seven dimensions on the PARS-R were evaluated. Alpha coefficients assessed latent traits within and internal consistency of the scale. Feasibility of administration of the test was assessed by determining whether the interviews were successfully completed without limiting problems. Specifically, administrators made qualitative judgments with regard to whether caregivers could accurately complete the interview. Inter-rater and cross-site reliability were assessed using a subset of video-recorded administrations selected at random (12 from RUMC, two from UC Davis). Twelve datasets were coded independently by all three raters, while fourteen datasets were independently coded by two raters (N.M.R.-P., D.H.). Kappas were used to assess agreement between items 1–61 and seven severity items. ICC were used to assess inter-rater reliability and preliminary cross-site reliability. Test-retest reliability was evaluated by comparing PARS-R Total and dimension scores from session one and session two using interclass correlation coefficients. Clinical convergent validity was determined using Pearson’s correlations between scores on the PARS-R with (a) caregiver-report scores on the ADAMS, (b) caregiver-report scores on the ABC-C, and (c) clinician-report on the CGI-S for anxiety. Divergent validity was evaluated by determining whether lower correlations were observed between PARS-R scores and measures of constructs other than anxiety.

Results

Internal Validity

Descriptive statistics.

For five individuals, zero symptoms were endorsed during the first administration. The minimum, maximum, mean, and standard deviation are reported for the total number of items endorsed and severity indices for the entire sample at the first administration (T1) and the smaller sample at a second administration (T2) in Table 2. Data are also reported for the young (age < 18) and adult groups (age ≥ 18) and for males and females in Table 3. Although the means suggest fewer symptoms were endorsed in the adult group, there were no significant differences in Total Number of Items Endorsed or Severity Indices between age groups (t ≤ 1.94, p ≥ .09, all comparisons) or between males and females (t ≤ .75, p ≥ .46, all comparisons).

Table 2.

PARS-R Score Range (Minimum and Maximum Number of Endorsements), Total Number of Symptoms Endorsed (Raw Score Total out of 61 Items), and Severity Score Raw Totals for the Five-Item and Seven-Item Scales (Maximum 35 and 25, Respectively) and Standard Deviation (SD) at Time 1 (n = 49) and Time 2 (n = 38)

	Minimum	Maximum	Mean (SD)
Time 1
Total number of symptoms endorsed	0	25	7.51 (6.86)
5 item scale	0	21	8.41 (5.63)
7 item scale	0	30	13.47 (8.16)
Time 2
Total number of symptoms endorsed	0	22	5.08 (5.94)
5 item scale	0	22	7.50 (5.50)
7 item scale	0	31	12.11 (7.74)

Table 3.

PARS-R Scores (Mean, SD) by Subgroup

Group	Time 1				Time 2
	n	Total items endorsed	Five-item scale	Seven-item scale	n	Total items endorsed	Five-item scale	Seven-item scale
Young (0–17 years)	29	8.34 (7.17)	9.52 (5.17)	14.93 (7.70)	20	7.15 (5.82)	8.05 (4.77)	12.95 (6.93)
Adult (≥ 18 years)	20	6.30 (1.42)	6.80 (6.00)	11.35 (8.53)	18	5.89 (6.27)	6.89 (6.29)	11.17 (8.66)
Male	33	7.61 (7.21)	8.55 (5.30)	13.58 (7.67)	25	5.92 (4.38)	7.44 (4.40)	12.08 (6.13)
Female	16	7.31 (6.30)	8.13 (6.43)	13.25 (9.34)	13	7.77 (8.36)	7.62 (7.38)	12.15 (10.47)

Internal consistency.

Both the five- and seven-item scales exhibited strong internal consistency (Cronbach α = .89 and .92, respectively). Replicating previous findings, correlations between the seven severity items and the five-item total score were calculated. All seven severity items were significantly correlated with the PARS-R five-item total score (.40 ≤ r ≤ .92, p ≤ .005, all comparisons).

Feasibility

Examiners’ qualitative assessment of the training process for the PARS-R indicated that the PARS-R was relatively easy and rapid to learn. The measure was also fairly quick to administer. Once reliable, the average administration of the PARS-R required only 30–45 minutes. Because the PARS-R is a clinician-rated parent interview and the patient was not required to complete any assessments, there was no refusal to participate on the part of the patient. Caregivers were able to easily report on presence/absence and severity of anxiety symptoms, with the exception of symptoms like paresthesias or feeling of dread, which are symptoms most patients may not be able to verbalize.

Reliability

Inter-rater and cross-site reliability.

Administrations and coding of the PARS-R by a trained medical student, a doctorate-level researcher, and clinical psychologist were reliable and consistent within and across sites (Table 4).

Table 4.

Inter-Rater Reliability

Reliability	Scale analysis 61 symptoms checklist items (k)	61 symptoms checklist (κ)	Five-item severity index (α)	Seven-item severity index (α)
Inter-rater (RUMC)
J.Y.:N.M.R.-P.	.87	.93	.98	.99
Cross-site
J.Y.: N.M.R.-P.:D.H.	-	.96	.97	.97
J.Y.:D.H.	.90	.95	.96	.93
N.M.R.-P.:D.H.	.85	.92	.96	.94

Test-retest reliability.

Within the dataset of 38 participants who had retest data available, there were no significant differences from T1 to T2 for the Total Number of Items Endorsed (paired Student’s t tests: t(37) = 1.49, p = .15, five-item total score, t(37) = 0.53, p = .60, or the seven-item total score, t(37) = 0.93, p = .36). There were no differences from T1 to T2 for either age or gender subgroups, t ≤ 1.59, p ≥ .12, all comparisons. Furthermore, the total number of symptoms endorsed and severity indices showed excellent test–retest reproducibility as measured by ICC for the entire group and all subgroups (Table 5).

Table 5.

Test–Retest Intraclass Correlations

Group	n	Total items endorsed	Five-item scale	Seven-item scale
All subjects	38	.90	.86	.86
Young (0–17 years)	20	.90	.79	.85
Adult (≥ 18 years)	18	.90	.90	.90
Male	25	.90	.76	.81
Female	13	.94	.94	.94

p ≤ .001, all comparisons.

Clinical Relevance

Pearson’s correlations were evaluated between the PARS-R severity indices, ADAMS (Table 6), ABC-C (Table 7), and CGI-S (Table 8). The PARS-R five- and seven-item severity indices demonstrated the strongest, and most significant, correlation with the Generalized Anxiety subscale on the ADAMS. Significant but weaker relationships were also identified with the Obsessive/ Compulsive Behavior, Social Avoidance, and Manic/Hyperactive Behavior subscales. Within the young subgroup, a significant correlation was found with respect to the Depressed Mood subscale; no other relationships were found with the Depressed Mood subscale. Results were largely identical when evaluating the young and adult subgroups and the male and female subgroups (data not shown).

Table 6.

Correlations (r Value) Between PARS-R (T1) and ADAMS Subscales

	Manic/hyperactive behavior	Depressed mood	Social avoidance	Generalized anxiety	Obsessive–compulsive behavior
Five-item scale	.44*	.25	.41*	.60***	.56**
Seven-item scale	.38*	.26	.47**	.61***	.54**

^*p ≤ .05

^**p ≤ .01

^***p ≤ .001.

Table 7.

Correlations (r Value) Between PARS-R (T1) and ABC-C Subscales

	Irritability	Lethargy	Stereotypy	Hyperactivity	Inappropriate speech
5 item scale	.57***	.30	.23	.51**	.21
7 item scale	.60***	.33	.27	.49**	.17

^**p ≤ .01

^***p ≤ .001.

Table 8.

Correlations (r Value) With CGI-S for Anxiety

Group	5 item scale	7 item scale
Entire sample (n=43)	.60***	.55***
Young (0–17y)	.97***	.46*
Adult (18+)	.65**	.60**
Male	.96***	.66***
Female	.53*	.47

^*p ≤ .05

^**p ≤ .01

^***p ≤ .001.

On the ABC-C, the PARS-R severity scores were best correlated with Irritability and Hyperactivity. When evaluating subgroups, the PARS-R again showed strong relationships with Irritability and Hyperactivity in the young group and males, while the strongest correlation was with Social Withdrawal in adults and females.

Again, the five - and seven-item severity indices on the PARS-R exhibited strong and significant correlations with the CGI-S for anxiety within the sample from the RUMC site altogether and within the young and adult subgroups and male participants. This relationship was moderate, though less significant, within female participants.

Preliminary item analysis.

As an initial qualitative analysis of symptom relevance, specific symptom endorsements by group were examined. Eight items were endorsed by at least 25% of the entire patient population: Has fear of and/or avoids talking with a stranger (Item 3); Has fear of and/or avoids talking on the phone (Item 4); Clings to parent, or follows parent around the house (Item 19); Irritability (Item 25); Restlessness or feeling keyed-up or on edge (Item 28); Crying spells when in anxiety-provoking situations (Item 54); Temper tantrums when in anxiety-provoking situations (Item 55); and Keeps distance from other people (Item 57). Meanwhile, nine items were never endorsed: Reluctant or refuses to eat in public (Item 7), Fear of dying (Item 42), Chest pain or discomfort (Item 43), Paresthesias (Item 44), Derealization (Item 45), Saving miscellaneous items for fears of bad things happening (Item 49), Has fears or worries about having a serious disease/illness (Item 50), Pains in his/her body that are feared to be serious (Item 51), and Headaches (Item 61).

Discussion

This study demonstrates the feasibility, reliability, and validity of the PARS-R when evaluating the presence and severity of anxiety symptoms in individuals with FXS. The results of this study demonstrated internal consistency of the PARS-R that superseded results from previous validation studies of the PARS. One potential explanation for this result is that the revisions made to the PARS-R improved its consistency. However, a direct comparison between the PARS-R and PARS was not conducted.

The staff on this pilot study reported the process for becoming reliable on the PARS-R as fairly rapid, especially compared with training procedures for other clinician-administered measures (e.g., ADIS-IV). The PARS-R did not provide training requirements for its use. Training techniques outlined in this study serve as suggested guidelines for others to adopt or modify. Specifically, those who have experience with the FXS phenotype and manifestations of anxiety in children and those with intellectual disabilities may be qualified to administer the PARS-R for those with FXS. Either a clinician or a research psychologist (or equivalent degree) may train less-certified staff. It is recommended that training include education about the manifestation of anxiety in FXS for those who are less familiar with the presentation of clinical anxiety. Administrators preferred the clinician-rated caregiver interview because it offered an opportunity to clarify the intent of items, which is not possible with a questionnaire.

Similar to the validation of the original scale, the PARS-R exhibited high inter-rater, test–retest, and preliminary cross-site reliability in our sample (Riddle et al., 2002; Storch et al., 2012). Each of the scores (total items endorsed, five-item total score, and seven-item total score) showed good test–retest reproducibility. The scale functioned largely similarly for both males and females.

The PARS-R showed excellent convergent validity based on significant correlations with other anxiety measures, including both caregiver-and clinician-report of anxiety (e.g., generalized anxiety subscale on the ADAMS and the CGI-S). The seven-item scale, as measured within females, was not significantly related to the CGI. However, the correlation was in the same direction and moderate in strength. The absence of a significant relationship on this scale within females was likely a function of the small number of data points available for this set of analyses and not an indication of the sensitivity of the PARS-R in females.

While the convergent validity data was supportive of the PARS-R, the PARS-R showed modest evidence of divergent validity as evidenced by weaker correlations with non-anxiety comparison measures (e.g., lethargy subscale on the ABC-C). The lack of divergent validity is not entirely unexpected. These results parallel what was found in the study of the PARS in individuals with ASDs (Storch et al., 2012). Furthermore, anxiety can contribute to or be associated with a range of other symptoms in persons with FXS, such as over-arousal, which might lead to hyperactivity, irritability, or aggression.

The PARS-R offers some advantages to other anxiety measures used in clinical trials. Pilot data from the current study demonstrate the utility of the tool to assess anxiety in pediatric and adult populations with FXS. Adoption of the PARS-R for adults with FXS would represent a major advantage over existing self-report anxiety assessments that offer only limited utility in a developmentally disabled and intellectually challenged population (Matson et al., 1997). The results of this study also support that, with training and standardization across administrators and sites, clinicians, highly trained clinical trial coordinators, research assistants, or graduate-level students with hands-on experience and knowledge of the FXS behavioral phenotype can acquire reliable and valid PARS-R data. Since many clinical trial studies have non-physician, non-psychologist research staff or clinical trial coordinators, this aspect is a potential benefit of its use in such studies. Another advantage to the PARS-R is that it may be completed in less than one hour. The PARS-R provides a continuous measure of DSM-IV-based symptoms spanning separation anxiety, social phobia, and generalized anxiety without requiring a much more extensive psychiatric interview.

Results from this initial exploration of the PARS-R addressed the types of anxiety identified by families of individuals with FXS. Specifically, parents most frequently noted symptoms of social anxiety, separation anxiety, irritability, and tantrums. Other prevalent concerns, such as obsessive–compulsiveness and gastrointestinal issues, were adequately covered by the symptoms checklist. Importantly, few caregivers indicated any additional anxiety symptoms present within the scope of the past week. That only 9 out of 61 items were never endorsed further supports the relevance of the PARS-R for assessment in FXS. Therefore, the PARS-R appears to capture the majority of anxiety symptoms present in patients with FXS. An in-depth factor analysis of the PARS-R and other anxiety related symptoms in larger samples of individuals with FXS may identify a more comprehensive and targeted item set for measurement with this population.

Limitations and Future Directions

The PARS-R exhibits many strengths as an anxiety assessment tool, but also some limitations. The clinician-rated interview format allows administrators to extract information about daily events to more accurately assess the frequency and interference of anxiety behaviors. However, since the PARS-R relies on parent-report, rather than on direct assessment, there is still likely to be subjectivity and response bias. This subjectivity is somewhat mitigated by the fact that an administrator with experience in FXS and anxiety can help facilitate the caregiver to provide examples and discussion leading to more reliable and valid responses. As with other caregiver-reported measures of anxiety, the PARS-R requires caregivers to infer some symptoms that are internally experienced and may not be manifested in behavior or verbal expression. Especially in populations of individuals who have intellectual disability or who are nonverbal, it may not be possible for parents to accurately report on the presence and severity of some symptoms, such as derealization or intrusive thoughts, because of the inability of individuals to convey this information to parents (Sullivan et al., 2007). Another potential limitation of the utility of this interview is that it does not provide an indication of specific subtype of anxiety experienced by the individual.

There are additional limitations beyond the measure itself. Because it was outside the scope of this initial study, a full chart review was not included. Therefore, the analysis of cognitive function was limited by the number of participants who had IQ scores available. Furthermore, a broad range of IQ assessments were used to establish IQ. Without a full chart review or current clinical assessment, it was not possible to confirm whether patients met formal DSM-IV criteria for anxiety. Thus, the classification of cognitive abilities and anxiety diagnoses in our sample was limited. Since the study included a relatively small sample, caution is advised when interpreting the results of this study. The results may not be fully representative of the population of individuals affected by FXS.

Based on this initial study, additional steps may be needed to fully establish the use of the PARS-R in studies of FXS. In the present study, interviews took place largely over the summer. It was important to establish whether the past week was “typical” for the patient. Thus, there should be some examination into whether the time period of the “past week” is an appropriate timeframe for most patients. Also, this study focused on a clinician-rated interview with the caregiver. The PARS-R also includes administration of an interview with the individual with FXS. This variation is not believed to be a viable option with this population due to lower intellectual functioning, verbal impairments, and/or lack of insight (Cordeiro et al., 2011).

Prior evidence demonstrated that the PARS is sensitive to treatment effects (Birmaher et al., 2003; Geller et al., 2007; Riddle et al., 2002; Walkup et al., 2001). Therefore, the PARS-R may be useful as an outcome assessment for clinical trials or behavioral interventions in FXS. Future controlled trials should examine the sensitivity to change of the PARS-R in FXS.

The high reliability achieved within this study is promising. However, cross-site reliability was only assessed with a limited number of PARS-R administrations. In order to be used in large-scale, multi-site clinical trials that span state and national boundaries, it is important to conduct larger-scale validation studies of cross-site reliability and phone administration techniques. Additionally, 12 PARS-R interviews were administered over the telephone for the convenience of families who otherwise would have needed to travel long distances for the study. Although a review of the preference for and utility of phone interviews in general suggested mixed results (Bowling, 2005), examination of the statistical validity of phone interviews was outside the scope of the present study. Other clinician-rated parent interviews, such as the Autism Diagnostic Interview-Revised (Rutter, Le Couteur, & Lord, 2003), are validated for administration via a telephone interview (Ward-King, Cohen, Penning, & Holden, 2010). Therefore, the further validation of the PARS-R as a phone interview would be valuable in extending its utility as an outcome measure in clinical pharmaceutical and behavioral trials.

Conclusions

The main goal of this pilot study was to explore the reliability, validity, and potential utility of the PARS-R as measure of anxiety in FXS. Based on the data presented here, the PARS-R shows promise as a potentially sensitive outcome measure for anxiety in FXS clinical trials. Extensions of this study are now needed to evaluate the validity of the measure in larger FXS cohorts and to examine the sensitivity of the PARS-R before and after treatment in clinic-based settings and in FXS clinical trials. Establishing use of the scale in FXS cohorts could serve as a model for other individuals with an intellectual disability or ASD. Further validation of the PARS-R in difficult-to-assess populations would make the scale more widely applicable for a range of studies and clinical uses.