Psychometric Properties of the PANS 31-Item Symptom Rating Scale

Abstract

Objectives:

Pediatric acute-onset neuropsychiatric syndrome (PANS) is characterized by sudden onset of obsessive-compulsive disorder and/or eating restriction with associated neuropsychiatric symptoms from at least two of seven categories. The PANS 31-Item Symptom Rating Scale (PANS Rating Scale) was developed to identify and measure the severity of PANS symptoms. The objective of this study was to define the psychometric properties of the PANS Rating Scale.

Methods:

Children with PANS (N = 135) and their parents participated. Parents completed the PANS Rating Scale and other scales on Research Electronic Data Capture. The PANS Rating Scale includes 31 items that are rated on a Likert scale from 0 = none to 4 = extreme. Pearson's correlations were run between the PANS Total score and scores on the Children's Yale–Brown Obsessive Compulsive Scale (CY-BOCS), Yale Global Tic Severity Scale (YGTSS), Modified Overt Aggression Scale (MOAS), Columbia Impairment Scale (CIS), PANS Global Impairment Score (GIS), and Children's Global Assessment Scale (CGAS).

Results:

Convergent validity was supported by significant correlations between the PANS Total and scores on the CY-BOCS, YGTSS, MOAS, CIS, GIS, and CGAS. The largest correlations were with measures of functional impairment: PANS Total and CIS (r = 0.81) and PANS Total and GIS (r = 0.74). Cronbach's alpha was 0.89 which demonstrates strong internal consistency of the 31 items. PANS Total score was significantly higher in children in a flare of their neuropsychiatric symptoms compared to children who were not in a flare.

Conclusions:

This study provides preliminary support for the PANS Rating Scale as a valid research instrument with good internal consistency. The PANS Rating Scale appears to be a useful measure for assessing children with PANS.

Introduction

Pediatric acute-onset neuropsychiatric syndrome (PANS) is a neuropsychiatric disease characterized by the sudden onset or relapse of obsessive-compulsive disorder (OCD) and/or eating restriction as well as acute onset of other neuropsychiatric symptoms from at least two of the following seven categories: anxiety, emotional lability or depression, irritability/aggression/oppositional behavior, behavioral/developmental regression, deterioration of school or cognitive skills, sensory or motor abnormalities (e.g., tics, clumsiness, and overflow dystonia), and somatic symptoms (e.g., sleep disruption, urinary symptoms) (Calaprice et al., 2017; Frankovich et al., 2015a; Frankovich et al., 2015b; Gamucci et al., 2019; Gaughan et al., 2016; Gromark et al., 2019; Johnson et al., 2019; Murphy et al., 2015; Swedo et al., 2012; Toufexis et al., 2015).

PANS is thought to be triggered by an infection but since proving a causal relationship between infection and individual cases is not always possible, the PANS criteria are agnostic to the triggering infection. Pediatric Autoimmune Neuropsychiatric Disorder Associated with Streptococcus (PANDAS) is a related condition that refers to cases of abrupt onset OCD and/or tics which have a temporal relationship to Group A streptococcal infections (Chang et al., 2015; Swedo et al., 2012). The connection between onset of OCD, tics, behavior changes, and eating restriction to Group A streptococcus is supported by robust epidemiological studies (Murphy et al., 2007; Orlovska et al., 2017). Imaging studies suggest neuroinflammation in the basal ganglia region (Cabrera et al., 2019; Giedd et al., 2000; Giedd et al., 1996; Kumar et al., 2015; Zheng et al., 2020) which is supported by recent findings of autoantibodies binding to cholinergic interneurons within the basal ganglia region (Xu et al., 2021) in both PANS and PANDAS.

Typically, PANS/PANDAS symptoms follow a relapsing and remitting course. Being in a “flare” refers to a dramatic worsening of neuropsychiatric symptoms (i.e., relapse). A flare is usually triggered by a new infection (e.g., strep throat). Without intervention, an average flare lasts ∼11 weeks (Brown et al., 2017a; Brown et al., 2017b).

The approach to intervention for PANS/PANDAS includes treating the underlying infection, treating the postinfectious inflammatory response, and managing psychiatric symptoms based on standard of care (Cooperstock et al., 2017; Frankovich et al., 2017; Swedo et al., 2017; Thienemann et al., 2017). There is a need for an instrument to identify and measure severity of PANS symptoms. The purpose of this investigation was to define the psychometric properties of the PANS 31-Item Symptom Rating Scale (PANS Rating Scale) (T. Murphy and G. Bernstein, pers. commun., 2023). It was predicted that the validity data would support the usefulness of this instrument for assessing children with PANS. Furthermore, we hypothesized that the PANS Total score would be higher in those in a flare compared to those not in a flare.

Methods

This study was conducted at Stanford University in affiliation with the Stanford Children's Immune Behavioral Health (IBH) Clinic, a multidisciplinary clinic where patients are followed by a team composed of psychiatrists, rheumatologists, an immunologist, a pediatrician, a nurse practitioner, social worker, and psychologist. The Stanford Panel on Human Subjects Institutional Review Board (IRB) approved this study (IRB No. 26922) in accordance with the latest version of the Declaration of Helsinki. Written informed consent from parents and adult participants and written informed assent from competent minor participants were obtained. Data were stored in a secure Research Electronic Data Capture (REDCap) database (Harris et al., 2009). Data were exported from the IBH REDCap clinical database for data analyses.

Study population

We selected consecutive patients who met PANS criteria and completed a 31-item PANS Rating Scale after March 9, 2021, the date at which the full 31-item PANS Rating Scale was administered in the IBH Clinic. The PANS Rating Scale was part of an online survey of rating scales that parents completed before every visit. Between March 9, 2021 and April 29, 2022, the IBH Clinic saw 214 unique patients. We excluded patients from the study who refused or withdrew consent (n = 5) and did not meet PANS or PANDAS criteria (n = 74). Total sample size was 135 patients. Patients were grouped according to their disease state (with respect to whether they were in a flare or not), with group placement based on the most impaired state (relative to the child's usual baseline) as determined by clinician assessment after the visit.

The following disease state groupings were used:

• 1) Not at Baseline, In Flare (n = 63): This group reflects patients who are impaired relative to their own baseline status. They may be in their first episode or a relapse. The flare state may have occurred on a previously good baseline or on chronic symptoms. The flare status may be new-onset, steady, but not resolving.

• 2) Not at Baseline, Resolving Flare, or Unclear Flare Status (n = 23): Patients who are impaired relative to their own baseline, but the patient is either in a resolving flare or unclear flare. This is often the case when the flare is mostly resolved but there are residual symptoms.

• 3) Significantly Improved, In Flare (n = 6): Patients who have significantly improved from their initial episode or a relapse and presented to clinic with a new but minor relapse of PANS symptoms.

• 4) Significantly Improved, Not in Flare (n = 28): Patients who have significantly improved from their initial episode or a relapse but have residual symptoms that keep them from being at their normal baseline.

• 5) Excellent Baseline, Not in Flare (n = 15): Patients who are back to their normal pre-PANS baseline and are currently not in a flare.

Measures

The PANS Rating Scale is composed of 31 items related to PANS symptomatology (Supplementary Table S1). Each item is rated on a 5-point Likert scale (0 = None, 1 = Mild, 2 = Moderate, 3 = Severe, 4 = Extreme) based on symptom severity during the previous week. The Total PANS score is the sum of the scores (0–4) on each of the 31 items with a higher score indicating a higher number of symptoms and/or greater severity of symptoms. The IBH Clinic collects the parent-reported PANS Rating Scale as part of a comprehensive REDCap survey of measures that parents complete online before each clinic visit. Psychometric properties of the PANS Rating Scale have not been studied previously.

When OCD or tics are present (based on the 31 items of the PANS Rating Scale), the REDCap questionnaire has branching logic to prompt the parents to fill out additional surveys. If OCD symptoms were reported, parents completed the Children's Yale-Brown Obsessive Compulsive Scale (CY-BOCS) that measures severity of obsessions and compulsions in youths with a total score of 0–40 (Scahill et al., 1997). The CY-BOCS has been shown to be a reliable and valid instrument for assessing OCD symptomatology in youth (Storch et al., 2004). If tics were reported, parents completed the Yale Global Tic Severity Scale (YGTSS) that measures the severity of motor and vocal tics. The convergent and discriminant validity of the YGTSS have been established (Leckman et al., 1989). Parents did not complete the CY-BOCS when obsessions or compulsions were absent, nor the YGTSS if tics were absent. This explains the missing data for 32 participants on the CY-BOCS and for 98 participants on the YGTSS (Tables 1 and 2).

Table 1.

Demographic and Clinical Characteristics of 135 Consecutive Patients with Pediatric Acute-Onset Neuropsychiatric Syndrome

Age at first neuropsychiatric deterioration ever
Mean (SD) [missing]	7.9 (3.5) [1]
Median [range]	7.6 [1.6, 17.7]
Age at neuropsychiatric decline that led to first clinic visit
Mean (SD) [missing]	9.4 (3.4) [4]
Median [range]	9.6 [3.0, 18.1]
Age at first neuropsychiatric decline that met strict PANS criteria
Mean (SD) [missing]	9.3 (3.8) [1]
Median [range]	9.3 [1.6, 26.3]
Biological sex, n (%)
Male	79 (58.5)
Female	56 (41.5)
Race, n (%)
Asian	3 (2.2)
Mixed race	13 (9.6)
Unknown	1 (0.7)
White	118 (87.4)
Ethnicity, n (%)
Hispanic/Latinx	19 (14.1)
Not Hispanic/Latinx	116 (85.9)
Children's Yale–Brown Obsessive Compulsive Scale
Mean (SD) [missing]	15.9a (8.21) [32]
Median [range]	15.0 [0.0, 40.0]
Yale Global Tic Severity Scale
Mean (SD)	19.8b (13.89) [98]
Median [range]	16.0 [0.0, 55.0]
Modified Overt Aggression Scale
Mean (SD)	2.9 (6.2)
Median [range]	0.0 [0.0, 40.0]
Columbia Impairment Scale
Mean (SD)	9.8 (9.3)
Median [range]	7.0 [0.0, 46.0]
PANS Global Impairment Score
Mean (SD)	33.4 (25.1)
Median [range]	25.0 [0.0, 95.0]
Children's Global Assessment Scale
Mean (SD) [missing]	68.5 (18.6) [3]
Median [range]	70.0 [17.0, 100.0]

Note: Variables without missing data noted in table have no missing data. Missing data on Children's Yale–Brown Obsessive Compulsive Scale and Yale Global Tic Severity Scale are due to those measures not being relevant for that patient. Participants presented March 2021 to April 2022.

^a Based on 103 participants.

^b Based on 37 participants.

PANS, Pediatric Acute-Onset Neuropsychiatric Syndrome; SD, standard deviation.

Table 2.

Correlations Between Psychometric Scales and PANS Total Score

Scale	n	Pearson's correlation with PANS Total score	95% CI	p
Children's Yale–Brown Obsessive Compulsive Scale	103	0.6	0.45 to 0.71	<0.001
Yale Global Tic Severity Scale	37	0.33	0.01 to 0.59	0.045
Modified Overt Aggression Scale	135	0.45	0.30 to 0.57	<0.001
Columbia Impairment Scale	135	0.81	0.74 to 0.86	<0.001
PANS Global Impairment Score	135	0.74	0.65 to 0.81	<0.001
Children's Global Assessment Scale	132	−0.66	−0.74 to −0.55	<0.001

Participants presented March 2021 to April 2022.

CI, confidence interval.

Other instruments filled out by parents before each visit included: (1) Modified Overt Aggression Scale (MOAS) that evaluates the severity and frequency of aggressive behavior in four categories: verbal aggression, aggression against objects, aggression against self, and aggression toward others. Preliminary data indicate that the MOAS is useful for retrospective reporting of aggressive behavior in adults (Sorgi et al., 1991). (2) Columbia Impairment Scale (CIS) is a 13-item scale to measure the functional impairment of children and adolescents and has been validated (Attell et al., 2020; Bird et al., 1993). (3) PANS Global Impairment Score (GIS) is a single item that is rated 0 to 100 with higher scores indicating greater impairment. GIS has been validated in the PANS population (Leibold et al., 2019). (4) Children's Global Assessment Scale (CGAS) is filled out by the clinician after each clinic visit and is based on data collected and reviewed during the parent/child interview and clinician review of the above measures. The concurrent and discriminant validity of the CGAS have been reported (Shaffer et al., 1983).

Statistical analysis

Participant demographic and clinical characteristics were summarized using mean (standard deviation) and count (%) for continuous and categorical variables, respectively. Pearson's correlations were used to explore the relationship between PANS Total score and other psychometric measures.

We used a t-test to evaluate whether the PANS Rating Scale differs by sex or children in a flare versus those not in a flare. For the later analysis, we used the following two patient groups: (1) Flare group (n = 63) was defined as children who are not at baseline and in a flare, and (2) Not Flare group (n = 43) included children who were significantly improved, not in a flare (n = 28) plus those at excellent baseline (n = 15). In addition, linear regression with the outcome of PANS Total score was tested for an association with age of PANS onset. All analysis was completed in R, version 4.2.2. P-values are all two-sided and evaluated at the 0.05 level for statistical significance.

Results

Participants

The sample included 135 children and adolescents. Biological sex was 58.5% male and 41.5% female. Racial composition was 2.2% Asian, 9.6% Mixed race, 0.7% unknown, and 87.4% White (Table 1). Ethnicity was 14.1% Hispanic or Latinx and 85.9% Not Hispanic or Latinx. Mean age when the children first met full PANS criteria was 9.3 ± 3.8 years. There was no association between age of PANS onset and PANS Total score [slope = 0.17, 95% confidence interval: (−0.43 to 0.78), p = 0.57]. There was no difference between mean PANS Total score for males (17.3 ± 14.8) versus females (17.5 ± 11.7) (p = 0.96).

Neuropsychiatric symptoms

The most common symptoms reported by parents on the PANS Rating Scale were obsessions (65.9%) and compulsions (68.1%) with 74.8% of parents reporting their children had obsessions and/or compulsions. Other common symptoms occurring in 60% or more of children included other anxiety/fears/phobias/panic attacks, mood swings/moodiness, irritability, and trouble paying attention (Supplementary Table S1).

Reliability

A test of internal consistency was completed on the 31-item PANS Rating Scale. Cronbach's alpha was 0.89.

Validity

There were significant correlations between the PANS Total and total scores on the CY-BOCS, YGTSS, MOAS, CIS, GIS, and CGAS (Table 2). All correlations are medium to large with the exception of the small correlation between the PANS Total and the YGTSS. The highest correlations were with the two measures of functional impairment: PANS Total with CIS (r = 0.81) and PANS Total with GIS (r = 0.74).

Flare status and PANS scores

The mean PANS Total score was 17.4 ± 13.6 for the entire sample. The mean PANS Total score was significantly higher in the Flare group (n = 63) at 24.6 ± 13.2 compared to 6.4 ± 6.2 in the Not Flare group (includes participants who are significantly improved and participants at an excellent baseline) (n = 43) (p < 0.001).

Discussion

There were significant correlations ranging from 0.33 to 0.81 between the PANS Rating Scale and other parent-report measures (i.e., CY-BOCS, YGTSS MOAS, CIS, GIS). CGAS, our only clinician-rated scale, showed a significant correlation of −0.66 with the PANS Rating Scale. These correlations indicate that higher scores on the PANS Rating Scale are associated with higher scores on measures of obsessions/compulsions, tics, aggression, and functional impairment (i.e., CIS, GIS) and are associated with lower scores on a measure of healthy functioning (i.e., CGAS). Our findings indicate that the PANS Rating Scale and the other study scales are measuring similar but not identical constructs and provide support for the convergent validity of the PANS Rating Scale. Of note, the highest correlations were between the PANS Total and CIS and GIS, measures of functional impairment, which suggests that the PANS Rating Scale is an excellent instrument for identifying youths with impairment due to their PANS symptoms.

This study also demonstrated that the PANS Rating Scale is an instrument with good internal consistency of items. The internal consistency measure showed a Cronbach's alpha of 0.89, which indicates that the items on the scale are measuring similar but not identical constructs.

The 31-Item measure was designed to be a complete survey of symptoms that is filled out by parents to compile all the observable psychiatric symptoms affecting a child. It does not include certain physical symptoms that have also been observed in the PANS population (e.g., postural orthostatic tachycardia syndrome symptoms, fatigue, movements during sleep, morning stiffness, rashes) as these are covered by a more traditional medical review of systems. During a clinic visit, it is not possible to review and discuss all 31 items so this instrument can serve as a starting point for clinicians and families to discuss the symptoms that most affect their child while allowing for the full inventory of symptoms to be recorded.

Data collection and COVID-19 pandemic

The data for this study were collected during the COVID-19 pandemic, and for at least half of the study, many children were in virtual or hybrid schooling and wearing masks to school and playdates. Generally speaking, the population in the Bay Area practiced caution with regards to spread of germs (e.g., masks in stores, social distancing, and small gatherings) until the Fall of 2022 (Judson et al., 2023). Up to 80% of adults living in the San Francisco Bay Area reported “always wearing a mask” in a study that tracked 3846 adults between August 2020 and March 2021 (Judson et al., 2023). These precautions resulted in fewer infections known to trigger PANS (Group A strep infections, other ear, nose and throat infections) (McBride et al., 2020), fewer new-onset PANS cases, and fewer flares compared to nonpandemic years (publication in process). Our data show a lower mean PANS Total score than we would expect during a nonpandemic year.

The frequencies of many of the PANS symptoms in our study (Supplementary Table S1) were less than those reported in earlier studies (Murphy et al., 2014; Murphy et al., 2012). This is partially explained by data collection occurring during the pandemic when there were fewer infections and thus fewer relapses and the intensity of flares were less (publication in process). Separation anxiety has been reported to be one of the most common comorbid symptoms in PANS patients (Bernstein et al., 2010; Murphy et al., 2014; Swedo et al., 2015); however, it was described in only 32% of our sample. This is likely due to children staying home with parents during the pandemic and not encountering situations requiring separation from attachment figures.

Limitations and strengths

Since the study took place during the pandemic when there were fewer flares compared to nonpandemic years, the PANS Total scores are lower than would be otherwise expected since fewer flares means that there was more time spent in recovery mode. Thus, the data are not necessarily generalizable to all children with PANS during nonpandemic years.

Strengths include that data were collected in a PANS specialty clinic with infrastructure for obtaining data that are directly inputted into REDCap through on-line parent rating scales. Each participant had a comprehensive multimodal, multiinformant assessment which allowed for comparison of scores on the PANS Rating Scale with other validated measures to determine convergent validity of our instrument.

Our sample of patients with PANS is more diverse than the sample recently described from chart reviews of patients in primary care clinics (Wald et al., 2023). A retrospective chart review of 900,497 children who had primary care visits in 2017–2019 at 3 academic medical centers (University of Arkansas Medical Center, Geisel School of Medicine at Dartmouth, University of Wisconsin School of Medicine in Madison) identified 357 possible cases of PANS/PANDAS including 13 actual cases of PANS/PANDAS (Wald et al., 2023). The sample of over 900,000 children was 85% non-Hispanic White, while the 13 definite cases of PANS were 100% non-Hispanic White. The racial composition of the current study is 2% Asian, 10% mixed race, 1% unknown, and 87% White. Ethnicity of participants in our study is 14% Hispanic and 86% non-Hispanic.

Conclusions

This study provides preliminary support for the PANS Rating Scale as a valid research instrument with good internal consistency of items. The instrument has solid convergent validity and appears to be an excellent instrument for identifying youths with functional impairment due to their PANS symptoms (i.e., very high correlations with the CIS and GIS). It can be used to help differentiate youths with PANS who are in a flare versus those not in a flare.

Clinical Significance

The PANS Rating Scale appears to be a useful instrument for identifying and measuring the severity of PANS symptoms in children and adolescents. The Scale permits the recording of a full inventory of neuropsychiatric symptoms. During a clinic visit, it can be used by clinicians and families to discuss the neuropsychiatric symptoms that are most impairing for the child. The PANS Rating Scale fills a void in the area of evaluation of youth with this impairing neuropsychiatric condition. A future direction is to determine whether this instrument is effective in identifying change in symptom severity with treatment.

Authors' Contributions

G.A.B.: Conceptualization, methodology, writing–original draft, writing–review and editing. M.H.K.: Data curation, writing-original draft. R.L.F.: Formal analysis, visualization, writing–original draft, writing–review and editing. C.M.: Investigation. M.S. and M.T.: Investigation, resources. S.A., B.F., and M.M.: Resources. T.K.M.: Resources, writing–review and editing. J.F.: Conceptualization, methodology, project administration, supervision, writing–original draft, writing–review and editing, and funding acquisition (to support RA to administer questionnaires).

Disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health's National Center for Advancing Translational Sciences.

Disclosures

The authors have no conflicts of interest.

Supplementary Material

Supplementary Table S1