The Parental Concerns Questionnaire: A Brief Screening Instrument for Potentially Severe Behavior Problems in Infants and Toddlers At-Risk for Developmental Delays

Abstract

The Parental Concerns Questionnaire (PCQ) was designed as a parent-interview screening instrument for young children with developmental concerns at risk for potentially severe behavior problems (SBDs). Parents of 262 young children (4 to 48 months) answered to the 15 dichotomous PCQ items interviewed by trained staff. Cluster analysis for items revealed three item clusters, which we labeled Developmental/Social (8 items), Biomedical (3 items), and Behavior Problems (3 items). This paper discussed primarily the Behavior Problems cluster, with items referring to self-injurious, aggressive, and destructive behaviors. Parents' concerns about behavior problems were high, with item-endorsements of the Behavior Problems cluster ranging from 41.8 % to 68.8 %. The Behavior Problems cluster was significantly correlated with all three subscales of the Behavior Problems Inventory (BPI-01), with select subscales of the Aberrant Behavior Checklist (ABC), and with the Repetitive Behavior Scale-Revised (RBS-R) providing some evidence for concurrent validity. Sensitivity and specificity data were computed for the three PCQ items as well as for the cluster score in comparison with the BPI-01, ABC, and RBS-R showing strong sensitivity. The PCQ Behavior Problems cluster is a useful screening checklist with high sensitivity for potential SBDs in young children at-risk for developmental delays.

Early identification and intervention for infants and toddlers at-risk for developmental psychopathology has become a field of increasing research interest in recent years (see Schroeder & Courtemanche, 2012 for a review). Emphasis on infant mental health and on early identification of autism spectrum disorders (ASD) has undoubtedly promoted more research on early screening of parents and younger siblings of children with ASD (Fodstad et al. 2012; Hess & Landa, 2012). There is now considerable research suggesting that signs of SBDs, i.e. aggression, self-injury (SIB), and their frequent co-variable, stereotyped behavior, may occur as early as 6 months in some children with DD (Berkson et al. 2001; Fodstad et al., 2012; Schroeder et al., under review). There is also mounting evidence for the importance of early behavioral and parent based intervention to prevent behavior problems in children with developmental disabilities (Einfeld et al. 2013). Early screening and tracking of these children might avoid more intensive intervention later, after SBDs are firmly established in their repertoires and when they may be more resistant to intervention. Oyama-Ganiko et al. (in press) have shown that a moderate early intervention program is sufficient to significantly decrease the SBDs in a majority (57 %) of such children.

While Matson and collaborators developed and validated the Baby and Infant Screen for Children with aUtIsm Traits (BISCUIT-Part 3; Fodstad et al., 2012; Matson et al. 2009) for individuals on the autism spectrum, Schroeder and his colleagues simultaneously developed the Parental Concerns Questionnaire (PCQ) for children with DD in general. The PCQ is a 15-item yes/no checklist intended as a screening instrument that families could use to decide whether or not to enter their child into an early intervention program to prevent the emergence or aggravation of SBDs. The items represent risk factors found in the research literature to be related to elevated incidence of aggression, SIB, and stereotyped behavior in (see Rojahn et al. 2008 for review). Schroeder et al. (under review) found that age, gender, diagnosis, intellectual functioning, communication, visual impairment, parent education, and family income were differentially related to the occurrence of SIB, aggression, and stereotyped behavior. The most strongly related risk factors to SBDs in these young children were age, gender, and diagnosis.

The current report presents a psychometric analysis of the PCQ Behavior Problems cluster, one of three PSQ item clusters, in comparison to three existing behavior rating scales. It was hypothesized that the PCQ Behavior Problems would be highly correlated with frequency and intensity scores on the Behavior Problems Inventory (BPI-01: Rojahn et al., 2001), the Aberrant Behavior Checklist (ABC: Aman et al. 1985) and Repetitive Behavior Checklist-Revised (RBS-R: Bodfish et al., 2000). It was also predicted that the PCQ Behavior Problems cluster item scores would vary as a function of the strongest risk factors for SBD found in the Schroeder et al. (under review) study, i.e. age, gender, and diagnosis.

Method

This protocol was approved by the Internal Review Board at CASP and by Human Subjects Committee at the University of Kansas.

Participant Recruitment and Ascertainment Procedures

Parents of infants and toddlers were solicited via television, radio, and newspaper across Peru who were concerned about early signs for developmental delay as well as aggression, destructive behavior, and SIB of their child. Peru has a population of 22 million, 11 million of whom live in Lima, the Capitol, so most of the responses came from residents of Lima. Parents called the CASP Telephone Triage Service, which helps families locate services for their children throughout Peru. After 1000 calls over a 2-month period, parents of 341 children brought their child for an interview. Ten “veteran” parents whose children had been treated for SBDs at CASP for several years and who had been trained on the PCQ, interviewed the parents. These “veteran” parents were highly trained over the years by CASP staff in behavior management (183 h per year), and they were able to offer support and encouragement to the parents. If the parents showed concerns for their child's having any of the risk factors listed on the PCQ, they were invited to an in-depth interdisciplinary evaluation at CASP on weekends in the fall of 2010. There were 262 children, aged 4–48 months, who met this criterion. A full account of the results of these evaluations and follow-ups is given in Schroeder et al. (under review). We will concentrate on the data relevant to SBDs.

Behavior Assessment Instruments

Parental Concerns Questionnaire (PCQ)

As mentioned earlier, the PCQ is a 15-item yes/no checklist developed by the senior author from an extensive review of the literature on risk factors for SBDs (see Dawson, 1996; Dunlap et al. 2006; Rojahn et al. 2008 for review). Risk factors were defined as behaviors or existing disorders that raised the probability of full-blown aggression, SIB, and stereotyped behavior in the future (for details see Mayo-Ortega et al. 2012). If parents responded affirmatively one or more of those PCQ items (or risk factors), they were queried further as to the nature and severity of the disorder. The PCQ questions are shown in Table 1.

Table 1. Percentage of positive endorsement of PCQ items.

Cluster	PCQ Items (abbreviated)		n
Developmental/Social	Is your child happy	65.90 %	246
	Motor development	56.50 %	246
	Learning abilities	61.70 %	243
	Attending	73.10 %	249
	Language development	87.00 %	246
	Funny movements	62.70 %	252
	Autism	56.60 %	242
	Sibling with autism	80.20 %	222
Biomedical	Genetic syndrome	30.40 %	257
	Neurological syndrome	52.50 %	261
	Seizures	12.20 %	262
Behavior Problems	Hits others	68.80 %	256
	Destroys objects	41.80 %	239
	Injures himself	63.10 %	255
	Other concerns	97.50 %	203

In-Depth Behavior Assessment

Three well-known rating scales were used for behavior assessment.

Behavior Problems Inventory (BPI-01)

The BPI-01 is a 49-item behavior-rating instrument with 14 specific self-injurious behavior, 24 stereotypic behavior, and 11 aggressive/destructive behavior items (Rojahn et al., 2001) Each of the three problem behavior groups is preceded by a generic definition that applies to all items within the group. Items are scored on a five-point frequency scale (0=never, 1=monthly, 2=weekly, 3=daily, 4=hourly) and a four-point degree-of-the-problem or severity scale (0=no problem, 1=a slight problem, 2=a moderate problem, 3=a severe problem). Only behaviors that have occurred at least once during the past 2 months are scored. For clinically significant behaviors that are not captured by of the 49 items, “other” categories are provided for each of the three behavior groups. The BPI has been used predominately in groups of individuals in the severe to profound range of intellectual disabilities who were residents of institutions. However, the BPI-01 was also found to be sensitive to drug effects in studies with community-based children with sub-average cognitive functioning. The instrument has also proved useful for longitudinal assessments for people who are at risk for behavior problems, as it permits monitoring changes in the frequency or intensity of existing behavior problems, the emergence of new behaviors, and the remission of others. Rojahn et al. (2012) have recently published norms on the BPI-01 across the life span.

Aberrant Behavior Checklist (ABC)

The ABC is one of the most widely used and psychometrically sound assessment instruments in research and clinical practice in developmental disabilities. It is an informant-oriented problem behavior rating scale that was empirically derived by factor analysis based on a large sample in New Zealand (Aman et al. 1985a). It consists of 58 items, each scored on a four-point scale (0=not a problem, through 3=problem is severe in degree). The items are divided into five subscales: (1) Irritability, Agitation, Crying, (2) Lethargy, Social Withdrawal, (3) Stereotypic Behavior, (4) Hyperactivity, Non-Compliance, and (5) Inappropriate Speech. Norms are available for adults and for children and adolescents (Aman & Singh, 1994; Aman et al., 1985; Marshburn & Aman, 1992). Three nonspecific self-injurious behaviors are part of the Irritability, Agitation, and Crying subscale.

Repetitive Behavior Scale-Revised (RBS-R)

The RBS-R is an empirically derived behavior rating instrument that was designed for the assessment of the presence and severity of abnormal, repetitive behaviors in persons with a variety of conditions, including intellectual disabilities, psychiatric, neurological, and behavioral disorders (Bodfish et al. 1999; Bodfish et al. 2000). The scale was developed by compiling items from several existing behavior-rating scales that measured intensity of repetitive behaviors. The selected 43 items were assigned by clinical staff to one of six dimensions or subscales of repetitive behavior: (1) Stereotyped behavior (six items), (2) Self-injurious behavior (eight items), (3) Compulsive behavior (eight items), (4) Ritualistic behavior (six items), (5) Sameness behavior (11 items), and (6) Restricted behavior (four items). The first version of the instrument, the RBS, consisted only of the first three subscales. Items are evaluated on a four-point Likert scale (from 0=behavior does not occur, to 3=behavior occurs and is a severe problem.) The scale has been used to measure repetitive behavior in people with intellectual disabilities and autism spectrum disorders.

Since these scales have not been used often among children as young as in the current study, we have conducted a validity study of the three of them and, we have shown that they do have good validity and reliability in this population (Rojahn et al., 2013).

Data Analysis

Data analysis was conducted with IBM SPSS Statistics 21. Initially descriptive statistics of the participants and percentage of endorsements of parental concerns on the PCQ were calculated. Correlation matrices (using Spearman's Rho, since the distributions of many variables were skewed) were also examined for potential relationships among PCQ, BPI-01, ABC, and RBS-R measures of SIB, stereotyped behavior, and aggression.

Since many of the questions on the PCQ were correlated, a cluster analysis, using Ward's procedure, was used to find the best cluster solution (Sheppard, 1996). A three-cluster solution proved the most clinically meaningful: (1) a Developmental/Social cluster (8 items); (2) a Biomedical cluster (3 items); and (3) a Behavior Problems cluster (3 items) (see Table 1). Total scores of these three clusters were then correlated with the BPI-01, ABC, and RBS-R subscale scores.

ANOVAS were conducted comparing gender, diagnosis, and age groups with PCQ Behavior Problems cluster scores. Finally, sensitivity and specificity of the PCQ were calculated.

Results

Descriptive Statistics

Of the 262 children who received the PCQ, 166 were male, 96 were female. Mean age was 27.06 months (SD=10.0, range=4 to 48 months). Distribution of age groups (12-month grouping) was: 31 were between 4 and12 months; 65 between 13 and 24; 109 between 25 and 36; and 57 between 37 and 48. This grouping was chosen in order to observe changes in SBDs expected near the age of 2 years, as noted by Berkson and Tupa (2000) in their review. Of the 262 children invited, 233 attended the confirmatory interdisciplinary evaluation, and they were grouped into three groups: Down syndrome (n=58), Autism or At-Risk for Autism (n=91), and Atypical Development (n = 84), which included a variety of impairments and syndromes, such as blindness, global developmental delay, Hydrocephaly, Microcephaly, Macrocephaly, Cerebral Palsy, and delays with unknown causes. All but 12 (95 %) had scores below average on the Cognitive Subscale of the Bayley Scales of Infant Development, Third Edition (Bayley 2006). Thus almost all of the children were considered “At-Risk-_for-Developmental-Delay”

Behavior problems exhibited during the interdisciplinary evaluations were mild for all children, hence our use of the term “potentially” SBDs.¹ Thus, all of the SIB cases in the study were exhibiting proto-SIB, according to Berkson's classification (Berkson et al., 2001).

Parental Endorsement of PCQ Items

Table 1 shows the item-cluster map and gives the percentage of positive endorsements of PCQ items. The rate was high, ranging from 12.2 % to 97.5 %. “Other Concerns” was a category left open for the parent to comment on something not covered in the other 14 items. A wide array of responses was given to this item, many of them unrelated to SBDs. Of the other 14 items, the highest concerns were with language development, attention and learning, and concerns about ASD and SBDs.

Table 2 presents descriptive statistics of the PCQ item clusters, including Cronbach α internal consistency scores. α- values were low for all three subscales if the standards for behavior rating scales are adopted. However, screening instruments with small item numbers cannot, by definition, be expected to yield high α -levels, as they increase with the number of items.

Table 2. Descriptive statistics and internal consistency (α) for the PCQ clusters.

	n items	Mean	SD	n	Skew	Kurtosis	α
Developmental/Social	8	5.08	1.64	201	−.28	−.63	.44
Biomedical	3	0.94	0.84	256	.48	−.58	.29
Behavior Problems	3	1.73	0.98	237	−.35	−.86	.42
Totala	14	7.79	2.24	193	−.09	−.39	.45

^aExcluding item 15 (“other concerns”)

Correlation of PCQ Cluster Scores with BPI-01, ABC, and RBS-R Subscale Scores

Table 3 gives the Spearman ρ (rho) correlations of the three PCQ endorsement cluster scores with BPI-01, ABC, and RBS-R rating subscale scores. A non-parametric was chosen due to the fact the behavior problem scales typically produce skewed, non-normally distributed data. There were several significant correlations among the PCQ Behavior Problem Cluster and all three BPI-01 subscale scores, the ABC Irritability subscale score, and the RBS-R subscale scores for SIB, Compulsivity, Ritualistic, Sameness subscale scores. There were also number of significant correlations between the PCQ Developmental/Social Cluster and BPI-01 subscale scores for SIB and Stereotypy and the ABC subscale score for Stereotypy. For the PCQ Biomedical Cluster, the only significant correlation was for Stereotypy on the BPI-01.

Table 3. Spearman ρ correlations of PCQ clusters with BPI-01, ABC, and RBS-R subscales.

PCQ Cluster	Developmental / Social	Biomedical	Behavior Problems
BPI-01
SIB	.26**	.09	.45***
Stereotypy	.42***	.02	.36***
Aggression/Destruction	.08	−.10	.53***
ABC
Irritability	.24*	−.15	.37***
Lethargy	.24*	−.20	.00
Stereotypy	.41***	−.00	.07
Hyperactivity	.14	−.26**	.09
Excessive Speech	.33**	−.13	.13
RBS-R
Stereotypy	.12	.03	.09
SIB	.05	.22*	.28**
Compulsivity	.10	−.19	.29**
Ritualistic	.04	.01	.33**
Sameness	.05	−.16	.23*
Restrictive/Repetitive	.17	−.14	.16

^*p<.05,

^**p<.01,

^***p<.001

Behavior Problem Cluster Scores As a Function of Demographic Variables

To examine the difference between PCQ Behavior Problem cluster scores between boys and girls, a t-test for independent groups was computed. No gender difference was detected. A one-way ANOVA was computed to examine the difference between PCQ Behavior Problem cluster scores between the three diagnostic groups with the PCQ Behavior Problem cluster Scores as the dependent variables. Again, no diagnostic group differences were detected. Finally, a one-way ANOVA was computed to examine the difference between PCQ Behavior Problem cluster scores between the four age groups with the PCQ Behavior Problem cluster Scores as the dependent variables. There were significant groups differences; F (3, 208)=2.78, p<.05. Post hoc chi-square analyses for each of the three Behavior Problem items revealed that group differences occurred only with the item “Destroys objects.” The youngest age group had lower scores on this item, while no difference appeared among the three older age groups.

Diagnostic Performance of the PCQ Behavior Problems Items and Cluster

The diagnostic performance of the PCQ Behavior Problems was examined on the level of the individual items (estimating their sensitivity and specificity) and for the cluster score (receiver-operating characteristics [ROC] analysis). Table 4 presents sensitivity and specificity values of the three PCQ clusters items as test variables vis-a-vis select dichotomized analog subscale scores² of the ABC, BPI-01, and RBS-R as state variables. Sensitivity values ranged from a low .45 to a high .81 while specificity values ranged from .40 to .79.

Table 4. Sensitivity and specificity of the PCQ behavior problem items as compared with select dichotomized analog subscale scores of the ABC, BPI-01, and RBS-R.

Analog Subscales	Dichotomized analog subscale scores	PCQ Items (abbreviated)
		No	Yes
		Injures self
BPI-01 SIB	No	16	9	Sensitivity=.71
	Yes	44	106	Specificity=.64
RBS-R SIB	No	18	19	Sensitivity=.81
	Yes	11	47	Specificity=.49
		Hits others
BP-01 Aggression/Destruction	No	22	12	Sensitivity=.72
	Yes	40	103	Specificity=.65
		Destroys objects
BPI-01 Aggression/Destruction	No	23	6	Sensitivity=.45
	Yes	75	61	Specificity=.79

Sensitivity refers to the rate of true positives (i.e., number of individuals where the PCQ items concurred with the respective analog subscale that a given behavior was present), divided by true positives plus false negatives. Specificity number of true negatives divided by true negatives plus false positives. Sensitivity and specificity of a test are inversely related

It is important to point out that state variables are, if possible, golden standards or true states of conditions. Obviously, those dichotomized subscale scores cannot claim to reflect the true state of affairs because of the error proneness of the instruments that produced them. Hence, it is possible than the PCQ actually performs better than is reflected in the sensitivity and specificity values reported.

Table 5 shows numerical results of ROC analyses that yielded area under the curve (AUC) values, and sensitivity³ and 1–specificity⁴ rates of the PCQ Behavior Problem cluster as it changes as a function of increasing subscale score cut-off points of the BPI-01, ABC, and/or RBS-R. The Behavior Problems cluster showed significant relationships with the BPI-01 subscales SIB and Aggression/Destruction, but not with the RBS-R SIB subscale. AUC values for the PCQ Behavior Problem cluster ranged from .58 to .69 in diagnosing SIB (according to the findings obtained with the RBS_R SIB subscale and the BPI-01 SIB subscale), and .75 for aggressive/destructive behavior (according to the Aggressive/Destructive behavior subscale of the BPI-01). AUC-values below .70 are often considered to signal poor performance, while values between .70 and .80 are fair.⁵

Table 5. Result for the PCQ behavior problems cluster as the test variables Vis-à-Vis dichotomized analog subscale scores.

State variable	AUCa	Cut-points	Sensitivity	1 - Specificity
BPI-01 SIB (freq.)	.69*	0.5	.88	.80
		1.5	.68	.40
		2.5	.30	.00
BPI-01 Aggression Destruction (freq.)	.75**	0.5	.89	.73
		1.5	.70	.20
		2.5	.29	.07
RBS-R SIB	.58	0.5	.91	.79
		1.5	.68	.57
		2.5	.27	.21

AUC Area under the curve

^aGuidelines for the interpretation of the AUC values: .90–1=excellent; 80–.90=good; 70–.80=fair; .60–.70=poor; .50–.60=fail (Tape 2013, August 21)

^*p<.05;

^**p<.001

Discussion

In summary, the PCQ Problem cluster is useful in identifying early potential SBDs in young children at risk for and with developmental delays, at least when used in parent interview format by well trained interviewers. In particular, it has good sensitivity in detecting proto-SIB and aggressive behavior. For instance, if the BPI-01 had a score of 1 or higher for the subscale SIB, the PCQ Problem cluster had a score >0 in 87.8 % of the cases (see Table 4). Similar sensitivity rates were found for the BPI-01 Aggression/Destruction subscale and the RSB-S SIB subscale. Its main shortcoming is the rate of “false alarm,” which means that a good portion of children identified as having proto-SIB and aggressive behavior by the PCQ Problem cluster were not identified as such by other, more comprehensive assessment instruments. However, given that sensitivity is the more important index of a screening instrument that serves to identify cases that could benefit from early intervention or treatment as in the case of this population, the PCQ Problem cluster did a useful job. The fact that the three-item PCQ Problem cluster had a low reliability (internal consistency) is likely at least in part due to the small number of items and may, therefore, contribute to the relative weakness of the PCQ Problem cluster's specificity. This may be seen as an acceptable tradeoff between feasibility of the instrument and its accuracy.

As far as the performance of the three individual PCQ Problem cluster items is concerned, their sensitivity was somewhat lower than that of the PCQ Problem cluster, but so were their false positive rates.

This study also has a few shortcomings. Most importantly, when it comes to the exploration of sensitivity and specificity of the PCQ, all three instruments that were used as standard (or state variables) have themselves varying degrees of psychometric flaws and thus contribute in these analyses to the shortcomings that are being attributed solely to the PCQ. In other words, some of the discrepancies between the PCQ on the one hand and the BPI-01, the ABC, and the RBS-R on the other may be rooted in measurement errors of the latter. Furthermore, this study did not allow the calculation of other sources of measurement error in the PCQ, such as inter-rater agreement and test-retest reliability.

All in all, the PCQ Behavior Problems cluster used in an interview format with trained interviewers can be a useful first screen with high sensitivity for potential SBDs in young children at-risk for developmental delays.