Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Jun 1.
Published in final edited form as: J Dev Phys Disabil. 2014 Feb 15;26(3):325–334. doi: 10.1007/s10882-014-9368-2

Functional Analysis Outcomes and Comparison of Direct Observations and Informant Rating Scales in the Assessment of Severe Behavior Problems of Infants and Toddlers At-Risk for Developmental Delays

Stephen R Schroeder 1, David M Richman 2, Layla Abby 2, Andrea B Courtemanche 4, Rosa Oyama-Ganiko 3
PMCID: PMC4000037  NIHMSID: NIHMS566476  PMID: 24778543

Abstract

Severe problem behaviors, like aggression, self-injury, and repetitive behaviors, in people with intellectual and developmental disabilities often appear during early development and may persist without early intervention. The frequencies of self-injurious behavior, aggression, tantrums, property destruction and stereotyped behavior among 17 infants and toddlers at risk for developmental delays and severe behavior problems were assessed using two methods: 1) direct observation of responses during 10 s partial interval recording during analogue functional analysis and 2) the Behavior Problem Inventory-01 (BPI-01; Rojahn et al, 2001), an informant rating scale. Analogue functional analysis results suggested that the most common function for problem behavior was automatic reinforcement, followed by negative reinforcement in the form of escape from demands. Agreement across the two types of measurement systems as to occurrence of the behaviors reported on the BPI-01 and direct observations during analogue functional analyses was greater than 75% across aggression, self-injury, and stereotyped behavior. Agreement at a more molecular level of the ranking of the most commonly occurring specific behaviors was considerably lower. Results are discussed in terms of future research on identifying conditions that set the occasion for high levels of agreement between indirect and direct measurement systems for severe behavior problems.

Keywords: Functional analysis, informant ratings, infants, toddlers, developmental delays

Recent research has shown that early signs of severe behavior problems, such as self-injurious behavior (SIB), aggression, and stereotyped behaviors, occur among some infants and toddlers at risk for developmental delays as early as one year of age and persist into later life if left untreated (Berkson, Tupa, & Sherman, 2001; Kurtz, Huete, Cataldo, & Chin 2012; Matson, Dempsey, & Fodstad, 2009; Schroeder, et al. in press). These results emphasize the need for early identification and intervention before the behavior problems develop complex response-reinforcer relations in a child’s behavior repertoire.

Information leading to early identification of behavior problems in this very young age group has often relied on a multimodal, multi-method approach to quantification of how often these behaviors occur and how severe the behaviors impact daily life functioning. Methods include parent informant ratings, descriptive direct observations, retrospective interviews, and standardized assessments of behavior problems (e.g., Behavior Assessment for Children-2) during this period of rapid developmental change in motor behavior, language/communication, and adaptive behavior. Questions often arise as to the agreement among these different measures, and as to which are the most valid and reliable sources of information for documenting levels of problem behavior longitudinally or as a method of documenting change from baseline to early intervention. Each type of measure has several strengths and limitations. Retrospective reports from caregivers may give a more complete picture of the past development of the child’s behavior, but these reports may be subject to bias due to the suggestibility of the interviewee or simple errors in recall. Standardized psychometrically-validated informant rating scales are relatively brief and easy to administer to a large group of participants, and they readily lend themselves to statistical group analysis. However, they are often believed to overestimate the frequency or severity of a behavior (Barlow, Nock & Hersen, 2008; Johnston & Pennypacker, 2009).

Direct observations of behavior may be a more preferred method to determine the frequency and severity of problem behaviors. They lend themselves to individual analysis and visual interpretation of graphically depicted data. Analogue functional analysis is one method of direct observation of behavior (Iwata et al., 1982) that involves direct observation of a target behavior under different environmental contingencies in a brief counterbalanced multi-element single-case design. The results of the functional analysis can be used to obtain a relative frequency of the problem behavior under certain conditions, and a literature of over 250 studies supports the use of functional analysis as an experimental method of determining the functions of challenging behaviors (Hanley, Iwata, & McCord, 2003; Beavers, Iwata, & Lerman, 2013). Direct observation procedures, however, can be labor intensive, expensive, and observers may require extensive training before they can obtain satisfactory level of interobserver agreement. Direct observations of behavior may also inaccurately estimate the frequency of the behavior (depending on the observation window chosen and other factors). The results of the observations can be variable unless they are conducted repeatedly until the problem behavior reaches steady-state responding.

Both informant ratings and direct observations are abstracted samples of behavior representing response probabilities (Thompson & Lubinski, 1986). The level of behavior analysis needs to be adjusted to the functional unit of behavior-environment interaction (Morris, Higgins, & Bickel, 1982), in order to assess their agreement adequately. The present paper explores a methodology to assess agreement between parent frequency ratings and direct observations of behavior problems during functional analyses conducted in the homes of infants and toddlers exhibiting behavior problems and at risk for developmental delays in Lima, Peru.

Method

Data from this study were part of a larger longitudinal study of early detection and prevention of severe behavior problems of young children at risk for developmental delays in Peru (Schroeder et al., in press). Parents, who were concerned about their infant’s or toddler’s development and behavior, were solicited by the newspaper, television, and radio to call in to the Centro Ann Sullivan del Peru (CASP), a state-of-the-art educational program for people with autism and developmental disabilities in Lima, the capitol of Lima. After 1000 parental calls to CASP, 341 families were invited to visit for a screening interview. Of those screened, 262 families were invited for an in-depth interdisciplinary evaluation involving language, cognitive, autism diagnostic evaluation, medical, and behavioral problem assessments. Details of the screening procedures are given in Mayo-Ortega et al. (2012) and in Schroeder et al. (2013) and a description of the interdisciplinary evaluations can be found in Schroeder, et al. (in press). Children were then followed for the duration of one year and were re-assessed by interdisciplinary teams after initial evaluation (Time 1), at six-months (Time 2), and at 12 months (Time 3). Of the 262 families invited for the interdisciplinary evaluation, 180 children had complete data over the one-year period. Of these 180, 17 children were randomly selected to participate in the functional analysis (FA) procedures.

Behavior Problem Assessment

Multimodal assessment of behavior problems was used, similar to previous studies on the early development of behavior problems such as SIB (e.g. Kurtz et al., 2012; Richman & Lindauer, 2005). The main dependent variable was the frequency scores the Behavior Problem Inventory (BPI-01; Rojahn, Matson, Lott, Esbensen, & Smalls, 2001). The BPI-01 contains topographically defined items that rate 49 specific behaviors on a frequency scale (0 = never, 1 = monthly, 2 = weekly, 3 = daily, 4 = hourly) and severity scale (0 = no problem, 1 = a slight problem, 2 = a moderate problem, 3 = a severe problem). Behaviors are divided into three subscales: SIB (14 items), stereotypic behavior (24 items), and aggressive or destructive behavior (11 items). Parents were asked to rate how frequent and severe the behaviors were during the last two months.

Several psychometric studies have shown the BPI-01 to have good internal consistency (Gonzalez et al., 2009; Rojahn et al., 2001), high test-retest reliability (Gonzalez et al., 2009), acceptable inter-rater agreement (Sturmey, Fink, & Sevin, 1993), good criterion-related validity (Rojahn, Aman, Matson, & Mayville, 2003; Rojahn, Wilkins, Matson, & Boisjoli, 2010), good factor validity (Gonzalez et al., 2009; Rojahn et al., 2012a, b), and acceptable item characteristics based on item response theory analyses (Barnard-Brak, Rojahn, & Wei, in press). All children received the BPI-01 at each of the three evaluations, separated by six-months.

In addition to the BPI-01, functional analyses were conducted by the children’s parents in the home using the methodology described by Wacker, et al. (1998). Each family received two home visits from CASP staff members. On the first home visit, a Functional Analysis Interview (FAI) was conducted with the parents to define the behavior problems and their possible functions. CASP staff were trained by the second author, who in-turn trained and coached the parents in the home to conduct 5 min analogue functional analysis conditions (i.e., play (control), escape, attention, ignore/alone). On the first visit each condition was conducted once. On the second visit (approximately 6 months later), each condition was conducted twice. All functional analysis sessions were videotaped and a 10 sec partial interval system was used to score each topography of problem behavior (i.e., aggression, property destruction, SIB, stereotypy, and tantrum) within each 10 sec interval. Interobserver agreement (IOA) was conducted on 33% of session. Total agreement was (m=97.8%, range = 80–100%) across all topographies of problem behavior assessed during analogue functional analysis observations.

Data Analysis Plan

Descriptive Statistics

We examined the descriptive statistics to assure that the present sample of participants was comparable to the whole cohort of 180 participants assessed in Schroeder, et al. (in press).

Functional Analyses

After examining the descriptive statistics, we first examined the functional analysis data using visual inspection, to determine the possible functions of the behavior problems (Hagopian et al., 1997). Functional analyses were conducted across five different categories of problem behavior for each child: aggression, property destruction, SIB, stereotypy, and tantrum. Graphs that included fewer than three data points across all conditions (i.e., control, attention, escape, and alone) were excluded from interpretation of the function of problem behavior. Additionally, graphs that showed very low occurrence of problem behavior (i.e., occurred in 10% or less of the intervals) were also excluded. Initially, graphs across all participants and categories of challenging behavior totaled 78 graphs. After applying the exclusion criteria described, we interpreted analogue functional analysis outcomes for 15 participants across 27 topographies of problem behavior.

Interobserver Agreement (IOA)

For IOA purposes, graphs that resulted in undifferentiated responding across all functional analysis conditions and interpretation of automatic reinforcement (e.g.,, differentiated responding in the alone condition) were collapsed and categorized as indicative of automatic reinforcement. The second and third authors interpreted 24 out of 27 (89%) graphs independently (the other three were interpreted by both authors but not included in IOA). Interobserver agreement was calculated by dividing the total number of exact agreements by the sum of agreements and disagreements, and multiplying the result by 100. Interobserver agreement resulted in 88% (21 out of 24 graphs) agreement for interpretation of the function for each topography of problem behavior.

Comparison of Functional Analyses with BPI-01 Ratings

For each participant, we summed the frequencies of 10 s intervals of aggression, property destruction, self-injury, stereotyped behavior, and tantrum behavior across all FA sessions at Time 1 and Time 2 in order to reduce the number of 10 s intervals with zero behavior. We then ranked participants according to their estimated behavior frequency from most to least. We correlated the ranks of the participants on total 10 s intervals of behavior problems during the FAs and BPI-01 frequency ratings at Time1 and six months later at Time 2. Since the distributions of scores were skewed, we used the Spearman’s Rank Difference Correlation (Guilford, 1956).

Agreement Between FAI Definitions by Parents and Observations by CASP Staff During Functional Analysis

Since CASP staff also recorded which problem behaviors parents reported that their child performed frequently (e.g., at least weekly), we compared percentage agreement between whether or not behaviors noted by parents on the BPI-01 and during this Functional Analysis Interview were actually observed by staff during the FA sessions. This could serve as a cross-check on the effects of a possible time lag of up to weeks hiatus between when some BPI-01s were recorded and when FAs were conducted.

Results

Descriptive Statistics

Ten males and seven females participated in the functional analysis conditions. Their mean age at Time 1 was 32 months. (range= 17–41 months). Mean IQ on the Cognitive Scale of the Bayley Scales of Infant and Toddler Development, Third Edition (Bayley, 2006) was a score of 79 (range 55–120), which is more than one standard deviation below the mean of 100. The mean Language/Communication scores on the Communication and Symbolic Behavior Scale (Wetherby & Prizant, 2002) was 69 (range=65–96), which is more than two standard deviations below the mean of 100. Their mean score on the Child Autism Rating Scale (Schopler, Reichler, & Renner, 1988) was 43 (range 34–55). The usually accepted cut-off score suggesting Autism Spectrum Disorder (ASD) is 35. Because these children had not received a full diagnostic examination for ASD, we have labeled them “At Risk for Autism.” In addition, two children had diagnoses of Down syndrome, two had Global Developmental Delay, two had seizures, and one had perinatal hypoxia. The demographics of the current sample were very similar to those in the larger cohort of 180 study participants (Schroeder, et al., in press).

Functional Analyses

Table 1 displays the combined results of the interpretation of the function(s) of the broad categories of problem behavior. Topographies of problem behavior were grouped by behavioral function of their problem behavior into the following groups: (1) automatic reinforcement/undifferentiated, (2) negative reinforcement (escape), (3) and positive reinforcement (attention). The vast majority (70%) of topographies of problem behavior were maintained by automatic reinforcement/undifferentiated functional analysis results. The most frequently occurring social function was negative reinforcement in the form of escape from demands (22%). Finally, only two topographies of problem behavior were maintained by multiple functions (escape and attention function; escape, attention, and automatic function). Surprisingly, no topographies were maintained by positive reinforcement in the form of social attention from the child’s mother during the FA.

Table 1.

Summary of analogue functional analysis outcomes

Categories of Challenging Behavior Automatic/Undifferentiated Escape Attention & Escape Attention, Escape & Automatic
All 19/27 (70%) 6/27 (22%) 1/27 (4%) 1/27 (4%)
Stereotypy 10/27 (37%) 0 0 0
SIB 2/27 (7%) 1/27 (4%) 1/27 (4%) 1/27 (4%)
Property destruction 3/27 (11%) 0 0 0
Tantrums 2/27 (7%) 4/27 (15%) 0 0
Aggression 2/27 (7%) 1/27 (4%) 0 0
Open in a new tab

Agreement between Direct Observations during FA 10 s Intervals and BPI-01 Subscale Frequency Ratings

Percentage agreement between challenging behaviors endorsed by parents on the BPI-01 were also directly observed during the analogue functional analysis conditions. Thus, parents’ ratings on the BPI-01 agreed with direct observations at home visits as to the presence of three broad categories of behavior problems assessed via the BPI-01 (i.e., aggression/destruction, SIB, and stereotypy). Overall agreement, defined as rated as occurring by parents on the BPI-01 and observed during the functional analysis, for SIB was 73%; for aggression, 91%; for stereotyped behavior, 83%. When agreement between direct observations and BPI-01 ratings of the most-to-least frequently occurring topographies were analyzed, however, agreement coefficients dropped dramatically to 48% for aggression; 50% for stereotyped behavior; 42% for SIB.

Rank Difference Correlations between FA behavior problem frequency of 10 s intervals from the FA conditions and BPI-01 subscale frequencies and Time 1 and Time 2 are displayed in Table 2. At Time 1, correlations were not significant. At Time 2, however, correlations were significant for Aggression/Destruction and SIB, but not for Stereotyped Behavior.

Table 2.

Spearman’s Rank Difference Correlations Between Behavior Frequency SummedAcross Functional Analysis Components and BPI-01 Subscale Frequency at Time 1and Time 2.

Aggression/Destruction SIB Stereotyped Behavior
Time 1 .207 .067 −.299
Time 2 .535* .477* .288
Open in a new tab
*

= p<.05

Agreement Between FAI Definitions by Parents and Behavior Observations by CASP Staff during Functional Analysis

Behavior topographies from the items of the BPI-01 were defined by parents during the FAI immediately before the functional analyses were conducted in the home. Observations during FA were videotaped, the definitions of both the FA and FAI agreed 76% the time.

Discussion

In summary, both parental ratings and interviews agreed with behavior observations as to the presence of SIB, aggression, and stereotyped behavior in their child approximately 75% of the time. In the present study, parental ratings of frequency of behavior problems and behavior observations tended to agree at the molar level (present or absent), but the degree of agreement at the more molecular level (rank order agreement) was lower. There are likely a number of mitigating factors that affected this relationship. Some examples include the comparability of response units, the underlying scales (nominal, ordinal, interval, ratio) represented by the different measures, the size of the behavior sample, and the skewness of the underlying distributions. In the present case, the first behavior sample during functional analysis at Time 1 contained so little data that behavioral functions could not be differentiated, whereas the data collected at Time 2 were more stable and interpretable. At Time 2, all parents had practiced the procedures and were likely to produce more stable implementation of the procedures for the functional analysis conditions. Training and coaching of the parents, as well as practice, may be an important ingredient in producing stable data during functional analysis.

The young children who participated in this study were already engaging in several topographies of problem behavior. The most common function was automatic reinforcement and a small proportion of problem behaviors were maintained by some form of social reinforcement. Brief functional analyses, like the ones conducted in this study, may not be sufficient to reliably evoke enough challenging behavior to determine behavioral function in very young children with behavior problems in their early stages of development. Children may not have had enough exposure to the different functional analysis conditions, which could result in undifferentiated functional analysis outcomes. Additionally, the tangible condition was never conducted with any of the children who participated in the functional analysis. Peruvian mothers reported that they found the tangible condition to be unacceptable as they commonly reported that they do not give their children tangibles when they engage in problem behavior, and that they were uncomfortable doing so for the current study. Another limitation of the functional analysis outcomes is that we could not conduct Alone conditions in the home environment. Instead, we had to conduct Ignore sessions that do not provide as good of a test for automatic reinforcement because stimuli such as the presence of another person signaled the potential availability of socially mediated reinforcers. This was another limitation of implementing analogue functional analyses in the participants’ home. The only way to confirm that topographies categorized as automatic/undifferentiated were indeed maintained by some form automatic reinforcement would be to implement a series of extended alone sessions to see if the behavior persisted in the absence of social consequences. The Peruvian mothers also viewed Alone/Ignore sessions negatively because they reported that their children were never left alone. Thus, it is possible that additional functions could have been identified if tangible and/or extended alone conditions were included with the functional analysis conditions. Identifying behavioral functions is an essential component to the development of early intervention procedures. Perhaps a more traditional or extended functional analysis may be needed with very young children with developmental delays in cognition and language/communication, as the children in the present study did.

In conclusion, parental reports of problem behaviors through standardized rating scales matched behaviors that were directly observed during parent-implemented functional analyses at the most basic level. Agreement decreased dramatically, however, to chance levels of agreement when the agreement data were analyzed in terms of most-to-least occurring topographies. Thus, additional research is needed to document the conditions that produce greater agreement between rating scales and direct observations. The use of parent ratings and interviews is a rich resource. In cases such as this, both types of measures can be important for accurate behavior assessment. Perhaps measures like the BPI-01 can be included during functional assessments to better inform the interpretation of functional analysis outcomes.

Fig. 1.

Fig. 1

Open in a new tab

Percentage of functions of aberrant behavior (AB), i.e. SIB, aggression/destruction and stereotyped behavior identified across all 17 participants.

Footnotes

Disclaimer: This research was supported by the Fogarty International Center and the NICHD of NIH, grant no. HD060500. The opinions stated herein reflect those of the authors and not necessarily of the NIH or the Centers for Disease Control and Prevention.

References

  1. Barlow DH, Nock M, Hersen M. Single-case experimental designs. 3. New York: Allyn & Bacon; 2008. [Google Scholar]
  2. Barnard-Brak L, Rojahn J, Wei T. Psychometric analysis of the Behavior Problems inventory using an item-response theory framework: a sample of individuals with intellectual disabilities. Journal of Psychopathology and Behavioral Assessment (in press) [Google Scholar]
  3. Bayley N. Manual of the Bayley Scales of Infant and Toddler Development. 3. San Antonio, TX: NCS Pearson; 2006. [Google Scholar]
  4. Beavers G, Iwata BA, Lerman D. Thirty years of research on the functional analysis of problem behavior. Journal of Applied Behavior Analysis. 2013;46:1–21. doi: 10.1002/jaba.30. [DOI] [PubMed] [Google Scholar]
  5. Berkson G, Tupa M, Sherman L. Early development of stereotyped and self-injurious behaviors: I. incidence. American Journal on Mental Retardation. 2001;106(6):539–547. doi: 10.1352/0895-8017(2001)106<0539:EDOSAS>2.0.CO;2. [DOI] [PubMed] [Google Scholar]
  6. Gonzalez ML, Dixon DR, Esbensen A, Rojahn J, Matson JL, Terlonge C, Smith KR. The behavior problems inventory: Reliability and factor validity in institutionalized adults with intellectual disabilities. Journal of Applied Research in Intellectual Disabilities. 2009;22:223–235. doi: 10.1111/j.1468-3148.2008.00429.x. [DOI] [Google Scholar]
  7. Guilford JP. Statistics in psychology and education. New York: McGraw-Hill; 1956. [Google Scholar]
  8. Hagopian L, Fisher W, Thompson R, Owen-DeSchryver J, Iwata BA, Wacker D. Toward the development of structured criteria for interpretation of functional analysis data. Journal of Applied Behavior Analysis. 1997;30:313–326. doi: 10.1901/jaba.1997.30-313. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Hanley G, Iwata B, McCord B. Functional analysis of problem behavior: A review. Journal of Applied Behavior Analysis. 2003;36:147–185. doi: 10.1901/jaba.2003.36-147. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Iwata BA, Dorsey MF, Slifer KJ, Bauman KE, Richman GS. Toward a functional analysis of self-injury. Analysis and Intervention in Developmental Disabilities. 1982;2:3–20. doi: 10.1901/jaba.1994.27-197. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Johnston JM, Pennypacker HS. Strategies and tactics of behavioral research. 3. New York, NY: Routledge; 2009. [Google Scholar]
  12. Kurtz P, Huete J, Cataldo M, Chin M. Identification of emerging self-injurious behavior in young children: A preliminary study. Journal of Mental Health Research in Intellectual Disabilities. 2012;5:260–285. doi: 10.1080/19315864.2011.600809. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Matson JL, Dempsey T, Fodstad J. Stereotypies and repetitive/restrictive behaviors in infants with autism and pervasive developmental disorder. Developmental Neurorehabilitation. 2009;12:122–127. doi: 10.1080/17518420902936730. [DOI] [PubMed] [Google Scholar]
  14. Mayo-Ortega L, Oyama-Ganiko R, LeBlanc J, Schroeder SR, Brady N, Butler MG, Marquis J. Mass screening for severe problem behavior among infants and toddlers in Peru. Journal of Mental Health Research in Intellectual Disabilities. 2012;5:246–259. doi: 10.1080/19315864.2011.590626. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Morris E, Higgins S, Bickel W. Comments on cognitive science in the experimental analysis of behavior. Behavior Analyst. 1982;5:109–125. doi: 10.1007/BF03392380. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Richman DM, Lindauer SE. Longitudinal assessment of stereotypic, proto-injurious, and self-injurious behavior exhibited by young children with developmental delays. American Journal on Mental Retardation. 2005;110(6):439–450. doi: 10.1352/0895-8017(2005)110[439:LAOSPA]2.0.CO;2. [DOI] [PubMed] [Google Scholar]
  17. Rojahn J, Aman MG, Matson JL, Mayville E. The Aberrant Behavior Checklist and the Behavior Problems Inventory: Convergent and divergent validity. Research in Developmental Disabilities. 2003;24:391–404. doi: 10.1016/s0891-4222(03)00055-6. [DOI] [PubMed] [Google Scholar]
  18. Rojahn J, Matlock ST, Tassé MJ. The stereotyped behavior scale: psychometric properties and norms. Research in Developmental Disabilities. 2000;21:437–454. doi: 10.1016/s0891-4222(00)00057-3. [DOI] [PubMed] [Google Scholar]
  19. Rojahn J, Matson JL, Lott D, Esbensen AJ, Smalls Y. The Behavior Problems Inventory: An instrument for the assessment of self-injury, stereotyped behavior, and aggression/destruction in individuals with developmental disabilities. Journal of Autism and Developmental Disorders. 2001;31:577–588. doi: 10.1023/A:1013299028321. [DOI] [PubMed] [Google Scholar]
  20. Rojahn J, Rowe EW, Sharber AC, Hastings RP, Matson JL, Didden R, Kroes DBH, Dumont ELM. The Behavior Problems Inventory-Short Form (BPI-S) for Individuals with Intellectual Disabilities I: Development and Provisional Clinical Reference Data. Journal of Intellectual Disability Research. 2012a;56:527–545. doi: 10.1111/j.1365-2788.2011.01507.x. [DOI] [PubMed] [Google Scholar]
  21. Rojahn J, Rowe EW, Sharber AC, Hastings RP, Matson JL, Didden R, Kroes DBH, Dumont ELM. The Behavior Problems Inventory-Short Form (BPI-S) for Individuals with Intellectual Disabilities II: Reliability and Validity. Journal of Intellectual Disability Research. 2012b;56:546–565. doi: 10.1111/j.1365-2788.2011.01506.x. [DOI] [PubMed] [Google Scholar]
  22. Rojahn J, Wilkins J, Matson J, Boisjoli J. A comparison of adults with intellectual disabilities with and without ASD on parallel measures of challenging behaviour: The Behavior Problems Inventory-01 (BPI-01) and Autism Spectrum Disorders-Behavior Problems for Intellectually Disabled Adults (ASD-BPA) Journal of Applied Research in Intellectual Disabilities. 2010;23:179–185. doi: 10.1111/j.1468-3148.2009.00519.x. [DOI] [Google Scholar]
  23. Schopler E, Reichler RJ, Renner BR. The Child Autism Rating Scale. Los Angeles, CA: Western Psychological Services Corporation; 1988. [Google Scholar]
  24. Schroeder SR, Rojahn J, An Xiaohu, Mayo-Ortega L, Oyama-Ganiko, LeBlanc J. The Parental Concerns Questionnaire: A valid brief screening instrument for severe behavior problems among infants and toddlers at risk for developmental delays. Journal of Developmental and Physical Disabilities. 2013 doi: 10.1007/s10882-013-9359-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Schroeder SR, Marquis J, Reese RM, Richman DM, Mayo-Ortega L, Lawrence LM. Risk factors for self-injury, aggression, and stereotyped behavior among young children at risk for intellectual and developmental disabilities. American Journal on Intellectual and Developmental Disabilities. doi: 10.1352/1944-7558-119.4.351. (in press) [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Sturmey P, Fink C, Sevin J. The Behavior Problems Inventory: A replication and extension of its psychometric properties. Journal of Developmental and Physical Disabilities. 1993;5:327–336. doi: 10.1007/BF01046389. [DOI] [Google Scholar]
  27. Thompson T, Lubinski D. Units of analysis and kinetic structure of behavioral repertoires. Journal of the Experimental Analysis of Behavior. 1986;46:219–242. doi: 10.1901/jeab.1986.46-219. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Wacker D, Berg W, Harding J, Derby M, Asmus J, Healy A. Evaluation and long-term treatment of aberrant behavior displayed by young children with disabilities. Developmental and Behavioral Pediatrics. 1998;19:260–266. doi: 10.1097/00004703-199808000-00004. [DOI] [PubMed] [Google Scholar]
  29. Wetherby AM, Prizant BM. Communication and Symbolic Behavior Scales Developmental Profile. Baltimore, MD: Paul Brookes Publishing Co., Inc; 2002. [Google Scholar]

RESOURCES