Development of the Communication Complexity Scale

Abstract

Accurate description of an individual's communication status is critical in both research and practice. Describing the communication status of individuals with severe intellectual and developmental disabilities is difficult because these individuals often communicate with presymbolic means that may not be readily recognized. Our goal was to design a communication scale and summary score for interpretation that could be applied across populations of children and adults with limited (often presymbolic) communication forms.

Methods

The Communication Complexity Scale (CCS) was developed by a team of researchers and tested with 178 participants with varying levels of presymbolic and early symbolic communication skills. Correlations between standardized and informant measures were completed, and expert opinions were obtained regarding the CCS.

Results

CCS scores were within expected ranges for the populations studied and inter-rater reliability was high. Comparison across other measures indicated significant correlations with standardized tests of language. Scores on informant report measures tended to place children at higher levels of communication. Expert opinions generally favored the development of the CCS.

Clinical implications

The scale appears to be useful for describing a given individual's level of presymbolic or early symbolic communication. Further research is needed to determine if it is sensitive to developmental growth in communication.

The purpose of the research reported in this paper was to investigate the creation of a scale of complexity of communication that emphasizes presymbolic stages of development. Before children speak their first words, they communicate using various gestures, body movements and vocalizations. These developmentally early forms of communication have been referred to as prelinguistic or presymbolic communication. For example, infants will often reach out to a parent to request being picked up. Even earlier than this, infants smile to convey contentment and fuss and cry to communicate displeasure.

The developmental changes that occur from birth through first words and/or early symbol use have been studied and documented for a variety of purposes in diverse populations. For example, Bates and colleagues have documented a typical progression in presymbolic communication and found that this progression relates to later symbol use and language (Bates & Dick, 2002; Thal, Bates, Goodman, & Jahn-Samilo, 1997; Volterra, Caselli, Capirci, & Pizzuto, 2005). Pre-intentional communication (also referred to as perlocutionary) includes behaviors that are purposeful but not clearly directed to another person—such as crying without accompanying gesture or eye gaze. Intentional communication includes gestures and vocalizations that are clearly directed to another person. Behavioral indications of directionality include eye gaze, touching, and body posture. Developmentally, these behaviors are followed by symbolic communication, typically spoken words (Brady, Bredin-Oja, & Warren, 2008).

Development of presymbolic communication has been described for individuals with different types of disabilities and disorders including hearing impairment (Zaidman-Zait & Dromi, 2007), brain lesions (Bates, Vicari, & Trauner, 1999), autism (Luyster, Qiu, Lopez, & Lord, 2007; Chian, Soong, Lin, & Rogers, 2008; Wetherby, et al., 2004), Down syndrome (Abbeduto, Warren, & Conners, 2007; Fidler, Philofsky, Hepburn, & Rogers, 2005), Williams syndrome (Singer-Harris, Bellugi, Bates, Jones, & Riossen, 1997), Angelman syndrome (Didden et al., 2009) and fragile X syndrome (Flenthrope & Brady, 2010). For example, children with Down syndrome typically have a profile of relative strength in gesture use across communicative functions (Abbeduto et al., 2007; Brady et al., 2008) whereas children with autism frequently have an uneven profile characterized by infrequent commenting gestures (Anderson et al., 2007; Kasari, Freeman, & Paparella, 2006)

Presymbolic behaviors have also been targeted in interventions aimed at ultimately improving language outcomes. Children with a variety of disabilities including those listed above have learned to communicate more often with presymbolic gestures and vocalizations following intervention (Fey et al., 2006; Pinder & Olswang, 1995; Yoder & Warren, 2001). Brady and Bashinski (2008) extended teaching paradigms aimed at increasing presymbolic communication to deaf-blind children.

It is important to accurately describe an individual's development in presymbolic behaviors for purposes of assessment and intervention planning. Accurate description of attainment of presymbolic levels can reveal if an individual is progressing along the expected continuum of development, or in some cases, whether patterns of performance are being demonstrated that might reflect particular disorders. For example, deaf-blind children may demonstrate interest in objects and events through focused attention, but may have difficulty demonstrating shared interest with another person through conventional means such as eye gaze and pointing (Bigelow, 2003; Rattray & Zeedyk, 2005).

In order to describe an individual's presymbolic communication, it would be extremely valuable to have a scale to measure increasingly complex, observable behaviors reflecting emerging levels of communication achievement. Further, a scale that captures the continuum of performance across differing ages and disorder types would enhance communication among researchers, practitioners and stakeholders. The challenge, of course, is defining behaviors along the continuum that can be reliably observed and still capture the nuances in performance of individuals who are presymbolic but who vary in age, disorder type, sensory and motor ability, and development. Ultimately, a reliable scale of communication complexity-- ranging from presymbolic to symbolic-- could generate a summary score for interpretation of performance across groups and populations. This summary score would reflect current status relative to the communication continuum itself, rather than a particular chronological age or other comparison group.

Parameters of Presymbolic Communication

The two main parameters that change during development of presymbolic communication, based on classic theories of early symbolization, are shift of attention toward the communication partner and an increased variety in form of communication (Bates, Benigni, Bretherton, Camaioni, & Volterra, 1979; Werner & Kaplan, 1984). The first parameter, shift in attention, reflects the extent to which infants (or older individuals with delays in development) direct their behaviors to their partner (Adamson & Chance, 1998; Baldwin, 1995; Tomasello, 1995; Trevarthen & Aitken, 2001). The major developmental milestone related to this parameter is the clear behavioral indication that the person is intentionally directing his/her behavior to the partner. For example, during mother-child interactions babies learn to direct their crying and gestures toward their communication partners by linking these potentially communicative behaviors (PCB) with some other behavior such as a shift in eye gaze, change in body posture or movement toward the communication partner. Shifts in orientation between referent and partner are a means to demonstrate coordinated joint engagement (Bakeman & Adamson, 1984).When infants start to reliably pair these shifts in directionality with their cries and gestures, their communication has been described as illocutionary or intentional communication (Bates et al., 1979; Brady, McLean, McLean, & Johnston, 1995; Ogletree, Wetherby, & Westling, 1992; Warren & Yoder, 1998).

The second main parameter that develops is the form of the communication behavior itself. In the first few months, infants are limited in terms of the variety of motor gestures or vocalizations that they can produce. However, towards the second half of the first year for most infants, motor development allows for a greater repertoire of gestures including pointing, waving, grasping and releasing objects, as well as increasingly complex vocal productions that include vowels and consonant sounds. Changes in the forms of early communication behaviors follow a developmental progression. Early gesture productions typically involve directly contacting the referent object or person, while later gestures include pointing toward referents and using representational gestures such as a “shh” gesture (Carpenter, Nagell, & Tomasello, 1998; Crais, Day Douglas, & Cox Campbell, 2004; McLean, Brady, McLean, & Behrens, 1998; Werner & Kaplan, 1984). Within a year or so, most children communicate with abstract symbols such as words or signs.

Changes across both of these parameters (directionality and forms) result in increasingly complex presymbolic communication. For example, cooing sounds emitted while a child stares at an interesting mobile may be interpretable by a familiar caregiver (e.g., the child “likes” the mobile). A more complex presymbolic communication would be looking back and forth between the mobile and Mom while cooing. And an even more complex communication act would include pointing to the mobile, then looking back at Mom while babbling.

In addition to these parameters, presymbolic communication varies in function. Some communication acts have behavior regulation functions such as requesting and rejecting and others have declarative/joint attention function such as pointing out something of interest (Brady, Marquis, Fleming, & McLean, 2004; Crais et al., 2004). It is important to note that children can typically use different forms to convey these different functions, hence complexity cuts across functions (McLean, Brady, McLean, & Behrens, 1998). Some types of disorders, such as autism, are marked by a delay in communicating certain functions (i.e., declaratives) while growth in communication complexity within other functions (i.e., requesting) may be observed.

Assessments of Presymbolic Expressive Communication

Typically developing children rapidly move through presymbolic communication and into word use. Hence, few standardized measures have been developed that assess communication at presymbolic levels. Those that are available rely primarily on parental (or other knowledgeable informant) report. For example, early items on the Mullen Scales of Early Learning (MSEL, Mullen, 1995) include questions about differential sound production in babbling and ask whether children play gesture/language games. The MacArthur-Bates Communication Development Inventory (MCDI) -Words and Gestures (Fenson et al., 1994) asks informants about production of many gestures, such as gives and points.

The Communication and Symbolic Behavior Scales Developmental Profile (CSBS DP) Caregiver Questionnaire (Wetherby & Prizant, 2003) is another informant report measure that includes presymbolic items. It consists of 41 multiple choice items covering the following areas: emotion and eye gaze, communication, gestures, sounds, words, understanding and object use. It is designed to be filled out independently by parents in about 20 minutes. Scores are totaled within each area and also combined for a total raw score. Both subscale and total scores can be compared to norms for children between the ages of 6–24 months. Raw scores may be used descriptively or in research with older children.

A few measures of presymbolic communication involve directly observing child communication behaviors. The behavior sample from the CSBS DP (Wetherby & Prizant, 2002) provides opportunities for children to communicate with gestures and vocalizations in addition to words. Children's productions are evaluated in comparison to a normative sample of children between 6 months and 2 years. The authors state that the CSBS may also be used with children up to the age of 6 if their developmental age is less than 2 years. In addition, the Early Social Communication Scales (ESCS, Seibert & Hogan, 1981) has been used in many research studies to describe children's presymbolic communication observed in a structured interaction.

Describing presymbolic communication complexity is important for individuals with more severe disabilities as well, specifically for those who may be functioning at a level equivalent to one year or below. A number of assessment tools have been developed for individuals with severe disabilities. Assessment protocols that are similar to the CSBS and ESCS have been tailored to meet the needs of older individuals with severe disabilities (McLean, McLean, Brady, & Etter, 1991) and to evaluate individuals needing specific accommodations for sensory limitations (Brady & Bashinski, 2008).

In addition to behavioral measures, several informant report measures have been developed for individuals with severe disabilities. Each of these assessments has been designed both to assess current communication and to design appropriate goals for individuals with severe disabilities. Several assessments rely on either checklists or interviews to help parents and other caregivers identify communication and communication-related behaviors. For example, The Checklist of Communicative Competencies (Triple C, Iacono, Bloomberg, & West, 2005; Iacono, West, Bloomberg, & Johnson, 2009) was developed to assess communication skills of adults with severe and multiple disabilities. The Triple C is a checklist of behaviors related to communication that was designed to be completed by support staff trained in observing particular behaviors. An advantage to this approach is that caregivers attend to communication-related behaviors over a length of time such as two weeks. Thus the behaviors recorded are likely to be highly typical for the individual being assessed. Examples of behaviors included in the Triple C are visually following slow moving objects or people and following a simple instruction out of a routine.

The Inventory of Potential Communication Acts (IPCA, Sigafoos et al., 2000) uses an interview format with open-ended questions to help respondents identify potential communication acts. For example, a respondent is asked to describe how the individual in question lets you know he/she wants something to eat. Depending on the talkativeness of the interviewee, the IPCA can be completed in under an hour. Responses are later summarized according to communication forms and functions (e.g. request, protest) using a grid.

The Communication Matrix is an interactive on-line assessment tool developed by Rowland and colleagues to obtain information about early communication (Rowland & Fried-Oken, 2010). The Communication Matrix is designed to be used by speech-language pathologists, educators, or parents to document the expressive communication skills of children who have severe or multiple disabilities. A familiar communication partner can complete the communication matrix in less than an hour. By means of a system of on-line questions and videotaped examples, informants describe the behaviors used to communicate four different functions: refuse things, obtain things, engage in social interaction, and seek information. For example, the informant would indicate what behaviors the target individuals would use to communicate the message “obtain more of something.” Results are presented in a matrix that describes levels of communication, emerging or mastered, for each of the four functions. Levels I–II are preintentional; levels III–IV are intentional presymbolic; levels V–VII are symbolic communication. Thus, several assessment tools are currently available that have been developed and used to describe individuals' expression of presymbolic communication. These tools have some limitations, however, that compelled the development of the CCS.

First, most available assessments rely on caregiver report. Although familiar partners have the advantage of knowing a child's behavioral repertoire, these respondents may over interpret early behaviors as being intentionally communicative and overestimate symbolic communication because of their familiarity with idiosyncratic behaviors and the contexts in which they are produced. Eadie and colleagues (Eadie et al., 2010) compared scores from the CSBS behavior sample to those from the parent report measures and found moderate correlations between measures, particularly between scales reporting gesture use. However, using a confirmatory factor analysis, the authors determined that results from the behavior sample had a more robust structure than did results from the parent report measure. These authors concluded that parents may have difficulty interpreting prelinguistic communication in comparison to more overt communication such as word production. Further, Eadie and colleagues point out that there are no gold standards for what to include on parent reports of early communication skills, and no agreement on what skills represent symbolic or communicative acts.

Differences in the results from parents and trained observers may reflect significant variability in prelinguistic communication ability across contexts. To provide reliable information about early communication development, observations by objective observers may offer a more balanced interpretation of how others might interpret presymbolic behaviors. Thus, the Communication Complexity Scale could offer supplemental information that could augment parent/caregiver report. Accumulating information from multiple sources is regarded as the optimum method for gaining a complete assessment (Brady & Halle, 1997; Ogletree & Fischer, 1996).

Second, many of the existing scales are not sensitive to subtle changes in complexity as communication develops or in response to intervention. This is particularly true for capturing subtle behavior changes that might occur across different aged and disordered populations. Further, having a tool that could be used to describe current communication functioning across populations of individuals with severe communication impairments would be useful in revealing similarities and differences in performance, providing important insights into this period of communication development irrespective of age and diagnoses.

A third limitation of current assessments of early communication is that most focus on a limited set of behaviors, particularly visual attending skills and communicative gestures, which are often not present or extremely limited in blind or multiply disabled individuals (Stephenson & Dowrick, 2005). Rattray and Zeedyk (2005) found that mothers and infants with visual impairments relied on touch, vocalizations and facial orientation to convey object interest and shared focus of attention. A scale that includes a broader spectrum of communicative prelinguistic signals to judge orientation to objects, people, or between objects and people would be valuable. For example, a blind individual may touch a vibrating object and then orient his/her body towards a person to indicate dual orientation.

Perhaps the most important limitation to the measures described above is that results don't yield a score that describes an individual's overall level of presymbolic/early symbolic communication development. Although a standardized score is often used with other populations to describe relative communication performance, many individuals with severe disabilities are too old to use the normative scales provided by tests such as the CSBS or the MCDI. Parent report measures such as the IPCA and the Communication Matrix provide information about current behaviors used to communicate many different functions such as refusing and asking questions, but not an overall level. For example, the Communication Matrix reports communication levels used to communicate 17 different functions (Rowland & Fried-Oken, 2010). This amount of detail can be extremely valuable in intervention planning; however, a single score may better communicate an individual's modal level of communication. An example of such a scale is the Rancho scale, often used to describe a patient's cognitive level after a stroke or traumatic injury (Hagen, 1987). A single score has the potential for reflecting change over time in populations of individuals who might be followed from childhood through adulthood, and for whom change is difficult to measure.

The scale described in the remainder of this paper was borne of the limitations and needs described above. The goal was to design a communication assessment scale for children and adults with limited (most often presymbolic) communication forms. The focus was on creating a tool that measured subtle, observable behaviors appropriate for varied populations, could be efficiently administered, and yield a summary score of performance.

Research Questions addressed

1. Can a scale be developed and reliably implemented that reflects differences in levels of presymbolic and early symbolic communication attainment?

2. Can such a scale be used with varying populations including toddlers, preschool age children and older individuals with severe disabilities?

3. How does this scale compare to existing measures of presymbolic and early symbolic communication, including standardized tests?

4. How do experts rate the value of the CCS?

Methods and Results

This paper presents the results of our efforts to develop and test the reliability and validity of Communication Complexity Scale (CCS). Three phases are described: 1) development of the scale including standardization across three projects, 2) reliability measures, 3) comparisons to other measures, and 4) expert opinions. The results of each of these phases are presented following a description of the methods used for each phase.

Participants

The CCS was developed by three teams of researchers at two different sites: University of Kansas and University of Washington. Our focus populations were (1) preschool-age children with intellectual and developmental disabilities including children with autism and Down syndrome who have been identified as candidates for AAC (Kansas-1; n = 93), all from English speaking homes, (2) infants 10–36 months of age with moderate-to-severe motor impairments who were nonverbal and not yet producing intentional communication signals, and thus identified as candidates for directed eye gaze intervention (Washington n = 28), 15 exposed to a language other than English, and (3) individuals with severe and multiple disabilities and suspected vision impairments of various ages (between 7–60 years) participating in a requesting via adaptive switch intervention (Kansas-2 n = 43), 3 exposed to a language other than English. Across all three projects, individuals with expressive vocabulary below 20 words (speech, signs or symbols) were included, and individuals with larger vocabularies were excluded. All participants had hearing levels within normal limits based on record reviews. Table 1 presents further descriptive information about the three populations. A variety of diagnoses, language exposures, motor abilities, and sensory abilities are represented across samples. The common characteristic is that communication was predominantly presymbolic.

Table 1.

Demographic information for participants in the Development of the CCS.

	Kansas 1		Washington		Kansas 2*
	Total Sample	Reliability Sample	Total Sample	Reliability Sample	Total Sample		Reliability Sample
					AD	CH	AD	CH
Number	91	47	28	11	26	33	11	12
Males	72	35	19	9	7	17	3	6
Mean C.A. in months and (SD)	49.1 (8.8)	48.8 (8.7)	17.1 (5.1)	20.2 (3.2)	592.1 (154.5)	125.3 (70.1)	586.4 (133.4)	115.2 (72.9)
Hispanic	6	3	4	2	0	12	0	6
Race
White	63	31	14	4	25	20	10	8
Black	14	11	3	1	1	8	1	2
Asian	3	1	3	1	0	1	0	0
Pacific Islander	1	0	1	1	0	1	0	0
Other	10	4	5	4	0	3	0	2
Unknown	0	0	2	0	0	0	0	0
Etiologies
Down Syndrome	15	4	5	1	0	0	0	0
ASD	44	24	1	0	0	0	0	0
Cerebral Palsy	5	1	3	2	0	0	0	0
Blindness	0	0	0	0	9	20	5	5
Deafness	0	0	0	0	0	0	0	0
Severe/Multiple	0	0	0	0	26	33	11	12
Other	32	18	19	8	0	0	0	0

Note.Kansas-2 sample is broken up into 2 parts, a child sample of those 21 years and under and an adult sample of those older than 21 years for descriptive purposes.

Assessment Contexts

The CCS is intended to be independent of the assessment context. Theoretically, CCS scores could be derived from various observational contexts. To date, however, the CCS has been applied to communication behaviors observed in scripted videotaped assessments of communication opportunities. All three of our assessment contexts would be described as “high structured” (Vandereet, Maes, Lembrechts, & Zink, 2010) because they were designed to elicit communicative behaviors (as opposed to low structured naturalistic observations). The scripted communication assessments were developed for each project based on unique attributes of the particular populations being investigated. The assessments themselves are not the focus of this paper, and we have therefore only provided a general description. However, further descriptions are available upon request.

In each of these assessment protocols, experimenters followed a script in which items or events (e.g., an unusual sound) were presented and the participant's response was scored with the CCS. The number of scripts varied across projects but each included a minimum of 12 different communication opportunities. Most opportunities were designed to elicit communication for behavior regulation. The experimenter stopped a desired activity, presented a choice between two objects/toys, or presented a desired object behind an obstacle that required experimenter assistance. At least three opportunities were designed to elicit joint attention through a violation of expected routine or the introduction of a novel event. Scripts and materials were designed to appeal to participant preferences for objects and activities and to efficiently provide opportunities for communication with an examiner. Administration took approximately 20–30 minutes for each of the populations.

For example, in the Kansas-1 study, a series of opportunities to request and comment, similar to those in the CSBS, were presented by an experimenter to each child. Activities were age appropriate for preschool age children, such as playing with a wind-up toy and or books whose pages had been disfigured (in hopes of eliciting a comment). In addition, a symbol book with symbols representing each activity was made available during the assessment. Children who were learning to use a speech generating device (SGD) had it available during the assessment (however, advanced programming of assessment vocabulary was not possible).

Opportunities were adapted for those with visual impairments in the Kansas-2 study assessment protocol. For example, some items had both sound and light such as a revolving LED display that whirred as it turned. Blind participants often held the item up to their ear, while those with some limited vision held the item close to and within their field of vision. Texture was also featured in items such as a furball that sang and spoke phrases while vibrating. Items such as vibrating toys were presented to participants with limited movement by moving the items lightly across an arm or hand. Assessments were conducted in low lighting to maximize luminosity of items used in the assessments.

In the Washington study, a variety of toys and books appropriate for infants and young children were presented in opportunities for communicative interactions and subsequent play. These materials included colorful toys with multiple parts (e.g., stacking rings) or required adult activation (e.g., musical top), which could be used to elicit turn-taking and requesting. A variety of simple sensory toys (e.g., textured balls, feathers, beads) were also available in an effort to appeal to individual child preferences. Books were simple, containing one picture per page.

Development of the Scale

The CCS has been developed to describe communication levels that reflect increasing degrees of coordination between referent and communication partner, and increasingly sophisticated forms of communication. Over the course of 18 months, authors from the three research sites viewed selected videotaped sessions of a scripted assessment for different participants from each site. During this time, a total of 12 videos were coded and scores compared, with subsequent discussions to develop and refine the content of the scale. The following adjustments and refinements were made to address challenges in interpreting behaviors of specific populations, and to meet the overall objectives of creating an instrument that could be used across ages and for older individuals with more severe disabilities.

The scoring is based on an ordinal scale with 11 different levels. Table 2 presents the scale, along with a short example of a behavior representative of each level. The 11 levels encompass the categories of perlocutionary (pre-intentional), illocutionary (intentional) and beginning locutionary (symbolic) communication development based on the literature (Wetherby, Warren & Reichle, 1998). The corresponding cut-offs for these stages are also noted on Table 2. One noteworthy aspect of our scale is the inclusion of points reflecting eye gaze behaviors (points 5, 7a, 8 and 9). For children with significant motor impairments, such as those being studied by the Washington project, eye gaze behaviors may signal both a shift in attention and a behavioral communication signal, reflecting both parameters of communication discussed above (Stephenson & Dowrick, 2005).

Table 2.

CCS Scale

Scale number	Definition	Example	Communication Level
0	No response	An opportunity is presented but the child looks away the entire time.
1	Alerting- a change in behavior	A vibrating toy stops vibrating and the child stops smiling	Preintentional
2	Single object/event/person orientation	Child handles a toy and focuses attention on only this toy	Preintentional
3	Single object/event/person orientation plus 1 PCB1	Child handles a toy and vocalizes	Preintentional
4	Single object/event/person orientation plus more than 1 PCB	Child reaches toward a toy and vocalizes	Preintentional
5	Scanning between objects/events	The child visually scans (shifts attention) between two different wind-up toys	Preintentional
6	Dual orientation between a person and an object or event	While playing with a wind up, the child looks up at partner	Preintentional
7a	Triadic eye gaze	Child looks from toy to partner and back to toy within a few seconds	Intentional nonsymbolic
7b	Dual orientation plus 1 or more PCBs	Child looks from toy to partner and vocalizes	Intentional nonsymbolic
8	Triadic eye gaze plus 1 PCB	Child looks from toy to partner and back to toy while vocalizing	Intentional nonsymbolic
9	Triadic eye gaze plus more than 1 PCB	Child looks from toy to partner and back to toy while vocalizing and giving toy to partner	Intentional nonsymbolic
10	One word verbalization, sign or AAC symbol selection	Child says “more” to request more cheerios	Intentional symbolic
11	Two or more word verbalizations, signs or AAC symbol selections	Child says “more please” to request more cheerios	Intentional symbolic

¹PCB = Potentially Communicative Behavior: Behaviors such as Vocalizations, gestures, eye gaze or switch closures that appear to be purposeful in response to the stimulus, and that could be viewed as communicating behavior regulation or joint attention or social interaction.

A coding manual was developed that further defined each behavior described in Table 2 along with examples of behaviors meeting each definition. The manual as well as the scale itself were revised through an iterative process between investigators that involved watching the videos independently, comparing scores, adjusting definitions and criteria, and then repeating these steps with a new video. More details on this process are provided below.

First, one of the authors from each project identified the start and stop time of each “opportunity” presented in the assessment. For example, in a task designed to assess joint attention, a brightly colored fan was started by a remote switch. The onset of this opportunity was the start of the fan and the stop time was approximately 30 seconds later. The average amount of time included in each opportunity was about 30 seconds; however longer increments were included for some individuals with motor impairments. A scoring form with the start and stop times was distributed to each coder (author) so that we could compare scoring for the same behaviors.

Next, we independently watched each identified opportunity on the video and recorded the most complex/highest scored communication behavior observed during that opportunity. For example, if both object scanning (number 5) and triadic eye gaze (number 7a) occurred during an opportunity, only a “7a” would be recorded. One scaled number was recorded per opportunity.

Achieving consistent scoring was particularly challenging for participants with severe motor and sensory impairments. We observed that some individuals would change or stop doing a behavior (e.g., start or stop body or head movements, smile or cease to smile, start or stop vocal fussiness, or open and close eyes) in response to, for example, a tactile stimulus such as vibrating toy placed in their hand. Although these behaviors appeared to be in response to the stimulus provided by the assessor, we were reluctant to describe these behaviors as orienting responses. Hence, we added `Alerting' to the low end of the scale; a behavior we placed higher than a `No Response' but below that of demonstrating orientation to an object/event or person.

A second challenge was scoring use of symbols via SGDs or Picture Exchange Communication System (PECS). Across different ages and ability levels, we observed that some individuals would select symbols but that these selections did not show clear evidence of a link with a specific desired event. There seemed to be two important distinctions to make in the use of these symbols. First, it was important to observe whether the individual selected the symbol (e.g. activated the SGD) as a response to the stimulus, which would suggest intentionality. We added a requirement that symbol selections needed to occur in conjunction with presentation/availability of corresponding referent stimuli. Symbol selections at other times (for example when she or he already had access to the corresponding referent) were not scored. Second, use of a switch or exchange of a symbol could not be considered symbolic if there was no evidence of discrimination. For this reason, if there was only one item in the selection set, for example a single PECS symbol or single message SGD, selection of that item could not be scored as a 10. Instead, this would be considered a 3 if there was no orientation to person, or a 7 if accompanied by orientation to person. The score of 10 was only given when the individual selected an appropriate symbol from a selection set with 2 or more symbols and this was accompanied by orientation to the examiner.

Another area of discussion focused on involvement of the examiner in the assessment, and how the examiner's behavior influenced the participant's performance. For example, if the examiner moved or talked to get the participant's attention, a response of looking at the examiner was not scored. Similarly, if the examiner's behavior influenced or caused triadic eye gaze such as comments that drew the participant's attention while he/she was looking at a toy, triadic eye gaze was not coded. This assisted the authors in agreement on what constituted a `prompted' as opposed to `unprompted' behavior. In this study, only unprompted behaviors were scored.

Given the physical and motoric limitations of participants across all three sites, another challenge was to decide what constituted a gesture as a PCB. After viewing videos of toddlers, preschoolers, and adults, the authors agreed upon detailed definitions for eligible gestures. For example, a complete `give' included both `push toward' clearly directed at the examiner and the release of the object. Similarly, coding of `reach' for an object required that the individual extend an arm/hand toward the object, but not grasp it. Throwing or manipulating toys were not considered gestures.

Finally, our reliability was enhanced by attending to procedural fidelity of videotaping the sessions across sites and timing issues. For example, a participant could only receive credit for `scanning' or eye gaze shifts between two objects if both objects were physically present and in view of the coder. An adaptation to this rule was necessary for individuals with severe visual impairments who showed scanning by exploring their environment with their hands and then touching each item. Timing issues were encountered related to varying lengths of time it took participants to shift their orientation between a toy and the examiner, or to engage in triadic eye gaze. If the amount of time required to physically demonstrate a shift in attention was individualized to accommodate a participant's physical abilities, this time adjustment was communicated to fellow raters.

Summarizing individual scores

It should be remembered that our goal was to quantify communication complexity for all participants in our various projects. Thus, we needed a way to summarize the results of each individual's responses across their respective scripted assessment. There are many different ways to summarize the complexity of communication for an individual. For example the scores for each response could be totaled. However, since each project used a different number of opportunities, the range of possible scores would vary across projects and thus, not be comparable. The overall mean could be used, but we felt that this score could be misleading. Through examination of the data, it was clear that many individuals responded with much lower levels of communication complexity in some of the scripted opportunities than in others. Responding varied across individuals and appeared not to be due to any consistent problem with the assessment script, but rather a lack of interest or even dislike for some particular stimulus on the part of the respondents. Thus, we concluded that using the average of the three highest scale scores recorded in a scripted protocol would provide a sensitive, overall picture of each participant's best communicative performance without relying on a single response. We calculated the average of the three highest scores for all the opportunities for each individual participant.

We set our criterion for cross-project scoring stability at +/− 1 CCS scores (the average of the top three responses). For example, if Rater A's three highest scores for participant 1 were 7, 7 and 8; the average top 3 CCS score would be 7.33. If rater B's CCS average of the top 3 scores for the same assessment was 7.00, these scores would fall within our criterion of stability. This criterion was met across the authors of this manuscript for the last three of the 12 assessments independently coded by each project, suggesting that cross-project coding had become stable.

Inter-rater Reliability

Within each project, a number of assessments were independently scored by two raters and compared for reliability calculations. For the Kansas-1 project, research assistants were taught to use the CCS in three steps. First, each research assistant viewed practice tapes with the first author. Second, research assistants coded practice video files independently and discussed discrepancies with the first author. Training continued in this fashion until research assistants met the established criterion (same as the stability criterion described above) for three practice assessment files. Approximately three hours of training (outside of the time to actually code tapes) were required to achieve this criterion.

For the Kansas-2 project, the investigator trained one research assistant by first discussing the definitions for each code and possible idiosyncratic behaviors that might be observed in this project's diverse population. The two researchers then viewed videotapes of several participants with differing sensory and motor disabilities and discussed how to code each script. Next, researchers viewed scripts separately and then compared scores.

For the Washington project, the first set of 8 videos was coded by two co-authors who helped develop the scale so no additional training was required. For a second set of 8 videos, a student coder was trained in the following manner. First the student read the manual in depth, and asked questions about the coding instructions. Second, the student coded practice video files independently and discussed discrepancies with the co-author who had already coded the same files. This training continued until the student coder met the established criterion on three videos. The student coder then completed a total of 8 additional videos, of which one co-author coded 25% for continued reliability checks.

Independent reliability was measured for a total of 92 different CCSs. Fifty-seven of the scores were from the Kansas-1 project; 11 were from the Washington project and 24 were from the Kansas-2 project. Information about the participants in the reliability sample is presented in Table 1. We used three different measures of inter-rater reliability for the average of the top 3 scores across the scripts administered: Intraclass Correlation Coefficients (ICCs), the number of scores by both independent raters that were within 1 point of each other, and the agreement across communication level categories. We also examined the point-by-point reliability of scores for each assessment.

Intraclass correlation coefficients were calculated for the total sample as well as within projects to assess the absolute agreement between two independent raters on this single score. The ICC for the total sample was .98 indicating consistent, reliable coding. Within project ICCs were also high: .95 for Kansas-1, .99 for Washington and .95 for the Kansas-2 project.

Another way to examine consistency is to examine how many of the CCS scores were identical, within one point of each other, or greater than 1 point difference across the independently coded assessments. Fifty-seven of the 92 independently rated assessments had an identical CCS score, and 28 were within 1 point of each other. Thus, 92% scores from independent raters fell within 1 point of each other.

An important consideration is the extent to which the ratings are reliable across three broad categories of communication levels. CCS scores of 0–11 map onto three broader Communication levels--pre-intentional (perlocutionary), intentional non-symbolic (illocutionary), and symbolic. Participants with scores of 6.499 and lower are considered Pre-Intentional. Participants with scores in the 6.50 – 9.499 range are considered to be Intentional non-symbolic, and participants with scores 9.50 and greater are considered to be communicating at the Symbolic level. Table 3 shows the agreement across categories of communication levels by independent raters. The data are separated out for each project. Across all projects, the vast majority of scores by independent raters fell within the same category of communication level.

Table 3.

Agreement across Communication Levels.

	Kansas-1 Project
	Rater 1
Rater 2	Pre-intentional	Intentional Non-symbolic	Symbolic	Total
Preintentional	2	1	0	3
Intentional Non-symbolic	2	14	6	22
Symbolic	0	2	20	22
Total	4	17	26	47

	Kansas-2 project
	Rater 1
Rater 2	Pre-intentional	Intentional Non-symbolic	Symbolic	Total
Preintentional	19	1	0	20
Intentional Non-symbolic	0	3	0	3
Symbolic	0	0	0	0
Total	19	4	0	23

	Washington Project
	Rater 1
Rater 2	Pre-intentional	Intentional Non-symbolic	Symbolic	Total
Preintentional	3	0	0	3
Intentional Non-symbolic	0	8	0	8
Symbolic	0	0	0	0
Total	3	8	0	11

Point-by-point reliability on the individual scripted communication opportunities within each assessment (i.e., scripted opportunities) was also quite high. Kansas Project 1 had a total of 672 items scored by two raters; 76% of the items were scored consistently (within 1 point) between the two raters and Kappa was .44. For the Washington sample, a total of 270 different items were scored by two raters. Eighty-nine percent (241) were within 1 point of each other. Kappa was .77. Kansas Project 2 had a total of 280 different items scored by two raters. Ninety-four percent (262) were within 1 point of each other. Kappa was .78.

Comparison to Other Measures

We compared scores on the CCS to scores on two standardized tests and two informant interview assessments for some participants. Although there isn't a gold standard available to compare CCS scores, we were interested to see how CCS scores compared to currently available measures that target similar developmental ages and stages.

Standardized tests

We compared the CCS to scores from the expressive language scale of the MSEL for both the Kansas-1 data and for the Washington data. In addition, we compared CCS scores to the expressive scale of the Preschool Language Scale (PLS-4, Zimmerman, Steiner & Evatt Pond, 2003) for the Kansas-1 project. Items on these tests are designed to be used with children below 1 year of age and thus their scores should reflect a comparable stage of communication development. Across both sites, the same examiner typically administered both the standardized measure and the scripted assessment. However, the CCS was scored by a different individual who was blind to the standard language assessment scores. Participants on the Kansas-2 project were unable to comply with the behavioral requirements of either the MSEL or the PLS-4.

For the MSEL, we had a total of 106 participants that had both the MSEL and the CCS. Sixteen of the participants were from the Washington project and 90 were from the Kansas-1 project. The Pearson correlation between the MSEL Expressive Scale Raw Score and the CCS score was .40, significant at the .01 level.

For the PLS-4, we had a total of 91 participants who had both the PLS-4 and the CCS. All of these participants were from the Kansas-1 project. The Pearson correlation between the total raw score for the expressive language scale and the CCS was .438, significant at the .01 level.

Informant report measures

The Communication Matrix and the CSBS-DP Caregiver Questionnaire are two measures of presymbolic communication that rely on parent (or other knowledgeable informant) report. Both of these measures were described in the Introduction.

Twelve mothers of participants in the Kansas-1 study and 3 parents of participants in the Kansas-2 study completed the Communication Matrix online using the procedures described on the website associated with this assessment, and cited earlier in this paper. The mothers were not aware of CCS scores at the time of completing the Communication Matrix. At the conclusion of the Communication Matrix, the respondent was asked if they would be willing to share results. At our request, each mother specified that results could be shared with members of our research team. Results from the Communication Matrix are organized according to the highest level (1–7) mastered or emerging within each function: refuse things, obtain things, engage in social interaction, or seek information. In order to obtain a score that was somewhat similar in scope to the CCS, we calculated two average scores for each participant: 1) the average of the highest level mastered across the four functions, and 2) the average of the highest level emerging across the four functions. Each of these levels (mastered and emerging) was then interpreted as corresponding to preintentional, intentional, or symbolic using the guidelines established by the author (Rowland, 2010).

Table 4 shows the comparison between the levels obtained from the 15 Communication Matrices completed and the CCS. In five cases, the highest emerging score on the Matrix was consistent with the CCS, and in seven cases the highest score mastered on the Matrix was also consistent with the CCS. In 9 cases the highest emerging score on the Matrix was a category higher than that indicated by the CCS; and five participants' highest mastered scores on the Matrix were a category higher than those on the CCS. These higher scores on the Matrix were expected due to the greater familiarity of informants, and the larger scope of behaviors inventoried; however, we also had one case of a higher score on the CCS than the highest emerging score on the matrix and two cases where the CCS was higher than the category for the highest mastered score on the Matrix. Hence, the CCS and the Communication Matrix scores were similar but not identical, suggesting an advantage to collecting both observational and informant measures.

Table 4.

Comparison between CCS and Communication Matrix Scores.

Participant	CCS score and category	Matrix: Mastered Category	Matrix: Emerging Category
1	9.9 Symbolic	6 Symbolic	6 Symbolic
2	7.0 Intentional	4 Intentional	6 Symbolic
3	6.7 Intentional	3 Intentional	6 Symbolic
4	8.6 Intentional	6 Symbolic	7 Symbolic
5	9.0 Intentional	7 Symbolic	7 Symbolic
6	9.3 Intentional	6 Symbolic	7 Symbolic
7	9.0 Intentional	7 Symbolic	7 Symbolic
8	8.0 Intentional	3 Intentional	6 Symbolic
9	11.3 Symbolic	4 Intentional	7 Symbolic
10	7.7 Intentional	4 Intentional	6 Symbolic
11	12.0 Symbolic	3 Intentional	3 Intentional
12	5.3 Pre-Intentional	3 Intentional	7 Symbolic
13	1.3 Pre-Intentional	1 Pre-Intentional	1 Pre-Intentional
14	1.3 Pre-Intentional	2 Pre-Intentional	2 Pre-Intentional
15	3.0 Pre-intentional	2 Pre-intentional	2 Pre-intentional

Scores from the CSBS-DP Caregiver Questionnaire were compared to CCS scores for 13 children from the Washington project. Scoring of the CSBS-DP was completed by the same examiner who scored the MSEL, but who was not involved in the CCS coding. The Spearman rho correlation for total raw score for the CSBS and the CCS was −.023 and insignificant (p = .94). The correlations between CCS and the individual raw subscales of the CSBS ranged from .17 to .42, with the highest correlation (.42) occurring between the Emotion and Eye Gaze score and the CCS. This score reflects in part the parents' impressions about their children's engagement with people and toys, which is similar to the scoring on the CCS that appeared most common. Low correlations on other scores and the overall CSBS score may reflect many questions requiring motor responses in the CSBS questionnaire, and low motor skills in the Washington participants.

Content Validity- Expert Opinions

We asked four nationally recognized experts in early communication development to review the CCS. Each expert has had over ten years of experience conducting clinical research on early communication development. Copies of the scale along with definitions and written examples of each level were provided via email along with the following questions:

1. Do you agree with the order of items on the scale?

   • 1a
     if not, how would you re-order the items?
2. Are there additional items you would add to the scale?

   • 2a
     if so, please describe where you would add these items
3. Are there items you would not include on this scale?

   • 3a
     if so, please list these items.
4. In your opinion, would other researchers find value in using a scale such as the CCS to describe participants' current levels of expressive communication development?

   • 4 a
     Why or why not?
5. In your opinion, would clinicians find value in using a scale such as the CCS to describe client/student's current level of expressive communication development?

   • 5a
     Why or why not?

6. Are there any other comments you would like to share regarding the CCS?

Several common themes were observed in the experts' answers to questions 1–3, regarding the inclusion of items and ordering of the scale. Regarding the lower end of the scale, two experts questioned assigning a value of level 5 to dual orientation to two objects, stating that this behavior could be viewed as either an expansion of level 4, or eliminated altogether “on the basis that this doesn't involve communication.” Regarding the upper end of the scale, experts questioned the ability to reliably identify level 10, intentional symbolic behavior (1 spoken word, sign, or AAC symbol). One expert felt it would be important to differentiate using a symbol to represent a specific object/event vs. using a single symbol such as the “more” sign in many contexts. Another expert thought that differentiating holophrases from generative multi-symbol utterances would be important.

Regarding the value of the scale (questions 4 and 5), three of the four experts felt the scale was extremely valuable and needed for both research and clinical practice. One expert stated that the scale is a distinct asset to the field. Another stated that she felt the scale could be helpful in providing descriptive information on a child's level of communication, and could also help with developing goals for intervention. One expert, however, did not feel that the current version of the scale was useful for research because “it doesn't get at communicative functions or the variety of symbols that they use.”

These expert opinions indicate that there is need for a scale such as the CCS but that some adjustments could improve the usability across individuals and contexts. The comments regarding scores for symbol use are guiding our research group to further delineate and expand the upper (symbolic) end of the scale for future applications. In response to the comment about communication functions, it is true that the CCS only reflects development in communication forms. However, a researcher or clinician could calculate CCS scores within specific communication functions, and compare these scores within or across individuals to obtain information about relative development within functions. The scripted opportunities within a given assessment may include opportunities for behavior regulation (including protests), joint attention, and social interaction. CCS scoring could capture an individual's performance within these functions. For example, all the assessments used in creating the CCS included both behavior regulation and joint attention opportunities. The Kansas-1 project plans to compare CCS scores within functions as a way to describe communication profiles for the participants.

Face Validity of CCS Scores

Finally, in terms of evaluating the validity of the CCS scores, we looked at the means and ranges and the number of scores that fell within the three categories of communication levels (preintentional, intentional nonsymbolic, symbolic) across the three projects. These figures are presented in Table 5. The Kansas-2 project focused on participants with the most severe communication impairments and the average CCS score for this project was 2.72. Ninety percent of the Kansas-2 participants fell within the preintentional communication level. The mean score for the Washington participants was 7.13 and 75% fell in the intentional, nonsymbolic range. Given that most of the participants in the Washington study had motor impairments and were learning to intentionally communicate with eye gaze/orientation, these figures seemed appropriate. Finally, the mean score for the Kansas-1 project was 8.87 and 39% participants had scores in the symbolic range. Many children in the Kansas-1 study were communicating with manual signs and other forms of AAC, thus these numbers also seemed appropriate.

Table 5.

CCS scores for each project.

Project and sample size (n)	Mean score	Number preintentional	Number intentional	Number symbolic
Kansas 1 (91)	8.83 (1.41)	5	52	34
Washington (28)	7.30 (1.52)	6	22	0
Kansas 2 (59)	2.97 (2.08)	52	7	0

Discussion

We created the CCS to provide a description of the level of communication observed in early/beginning communicators. Based on the data provided in this current paper, we conclude that the CCS is applicable across varied populations and age ranges. Reliable scores were obtained with three different research populations, and the scores appeared to reflect expected ranges of performance. The CCS appears to accurately describe individuals' levels of early expressive communication based on comparison to other measures and expectations for the different populations investigated. In addition, correlations with other formal and non-formal assessments indicate that our results are consistent with--but not redundant to-- these measures.

The CCS may fill a need in research with individuals in early stages of communication by providing a means to describe observed communication levels. The CCS score could facilitate communication between researchers and interventionists in terms of describing a particular individual's level of expressive communication development, the average communication level for a particular group of individuals (e.g., those involved in an intervention study), and may reflect changes in communication over time.

One potential advantage of the CCS is that it could be used to supplement scores from other similar assessments such as the ESCS (Seibert & Hogan, 1981), the CSBS behavior sample (Wetherby & Prizant, 2002), or assessments constructed for particular populations (e.g., Brady & Bashinski, 2008). These assessments also providescripted opportunities to communicate using nonsymbolic forms of communication; yet results are not summarized with a score such as the CCS. CCS scoring could be used to describe performance in these assessments using the same methods reported in this study. The CCS score wouldn't replace results such as rates of communication, or forms and functions available but would provide a consistent descriptor of an individual's expressive communication level observed in the assessment.

Further, in clinical settings that serve nonverbal individuals of different ages and a variety of disorders, the CCS could serve as a common behavioral assessment that should be valuable for planning intervention and measuring progress. For example, if an individual had an initial CCS score of 6 (near the top of the preintentional range), interventions aimed at increasing intentional communication would seem appropriate. Subsequent evaluations may show increases in scores on the CCS, reflecting successful intervention efforts.

Limitations and future directions

The CCS was developed by a team of researchers to facilitate description of participants across projects. Our sample population included individuals participating in three research sites, but the participants were not randomly selected. Although our intent is to use this measure to also measure progress in intervention, we have not yet determined if the CCS will show variability in response to interventions and/or correspond to social validation measures of treatment effects.

Two findings suggest the need to further develop the upper ends of the CCS. First, expert evaluators indicated that they would have difficulty discriminating between behaviors viewed as “symbolic” and rote associative responses using the definitions provided. For example, if a child selects a symbol on a speech generating device to request a break, but this is the only symbol available on the device, should this be considered a single symbol response, or receive a lower score on the scale? Second, reliability for individual assessment opportunities was better for both the Kansas-2 and the Washington study than for the Kansas-1 study. One reason that the reliability for the Kansas-1 study may be lower is that children were generally performing at higher levels compared to the other two projects' participants. Further clarification and perhaps further delineation of the upper end of the scale may increase both validity and reliability of the scale.

The scale presented here describes levels of performance based on the form of communication behavior, without regards to function. Our assessments included opportunities to produce different communicative functions such as behavior regulation and joint attention, but the scale itself doesn't differentiate or even weight responses across these different functions. Differences in functions are frequently of interest to both researchers and clinicians. Although we suggested that different CCS scores could be computed within functional classes and that comparing these sub-scores would likely provide information about profiles of responding within and across functions, we did not provide any data demonstrating this. The information provided within this paper is limited to describing the level of communication forms (regardless of function) observed by a particular individual.

Our future research plans include investigating the degree to which CCS scores reflect changes over time. Changes may be due to development or regression (associated with some debilitating conditions) or may result from interventions. The authors of this manuscript are following three different populations over several years and will be able to evaluate the degree to which CCS scores reflect change over time, including as related to intervention.

Summary

The CCS may fill a need in research with individuals in early stages of communication by providing a means to describe observed communication levels. The ability to describe these pre and early symbolic communication behaviors across different populations may also enlighten our understanding of communication development in individuals with severe disabilities. The CCS score could facilitate communication between researchers and interventionists in multiple ways. The CCS could reflect current level of performance or the change over time in a particular individual or in a group of individuals, for example those involved in an intervention study. A CCS score is viewed as a complement to information obtained from parent/informant report assessments. Combined, these measures present a more complete picture of a given individual's current communication.