The influence of scale structure and sex on parental reports of children’s social (pragmatic) communication symptoms

Abstract

The addition of social (pragmatic) communication disorder [S(P)CD] to the DSM-5 taxonomy has left clinicians and researchers searching for appropriate diagnostic measures. Factor analysis procedures examined the extent to which S(P)CD symptoms presented within the Children’s Communication Checklist-Second Edition (CCC-2) represented a unique construct and whether these factors were influenced by children’s sex. Parents of 208 children (males = 125 and females = 83) from a community-based sample completed the CCC-2. Two pragmatic scores from the CCC-2 were analysed as follows: the social interaction difference index (SIDI) and a pragmatic composite from the original CCC (PC-5). Factor analysis failed to find a unique factor structure for either pragmatic composite. Analyses uncovered different factor structures for the CCC-2 SIDI and PC-5 composites and for boys and girls. S(P)CD represents a complex combination of symptoms that are poorly differentiated from other language and socioemotional behavioural difficulties.

The recent addition of the social (pragmatic) communication disorder [S(P)CD] designation into the Communication Disorders section of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5: American Psychiatric Association, 2013) ascribes a diagnostic entity to symptoms that have been recognised in the field of speech-language pathology for over thirty years (Bishop & Rosenbloom, 1987; Rapin & Allen, 1983). Pragmatic symptoms frequently associated with earlier designations included inadequate conversational skills, poor topic maintenance and answering beside the point of a question (Rapin, 1996). S(P)CD offers speech-language pathologists a formal mechanism to provide services to children whose limitations in language may not otherwise qualify for clinical services on the basis of deficiencies in their structural language performance (syntax and morphology).

The diagnostic criteria for S(P)CD expand upon early descriptors, as demonstrated in the current inclusionary and exclusionary criteria presented in Table 1 (American Psychiatric Association, 2013, p. 48). Notably, all of the components under Diagnostic Criteria A must manifest in order to receive a positive diagnosis of S(P)CD. Deficits must result in a functional communication deficit with the onset of symptoms in early development although these may not be apparent until social communication demands exceed individuals’ capacities. Exclusionary criteria include other medical or neurological conditions, such as autism spectrum disorder, and low abilities in grammar. The latter criterion will make the routine diagnosis of S(P)CD challenging because the most common associated feature of the disorder is language impairment, and one of the symptoms of language impairment is problems with discourse management (APA, 2013). Therefore, identification methods for S(P)CD will need to account for social/pragmatic symptoms and structural language deficits, providing practitioners with a way to disentangle the relative contributions of these two sources to children’s observed social difficulties. Additionally, the identification of S(P)CD requires differential diagnosis from conditions which may present with similar secondary symptoms, such as anxiety disorders or attention deficit/hyperactivity disorder (ADHD).

Table 1.

DSM-5 Social (pragmatic) communication disorder inclusionary and exclusionary criteria (American Psychiatric Association, 2013).

Diagnostic criteria	Notes
Inclusionary
A. Persistent difficulties in social use of verbal and nonverbal communication in the following: Deficits in communication for social purposes Impairment in the ability to change communication to match context or needs of the listener Difficulties of following rules for conversation or storytelling Difficulties of understanding what is not explicitly stated and nonliteral or ambiguous meanings of language	All four must be present
B. Deficits result in functional limitations in effective communication, social participation, social relationships, academic achievement or occupational performance	May occur individually or combined
C. Onset of symptoms is in the early developmental period	Deficits may not manifest until social communication demands exceed capacity
Exclusionary
D. Symptoms are not attributable to another medical or neurological condition or to low abilities in the domain of word structure and grammar and are not better explained by autism spectrum disorder, intellectual disability, global developmental delay or another mental disorder.

Children’s Communication Checklist-2

The Children’s Communication Checklist-2 (CCC-2: Bishop, 2006) is a widely studied parent report measure designed to account for both social/pragmatic and structural language deficits in school age children and may represent one of the few psychometrically valid options for screening for S(P)CD (Norbury, 2014; Swineford et al., 2014; Volden & Phillips, 2010). The CCC-2 is a standardised 70-item parent questionnaire that measures children’s structural and pragmatic language abilities. The ten scales include Speech, Syntax, Semantics, Coherence, Initiation, Scripted Language, Context, Nonverbal Communication, Social Relations and Interests. Scales may be grouped together as composites to assess potential pragmatic difficulties, specific language impairment (SLI) and autism spectrum disorder (ASD). The CCC-2 provides users with individual scaled scores and composite scores, such as, the General Communication Composite and the Social Interaction Difference Index (SIDI).

Historically, the central features of pragmatic disorder have been difficult to assess. Norbury (2014) suggested that the Social Interaction Difference Index (SIDI) may represent one of the few validated options for screening for pragmatic difficulties. The SIDI captures potential differences in children’s performances between structural language dimensions and pragmatic difficulties, a key element of the exclusionary associated with the S(P)CD designation. The SIDI is calculated by subtracting the sum of the sum of the pragmatic/social-interaction scales (i.e. Initiation, Nonverbal Communication, Social Relations and Interests) from the sum of the structural language scales (i.e. Speech, Syntax, Semantics and Coherence). Accordingly, positive SIDI scores are characteristic of stronger pragmatic abilities relative to structural language skills, while negative scores suggest stronger structural language abilities relative to pragmatics.

Use of the CCC-2 in classification studies

The SIDI has been used to document pragmatic performance across a wide range of neurodevelopmental disorders (Adams et al., 2012; Botting, 2004; Freed et al., 2015; Geurts & Embrechts, 2010; Helland, 2014; Laws & Bishop, 2004; Timler, 2014). However, attempts at using the SIDI or SIDC (SIDC, called the Social Interaction Deviance Composite [SIDC] in the U.K. version) to differentiate pragmatic language difficulties from symptoms of language impairment or autism spectrum disorder (ASD) have had various levels of success (Norbury & Bishop, 2005; Norbury, Nash, Baird, & Bishop, 2004). As a result, multiple investigations utilising the CCC-2 as a pragmatic measure have resorted to creating their own versions of pragmatic composites for experimental purposes in order to capture clinically significant pragmatic weaknesses (Adams et al., 2012; Bignell & Cain, 2007; Geurts & Embrechts, 2010; Helland, 2014; Law, Rush, & McBean, 2014; Leonard, Milich, & Lorch, 2011; Timler, 2014). Composites have varied across studies, reflecting different combinations of the individual CCC scales. Table 2 summarises the pragmatic composites used across various studies. It is unknown what, if any, impact the different pragmatic composites would have on the capacity of the CCC-2 to identify S(P)CD symptoms.

Table 2.

CCC-2 pragmatic composites.

Citation	Pragmatic composite from the CCC-2
Adams et al. (2012)	15 items from list of 20 items
Bignell & Cain (2007)	coherence, context, conversational rapport, initiation and stereotyped language
Geurts & Embrechts (2010)	coherence, context, initiation, nonverbal communication and scripted language
Helland (2014)	initiation, interests, nonverbal communication, stereotyped language and social relations
Law et al. (2014)	coherence, context, initiation, nonverbal communication and stereotyped language
Leonard et al. (2011) Timler (2014)	context, initiation, interests, nonverbal communication, scripted language and social relations

Previous studies comparing children with grammatical language impairments and pragmatic difficulties have found overlapping symptoms on measures of language and social/pragmatic abilities. Botting and Conti-Ramsden (2003) investigated the ability to distinguish specific language impairment (SLI), pragmatic language impairment (PLI) and autism spectrum disorders (ASD) using psycholinguistic markers. Children were identified with PLI based upon scores on the CCC pragmatic scale score < 132 (CCC: Bishop, 1998). Observed group differences were complex and inconsistent across psycholinguistic indices measuring nonword repetition, past tense marking and sentence recall. The authors concluded that the PLI groups could not be reliably distinguished from either the ASD or SLI groups on the basis of psycholinguistics markers.

Norbury et al. (2004) looked at group differences among participants with SLI, PLI, PLI + autism features, high functioning autism (HFA) and ASD on the CCC-2. They compared the new CCC-2 pragmatic composite and the original CCC pragmatic composites, reporting that they were generally unable to distinguish between groups. The exception was the group with HFA, who performed significantly lower on the original CCC pragmatic composite. When the SIDC was used to examine differences between pragmatic and structural abilities, they found that when the GCC score was < 55 (standard score of < 80), the SIDC could be used to identify children with pragmatic difficulties, although the SLI group was not significantly higher than the PLI group.

While these studies indicate that social/pragmatic difficulties may be difficult to disambiguate from other developmental disorders, other research has provided evidence that pragmatic difficulties can present as a unique disorder. Helland (2014) used a Norwegian version of the CCC-2 to investigate language profiles of children with specific language impairment (SLI) and children with an Asperger syndrome (AS) diagnosis. Children with AS significantly outperformed children with SLI on the structural scales of the CCC-2, while children with SLI had significantly higher pragmatic scores than the AS group, indicating that the groups had distinctly different language profiles as measured by the CCC-2. Tomblin et al. (2014) examined CCC-2 scores from eighth grade students with and without language impairments. Three-hundred and ninety-four participants completed multiple language measures of discourse comprehension and production, as well as parental reports on the CCC-2. Principal component analysis identified pragmatic skills measured by the CCC-2 SIDC as a unique construct when analysed along with standardised language tests and conversational-based language measures.

Bishop and Norbury (2002) examined children with specific language impairment (SLI) and pragmatic language impairment (PLI) to investigate the independence of PLI from autistic disorder. Autism measures, the CCC, and Social Communication Questionnaire (SCQ) were used to uncover which children had autism symptoms. In the PLI group, 4 of 13 children met the criteria for an autism diagnosis, with 2 additional children meeting the criteria for pervasive developmental disorder not otherwise specified (PDDNOS). While this finding indicates potentially overlapping symptoms in PLI and autism in some children, other children in the PLI group did not demonstrate symptoms of autism, supporting PLI as a potentially unique language disorder.

Sex and social-pragmatic performance

Sex differences in rate and symptom severity across psychiatric disorders have been well documented (Alegria, Jackson, Kessler, & Takeuchi, 2001–2003; Eaton et al., 2012). As a result, many psychiatric assessments for children and adolescents provide norms disaggregated by sex (Achenbach & Rescorla, 2001; Conners, 2008; Reynolds & Kamphaus, 2015). In contrast, in communication assessment sex-based norming has been limited (Fenson et al., 2007; Goldman & Fristoe, 2015). Despite infrequent sex-based norms in assessment tools, sex differences in language performance across childhood have been reported. A study examining oral and fictional written stories in 538 children with and without language disorder observed a main effect for sex, with girls performing higher than boys on story structure, linguistic complexity and narrative quality (Fey, Catts, Proctor-Williams, Tomblin, & Zhang, 2004). Rice and Hoffman (2015) longitudinally examined receptive vocabulary growth in those with and without SLI. Their findings revealed different growth patterns influenced by age and sex interactions, such that unaffected girls at 21 years had similar vocabulary levels to unaffected boys at 18 years. Additive effects for gender and SLI were also found. Affected boys at 21 years had similar vocabulary levels as unaffected girls at 18 years, while the vocabulary of affected girls at 21 years was similar to unaffected girls at 15 years.

Reports on pragmatic differences on the basis of sex provide mixed findings. Some studies report no significant differences in pragmatic abilities based upon sex (Collins, Lockton, & Adams, 2014; Law et al., 2014; Leonard et al., 2011). However, Ketelaars et al. (2010) found sex-based differences in pragmatic competence in a community-based sample of 553 girls and 548 boys. Results on the Dutch translation of the CCC (Bishop, 1998; Hartman et al., 1998) indicated significant sex differences across all of the scales and the Pragmatic Composite, with girls demonstrating higher levels of pragmatic ability than boys. Geurts et al. (2009) also reported that girls had higher performance on the CCC Pragmatic Composite when using the Dutch CCC in a study of 797 boys and 792 girls. The influence of sex on parent ratings of pragmatic symptoms is unresolved.

Current study

The primary aim of the current study was to investigate further the nature of clinical symptoms associated with S(P)CD as measured by the CCC-2. We used exploratory factor analysis to determine the underlying structure of S(P)CD symptoms within the context of other linguistic and socioemotional behavioural symptoms. Due to the various scoring schemes utilised in previous investigations, we also examined the extent to which different composites yielded different solutions. Sex differences in overall CCC-2 scores have been reported by some investigators. If factor analyses were to indicate that sex differences extend as well into the underlying trait structures of S(P)CD symptoms, this would encourage the development of sex-based criteria and other service accommodations

Method

Approval for this project was granted by the institutional review board at the University of Utah. Written parental consent and child assent to participate in the study were obtained.

Participants

Participants were recruited from a larger community-based sample of 1,060 children who took part in an investigation of language screening measures for students from kindergarten to grade 3. All students within targeted elementary schools who were enrolled in regular education and those who were receiving services for communication disorder, emotional behavioural disorder, reading disability or learning disability were invited to participate. Participants who failed the locally normed language screening measures were invited to participate in confirmatory assessments conducted by examiners blinded to children’s clinical status and screening performance. A grade-matched group of children who passed the screenings were also recruited based upon random number assignment. In total, 125 boys and 83 girls participated in this study.

All participants were monolingual speakers of Standard American English. Potential participants were excluded if they (1) were enrolled in ‘gifted’ or ‘enrichment’ programmes or (2) failed a hearing screening (at 25 dB across 500, 1000at 25 dB across 500, 2000 and 4000 Hz). Children were not excluded from confirmatory testing if they had positive histories of developmental or health difficulties or if they were receiving services for learning disabilities or emotional behavioural disorders. Table 3 provides racial and ethnicity information, rates of service receipt by a speech-language pathologist, children’s age, maternal education and nonverbal ability. There were no significant differences between boys and girls on these variables.

Table 3.

Descriptive information for boys and girls: Group means (M), standard deviations (SD) and range.

	Boys (n = 125) M (SD) Range	95% CI	Girls (n = 83) M (SD) Range	95% CI	F(1, 207)	η2p
Caucasian	87.02%	–	94.00%	–	2.21	.011
Hispanic	7.4%	–	4.80%	–	.540	.003
Receiving SLP services	36.8%	–	28.9%	–	1.39	.007
Child’s age	7;9 (1;1) 6;0–10;0	–	7;8 (0;11) 6;0–10;3	–	.086	.000
Mom’s educationa	3.65 (.93) 1–5	–	3.53 (.91) 1–5	–	.864	.004
Naglierib	106.60 (16.08) 73–15	103–109	104.70 (12.15) 77–128	102–107	.841	.004

^*p < .05,

^**p < .01,

^***p < .001.

^a5-point scale where 1 = some high school, 2 = high school degree, 3 = some college, 4 = college degree and 5 = some graduate school/advanced degree.

^bStandard score (M =100, SD = 15).

Measures

Nonverbal measure

Standard scores from the Naglieri Nonverbal Abilities Test (NNAT: Naglieri, 2003) were used to measure participants’ nonverbal abilities. The 72-item NNAT assess individual differences in pattern completion, reasoning by analogy and serial reasoning.

Socioemotional behavioural measure

The Child Behaviour Checklist (CBCL: Achenbach & Rescorla, 2001) was completed by parents to examine potential symptom overlap between children experiencing socioemotional problems consistent with the S(P)CD diagnosis versus those associated with other DSM diagnoses. The CBCL consists of 113 items representing clinical symptoms and provides T scores (M = 50, SD = 10). Higher scores on the CBCL are indicative of higher levels of parental concern (T scores 60 and 70 represent 1.0 and 2.0 SDs above the mean). In accordance with the instrument’s scoring procedures, separate norms were used for boys and girls. The CBCL includes six DSM syndrome scales (i.e. Affective Problems, Anxiety Problems, Somatic Problems, ADHD, Oppositional Defiant Problems and Conduct Problems). The CBCL is a widely used measure and has demonstrated strong psychometric properties (Nakamura, Ebesutani, Bernstein & Chorpita, 2009).

Language measures

Clinical Evaluation of Language Fundamentals - Fourth edition (CELF-4)

The CELF-4 (Semel, Wiig, & Secord, 2003) is an omnibus test of language ability that was used to measure general language abilities. CELF-4 subtests necessary for obtaining the Core Language Score (CLS), which yields a standard score (M = 100, SD = 15), were used. The CELF-4 has adequate levels of sensitivity and specificity needed to identify language impairment (Se = 87%, Sp = 96%: Spaulding, Plante, & Farinella, 2006, p. 69).

Nonword repetition (NWR)

Nonword repetition tasks have been used extensively as a measure of children’s phonological working memory (see Estes, Evans, & Else-Quest 2007 for a recent meta-analysis), representing a well-established area of weakness for children with language impairment. We administered the Dollaghan and Campbell NWR (1998) which contains 16 nonwords that gradually increase in length from one to four syllables. Each phoneme was counted as correct or incorrect, yielding a percentage of total correct phonemes.

Rice Wexler Test of Early Grammatical Impairment (TEGI)

The TEGI (Rice & Wexler, 2001) is a norm-referenced measure of children’s finiteness production used here to measure finiteness. Limitations in the development of finiteness have been one of the core deficits associated with language impairment in English-speaking children (see Ash & Redmond, 2014 and Leonard, 2014 for reviews), with evidence suggesting that finiteness deficits persist through adulthood (Poll, Betz, & Miller, 2010). The TEGI manual reports adequate levels of sensitivity (81%) and specificity (95%) when its cut-offs are used (Spaulding et al., 2006, p. 69).

Pragmatic measure

Children’s Communication Checklist – Second edition (CCC-2)

The CCC-2 (Bishop, 2006) was used to measure children’s S(P)CD symptoms. The CCC-2 represents one of the most prominent checklists in clinical and research applications (Norbury, 2014). The CCC-2 standardisation sample included 950 children between the ages of 4;0–16;11 from across the US.

Per above, the SIDI is calculated by subtracting the sum of the pragmatic/social-interaction scales from the sum of the structural language scales. Because various pragmatic composites have been used by researchers to capture S(P)CD symptoms, in addition to the SIDI measure, a Pragmatic Composite-Five (PC-5) was calculated based upon the pragmatic measure reported in the original CCC (Bishop, 1998). In order to ensure the similarity of the items across the two versions, we matched the scales in our PC-5 to the original pragmatic composite. The following scales were summed to produce the PC-5 pragmatic composite score: Coherence, Initiation, Scripted Language, Context and Nonverbal Communication. Support for use of these five scales to identify S(P)CD is provided in Table 4 which displays individual CCC-2 items that align with S(P)CD symptoms. As noted in Table 4, all seven items within each of the PC-5 scales can be applied to one of the four primary S(P)CD symptoms.

Table 4.

Alignment of CCC-2 scales and items with the DSM-5 S(P)CD diagnostic criteria.

CCC-2 Scale	A-1: Deficits in using communication for social purposes in a manner that is appropriate for the context	A-2: Impairment in ability to change communication to match context or listener needs	A-3: Difficulties following rules for conservation and story-telling including knowing how to use verbal and nonverbal signals to regulate interaction	A-4: Difficulties understanding what is not explicitly stated such as nonliteral and figurative language
Speech (0 items)
Syntax (0 items)
Semantics (0 items
Coherence (7 items)		10. Uses terms like he or it without making it clear what he or she is talking about 25. It is difficult to tell if he or she is talking about something real or make believe 48. does not explain what he/she is talking about to someone who does not share his/her experiences 50. It is hard to make sense of what he/she is saying, even though words are clearly spoken 53. Talks clearly about what he/she plans to do in the future	40. Confuses the sequence of events when trying to tell a story or describe a recent event 68. Explains a past event clearly
Initiation (7 items)	21. Talks to people without any encouragement or starts conversations with strangers	5. Talks repetitively about things that no one is interested in 37. Tells people things they already know 59. Keeps quiet in situations when someone else is trying to talk or concentrate 70. Talks to others about their interests	35. It is difficult to stop him/her from talking 45. Asks a question, even though he/she has been given the answer
Scripted Language (7 items)		18. Uses favourite phrases, sentences or longer sequences in inappropriate contexts 23. Pronounces words in an over-precise manner 42. Provides over-precise information in his/her talk 61. When answering a question, provides enough information without being over-precise	11. Says things he/she does not seem to fully understand or seems to be repeating something he/she heard an adult say 30. Repeats what others have just said 62. You can have an enjoyable, interesting conversation with him/her
Context (7 items)		28. Ability to communicate varies from situation to situation 60. Realises the need to be polite	34. Takes in just one to two words in a sentence, and so misinterprets what has been said	15. Misses the point of jokes and puns 19. Gets confused when a word is used differently from its usual meaning 41. Is too literal 54. Appreciates the humour expressed by irony
Nonverbal Communication (7 items)			8. Looks blank in situations where most children would have clear facial expression 14. Does not look at person he/she is talking to 20. Stands too close to other people when talking to them 31. Ignores conversational overtures from others 39. Does not recognise when other people are upset or angry 56. Makes good use of gestures to get his/her meaning across 65. Smiles appropriately when talking to other people
Social Relations (3 items)		57. Shows concern when other people are upset 67. Talks about his/her friends; shows interest in what they do and say	33. Hurts or upsets other children without meaning to
Interests (4 items)	22. Talks about list of things he or she has memorized	26. Moves the conversation to a favourite topic, even if others do not seem interested in it 49. Surprises people by his/her knowledge of unusual words 63. Shows flexibility in adapting to unexpected situations

Analytic approach

Data were analysed using a two-step approach. Initially, a series of one-way ANOVAs were conducted to investigate potential sex differences in participants’ performance on the socioemotional behavioural, language and pragmatic measures. Alpha levels less than 0.05 were considered significant.

To address our primary research question, exploratory factor analyses (EFA) were conducted to determine whether or not the associations among measures of nonverbal abilities, pragmatic skills, language and behaviour could be explained by a more parsimonious underlying structure. Exploratory factor analyses were used because we had no theoretical or empirical assumptions about the underlying factor structure (Fabrigar, Wegener, MacCallum, & Strahan, 1999). These analyses included NWR, CELF-4 CLS, and the TEGI EGC, six CBCL syndrome subscales (Affective, Anxiety, Somatic, ADHD, Oppositional Defiant and Conduct) and either the SIDI or PC-5. All factor analyses were conducted separately for boys and girls. The subject-to-factor ratio for all analyses exceeded the minimum (5:1) suggested by Gorsuch (1983).

All measures correlated at least 0.3 with at least one other measure, suggesting reasonable factorability. Bartlett’s test of Sphericity was significant for the SIDI and PC-5 analyses for boys and girls (p < .001), indicating that the variables were correlated at a level sufficient for factor analysis (Gorsuch, 1997). Additionally, the Kaiser-Meyer-Olkin Measure of Sampling Adequacy ranged from 0.76 to 0.83 for the four analyses, indicating that the minimum amount of data for the factor analysis was satisfied (Kaiser, 1974).

Results

Complete data were available for all of the participants (Table 5 and Table 6). One-way ANOVAs indicated that there were no significant differences between sexes on any of the psycholinguistic and socioemotional behavioural measures. As indicated by the ranges presented in Table 5, both groups contained cases within the clinical range. Table 6 provides scores on the CCC-2 across both groups. One-way ANOVAs demonstrated no significant difference between boys and girls on the Speech, Syntax, Semantics, Coherence, Scripted Language, Context and Social Relations scales. Boys, however, had significantly lower performance than girls on the Nonverbal Communication [F(207) = 9.15, p = .003] and Interests [F(207) = 8.71, p = .004] scales. The performance of boys on the PC-5 composite was also significantly lower than girls [F(207) = 4.54, p = .022]. The observed differences between boys and girls on the CCC-2 PC-5 and two of the pragmatic scales motivated the remaining analyses in this study to be conducted separately for each sex.

Table 5.

Performance of boys and girls on psycholinguistic and socioemotional behavioural measures: Group means (M), standard deviations (SD) and range.

	Boys (n = 125) M (SD) Range	95% CI	Girls (n = 83) M (SD) Range	95% CI	F(1, 207)	η2p
NWRa	84.52 (11.62) 32–100	82.46–86.58	84.51 (10.81) 30–98	82.15–86.87	.000	.000
CELF-4 CLSb	94.10 (19.73) 40–132	90.70–97.49	95.69 (19.31) 40–123	91.47–99.90	.341	.002
TEGI EGCc	90.83 (19.12) 10–100	87.45–94.22	91.69 (15.44) 4–100	88.32–95.06	.116	.001
CBCL DSM-IV scales
Affective Problemsd	55.49 (6.49) 50–83	54.34–56.64	54.57 (6.41) 50–78	53.17–55.97	1.02	.005
Anxiety Problemsd	55.53 (6.99) 50–77	54.29–56.76	54.95 (8.04) 50–97	53.20–56.71	.301	.001
Somatic Problemsd	54.69 (7.38) 50–80	53.38–55.99	53.36 (5.69) 50–70	52.12–54.60	1.92	.009
Attention Deficit/Hyperactivityd	55.98 (7.98) 50–77	54.56–57.39	54.94 (7.10) 50–75	53.39–56.49	1.28	.006
Oppositional Defiant Problemsd	56.16 (7.21) 50–77	54.88–57.44	54.72 (6.27) 50–77	53.35–56.09	.917	.004
Conduct Problemsd	54.22 (6.61) 50–78	53.05–55.39	54.28 (6.84) 50–78	52.78–55.77	.003	.000

^*p < .05,

^**p < .01,

^***p < .001.

^aPercentage of phonemes correct,

^bStandard score (M = 100, SD = 15),

^cPercentage of items correct,

^dChild Behaviour Checklist scales (M = 50, SD = 10): higher scores indicate elevated levels of difficulty.

Table 6.

Performance of boys and girls on the Children’s Communication Checklist-2: Group means (M), standard deviations (SD) and range.

	Boys (n = 125) M (SD) Range	95% CI	Girls (n = 83) M (SD) Range	95% CI	F(1, 207)	η2p
Speech	8.28 (3.24) 1–13	7.71–8.85	8.71 (3.01) 1–13	8.05–9.37	.934	.005
Syntax	9.62 (2.94) 1–13	9.10–10.14	9.70 (3.06) 1–13	9.03–10.37	.031	.000
Semantics	9.42 (3.05) 2–15	8.88–9.96	9.69 (3.45) 2–15	8.93–10.44	.334	.002
Coherence	9.35 (2.80) 2–14	8.86–9.85	9.78 (2.71) 3–14	9.19–10.38	1.21	.006
Initiation	9.66 (2.70) 3–16	9.19–10.14	10.36 (2.84) 3–16	9.74–10.98	3.19 (p = .076)	.015
Scripted Language	10.03 (2.45) 2–14	9.60–10.47	10.58 (2.47) 3–14	10.04–11.12	2.47	.012
Context	9.44 (2.58) 3–15	8.98–9.90	10.04 (2.83) 3–15	9.42–10.65	2.47	.012
Nonverbal Communication	9.36 (2.77) 1–14	8.87–9.85	10.48 (2.38) 1–14	9.96–11.00	9.15*	.043
Social Relations	9.98 (2.48) 3–13	9.55–10.42	10.27 (2.72) 3–13	9.67–10.86	.594	.003
Interests	10.14 (2.76) 1–16	9.66–10.63	11.25 (2.49) 5–17	10.71–11.80	8.71*	.041
General Communication Composite	96.86 (15.10) 51–130	94.18–99.53	100.65 (16.87) 59–132	96.97–104.33	2.87 (p = .092)	.014
Social Interaction Difference Index	2.46 (8.92) −23–29	.89–4.04	4.39 (6.73) −12–21	2.92–5.85	2.79 (p = .096)	.013
Pragmatic Composite– Five	47.85 (11.24) 15–73	45.86–49.84	51.24 (11.26) 21–71	48.78–53.70	4.54*	.022

^*p < .05,

^**p < .01,

^***p < .001.

Factor analysis

Principal axis factoring (PAF) was used to analyse the variables because the assumption of multivariate normality was severely violated due to the inclusion of non-normally distributed clinical symptoms (Osborne, Costello, & Kellow, 2008). Direct oblimin, an oblique rotation, was chosen for all PAFs because it produces considerably fewer cross-loadings as a result of minimising squared loading covariance between factors (Fabrigar et al., 1999). Delta and kappa values were not altered from the SPSS default settings, which were 0 and 4, respectively. Factors were identified through visual inspection of the screen plots. Loadings between −.30 and .30 were not included in the factor.

Factor analysis: SIDI¹

In the sample of 125 boys, a factor solution was identified accounting for 55.35% of the total variance and consisting of two factors, which we labelled Socioemotional and Behaviour Competence and Linguistic Proficiencies. The Socioemotional and Behaviour Competence factor accounted for 30.68% of the variance, while the Linguistic Proficiencies factor accounted for 24.67% of the variance. Table 7 displays the items, factor loadings and factor correlations. The first factor included all of the scales from the CBCL. This factor seemed to reflect general socioemotional and behavioural competence. The Oppositional Defiant and Affective syndrome scales from the CBCL carried the highest loadings on the first factor. The second factor included the NWR, the CELF-4 CLS, the TEGI EGC and the CCC-2 SIDI. Factor two appeared to index linguistic proficiencies and also included the pragmatic measure from the CCC-2 SIDI. This factor consisted of high loadings for CELF, TEGI and NWR and a negative loading for the SIDI. This was expected because negative SIDI scores are indicative of higher structural versus pragmatic language performance.

Table 7.

Factor loadings for boys using the CCC-2 SIDI.

Item	Factor loading		Communalities
	1	2
	3.48	2.81
NWR		.896	.826
TEGI EGC		.821	.673
CELF-4 CLS		.816	.677
CCC-2 SIDI		−.606	.371
CBCL Affective	.834		.705
CBCL Anxiety	.515		.266
CBCL Somatic	.422		.179
CBCL ADHD	.776		.601
CBCL Oppositional Defiant	.866		.752
CBCL Conduct	.696		.484
% of variance	30.68	24.67
Factor correlations
Factor 1	–
Factor 2	−.049	–

In the sample of 83 girls, a factor solution was identified that accounted for 61.74% of the total variance and consisting of three factors, which we labelled Socioemotional and Behavioural Competence (35.07%), Linguistic Proficiencies (21.15%) and Internalising Behaviours (5.52%). Table 8 displays the items, factor loadings and factor correlations. The first factor included most of the scales from the CBCL. This factor seemed to reflect general socioemotional and behavioural competence. The Oppositional Defiant and Conduct syndrome scales from the CBCL carried the highest loadings on the first factor. The second factor consisted of high loadings for the NWR, the CELF-4 CLS, the TEGI EGC and a negative loading for CCC-2 SIDI. Factor two appeared to index linguistic proficiencies including the pragmatic measure from the CCC-2 SIDI. The third factor included the CBCL Affective, Anxiety and Somatic syndrome scales. The third factor included the scales from the CBCL that capture internalising behaviour difficulties.

Table 8.

Factor loadings for girls using the CCC-2 SIDI.

Item	Factor loading			Communalities
	1	2	3
	3.82	2.46	1.11
NWR		.862		.756
TEGI EGC		.790		.621
CELF-4 CLS		.818		.802
CCC-2 SIDI		−.533		.346
CBCL Affective	.583		.348	.601
CBCL Anxiety	.366		.513	.519
CBCL Somatic			.543	.308
CBCL ADHD	.789			.687
CBCL Oppositional Defiant	.897			.784
CBCL Conduct	.860			.748
% of variance	35.07	21.15	5.52
Factor correlations
Factor 1	–
Factor 2	−.114	–
Factor 3	.315	−.078	–

Factor analysis: PC-5²

The PC-5 was substituted for the CCC-2 SIDI, and the analyses were conducted first in the sample of 125 boys. Factor analysis suggested a factor solution that accounted for 61.73% of the variance. Table 9 displays the three identified factors that we labelled as follows: Socioemotional and Behavioural Competence (35.72%), Linguistic Proficiencies (21.23%) and Internalising Behaviours (4.78%). The highest factor loadings on factor one included the CBCL Oppositional Defiant and Conduct syndrome scales, with a negative loading of the PC-5. Factor two included the three linguistic measures, NWR, CELF-4 CLS and the TEGI EGC. Factor three included the three scales from the CBCL that capture internalising behaviours, the Affective, Anxiety and Somatic syndrome scales.

Table 9.

Factor loadings for boys using the CCC-2 PC-5.

Item	Factor loading			Communalities
	1	2	3
	3.93	2.39	1.02
NWR		.907		.834
TEGI EGC		.804		.636
CELF-4 CLS		.843		.741
CCC-2 PC-5	−.699			.831
CBCL Affective	.316		.682	.338
CBCL Anxiety			.503	.310
CBCL Somatic			.595	.647
CBCL ADHD	.731			.782
CBCL Oppositional Defiant	.826			.540
CBCL Conduct	.741			.514
% of variance	35.72	21.23	4.78
Factor correlations
Factor 1	–
Factor 2	−.147	–
Factor 3	.592	−.045	–

The final analysis included 83 girls. A two factor solution explained 58.27% of the variance, as shown in Table 10. Factor one included all of the subtests from the CBCL and the CCC-2 PC-5 and was labelled Socioemotional and Behavioural Competence (39.43%). Factor two included the NWR, CELF-4 CLS, TEGI EGC and a cross-loading of the CCC-2 PC-5 and was labelled Linguistic Proficiencies (18.84%). The cross-loading of the CCC-2 PC-5 was −.581 on factor one and −.337 on factor two.

Table 10.

Factor loadings for females using the CCC-2 PC-5.

Item	Factor loading		Communalities
	1	2
	4.32	2.21
NWR		.863	.731
TEGI EGC		.792	.613
CELF-4 CLS		.848	.829
CCC-2 PC-5	−.581	−.337	.518
CBCL Affective	.752		.572
CBCL Anxiety	.635		.399
CBCL Somatic	.322		.104
CBCL ADHD	.789		.688
CBCL Oppositional Defiant	.823		.690
CBCL Conduct	.832		.684
% of variance	39.43	18.84
Factor correlations
Factor 1	–
Factor 2	−.170	–

Given the difference between the results presented here and those of Tomblin et al. (2014), we conducted a post hoc analysis using a Principal Components Analysis (PCA) approach instead of the PAF factor approach. Boys and girls were combined in this analysis (n = 208) in order to replicate the methods of Tomblin et al. (2014). The results of the post hoc analysis are presented in Table 11. Briefly, the PCA factoring method did not identify a unique factor for the SIDI, instead revealing three factors, of which the SIDI cross-loaded on two of the factors including one with several of the CBCL scales.

Table 11.

Principal component analysis (PCA) using CCC-2 SIDI including boys and girls (n = 208).

Item	Factor loading			Communalities
	1	2	3
	4.09	2.70	1.00
NWR	−.469	.753		.834
TEGI EGC	−.361	.778		.636
CELF-4 CLS	−.542	.689		.741
CCC-2 GCC	−.762			.608
CCC-2 SIDI		−.675	.310	.831
CBCL Affective	.783			.338
CBCL Anxiety	.597		.366	.310
CBCL Somatic	.397		.738	.782
CBCL ADHD	.773	.309		.514
CBCL Oppositional Defiant	.462	.412		.540
CBCL Conduct	.716	.340		.647
% of variance	37.19	24.56	9.11

In summary, the findings from the four factor analyses revealed different factor structures for boys and girls based upon the inclusion of the CCC-2 SIDI or the CCC-2 PC-5. Notably, the CCC-2 SIDI scores clustered with other language measures while the CCC-2 PC-5 scores grouped with socioemotional behavioural difficulties captured by the CBCL DSM syndrome scales. The PC-5 scores for girls cross-loaded on the behavioural difficulties and linguistic competencies factors. The presence of different solutions for boys and girls suggests that pragmatics, like phonology and semantics, but unlike grammar or verbal memory appears to represent a linguistic domain which is moderated by sex. In this regard, it is also similar to ADHD, Anxiety, and other socioemotional behavioural disorders.

Discussion

The inclusion of S(P)CD in the DSM-5 framework occurred before empirically validated measures were available to identify the disorder. Accordingly, clinicians and researchers have been left without robust protocols for assessing S(P)CD. The CCC-2 is one of the most frequently used parent report measures that has the ability to identify clinically significant pragmatic difficulties. Parental measures such as the CCC-2 hold promise for accurately identifying children who have S(P)CD because they require relatively little time for parents to report on behaviours that might otherwise be difficult to observe in the context of an assessment (Norbury, 2014). Therefore, it is valuable to examine the potential consequences of using different aspects of this instrument.

Social and pragmatic difficulties as overlapping constructs

Our results suggest that it will be challenging for clinicians and researchers to differentiate pragmatic difficulties from other language and socioemotional behavioural problems. This finding was somewhat unexpected given that previous work by Tomblin et al. (2014) found that the SIDI constituted its own factor when grouped with measures of vocabulary, omnibus language and language sample measures in a community sample of eighth grade students. One explanation for the different results was that the present study used a PAF analysis and Tomblin and colleagues’ study used a Principal Components Analysis (PCA). The PCA approach does not discriminate between shared and unique variance, and as a result, may produce inflated values of variance accounted for by the components (Osborne et al., 2008). Additional differences between our study and Tomblin et al.’s study include the age of the participants and the measures. The different outcomes likely reflect a complex interaction among children’s age and social-emotional behaviours and warrant further investigation.

Previous research has attempted to bypass the complications of the SIDI by creating other pragmatic composites out of the CCC-2 scales. Our use of the PC-5 was an attempt to offer an alternative to the SIDI that better aligned with the S(P)CD definition offered in the DSM-5 (see Table 4). None of the factor analyses conducted in this study revealed a unique factor structure for either the SIDI or the PC-5. An additional objective of this study was to determine whether the underlying trait structure for S(P)CD symptoms varied dependent upon the SIDI or PC-5. Based upon the evidence provided in the factor analyses, the SIDI and the PC-5 appeared to represent different aspects of pragmatic ability. This finding aligns with other reports, suggesting that the SIDI may not adequately characterise variation in children’s pragmatic abilities (Adams et al., 2012; Geurts & Embrechts, 2010; Helland, 2014; Leonard et al., 2011; Timler, 2014). It may be that the combined use of the SIDI and PC-5 scores could provide a broader view of children’s pragmatic performance. However, one of the key features of the PC-5 is that it more closely aligns with the inclusionary criteria for S(P)CD than the SIDI, which implies that it would be the preferred composite for screening children for pragmatic difficulties with the caveat that these difficulties may be associated with other social-emotional and behaviour disorders.

Some investigators have suggested that children who could currently be characterised as having S(P)CD might have it as a conjunct with other structural language deficits or represent a subgroup of the ASD continuum (Boucher, 1998; Brook & Bowler, 1992). Clinicians and researchers need measures that are able to partition those pragmatic difficulties caused by structural language problems or behavioural/emotional disorders because appropriate intervention bears directly upon successfully addressing the underlying cause of these difficulties. For example, pragmatic difficulties resulting from disorders such as ADHD may require pharmaceutical interventions in situations where behavioural interventions alone are unable to address children’s social difficulties. Alternatively, children with pragmatic difficulties who do not meet DSM criteria for ADHD may need an intervention focused directly on language use skills delivered by a speech-language pathologist. Clearly, moving forward differential diagnosis will need to be a crucial aspect of implementing the S(P)CD diagnosis.

Sex differences

Our results indicated that performance on the CCC-2 varied by sex, such that girls had significantly higher performance on the Nonverbal Communication and Interests scales, and on the Pragmatic Composite-Five. This finding was inconsistent with Collins et al., (2014); Law et al., (2014); Leonard et al. (2011) but replicated Ketelaars et al. (2010) and Geurts et al. (2009) who similarly reported higher performance for girls across various CCC-2 scales and pragmatic composites.

Our factor analyses suggested different underlying factor structures for boys and girls on both the SIDI and the PC-5. To summarise, boys and girls demonstrated differing factor structures on the SIDI and the PC-5 based upon internalising behaviours as measured by the CBCL. Specifically, the structure of the SIDI analysis resulted in two factors for boys: Socioemotional and Behavioural Competence and Linguistic Proficiencies, and three factors for girls: Socioemotional and Behavioural Competence, Linguistic Proficiencies, and Internalising Behaviours. For both groups, the SIDI loaded on the Linguistic Proficiencies factor, supporting the notion that this measure of pragmatics shares an underlying relationship with other language areas such as phonological memory and finiteness production. This was not surprising given that the SIDI is the difference score between structural language and pragmatic language on the CCC-2. The differing factor structures demonstrated in the SIDI analysis, that is, two factors for boys and three for girls, supports previous research indicating differing socioemotional profiles based on sex (Alegria et al., 2001–2003, Eaton et al., 2012, Hartung & Widiger, 1998). However, our finding of differences between boys and girls on the Internalising Behaviours factor was unexpected because the CBCL had already incorporated sex-based norms which should have prevented an internalising factor from being identified. The factor analysis for boys using the PC-5 revealed a three factor solution; Socioemotional and Behavioural Competence, Internalising Behaviours and Linguistic Proficiencies. The factor analysis for girls using the PC-5 resulted in a two factor solution: Socioemotional and Behavioural Competence and Linguistic Proficiencies. The PC-5 loaded on the Socioemotional and Behavioural Competence factor for boys, and cross-loaded on the Socioemotional and Behavioural Competence and Linguistic Proficiencies for girls. The observed overlap of pragmatic difficulties and socioemotional and behavioural problems will likely require a multi-disciplinary approach to differentially diagnose S(P)CD from psychiatric conditions. Although these interpretations are based on factor analysis, which can produce unstable estimates, the results here are consistent with other studies in the area.

Currently, similar questions about underappreciated sex effects in Autism Spectrum Disorder have been raised, and subtle differences in sleep patterns, repetitive behaviour, externalising behaviour and social problems have been reported (Hartley & Sikora, 2009; Holtmann, Bölte, & Poustka, 2007). Research in language impairment and autism appears to have converged on the existence of subtle sex variations associated with these phenotypes. The results presented here motivate further study of sex differences in the symptomatology of S(P) CD as well.

Limitations and future directions

Future research is needed to examine the complexities that accompany the nascent S(P)CD label. One limitation of this study was that the CCC-2 was the only measure used to examine children’s social and pragmatic difficulties. Generally speaking, multiple assessment methods should be used to make an S(P)CD diagnosis (e.g. conversational analysis, standardised measures and observation of peer interactions). Nonetheless, we focused on key pragmatic weaknesses that would likely be included in symptoms of the disorder. Due to the various CCC-2 pragmatic composites that have been used by researchers and our findings that the underlying properties of these symptoms change depending on which composite is used, additional research is needed to determine which pragmatic metrics best capture S(P)CD – with the possible consequence that S(P)CD will be regarded in the future as primarily a subtype of language impairment or as a subtype of socioemotional disorder.

Exploratory factor analysis is a procedure that is designed to explore new constructs and generate new hypotheses. Findings from study to study can be unstable, particularly with small sample sizes. Accordingly, results from this study should be considered provisional and are meant to add to the larger discussion surrounding S(P)CD and the emergence of best practice standards. Our results, along with those from previous reports, indicate that the impact of sex on S(P)CD presentations should continue to be an integral part of this line of inquiry.

Conclusions

We examined the extent to which a commonly used parent rating scale could be adapted to identify S(P)CD symptoms in accordance with the DSM-5 scheme. We demonstrated that in order to move forward as a coherent, separate clinical entity additional diagnostic refinements of the S(P)CD construct are needed. One key refinement will be accommodating for sex differences in the expression of S(P)CD. Speech-language pathologists possess expertise in characterising and addressing a range of pragmatic deficits as they appear in a variety of clinical populations. Whether the new S(P)CD label captures a segment of the population that requires a unique set of diagnostic and intervention procedures or if the needs of these individuals can adequately be addressed using existing protocols remains an open question awaiting convincing evidence.