Recommendations for Social Skills End Points for Clinical Trials in Neurofibromatosis Type 1

Abstract

Objective

To review parent-report social skills measures to identify and recommend consensus outcomes for use in clinical trials of social deficit in children and adolescents (ages 6–18 years) with neurofibromatosis type 1 (NF1).

Methods

Searches were conducted via PubMed and ClinicalTrials.gov to identity social skills outcome measures with English language versions used in clinical trials in the past 5 years with populations with known social skills deficits, including attention-deficit/hyperactivity disorder and autism spectrum disorder (ASD). Measures were rated by the Response Evaluation in Neurofibromatosis and Schwannomatosis (REiNS) Neurocognitive Committee on patient characteristics, use in published studies, domains assessed, availability of standard scores, psychometric properties, and feasibility to determine their appropriateness for use in NF1 clinical trials.

Results

Two measures were ultimately recommended by the committee: the Social Responsiveness Scale–2 (SRS-2) and the Social Skills Improvement System–Rating Scale (SSIS-RS).

Conclusions

Each of the 2 measures assesses different aspects of social functioning. The SSIS-RS is appropriate for studies focused on broader social functioning; the SRS-2 is best for studies targeting problematic social behaviors associated with ASD. Researchers will need to consider the goals of their study when choosing a measure, and specific recommendations for their use are provided.

The neurocognitive effects of neurofibromatosis type 1 (NF1) have been well-delineated and include impairments in attention, behavior regulation, executive function, language, visuospatial abilities, and academic skills.^1,2 Assessments of social functioning in children with NF1 suggest decreased social competence and poor social information processing.^2-4 Attention-deficit/hyperactivity disorder (ADHD) and autism spectrum disorder (ASD), 2 neurodevelopmental disorders associated with poor social outcomes, have high prevalence within the NF1 population, with studies suggesting 30%–50% of children with NF1 meet criteria for ADHD⁵ and up to 25% meet criteria for ASD.^6,7 In populations other than NF1, clinical trials have focused on improving social skills and social outcomes through behavioral intervention, such as social skills training programs,⁸ and pharmacologic treatment, such as oxytocin.⁹ Pharmacologic interventions for social skills have not been trialed in the NF1 population; however, preclinical models using genetically engineered NF1 mice show promise for targeted pharmacologic intervention improving social functioning. For example, inhibition of Pak1 in the amygdala of NF1 mice improves social functioning.¹⁰

With the potential for clinical trials targeting social outcomes, there is a critical need to identify and evaluate potential outcome measures for clinically meaningful targets in the NF1 population. Consistency in end points in clinical trials improves the ability to assess therapeutic efficacy and allows greater generalization of findings. Furthermore, in a population such as NF1, where sample sizes may be relatively small, the use of harmonized outcomes will allow for samples to be combined to yield greater power to detect meaningful change.¹¹ For ADHD and ASD populations, guidelines have been published with recommendations for primary end point measures in clinical trials.^12,13 These guidelines were developed by expert panels and involved extensive review of measures regarding clinical relevance, psychometric properties, and burden on the patient, parent, and clinician. However, given the cognitive and behavioral difficulties associated with NF1, which often occur in the context of unique medical complications/morbidity, it is important to identify measures appropriate for this specific population. This article describes the work of the Response Evaluation in Neurofibromatosis and Schwannomatosis (REiNS) Neurocognitive Committee in reviewing specific measures of social functioning and developing recommendations for their use in clinical trials with the NF1 population with a primary or secondary neurocognitive or behavioral aim.

Social Skills, ADHD, and ASD in Children With NF1

Available data on social functioning in the NF1 population is based on a limited number of studies using direct child assessment and a larger group of studies assessing symptomatology and functioning using parent-report questionnaires. Existing data can be grouped into 2 broad categories: general social functioning and the presence and severity of autism symptomatology. Frequently used broad-based measures of social functioning include the Child Behavior Checklist (CBCL)¹⁴ and the Behavior Assessment System for Children, second edition (BASC-2).¹⁵ These measures assess more general behavioral and emotional functioning, with only a subset of items focused on social functioning. The Social Skills Rating System (SSRS)¹⁶ and its more recent version, the Social Skills Improvement System–Rating Scale (SSIS-RS),¹⁷ focus solely on components of social functioning, and both have been used in many studies. Frequently used questionnaires of autism symptomatology include the Social Communication Questionnaire (SCQ)¹⁸ and Social Responsiveness Scale/Social Responsiveness Scale–2 (SRS/SRS-2).^19,20

In studies using these measures, problems with social functioning are widely reported, with studies documenting that upwards of 40% of children with NF1 have poor social functioning (for a comprehensive review, see Chisholm et al.²¹). Children with NF1 are rated as less liked than their peers, are more socially isolated, and have fewer reciprocated friendships. They also engage in fewer prosocial behaviors and are not viewed as leaders.³ The cognitive drivers of these social problems are unclear, but there is increasing evidence for deficits in social cognition; that is, the cognitive processes related to the perception and understanding of cues in the environment that communicate social and interpersonal information, and the ability to use those cues to interpret the thoughts and behaviors of others, and to modify one's own behavior to fit the social context.²² Deficits in social cognition and perspective-taking have been reported in children with NF1,²³ as have difficulties with face and emotion recognition.^2,24

Common comorbid disorders, including ADHD and ASD, are significant risk factors for social problems in NF1. Measures of social functioning difficulties are significantly correlated with ADHD symptomatology,^25,26 and children with NF1 and comorbid ADHD show poorer social competence, poorer social skills, and more social problems than children with NF1 only and children with NF1 and comorbid learning deficits.^5,25

Emerging data suggest higher rates of ASD symptoms in children with NF1 compared to the general population and there is growing evidence to indicate the typical presentation of ASD symptoms in children with NF1 may be distinct. Children with NF1 demonstrate fewer stereotyped behaviors and greater engagement and interest in social interaction than children with idiopathic ASD.^7,27 In addition, elevations on parent-reported scales of restricted and repetitive behaviors may reflect endorsement of items that are less specific to autism and more common to both ADHD and well-known executive function and motor deficits in children with NF1, such as inflexibility, poor planning and organization, and limited self-starting/initiation.^7,27,28 There also appears to be a relationship between ADHD and ASD symptoms in NF1, with a high co-occurrence of these disorders.^6,27,29 Social problems and ASD symptoms are not driven by overall cognitive ability, as studies have found little or no relationship between IQ and ASD symptoms in other populations.^6,27

Clinical Trials Related to Social Skills

Although no clinical trials focused on social skills have been conducted in the NF1 population, some pharmacologic interventions have included social functioning as a secondary end point. Mautner et al.⁵ investigated the effect of methylphenidate on children with NF1 and ADHD. In addition to showing improvement on measures of attention, parents reported fewer problems on the CBCL Social Scale¹⁴ after treatment with methylphenidate. Another study is currently examining the effect of methylphenidate on cognition and behavior in NF1 and includes the SSIS-RS¹⁷ as a secondary outcome measure (ACTRN12611000765921).

Within the ADHD population, clinical trials have focused on the effect of behavioral intervention (typically social skills training), psychopharmacologic treatment, or a combination of the 2 on social skills. The most frequently used measures are the SSRS/SSIS-RS,^16,17 BASC-2,¹⁵ and CBCL.¹⁴ Studies investigating medication effects have shown improvements on the SSRS and CBCL following treatment with stimulant and nonstimulant medication.^30,31 The SSRS has also been used as an outcome measure in trials that combined behavioral and medication treatment.³¹ These tools have been able to detect changes in social function associated with therapy, demonstrating their utility in clinical trials.

In the autism population, there is a wider range of measures used to assess social functioning in clinical trials. The SRS/SRS-2^19,20 is the most frequently used measure, with 88 completed studies reporting its use on clinicaltrials.gov (retrieved 31 March, 2020). The SRS/SRS-2 effectively measured change in studies assessing the effect of social skills groups,^8,32 as well as pharmacologic interventions.^33,34 The Aberrant Behavior Checklist³⁵ is primarily used in pharmacologic trials.^36,37 Adaptive measures including the Vineland Adaptive Behavior Scale–II³⁸ and the Adaptive Behavior Assessment System, second edition (ABAS-2)¹⁵ are frequently used with trials investigating early intervention and behavioral intervention with infants and young children.³⁹ Use of other measures in clinical trials is less consistent.

The aims of this article are to recommend specific measures of social skills for clinical trial end points in NF1 based on review of current available literature and discuss considerations for their use in future NF1 clinical trials.

Methods

The REiNS Neurocognitive Committee evaluated measures of social skills identified through an extensive review of relevant literature. PubMed and ClinicalTrials.gov were used to identify parent-report measures with an English language version and used within the past 5 years for children 6–18 years of age. All studies were included for review, irrespective of recruitment status. Search terms included “social,” “social skills,” “social function,” or “social behavior” combined with disorder specific terms, including (“ADHD” or “attention deficit hyperactivity disorder”) or (“autism,” “Asperger,” “ASD,” or “autism spectrum disorder”) or (“neurofibromatosis,” “neurofibromatosis type 1,” or “NF1”). Additional searches were conducted with the previous terms and including “clinical trial,” “randomized controlled trial,” “RCT,” or “intervention.” We also examined measures that had been used previously with the NF1 population.

The procedure to evaluate measures has been previously used by the REiNS Neurocognitive Committee and is described at length in Walsh et al.¹¹ Committee members used the Cog-RATE form to review each measure on 6 criteria: patient characteristics, use in published studies, domains assessed, availability of standard scores, psychometric properties, and feasibility for use in clinical trials. Each area was rated on a scale of 0 (lowest rating/unacceptable) to 3 (highest rating). A decision was made about each measure largely based on the patient characteristics, psychometrics, and feasibility category ratings. A summary of the Cog-RATE scoring criteria is presented in table 1.

Table 1.

Cog-RATE Scoring Criteria

Results

The literature review identified 14 published measures used in social skills clinical trials. Based on initial screening review and committee consensus, 7 measures did not undergo full review. The most frequent reasons included concerns about poor or outdated normative data or limitations in age range. Table 2 lists measures that did not receive a complete review and the reason for exclusion. Whereas the SRS¹⁹ and the SSRS¹⁶ both have extensive use in the broader clinical trial literature as well as in descriptive NF1 studies, the updated versions of these measures were reviewed.

Table 2.

Measures Not Receiving Full Committee Review

The committee reviewed and rated 7 measures of social functioning (table 3). The SCQ¹⁸ received a low rating and did not receive further consideration. The SCQ was developed to screen for ASD diagnosis, and the committee believed this measure was too limited in scope to be useful as a social functioning end point in the NF1 population.

Table 3.

Measures Reviewed

The Children's Communication Checklist–2 (CCC-2)⁴⁰ received a high rating because of its strong psychometric properties and previous use in clinical trials. However, the measure has a narrow focus on pragmatic language skills within the social context. Whereas it may be an appropriate measure for a specific trial focused on social language, it was not considered an appropriate measure for trials focusing on broader social behaviors. The Aberrant Behavior Checklist³⁵ also received a relatively high rating, due to its wide use in clinical trials. However, this measure was developed for use primarily to assess behavior problems in individuals with developmental and intellectual disabilities, making it less applicable for the NF1 population.

The Vineland Adaptive Behavior Assessment Scale, second edition (VABS-II) and Vineland Adaptive Behavior Assessment Scale, third edition (VABS-3)^38,41 and the ABAS-2/Adaptive Behavior Assessment System, third edition (ABAS-3)^42,43 also were rated highly. However, both are broader measures of adaptive functioning, with a subset of items assessing social skills. Administering the measures in their entirety would increase participant burden, and it is unclear whether the tools retain their psychometric properties if only the social scales are administered. In addition, the VABS requires extensive training for its administration, making it a less feasible measure to include in a multicenter clinical trial.

The SRS-2²⁰ received the highest rating. The SRS-2 is an updated version of the SRS.¹⁹ It contains the same items as the original SRS, but scores are based on updated norms. The SRS-2 focuses on social deficits associated with ASD. Scales include Social Awareness, Social Cognition, Social Communication, Social Motivation, Restricted Interests, and Repetitive Behavior. It consists of 65 items rated on a 4-point Likert scale and separate forms are available for parent and teacher ratings. A T score is calculated for each scale, as well as for a total score. Descriptive ranges are provided for T scores (<59: within normal limits; 60–65: mild range; 66–75: moderate range; 76+: severe range).²⁰ The SRS-2 has good psychometric properties. Internal consistency ranges from 0.93 to 0.97 depending on the scale and it has adequate test–retest reliability and interrater agreement. The SRS-2 is widely used in clinical trials targeting social outcomes in ASD. In NF1, the SRS has been used in 13 descriptive studies^{2,6,7,27,29,44-51} and the SRS-2 has been used in 5 studies.^{26,27,46,52,53} These studies have found increased rates of ASD symptoms compared to population norms, with large effect sizes.^{6,26,27,29,52,53} Rates of elevations on the SRS-2 have been generally consistent across studies, with 39%–45% of individuals with NF1 scoring in the mild–moderate range and 13%–16% scoring in the severe range.^26,27,29

The SSIS-RS¹⁷ was also rated highly. The SSIS-RS is a revised and modified version of the SSRS,¹⁶ one of the most widely used measures of prosocial skills. A number of substantial modifications from the SSRS were made that include changes in (1) item content (a large number of items added and deleted in the revised version), (2) item rating scale (changed from a 3-point to 4-point scale), and (3) forms (moved from different forms for specific age ranges to the same questions across all ages). A direct comparison between the SSRS and the SSIS-RS⁵⁴ found that the SSIS-RS has an expanded definition of social skills to include communication and engagement and shows stronger internal consistency than the SSRS. Rather than working from a “deficit” model, the SSIS-RS places emphasis on quantifying the ways children engage in socially acceptable or prosocial behavior, such as engaging in cooperative play, expressing empathy, and sharing. It consists of 79 items with forms available for parents, teachers, and self-ratings. The SSIS-RS yields an overall Social Skills standard score as well as categorical scores (below average, average, above average) for subscales of communication, cooperation, assertion, responsibility, empathy, engagement, and self-control. Internal consistency ranges from 0.70 to 0.95, and the measure has adequate test–retest reliability and interrater agreement. The SSIS-RS has been used in several clinical trials of children with ASD and ADHD, which have documented its sensitivity to change.^55,56 Descriptive studies in the NF1 population have primarily used the SSRS, with findings indicating poor social skills compared to normative expectations, typically developing peers, or unaffected siblings.^2,25,48,50 The SSIS-RS has been used in one study with an NF1 population,²⁶ which also found similarly significant deficits relative to population norms, with a medium effect size.

Discussion

The REiNS Neurocognitive Committee's thorough review of available measures to assess social functioning as a potential outcome measure in NF1 clinical trials was revealing. Several measures were outdated or had poor psychometric properties. Others were highly rated but were ultimately deemed less than optimal for use as an outcome measure in NF1 clinical trials because they were considered too narrow in scope to be appropriate for clinical trials targeting broader social skills; required specialized training for administration, making their use in the clinical trial setting burdensome; or gathered more information than necessary.

The committee believed that the differences in the underlying constructs of the measures necessitated recommendation of 2 assessment measures, one focused on broader social functioning and prosocial behaviors (SSIS-RS) and one focused on problematic social behaviors associated with ASD (SRS-2). Both measures are considered useful, depending on the focus of the research, the intervention being evaluated, and the study design. For most studies, it will be most appropriate to choose only one of these measures as a primary end point in order to preserve resources and aid interpretability. However, for some studies, these tools may provide complimentary information and it may be useful to include the alternate measure as a secondary end point. For example, a behavioral intervention may have reducing ASD symptoms as a primary goal and improving prosocial behaviors as a secondary goal. Furthermore, because children with NF1 demonstrate deficits in various aspects of social functioning, using multiple measures can provide information on different aspects of social skills.

It is important to note that these measures are not interchangeable. In choosing a measure, researchers will need to consider several factors. First, each measure has a different underlying theoretical basis. As a result, similar domains are assessed in unique ways. For example, both measures have a “communication” scale. On the SSIS-RS, communication items address developmental skills related to language and pragmatics (“Takes turns in conversations,” “Speaks in an appropriate tone of voice”). In contrast, items on the SRS-2 are framed within the context of characteristics specific to ASD (“Is awkward in turn-taking interactions with peers, for example, doesn't seem to understand the give-and-take of conversations,” “Is able to imitate others' actions”). As such, researchers cannot assume that scales with similar names are necessarily measuring the same construct, and careful consideration should be given to which model and specific items fit best with their study.

The target behavior and the goals for the intervention will also be important when choosing one as an end point in a clinical trial. In general, the SSIS-RS may be best suited for trials where the goal of the intervention is improving positive social behavior; for example, a behavioral intervention to increase social cooperation and engagement. In contrast, the SRS-2 may be a superior end point where the intervention is focused on reducing ASD-type behaviors or more problematic social behaviors; for example, a pharmacologic intervention to reduce behaviors associated with autism. The SRS-2 may also be more appropriate when the intervention is expected to have more generalized developmental benefit, as the SRS-2 may capture more severe deficits than the SSIS-RS.²⁶ Consultation with a psychologist or a member of the REiNS Neurocognitive Committee regarding outcome choices is recommended to carefully consider the options and assist in choosing the most appropriate methodology for a given trial.

The committee discussed the need to emphasize appropriate use and limitations in using the SRS-2 in research. The SRS-2 was not developed to diagnose ASD, but rather to assess the presence and severity of behavioral traits that are common in ASD. As such, scores on the SRS-2 do not reflect rates of ASD in NF1, but rather prevalence of ASD characteristics.

Finally, for measures to be considered appropriate end points for NF1 neurocognitive and behavioral clinical trials, it is important to have population-specific baseline data, as well as documentation of the utility of multiple measures on the same sample. Since the completion of ratings, Payne et al.²⁶ have published such data on the SRS-2 and SSIS-RS in a large pooled sample of children with NF1. Both measures were able to adequately capture social difficulties in this sample across age and sex, suggesting that both are appropriate for use with the NF1 population.

Only parent-reported symptom and observer rating scales were reviewed. Parent-report measures provide information on a child's social skills in a way that is easily accessible and repeatable within the clinical trial setting. Although parent-reported measures are our recommended measure for clinical trials, studies that want a broader perspective of social functioning could also use teacher- or self-report forms to provide additional ancillary information. Furthermore, there are also other ways to assess social functioning, including direct observation to assess a child's social skills in various settings or in response to social situations. Indeed, much of social skills research conducted in children without neurodevelopmental or medical conditions relies exclusively on systematic observation or peer reports. Direct observation and peer-completed measures may be appropriate and feasible for clinical trials conducted in a therapeutic treatment or school setting. However, such measures would be challenging to complete in most clinical trial settings. As such, the committee determined that rating scales primarily completed by parents will be the most feasible way of assessing social functioning within an NF1 clinical trial setting.

Social skills are not a unitary construct, and no one measure will assess all aspects of social functioning. Careful consideration will be needed when deciding which instrument is most appropriate for any given trial. However, as the 2 recommended measures assess a range of social functioning, covering both positive, prosocial behaviors and more problematic social behaviors, it is expected that one of these measures will be useful for most intervention trials in NF1.

Glossary

ABAS-2

Adaptive Behavior Assessment System, second edition

ADHD

attention-deficit/hyperactivity disorder

ASD

autism spectrum disorder

BASC-2

Behavior Assessment System for Children, second edition

CBCL

Child Behavior Checklist

NF1

neurofibromatosis type 1

REiNS

Response Evaluation in Neurofibromatosis and Schwannomatosis

SCQ

Social Communication Questionnaire

SRS

Social Responsiveness Scale

SRS-2

Social Responsiveness Scale–2

SSIS-RS

Social Skills Improvement System–Rating Scale

SSRS

Social Skills Rating System

VABS

Vineland Adaptive Behavior Assessment Scale

Appendix. Authors

Study Funding

The REiNS International Collaboration is supported by the Children's Tumor Foundation. This study is not industry sponsored.

Disclosure

J.M. Payne holds a clinician-scientist fellowship awarded by the Murdoch Children's Research Institute. J.A. Janusz and Dr. Klein-Tasman report no disclosures. P.L. Wolters receives funding from the Neurofibromatosis Therapeutics Acceleration Program. H.L. Thompson, S. Martin, P. de Blank, N. Ullrich, A. del Castillo, M. Hussey, K.K. Hardy, K. Haebich, T. Rosser, M.A. Toledo-Tamula, and K.S. Walsh report no disclosures. Go to Neurology.org/N for full disclosures.