Outcome Measures for Clinical Trials in Fragile X Syndrome

Abstract

Objective

Progress in basic neuroscience has led to identification of molecular targets for treatment in fragile X syndrome (FXS) and other neurodevelopmental disorders, however, there is a gap in translation to targeted therapies in humans. One major obstacle to the demonstration of efficacy in human trials has been the lack of generally accepted endpoints to assess improvement in function in individuals with FXS. To address this problem, the NIH convened a meeting of leading scientists and clinicians with the goal of identifying and standardizing outcome measures for use as potential endpoints in clinical trials in FXS.

Methods

Participants in the meeting included FXS experts, experts in the design and implementation of clinical trials and measure development, and representatives from advocacy groups, industry, and federal agencies.

Results

The group generated recommendations for optimal outcome measures in cognitive, behavioral, and biomarker/medical domains, including additional testing and validation of existing measures, and development of new measures in areas of need. Although no one endpoint or set of endpoints could be identified that met all criteria as an optimal measure, recommendations are presented in this report.

Conclusion

The report is expected to guide the selection of measures in clinical trials and lead to the use of a more consistent battery of measures across trials. Further, this will help to direct research toward gaps in the development of validated FXS-specific outcome measures, and to assist with interpretation of clinical trial data by creating templates for measurement of treatment efficacy.

Introduction

An important goal of research in the field of intellectual and developmental disabilities is to discover methods for ameliorating the impact of these disorders on health and quality of life of affected individuals and their caregivers. One such disorder is fragile X syndrome (FXS). FXS is an X-chromosome linked single-gene disorder and the most common known, heritable cause of cognitive and behavioral dysfunction in humans, with prevalence rates as high as 1 in 2,500 (1,2). FXS is caused by a trinucleotide repeat expansion mutation in the regulatory region of the Fragile X Mental Retardation 1 gene (FMR1) which results in hypermethylation and transcriptional silencing of the gene, and the resulting loss or reduction of expression of the gene product, Fragile X Mental Retardation Protein (FMRP). Insufficient expression of FMRP, an mRNA-binding translational regulator, has a broad array of effects on cellular signaling pathways and on synaptic plasticity, morphology and function, thereby leading to abnormalities in brain connectivity and neurodevelopmental processes (3–6). In addition to intellectual disability (ID), individuals with FXS often demonstrate clinical features that overlap with other frequently occurring psychiatric and developmental disorders including autism, ADHD, anxiety disorders, and language disorders (7).

A number of symptom-based pharmacological treatments for individuals with FXS have been employed, including stimulants for attention deficits, selective serotonin reuptake inhibitors (SSRIs) for anxiety and atypical neuroleptics for aggression (8,9). Unfortunately, there is limited available data regarding the efficacy of symptom-based pharmacological or behavioral treatments in FXS. Clinical trials of sufficient size and power to detect definitive treatment effects have not been conducted. Studies have primarily consisted of open-label pilot trials and small pilot placebo-controlled double-blind trials of symptom-based or mechanistically-targeted pharmacotherapy (Table 1). While these trials have provided valuable information about the potential efficacy of approved and investigational drugs in FXS, they have also demonstrated multiple shortcomings of currently available outcome measures. Further, clinical endpoints used to assess efficacy have differed across medication trials in FXS, making cross-study comparisons difficult (Table 1). Moreover, the adequacy of many of those clinical measures remains to be determined.

Table 1.

Outcome Measures Used in Prior Published Trials in FXS

Trial (First author, Year published (reference))	Design1	N2	Measures Used3			Measures Reliable4	Measures Change- Sensitive5
			Behavioral	Cognitive	Biomarker
High-dose folate, (Hagerman, 1986 (112))	DB, PC, XO	25	CARS, AutBC	PPVT, TOLD, IQ		ND	None
Methylphenidate, (Hagerman 1988 (113))	DB, PC, XO	15	Conner Scales			ND	Conner
Folinic acid (Strom 1992 (114))	DB, PC, XO	21	CTRS, CPRS ACTeRS	VABS, PPVT		ND	None
Methylphenidate, Amphetamine salts, (Hagerman 2002 (115))	OL	19			EDR	ND	EDR
CX516, (Berry-Kravis 2006 (116))	DB, PC	48	ABC, SNAPIV, VAS, CGI, CARS, GARS, ADOS	RBANS, CELF, PLS-4, PPVT, TVPS, W- JMem, IVA, VABS		PPVT, PLS-4, RBANS (LR, LL, DS, SM subtests), W-JMem, SNAPIV, ADOS Comm, CARS, ABC Subscales, VAS Behavior	None
N-acetyl-carnitine (Torrioli 2008 (117))	DB, PC, XO	63	CGI-T, CGI-P	VABS, WISC		ND	CGI-T, CGI-T, VABS
Lithium, (Berry-Kravis 2008 (80))	OL	15	ABC, VAS, CGI	RBANS, PPVT, CFXCPT, NEPSY Tower, NVALT, Card Task, VABS	ERK, Eye Tracking, Auditory Filtering, Autonomic Testing – RSA, HR, HRV	CFXCPT (in concurrent study)	ABC, VAS, CGI, RBANS (LL), VABS-L, ERK
Fenobam, (Berry- Kravis 2009 (96))	OL	12		CFXCPT	PPI	PPI (concurrent study)	PPI
Donepezil, (Fung 2012, Kesler 2009, (73,111))	OL		ABC, CBCL	CNT, HVLT	MRS
Memantine, (Erickson 2009 (118))	OL	6	ABC, CGI, ADHDRS-IV, SRS			ND	None significant
Valproic acid, (Torrioli 2010 (119))	OL	10	CTRS, CPRS, CGI, SNAPIV, K-SADS-PL	VABS	FMR1 re- activation	ND	CPRS-R
Minocycline, (Paribello 2010 (120))	OL	20	ABC, CGI, VAS	RBANS, PPVT, NVALT		ND	ABC, CGI, VAS, RBANS (LL, SM)
Aripiprazole, (Erickson 2010 (121))	OL	15	ABC, CGI, CY-BOCS			ND	ABC, CY-BOCS, CGI
Riluzole, (Erickson 2011 (90))	OL	6	CY-BOCS, CGI ADHDRS-IV, ABC, SRS	PPVT	ERK	ND	ADHDRS-IV, ERK
AFQ056, (Jacquemont 2011 (76))	DB, PC, XO	30	ABC, RBS, CGI, VAS, SRS	KiTAP, RBANS, PPVT	ERK, Eye Tracking, PPI, FMR1 methylation, FMR1 mRNA	ND	CGI, ABC, RBS, VAS (in fully methylated group)
Oxytocin, (Hall 2012 (122))	DB, PC	8			Eye tracking, cortisol, Autonomic Testing – RSA, HR, HRV	ND	Eye tracking, cortisol
Arbaclofen, (Berry- Kravis 2012 (72))	DB, PC, XO	62	ABC, RBS, VAS, CGI, SRS, CASI, ADHDRS-IV	RBANS, PPVT, VABS, SB5- WM	PPI, APP		ABC-SW, VABS- P/L, CGI, VAS
Acamprosate, (Erickson 2012 (123))	OL	12	ABC, CGI, SRS, CASI, ADHDRS-IV	PPVT, VABS	APP		APP, ABC, CGI, SRS, ADHDRS-IV

¹DB=Double-blind, PC=Placebo-controlled, XO=Crossover, OL=Open label

²Number of subjects in trial

³Adaptive measures are included in the Cognitive section. Abbreviations for measures as follows: ABC=Aberrant Behavior Checklist-Community Edition (SW=Social Withdrawal Subscale), ACTeRS=ADD-H Comprehensive Teacher’s Rating Scales, ADHDRS-IV=ADHD Rating Scale IV, ADOS=Autism Diagnostic Observation Scale, APP=Amyloid precursor protein, AutBC=Autism Behavior Checklist, Card Task=Card Task Test of Visual Sequential Memory, CARS=Childhood Autism Rating Scale, CASI=Childhood Anxiety Sensitivity Index, CBCL=Child Behavior Checklist, CELF=Clinical Evaluation of Language Skills, CFXCPT=Carolina Project Fragile X Continuous Performance Test, CGI=Clinician Global Impression, CGI-P=Conner Global Index for Parents, CGI-T=Conner Global Index for Teachers, CNT+=Contingency Naming Task, CPRS=Conner Parent Rating Scale, CTRS=Conner Teacher Rating Scale, CY-BOCS =Children’s Yale-Brown Obsessive Compulsive Scale modified for pervasive developmental disorders, EDR=Electrodermal responses, ERK=ERK-related kinase activation rate, GARS=Gillam Autism Rating Scale, HR=Heart Rate, HRV=Heart Rate Variability, HVLT=Hopkins Verbal Learning Test, IVA=Integrated Visual-Auditory CPT, KiTAP=Tests of Attentional Performance for Children, K-SADS PL=Kiddie Schedule for Affective Disorder and Schizophrenia for School Age Children - Present and Lifetime Version, NVALT=Non-Verbal Associative Learning Task, PLS-4=Preschool Language Scale – Version 4, PPI=Prepulse Inhibition, PPVT=Peabody Picture Vocabulary Test, RBANS=Repeatable Battery for the Assessment of Neurological Status (LL=List Learning, SM=Story Memory, DS=Digit Span, LR=List Recall Subtests), RBS=Repetitive Behavior Scale, RSA=Respiratory Sinus Arrythmia, SB5-WM=Stanford-Binet Version 5 Working Memory Index, SNAPIV=Swanson, Nolan, and Pelham Questionaire for ADHD, SRS=Social Responsiveness Scale, TOLD=Test of Language Development, TVPS=Test of Visual Perceptual Skills, VABS=Vineland Adaptive Behavior Scale (P/L=Play and Leisure Subscale), VAS=Visual Analog Scale, W-JMem=Woodcock-Johnson Memory for Words

⁴Reliability not assessed in all trials, data presented includes only those analyzed that had ICC or weighted Kappa>0.8 on test retest

⁵Measures change-sensitive in trial are presented, it must be recognized that lack of change-sensitivity in the absence of a drug effect does not rule out change sensitivity for the measure, change sensitivity may be exaggerated or reported as present when measuring only placebo effect in open-label studies, if just a subtest was change-sensitive-subtest name given in parethesis

New knowledge derived from studies of translational models of FXS, as well as genetic, imaging, and neuropsychological investigations of people with FXS has opened the door to the development of disease-specific pharmacological treatments (10), with several drugs currently being tested in Phase 2 and 3 clinical trials (Table 1). Prompted by the extraordinary research progress, and in anticipation of ongoing acceleration of the clinical trial effort in humans with FXS, the National Institutes of Health convened a meeting of leading scientists and clinicians with the goal of identifying outcome measures for use as potential endpoints in clinical trials in FXS. Participants in the meeting included FXS experts, along with experts in the design and implementation of clinical trials and measure development, and representatives from advocacy groups, the pharmaceutical industry, and various federal agencies. The overarching objective of this meeting was to facilitate discussion that would lead to the identification and/or development of a set of outcome measures that could be used to document efficacy in current and future clinical trials.

The work of the participants in the meeting was guided by three principles. First, although it is unlikely that a single battery of measures would be appropriate for all clinical trials, identification of a core set of widely applicable measures would facilitate comparability across different agents, research centers, and methodological approaches. Second, outcome measures that have been developed for symptom-based clinical trials in behaviorally defined disorders (e.g., autism, ADHD) might not be sufficiently sensitive or specific for disease-oriented interventions in FXS and thus, measures must be validated specifically for FXS. Third, the results from the meeting should generate a set of criteria for identifying appropriate outcome measures that could help inform the design of clinical trials, improve interpretation of clinical trial data by regulatory agencies such as the FDA, and help guide governmental and private sector funding entities on this topic.

Methods and Procedures

Working groups were created to identify outcome measures in three areas relevant to the broad phenotype of FXS: (1) Cognition; (2) Behavior/Emotion; and (3) Medical/Physical including Biomarkers (Figure 1). Each working group included individuals with clinical, research, and measurement expertise in developmental disorders. Each of the three working groups participated in a series of meetings, during which the process described in Figure 1 was carried out and all relevant published literature was evaluated. Following several months of virtual meetings, a 2-day face-to-face meeting was convened in Washington, D.C. The meeting included FDA input on the process, presentation of working group progress, intensive work within working groups to refine their list of optimal endpoints, and a final open discussion.

Figure 1.

Process for developing recommendations for the optimal outcome measures - based on FDA Guidance for Industry Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims

Results

Summaries of data analyses and discussion from each working group are presented in the following sections. It should be noted that although much of the activities of the working groups focused on parsing cognition, behavior and emotion, and biological systems into distinct subdomains, there was also the recognition of the possibility of developing a single composite measure for each domain – a composite that aggregated the core features of FXS within that domain. In this way, investigators might seek approval for a therapeutic designed to treat a cluster of related features associated with FXS rather than just specific symptoms. This single-composite approach is likely to have some practical challenges, such as how to identify features that will be included in the measure, as well as variation in symptom presentation across age and level of impairment. Nonetheless, such an approach is appealing in light of neurotherapeutics in development, which target the underlying neurobiology of FXS. A potential drawback of a composite measure is that it may fail to identify true clinical improvement in a single functional domain, as may occur when an intervention has a narrow, yet clinically significant effect, on only one of the aggregated subdomains. The single-composite approach was endorsed most enthusiastically by the Behavior and Emotion working group.

Cognitive Measures

Although use of a single measure of cognitive function is appealing, the working group was skeptical that this approach would be robust for clinical trials in FXS given the rather modest changes in cognition expected from various classes of drugs being tested. Focused assessments of core cognitive features seem better suited for clinical trials at this time. Given this starting point, the working group was guided by the following:

1. The therapeutics currently being developed are designed to impact neural systems (e.g., the mGluR5 pathway) that are related to numerous subdomains of cognition as well as aspects of emotion and behavior. For cognition, clinical effects are likely to be observed in both FXS-specific features of cognition (e.g., high rates of perseverative language) and in features that are shared by other neurodevelopmental disorders (e.g., limited short-term memory).

2. Improvements in cognition can be measured in a variety of ways. On the one hand, improvements can be reflected in the “products” of learning, for example an increase in the number of words that an individual can produce. On the other hand, improvements can be reflected in the “processes” underlying learning, as in the case of an an accelerated learning rate (11). Process measures may be more valuable for evaluating clinically meaningful effects because they reflect improvements in the capacity to learn and adapt, which should have significant effects on daily functioning and quality of life. Process measures may also be more sensitive to drug effects than “product” measures, which are essentially the result of a learning process and therefore “accumulate” to measurable levels gradually.

3. The scope and nature of changes in cognition vary with age. Thus, different measures will be needed for participants of different ages.

4. Although it is useful to distinguish different areas of functioning, such as cognition and behavior, these domains are interrelated and measurement strategies should be designed accordingly. The working group found value in partitioning cognition into the subdomains of language, memory and learning, executive functioning, social cognition, and academic achievement; again, recognizing that these areas are inter-related. A list of promising outcome measures for the cognitive domain can be found in Table 2.

Table 2.

Promising Outcome Measures for the Cognitive Domain

Domain	Outcome Measure	Pros	Cons
Language	Standardized Language Sampling Procedures (e.g., narration)	Useful for wide age/ability range	Labor-intensive transcription; psychometrics not fully known for FXS
Memory	WJ Auditory Working Memory WJ Digits Reversed Corsi Blocks CANTAB Object Memory RBANS List Learning	Collectively, broad coverage of domain; good psychometric properties in general population; evidence of reliability and validity in FXS	Not all subtests appropriate for lower-functioning people
Executive Function	WJ Planning Contingency naming KiTAP (4 subtests) WJ Rapid naming	Collectively, broad coverage of domain; good psychometrics in general population; evidence of reliability and validity in FXS	Not all subtests feasible for lower- functioning people
Social Cognition	Social Responsiveness Scale	Broad coverage of social behaviors; easy to administer; strong psychometrics in other populations Useful for wide age/ability range	Unknown validity for FXS
	Eye Tracking (to measure emotion processing)	Useful for wide age/ability range	Unknown validity for FXS; technology not widely available
Academic Achievement	Woodcock-Johnson Tests of Achievement-III	Wide age range; good reliability; alternate test forms	Unknown validity for FXS
	Wechsler Individual Achievement Test-II	Wide age range; validity studies with clinical populations	Unknown validity for FXS

Language

The linguistic phenotype of FXS is characterized by language delays relative to chronological age expectations (12–17). Atypical linguistic behaviors are also common, including high rates of verbal perseveration (18,19), and bursts of rapid, poorly articulated speech (20). The phenotype varies with the occurrence of co-morbid conditions, with lower IQ and more severe autism symptoms associated with more serious language problems (16,21–25).

The working group considered a number of measures, including standardized tests, caregiver reports, and laboratory-based tasks. Significant limitations of most measures were noted, including a lack of sensitivity to clinically meaningful change and limited or unknown reliability and validity (26). In the end, the working group concluded that expressive language sampling procedures held the most promise for immediate use in clinical trials (see Table 2). These procedures are common in research settings and involve recording samples of spontaneously produced language in a structured social interaction with an examiner (26). Objective variables can be created from transcripts of these samples to reflect numerous aspects of the linguistic phenotype. These procedures are applicable (with minor modifications) over a wide developmental range (26).

There is considerable evidence that measures derived from language samples collected under standardized conditions are related to chronological age in typically developing children and mental age in many language-disordered populations. These measuresdistinguish language-typical from language-disordered populations (27) and evidence is accumulating that these measures can distinguish individuals with FXS from those with other neurodevelopmental disorders, such as Down syndrome (13,22,28–31). Guidelines for standardization for a number of language sampling procedures (e.g., conversation, narration) exist (32), and these procedures can be mastered with rather minimal training (26).

Memory and Learning

The working group distinguished between several types of memory based on the duration and type of information stored, as well as the neural systems involved (33–39). In terms of explicit memory, which refers to those aspects of memory that involve effortful attempts to store and retrieve information, individuals with FXS are more impaired in remembering sequentially presented information than simultaneously presented information (40). Individuals with FXS also perform below expectations on measures of working memory (41), which involves rehearsal and allocation of mental resources (42,43). Individuals with FXS also are especially impaired at working memory tasks requiring the manipulation of auditory (as opposed to visual) information (44–46) and when memorization requires considerable attention and strategic planning (47,48). The implicit memory system, which involves the storage of information without conscious effort, is largely unexplored in FXS, although anecdotal experience suggests that implicit memory may not be as impaired as other types of memory in FXS.

The working group identified several memory tasks that mapped onto the FXS memory phenotype (see Table 2), for example, the Auditory Working Memory and Digits Reversed subtests of the Woodcock-Johnson III Tests of Cognitive Abilities (49,50), although the latter subtest may be beyond the ability of individuals with a developmental level less than four years. The Corsi Blocks (51) test has been used successfully to study visual-spatial sequential memory in FXS. The working group also proposed assessing long-term memory with the list learning and story memory subtests of the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; 52, 53).

Executive Functioning

Executive functioning involves setting goals, developing plans, and implementing goals (54). The functioning of this system is typically indexed by tasks that require behaviors such as inhibition of prepotent responses, monitoring the success of a problem-solving strategy, the ability to shift between responses or strategies, and the efficiency of applying cognitive processes (55). There is compelling evidence that deficits in executive functioning are highly characteristic of individuals with FXS (56,57), with performance often below mental age expectations (58). These impairments in executive function map on to abnormalities in brain function, particularly in frontal regions in FXS (59,60).

The working group identified several measures of executive function that appear well-suited for use in some clinical trials targeting FXS (see Table 2), however, some of these measures may be beyond the ability of lower-functioning individuals. The working group also suggested the use of informant report measures (e.g., the Behavior Rating Inventory of Executive Function; BRIEF; 61), which may prove useful when direct assessment is not possible.

Social Cognition

Social cognition refers to the processing of information about the social world (e.g., the ability to recognize differences in people’s knowledge or perspectives). These skills are impaired relative to chronological age expectations in FXS (62–64). Impairments in some aspects of social cognition exceed the impairments seen in other domains, especially in individuals with co-morbid FXS and autism (23).

The working group noted that measures of social cognition have not been well-studied for FXS. More generally, it was noted that the relationship between performance on such measures and actual social behavior is not strong. The working group concluded that the Social Responsiveness Scale (65), which is a caregiver report, and the use of eye-tracking technology to assess attention to social events, were promising but required further validation in FXS.

Academic Achievement

The working group recommended the use of academic achievement measures as indicators of long-term outcomes and real-life functioning, particularly in clinical trials of longer duration. The expectation is that changes in language, memory, and executive function are likely to lead to functional changes in the academic domain. However, no specific measure was found to be adequate at this time.

Summary

Although promising measures were identified for several key domains, each requires further evaluation before being endorsed for clinical trials.

• Additional data are need on test-retest reliability, sensitivity, and validity.

• Validation over an expanded age range is needed.

• Feasibility and validity data are needed across the full range of affectedness.

Behavior and Emotion Measures

The working group on Behavior and Emotion considered the following points in their evaluation of measures that could be appropriate for use in a clinical trial: 1. The measure should adequately cover the type and severity of problems that most interfere with daily functioning and quality of life for individuals with FXS and their families. In this regard, the FDA clearly emphasizes that a patient reported outcome (PRO) measure, such as a rating scale, must include symptoms that affected individuals or their proxies (caregivers), clinicians, and investigators report to be characteristic of the specific condition to be treated (66). 2. The measure should have documented reliability and validity for the age range and level of functioning of individuals likely to be recruited into clinical trials, and preferably the psychometric properties should be established for the FXS population rather than for intellectual disability more generally. 3. The measure should have a proven capacity to detect improvement in the specified domain (i.e., sensitivity), preferably in the context of a controlled trial. The working group identified several maladaptive behaviors that are over-represented in FXS relative to other intellectual disabilities: 1) inattention; 2) hyperactivity/impulsivity; 3) irritability/aggression; 4) self-injury; 5) anxiety; 6) repetitive/compulsive behavior; 7) sleep problems; and 8) social avoidance/reciprocity. Adaptive behavior (the ability to complete the activities necessary for successful navigation of independent daily life) was also identified as a potential target. A list of promising measures of maladaptive behaviors can found in Table 3.

Table 3.

Promising Measures of Maladaptive Behavior

Domain	Potential Outcome Measure	Pros	Cons
Irritability/aggression	ABC-Irritability	Known sensitivity in ID/AUT. FXS psychometric data	Sensitivity in FXS unknown. Limited to lower functioning patients
Anxiety	PARS	Good phenotype coverage; sensitive in pediatric anxiety trials, existing FXS psychometric data Validated for ID; limited FXS psychometric data	Limited data in ID and FXS sensitive unknown.
	ADAMS		Unclear validity for higher functioning patients
Hyperactivity/impulsivity	Connors/SNAP-IV	Sensitive in ADHD trials	Validity in FXS unknown
Inattention	Connors/SNAP-IV	Sensitive in ADHD trials	Validity in FXS unknown
Social reciprocity/avoidance	SRS	Good coverage of domains; linked to broad autism phenotype	Validity in FXS unknown
	ADAMS (social avoidance) ABC – Lethargy/withdrawal	FXS reliability and validity data FXS reliability and validity data	Sensitivity unknown Lethargy confounded with social items (however, see 81)
Repetitive/compulsive behaviors	Repetitive Behavior Scale-Revised	Good coverage of domains; validated in autism/ID	Limited FXS data.
Adaptive behavior	Vineland	Extensive FXS data; sensitive in double blind FXS trials	A by parental expectations and environment; wide period of observation
Self-injury	Behavior Problems Inventory	Published FXS data; severity and frequency ratings	Sensitivity to treatment unknown

The working group considered informant-based rating scales, such as those completed by a parent, caregiver, or clinician, because of their ease of use, cost, and utility in multi-site trials. Some of these measures have a track record documenting behavioral improvement in individuals with intellectual disability or autism spectrum disorders, such as the Aberrant Behavior Checklist (67–69). The working group also acknowledged several limitations and concerns about informant reports, including a reliance on respondents who may be unintentionally biased such as parents of affected children. In addition, informant-based rating scales are quite variable with some rating severity of symptoms and others rating frequency. In addition, well-defined “behavioral anchors” to help guide respondents are often absent. This can be problematic for caregivers, many of whom have minimal exposure to other children with FXS and thus, may have little experience with the full range of symptom severity. The working group concluded, however, that the sensitivity of these rating scales to treatment effects is likely to be dependent on test-retest reliability within-respondent and within-subject. Thus, such rating scales remain a viable option, especially given the paucity of other validated measures.

A challenge for any rating scale is providing coverage of the wide range of symptom severity and expression across individuals with FXS. An additional challenge arises in covering the variability in symptom presentation across developmental and mental ages.

The working group identified several behavioral rating scales that meet important criteria, among these is the Aberrant Behavior Checklist (67), which has good psychometric properties when used with individuals with intellectual disability and has been used successfully to document improvements in irritability/aggression in many controlled trials of autism (70–73). Berry-Kravis et al (72) and Fung et al. (73) reported on the successful use of the ABC total score to document behavioral improvement in an open label trial of lithium in 15 children and adults with FXS and donepezil in 12 children and adults with FXS, respectively.

At the time of the working group meeting, the ABC had not been extensively validated for FXS and thus the working group endorsed further study on this measure, especially in light of the fact that several ongoing industry-sponsored clinical trials have utilized the ABC as a primary endpoint for FXS. This recommendation led to a successful five-site collaborative study of the psychometric properties of the ABC in over 600 individuals with FXS (74) and utilization in clinical trials (75,76). The working group also identified both the SNAP-IV (77) and the Connors Rating Scales (78), each designed to measure hyperactivity/impulsivity, as having utility in clinical trials for FXS, however it was noted that only limited data are available for those with intellectual disabilities. Although anxiety is an important problem in FXS, the working group concluded that few suitable measures are currently available. The Anxiety, Depression and Mood Screen (ADAMS) (79) was identified as a useful tool for screening anxiety and mood symptoms in individuals with intellectual disabilities, and has since shown evidence of validity for FXS (80); however, its sensitivity to treatment effects is unknown. The working group also saw promise in the Pediatric Anxiety Rating Scale (PARS) (81), which was used successfully to document improvement in pediatric anxiety disorders with SSRIs. A small reliability and validity study of the PARS for FXS has been completed recently and is in press (82).

Repetitive behaviors, such as perseverative speech and stereotypies, are often problematic in FXS. The Children’s Yale Brown Obsessive Compulsive Scale, modified for pervasive developmental disorders (CYBOCS-PDD) (83), has excellent psychometric properties when used with children with an autism spectrum disorder and has documented sensitivity to psychopharmacological intervention (84), although its reliability, validity and sensitivity in FXS is unknown. The Repetitive Behavior Scale-Revised (RBS-R) (85,86) was designed for use in populations with an autism spectrum disorder and related developmental disorders and was recently used in a study of young boys with FXS or idiopathic autism (87), with different profiles of repetitive behaviors being observed in the two groups.

Although the main focus in clinical trials in FXS has been on maladaptive behaviors shown to improve with short-term treatment in other disorders, it is recognized that adaptive behavior may also be responsive to intervention, especially in studies providing treatment over extended periods. Adaptive behavior scales are expected to be especially important in assessing treatment impact in very young children with FXS, as fewer measures of cognitive function exist. It is also possible that older children and adults may demonstrate improvements in adaptive behaviors (e.g., improved communication, greater independence with daily living skills, or improved socialization) that are secondary to reductions in maladaptive behaviors or enhanced cognition. The relative value of different adaptive behavior measures has not been explored extensively in FXS. However, improvement in adaptive behavior using specific subscales of the Vineland (88) has been demonstrated in the small open label trial of targeted treatment with lithium (Expressive Language subdomain) (72), as well as the recent placebo-controlled trial of arbaclofen (Play and Leisure subdomain) (75), suggesting that the measure is likely to be sensitive.

In light of the limitations of informant-report rating scales, the working group also concluded that there is a pressing need for the development of more objective, direct observation or assessment of individuals with FXS by trained clinicians or experimenters. Such measures could include behavioral sampling, in which individuals with FXS are observed directly by trained and reliable experimenters across several contexts. The advantages of direct behavioral observation should be balanced, however, by the time commitment and potential cost associated with such an approach for a large multi-site trial.

The working group also discussed the feasibility and advantages of developing a new behavioral scale specifically designed for individuals with FXS. It was suggested that a measure could be developed by combining items selected from other instruments or developed anew by a diverse panel of parents, clinicians, and investigators, and empirically developed using established methods of factor analytic techniques, as well as reliability and validity studies. Such an approach could yield a tool measuring behaviors closer to the “core” of the FXS phenotype and a scale or scales that would be more sensitive to interventions designed to target the neurobiology of the disorder.

Summary

The working group endorses efforts:

• To establish the reliability, validity and sensitivity (including sensitivity to change) of several currently available measures within the FXS population

• To establish the specificity of the measured constructs to FXS by examination of correlations between such measures and valid FXS biomarkers (e.g., brain imaging) and gene-dose (e.g. FMRP)

• To consider development of a new behavior rating scale for FXS to cover the phenotype and the full range of associated symptoms

• To supplement traditional psychometric studies with data from focus groups that include patients or their proxies, and other caregivers to provide input on the construct validity, interpretability, and feasibility of measures of interest

Biomarkers and Medical Measures

Biomarkers as endpoints for treatment response are attractive because of their potential to be objective, repeatable, and quantifiable measures of the biology of the disorder or a physical manifestation of the underlying biology. Such biomarkers might assess the impact of a treatment on mechanisms underlying FXS as opposed to evaluating behavioral symptoms alone. In practice, however, it has been difficult to ascertain whether and how changes in available biomarkers correlate with meaningful improvements in function or behavior (89). Thus, the working group concluded that our understanding of biomarkers in FXS is not sufficiently developed to be useful for product registration with the FDA at this time. The need for clinically meaningful, treatment-sensitive biomarkers in FXS, however, cannot be over-stated.

Blood and Tissue Biomarkers

Research on peripheral tissue biomarkers in FXS was noted by the working group to be at a very early stage. FMR1 methylation status, FMR1 mRNA levels and FMRP levels have been proposed as obvious markers of the disorder but may or may not predict response to disease-targeted pharmacological interventions, as they do not represent the target of current therapies. In a small pilot trial of mGluR5 blocker AFQ056, response was substantially better in the sub-group of FXS males with complete gene methylation and undectable FMR1 mRNA levels (76)(see Table 1). These results need to be replicated in a larger trial, but suggest that FMR1 methylation and FMRP (not measured in the AFQ056 study) should continue to be studied as biomarkers that may help predict treatment response in future FXS trials.

Although predictors of treatment response are critical, a major focus of this working group was on development of measures that reflect the impact of a drug on FMRP-regulated pathways, including activity of 1) proteins in the translational activation pathway and 2) proteins whose translation is regulated by FMRP (some proteins may fit in both of these categories). ERK activation rate, a translational pathway biomarker, abnormal in FXS mice and humans, is responsive (normalizes) to lithium (72) and riluzole (90) treatment in individuals with FXS (see Table 1). Amyloid precursor protein (APP), a potential biomarker whose translation is regulated by FMRP (91), is a target measure in several FXS trials; for example, APP was shown to normalize during treatment with acamprosate (Table 1). Other measures, such as phosphoinositide 3-kinase (92), mTOR and S6 kinase activity (93), are promising, but need further development and validation for meaningful implementation.

In the development of peripheral biomarkers, it is important to recognize that pathways and receptors are not always connected in the same fashion in different tissues. For biomarker discovery, careful consideration must be given to: 1) tissue differences in the interaction of the biomarker with pathways affected by FMRP and the drug under study; 2) individualization of biomarkers for agents with different targets; 3) peripheral biomarkers index tissues outside the CNS and may be very difficult to correlate with a CNS-mediated behavioral or functional improvement. Consequently, biomarkers in FXS may be, at best, an indication that the drug is hitting a desired cellular target, but may not fully predict treatment response. In addition, a reasonable goal could be to identify surrogate biomarkers that reliably track with the disorder or treatment regimen, but may not be central to the biological pathways in FXS.

Electrophysiological Measures

Prepulse inhibition (PPI) of the startle response, the most extensively studied electrophysiological measure, is reduced in individuals with FXS and has good test-retest reproducibility (94). Moreover, PPI is attractive because (a) mGluR5 pathways are involved in both PPI regulation and FXS, (b) there are prominent sensory inhibitory (gating) deficits in FXS, and (c) PPI appears deficient in both the Fmr1 knockout mouse and in humans with FXS (95,96) (see Table 1). There are problems, however, with establishing functional behavioral correlates for PPI, and achieving inter-site reliability. Consequently, PPI may be most useful in early proof-of-concept trials as opposed to large multi-site trials.

Autonomic measures including heart rate, R-R interval, and beat-to-beat variability were not found to be responsive to lithium treatment but have not been utilized in other treatment studies (97). Other psychophysiological measures are under investigation as potential biomarkers for treatment response in individuals with FXS.

Eye Tracking and Pupillometry

Eye gaze aversion and pupillometry measured with newer eye tracking systems represent potential biomarkers of anxiety and excess sympathetic outflow.. Both eye gaze and pupillometry measures are abnormal in FXS (98) and have good test-retest reproducibility (99). This measure is being used in clinical trials currently (Table 1) but data are not yet available on sensitivity to treatment-induced change.

Although the behavioral correlates of eye gaze aversion and social anxiety would seem obvious, it has been difficult to show correlations of pupillometry and eye tracking response profiles with validated instruments for rating such behaviors and thus, additional research is needed before eye tracking could be considered a possible endpoint supporting registration of pharmaceuticals for treatment of FXS. Moreover, the cost of eye tracking systems may limit use of these measures to smaller proof-of-concept trials.

Neuroimaging Studies

Thus far, early phase clinical trials in FXS have utilized behavior checklists or cognitive tests developed for general use in non-FXS populations, as clinical trial endpoints. Because neurobiological mechanisms associated with FXS interact with environmental influences to affect brain development and function from birth, the use of relatively non-specific behavioral or cognitive endpoints increases the likelihood of a false negative (Type II) error, even in the presence of a biologically meaningful effect.

Because direct measures of brain structure, function and neurochemistry might be more closely and directly related to fundamental neurobiological etiology arising from reduced FMRP, the possibility of employing in vivo neuroimaging biomarkers as disease biomarkers or treatment endpoints in FXS has attracted considerable interest (100). A growing number of brain imaging studies indicate that distinct and, in some cases, unique neuro-phenotypic features exist in individuals with FXS relative to age and sex matched neuro-typical controls, as well as IQ matched idiopathic (i.e., non-FXS) developmentally disabled or autistic controls (101–108). These features include a significantly (≥40%) larger caudate nucleus (a prominent finding detectable as early as one year of age), aberrant white matter connectivity or activation of prefrontal-striatal pathways, and significant anatomical and/or functional activation differences in the insular, fusiform and superior temporal cortices, amygdala, hippocampus and cerebellar vermis.

The few longitudinal neuroimaging studies conducted in children and adolescents to date indicate that the trajectory of brain development in FXS diverges from typical neurodevelopment at distinct time periods during childhood and adolescence as well as in young adults (109,110). These critical windows of brain maturation may correspond to neurodevelopmental epochs when FMRP is particularly important for achieving optimal synaptic maturation and connectivity in the brain and could in the future, be used to evaluate treatment outcome.

An initial magnetic resonance spectroscopy (MRS) study in a small sample of individuals with FXS showed reduced choline levels in a region of the prefrontal cortex (111). A subsequent study replicated this finding in the caudate nucleus of individuals with FXS relative to IQ-matched controls and also found reduced levels of GLX, an MRS measure comprised of glutamine and glutamate (submitted). Because MRS techniques can detect and measure several neurometabolites implicated in the neurobiology of FXS, this neuroimaging modality holds particular promise for identifying potential disease biomarkers and clinical trial endpoints.

In summary, because of the seemingly greater proximity to fundamental mechanisms underlying FXS, neuroimaging biomarkers may hold promise as potential endpoints in clinical trials. Notwithstanding the cost and training involved, additional research is needed to demonstrate the feasibility and utility of using neuroimaging biomarkers in research and clinical settings for this purpose.

Side Effect Monitoring, Side effects may be difficult to detect in cognitively impaired individuals who may not be able to report symptoms accurately. Formal comprehensive questionnaires have been used such as the SMURF (72); however, these are time consuming and some studies have used lists for screening of an inventory of specific targeted side effects (100) depending on the particular concerns associated with the drug being tested. Considerable care should be exercised in this arena to monitor adverse events and protect study subjects.

Summary

• Research on biomarkers for detecting intervention-induced improvement in fundamental neurobiological deficits of FXS is in its infancy.

• Continued research is greatly needed at multiple levels: cellular, physiological, neuroimaging and (core) behavioral.

• Measures under development have yet to be directly linked to the underlying neurobiology of FXS.

• The future availability of robust biomarkers for elucidating the neurobiology of, and treatment-response in humans with FXS is of utmost importance.

Discussion

The overarching objective of these NIH-initiated meetings was to facilitate discussion that would ultimately lead to treatment outcome measures that can be utilized in current and future clinical trials. Particularly motivating to the participants was the knowledge that a core set of relevant clinical endpoints would significantly improve our ability to interpret and compare results from clinical trials taking place at different sites or using different pharmacologic agents and/or behavioral and cognitive interventions. This may also increase the likelihood that future Phase 3 trials would result in more rapid FDA approval of drugs for indications related to FXS with labeling of potential therapeutic agents for clinical use. The process may also serve as a model for other neurodevelopmental disorders.

The working group suggests that no single measure currently exists that meets all criteria for an ideal clinical endpoint that can be used to evaluate treatment for FXS. Although the ABC is proving quite useful in current clinical trials, it is unlikely to meet the long-terms needs of the field given the limitations noted in the Results section. The working groups endorsed the need to devise a set of measures that reliably captures the core cognitive impairments and underlying neurobiological mechanisms of FXS, as well as the impairments that define key behavioral and emotional domains. Consequently, we conclude that:

1. Scientists, funding agencies, and private industry should collaborate to support a broad-based program of research in FXS to arrive at a set of core measures tapping the domains of cognition, behavior and emotion, and neurobiology.

2. This research should strive to identify and develop measures that (a) are psychometrically sound with high reliability, validity, and sensitivity to change for the FXS population; (b) index core features of the FXS phenotype, at the neurobiological level or the level of cognition, emotion, and behavior; (c) focus on constructs that translate to improved quality of life for individuals with FXS; (d) are feasible across a wide range of ages and levels of functional impairment or have well-documented conditions of use with respect to age and degree of impairment; (e) are relatively insensitive to placebo effects and (f) are logistically feasible for use in large multisite trials.

3. Several measures appear particularly promising and thus, should be a high priority for further development. Arguably, the single measure that appears closest to reaching the threshold of the criteria identified is the Aberrant Behavior Checklist, which has a long history of use in research, including clinical trials and has now been revised to better accommodate the FXS phenotype. Importantly, the ABC is being used in clinical trials targeting FXS and is proving useful to detecting treatment-induced change. In the cognitive domain, expressive language sampling holds promise and has been refined in ways that make it feasible for use in multisite clinical trials with individuals affected by FXS. In the domain of biomarkers, PPI and neuroimaging appear promising at least for proof-of-concept trials and perhaps more broadly.

4. Rather than begin anew, there are measures that have been (or are being) developed for other clinical populations and many of the constructs being measured overlap the core symptoms of FXS. These measures should be evaluated for their feasibility and face validity for FXS. This approach of “borrow, adapt, and evaluate” should prove useful for all domains, including that of biomarkers

5. The development of outcome measures would benefit from greater collaboration between scientists working on different models systems. There is currently a disconnect between efforts to measure the phenotypes of FXS in humans and those in mice and other non-human model systems. An attempt to create parallel measures that index similar constructs across different model systems is likely to assist in measure development and accelerate translation of therapeutics from preclinical work to human clinical trials, and has been successful in other disorders.

6. Although FDA approval of a drug is likely to require selection of a single measure in advance of the trial, it is important to acknowledge that our understanding of the causal pathways from genetic mutation to individual phenotype in FXS is incomplete. Thus, a useful strategy would be to select a small set of core measures, with the goal being to advance understanding of FXS behavioral domains and biomarkers and the targeted neurobiological systems and pathways.. In this way, we will document efficacy of the therapeutic agent and simultaneously provide new insights into the pathophysiology of FXS. Ultimately, however, we must arrive at a primary measure for any given trial, although this measure is likely to be different for different therapeutic agents or for short- versus long-term trials, with the choice being informed by knowledge about the causal pathways of the disorder and the mechanisms of action attributed to the therapeutic agent.

In conclusion, this is an exciting era in the field of FXS. The interest in potential therapeutics has increased dramatically and exciting partnerships are being forged between academia, government agencies, and private industry with the goal of moving us closer to a cure. Moreover, several promising agents are already available for testing, with some already have documented positive effects. As a result, optimism among families affected by FXS is high. Further progress will be limited, however, unless the field can quickly arrive at a set of outcome measures to be used in clinical trials. If we do not, we run the risk of abandoning potentially effective treatments and adding to the stress and sense of loss of affected families. The stakes could not be higher.

Table 4.

Promising Biomarkers

Domain	Potential Outcome Measure	Pros	Cons
FMR1/FMRP expression measures in blood	FMR1 methylation	Predicted response to AFQ056 in small study, stable, easy to store/ship	True predictive effect unknown, variability between assays
	FMRP levels	Key factor in severity of molecular deficit being targeted	No easy and accurate assay, no association with outcome of treatment yet, not reflect downstream events from FMRP deficit
Signaling Pathway Measures proposed as biomarkers in blood	ERK activation rate	Abnormal in FXS blood, responsive to lithium tx	Not responsive to tx in some trials, not stable, assay difficult, requires high blood volume
	APP and metabolite levels	Abnormal in FXS blood, protein regulated by FMRP, stable in serum. easy to collect, store and process samples,	Treatment response not yet shown
	PI3K activity		Treatment response not yet shown
	mTOR activity	One study showed abnormal in FXS	Treatment response not yet shown
	S6 Kinase activity	One study showed abnormal in FXS	Treatment response not yet shown
	MMP activity	Levels abnormal, normalized in minocycline trial	Variable results between laboratories for levels in individuals with FXS
Biophysical Measures	Prepulse Inihibtion (PPI)	Good reproducibility, treatment responsive in fenobam trial	Lack of demonstration of clinical validity, expensive and difficult to standardize across sites, younger patients don’t tolerate it
	Eye Tracking and Pupillometry	Easy to do, feasible for most patients, good reproducibility	Clinical validity not clear, not yet shown to be treatment responsive, expensive
	Heart Rate Variability	Feasible for most patients	Not treatment responsive in lithium study, inconsistent methods across labs, reproducibility not defined, expensive
Neuroimaging	Structural MRI DTI MRS rsfMRI fMRI PET		Some procedures may require behavioral preparation or sedation in lower functioning affected males Functional MRI is not feasible in many affected males