Development of a Novel Observer Reported Outcome Tool as the Primary Efficacy Outcome Measure for a Rare Disease Randomized Controlled Trial

Abstract

We developed an Observer-Reported Outcome (ObsRO) survey instrument to be applied in a multicenter, placebo-controlled, crossover randomized controlled trial of dichloroacetate in children with pyruvate dehydrogenase complex deficiency. The instrument quantifies a subject’s at-home level of functionality, as reported by a parent/caregiver, who were instrumental in providing the clinical descriptors and domains that formed the instrument’s content. Feasibility testing of the ObsRO tool showed it to be easy to use and comprehensive in capturing the major clinical functional limitations of affected children and requires less than five minutes for a parent/caregiver to complete daily.

Introduction

No therapy for any primary mitochondrial disease has been approved by the Food and Drug Administration (FDA). Among the reasons contributing to this stark deficiency are a historical aversion by both practitioners and patients to participate in controlled clinical trials; an over-reliance on unproven nutritional supplements; heterogeneity in the clinical presentation and course of disease, even among cohorts expressing an identical molecular genetic etiology; and lack of validated biochemical biomarkers or clinical assessment tools applicable as primary outcome measures for randomized controlled trials (RCTs) of investigational therapies (1–6). Given these challenges, and the inherent logistical difficulties in conducting multicenter RCTs in rare disorders, it is not surprising that no drug has advanced to filing in the United States of a New Drug Application for treating a mitochondrial disease.

Pyruvate dehydrogenase complex deficiency (PDCD) is due to any of several loss-of-function mutations in the nuclear-encoded genes that give rise to PDC, a mitochondrial matrix enzyme mega-complex that is essential to cellular bioenergetics (Fig. 1). PDC irreversibly decarboxylates pyruvate to acetyl coenzyme A (acetyl CoA) and thus provides a requisite metabolic bridge between cytoplasmic glycolysis and mitochondrial oxidative phosphorylation. A functional PDC is also required for the oxidative removal of lactate. Consequently, PDCD is considered the most common cause of congenital lactic acidosis during the neonatal period. Patients who survive early life typically exhibit progressive neurological and neuromuscular degeneration and early death due to end organ failure and/or to over-whelming metabolic acidosis (3–6).

Fig. 1.

Role of PDC in cellular energy metabolism and site of action of DCA. PDP, pyruvate dehydrogenase; PDK, pyruvate dehydrogenase kinase; TCA, tricarboxylic acid; e-, electrons; DCA, dichloroacetate.

Dichloroacetate (DCA) is an investigational drug having Orphan Product designation for the treatment of congenital lactic acidosis (7, 8). The drug inhibits pyruvate dehydrogenase kinase (PDK) isoforms that inhibit PDC by reversible phosphorylation of the E1 alpha (E1α) subunit of PDC. Repeated DCA administration also decreases turnover of the complex. Oral DCA is rapidly absorbed, readily crosses the blood-brain and lowers blood lactate concentrations within minutes of its administration. The drug is biotransformed by glutathione transferase zeta 1 (GSTZ1) to glyoxylate, which is inactive towards PDK. Haplotype variations in GSTZ1 influence the kinetics and biotransformation of DCA, distinguishing “slow” from “fast” drug metabolizers. Fast metabolizers possess at least one wild type (EGT) GSTZ1 allele, while slowmetabolizers lack this allele (7, 8). This genetic distinction has been exploited in a phase 1 trial of DCA in adults with recurrent malignant brain tumors to mitigate or prevent drug-induced reversible peripheral neuropathy (9), which is the only known dose-limiting toxicity associated with chronic DCA administration (reviewed in 10) and is also incorporated into the PDCD trial (see below). Anecdotal reports and a RCT indicate chronic, oral DCA administration is generally well tolerated and safe in children with primary mitochondrial diseases, including PDCD, and is effective in causing sustained reductions in blood and cerebrospinal lactate (11, 12). Open label studies in PDCD patients also suggest DCA may improve clinical functionality within 24 hours of initiating treatment and, may also increase survival (13, and personal observ.). Therefore, in 2013, we began to plan a multicenter phase 3 trial of DCA (IND028625) in PDCD ( NCT02616484) as the first such RCT of any therapy for this disease. Here we report on the process of developing a unique Observer-Reported Outcome survey of at-home patient functionality. The use of such a tool has been endorsed by the FDA as the primary efficacy outcome measure for the trial.

Development of Survey Tool

Initial discussions with the Division of Gastroenterology and Inborn Errors Products (GIEP), FDA, made clear that surrogate metabolic markers, such as blood lactate, could not be used as primary efficacy outcome measures and that, instead, we should apply an outcome measure that would identify changes in how patients in the proposed treatment arms felt and functioned. This posed an interesting challenge. Although prior clinical data suggested that DCA may mitigate, or at least stabilize, some of the common clinical signs of PDCD and improve functionality, there are no validated tools to evaluate how young children with PDCD feel or function before or after a therapeutic intervention. The Newcastle Pediatric Mitochondrial Disease Scale (NPMDS) is a validated clinician-observer reported outcome tool in which the content was derived from expert clinical opinions rather than from parents/caregivers (14). It has been applied to an open label study of a synthetic coenzyme Q₁₀ analog for treating patients with Leigh syndrome (15), a neurodegenerative complication of many mitochondrial disease patients, including some with PDCD (3–6). The scale is quite time-consuming and, although completed by the clinician, it relies heavily on recall of the parent/caregiver. Comparisons are made between a patient’s development and functional abilities to those expected from a child of that age. Due to the necessity of this baseline knowledge, the NPMDS was deemed usable only by pediatricians, preferably with experience in mitochondrial diseases. The NPMDS is also dependent on up to four-week recalls by the parent/caregiver to evaluate “current function.” This limitation was identified by a National Institute for Neurological Diseases and Stroke-sponsored committee charged with evaluating current biobehavioral tools for the purpose of developing appropriate Common Data Elements for both future mitochondrial disease clinical trials and primary clinical care (16). In addition, GIEP informed us that it does not evaluate drugs for approval based on QOL assessments and discourages tools that rely on patient or observer recall, because of the likelihood of inaccurate reporting. The Karnofsky/Lansky scale (17) is a validated clinician-observer instrument for measuring at-home functionality in children with cancer and other certain other life-threatening diseases. It should be applicable to children with PDCD, although it has never been applied in studies of pediatric primary mitochondrial diseases. The GIEP recommend that the Karnofsky/Lansky scale could be used as a secondary (exploratory) endpoint in our trial. Thus, we and the FDA recognized there was a need for a robust clinical outcome assessment measure able to capture and quantify functional signs in young children with PDCD over time, especially during the course of an interventional trial.

Accordingly, following multiple discussions with GIEP, and cognizant of published federal regulatory guidelines regarding the development and application of such outcome measures ( 18–20), we sought to generate an Observer-Reported Outcome (ObsRO) tool for this trial. Given the youth and cognitive disabilities of most, if not all, children to be entered into our study, we decided that a Patient-Reported Outcome Measure would be impracticable. Furthermore, because the study design (described below) limited the number of patient visits to participating clinical sites, a Clinician-Reported Outcome Measure would, by itself, be insufficient in monitoring progress during the trial.

The evaluation of the ObsRO assessment began with a questionnaire sent in April, 2013, to parents of 25 PDCD children, asking what signs they observed when their child became ill, presumably associated with metabolic decompensation. Patients were asked to record the frequency of each sign, its relative severity on a Likert scale (i.e., the degree of “worry”, from least to most worrisome) and the sign they observed first when their child became sick. The parents were members of an international web-based PDCD support group organized by a mother of a PDCD child. All responses were transmitted to this individual, who de-identified the data prior to forwarding the responses to one of us (PWS) by a process approved by the University of Florida Institutional Review Board.

Twenty-four evaluable responses were received and all raw data were submitted for review to GIEP. From these results, we determined the most prevalent and clinically important signs and grouped them initially under four domains: Muscular; Neurological; GI/Eating Problems and General Health, using the specific descriptive words and phrases provided by the parents. In June, 2013, three of the study investigators met with six mothers of PDCD children to revise the survey further, clarifying some wording and adding a few more signs the parents considered both crucial and generally applicable, yet also sufficiently user-friendly to be applied daily throughout each treatment phase of the future trial.

The revised questionnaire was submitted to GIEP and to the Study Endpoints and Label Development group, now known as the Clinical Outcome Assessments in the Office of New Drugs, Center for Evaluation and Research (CDER), FDA. Subsequently, additional iterations in the survey content and format were made, through written communications and Type C meetings or teleconferences with GIEP. In 2016, a four-month, IRB-approved, feasibility study was conducted in 10 PDCD families recruited from the PDCD advocacy organization, using the version of the ObsRO tool that reflected the most current FDA recommendations. Participant families were provided an electronic tablet device configured by the University of Florida Information Technology Department to allow daily survey completion, using a wireless internet connection for immediate transmission of data entered into the ObsRO database. The Research Electronic Data Capture (REDCap) secure web application was utilized and the University of Florida served as Data Coordinating Center (DCC).

The basic components of the survey instrument are summarized in Table 1. Parents/caregivers were instructed to complete the ObsRO survey each evening. For each day’s survey, the day of the week and date of the survey were entered by the participant and were electronically time-stamped within the database. Once respondents were familiar with its content and operation, they could complete the 16-question survey in less than five minutes. An additional one-two minutes were required to complete a Weekly Global Assessment questionnaire. Individual scores for severityitems were graded on a Likert scale from 0–4, with 0 being mild or absent signs and 4 being very frequent and/or extremely severe signs. Scores for frequency items were also entered as a number. The study coordinator for this project at the University of Florida maintained bi-weekly telephone or email contact with each participant family to solicit the status of the child and provide encouragement regarding compliance with the survey. Reported outcome variables included 1) the percentage of surveys completed; 2) the number of parental inquiries concerning procedure/process and analyses focusing on inter-item correlations; 3) the relationship between ratings on the individual items and overall health ratings provided by the parents (thereby testing the informative value of the items); and 4) results of an exit questionnaire relevant to cognitive debriefing of each participant parent/caregiver. Nine of the 10 families completed the four-month study and 8 families completed the exit questionnaire (Table 2) (21). The data confirmed the ObsRO survey as a measure that captured the major signs affecting PDCD children and was easy to complete, once respondents gained familiarity with the instrument.

Table 1:

Basic Elements of the Observer Reported Outcome Measure

Motor	• Weakness
	• Incoordination
	• Hypotonia/low muscle tone
	• Rigidity/hypertonia
	• Involuntary movements
Breathing	• Breathing difficulty
Seizures	• # seizures in the last 24 hours
Eating	• # vomiting or retching episodes in the last 24 hours
	• Feeding tube present
	○ If Yes, proportion of tube feedings
	• Refusal to eat by mouth
Fatigue	• Tires more easily
Sleep	• Sleep disturbance
General health	• Irritability (emotional)
	• Irritability (interactive)
Other health issues	• Emergency room visit in the last 24 hours
	• Hospitalized in the last 24 hours

Table 2:

Exit Questionnaire

SURVEY QUESTION	YES	NO
Were the survey instructions clear?	8	0
Was the purpose of the survey clear?	8	0
Did the survey capture the main signs affecting your child?	7	1
Were there problems learning to use the electronic tablet device?	0	8
Were there problems transferring the data electronically?	3	5
Did your child’s condition improve during the 4-month assessment?	0	8
Did your child’s condition worsen during the 4-month assessment?	2	6

Incorporation of Survey in Study Design

PDCD is a rare disease, with a postulated prevalence of less than 300 patients in the U. S., and the life expectancy of most affected children is severely truncated. Thus, the pool of patients available to participate in clinical trials is limited. To overcome this potential barrier, we established a national network of clinical investigators experienced in the evaluation and care of PDCD patients; engaged an international parent organization of affected children; and incorporated the ObsRO survey as the primary efficacy outcome measure to optimize the probative value from the small sample size employed in this trial. Locations of study sites and of families who participated in the ObsRO survey developments are depicted in Figure 2.

Fig. 2.

Location of families who participated in initial ObsRO development (yellow) and of institutions participating as study sites in PDCD trial (red).

Initial funding for the study is provided by a four-year grant from the Office of Orphan Products Division (OOPD), FDA. Therefore, the trial, including organization, training of study personnel, patient recruitment and treatment and closeout statistical analysis is to be completed within this period. We took advantage of the rapid pharmacodynamics effects of DCA to construct a double-blind, placebo-controlled, crossover trial of 24 evaluable children, aged 6 months through 17 years, with genetically proven PDCD. To account for dropouts, we aim to recruit 30 subjects to the trial. Patients will initially be screened for eligibility and genotyped to determine GSTZ1 haplotype status, which will stratify subjects into one of the two dose regimens (Fig. 3). EGT carriers will receive 12.5 mg/kg/12hr DCA, which is the standard dose administered in most prior trials. EGT non-carriers will receive half that dose. Each treatment arm lasts four months, with arms separated by a one month washout. Following the nine month double-blind period, families are provided the opportunity to continue on open label DCA, for further assessment of chronic drug safety.

Fig. 3.

Study design of PDCD trial. Each treatment arm lasts four months, separated by a one-month washout (w/o).

The DCC for the trial is at Columbia University, which has created a web-based electronic data capture (EDC) system that is 11 CFR subpart 21-compliant. The EDC facilitates collection of essential trial data and communication of safety information to members of the study team and the trial’s Data and Safety Monitoring Board.

The ObsRO survey instrument used in the Phase 3 Trial of DCA in PDCD may be accessed from a secure web-based electronic data capture system maintained at Columbia University (22). This final version includes both severity and frequency questions in seven domains identified as important by parents/caregivers: motor; breathing; seizures; eating; fatigue; sleep; and general health. The severity questions include descriptors and anchors using a Likert scale (0 = none, not present; 4 = worst ever) to guide the respondent in defining each sign for their child. Once eligibility has been confirmed and dose - stratification has been accomplished by GSTZ1 haplotype analysis, the subject and family will begin a one-month baseline period, in which familiarity with the ObsRO instrument is gained. At least 80% compliance with complete surveyreporting must be achieved during this period for the subject to be randomized. This period also establishes the study participant’s baseline level of health prior to beginning treatment. The approach to missing data is strongly proactive, with a goal of 100% daily survey completion compliance. A detailed review of the ObsRO tool is completed with the parent/caregiver at the enrollment visit, including selection of 3–7 of the ”sentinel signs” among all 11 items listed in the survey that best represent their child’s disease signs. Bi-weekly communication with the parent/caregiver during the trial by the local study site is aimed to further ensure complete data collection.

The primary analysis will adopt an intent-to-treat strategy, with an appropriate imputation scheme for missing data (although, as noted, the primary goal is to minimize this). The ObsRO scoring analysis includes 11 equally-weighted items reflecting at-home functionality. The primary endpoint will be the difference between the subject’s average slope in Treatment Period 2 minus the average slope in Period 1, averaged over all 11 outcomes, regardless of treatment ordering. A negative slope indicates functional improvement during the recording period.

Conclusions

The potential innovation and impact of this trial are several-fold. Perhaps most importantly, it offers the first opportunity for the PDCD community to engage in targeted treatment of the disease. In this regard, the content of the ObsRO tool incorporates terms, words, descriptors and phrases commonly used and recommended by PDCDfamilies - an empowerment of the lay community that may serve as an example for engaging families and communities in trial development. Our experience also emphasizes the close engagement of the FDA in evolving the ObsRO instrument and provides lessons in regulatory review and approval and a template for future trials of rare diseases. Indeed, we know of no precedent for using an ObsRO tool as an FDA-sanctioned primary outcome measure in a pivotal trial for any rare disease. Thus, successful application of our survey in the DCA/PDCD trial may have far-reaching consequences for the design and implementation of future interventional trials of mitochondrial and other rare disorders.

Highlights.

• There are no Food and Drug Administration (FDA)-approved therapies for primary mitochondrial diseases and there are no validated biomarkers that predict patient functionality following therapeutic intervention.

• In conjunction with families of children with pyruvate dehydrogenase complex deficiency (PDCD) and the FDA, we developed and determined the feasibility of Reported Outcome (ObsRO) survey tool to measure home functionality.

• The ObsRO survey is being applied as the primary efficacy outcome measure in a federally funded multicenter randomized controlled trial of dichloroacetate in children with PDCD.