Assessing the Efficacy of Cell Transplantation for Parkinson’s Disease: A Patient-Centered Approach

Abstract

Background:

Evidence from a growing number of preclinical studies indicate that recently discovered stem cell lines may be translated into viable cellular therapies for people with Parkinson’s disease.

Objectives:

In a brief but critical review, we examine the use of primary and secondary outcome measures currently used to evaluate the efficacy of cellular therapies.

Methods:

The current practice of relying on a single primary outcome measure does not appear to provide the evidence required for demonstrating the robust, life-changing recovery anticipated with the successful implementation of cellular therapies.

Results:

We propose a 360-degree assessment protocol, which includes co-primary and composite outcome measures to provide accurate and comprehensive evidence of treatment efficacy, from the perspectives of both the researchers and the patients.

INTRODUCTION

Remarkable advances in the field of stem cell technology are anticipated to provide a supply of safe and standardized cell lines suitable for transplantation in people with a range of neurological disorders including Parkinson’s disease (PD) [1–4]. Stem cells, when shown to be safe and ready to be efficiently manufactured, may be used for replacing dopamine (DA) expressing cells in suitable patients with PD [5–7]. It is understood, however, that the successful translation of the new cell lines into effective treatments depends on taking into account the lessons learned over three decades of pre-clinical and clinical research for replacing DA expressing cells [8–10].

One of the important lessons is that confirmatory trials for evaluating the functional efficacy of DA expressing cells should be initiated only in the light of clear preliminary evidence of their safety and efficacy [11, 12]. It has been argued that the prematurely implemented double-blind randomized control trials (RCTs) may have impeded the clinical implementation of cellular therapies for the treatment of PD [12]. A multi-center project (TRANSEURO) is currently being completed to demonstrate the effect of key factors, such as patient selection, tissue composition, surgical techniques and trial designs for transplanting DA expressing fetal cells [6, 9].

Ideally, pre-clinical and clinical evidence should be integrated into a coherent treatment protocol for guiding further advances in stem cell research leading up to the evidence-based implementation of cellular therapies. To quote Dunnett and Rosser [13], “Only once a fully reliable and effective standardized treatment protocol is determined is it appropriate to seek validation in a randomized control trial” (p.85)

In order to develop an effective treatment protocol, it is essential to explore the psychosocial processes which enable patients to achieve optimal recovery following the transplantation of cells [12, 13]. It has been argued previously that a multi-disciplinary patient-centered approach is most suitable for ensuring the best possible health outcomes for people with chronic, disabling conditions such as PD [14]. Therefore, we propose a patient-centered approach to data collection for identifying the anticipated structural and concomitant functional improvements in people undergoing stem cell transplantation.

The aims of this article are to review the limitations of current strategies for producing evidence to demonstrate the efficacy of cellular therapies and to propose a patient-centered, biopsychosocial approach to evaluating the preliminary and confirmatory efficacy of cell-based therapies.

APPROACHES TO EVALUATION OF TREATMENT EFFICACY

It is now accepted that the transplantation of DA expressing stem cells will not provide a cure for Parkinson’s disease [9–11]. Rather, if successful, cell replacement therapies are expected to reverse the progressive loss of DA which follows the degeneration of the nigrostriatal system and to improve motor and psychosocial functioning [10, 13, 15]. This section provides a brief overview of how measurement tools have been used by researchers to evaluate the preliminary and confirmatory efficacy of cell or gene-based interventions for PD.

Protocols for selecting measurement tools

Owing to the progressive damage to multiple neural systems and the fact that the presentation and severity of the biological impairments and psychosocial dysfunctions vary markedly across people with PD [16], there has been a determined effort to select and organize outcome measures into coherent assessment batteries.

A protocol for selecting standardized assessment tools, referred to as the Core Assessment Program for Intracerebral Transplantation (CAPIT) was devised to guide research and evaluation of intracerebral neurotransplantation for PD. The CAPIT protocol provided a comprehensive list of outcome measures applicable to collect data on key dimensions of PD [17]. The CAPIT protocol was revised to create the Core Assessment Program for Surgical Intervention Therapies (CAPSIT-PD) for the evaluation of the safety of different types of surgical interventions for PD. Further, in recognition of growing evidence for non-motor impairments, the CAPSIT-PD also included standardized tests of memory, cognition and emotional states in people with PD following surgical interventions [18]. Although the CAPIT protocol seems to be no longer in general use, the idea of the importance of a standardized, multi-source assessment tool remains relevant to our present discussion.

Primary and secondary outcome measures

There are numerous standardized measurement tools available and applicable for clinical evaluation and research. The United Parkinson’s Disease Rating Scale (UPDRS) is an example of a comprehensive tool, which has been used in numerous research studies to track the progression of PD. The UPDRS consists of four subscales to assess functional changes including cognition and mood, activities of daily living (ADL), motor symptoms UPDRS motor and the complications of therapy [19] The UPDRS is typically administered under two conditions; patient without medications (UPDRS ‘off’) or while the patient is using medications (UPDRS ‘on’). The UPDRS was revised by the Movement Disorder Society to produce the MDS-UPDRS [20]; however, most of the available evidence for the efficacy is based on the original version of the UPDRS.

In order to confirm the efficacy of an intervention which involves neurosurgical procedures, researchers follow a general strategy as described, for example in the ‘CONSORT’ statement [21]. Briefly, having designed a randomized controlled trial (RCT) researchers nominate a primary outcome measure and specify an endpoint for completing the trial. The efficacy of the intervention is demonstrated by statistically and clinically significant differences on the primary outcome between the treatment and control groups. The primary outcome measures are also important to conduct power analyses and are relied on to conduct meta-analyses to synthesize the results of related trials [9, 12].

A host of secondary outcomes are also used to evaluate changes on constructs relevant to cellular therapies for PD, such as cognitive and emotional changes. However, research groups tend to differ on how they report the data on secondary outcome measures and the degree of importance they attribute to the secondary outcomes [9, 22, 23]. There appear to be no explicit rules for deciding how primary and secondary outcomes should be selected to determine treatment efficacy. Considering the decisive importance of primary outcome measures, there is a need for evidence-based discussion for identifying the criteria required for designating the available assessment tools into primary and secondary categories.

A CRITICAL LOOK AT USING DATA GENERATED BY CURRENT ASSESSMENT TOOLS

It is understood that there is a need for an outcome measure which clearly indicates a long-term, life changing recovery for the therapeutic benefits of the cell transplantation program [24, 25]. In this section, we will examine if a single primary outcome measure is adequate for the task of establishing the efficacy of cellular therapies.

Selection of the primary outcome measure

Research groups developing and evaluating surgical interventions differ in their selection of primary outcome measures for assessing the outcomes of surgical treatments for PD. For example, groups evaluating the efficacy of deep brain stimulation (DBS) nominated a variety of primary outcomes, such as, time spent in the ‘on’ state without troubling dyskinesia [26] or the “total score on PDQ39” [27]. In contrast, groups evaluating cellular therapies have generally selected the UPDRS (motor, off) as the primary outcome for evaluating efficacy [23].

In some studies, the degree to which transplanted dopamine cells survive, as measured by F-fluorodopa PET scans, correlates positively with symptomatic recovery as measured by UPDRS (motor) in open-label studies of fetal cell transplantation [28]. However, in other studies where significant increase in DA activity were found there were no significant benefits on UPDRS outcomes in the transplanted groups [29]. Further, statistically significant improvements on UPDRS (motor) do not consistently correlate with significant improvements on secondary outcomes, such as Quality of Life (PDQ39) outcomes [30]. The uncertainty in confidently predicting correlations among improvement on the primary and secondary measures of efficacy indicates the need to critically evaluate the construct validity of current assessment tools.

Another problem with primary outcome measures is the uncertainty with deciding on the magnitude of improvement which indicates a clinically significant effect size. For example, Ko and colleagues [31] nominated a threshold of 2 point (approximately 5%) improvement on UPDRS (motor) as their criterion for identifying placebo responders. In contrast, LeWitt and colleagues [30] selected a minimal 9-point difference which has been previously identified as an indicator of strong efficacy [32]. Further, a meta-analysis relied on a 33% or greater improvement on UPDRS (motor) to indicate recovery [12]. The rationale for choosing this threshold was that this effect size has been used as a criterion for identifying L-DOPA related medication benefits [17]. Barker and colleagues, anticipated a 50% improvement on UPDRS (motor) improvement in the group receiving DA expressing cells in an ongoing (TRANSEURO) trial [9]. Unquestionably, this would be a decisive effect size, comparable to the efficacy of DBS which has been adopted for clinical treatment [26, 27]. The question, from a person-centered approach to data collection, is if motor improvements by themselves constitute adequate evidence for the clinical efficacy.

Perspectives of the cell recipients

While significant improvements, as exemplified by UPDRS (motor, off) are important indicators of efficacy from the researchers’ perspective, there is the question of whether the patients with advanced PD also see this as a meaningful, life changing recovery. There is evidence that the post-treatment changes in motor symptoms before taking medications are not necessarily the primary concerns of people with advanced PD [33].

The issue here is that standardized tests, such as UPDRS (ADL) or PDQ39 to assess patients’ experiences on predetermined questions, thereby confining possible responses to quantifiable characteristics of the patients on operationally defined factors. There may be additional information that participants need to communicate regarding their experiences of the recovery process and the perceived benefits and limitations of the treatment. It has been suggested that additional evidence, based on the rigorous application of qualitative methods, would provide the detailed information necessary for understanding recovery from the participant’s perspectives [12, 34]. Qualitative research has a long history to provide essential evidence for understanding the experiences of people with PD [34]. We propose that this method should be also an integral component of conducting research and evaluation in the field of reconstructive therapies using stem cells [6, 34, 35].

This brief critique questions the validity of relying on a single primary outcome measure, such as UPDRS (motor off) as the evidence for a meaningful, life changing measure of treatment efficacy. Unquestionably, improvement of motor symptoms are necessary components of the evidence, but are not sufficient by themselves to confirm the efficacy of cell transplantation.

CO-PRIMARY ENDPOINTS FOR THE QUANTITATIVE ASSESSMENT OF RECOVERY

Researchers and regulators previously addressed issues associated with evaluating treatments for complex disorders where multiple endpoints are required for establishing efficacy [36–38]. The use of co-primary endpoints and composite outcome measures were recommended for evaluating the results of trials where two or more outcomes were considered to be essential components of the evidence for efficacy [36]. In the light of the problems associated with relying on a single primary outcome, we propose an alternative, multi-source assessment protocol, based on the biopsychosocial model. Application of this approach to collecting data for demonstrating the efficacy of cellular therapies requires evidence on the essential dimensions of the recovery process.

The following five dimensions of recovery (see Table 1) for cellular therapies are suggested, based on the biopsychosocial categories recommended by McGinley and Danoudis for organizing currently used assessment tools for PD [39] (Table 1).

• I.Reversal of neuropathology: For cell replacement therapies, it is essential to provide evidence for significant levels of cell survival and significant increases in DA activity [28, 29]. Measurements tools, which produce data for changes in DA functioning, such as F-Flurodopa PET scans, are absolutely necessary for evaluating the efficacy of DA replacement therapies.
• II.Improved response to medications: Although medications such as L-DOPA are effective in the earlier stages of PD, with the inexorable degeneration of the nigrostriatal system there is a gradual loss of their efficacy. Quantity of medication use is recorded by treating physicians and changes are usually reported as L-dopa equivalents. Adverse side effects, such as dyskinesias and also the time intervals during which drugs are not effectively controlling motor symptoms are also recorded by the research participants, in diaries designed for this purpose. If the treatment is effective then the increased DA turnover should be correlated with an improved response to medications.
• III.Symptomatic changes in motor functioning: Improvement on motor functioning, as for example assessed by UPDRS (motor) is essential for demonstrating the efficacy of stem cell therapies. Symptomatic improvements should be correlated with positive outcomes on (I) and (II).
• IV.Activity levels: We suggest that the cell transplantation cannot be judged as effective without significant improvements on ADL outcomes. The ADL subscale of the UPDRS is used to assess the subjective reports of people with PD regarding their ability to perform basic activities, indicating toileting, preparing meals or house maintenance.
• V.Changes in social participation and quality of life: As with activities of daily living, meaningful improvements in quality of daily life are necessary attributes of a clinically effective treatment of PD. The Parkinson’s Disease Questionnaire (PDQ39) is a standardized questionnaire, which has been relied on as an overall indicator of the quality of life (QoL) [40].

Table 1.

Dimensions of recovery and examples of relevant standardized outcome measures.

	Dimensions of Recovery following Cell Transplantation	Examples of relevant Measurement Tools
I.	Substantial number of transplanted cells survive and express DA,	18F-flurodopa PET scan
II.	The participants experienced an enhanced response to medications,	Reduction in levodopa equivalent
III.	Improvement in motor symptoms,	UPDRS (motor)
IV.	Participants experienced improved levels of activity,	UPDRS (ADL)
V.	Clear evidence for enhanced social participation and quality of life.	PDQ 39

All the above factors should be correlated for indicating a clinically meaningful recovery, and thereby confirming the efficacy of stem-cell transplantation.

There are many other measurement tools currently available to assess changes in the progress of PD; we only presented a very limited sample in this brief overview. There are ongoing technological changes which may lead to new and more accurate assessment tools which could lead to improved assessments of efficacy [41].

The key point here is that the five domains of recovery described above are hypothesized to constitute the interacting components of an integrated system representing the recovery process following the grafting of cells. By postulating an integrated system, we propose that the evidence for improvements in each of the numerical components is essential to represent and understand the mode of action of the transplanted cells. When the specific tools are selected, it may be possible to synthesize the 5 scores (see Table 1) into a ‘global’ quantitative indicator measure of efficacy, as previously adopted by Kieburtz and colleagues [42].

COMPOSITE RESULTS

In addition to the five quantitative domains of the recovery process described above, a patient-centered approach also requires detailed evidence from the perspectives of the participants. The original idea for the proposed 360-degree assessment protocol comes from the discipline of organizational psychology (e.g., [43, 44]). In this discipline, multi-source assessment protocols (such as self-reported, supervisor/peer reported) have been used for the evaluation of organizational factors such as performance or job-satisfaction. Although the evaluation of performance in organizational settings is very different to that of conducting cellular therapies, we suggest that the logic underlying the two multi-perspective assessments is arguably identical.

As shown in Fig. 1, the proposed 360-degree multi-source assessment entails two orthogonal factors; the source of the data (subjective/objective) and the form of the data (numerical/narrative). The model postulates that data collected from the four different domains (A, B, C and D) integrated into a combined response. As discussed below, combined response provides the evidence for a meaningful, life-changing recovery following cell transplantation.

Fig. 1.

Model for a 360-degree assessment of functioning in PD. A: represents numerical, self-reported data generated by participants using standardized questionnaires. B: represents numerical data assessed by observers or clinicians using standardized measurement tools. C: represents narrative data recorded by observers or clinicians. D: represents narrative data reported by people with PD — combined results. CR: represents the composite or combined response which ideally includes improvement on all the four dimensions.

The components of the multi-source model include existing quantitative measures (dimensions A and B) as those listed in CAPSIT-PD. It is understood that these remain essential for the assessment of changes of variations representing treatment efficacy.

• •‘A’ represents numerical, self-reported data generated by participants using standardized questionnaires such as data from the PDQ39 or UPDRS (ADL).
• •‘B’ represents numerical data assessed by observers or clinicians using instrumentation such as PET-scans or standardized assessment scales, as for instance UPDRS (motor).

In addition to the numerical measures, the multi-source protocol explicitly incorporates two qualitative dimensions (C and D). The narratives are included in the assessment protocol to enable understanding the therapeutic outcomes from the perspectives of the people with PD, as well as the health professionals who are working with the participants following the interventions. The themes emerging from these narratives provide detailed information about the experiences of the participants and the social context in which recovery takes place following cell transplantation.

• •‘C’ represents narrative data recorded by observers or clinicians describing changes in the participants’ symptoms and level of activity and social participation.
• •‘D’ represents narrative data reported by people with PD describing their personal experiences and the perceived impacts of the intervention for overcoming specific aspects of activity limitations and levels of social participation associated with PD.

Data collection for C & D dimensions do not require standardized measurement tools but rather face-to-face interactions including semi-structured and in-depth interviews, and the observation of and interactions with participants over a period of time in their physical environments and social settings.

Brief case studies (C) written of each patient by researchers provides valuable insights into the progress of the patients. An example of how this has been implemented can be seen in the reports of Swedish researchers [22] who provided brief summary for the progress of each of their participants in a longitudinal study of the benefits of cell transplantation for PD. For example, quoting the report on Patient 7: “…This patient is now running his own business on a full-time basis (he previously worked part-time) and has been able to take up car driving again, which was impossible before surgery” [22, p.1124]. This information by itself is not decisive, as it is colored by the biases, expectations and values of the expert observers. However, when triangulated with relevant quantitative sources of evidence, the narrative data contributes valuable information for enabling a holistic overview of the recovery process [12].

Component ‘D’ describes the experiences and consequences of cellular therapies from the participants’ perspectives. It is essential to establish that objectively measured improvements are consistent with the expectations and values of the people with PD. This evidence is best collected by interacting with research participants in the context of rigorously applied qualitative methods [34, 45]. Qualitative data are analyzed to produce the themes, which enable researchers to interpret empathically the experiences and the reasons for the actions of people with PD undergoing cellular therapies.

The concept of themes is illustrated by a study by O’Brian and colleagues [46] who conducted semi-structured interviews to collect data to explore the personal meanings of participating in exercise program for people with PD. Analysis of the narrative data generated four major themes: adapting to pain and loss, the influence of others, making sense of the exercise experience and hope for a more active future. These themes provided the information applicable to understanding the barriers and facilitators which influenced the participants’ successful completion of the exercise programs.

A study by Fisher and colleagues [47] demonstrates how data from quantitative and qualitative sources can be combined to evaluate the acceptability of body-worn remote movement sensors. The experiences of participants with PD were evaluated by three different methods of data collection; a qualitative ‘free text response’, a structured quantitative questionnaire and the recording of estimated ‘non-wear time’. The synthesis of data from the three sources were triangulated to provide ecologically valid evidence applicable to facilitating the wearing of the sensors by the research participants with PD in their everyday environment.

As stated earlier, the composite results refer to the proposed synthesis of the results on each measure from the dimensions A, B, C, and D. The scores are used to determine the overall positive response of each participant to the cellular therapies. Composite result is not a specific, weighted statistic but rather the demonstration of satisfactory outcomes on each dimension, as reported from the perspectives of the researchers and the participants. Ideally, a ‘satisfactory score is represented by, (1) improvement on each of the five quantitative measurements in A and B also and (2) the report of perceived improvements in activity levels and the quality of everyday life (C and D).

APPLICATION TO STEM CELL RESEARCH

The translation of stem cell research into viable treatments begins with extensive and detailed pre-clinical experiments. The aim of pre-clinical research is to demonstrate, the mode of actions of the cells in animal models and to estimate the probable safety and efficacy of the cells in human participants [7, 48].

Phase 1, preliminary safety and feasibility studies with human participants aim to identify adverse events associated with the transplantation of the cells to determine if the dose levels are adequate and ultimately, to enable the researchers to decide if further efforts to develop the stem cell line is justified. In general, preliminary studies involve small sample sizes and open-label designs, where a single primary outcome measure of efficacy such as UPDRS motor, appears sufficient for further trials [49].

Phase 2, preliminary efficacy studies [49] aim to provide detailed evidence for the safety and clinical benefits of stem cell transplantation for PD. As stated by [49] this stage of the research program requires a primary outcome measure which “…must show an improvement that is clinically meaningful to a patient so that the risk of a neurosurgical procedure is reasonably balanced by the potential clinical benefits and the value of the generalizable knowledge.” (p.647)

We suggest that the 360-degree assessment protocol is well suited to the task of assessing the preliminary efficacy of DA expressing stem cells. Of course, the results of additional secondary outcome measures and most importantly, the occurrence of adverse events must be also included in the data. As discussed earlier, the preliminary efficacy measure should clearly indicate significant effect sizes at the endpoint of open-label studies before confirmatory studies are initiated.

Phase 3, confirmatory studies rely on RCT designs where a sample of participants with PD are randomly assigned to either stem cell treated, active group(s) or to the control groups(s) [49]. Statistically significant and clinically meaningful difference in favor of the transplanted groups at the designated endpoint is the accepted criterion for treatment efficacy. Significant differences in favor of the transplanted group on the five co-primary outcomes, if consistent with qualitative reports, would provide convincing evidence for the efficacy of stem cell transplantation.

Further, the 360-degree, person-centered evaluation enables the accurate identification of individual research participants who have definitely improved by the endpoint of the trial. The combined response may be used to indicate the proportion of people in the control and stem cell treated groups who have convincingly improved over a given period of time. The confirmation of the efficacy of stem cell lines would be decided by the proportion of participants in the control and treated groups identified as achieving a satisfactory composite result. The results of the RCT would be analyzed using inferential statistics for nominal data, such as odds ratios, to demonstrate the statistical and practical significance of the efficacy of the transplantation of stem cells.

SUMMARY

A compelling lesson learned from previous research is that confirmatory trials of the efficacy of new stem cell lines for the treatment of PD should be initiated only after an evidence-based, conceptually sound treatment protocol is constructed. From a patient-centered perspective, the current practice of relying on a single primary outcome measure does not provide the evidence required to demonstrate the life-changing recovery which would confirm the efficacy of transplanting stem cells in people with PD.

We proposed an alternative evaluation strategy, which uses five co-primary outcome measures (Fig. 1) to demonstrate that the transplantation of the cells increases the expression of DA, improves response to medications, ameliorates motor symptoms, and enhances activity levels and social participation in patients. Further, the 360-degree protocol explicitly takes into account the participant’s experiences as a necessary component of the evidence for determining efficacy. The proportion of people with PD in the transplanted group may be compared to the controls to determine the efficacy of stem cell therapies in comparison to control treatment.

We recognize that this is a very preliminary presentation of the proposed assessment protocol. Many questions need to be answered before we can decide if the 360-degree protocol improves the accuracy of the evidence for evaluating the efficacy of stem cell therapies for PD. If further evidence supports the validity of this assessment protocol then, it may be perhaps also be considered for evaluating other novel interventions for the treatment of PD.

CONFLICT OF INTEREST

The authors have no conflict of interest to report.