Gene Therapy Researchers' Assessments Of Risks And Perceptions Of Risk Acceptability In Clinical Trials

Abstract

Although recent clinical trials have demonstrated the increasing promise of gene therapy, they have also illustrated the difficulties of assessing risks, given the inherent uncertainty of trial outcomes. An international survey was conducted to investigate gene therapy researchers' perceptions and assessments of risks in clinical trials. Data from respondents (n = 156) demonstrated researchers' perceptions of clinical context and the strength of preclinical evidence strongly influenced risk assessments and judgments of acceptable risk levels. Professional experience in clinical care, and particularly care of children, predicted favorable attitudes toward nonanimal preclinical models and trial initiation when sub-optimal treatments were available. The potential for adverse events to impact negatively on the gene therapy field and on public trust were relevant considerations when planning a trial. Decisions about clinical trials appear to be influenced not only by the clinical context and preclinical evidence, but also subjective factors reflecting the experience of researchers, value-judgments about risk and benefit, and attitudes toward preclinical models, uncertainty, adverse events, and the perceived needs of patients. It is clear that risk assessment in clinical research involves moral and scientific judgment. Identifying moral assumptions and qualitative assessments underpinning the design and conduct of research may facilitate future decision-making in clinical trials.

Introduction

Although the promise of gene therapy has long been recognized, it is only in recent years that this promise has begun to be realized. Therapeutic efficacy following gene therapy has now been reported for several inherited diseases, including X-linked severe combined immunodeficiency, adenosine deaminase deficiency, chronic granulomatous disease, Leber's congenital amaurosis, adrenoleukodystrophy, β-thalassemia, and hemophilia B; as well as leukemia and HIV.^{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18} The lag in the transition from promising early laboratory results to clinical benefit underscores the challenges and uncertainties faced in the clinical translation of complex therapeutic products. Such challenges continue to pose difficulties for gene therapy research and are likely to confront other emerging research fields, including research in stem cell therapy and nanomedicines.

One of the most significant areas of translational uncertainty concerns the extent to which therapeutic efficacy and theoretically predicted risks evaluated in preclinical models will be observed in human research subjects, and the possibility that unknown side-effects may occur. The development of leukemia in 5 of 20 infants treated in the French and British gene therapy trials for X-linked severe combined immunodeficiency particularly demonstrates the difficulties with predicting risk in clinical research, when the only available knowledge about safety and efficacy is generated in preclinical models with limited predictive capacity.^19,20,21 Although vector-mediated insertional mutagenesis had been predicted as a possible risk, the likelihood of this resulting in leukemia was considered remote on the basis of preclinical data obtained from cell culture and small animal models.^22,23,24,25

Although preclinical studies provide quantitative read-outs of safety and efficacy in disease models, the interpretation of data during the planning and conduct of clinical trials involves extrapolation, and may depend upon the attitudes of researchers to risk, benefit, and the validity of the preclinical models used. For example, attitudes toward risk may become more influential when there is greater uncertainty about extrapolating from preclinical safety and efficacy testing. Little is known, however, about how researchers view risks in clinical research, and about how the attitudes of researchers toward risks and perceptions of acceptable risk levels can influence their assessments of risks during the design and conduct of clinical trials.

We have conducted a survey to investigate gene therapy researchers' assessments of risks and benefits and their perceptions of acceptable levels of risks in phase I clinical trials. The factors selected for investigation in this study as potential contributors to decision-making about clinical trials of gene therapy include the nature of the clinical context for decision-making and the attitudes of researchers toward uncertainty, adverse events (AEs) and the validity and utility of preclinical models, as revealed by researchers' interpretations of different levels of preclinical evidence. Demographic factors, such as age, gender, professional background as clinician or scientist, and cultural background, were also investigated.

Results

Demographics of respondents

A total of 156 complete survey responses were received from the mail-out to 1,733 members of gene therapy societies and 287 principal investigators of phase I gene therapy trials who received direct invitations to participate. This yields a response rate in the range of 7.7–9.0%, depending upon the extent of cross-membership between these groups, which is unknown because data from respondents was deidentified. To illustrate the representativeness of this sample of 156 responses with respect to the general population of 2,020 researchers who were invited to participate, it is possible to calculate the margin of error associated with estimating a proportion of 50% in the general population, which is the most conservative proportion to estimate.²⁶ The margin of error associated with a sample of 156 respondents is 7.6% with a 95% confidence interval. To achieve a margin of error of 5.0% with a 95% confidence interval, 323 respondents would be required. This suggests that the accuracy of proportions estimated using this sample population would not be substantially increased if the sample size were increased by more than twofold.

All major demographic groups of interest were represented, including gender, age, experience in a clinical trial, country in which research is conducted, years of research experience, professional role, and involvement in the clinical care of children (Table 1). These demographic groups were used for the subsequent cross-tabs analyses. Seventy-five per cent of respondents reported involvement in a clinical trial, almost 60% of respondents conducted their research in Northern America and almost 70% of respondents identified themselves as scientists.

Table 1. Demographic details of survey respondents.

Influence of perceptions of clinical context and attitudes toward preclinical evidence on assessments of risks in a hypothetical clinical trial

Overall, the proportion of respondents who agreed that it would be appropriate to recruit subjects to a phase I clinical trial depended upon both the hypothetical clinical scenario and the strength of the preclinical evidence (Figure 1). The least amount of support for a trial was observed with the combination of the scenario where disease could be controlled by diet and the cell culture model when a knock-out mouse could be generated within 2 years. The greatest amount of support was observed with the scenario involving an untreatable disease with death in infancy and a large animal model. For the scenarios where no treatments were available, support for initiating a clinical trial generally increased with disease severity, more rapid disease progression and younger patient age. Support was lower for the scenarios where treatments with varying limitations were available.

Figure 1.

Percentage of respondents who agreed it would be appropriate to recruit subjects to a phase I clinical trial for each of the hypothetical clinical scenarios based on data generated in each of the preclinical models. The radii of the circle behind each of the percentage values represents the relative proportions of respondents who agreed it would be appropriate to recruit subjects to a phase I trial. AEs, adverse events.

Analysis of population-averaged effects across the responses to the panel of questions using generalized estimating equations (GEEs) demonstrated levels of support for initiating a clinical trial were highly dependent on both the clinical scenario (P < 0.001) and the preclinical model (P < 0.001). There was also a significant interaction between the clinical scenario and preclinical model variables (P < 0.001). For each preclinical model, the effects of changing the clinical scenario were examined relative to the scenario which invoked the greatest support, where disease was untreatable and resulted in death in infancy (Figure 2a and Supplementary Table S1 online). The likelihood of disagreement with initiating a trial progressively decreased as treatments became unavailable, life expectancy was reduced and the disease became more severe and progressive. Changing the clinical scenario to any where treatments were available had a highly significant effect, regardless of the type of preclinical model (Supplementary Table S1 online). Changing the scenario to either of the two scenarios where no treatments were available and life expectancy extended into adulthood had a significant effect for most of the preclinical models. Overall, the effects of changing the clinical scenario from one which was untreatable and caused death in infancy were the strongest for the types of preclinical models which involved small or large animals.

Figure 2.

Consistent patterns of odds ratios when separate GEE models were fitted for each of the preclinical models and clinical scenarios. (a) Odds ratios representing the likelihood of disagreeing with the initiation of a clinical trial when the hypothetical clinical scenario was changed from the scenario involving an untreatable disease and death in infancy, which invoked the greatest support. Separate GEE models were fitted for each of the types of preclinical model. (b) Odds ratios representing the likelihood of agreeing with the initiation of a clinical trial when the type of preclinical model was changed from a cell culture-based model when a mouse model could be generated in 2 years. Similarly, separate GEE models were fitted for each of the clinical scenarios. AEs, adverse events; BMT, bone marrow transplant; freq., frequency; GEE, generalized estimating equations.

Similarly, separate GEE models were fitted for each of the clinical scenarios to examine the effects of changing the preclinical model, relative to the model that generated the least support for a trial, which was the cell culture model where a knockout mouse could be generated within 2 years (Figure 2b and Supplementary Table S2 online). The likelihood of agreeing with the initiation of a trial progressively decreased as the strength of preclinical evidence decreased, from the large animal model through to the cell culture model (Figure 2b). For all of the clinical scenarios, the effects of changing the type of preclinical model to a small or large animal model were highly significant (Supplementary Table S2 online). In general, the effect of changing the type of preclinical model was strongest for the clinical scenarios where treatments were available and quality of life was considered acceptable. Taken as a whole, the GEE analyses confirmed the overall pattern that support for a clinical trial increased with stronger preclinical models, greater disease severity and reduced life expectancy and treatment availability for the condition under study.

Differential risk assessment by demographic groups when making decisions about initiating a hypothetical clinical trial

To explore the potential influence of demographic factors, responses were aggregated according to gender, age, geographic location, professional role, involvement in pediatric care, and years of research experience. A heat-map was employed as a visualization tool for the purpose of guiding the analytical process. The proportions of respondents in each demographic group who supported a hypothetical clinical trial in each of the clinical scenarios and for each of the preclinical models were represented using a heat-map (Figure 3). The pattern of colors displayed for the total cohort of respondents in the first column of the heat-map indicated that higher support for a trial was correlated with increased disease severity and strength of preclinical evidence. When the respondents were aggregated according to demographic factors, the heat-map patterns for the different demographic groups were similar to the pattern for the total cohort, indicating overall that levels of support were similar. Similar color patterns were also observed between demographic categories. In general, demographic factors appeared to exert little effect when the preclinical evidence was strong and when the disease scenarios were incurable and associated with higher severity, lower quality of life and shorter life expectancy. Gender, age, regional location, and years of research experience did not appear to influence decision-making outcomes. However, in some situations where the preclinical evidence was weaker and the disease scenarios were less severe, it appeared that there were some differences in levels of support according to professional background of the respondents as a clinician or scientist and involvement in pediatric care. Therefore, the effect of professional experience on decision-making was examined for the questions involving the two least severe disease scenarios and the two cell-culture preclinical models (Table 2).

Figure 3.

Heat-map representing the proportions of respondents in each demographic group who supported initiation of a clinical trial, for each of the clinical scenarios and types of preclinical models. Dark green represents 100% agreement with initiating a trial, while dark red represents 100% disagreement (0% agreement). The columns are separated by white spaces according to the demographic groups of interest, with the total cohort represented in the left-hand-most column. Each of the eight groups of seven rows separated by a white space represents one of the eight types of preclinical evidence, ordered according to decreasing strength from top to bottom. These eight rows contain groups of seven rows, which represent each of the seven different clinical scenarios. The seven clinical scenarios are ordered according to decreasing disease severity from top to bottom: (i) no available treatment, death in infancy; (ii) no available treatment, death in childhood or adolescence; (iii) no available treatment, death in early adulthood; (iv) no available treatment, life expectancy reduced by 20 years; (v) disease curable by bone marrow transplantation, with risks and side-effects; (vi) disease can be controlled by blood transfusions; (vii) disease can be controlled by diet. AEs, adverse events.

Table 2. Effects of professional experience on decision-making in the least severe clinical scenarios when preclinical evidence involved cell culture-based models or there was a related phase I study with low frequency adverse events.

When the effects of professional experience were examined, clinicians and clinician-scientists were more likely than scientists to support a phase I trial on the basis of data generated in cell culture models when disease could be treated by either blood transfusion or diet. Respondents involved in the clinical care of children were similarly more likely to support a phase I trial when the preclinical evidence was weaker and the disease was less severe. These associations persisted when nonclinicians were excluded from analysis for the cell-culture model when disease could be treated by blood transfusion, but not when disease could be treated by diet. On the other hand, scientists and respondents not involved in pediatric care were more likely to decide against initiating a trial when low frequency adverse events occurred in a related phase I trial. When the preclinical model was changed from a cell culture model to a good mouse model, scientists were more likely than clinicians and clinician-scientists to change their decision from disagree to agree in three of the scenarios where treatments were unavailable and the scenario where disease could be treated by bone marrow transplantation.

Involvement in pediatric care had a marked impact upon decision-making when included as a variable in the GEE models fitted for the cell culture model, the cell culture model when a mouse model could be available in 2 years, the good mouse model when there was a related phase I study with no AEs, and the good mouse model when there was a related phase I study with a low frequency of AEs (odds ratios of 7.42, 8.54, 4.06, and 5.13, respectively; P values of 0.006, 0.003, 0.044, and 0.024, respectively). Similarly, professional experience in pediatric care affected decision-making outcomes when analyzed as a variable in the GEE models fitted for the scenarios where disease could be controlled by blood transfusions or by diet (odds ratios of 9.55 and 6.50, respectively; P values of 0.002 and 0.011, respectively). When nonclinicians were excluded from the GEE analysis, the effect of involvement in pediatric care remained for the GEE models fitted for the cell culture model, the cell culture model when a mouse model could be available in 2 years, and also the scenario where disease could be cured by bone marrow transplantation (odds ratios of 2.75, 2.87, and 2.52, respectively; P values of 0.049, 0.041, and 0.034, respectively).

Attitudes toward research involving human subjects

Responses to questions about attitudes toward research involving human subjects correlated with the responses to decisions about initiating a hypothetical clinical trial regarding the types of clinical scenarios and preclinical model more likely to lead to support for a trial. Respondents emphasized the importance of benefit in pediatric trials and preclinical safety testing, despite recognizing the limitations of small animal models. The importance of benefit in trials involving children was expressed by almost two-thirds of respondents (Table 3). Over two-thirds of respondents preferred recruitment of patients with advanced disease as compared with the patients with early stage disease.

Table 3. Percentage of respondents who agreed with statements about attitudes toward research involving human subjects.

Although over 80% of respondents considered the absence of AEs in small animal models did not facilitate meaningful predictions about long-term safety outcomes in humans, ~70% thought a clinical trial should not proceed without extensive toxicological data for a severe and incurable disease (Table 3). On the other hand, over 70% of respondents considered it may be acceptable to proceed to clinical trial in the context of a severe and incurable disease, when preclinical measurements of therapeutic efficacy fell short of clinical significance in a statistically powered animal study.

When asked about their own decision-making about a clinical trial, respondents considered the potential negative impact of AEs as a relevant factor, and indicated their personal decision-making had been affected by the report of an AE in their own field. Nearly 80% of respondents identified the potential for AEs to have a negative effect on public support and trust or on their own field of research as relevant considerations when planning a clinical trial (Table 3). Almost 60% of respondents indicated their own decision-making had been affected by the report of an AE in a trial in their field.

Discussion

This study investigated gene therapy researchers' perceptions of risk, risk assessment, the validity and utility of preclinical models and AEs, as well as the potential influence of these perceptions on decision-making about gene therapy trials. Our data obtained from a subset of gene therapy researchers reveals that the characteristics of the target disease and existing therapies, as well as the availability and practicalities of animal models, are highly influential when decisions are made about the design and conduct of clinical trials. Decisions about clinical trials are influenced by a number of subjective factors, such as length and kind of experience, value-judgments regarding risk and benefit, attitudes toward the strength and validity of preclinical evidence, and perceptions of the need for a novel treatment. This represents the first empirical study of how gene therapy researchers assess risks in clinical trials, the factors which influence decision-making in clinical research, and how gene therapy researchers balance these factors. Previous empirical studies of decision-making about clinical trials have evaluated the experience of research participants, with a particular focus on consent and motivations for research participation.^{27,28,29,30,31,32}

There are a number of potential limitations to this study which may arise from the relatively small sample size of 156 respondents, the heterogeneity of respondents, and limited number of qualitative questions included. The low response rate (7.7–9.0%) is consistent with the literature regarding online surveys, which describes response rates ranging from 1% to more than 60% depending upon the characteristics of the target population, the survey length, mode and strategy for delivery, and salience of the survey topic.^33,34 In addition, our response rate is likely to reflect both the large number of complex questions in the survey, necessary for obtaining meaningful data, and also the likely time-poor nature of the target research population, which renders any dataset of this nature difficult to obtain. The consequence of this low response rate is that these findings are exploratory in nature, have limited potential for generalizability, and need confirmation in larger studies.

Related to the low response rate is the possibility of an ascertainment bias in our data, which may arise if the characteristics of survey respondents differ significantly from those of the general population of gene therapy researchers. Despite these limitations to generalizability, the findings and analyses reported here nevertheless have internal validity with respect to the sub-population of gene therapy researchers who responded to the survey. Furthermore, it is also possible that the data may be influenced by a positive ascertainment bias. Seventy-five per cent of our respondents reported having previous involvement in a clinical trial, which is likely to be a higher proportion than in the general population of gene therapy researchers. Although the low response rate necessitates careful and qualified interpretation of the data, the 156 responses obtained in this study reflect opinions in the field and are useful for identifying factors which may influence decision-making and which can be followed up in further studies.

Although the cohort of respondents were heterogeneous with respect to professional role, years of research experience, age, and cultural background, this heterogeneity also enabled identification of demographic factors which may predict attitudes and perceptions, such as professional experience. It should also be noted that few of the “clinicians ” in this analysis identified themselves solely as clinicians, with the majority indicating a dual role as scientists and clinicians. Although this probably reflects the targeting of the survey to a community of researchers, further research and larger numbers of responses is required to further explore differences between individuals who identify solely as either scientists or clinicians.

Details of the clinical context for a trial, including the particular characteristics of the disease and the availability of alternative or effective therapy, were key factors in decision-making about clinical research. This was evident from both the responses of participants to hypothetical clinical trials and to the questions regarding their attitudes to research. When the needs of the patients are perceived as greater and there is increased certainty of a negative outcome under the existing clinical course, then researchers were more likely to support a clinical trial, even when the preclinical evidence was relatively weak. By contrast, when patients generally experienced an acceptable quality of life and sub-optimal treatments were available, then respondents were more divided about the appropriateness of initiating a phase I trial. In these situations, differences emerged between demographic groups, and particularly between scientists and clinicians.

What is clear from our data is that consideration of the particular clinical context and the potential for benefit shaped judgments of acceptable risk during the risk-benefit calculus for phase I trials, even though the primary goal of such trials is to evaluate safety, and not therapeutic efficacy. Respondents to this survey preferred to recruit patients with advanced disease as subjects in trials involving adults, suggesting exposure to uncertain risks was considered more acceptable when research participants had “less to lose”, even when the state of disease progression would reduce the potential for benefit. By contrast, in the pediatric setting, respondents generally considered the risk-benefit ratio should be skewed in favor of benefit. At the same time, when the target disease was severe and incurable, it was considered acceptable to proceed to a clinical trial even when measurements of therapeutic efficacy fell short of statistical significance in a statistically powered preclinical study, such that the prospect of benefit was unlikely. In other words, if the clinical context was dire, then it was more acceptable for the risk-benefit ratio to be skewed in favor of risk.

Gene therapy researchers' support for a hypothetical clinical trial was also strongly influenced by their attitudes toward the strength and utility of evidence generated in different preclinical models. Even though respondents placed great store in the use of preclinical models, they were clearly mindful of the limited predictive capacity and relatedness to human disease of some preclinical models, particularly cell culture-based models and small animal models. Attitudes toward the preclinical models perceived as weaker, particularly primary human cells, were also a key point of difference between scientists and clinicians, and pediatricians in particular.

Conversely, support for initiating a clinical trial was greatest when a large animal was used as a preclinical model across all clinical scenarios, with support being almost universal when treatments were nonexistent. Although it is self-evident that large animal models would be a preferred standard of evidence for preclinical studies, in reality such models are not readily available, are costly to maintain and impose protracted timelines. As a consequence, it is inevitable that researchers must consider the best way to communicate to prospective research participants the remaining uncertainties about safety and efficacy following preclinical studies in small animals. These may include how the limitations of models may affect attempts to extrapolate stable phenotype correction in mice (or other demonstrations of efficacy) to long-term potential benefit in humans, the potential for differences in immune responses between species, and the impact of short life-spans of small animals on assessments of long-term safety.

The possibility that scientists and clinicians may assess risks in clinical research differently highlights the influence of personal experience of disease upon decision-making; in this instance, gained through a professional role in clinical care and the care of children in particular. This finding is consistent with earlier research comparing perceptions of risk held by experts and laypersons.^35,36 It seems likely that those who work in healthcare may have a greater “everyday ” experience of the burden of severe and incurable disease and the impact of such diseases on quality of life. This understanding may inform researchers' evaluations of acceptable levels of risk in research and decisions about when it is appropriate to initiate a trial, such as whether to delay a trial to develop a better preclinical model or an improved vector. Conversely, scientists may have greater experience in interpreting research evidence. Involvement of both scientists and clinicians, therefore, is necessary for maximally informed decisions. Discussion and decision-making in ethics committees may also be more rigorous when committee members have a heterogeneous range of personal experiences of disease.

This study also provides preliminary empirical support for the notion that SAEs can delay research, not from an ethical or regulatory perspective, but as a result of researchers' own decision-making about clinical trials. The gene therapy researchers in our study were clearly concerned not only about the short-term negative consequences of AEs for research participants and their families, but also about the longer-term negative consequences of AEs for public support and trust and for the gene therapy field as a whole. The majority of respondents took these latter, longer-term consequences into account when planning a clinical trial. Moreover, most researchers who had been involved in a clinical trial stated that their own decision-making had been influenced by the report of an AE in their field. These findings suggest there can be unrecognized variables which affect decision-making and it would be interesting to investigate whether other fields, which have experienced SAEs but without the same degree of media attention, would be similarly influenced.

This study has identified a complex matrix of ethical, social, experiential and evidence-related factors which may influence decisions about clinical trials, and which may be differentially evaluated by different individuals. Although the results of our study are intriguing, further research is required to confirm the demographic predictors identified. It is unclear how explicitly and transparently these factors are considered and how they are weighed during decision-making. Judgments about risk, benefit, uncertainty and the adequacy of evidence are inescapably value-laden. This suggests that what is needed is a more sophisticated discussion about how values influence the decision-making and the risk assessments of not only researchers, but also regulators, ethics committees and research participants, and how decision-making within and between these groups may differ. It also suggests that optimal decision-making requires involvement of a mixture of people with diverse experience. A list of factors which have been identified as potential influences on decision-making and specific questions that need to be asked of any clinical study may assist the evaluation of such factors and improve transparency and rigor in decision-making about clinical trials. Such questions could be used to guide researchers' evaluation of the preclinical evidence and clinical context, and their design of studies, selection of participants and communication of risks to potential participants.³⁷ This research represents a first step toward identifying some of the factors which require consideration during decision-making.

Gene therapy researchers' assessments of risks in clinical trials are influenced strongly by their perceptions of the clinical context, the characteristics and needs of the patient cohort, and the strength and validity of preclinical evidence. Other nonscientific factors, including personal experience of disease and concerns regarding the potential negative effects of AEs, may also influence the design and conduct of clinical trials in gene therapy. This exploratory study provides important insights into how researchers make decisions about clinical trials and makes it clear that trial design is as much a moral issue as a scientific one. Recognition that not only the decisions of researchers, but also those of research participants, ethics committees and regulatory authorities, requires the exercise of moral judgment may facilitate more sophisticated thinking about the design of first-in-human trials and the means by which such trials can be effectively communicated to potential participants.

Materials and Methods

Study design. A quantitative internet-based survey was developed for investigating researchers' assessments of risks and perceptions of acceptable risk levels in phase I clinical trials, using a combination of short vignette questions, questions about the attitude of researchers to specific ethical issues about risks in clinical research, and questions about researchers' personal experience of decision-making about risk assessment (Supplementary Appendix online). The vignette questions required respondents to make hypothetical decisions about whether it would be appropriate to recruit participants to a phase I clinical trial. Respondents were presented with eight different descriptions of preclinical data (Table 4) in the context of seven different clinical scenarios (Table 5), and thus made a total of 56 hypothetical decisions. This approach required respondents to assess the risks in each hypothetical vignette and also revealed respondents' perceptions about acceptable levels of the potential risks in each of the hypothetical scenarios. The descriptions of preclinical data in this section were designed to reveal the attitudes of respondents toward uncertainty, AEs and the validity and utility of preclinical models. The scenarios were constructed to elucidate respondents' perceptions of the need for a novel treatment, and incorporated a range of disease severities, life expectancies, patient ages, qualities of life, availabilities and limitations of existing treatments, and uncertainties of outcome if the existing clinical course was followed. The questions about specific ethical issues about risks were designed to show the attitudes of researchers toward the needs of the patients, the strength of preclinical evidence, and the acceptability of research involving children. The questions about researchers' personal experience of decision-making asked about the influence of potential and actual AEs. These questions about researchers' attitudes and experience were designed to give further insights into how researchers assess risks and perceive acceptable risk levels.

Table 4. Descriptions of preclinical evidence provided to respondents for decision-making about the initiation of a clinical trial.

Table 5. Clinical scenarios provided to respondents for decision-making about the initiation of a clinical trial.

An internet-based approach for the survey was chosen to minimize the time required to complete the survey and to facilitate representation of international gene therapy researchers, even though the trade-off was likely to be a low response rate. The survey was distributed on the authors' behalf by the American Society for Gene and Cell Therapy (ASGCT), the European Society for Gene and Cell Therapy (ESGCT), the Australasian Gene Therapy Society (AGTS), the British Society for Gene and Cell Therapy (BSGCT), and the French Society of Cell and Gene Therapy (SFTCG) to their respective mailing lists. In addition to this mail-out, the survey was sent directly to principal investigators of phase I trials of gene therapy, whose names were identified using the clinical trial registration website ClinicalTrials.gov and whose contact details could be accessed publicly. This enabled the most directly relevant data to be obtained from researchers who had personally been involved in clinical trial decision-making. Research was conducted with the approval of the Human Research Ethics Committee of the Children's Hospital at Westmead, Sydney, NSW, Australia.

Construction and validation of the survey instrument. Survey questions were developed to investigate researchers' perceptions and assessments of risks in clinical trials (Supplementary Appendix online). Survey questions were all multiple choice, with the exception of two optional open-ended text questions (data not shown). Part A of the survey instrument collected the demographic details of respondents. Part B of the survey contained the 56 hypothetical questions about whether it would be appropriate to recruit participants to a phase I trial, and respondents were given the options of “strongly agree”, “agree”, “disagree”, and “strongly disagree”. The hypothetical clinical scenarios were designed to represent examples of real diseases which are treated in phase I trials of gene therapy. Naming of specific diseases and usage of detailed medical jargon were avoided to ensure accessibility for researchers without medical training. The numbers of scenarios and descriptions of preclinical data in Part B of the survey reflected a balance between covering a representative range of variables contributing toward the nature of the clinical scenario, and minimizing the amount of time required to complete the survey. Parts C and D of the survey instrument examined the attitudes of researchers to ethical issues about risks in clinical research and AEs, and included questions about researchers' personal experience of decision-making about the conduct of clinical trials.

The internet-based survey instrument was constructed using the open source online survey application LimeSurvey, hosted by the LimeService platform (LimeSurvey.com, Hamburg, Germany). The survey instrument was validated for readability and content validity by piloting on a representative sample of researchers involved in clinical research.

Data analysis and interpretation. Statistical analysis of the survey data was conducted using the Statistical Package for the Social Sciences software (SPSS; Version 20, IBM, Armonk, NY) and heat-maps were generated using R.³⁸ Related responses were combined, such as “agree ” and “strongly agree”, “disagree ” and “strongly disagree”, “very relevant ” and “somewhat relevant”, and “completely irrelevant ” and “somewhat irrelevant”. Trends across the whole survey cohort were analyzed using GEEs, a method for modeling binary outcomes and which allows for intra-individual correlations.^39,40,41 GEE analysis enabled examination of patterns in each individual respondent's willingness to support a clinical trial, averaged across the whole survey cohort. The clinical scenario, the type of preclinical model and professional experience in the clinical care of children were examined as variables in the GEE model. The effects of changing the clinical scenario and preclinical model were also examined individually, by fitting separate GEE models for each preclinical model and for each clinical scenario, respectively.

Cross-tabs analyses were performed to analyze the association of binary outcomes with different demographic groups, using Pearson χ² tests, Fisher's exact tests and Mantel–Haenszel odds ratios. In all analyses, a P value of <0.05 was deemed statistically significant.

Respondents were aggregated on the basis of gender (female or male), age (40 years and under or over 40 years), regional location (North American or Europe), years of research experience (10 years and under, 11–20 years or more than 20 years), professional role (scientist or clinician/clinician-scientist), and involvement in the clinical care of children (yes or no/not applicable; with and without the exclusion of nonclinicians). Responses from Australia and Asia were excluded from the analysis based on regional location, to enable investigation of cultural differences in risk perceptions in the regions with the greatest volume of clinical trial activity, North America and Europe.

SUPPLEMENTARY MATERIAL Table S1. Analysis of the effects of varying clinical scenarios for GEE models fitted individually for each of the type of preclinical evidence. Table S2. Analysis of the effects of varying preclinical evidence for GEE models fitted individually for each clinical scenario. Appendix Survey instrument

Supplementary Material

Supplementary Appendix

Survey instrument