Sample-size calculation for preclinical dose-response experiments using heterogeneous tumour models

Abstract

Background:

In preclinical radio-oncological research, local tumour control is considered the most relevant endpoint as it reflects the inactivation of cancer stem cells. Preclinical tumour-control assays may compare dose-response curves between different radiotherapy strategies, e.g., assessing additional targeted drugs and immunotherapeutic interventions, or between different radiation modalities. To mimic the biological heterogeneity of human tumour populations and to accommodate for approaches of personalized oncology, preclinical studies are increasingly performed combining larger panels of tumour models. For designing the study protocols and to obtain reliable results, prospective sample-size planning has to be developed that accounts for such heterogeneous cohorts.

Methods:

A Monte-Carlo-based method was developed to estimate the sample size of a comparative 1:1 two-arm prospective tumour-control assay. Based on repeated logistic regression analysis, pre-defined dose levels, assumptions on the dose-response curves of the included tumour models and on the dose-modifying factors (DMF), the power is calculated for a given number of animals per dose group.

Results:

Two applications are presented: (i) For a simple tumour-control assay with the head and neck squamous cell carcinoma (HNSCC) model FaDu, 10 animals would be required for each of 7 dose levels per arm to reveal a DMF of 1.25 with a power of 0.82 without drop out (total: 140 animals). (ii) In a more complex experiment combining six different lung tumour models to a heterogeneous population, 21 animals would be required for each of 11 dose levels per arm to reveal a DMF of 1.25 with a power of 0.81 without drop out (total: 462 animals). Analyzing the heterogeneous cohort reduces the required animal number by more than 50% compared to six individual tumour-control assays.

Conclusion:

An approach for estimating the required animal number for comparative tumour-control assays in a heterogeneous population is presented, allowing also the inclusion of different treatments as a personalized approach in the experimental arm. The software is publicly available and can be applied to plan comparisons of sigmoidal dose-response curves based on logistic regression.

Graphical Abstract

Introduction

The development of new or optimized cancer-treatment strategies, e.g., combining novel drugs with radio(chemo)therapy, is commonly started with several phases of preclinical research before translation into clinical trials [1–3]. The same may hold for the development and validation of novel biomarkers for treatment intervention in the context of personalized therapy of heterogeneous tumour populations, e.g., based on clinical imaging data or molecular analyses [4]. For preclinical experiments, the in-vivo tumour-control assay remains the most relevant endpoint, since it reflects the inactivation of cancer stem cells [2,3,5,6]. In dependence of the applied dose the tumour-control fraction (TCF) is assessed, which is closely related to the commonly used endpoint local control in clinical trials. However, tumour-control assays are time and cost intense and require a larger number of animals than alternative approaches, such as growth-delay studies [3,5].

Typically, tumour-control assays on mice are performed for every type of tumour model individually, see for example our previous studies on head and neck squamous cell carcinoma (HNSCC) xenografts [7,8]. The required animal number for experiments testing the additional effect of novel drugs on tumour control compared to a control group may thus easily reach more than 1000 animals if several tumour models are analysed. To reduce sample size and to mimic the situation in clinical trials, in which a heterogeneous tumour population may cause a heterogeneous population response [4,9], a tumour-control assay may include several tumour models of different radiosensitivity simultaneously. The overall dose response is then compared between the study arms based on this heterogeneous population. For the design of the corresponding study protocols, this population-based approach complicates the calculation of the required sample size, since different dose-response curves with different location and slope would be mixed and the applied drugs may show a different effect for each tumour model.

In this manuscript, we therefore present a Monte-Carlo-based approach for sample size calculation in tumour-control assays that compares dose-response curves between a control and an experimental group. The approach estimates the required animal number to reach a specific power based on given dose-modifying factors (DMF). Several tumour models can be included in the study arms to simulate heterogeneous populations. We present the application of the approach for two example cases: (i) using the HNSCC-based xenograft FaDu and (ii) modelling heterogeneous arms containing six lung-based tumour models of different radiosensitivity. For approach (ii), a substantial reduction of the required animal number is observed in comparison to six individual experiments.

Material and Methods

Tumour-control assays

In tumour-control assays, the response of individual tumours to treatment is observed until a pre-defined follow-up time is reached [10]. If the tumour did not recur during this time, it is labelled as controlled (label 1), otherwise as not controlled (label 0). This experiment is performed for a given number of individuals that are treated with different pre-defined dose levels d. The observed fraction of controlled tumours per dose level is called the tumour-control fraction, TCF(d), which typically shows a sigmoidal dependency on the dose, i.e., it is close to zero for small doses and increases to one for high doses. Common sigmoidal fit functions are based on logistic regression, log-logistic regression, or Poisson regression [11]. Logistic regression, for example, estimates the tumour control probability (TCP) by the following formula

$$TCP(d)=\frac{1}{1+\exp \left(-{\beta }_0-{\beta }_{1}d\right)}$$

with the fit coefficients β₀ and β₁. The dose to control 50% of the tumours, TCD₅₀, is calculated from the regression coefficients using TCD₅₀ = −β₀/β₁. To assess the effect of a treatment modification, e.g., by a novel drug, it can be tested if the dose-modifying factor (DMF) between the control arm and the experimental arm, here defined by DMF = TCD_50,control/TCD_50,exp., is significantly larger than 1 (null hypothesis H₀: DMF = 1, alternative hypothesis H₁: DMF > 1). Based on this test, an assumed (or minimally required) DMF, and the planned experimental parameters, the required number of animals N_a has to be calculated for the prospective study protocol.

Sample-size calculation

A Monte-Carlo simulation was developed to estimate the sample size N_a for a two-arm tumour-control study, following ideas from [12]. The algorithm repeatedly simulates the experimental outcome in-silico to determine the power at a given animal number. It requires the following input: dose levels to be used (number N_d), number N of animals per dose level of each arm, significance level α (e.g., 0.05), for every tumour model (x): relative frequency in heterogeneous cohort f_x, expected fit coefficients β_0,x and β_1,x of the dose-response curve for the control arm, expected DMF_x. The fit coefficients β_0,x and β_1,x can be estimated from the TCD_50,x and the slope of the dose-response curve of previous experiments if available for that tumour model.

The following assumptions are made by the simulation: dose response curves are described by a logistic regression for every tumour model and for the combined cohorts, the steepness of the dose response curve may differ for every tumour model but is the same in the control and experimental arm, the number of animals is the same for every dose group, dose levels for both arms are the same, assignment of mice to study arms is 1:1.

The following simulation is repeated a specified number of times (k) to estimate the statistical power for detecting a significant difference between the two study arms for the selected number N of animals per dose group: For every dose group, N tumour models are randomly selected according to their specified relative frequencies f_x. The expected tumour-control probability TCP_x(d_i) of mouse i is calculated for these models x according to their specified fit coefficients β_0,x and β_1,x at the current dose level d_i using the logistic function for the control arm. For the experimental arm, β_0,x is replaced by β_0,x/DMF_x. For every animal, a uniformly distributed random number u_i in (0,1) is drawn. If u_i < TCP_x(d_i) the tumour is considered as controlled (label 1) otherwise as not controlled (label 0), i.e., a binomial distribution with a success probability of the expected TCP is used. This information is encoded in a binary event vector 𝒆 including the simulated animals of both study arms and all dose levels (i = 1, …, 2 ∙ N ∙ N_d entries). In addition, a vector containing all dose values of both arms d and a vector encoding the arm to which the individual mice belong g (0: control, 1: experimental) is created. Based on these data, two logistic regressions are performed: The first logistic regression estimates the TCP based on the event vector as dependent and the dose vector as independent variable,

$$\mathit{T}\mathit{C}{\mathit{P}}_{\mathbf{1}}=\frac{1}{1+\exp \left(-{\beta }_{0,1}-{\beta }_{1,1}\mathit{d}\right)}.$$

The second logistic regression in addition includes the vector encoding the treatment arm as dependent variable,

$$\mathit{T}\mathit{C}{\mathit{P}}_{\mathbf{2}}=\frac{1}{1+\exp \left(-{\beta }_{0,2}-{\beta }_{1,2}\mathit{d}-{\beta }_{2,2}\mathit{g}\right)}.$$

For both models, the log-likelihood is calculated using

$$L{L}_{1/2}={\sum }_i\left(e_i\ln \left(TC{P}_{1/2,i}\right)+\left(1-e_i\right)\ln \left(1-TC{P}_{1/2,i}\right)\right).$$

The negative difference of both log-likelihoods multiplied by a factor two, −2(LL₁ − LL₂), is chi-squared distributed with one degree of freedom, which is the basis for a likelihood-ratio test. This test is used to calculate a p-value assessing if there is a significant difference between the dose-response curves of both arms for this particular simulation. The TCD₅₀ is calculated for both arms from the coefficients of the second model using $TC{D}_{50,\text{control}}=-\frac{{\beta }_{0,2}}{{\beta }_{1,2}}$ and $TC{D}_{50,\text{exp}.}=-\frac{{\beta }_{0,2}+{\beta }_{2,2}}{{\beta }_{1,2}}$, i.e., the dose modifying factor is $DMF=\frac{TC{D}_{50,\text{control }}}{TC{D}_{50,\text{exp}.}}=\frac{{\beta }_{0,2}}{{\beta }_{0,2}+{\beta }_{2,2}}$.

The described simulation is repeated k times. The power is finally obtained as the fraction of significant results, which in addition fulfill the condition that $TC{D}_{50,\text{control }}>TC{D}_{50,\text{exp}}$, i.e., only significant results in which the experimental group shows a reduced TCD₅₀ are considered.

As the final result one obtains the power of the trial for a specified number of animals per dose group N. Starting at a small N (e.g., N = 5) and increasing N until a specified power (e.g., 0.8) is reached, defines the final sample size per arm as N_𝑎 = N ∙ N_d. In addition, potential drop-out has to be considered.

The simulation was written in Python 3.8.5 using the scikit-learn package for logistic regression. A graphical user interface was generated for easy application. The software and example scripts can be downloaded at https://github.com/oncoray/powdrc.

Results

We present two applications for sample-size calculation: (i) a simple situation with one tumour model and (ii) a more complex situation with a heterogeneous population of six tumour models of different radiosensitivity.

(i) Exemplarily, we used results of previously performed experiments on the dose-response of the HNSCC xenograft FaDu in seven to 14-week-old athymic nu/nu mice that were treated with radiotherapy [7]. Parameters for the sample size calculation were: ${\beta }_{0,\text{FaDu}}=-7.2$, ${\beta }_{1,\text{FaDu}}=0.115G{y}^{-1}$, $TC{D}_{50,\text{FaDu}}=62.4\text{Gy}$, dose values [30, 40, 50, 60, 72.5, 80, 100] Gy, α = 0.05, and k = 10000. We assumed a DMF of 1.25 for a potential radio-sensitization comparing a control arm of mice treated with fractionated radiotherapy vs an experimental arm that in addition includes treatment with a hypothetical drug.

Figure 1a shows the simulated dose-response curves of both arms based on the median fit parameters of the logistic regression for N = 10 animals per dose level. The simulated median parameters $\left({\beta }_{0,\text{Sim}}=-7.42,{\beta }_{1,\text{Sim}}=0.119{\text{Gy}}^{-1},TC{D}_{50,\text{Sim}}=62.8\text{Gy},DM{F}_{\text{Sim}}=1.25\right)$ of the control arm were similar to the input values (see previous paragraph). The slopes of both curves were identical, which is an assumption of the simulation. The dependence of the power on the number of animals per dose level N is shown in Figure 1b. A power of 0.82 was reached for 10 animals per dose group. Adding 10% of drop-out leads to 12 animals per dose group, i.e., 84 animals for each arm and 168 in total. The dependence of the animal number on the DMF is shown in Figure 1c. Further validation was performed on additional HNSCC models [7,8,13].

Figure 1:

(a) Estimated tumour control probability (TCP) in dependence of dose (𝐷) for the xenograft FaDu treated with radiotherapy (control arm, Cont., solid line) and treated with radiotherapy+drug (experimental arm, Exp., dashed line) with a hypothetical dose-modifying factor (DMF) of 1.25, based on the median regression parameters. The TCD₅₀ of both arms is marked (dotted lines). Symbols represent the simulated average tumour-control fractions for the control (squares) and experimental (circles) arm. (b) The power increases with the number of animals per dose group (N). A power above 0.8 is reached for N ≥ 10 animals (dotted lines). (c) The required number of animals per dose level decreases with increasing DMF (shown for power ≥ 0.8). k = 10000 repetitions were used for all subplots.

(ii) For the second example, the overall dose response of a heterogeneous population of mice with different lung tumour models should be compared between radiotherapy and radiotherapy combined with one or potentially several experimental drugs. Such an approach could be used when a novel treatment with personalized decisions according to tumour features is evaluated. Six tumour models were considered with limited available information: H23, Calu-6, H1703, H441, H460, SW1573, hypothetically ordered from radiosensitive to radioresistant [14,15]. Since the actual TCD₅₀ was unknown for most of these models (except for H1703 with TCD₅₀ = 43.1 Gy [16]), we assumed a range of TCD₅₀ between 30 Gy and 100 Gy for the control arm, similar as reported for HNSCC-based models [7]. Since also the shapes of the dose response curves were unknown, we used 11 different dose levels: [0, 20, 30, 40, 50, 60, 70, 80, 90, 100, 120] Gy to cover a wide dose range. As the effect of the individual drug on every tumour model was unclear, we assumed a general DMF of 1.25 as a lower limit of relevance. All tumour models should be treated with the same frequency, f_x = 1/6. The estimated parameters are given in Table 1.

Table 1:

Hypothetical assumptions for the six lung tumour models used in application (ii). The dose-modifying factor (DMF) was set to 1.25 for all models, representing a potential lower limit of relevance for a situation in which no quantitative information on the expected effect of the intervention is available. TCD₅₀: dose to control 50% of tumours.

Tumour model	Coefficient β0	Coefficient β1/Gy−1	DMF	Frequency fx	TCD50/Gy
H23	−4.5	0.15	1.25	1/6	30
Calu-6	−6.0	0.15	1.25	1/6	40
H1703	−5.625	0.125	1.25	1/6	45
H441	−6.0	0.10	1.25	1/6	60
H460	−6.4	0.08	1.25	1/6	80
SW1573	−6.0	0.06	1.25	1/6	100

Figure 2a shows the assumed dose-response curves of the six tumour models for the control group. In Figure 2b, the dose-response curve of the total heterogeneous cohort is shown for both arms using the median fit parameters of the logistic regression for N = 21. The simulated DMF was 1.26. The heterogeneous cohort showed a shallower dose-response curve than the individual tumour models, caused by the combination of individual dose-response curves with different radiosensitivity [17]. The dependence of the power on the number of animals per dose level is shown in Figure 2c. A power of 0.81 was reached for 21 animals per dose group. This would give rise to a sample size of 231 per arm. Adding 10% drop-out, this number increases to 264 per arm, in total 528 animals. Figures 2d–f show additional exemplary analyses: (d) the dependence of the power on the selected dose levels, (e) the reduction in power if 3 of 6 tumour models are non-responders (DMF = 1), and (f) the dependence of the animal number on the DMF for a power of 0.8 (same DMF for all models).

Figure 2:

(a) Assumed tumour control probability (TCP) in dependence of dose (𝐷) for the tumour models H23, Calu-6, H1703, H441, H460, SW1573 (from left to right). (b) Estimated dose response for a cohort including all tumour models from (a) with the same frequency treated with radiotherapy (control arm, Cont., solid line) and treated with radiotherapy+drug(s) (experimental arm, Exp., dashed line) with a hypothetical dose-modifying factor (DMF) of 1.25 for every tumour model and drug. The TCD₅₀ of both arms is marked (dotted lines). Symbols represent the simulated average tumour-control fractions for the control (squares) and experimental (circles) arm. (c) The power increases with the number of animals per dose group (N). A power above 0.8 is reached for N ≥ 21 animals (dotted lines). (d) The distribution of dose levels influences the power (black circles: [0, 20, 30, 40, 50, 60, 70, 80, 90, 100, 120] Gy [same as (c)], gray crosses: [20, 25, 30, 35, 40, 45, 50, 60, 70, 85, 100] Gy). (e) Non-responding models reduce the power (black circles: all models with DMF = 1.25 [same as (c)], gray crosses: Calu-6, H441, SW1753 with DMF = 1, others with DMF = 1.25; gray plusses: Calu-6, H441, SW1753 with DMF = 1, others with DMF = 1.5). (f) The required sample size decreases with increasing DMF (parameters as in Table 1, dose levels as in Table 2, power ≥ 0.8). k = 10000 was used for all subplots.

Discussion

In this manuscript, we present a Monte-Carlo-based approach to estimate the required number of animals in a two-arm tumour-control assay comparing the DMF between a control arm and an experimental arm. Two applications were presented, (i) for the dose response of a single tumour model and (ii) for a heterogeneous cohort including six tumour models of different radiosensitivity.

The problem of calculating the sample size for comparing dose-response curves can typically not be solved by standard sample size software. While often a function for estimating the sample size for a binary covariate in logistic regression is available, the impact of the different dose levels is not sufficiently reflected. Therefore, several authors have proposed methods for calculating the sample size in dose-response trials, in particular for pharmaceutical studies, for example [18,19]. While these approaches can be used for the presented application (i), we developed a more general method that can assess heterogeneous cohorts of individuals with differing dose response, as presented in application (ii).

In application (ii), we used the same dose levels for the irradiation of all six tumour models irrespective of their assumed radiosensitivity. In principle, however, dose levels, for which the individual TCP is assumed to be close to 0 or close to 1 could be skipped in sample size planning and assumed to be exactly 0 or 1, respectively [12]. This would further reduce the sample size. Exemplarily, for dose levels above 60 Gy, all radiosensitive H23, Calu-6 and H1703 tumours are estimated to be cured (Figure 2a). Removing these dose levels for the three tumour lines would reduce the required animal number by 52 per arm (without drop-out). However, such a modification requires accurate estimates of the TCD₅₀ for these tumour models. If they turn out to be more radioresistant in the experiment, the study could be substantially underpowered. Therefore, adapting the used dose levels per tumour line is reasonable only if trustworthy dose-response curves for the respective lines are available before the study, at least for the control group. To further reduce the sample size, the distance between the dose levels can be chosen higher for regions in which most dose-response curves are expected to be shallow and lower for regions in which they are expected to be steep (see Figure 2d for an example). A detailed discussion on the selection of dose levels is given in [12].

More advanced study designs may include an automated adaptation of dose levels and animal number assignment during the experiment. For example, a prior function describing the initial assignment of mice to the dose levels may be continuously updated based on the available results [20]. While this procedure can be applied for assays with short-term endpoints, adaptive designs may be more challenging in radiotherapy tumour-control studies because of the long follow-up periods.

For application (ii), individual comparisons between the control and the experimental arm could be performed for every tumour model as an alternative to the presented comparison of the heterogeneous arms. For this experiment, the required total sample size of the six individual trials was estimated as 1056 animals per arm without drop-out using the same 11 dose levels (with the same assumptions as previously, Table 2). For the heterogeneous cohort, 231 animals per arm were estimated. Thus, this approach reduces the required animal number by 78%. A more realistic experiment with the six individual tumour models may exclude dose-levels for which the expected TCP is very high, i.e., where all tumours are expected to be controlled. Removing all dose levels with an expected TCP above 90% for all models individually would lead to a required sample size of 598 animals per arm without drop-out. Still the approach using the heterogeneous cohort reduces the required animal number by 61%. While the six individual dose-response experiments would reveal the TCD₅₀ and the DMF for every tumour line, the heterogeneous experiment gives one DMF for the whole cohort, which is statistically tested. The effect of the individual drugs on the individual tumour lines remains unknown and only a general approach, e.g., on personalized combined treatment can be evaluated. While subgroup analyses are possible, they may be substantially underpowered, in particular for tumour models with steep dose-response curves. Thus, the design will only lead to a positive result, if the majority of drug combinations in most tumour models is superior to the standard treatment (Figure 2e), which is a very relevant clinical question but may not help if the drug mechanisms are not well understood yet.

Table 2:

Number of animals N per dose value and arm with a power >0.8 to reveal a dose-modifying factor 1.25 for every individual lung tumour model. Used parameters are given in Table 1. Dose values were fixed at [0, 20, 30, 40, 50, 60, 70, 80, 90, 100, 120] Gy and k = 10000. Models with a steep dose-response relationship (high β₁ in Table 1) require more animals per dose level, since fewer of the fixed levels are populated with meaningful data, i.e., have a tumour-control fraction substantially larger than 0 and smaller than 1.

Tumour model	N
H23	33
Calu-6	17
H1703	16
H441	12
H460	9
SW1573	9

The presented approach is limited by its assumptions: logistic dose-response curves for every tumour model and for the combined population, same steepness for control and experimental arm, same number of animals per dose group, same dose groups for both arms, 1:1 assignment of individuals to arms. In principle, these limitations can be solved by extending the Monte-Carlo simulation code. While this may be done in the future, it would introduce additional degrees of freedom that would have to be estimated before the study. In practice, however, good estimates for these parameters are rarely available and a conservative but robust sample size planning may be preferred to a calculation with somewhat reduced sample size but based on assumptions that may not hold.

In conclusion, the presented approach for estimating the required animal number may be applied to preclinical tumour-control assays comparing the dose-response of two different treatments when a clinically relevant population is aimed to be investigated.

Highlights:

• Presentation of sample-size calculation for comparing dose-response curves

• Application to simple experiment with one tumour model

• Application to complex experiment with heterogeneous tumour models