DESIGN AND POWERING OF CYSTIC FIBROSIS CLINICAL TRIALS USING RATE OF FEV₁ DECLINE AS AN EFFICACY ENDPOINT

Abstract

Background

Rate of lung function decline (RLFD) (as FEV₁ percent predicted/yr) is a robust measure of CF therapeutic efficacy rarely used as a study endpoint, in part due to uncertainty of sample size requirements.

Methods

Sample size requirements for 1:1 randomizations to detect RLFD treatment effects from 20%-80% were assessed in Epidemiologic Study of CF (ESCF) patients. Effects of measuring FEV₁ 1-4 times per year in studies of 1- to 4-year durations were assessed in 399 patients age ≥6 years with FEV₁ ≥70%. Impacts of inclusion/exclusion based on risk factors in 2,369 ESCF patients were assessed over 1.5 years using semi-annual FEV₁ measures.

Results

Increasing study duration and exclusion of lower risk patients (e.g., no P. aeruginosa infection) both substantially reduced requirements.

Conclusions

CF RLFD studies of 1.5 years in duration appear feasible provided investigators account for the beneficial effects of subject inclusion/exclusion based on risk factors in power estimates.

INTRODUCTION

Cystic fibrosis (CF) is a life-shortening genetic disease in which 80% of deaths directly or indirectly result from loss of pulmonary function [1]. Previous CF studies have suggested that: patients who die at an earlier age experience, on average, a more rapid rate of decline of forced expiratory volume in 1 second (FEV₁) over their lifetimes [2]; a patient’s rate of FEV₁ decline can be used to estimate their age of death [3]; the presence or absence of certain risk factors can help predict future rate of decline [4]; and, chronic therapeutic interventions can slow FEV₁ decline [5–7]. For these reasons, demonstration of a reduction in the mean rate of FEV₁ decline compared with placebo has been proposed as a basis by which a CF respiratory therapy can be determined to be disease modifying [8].

Chronic respiratory therapies have become an increasingly prominent component of CF patient management today, although the most commonly used therapies have not been shown to reduce rate of FEV₁ decline, but rather to produce a sustained improvement in FEV₁ compared with placebo-treated controls [9]. Use of rate of FEV₁ decline as a prospective clinical trial endpoint remains a desirable but elusive goal for CF drug development in part because of individual variability in FEV₁ over time [8]. The effect of this variability is mitigated as study duration increases, which may partially explain why the only prospective controlled clinical trial in CF to demonstrate a significant reduction in the rate of FEV₁ decline to date has been a 4-year study of high-dose ibuprofen [5]. A 2-year study of ibuprofen demonstrated an effect on rate of forced vital capacity (FVC) decline, but did not reach significance for FEV₁ decline [10], and both a 56-week study of inhaled tobramycin and a 48-week study of inhaled hypertonic saline failed to reach primary FEV₁ rate of decline endpoints [11,12] Although challenging, it has been possible to establish a CF therapy’s impact on rate of FEV₁ decline using patient registries. The efficacy of high-dose ibuprofen has been recently confirmed using the Cystic Fibrosis Foundation (CFF) Patient Registry [6] and an effect of inhaled corticosteroids on rate of FEV₁ decline has been demonstrated using the Epidemiologic Study of Cystic Fibrosis (ESCF) [7].

These results suggest that retrospective analyses of CF patient registries might help better define design parameters for prospective randomized controlled studies that employ rate of FEV₁ decline as an efficacy endpoint. We have used data from the ESCF [13] to assess the impact of study duration, sampling frequency, inclusion/exclusion criteria, and magnitude of proposed treatment effects on study power and sample size requirements using rate of FEV₁ decline as an efficacy endpoint.

METHODS

Data were obtained from ESCF, a prospective, encounter-based, multicenter, observational study designed to evaluate the natural history of CF patients in North America from 1994 to 2005. [12] Informed consent was obtained according to local human subjects review boards. Pulmonary function test (PFT) results were reported as measured values and converted to percent predicted using reference equations from Wang et al. [14] for females through age 15 and males through age 17, and Hankinson et al. [15] at older ages.

Analyses were performed in two parts using SAS version 9.1 (SAS Institute, Inc., Cary, NC). First, the effects on sample size requirements of frequency of FEV₁ measurement and the duration of time over which measures are collected were assessed in a development population set. Next the effects of various risk factors for lung function decline [4] were assessed in a separate stratification population set using a single frequency and duration of measure selected from the development set analysis. For the development set, patients from ESCF were selected who had a clinic visit during which they were clinically stable >1 year after ESCF enrollment and within ± 1 calendar year of their 1998 birthday, were ≥6 years of age and had an FEV₁ >70% predicted at that visit, and had at least quarterly PFT results available over the subsequent 4-year time period. Only one PFT measure closest to each quarterly interval was used for analyses, with additional measures discarded. Rates of FEV₁ decline (as percent predicted/yr) and associated SD were calculated for the development set as functions of sampling frequency (ranging from quarterly to annual FEV₁ measures) and duration of measure (ranging from 1 to 4 years). Sample sizes required to retain 80% power to detect significant (p = 0.05) relative differences in rate of FEV₁ decline ranging from 20–80% in 1:1 randomized studies were estimated for each frequency/duration combination. This analysis was then used to choose a single frequency of FEV₁ measure and duration of study for subsequent stratification set analyses. The single frequency and duration of measure chosen for subsequent analyses was infrequent and short (respectively) in order to increase the number of available ESCF patients, while at the same time retaining sample sizes comparable to previous studies conducted in CF patients.

Relaxed inclusion criteria identified by development group analyses were then used to identify a group of ESCF patients large enough that sample size calculations could be conducted on subgroups stratified by risk factors for rate of FEV₁ decline (the stratification group). The stratification dataset was composed of patients from ESCF that met all of the development set inclusion criteria except that they only needed FEV₁ measures available at the selected single sampling frequency and duration of measure chosen based on the results of the development set analyses. In order to assess the impact of risk factors [4] on sample size requirements, these stratification group patients were divided by age group (6–12, 13–17, 18–24, ≥25 years of age), FEV₁ (70 to <85, 85 to <100, ≥100% predicted), weight-for-age percentile (<25th, ≥25th percentile), prior year history of P. aeruginosa culture results (all negative, any positive), and prior year history of exacerbations treated with intravenous (IV) antibiotics (IV exacerbations; 0, 1, 2, ≥3). Rates of FEV₁ change (as percent predicted/yr) and sample sizes required for 1:1 randomized studies to retain 80% power (α = 0.05) to detect effects on rate of FEV₁ decline ranging from 20–80% were calculated for stratification set subgroups.

RESULTS

A total of 399 of 32,585 ESCF patients met the inclusion criteria for the development set, with a mean age of 12.5 ± 5.7 (SD) years and mean FEV₁ of 89.8 ± 14.0% predicted (Table 1). Estimated mean rates of FEV₁ decline and associated SD decreased incrementally as the duration of measure increased from 1 year to 4 years and as frequency of measure increased from annual to quarterly (Fig. 1A and 1B). Sample sizes required to detect various treatment effects with 80% power and α = 0.05 assuming a 1:1 randomization were plotted for different frequencies of measure and durations of study (Fig. 2; Table 2). Changing the frequency of FEV₁ measurement had only a modest effect on sample sizes required to detect a given treatment effect, with quarterly measures requiring slightly higher sample sizes than other frequencies. For example, detection of a 50% treatment effect in a 4-year study using annual, semiannual, and quarterly measures required 138, 143, and 175 subjects per arm, respectively (Fig. 2A). Increasing study duration had a more dramatic effect on required sample sizes, with each increase in study duration leading to a further decrease in required sample sizes (Fig. 2B). Detection of a 50% reduction in rate of FEV₁ change using semiannual FEV₁ measures for 1 year required 315 subjects per arm, whereas increasing study duration by an additional 6 months reduced the required sample size by more than a quarter to 223 subjects per arm. Increasing study duration an additional 6 months (to 2 years) only decreased the required sample size an additional 14.8% to 190 subjects per arm (Fig. 2B). Based on these results, we selected a semiannual sampling frequency and 1.5-year study duration for subsequent analyses with the expectation that more ESCF patients would be available for analyses with these less stringent inclusion criteria.

Table 1.

Demographics of study population

Variable	Development	Stratification
Sample size, n	399	2,369
Age, yrs
Mean ± SD	12.5 ± 5.7	12.8 ± 5.9
Median	11.2	11.7
Range	34.9 (6.0–40.9)	52.0 (6.0–58.0)
Baseline FEV1, % predicted
Mean ± SD	89.8 ± 14.0	91.2 ± 14.5
Median	88.0	90.1
Range	62.7 (70.0–132.7)	72.3 (70.0–142.3)

Figure 1. Effect of frequency and duration of measure on estimated means and standard deviations of rate of FEV₁ change.

Panel A, estimated mean rates of FEV₁ change as a function of duration of measure beginning at the index visit. Bars represent standard errors (SE) around each mean. Panel B, observed standard deviations for estimated rates of FEV₁ change in Panel A. Estimates are derived using annual (black circles), semiannual (gray circles), and quarterly (white circles) FEV₁ measures.

Figure 2. Samples sizes required to detect treatment effects as functions of frequency of measure and study duration.

The number of subjects required per arm in a 1:1 randomized study (80% power, α = 0.05) to detect a given treatment effect on rate of FEV₁ change are plotted. Horizontal dotted line highlights a 50% treatment effect. Panel A, effect of collecting FEV₁ measures quarterly (light gray line), semiannually (dark gray line), and annually (black line) for a duration of 4 years. Panel B, effect of study durations of 1 year (gray dashed line), 1.5 years (black dashed line), 2 years (light gray line), 3 years (black line), and 4 years (dark gray line) using semiannual measures. Note that dark gray lines (semiannual FEV₁ measures for 4 years duration) are the same in each panel.

Table 2.

Sample sizes required to detect a 50% reduction in rate of decline as a function of study duration and measurement frequency

	Subjects required per arm*
Study Duration, yrs	Quarterly Measures	Semiannual Measures	Annual Measures
1.00	374	315	320
1.25	321	-	-
1.50	274	223	-
1.75	301	-	-
2.00	268	190	200
2.25	239	-	-
2.50	223	171	-
2.75	221	-	-
3.00	211	162	164
3.25	193	-	-
3.50	182	148	-
3.75	180	-	-
4.00	175	143	138

^*1:1 randomization, 80% power, α = 0.05

When inclusion criteria for the stratification set were relaxed to include patients having at least semiannual FEV₁ measures for 1.5 consecutive years after the index visit, a total of 2,369 patients were identified. The mean age of the stratification set was 12.8 ± 5.9 years and mean FEV₁ was 91.2 ± 14.5% predicted (Table 1). The mean rate of FEV₁ change in the stratification set over the 1.5 years from their index visit was -−4.1% predicted/yr (SD, 9.3), a smaller rate of FEV₁ change than observed under the same conditions for the development set (−5.1 ± 9.7% predicted/yr; Table 3). The stratification set was composed of substantially more patients <18 years (2,057) than ≥18 years of age (312) (Table 4). The status of specific baseline risk factors previously shown to affect rate of FEV₁ decline in CF children and adolescents [4] was not available for all patients in the stratification group, but distributions of presence or absence of risk factors among those with available data appeared to reflect demographics of the general CF population. For example, older age groups had proportionally fewer patients with FEV₁ ≥100% predicted when compared with younger groups. There was evidence of a survivor effect among the 106 patients ≥25 years of age, in that the proportion with weight– for-age ≥25th percentile was higher than that of the 18- to 24-year-old group (71.7% versus 58.3%, respectively), whereas the proportion having ≥2 IV exacerbations in the previous year was lower (24.5% versus 29.1%, respectively) (Table 4).

Table 3.

Comparison of development and stratification populations

Population	Sample size	FEV1 rate estimate§ (% predicted/yr)	Standard deviation (% predicted/yr)	Sample size per arm to detect a 50% effect§†
Development	399	−5.1	9.7	223
Stratification	2,369	−4.1	9.3	323

^§Semiannual FEV₁ measures for 1.5 yrs.

^†1:1 randomization, 80% power, α = 0.05.

Table 4.

Distribution of risk factors for decline in the stratification set

	Age range, yrs
	6–12	13–17	18–24	≥25
Sample size, n	1,441	616	206	106
FEV1 % predicted, n (%)
70 to <85	439 (30.5%)	288 (46.8%)	133 (64.6%)	74 (69.8%)
85 to <100	520 (36.1%)	210 (34.1%)	52 (25.2%)	29 (27.4%)
≥100	482 (33.4%)	118 (19.2%)	21 (10.2%)	3 (2.8%)
Weight-for-age percentile, n (%)
≥25th	803 (55.7%)	369 (59.9%)	120 (58.3%)	76 (71.7%)
<25th	637 (44.2%)	246 (39.9%)	85 (41.3%)	30 (28.3%)
P. aeruginosa culture history in the past year, n (%)
All Negative	658 (45.7%)	179 (29.1%)	32 (15.5%)	13 (12.3%)
Any Positive	685 (47.5%)	394 (64%)	157 (76.2%)	81 (76.4%)
IV exacerbations in the past year, n (%)
0	1001 (69.5%)	385 (62.5%)	93 (45.1%)	50 (47.2%)
1	274 (19%)	125 (20.3%)	53 (25.7%)	30 (28.3%)
2	95 (6.6%)	51 (8.3%)	33 (16%)	17 (16%)
≥3	71 (4.9%)	55 (8.9%)	27 (13.1%)	9 (8.5%)

Stratification by baseline FEV₁ and subject age dramatically affected estimated mean rates of FEV₁ change and corresponding sample size estimates (Fig. 3). In general, estimated sample sizes decreased as patients with higher baseline lung function were included, in all age groups studied (Fig. 3B). Inclusion of children between 6 and 12 years of age substantially increased sample sizes required to detect a 50% treatment effect when compared with sizes required for other age groups, particularly when baseline FEV₁ was <100% predicted. For example, studies of 6- to 12-year-olds with baseline FEV₁ between 85 to <100% and 70 to <85% predicted were estimated to require 1,078 and 1,378 subjects per arm, respectively (Fig. 3B). By comparison, studies conducted in 13- to 17-year-olds within the same pulmonary function categories were estimated to require only 170 and 127 subjects per arm, respectively.

Figure 3. Effect of stratification by age and pulmonary function on rate of decline and sample size requirements.

Panel A, mean rates of FEV₁ change when measured semiannually over 1.5 years for 2,369 subjects stratified by age and pulmonary function. Bars represent standard errors. Panel B, number of subjects required per arm to detect a 50% treatment effect in a 1:1 study with 80% power (α = 0.05) in the same population.

Stratification of patients by weight-for-age percentile at their index visit and by prior year history of P. aeruginosa culture positivity or IV exacerbation also affected required sample sizes (Fig. 4, Table 5). The sample size required to detect a 50% treatment effect for subjects with an index weight-for-age below the 25^th percentile was 35.6% lower than for subjects with higher weight-for-age percentiles (255 versus 396 subjects per arm). More dramatically, 68.9% fewer subjects with a history of at least 1 respiratory culture positive for P. aeruginosa in the prior year were required to detect a 50% treatment effect when compared with subjects with no positive cultures in the prior year (215 versus 691 subjects per arm). History of IV exacerbation in the prior year also affected sample sizes; a study in subjects with 1 prior IV exacerbation required 50.8% fewer subjects to detect a 50% treatment effect than a study in subjects with no exacerbations in the prior year (242 versus 492 per arm). As the number of IV exacerbations in the prior year increased, sample sizes required to detect a 50% treatment effect decreased correspondingly (Fig. 4, Table 5). Further stratification of risk factors by subject age provided even greater decreases in sample size estimates. For example, exclusion of 6- to 12-year-olds from analyses uniformly reduced sample size estimates, including a 57.1% reduction in the estimate for subjects with no history of IV exacerbations in the prior year and a 60.0% reduction for subjects with weight-for-age <25th percentile (Table 5).

Figure 4. Effects of inclusion or exclusion of other risk factors for FEV₁ decline on sample size estimates.

Sample sizes required per study arm to retain 80% power (α = 0.05) to detect a 50% change in rate of FEV₁ decline over 1.5 years using semiannual measures as a function of risk factor status. Risk factors are shown at the top separated by dashed lines. Status for each risk factor is shown at the bottom. Estimated sample sizes required per arm are shown above the bars.

Table 5.

Estimated RLFD^† and samples size requirements^§ by age and risk factors

	Age range, yrs
		≥6	≥13	≥18	≥25	6–12	13–17	18–24
Weight-for-age percentile
≥25th	FEV1 change (% pred/yr)	−3.5	−5.0	−4.7	−3.7	−2.5	−5.2	−5.4
	Sample size	396	192	170	182	787	201	158
<25th	FEV1 change (% pred/yr)	−4.9	−6.9	−7.0	−10.7	−3.8	−6.8	−5.7
	Sample size	255	102	93	58	464	106	109
P. aeruginosa culture history in the past year
All negative	FEV1 change (% pred/yr)	−2.7	−4.8	−4.4	−2.5	−2.0	−4.9	−5.1
	Sample size	691	236	199	499	1208	243	153
Any positive	FEV1 change (% pred/yr)	−5.2	−6.2	−5.8	−6.2	−4.3	−6.4	−5.6
	Sample size	215	124	127	110	364	122	137
IV exacerbations in the past year
0	FEV1 change (% pred/yr)	−3.1	−4.5	−3.7	−3.8	−2.4	−4.7	−3.7
	Sample size	492	211	205	228	852	210	194
1	FEV1 change (% pred/yr)	−4.9	−5.9	−5.8	−6.6	−4.1	−6.1	−5.3
	Sample size	242	134	103	106	386	153	100
2	FEV1 change (% pred/yr)	−5.8	−7.9	−7.7	−8.7	−3.6	−8.1	−7.3
	Sample size	172	73	85	60	519	78	101
≥3	FEV1 change (% pred/yr)	−9.0	−10.3	−9.0	−7.5	−7.4	−11.1	−9.6
	Sample size	98	68	94	108	162	56	92

^†derived from 1.5 years of semiannual measures

^§80% power to detect a 50% change in rate of FEV₁ change over 1.5 years with semiannual FEV₁ measure, α = 0.05, in CF subjects with FEV₁ ≥ 70% predicted.

DISCUSSION

Demonstration of a reduction in the mean rate of FEV₁ decline compared with placebo has been proposed as a basis by which a CF respiratory therapy can be determined to be disease modifying [8]. This standard has been met in a prospective clinical trial by only 1 chronic CF therapy [5,10]. For other (more commonly used) CF respiratory therapies, claims of a pulmonary function benefit have been based on sustained improvement in FEV₁ in treated subjects compared with subjects receiving placebo, not rate of decline [16–18]. Although sustained FEV₁ improvement is an endpoint that can be reached in studies of shorter duration, sustained improvement does not necessarily lead to a decreased rate of FEV₁ decline [8]. In addition, recent and ongoing shifts in the demographics of the CF population have made it a more challenging endpoint to execute than in the past [9].

We have used ESCF data to evaluate important factors in the design and powering of CF studies that use rate of FEV₁ decline as an efficacy endpoint. Our analysis complements a previous regression analysis estimating future rate of FEV₁ decline in children and adolescents with CF as a function of the presence or absence of specific risk factors [4]. We have focused on CF patients with a baseline FEV₁ ≥70% predicted because these patients have been shown to be at the greatest risk for FEV₁ decline in the near future [4].

For simplicity, we chose to examine the effects of risk factors for lung function decline on sample size requirements using a single study duration (1.5 years) and frequency of FEV₁ measure (semi-annual). Although best-suited for our retrospective analysis, these choices will not necessarily be ideal for future studies. For instance, although the choice of semi-annual sampling allowed us to increase the number of available ESCF patients, quarterly assessments are likely to be necessary in future controlled clinical trials to ensure collection of additional efficacy and safety data. For this reason, the specific sample sizes arising from our analyses are perhaps less important than suggestions of how study designs and population selection can increase the power of studies using rate of FEV₁ decline as an endpoint. Increasing the duration (and to a lesser extent, the frequency) of measure reduces both the observed rate of decline (Figure 1A) and its associated variance (Figure 1B), probably by regression to the mean. This produces opposing effects on sample size requirements, with decreasing rates of FEV₁ change requiring relatively more subjects to detect a given treatment effect and lowering variance having the opposite effect. Our results suggest that the benefit of reduced variance eclipses the disadvantage of a lower rate estimates as study duration increases, with studies of 1.5 years of sufficient duration to reduce variance to levels where reasonable sample sizes can be chosen while retaining 80% power to detect a treatment difference.

Stratification of study populations by exclusion of subjects at lower risk of immediate FEV₁ decline (e.g., subjects 6–12 years old or without specific risk factors) further reduces sample size requirements by both increasing the expected mean rate of FEV₁ decline and decreasing the variance associated with these rates. It is important to note that conclusions drawn for certain subpopulations have been made with relatively small numbers of patients. For example, few patients that were ≥18 years of age with FEV₁ ≥100% predicted were available for this analysis (Table 1).

Our results suggest that two previous randomized controlled 1-year-long studies of CF respiratory therapies appear to have been substantially underpowered to reach their primary FEV₁ rate of decline endpoints, primarily due to short trial durations. The open-label study of tobramycin inhalation solution in CF children with early stage lung disease [11], in which 63% of subjects were between 6 and 10 years of age, likely would have required nearly a thousand subjects (as opposed to the 181 enrolled when the study was halted) to retain adequate power to detect a difference in rate of FEV₁ decline over the study period. Similarly, the placebo-controlled study of hypertonic saline [12], which included a substantial number of subjects with FEV₁ below 70% predicted (mean, 74% predicted; SD, 21% predicted; range 40–132% predicted) was also underpowered with only 164 enrolled subjects. However, retrospective analyses of the powering of previous CF studies using rate of lung function decline should be considered in the context of how changes in management and demographics may have altered rates of FEV₁ decline in the population. For instance, the placebo rate of decline in the high-dose ibuprofen study over 4 years was –3.60 % predicted/yr [5], while the corresponding rate from our ESCF development set was predicted to be –2.42 % predicted/yr. This one-third reduction in decline rate has a dramatic impact on sample size estimates: our analyses suggest that over 270 subjects per arm (Figure 2B) would be required from our development set to detect the 40% reduction in rate of decline observed in the ibuprofen trial, while a little over 40 subjects per arm successfully demonstrated the effect in the actual study. Interestingly, investigators designing a 2-year high-dose ibuprofen study overestimated their subsequently observed placebo rate of decline (−2.7% predicted/yr) [10]. The feasibility of increasing power in a study using rate of FEV₁ decline as an endpoint by including subjects at higher risk for decline should be considered in the context of the overall distribution of CF patients in the population (Table 6). For example, we report in our analysis that only 56 subjects per arm ages 13 to 17 years old, with FEV₁ ≥70% predicted and ≥3 IV exacerbations in the previous year, would have been required to retain 80% power to detect a 50% reduction in rate of FEV₁ decline over 1.5 years (Table 5). However, only 224 of the patients reported in the CFF Patient Registry in 2008 [1] had these characteristics (Table 6). Thus, a study limited to this population would require enrollment of over half of all eligible patients in the US, with allowances for anticipated dropouts further increasing requirements. The challenge going forward will be to account for the prevalence of the most potentially useful CF subpopulations when designing FEV₁ rate of decline studies intended to have increased power.

Table 6.

Distribution of CF patients with FEV₁ ≥70% predicted followed in the 2008 Cystic Fibrosis Foundation Patient Registry [1]

	Age range§, yrs
	≥6	6–12	13–17	18–24	≥25
Patients, n	12,972	4,992	3,300	2,427	2,253
FEV1 % predicted * , n
70 to <85	5,108	1,267	1,266	1,288	1,287
85 to <100	4,281	1,681	1,129	786	685
≥100	3,583	2,044	905	353	281
Weight-for-age percentile † , n
≥25th	8,910	3,234	2,232	1,633	1,811
<25th	4,060	1,758	1,068	794	440
P. aeruginosa culture history in 2007, n
All negative	3,143	476	802	962	903
Any positive	8,478	4,263	2,284	1,151	780
IV exacerbations in 2007, n
0	9,010	3,657	2,097	1,575	1,681
1	2,527	956	714	502	355
2	851	261	265	194	131
≥3	584	118	224	156	86

^§Age as of Jan 1, 2008.

^*First available FEV₁ measure in 2008.

^†First available weight-for-age percentile measure in 2008.