Study Design Implications of Death and Hospitalization as End Points in Idiopathic Pulmonary Fibrosis

Abstract

BACKGROUND:

The feasibility of an interventional clinical trial in idiopathic pulmonary fibrosis (IPF) using death and hospitalization as primary end points is an area of uncertainty. Using data from a large well-characterized clinical trial population, this article aims to illustrate the impact of cohort enrichment and study duration on sample size requirements for IPF clinical trials in which death alone or death plus hospitalization serve as the primary end point.

METHODS:

Event rate estimates for death and hospitalization were determined from patients enrolled in National Institutes of Health-sponsored IPF Clinical Research Network clinical trials. Standard equations were applied to estimate the total sample size required for varying gender, age, and pulmonary function (GAP) stage-based cohorts.

RESULTS:

Risk estimates for death and hospitalization in the clinical trial cohort were substantially lower than those published. An IPF trial with death as its primary end point enrolling subjects designated as GAP stage 1 and 2 over 1 year with a minimum follow-up of 1 year would require an estimated 7,986 subjects to achieve 90% power for a hazard ratio of 0.70. Alternatively, an IPF trial with death plus hospitalization as its primary end point enrolling subjects with GAP stage 2 and 3 over 2 years with a minimum follow-up of 1 year would require an estimated 794 subjects for the same power and hazard ratio.

CONCLUSIONS:

Study design decisions, in particular cohort enrichment strategies, have a substantial impact on sample size requirements for IPF clinical trials using time-to-event primary end points such as death and death plus hospitalization.

Idiopathic pulmonary fibrosis (IPF) is a chronic, progressive, and ultimately fatal disease.¹ Because of this, there has been an increasing number of clinical trials testing potential therapies and an increasing focus on the choice of end points used to assess efficacy.^2‐4 One of the central areas of uncertainty in IPF clinical trial design is the feasibility of using mortality and hospitalization as primary end points, in particular, the number of subjects required for adequate statistical power.⁵

The impact of cohort selection on the incidence of death and other clinical events, such as hospitalization, in clinical trial populations is an important, undercharacterized issue in IPF clinical trial design. Median survival in patients with IPF enrolled in longitudinal cohorts has been largely reported as 3 years from the time of diagnosis,^6‐8 but clinical trials of patients with IPF have demonstrated substantially lower numbers of deaths than this estimate would predict.^9‐13

Data from tertiary care cohorts provide a framework for cohort selection in IPF based on risk of death.¹⁴ Patients can be classified into three stages (gender, age, and pulmonary function [GAP] stages) with varying risks of death over time using the baseline clinical variables GAP (FVC and diffusing capacity of lung for carbon monoxide). This article uses the GAP construct to provide model-based estimates of sample size requirements for IPF clinical trials in which time to death and time to death plus hospitalization are the primary end points, with event rates for death and hospitalization determined from a large, well-defined cohort of patients with IPF enrolled in IPF Clinical Research Network (IPFnet) clinical trials.

Materials and Methods

Cohort Description and Enrichment Strategies

The study cohort comprised patients enrolled in the following IPFnet clinical trials: STEP-IPF (Sildenafil Trial of Exercise Performance in Idiopathic Pulmonary Fibrosis), ACE-IPF (Anticoagulant Effectiveness in Idiopathic Pulmonary Fibrosis), and PANTHER-IPF (Prednisone, Azathioprine, and N-acetylcysteine: a Study That Evaluates Response in Idiopathic Pulmonary Fibrosis).^15‐17 Patients randomized to warfarin in ACE-IPF or the three-drug regimen (prednisone, azathioprine, acetylcysteine) in PANTHER-IPF were excluded due to harm from these therapies. All patients were given a diagnosis of IPF according to consensus criteria at the time the studies were conducted.¹⁸ All patients provided informed consent for research participation. The research described in this article was exempt from institutional review board review because of its use of existing anonymous clinical data.

Patients were stratified into three groups using the GAP model.¹⁴ Briefly, points were assigned to each patient for sex (female = 0, male = 1), age at enrollment (≤ 60 years = 0, 61-65 years = 1, > 65 years = 2), enrollment FVC % predicted (> 75% = 0, 50%-75% = 1, < 50% = 2), and enrollment diffusing capacity of lung for carbon monoxide % predicted (> 55% = 0, 36%-55% = 1, ≤ 35% = 2, unable to perform = 3) based on risk profiles, with the sum determining the GAP stage (≤ 3 = GAP stage 1, 4-5 = GAP stage 2, ≥ 6 = GAP stage 3).

End Point Definition

Time from randomization to death and time from randomization to hospitalization (ie, hospital admission) were collected prospectively in the parent IPFnet clinical trials as components of the primary end point (ACE-IPF) or as secondary end points (STEP-IPF and PANTHER-IPF). The IPFnet investigators prospectively adjudicated all nonelective hospitalizations for specific cause of hospitalization. The two primary end points used for the current analyses were all-cause death and death plus all-cause hospitalization, where included hospitalizations were in patients who did not also die during the study period.

Statistical Methods

Event rates at 1 year were estimated for the death and death plus hospitalization end points based on IPFnet data and were assumed to have a constant hazard over time (exponential distribution). Because there were a limited number of deaths in patients with GAP stage 1 from which to derive an estimate, we estimated the event rate for this cohort by taking the published event rates¹⁴ and reducing them by the overall average difference between IPFnet and published rates. Standard equations were applied to estimate sample size for clinical trials of varying subject characteristics, effect size, and study duration using nQuery Advisor 7.0 software (Statistical Solutions). Sample sizes reported are the total sample size required for a two-arm study using a 1:1 allocation ratio. For illustrative purposes, selected subject cohorts were analyzed as follows: only patients with GAP stage 1; only patients with GAP stage 2; only patients with GAP stage 3; an equal mix of patients with GAP stage 1 and 2; an equal mix of patients with GAP stage 2 and 3; and an equal mix of patients with GAP stage 1, 2, and 3. Effect size was varied using hazard ratios ranging from 0.50 to 0.80. Conventional type I and type II error rates were included (two-sided α = 0.05, β = 0.10), and a 5% loss-to-follow-up rate per year was assumed based on what occurred in the IPFnet clinical trials. Enrollment was assumed to be uniform over either 1 year or 2 years, and the final patients in either case was followed for an additional year.

Results

Cohort Description

The study cohort comprised 517 patients enrolled in the IPFnet clinical trials (Table 1). These patients were older, primarily men, and primarily former smokers. Patients with GAP stage 1 comprised 26.3% of the cohort; GAP stage 2, 43.9%; and GAP stage 3, 29.8%. Twenty-seven deaths and 77 hospitalizations were observed in the study cohort over a total follow-up time of 22,135 weeks.

TABLE 1 ] .

Cohort Characteristics

Characteristic	Value
No. patients	517
Age, y	68 ± 8.5
Female sex	104 (20)
Former or current smoker	327 (63)
BMI, kg/m2	29.3 (26.7-32.8)
Comorbidities
CAD	58 (11)
Emphysema/chronic bronchitis	23 (4)
Diabetes	56 (11)
GERD	149 (29)
Baseline physiology
FVC, %	65.3 ± 16.8
FEV1, %	75.3 ± 14.1
FEV1/FVC ratio	0.77 ± 0.02
Dlco, %	37.2 ± 13.5
Baseline 6-min walk distance, m	322 ± 128
Baseline oxygen use	96 (18.6)
Surgical lung biopsy	275 (53)
GAP stage
1	136 (26.3)
2	227 (43.9)
3	154 (29.8)

Data are presented as mean ± SD, No. (%), and median (range). CAD = coronary artery disease; Dlco = diffusing capacity of lung for carbon monoxide; GAP = gender, age, pulmonary function; GERD = gastroesophageal reflux disease.

Estimated Rates of Death and Death Plus Hospitalization

Estimated rates of death and death plus hospitalization from IPFnet data by individual and combined GAP stages at 6 and 12 months are shown in Table 2. Estimated event rates for the IPFnet-based cohort were substantially lower for each GAP stage than those published for tertiary care-based cohorts (Fig 1).

TABLE 2 ] .

Estimated Rates of Death and Death or Hospitalization by GAP Stage

	Event Rates, %
Cohort/End Point	6 Mo	12 Mo
GAP stage 1
Death	0.0	0.0
Death or hospitalization	1.5	7.3
GAP stage 2
Death	3.3	4.8
Death or hospitalization	12.4	18.5
GAP stage 3
Death	10.4	20.3
Death or hospitalization	34.4	41.9

Data represent Kaplan-Meier event rates. See Table 1 legend for expansion of abbreviation.

Figure 1 –

Estimated event rates by GAP stage for the IPFnet cohort compared with published results. The GAP staging system effectively risk stratified patients, but overall rates of death were substantially lower than published rates for tertiary care cohorts.¹⁴ GAP = gender, age, pulmonary function; IPFnet = Idiopathic Pulmonary Fibrosis Clinical Research Network.

Sample Size Estimates by Cohort and Study Duration

Sample size estimates for death and death plus hospitalization for selected GAP-based subject cohorts, hazard ratios, and study durations are shown in Tables 3 and 4. A trial with death as its primary end point enrolling subjects with GAP stage 1 and 2 over 1 year with a minimum follow-up of 1 year (total study duration, 2 years; median follow-up, approximately 1.5 years) would require an estimated sample size of 7,986 to adequately power for a hazard ratio of 0.70 (Fig 2A). A trial with death plus hospitalization as its primary end point enrolling subjects with GAP stage 2 and 3 over 2 years with a minimum follow-up of 1 year (total study duration, 3 years; median follow-up, approximately 2 years) would require an estimated sample size of 794 for the same power and hazard ratio (Fig 2B).

TABLE 3 ] .

Sample Size Estimates for Selected Cohorts in a Two-Year Study

	Sample Size Estimate (90% Power)
Cohort/End Point	HR = 0.50	HR = 0.60	HR = 0.70	HR = 0.80
GAP stage 1
Death	4,548	7,454	13,938	33,024
Death or hospitalization	1,302	2,140	4,014	9,532
GAP stage 2
Death	1,820	2,988	5,596	13,280
Death or hospitalization	456	758	1,430	3,420
GAP stage 3
Death	456	758	1,430	3,420
Death or hospitalization	230	386	738	1,784
GAP stages 1 and 2
Death	2,602	4,268	7,986	18,936
Death or hospitalization	676	1,116	2,098	5,000
GAP stages 2 and 3
Death	730	1,204	2,264	5,390
Death or hospitalization	306	510	968	2,328
GAP stages 1, 2, and 3
Death	1,012	1,666	3,128	7,436
Death or hospitalization	410	680	1,286	3,076

To achieve 90% power with a two-sided type I error of 0.05, an HR of 0.50 would require 87 events; an HR of 0.60, 161 events; an HR of 0.70, 330 events; and an HR of 0.80, 844 events. Samples sizes reported are the total sample sizes required for a two-arm study using a 1:1 allocation ratio, assuming uniform enrollment over 1 y followed by a minimum of 1-y follow-up for the last patient enrolled. One-year death rates for the control group are assumed to be 2%, 5%, and 20% for GAP stages 1, 2, and 3, respectively. One-year death or hospitalization rates for the control group are assumed to be 7%, 20%, and 40% for GAP stages 1, 2, and 3, respectively. HR = hazard ratio. See Table 1 legend for expansion of other abbreviation.

TABLE 4 ] .

Sample Size Estimates for Selected Cohorts in a Three-Year Study

	Sample Size Estimate (90% Power)
Cohort/End Point	HR = 0.50	HR = 0.60	HR = 0.70	HR = 0.80
GAP stage 1
Death	3,476	5,700	10,662	25,274
Death or hospitalization	1,006	1,656	3,108	7,390
GAP stage 2
Death	1,400	2,300	4,314	10,244
Death or hospitalization	362	604	1,144	2,746
GAP stage 3
Death	362	604	1,144	2,746
Death or hospitalization	190	324	622	1,514
GAP stages 1 and 2
Death	1,994	3,274	6,132	14,548
Death or hospitalization	528	876	1,652	3,946
GAP stages 2 and 3
Death	570	942	1,776	4,242
Death or hospitalization	248	416	794	1,920
GAP stages 1, 2, and 3
Death	784	1,294	2,434	5,796
Death or hospitalization	326	626	1,034	2,486

To achieve 90% power with a two-sided type I error of 0.05, an HR of 0.50 would require 87 events; an HR of 0.60, 161 events; an HR of 0.70, 330 events; and an HR of 0.80, 844 events. Samples sizes reported are total sample sizes required for a two-arm study using a 1:1 allocation ratio, assuming uniform enrollment over a period of 2 y followed by a minimum of 1-y follow-up for the last patient enrolled. One-year death rates for the control group are assumed to be 2%, 5%, and 20% for GAP stages 1, 2, and 3, respectively. One-year death or hospitalization rates for the control group are assumed to be 7%, 20%, and 40% for GAP stages 1, 2, and 3, respectively. See Table 1 and 3 legends for expansion of abbreviations.

Figure 2 –

Relationship among end point choice, cohort selection, hazard ratio, study duration, and sample size. A, Two representative clinical trials with death as the primary end point; the first enrolls an equal number of subjects with GAP stage 1 and 2 over 1 y with a minimum follow-up of 1 year (total study duration, 2 y; median follow-up, approximately 1.5 y) and the second enrolls an equal number of subjects with GAP stage 2 and 3 over 2 y with a minimum follow-up of 1 year (total study duration, 3 years; median follow-up, approximately 2 y). B, The same two representative clinical trials with death plus hospitalization as the primary end point. See Figure 1 legend for expansion of abbreviations.

Discussion

The results demonstrate that study design decisions, particularly those regarding cohort enrichment strategies, have a substantial impact on the sample size requirements for IPF clinical trials using clinical end points. A clinical trial with death plus hospitalization as the primary end point in patients at low risk of death (ie, GAP stage 1) would require approximately five times the number of subjects as the same trial in patients with IPF and a high risk of death (ie, GAP stage 3).

For most potential sponsors of late-stage clinical trials in IPF, the primary concern with death and hospitalization as end points is feasibility. This is particularly true for mortality-based studies (ie, studies with a primary end point of time to death). A recent analysis of data from 622 patients with relatively preserved pulmonary function enrolled in industry trials conducted by InterMune found that a mortality-based clinical trial in this patient population would require several thousand patients.⁵ The present results are consistent with these findings; a mortality-based clinical trial enrolling patients with GAP stage 1 and 2 with similar assumptions would require subject numbers much greater than any trial performed in IPF to date. However, the present data demonstrate that a mortality-based trial enrolling patients with more severe physiologic impairment (eg, GAP stages 2 and 3) would reduce the sample size requirements by > 70%. Ultimately, the feasibility of mortality-based studies depends on the sponsor’s available resources and priorities.

One potential way to improve feasibility in a mortality-based IPF clinical trial would be to include a second clinical event, such as hospitalization, in the primary end point. Including hospitalization along with death as a composite end point reduces the sample size estimate by an additional 50% to 75%, depending on the cohorts being enrolled. The biggest impact of hospitalization on sample size in the present study was seen in patients with GAP stage 2, where there were three or four hospitalizations for every death. The addition of hospitalization to death in a composite end point has advantages and disadvantages that have been reviewed elsewhere.^2,5 The primary strength has been that hospitalization (in particular, nonelective hospitalization) is almost always of clinical significance to the patient and is associated with a high risk of subsequent mortality. The primary concern has been its lack of precision (in particular, for all-cause hospitalization that arguably includes events not reflecting progression or complications of the underlying disease) and the regional variation in clinical criteria for hospital admission. Unfortunately, analysis using potentially more-precise measures of hospitalization (eg, respiratory hospitalization) is not possible because this was not prospectively determined. The impact of other potential end points as components of a mortality-based composite (eg, change in FVC, change in 6-min walk distance) on sample size and their strengths and weaknesses were not assessed in this study.

Study duration had less of an impact on sample size requirements in the present model. For example, a trial enrolling patients with GAP stage 2 and 3 with death plus hospitalization as its primary end point and a median of approximately 1.5 years of follow-up (1 year for enrollment and a minimum of 1 year of follow-up) would require 968 patients for 90% power to detect a hazard ratio of 0.7. By increasing the median follow-up to approximately 2 years (2 years for enrollment and a minimum of 1 year of follow-up), this number decreases to 794 (an 18% reduction in sample size). These estimates assume a constant hazard of death over time; if rates of death and hospitalization increase (or decrease) in later months and years of the study, the impact of study duration on sample size requirements would change.

The observed event rates by GAP stage for the IPFnet cohort used in the present study demonstrate another important consideration. Data generated from general population and tertiary care-based cohorts may not accurately inform what is seen in the clinical trial population. Across all three GAP stages, there was a marked reduction in the 1-year mortality estimate for the IPFnet clinical trial population compared with published estimates. The reason for this difference is unknown but may reflect differences between the average patients seen at tertiary centers and those enrolled in clinical trials (eg, differences in diagnostic approaches, natural histories, overall health). Sponsors planning future clinical trials in IPF should focus on epidemiologic data from representative clinical trial cohorts whenever possible.

This study is not intended to give precise estimates of sample size requirements for clinical trialists, and it is not intended to define whether the end points of death or death plus hospitalization are feasible for any individual study sponsor. It also does not comment on the implications of study design decisions for cost, which is another major determinant of feasibility. However, we hope that this article provides clinical trialists with a resource for data-driven decision-making regarding the practical implications of study design decisions and illustrates how cohort enrichment strategies and other study design decisions have a major impact on sample size requirements. The choice of primary end point ultimately depends on other factors in addition to those discussed in this article that are specific to the intervention, sponsor, and regulatory agencies involved.

ABBREVIATIONS

ACE-IPF

Anticoagulant Effectiveness in Idiopathic Pulmonary Fibrosis

GAP

gender, age, pulmonary function

IPF

idiopathic pulmonary fibrosis

IPFnet

Idiopathic Pulmonary Fibrosis Clinical Research Network

PANTHER-IPF

Prednisone, Azathioprine, and N-acetylcysteine: a Study That Evaluates Response in Idiopathic Pulmonary Fibrosis