Application of Multiple Event Analysis as an alternative approach to studying pulmonary exacerbations as an outcome measure

Abstract

Background.

Pulmonary exacerbations (PEx) are important contributors to morbidity and mortality in cystic fibrosis (CF). Understanding risk factors for PEx is critical to improve treatment; pulmonary exacerbations also serve as an important outcome in CF clinical trials. Current risk estimates generally only evaluate time to the first PEx. Methods accounting for multiple exacerbations during the observation period could provide more power to detect significant risk factors.

Methods.

The Early Pseudomonas Infection Control (EPIC) Observational Study enrolled participants between 2004 and 2006 who were ≤ 12 years of age and negative for Pseudomonas aeruginosa. First and multiple event analyses were used to investigate risk factors for pulmonary exacerbations.

Results.

We evaluated a total of 5,129 PEx from 1,734 CF patients in the EPIC study. Multiple event analysis identified 2 more factors associated with occurrence of PEx compared to first event analysis. After adjusting for multiple factors, the following were associated with higher occurrence of PExs: female gender, older age at enrollment, household cigarette smoke exposure, increased cough at the most recent encounter, having used antibiotics since the previous encounter, a positive culture for any CF organism at the most recent encounter, and having had a PEx in the last 30 days.

Conclusions.

Multiple event analyses use all PEx events and may identify more risk factors for PEx than analysis of time to first PEx. We have provided an example of how to apply this type of analysis and how to interpret estimates in the context of the EPIC study.

Introduction

Pulmonary exacerbations (PEx), episodes of acute worsening of respiratory symptoms, are a leading cause of morbidity and mortality in cystic fibrosis (CF) and are important endpoints in CF clinical trials [1-6]. Recurrent PEx are associated with long term decline in lung function and shortened survival [1,3,7]. In addition, approximately 25% of patients fail to return to baseline lung function following a PEx even with aggressive treatment [8,9]. Despite their importance in CF, contributing factors to PEx, especially in children, are not fully understood. Pulmonary exacerbation outcomes are events with corresponding times of occurrence recorded, offering an opportunity to gain a more complete understanding of how these events evolve over time.

Although many studies have investigated PEx as an outcome in CF, most have analyzed only the time to the first event or a summary count of the events, such as the rate of PExs over a specified follow-up time. [10-16] This oversimplification results in a loss of information by ignoring subsequent events or the time course of the events[17]. The use of only first event or summary counts often involves an assumption that the risk of an event is constant over time (constant hazard assumption). Studies in COPD[18] and CF [19,20] have shown that a previous PEx event was a significant predictor for a subsequent event, thus violating the constant hazard assumption. Methods that capitalize on the occurrence of multiple PEx events and relax the constant hazard assumption have successfully been applied in CF, [21-23] but have been restricted to the statistical literature and have not yet been utilized to study PEx in children with CF.

Analysis of recurrent multiple events has been described in a few sources [24,25]. Advantages of analyzing all events include prediction of new events based on previous information, increased power, and ability to describe the progression of occurrence over time [17,26]. A fundamental concept in the analysis of recurrent multiple events is the instantaneous probability of occurrence of a new event given the history of events and covariates up to the given time; this is known as the intensity or hazard, similar to the hazard of death in survival analysis. The analysis of multiple events is similar to survival analysis, except that in the multiple event case, an individual is still at risk even after the occurrence of an event, whereas in a survival analysis an individual is no longer at risk once the event (e.g. first PEx) has been observed. Even though the two analyses estimate different quantities (hazard of the event in First Event Analysis, FEA, and hazard of a new event in Multiple Event Analysis, MEA), the power to detect associations with risk factors may be compared. It has been shown that multiple event analysis, that models the total rate of events over the entire follow-up, has on average more power for detecting effects compared to the first event analysis [26,27]. Our objective was to compare FEA and MEA for evaluating risk factors for pulmonary exacerbations in the Early Pseudomonas Infection Control (EPIC) Observational study.

Methods

Study Population

The EPIC Observational study, is a multicenter longitudinal observational study that enrolled children with a confirmed diagnosis of CF ≤12 years of age at 59 centers with respiratory cultures negative for Pseudomonas aeruginosa (either never infected or culture negative for at least 2 years). The design of the EPIC study has been previously reported [28,29]. EPIC enrolled 1,797 participants between 2004 and 2006 and included both study-specific and CF Patient Registry data. The annual EPIC family survey queried parents/guardians on time-varying factors such as influenza vaccination, exposure to cigarette smoke in the household, and hot tub and swimming pool use in previous year. Other information collected at every encounter included cold and cough symptoms in the previous week, use of oral antibiotics since previous visit, and whether a respiratory culture was positive for any CF related microorganism (the latter from the CF Registry). The data cut-off for the current analysis was December 2013.

Pulmonary Exacerbation (PEx) Definition

Pulmonary exacerbations, characterized by intravenous (IV) antibiotic treatment in the hospital or at home, [30,31] were captured from the CF Registry. The length of each PEx was defined based on the antibiotic treatment period. PExs within less than seven days of completing prior treatment were considered the same exacerbation [20,21] and PEx lasting more than 90 days were excluded from the analysis.

Statistical Analysis

Descriptive statistics were summarized for characteristics at enrollment. A counting process format of the data was used to fit the models. Given that individuals are not at risk of another exacerbation during the occurrence (length) of a PEx, these time periods were removed from the risk set. The time varying variables were assumed constant over the interval of time until the next follow-up. Cox regression analysis with time varying covariates was conducted for the first PEx analysis, and an Andersen-Gill model, which requires the proportional hazards assumption and is similar to the Cox model but for multiple events [24], was used for the multiple event analysis. The interpretation of regression coefficients is similar between the two models, except in the MEA, the hazard ratio refers to the hazard of experiencing a subsequent (or new) PEx whereas in the FEA it refers to the hazard of experiencing the first PEx. Variables were considered one at a time while adjusting for age at enrollment for both first and multiple event analyses. Finally, a multiple event model was constructed adjusting for all important variables, as identified by the p-value (<0.2) from the individual models. The cumulative mean function of events was calculated for key covariates including F508del CFTR-mutation. The cumulative mean function of events estimates the sum of events at a given time; for instance, at the end of the follow-up it provides the mean total number of events expected per individual categorized by F508del or other risk factor. The shape of the cumulative mean function of events provides a sense of the rate at which events are accumulated through time. The analyses were conducted using R (coxph function[32]). Additional description of the models as well as R code is provided in the supplement. The recurrent event model we use here is slightly different to the model that was compared to the first event analysis previously reported in the literature [26] as it models the timing of the events not the rate. [24,33] To provide evidence of the comparison of FEA versus MEA to detect risk factors, we have included in the supplementary material, a simulation of data to assess power under the MEA where we fit both MEA and FEA.

Results

Of the 1,772 children enrolled in the EPIC OBS study with PEx data, we identified 1,734 individuals with usable data; 38 individuals were excluded because of missing, after imputation by last observation carried forward, in the time-varying covariates (27), or having an either single EPIC annual survey or single EPIC encounter (11) (Figure 1). Out of the 1,734 individuals, 1.5% (26) had 1 EPIC survey, 4.8% (84) had 2, 4.4% (76) had 3, and the remaining 89.3% (1548) had at least 4 annual EPIC surveys. A total of 5,129 PEx in 788 individuals were identified after eliminating five events with length of more than 90 days.

Figure 1. Selection of Study Cohort.

Data were obtained from the Early Pseudomonas Infection Control Observational Study (EPIC Obs) with data cut off December 2013.

*Exclusion occurs after last carried forward imputation has been used for time-varying covariates.

About half of the cohort was female (49%), most were Caucasian (95%), the majority were F508del homozygous (51%), and the median weight percentile at enrollment was 36.7 (IQR: 16.5-63.4) (Table 1). The median age at enrollment was 5.8 (IQR: 3-8.8) years, and median length of follow-up 7.8 (IQR: 6.3-8.3) years. A total of 788 children experienced at least one PEx event during follow-up (45%), and 75% of the entire cohort experienced 2 or fewer PExs during the follow-up period.

Table 1:

Characteristics of the cohort at enrollment and follow-up (N=1734).

Characteristic		% (N) or Median (IQR)
Gender	Female	49% (854)
Race	Caucasian	95% (1648)
Ethnicity 1	Non-Hispanic	91.9% (1593)
	Unknown	4% (70)
CFTR mutations	Two alleles F508del	51% (890)
	One allele F508del	39% (669)
	Other or unknown	10% (175)
Weight percentile		36.7 (16.5-63.4)
Follow-up, years		7.8 (6.3-8.3)
Occurrence of one or more PExs during follow-up		45% (788)
Number of PExs recorded during follow-up		0 (0-2)
PEx annual rate during follow-up		0 (0-0.3)
Age at enrollment, years		5.8 (3-8.8)
Age at end of follow-up, years		12.7 (9.7-16)

¹Hispanic is the reference.

A qualitative comparison of results from both FEA and MEA analyses, adjusting for age at enrollment, indicate that the estimates from the MEA are in the same direction as FEA with higher hazards in time to first event corresponding to higher hazards of new events (Table 2). In addition, two variables (sex and influenza vaccination) were significant in the MEA model but not in the FEA model. Conversely, hot tub use in previous year is significant in the FEA, but not in the MEA. However, the association is near borderline in FEA (p=0.045), but not in MEA (p=0.109). Finally, the MEA model found that the risk of experiencing a subsequent PEx increased 2.8 times (95% CI 2.2 – 3.6) when there was an exacerbation within the previous 30 days.

Table 2.

Association of risk factors with risk of pulmonary exacerbations from First and multiple event models.

	FEA		MEA		FEA Adjusted		MEA Adjusted
Characteristic	95% CI	P- value	95% CI	P- value	95% CI	P- value	95% CI	P-value
Gender: Female	1.12 (0.98-1.27)	0.084	1.29 (1.1-1.51)	0.002			1.17(1.02,1.35)	0.024
Caucasian	0.83 (0.62-1.1)	0.194	1.04 (0.71-1.52)	0.9
2CFTR mutation: heterozygous F508del	0.86 (0.75-0.99)	0.03	0.87 (0.73-1.03)	0.11
CFTR mutation: Other	0.74 (0.58-0.94)	0.01	0.68 (0.51-0.9)	0.008
Age at enrollment (years)	1.04 (1.03-1.06)	<0.001	1.09 (1.07-1.12)	<0.001	1.03 (1.01-1.05)	<0.001	1.07(1.05,1.09)	<0.001
Weight percentile at enrollment	0.83 (0.78-0.89)	<0.001	0.75 (0.7-0.81)	<0.001	0.87 (0.81-0.93)	<0.001	0.79(0.73,0.84)	<0.001
Pseudomonas + culture at most recent encounter	1.58 (1.31-1.9)	<0.001	1.13 (1.02-1.25)	0.02	1.29 (1.06-1.56)	0.009
Flu vaccination in previous year	0.87 (0.72-1.05)	0.156	0.8 (0.72-0.89)		<0.001		0.84(0.75,0.94)	0.002
Cigarette smoke exposure in previous year	1.33 (1.16-1.52)	<0.001	1.33 (1.18-1.49)	<0.001	1.27 (1.1-1.46)	<0.001	1.21(1.08,1.35)	<0.001
Hot tub use in previous year	0.86 (0.75-1)	0.045	0.93 (0.85-1.02)	0.109
Swimming pool use in previous year	0.7 (0.57-0.86)	0.001	0.81 (0.72-0.91)	0.001	0.76 (0.62-0.94)	0.010	0.82(0.73,0.93)	0.002
Cold symptoms in week prior to most recent encounter	1.76 (1.55-2)	<0.001	1.42 (1.33-1.52)		<0.001
Increased cough in week prior to most recent encounter	2 (1.88-2.12)	<0.001	1.65 (1.59-1.71)	<0.001	1.9 (1.79-2.02)	<0.001	1.61(1.56,1.67)	<0.001
Oral antibiotics used since previous encounter	2.08 (1.82-2.38)	<0.001	1.63 (1.5-1.76)	<0.001	1.69 (1.47-1.94)	<0.001	1.49(1.37,1.61)	<0.001
Respiratory culture at previous encounter positive for any CF pathogen3	1.4 (1.18-1.65)	<0.001	1.27 (1.14-1.41)	<0.001	1.18 (0.99-1.4)	0.062	1.17(1.05,1.31)	0.003
<30 days since previous PEx			2.84 (2.24-3.61)		<0.001		2.73(2.14,3.47)	<0.001

PEx – Pulmonary exacerbation; FEA – first event analysis adjusting only for age at enrolment; MEA – multiple event analysis adjusting only for age at enrolment; FEA adjusted - multivariable first event analysis.

MEA adjusted - multivariable multiple event analysis.

¹The hazard ratio in the MEA is the hazard (instantaneous probability) of a new event given the history of the events and covariates. For instance, the hazard of experiencing a new exacerbation is 29% higher for females than for males.

²The reference level of CFTR mutation is homozygous F508 del.

³The pathogens are Staphylococcus aureus, Haemophilus influenzae, and Pseudomonas aeruginosa.

Adjusting for multiple factors, the following factors were associated with higher occurrence of subsequent PExs using both approaches: older age at enrollment, household cigarette smoke exposure, increased cough at the most recent encounter, having used oral antibiotics since the previous encounter, and a positive culture for any CF organism at the most recent encounter. Higher weight percentile at enrollment, and having used a swimming pool in the last year decrease the risk of experiencing a subsequent PEx. The three variables that differ between the adjusted FEA and adjusted MEA are Pseudomonas positive culture at most recent encounter, gender and flu vaccination in previous year, the first significant in FEA and the last two significant in MEA.

The cumulative mean function of events over time by F508del status, which represents estimates of the cumulative mean number of events over time, is highest for homozygous F508del (Figure 2). At the end of follow-up it provides the total mean number of events for a patient in each of the F508del allele groups. For an individual with two F508del alleles, the expected total mean number of events at the end of the follow-up was about 3.5 events, roughly twice the number of events in the other/unknown allele group. The lack of distinction between the curves at the early stage but differences in the late stage suggests that there might be differences in PEx rates over long follow-up times. The rate at which events accumulate was lower in the other/unknown group than the other two groups.

Figure 2. Cumulative Mean Number of events for F508del groups over time.

CFTR mutation F508del genotype: homozygous (solid line), heterozygous (dashed line), and other/unknown (dotted line).

A simulation study, reported in the supplementary material, comparing power between the two methods FEA and MEA shows that in general MEA has more power to detect risk factors. For instance, under no heterogeneity between individuals, if the log hazard ratio is 0.3 (HR=1.35), with 200 individuals, power for the FEA is 45% compared to 80% power under the MEA.

Discussion

To our knowledge, this is the first study in CF to compare multiple event analysis (MEA) with time to first event analysis (FEA) and to demonstrate the advantages of MEA over FEA. Using data from the EPIC Observational study, we show that MEA, which allows adjustment for previous PEx as a risk factor, identified more risk factors compared to FEA.

PEx are major contributors to morbidity and mortality in CF, and yet we do not have a good understanding of the time course of these events or their risk factors. The commonly used time to first event analyses do not take full advantage of the fact that patients typically have multiple exacerbations during an observation period or that the association of risk factors with each exacerbation may not be the same. Approaches that model total number of events over a specified time frame ignore the timing of the events. MEA allows more complete use of the data. [27] Indeed, we were able to detect associations with MEA that were not seen with FEA. MEA resulted in the detection of gender and influenza vaccination as significant risk factors that would have been missed in the FEA model. However, Pseudomonas positive culture (Pa+) was not a significant factor in MEA but it was in FEA. This may be because Pa+ is so intrinsically related to pulmonary health status and occurrence of PEx[34] that it may be more appropriately modeled by methods that handle internal covariates. [35-37] It may also be more associated with time to first PEx when the subject is not chronically infected, and therefore may not be associated with multiple PEx events as pulmonary disease progresses.

There are some important methodological points regarding MEA that we would like to emphasize. First, the estimates of the FEA and MEA are not comparable since they are not estimating the same quantities (parameters), but power to detect risk factors may be compared between methods. [26,27] Second, there may be situations where using FEA may be preferred. For instance, if most individuals have only a very small number of multiple events, then fitting a MEA and assessing fit may be challenging due to instability of estimators.

The few studies that have used statistical methods to analyze multiple PExs in CF have mainly focused on illustrating the use of sophisticated methods for analyzing multiple pulmonary exacerbations and not necessarily on evaluating risk factors for the occurrence of PExs. Yan[21] and Liu[22] analyzed data from a CF trial with 645 patients assessing only treatment and FEV1 as covariates; the data are described and provided as an example in a multiple event analysis statistical text [40]. Milovslaski [23] used a similar model to ours (Anderson-Gill model) and considered a comprehensive list of risk factors in the Epidemiologic Study of CF with about 20,000 patients between ages of 6 and 14 years[41]. Milovslaski [23]'s focus was to present methodology that could address the potentially informative censoring when analyzing multiple events. All of these studies have been confined to the statistical literature and have yet to be applied to CF studies in the clinical literature.

The current study has a few limitations. First, since it is an observational study, there might be unmeasured confounders that could potentially affect some of the relationships observed; for instance, the protective effect of swimming pool usage may be reflective of general health or socioeconomic status. Second, methods that address the duration of pulmonary exacerbations in a more advanced way rather than just removing the time from the risk set may more appropriately capture distinctive features of the occurrence of pulmonary exacerbations. However, more complex statistical methods usually come at the price of limiting clinical interpretation. Third, the diagnosis of PEx events is not standardized and can be subjective. We only analyzed exacerbations requiring intravenous antibiotics and are unable to comment on risk factors for all (e.g. treated with oral antibiotics) exacerbation events. Further work to analyze all PEx is needed. Fourth, more investigation may be needed regarding the most appropriate time to consider for previous PEx; other windows than the 30-day used here may be more optimal and informative. More advanced modeling is needed for prediction of future PEx using occurrence of previous PEx in a dynamic way. Dynamic prediction of future PEx would allow the use of previous and current time-varying covariates as well as previous PEx to estimate the likelihood of developing a new PEx within a specified window of time[42]. In addition, the multiple event analysis used in this paper assumes a constant hazard rate over time. An important extension worth considering in future studies is a gap time analysis in which the “system” that is generating the events is renewed every time there is a new event (exacerbation) or that the hazard rate of subsequent exacerbations changes over time. This would provide estimates of specific periods of time associated with higher risk of a subsequent PEx event.

In conclusion, analysis of multiple pulmonary exacerbations provides, in general, more power to detect risk factors than only analyzing the first pulmonary exacerbation. Risk factors may be missed when only analyzing the first PEx and fewer individuals may be needed to detect associations. Multiple pulmonary exacerbations may be a promising new endpoint allowing for more powerful estimation of treatment effects.