DEVELOPMENT OF A PREDICTIVE MODEL FOR LONG-TERM SURVIVAL AFTER LUNG TRANSPLANTATION AND IMPLICATIONS FOR THE LUNG ALLOCATION SCORE

Abstract

Background

Improving long-term survival after lung transplantation can be facilitated by identifying patient characteristics that are predictors of positive long-term outcomes. Validated survival modeling is important for guiding clinical decision-making, case-mix adjustment in comparative effectiveness research, and refinement of the lung allocation system (LAS).

Methods

We used the ISHLT registry to develop and validate a predictive model of 5-year survival after lung transplantation. A total of 18,072 eligible cases were randomly split into development and validation datasets. Pre-transplant recipient variables considered included: age, gender, diagnosis, body mass index, serum creatinine, hemodynamic variables, pulmonary function variables, viral status and co-morbidities. Predictors were considered in a stepwise approach with Akaike Information Criteria. Time-dependent receiver operator characteristic (ROC) curves assessed predictive ability. A 1-year conditional model and three models for disease subgroups were considered. ROC methods were used to characterize the predictive potential of the LAS post-transplant model at 1- and 5-years.

Results

Baseline model included: age, diagnosis, creatinine, bilirubin, oxygen requirement, cardiac output, Ebstein Barr virus status, transfusion history, and diabetes history. Prediction of long-term survival was poor (area under the curve (AUC)=0.582). Neither the 1-year conditional model (AUC=0.573), nor models designed for separate diseases (AUC=0.553-0.591) improved survival prediction. Predictive ability of the LAS post-transplant parameters was similar to our model (1-year AUC=0.580 and 5-year AUC=0.566).

Conclusions

Models developed from pre-transplant characteristics poorly predict long-term survival. Models for separate diseases and 1-year conditional models did not improve prediction. Better databases and approaches to predict survival are needed to improve lung allocation.

Introduction

With an increasing focus on improving long-term survival after lung transplantation, identifying characteristics of lung transplant recipients who survive greater than 5-years is an important undertaking. In addition, development of an accurate predictive model based on a large registry of lung transplant recipients is necessary for case-mix adjustment in comparative effectiveness research and potentially improving the lung allocation system (LAS).

Prior to implementing the LAS, priority for US lung allocation was centered on accumulated waitlist time.(1) Given differing severities and rates of disease progression in candidates, a system based on acuity rather than accrued time was necessary.(2) In May 2005, the Organ Procurement and Transplantation Network (OPTN) implemented the LAS to prioritize US candidates for lung transplantation by waitlist urgency and transplant benefit.(3) In development of this model, two sets of parameters were identified to calculate the waitlist urgency and transplant benefit: waitlist model parameters and post-transplant survival model parameters.

Several controversies surrounding the development of the LAS have been raised. One of the main controversies revolves around the appropriate follow-up time after transplantation for the derivation of the LAS. Critics argued that 1-year survival is not a surrogate marker for long-term survival and therefore, cannot entirely represent transplant benefit. In contrast, others suggested that pre-transplant characteristics are only predictive of short-term survival and therefore a model based on 1-year survival is appropriate.(4) Despite this debate, no study has investigated whether pre-transplant characteristics can adequately predict long-term survival.

As more organ exchange organizations outside the US consider development of their own allocation scores, and as the OPTN considers further LAS refinement, it is important to investigate other models of post-transplant survival including assessing the performance of the LAS with respect to long-term survival. In doing so, we may be able to identify additional variables that should be considered in future LAS refinements.

In this study, we developed and validated a predictive model of survival up to 5-years after transplantation using pre-transplant characteristics. We used Cox regression to develop a predictive survival model and then evaluated the predictive accuracy of the model score using time-dependent Receiver Operator Curve (ROC) methods.(5) We also developed a long-term survival model conditioned on 1-year survival. We also developed three predictive models restricted to patients within major disease subgroups. Finally, we used time-dependent ROC methods to also assess the ability of the post-transplant factors in the LAS to predict 1- and 5-year survival.

Methods

Population

The ISHLT has maintained a registry for lung transplants since 1988. This database is compilation of data from registries around the world including that of the United Network of Organ Sharing (UNOS). As of June 2008, the registry logged a total of 27,075 lung transplants recipients. Because practices have changed substantially since the first recorded lung transplant, we excluded transplants prior to 1997. In addition, lung transplant patients who were younger than 18, re-transplanted, or had multiple organs transplanted or had no follow up status available were excluded, leaving 18,072 lung subjects available for analysis (Figure 1a).

Figure 1a.

Development of Study Sample in the ISHLT database.

Candidate Predictors of Long-term Survival

We considered pre-transplant patient characteristics such as: age, gender, diagnosis, body mass index (BMI), serum creatinine, hemodynamic variables, pulmonary function variables, viral status measures and common co-morbidities. Age was modeled with a linear spline by decade to capture the complex relationship between survival and age.

Univariate Analysis

Candidate Predictors were assessed for univariate association with survival for up to 5-years after transplant using Cox regression methods. Significance was reported at p-value <0.05.

Missing Data

Many of the candidate predictors had some missing data with rates between 14% and 55% across all study years. In order to mitigate the impact of the missing data, we used multiple imputation.(6) For each missing value, 10 values were imputed using the ICE package in STATA v.10.1 (College Station, TX). This resulted in 10 imputed data sets. All subsequent analyses were performed for each imputed data set separately and then combined according to Rubin’s rules.(7)

Model Development

We set aside half of the transplants cases (N = 9,036) for model validation and used the remaining half to develop predictive models. The primary goal was to construct a Cox model(8) for predicting risk of death within 5-years after transplant. This was done in a stepwise approach. At each step, a model was fit for each of the imputed data sets separately and then the results were combined.(9). Predictor variables were considered for inclusion or exclusion based on the combined Akaike Information Criteria (AIC)(10). Coefficients were obtained for the final selected model.

In addition to the baseline model of pre-transplant patient characteristics, we explored other modeling approaches. First we developed a model of observations which had no missing data (n=1536). Then, we developed a model that was conditioned on survival to 1-year after transplantation. In addition, we developed disease subgroup predictive Cox models for the three most common diagnoses in the registry (chronic obstructive lung disease/alpha-1 antitrypsin deficiency [COPD], idiopathic pulmonary fibrosis [IPF] and cystic fibrosis (CF).

Predictive Ability

To quantify the predictive ability of a model, predictive scores for the validation cases were generated based on estimated Cox regression coefficients identified with development data. Scores were compared to observed follow-up times and status using a time dependent receiver operator characteristic (ROC) curve accounting for censored survival times.(5) A time-dependent ROC curve characterizes the ability of a predictive score to discriminate between “cases” defined as subjects that die prior to a pre-specified time and “controls” defined as subjects that lived beyond the specified time. The area under the curve (AUC) at 5-years was calculated as a summary measure. Potential values of AUC range from 0.5 (no predictive value) to 1 (perfect prediction). For each model evaluated, the AUC was calculated separately for each imputed data set and then averaged.

Comparison with LAS

Finally, we assessed the ability of the parameter estimates from the LAS post-transplant survival model to predict 1-year and 5-year survival. The LAS post-transplant model was derived from US data collected in the UNOS database. As described above, the UNOS database is a part of the ISHLT database; however some of the LAS variables are not available in the ISHLT database. Therefore, to assess the predictive ability of the LAS, we used the UNOS database. We used similar exclusion criteria as described earlier; however, we additionally excluded cases with one or more missing LAS variable rather than perform multiple imputation so that we did not underestimate the sensitivity of the LAS. Therefore, a total of 1446 cases that had one or more missing LAS variables were excluded. Our final database had 10,128 lung transplants available for analysis. (Figure 1b)

Figure 1b.

Development of Study Sample in the UNOS database.

Using the post-transplant model parameter estimates(11), 1-year survival scores were calculated for all cases. The ability of this score to distinguish those who survived at 1-year and those who did not was assessed with AUC. For comparison, we obtained the AUC at 1-year for our long-term survival model. In addition, we assessed how well the post-transplant model variables in the LAS predicted 5-year mortality.(11) We calculated parameters for the variables in the post-transplant survival model based on 5-year survival data for development cases and calculated 5-year survival scores for validation cases. The ability of this score to distinguish those who survived at 5-years and those who did not was assessed with time dependent AUC.

Results

Baseline characteristics

Baseline characteristics revealed that median (IQR) age of recipients was 54 (42, 59), most recipients were male (53%) and most had COPD (44%; Table 1). Although data on age, gender and diagnosis was complete by design, there was a significant amount of missing data for other variables (Table 1).

Table 1.

Baseline pre-transplant recipient characteristics and percent of missing data in the International Society of Heart and Lung Transplantation (ISHLT) database.

Recipient Characteristic		Percent Missing (%)
Age median (IQR)	54 (42, 59)	0
Female % (n)	46.4 (8,383)	0
Diagnosis % (n)		0
COPD1	44.1 (7,961)
Idiopathic Pulmonary Fibrosis	21.7 (3,916)
Cystic Fibrosis	16.1 (2,912)
Other Fibrosis2	7.2 (1,307)
Pulmonary Hypertension	4.6 (830)
Bronchiectasis	2.9 (526)
Lymphangiomyomatosis	1.0 (173)
Other	2.5 (447)
Body Mass Index median (IQR)	23.4 (19.9 – 27.0)	13.5
Creatinine median (IQR)	0.8 (0.7 -1.0)	29.7
Total Bilirubin median (IQR)	0.5 (0.3 - 0.8)	34.5
Pulmonary Variables
Oxygen Requirement, L/min median   (IQR)	2 (2 – 4)	42.8
Carbon Dioxide, mEq/L median (IQR)	44 (39 – 51)	54.4
FEV1 (L) mean (sd)	34.7 (21.0)	38.2
FVC (L) mean (sd)	50.4 (17.9)	38.6
6 minute walk test < 150 feet % (n)	11.4 (1,301)	36.6
Mechanical ventilation % (n)	6.0 (573)	46.8
Hemodynamic Variables median (IQR)
Cardiac Output (L/min)	5.1 (4.3 – 6.0)	53.8
Wedge Pressure (mm Hg)	11 (8 – 14)	50.8
Systolic Pulmonary Artery Pressure	37 (31 – 45)	47.0
Recipient Viral Status % (n)
CMV positive status	60.2 (6,958)	36.0
EBV positive status	84.2 (6,960)	54.2
HCV positive status	2.1 (224)	41.5
Co-morbidities % (n)
Diabetes	14.9 (1,966)	27.0
Hypertension	17.4 (2,193)	30.1
Cancer	3.6 (470)	28.3
Transfusion History	4.5 (496)	38.8
Peptic Ulcer	5.1 (589)	35.9
Cerebral Vascular	1.0 (83)	31.7
Corticosteroid user	36.9 (4,366)	34.6

¹COPD: Chronic Obstructive Pulmonary Disease and alpha-1 antitrypsin deficiency

²Other fibrosis includes: rheumatoid arthritis, wegner’s granulomatosis, pulmonary telengectasia, mixed connective tissue disease, polymyositis, silicosis, CREST, scleroderma, collagen vascular disease, connective tissue disease, desquamative interstitial pneumonitis, lymphocytic interstitial pneumonitis, restrictive lung disease, sjogren’s, occupational lung disease-other, pulmonary fibrosis(other specify).

Univariate analysis

In comparison to patients with COPD, patients with IPF, other types of fibrosis or pulmonary hypertension were significantly less likely to survive to 5-years; while patients with CF and lymphangioleiomyomatosis (LAM) were more likely to survive to 5-years (Table 2). Other significant variables are listed in Table 2.

Table 2.

Univariate associations of pre-transplant characteristics with 5-year survival

Recipient Characteristic	Hazard Ratio	P-value
Age
18 – 25	REF
26 – 35	0.84	0.008
36 – 45	0.81	0.001
46 – 55	0.93	0.236
56 – 65	1.12	0.031
< 65	1.41	<0.001
Female vs. Male	1.08	0.003
Diagnosis
COPD1	REF
Idiopathic Pulmonary Fibrosis	1.30	<0.001
Cystic Fibrosis	0.88	0.001
Other Fibrosis2	1.23	<0.001
Pulmonary Hypertension	1.29	<0.001
Bronchiectasis	1.08	0.306
Lymphangiomyomatosis	0.43	<0.001
Other	1.07	0.409
Body Mass Index	1.02	<0.001
Creatinine	1.15	<0.001
Total Bilirubin	1.02	<0.001
Pulmonary Variables
Oxygen Requirement, L/min	1.06	<0.001
Carbon Dioxide, mEq/L	0.995	0.003
FEV1 (L)	1.003	<0.001
FVC (L)	1.000	0.711
6 minute walk test < 150 feet	1.24	<0.001
Mechanical ventilation	1.18	0.031
Hemodynamic Variables
Cardiac Output (L/min)	0.966	0.008
Wedge Pressure (mm Hg)	0.998	0.468
Systolic Pulmonary Artery Pressure	1.003	0.004
Recipient Viral Status
CMV positive status	1.017	0.584
EBV positive status	0.86	0.003
HCV positive status	1.24	0.039
Co-morbidities
Diabetes	1.13	0.002
Hypertension	1.16	<0.001
Cancer	1.10	0.232
Transfusion History	1.30	<0.001
Peptic Ulcer	1.18	0.011
Cerebral Vascular	1.36	0.061
Corticosteroid user	1.22	<0.001

¹COPD: Chronic Obstructive Pulmonary Disease and alpha-1 antitrypsin deficiency

²Other fibrosis includes: rheumatoid arthritis, wegner’s granulomatosis-restrictive, pulmonary telengectasia, mixed connective tissue disease, polymyositis, silicosis, CREST, scleroderma, collagen vascular disease, connective tissue disease, desquamative interstitial pneumonitis, lymphocytic interstitial pneumonitis, restrictive lung disease, sjogren’s, occupational lung disease-other, pulmonary fibrosis(other specify).

Multivariate analysis

Adjusted overall model

The AUC at 5-years was low for the adjusted overall model (0.582; Table 3). Variables that were selected and were significantly associated with a lower survival included: 1) age (45-55 years and 55-65 years); 2) higher creatinine; 3) higher total bilirubin; 4) higher oxygen requirement; 5) lower cardiac output; 6) negative EBV status; 7) diabetes history 8) transfusion history and 9) use of chronic systemic steroid (Table 3). Patients with IPF, other fibrosis, pulmonary hypertension and bronchiectasis were less likely to survive, and patients with LAM were more likely to survive to 5-years than patients with COPD in the adjusted overall model (Table 3). We also performed a similar analysis of subjects who had no missing data. The AUC of this model was still low at 0.553 (data not shown).

Table 3.

Baseline model parameters and AUC of 5-year survival using pre-transplant patient characteristics and 5-year survival conditioned on surviving to one year

	Baseline Model	Conditioned on 1-year survival
Area Under the Curve	0.582	0.573
Variable	B coef	B coef
Age 18 – 25	−0.0861***	−0.110***
Age 25 – 35	−0.00826	−0.0347
Age 35 – 45	−0.00239	−0.0108
Age 45 – 55	0.0230**	0.02450*
Age 55 – 65	0.0249**	0.0291**
Age 65+	0.0277	0.0925*
Female vs. Male	---	---
Diagnosis
COPD1	REF	REF
Idiopathic Pulmonary Fibrosis	0.232**	−0.100
Cystic Fibrosis	−0.0146	−0.221
Other Fibrosis2	0.270**	−0.303*
Pulmonary Hypertension	0.361**	−0.157
Bronchiectasis	0.290**	0.0433
Lymphangiomyomatosis	−1.01**	−1.73**
Other	0.0258	−0.0965
Body Mass Index	0.00966	0.0203**
Creatinine	0.109*	---
Total Bilirubin	0.0173**	0.0156
Pulmonary Variables
Oxygen Requirement, L/min	0.0475***	0.0397*
Carbon Dioxide, mEq/L	−0.00269	---
FEV1 (L)	−0.00313	---
FVC (L)	---	---
6 minute walk test < 150 feet	0.0850	---
Mechanical ventilation	0.138	---
Hemodynamic Variables
Cardiac Output (L/min)	−0.0392*	−0.0355
Wedge Pressure (mm Hg)	−0.00390	---
Systolic Pulmonary Artery Pressure	0.00259	---
Recipient Viral Status
CMV positive status	---	---
EBV positive status	−0.165**	−0.204**
HCV positive status	---	0.235
Co-morbidities
Diabetes	0.117*	---
Hypertension	---	0.109
Cancer	---	−0.238
Transfusion History	0.217*	0.211
Peptic Ulcer	0.102	---
Cerebral Vascular	---	---
Corticosteroid user	0.101*	0.183**

¹COPD: Chronic Obstructive Pulmonary Disease and alpha-1 antitrypsin deficiency

²Other fibrosis includes: rheumatoid arthritis, wegner’s granulomatosis-restrictive, pulmonary telengectasia, mixed connective tissue disease, polymyositis, silicosis, CREST, scleroderma, collagen vascular disease, connective tissue disease, desquamative interstitial pneumonitis, lymphocytic interstitial pneumonitis, restrictive lung disease, sjogren’s, occupational lung disease-other, pulmonary fibrosis(other specify).

^*p < .05,

^**p < .01,

^***p < .001

Overall model conditioned on surviving to 1-year

The AUC for the 5-year prediction model conditioned on survival to 1-year after transplant was lower than the adjusted overall model (0.573 vs. 0.582; Table 3). After conditioning on 1-year survival, there was no longer a significant difference in survival between patients with COPD and IPF, pulmonary hypertension or bronchiectasis. After conditioning on 1-year survival, total bilirubin, cardiac output, diabetes history, and transfusion history were no longer significant predictors of survival (Table 3). Additional variables that were significantly associated with a lower survival at 5-years after conditioning on 1-year survival included: age ≥ 65 and higher BMI (Table 3).

Models predicting 5-year survival for separate disease subgroups

The AUCs for predicting survival at 5-years for all three disease specific models were low (0.553-0.591; Table 4). Different variables were selected by AIC criteria for each disease specific model. Gender was not included in the models for COPD or IPF; however, male gender was significantly associated with a lower survival in the CF model (p<0.01; Table 4). Oxygen requirement was included in the COPD and IPF model; however it was only significantly associated with survival in the IPF model (p<0.001; Table 4). FEV₁ was included in the COPD and the IPF model but was not significantly associated with survival. Neither six minute walk test nor FVC was included in any of the disease specific models (Table 4).

Table 4.

AUC and parameters of 5-year survival using disease specific models

	COPD1	IPF2	CF3
Area Under the Curve	0.553	0.591	0.584
Variable
Age 18 – 25	6.20	14.4	−0.0959***
Age 25 – 35	−0.0450	−0.0268	−0.0257
Age 35 – 45	−0.0488*	0.0421	0.0149
Age 45 – 55	0.0310*	0.0208	0.00333
Age 55 – 65	0.0338***	0.00613	0.149
Age 65+	0.132**	−0.0121	−0.418
Female vs. Male	---	---	−0.220*
Body Mass Index	---	0.0168	---
Creatinine	0.161*	---	---
Total Bilirubin	0.0141	0.0409***	---
Pulmonary Variables	---	---	---
Oxygen Requirement,   L/min	0.0337	0.0525**	---
Carbon Dioxide, mEq/L	---	---	---
FEV1 (L)	−0.00230	−0.00595	---
FVC (L)	---	---	---
6 minute walk test < 150   feet	---	---	---
Mechanical ventilation	---	0.353	---
Hemodynamic Variables
Cardiac Output (L/min)	−0.0417	−0.0452	−0.0538
Wedge Pressure (mm Hg)	---	---	---
Systolic Pulmonary Artery  Pressure	---	---	---
Recipient Viral Status
CMV positive status	---	---	−0.142
EBV positive status		−0.253	−0.136
HCV positive status	---	---	---
Co-morbidities
Diabetes	0.183	---	0.264*
Hypertension	---	---	---
Cancer	−0.218	---
Transfusion History	0.235	---	0.337
Peptic Ulcer	---	---	0.322
Cerebral Vascular	---	---	---
Corticosteroid user	0.153*	---	---

¹COPD: Chronic Obstructive Pulmonary Disease and alpha-1 antitrypsin deficiency

²IPF: Idiopathic Pulmonary Fibrosis

³CF: Cystic Fibrosis

^*p < .05,

^**p < .01,

^***p < .001

Ability of the variables in the LAS model to predict survival

The ability of the variables in the post-transplant model of the LAS to predict 1-year and 5-year survival was poor (AUC=0.580 and AUC=0.566 respectively; Table 5). The predictive ability of the LAS post-transplant survival model for 1- and 5-year survival was similar to the ability of our model (1-year AUC=0.606 and 5-year AUC=0.582, data not shown and Table 3). Parameters from the LAS post-transplant model that were not significantly associated with a 5-year survival included FVC, PCW, and having a diagnosis of pulmonary vascular disease, CF/immunodeficiency disorder, interstitial lung disease, bronchiectasis, obstructive bronchiolitis, or sarcoidosis in comparison to patients with obstructive disease (Table 5).

Table 5.

AUC of the LAS post-transplant model to predict 1-year survival and the AUC and the parameters of LAS post-transplant model to predict 5-year survival

	LAS model & 1- year survival	LAS variables & 5-year survival
Area under the curve	0.580	0.566
Variables	B coefficient1	B coefficient
Recipient age	0.00351	0.00955***
Creatinine	0.0620	0.129***
NYHA Class	−0.489	−0.364***
FVC	−0.00275	−0.0000766
PCW	0.0330	−0.0197
Mechanical Ventilation	0.313	0.681***
Diagnosis Group2
B	0.623	0.0991
C	0.00851	0.0911
D	0.413	0.174
Diagnosis detailed
Bronchiectasis	0.0561	0.0297
Eisenmenger’ s	0.394	0.583**
LAM	−0.624	−0.619**
Obstructive Bronchiolitis	−0.444	−0.153
Pulmonary Fibrosis other	0.172	0.344***
Sarcoid PA mean>30	−0.122	0.157
Sarcoid PA mean<30	−0.0165	0.135

^*p < .05,

^**p < .01,

^***p < .001

¹B coefficient derived in the LAS post-transplant model. P-values are N/A.

²

Diagnosis groups are defined as:

• Group A = Obstructive lung disease (e.g., emphysema)
• Group B = Pulmonary vascular disease (e.g., primary pulmonary hypertension)
• Group C = Cystic fibrosis or immunodeficiency disorder
• Group D = Restrictive lung disease (e.g. IPF, NSIP, sarcoidosis)

Discussion

This study suggests that the ability to predict long-term survival with pre-transplant characteristics is poor. In addition, although different covariates were identified among our disease subgroup models, our ability to predict long-term survival was not improved. Finally, the LAS was shown to have a poor ability to predict both 1-and 5-year survival.

Despite multiple modeling approaches, the ability to predict 5-year survival was not much better than chance. There are many possible explanations for the poor predictive ability. First, the ISHLT database has a significant amount of missing data (Table 1) for many pre-transplant variables. This degree of missing data might lead to a lower ability to predict long-term survival. Also, little information about the severity of co-morbidities such as the duration or control of the patient’s hypertension or diabetes is included in the database. These co-morbidities have been shown to be associated with decreased survival(12); therefore, inclusion of more detailed co-morbidity information may increase the predictive ability of the model.

Secondly, other variables that may be important to predicting long-term survival might not be included in the ISHLT transplant registry. In particular, information such as psychological disease, medical adherence and social support is not included, but has been shown to be associated with outcomes in other types of solid organ recipients.(13) For example, patients who demonstrated symptoms of post traumatic stress disorder after heart transplantation had more subsequent episodes of acute rejection and were more likely to die than those who did not exhibit these symptoms.(13) Therefore, some of these socio-behavioral factors may be important to consider in development of a long-term survival predictive model.

Finally, patient characteristics collected prior to transplantation may only be predictive of short term survival.(4) This is suggested by the differences seen between the baseline model and the model conditioned on 1-year survival (Table 3). In the baseline model, patients with IPF, other fibrosis, pulmonary hypertension and bronchiectasis are significantly less likely to survive to 5-years in comparison to patients with COPD. However, in the model of 5-year survival conditioned on 1-year survival, there is no significant difference between patients with COPD and those with IPF, CF, pulmonary hypertension or bronchiectasis (Table 3). This finding suggests that although the pre-transplant diagnosis may be associated with short term survival, it may be less important in predicting survival after the first year.

Our study also demonstrated that creating models of long-term survival designed specifically for disease subgroups did not significantly improve our model’s predictive ability; however significant differences are seen among the models’ covariates. In patients with CF, pre-transplant variables that were significantly associated with a lower 5-year survival included: age, male gender and a diabetes history. In comparison, patients with IPF who had a higher pre-transplant total bilirubin or oxygen requirement were less likely to survive to 5-years. Finally, patients with COPD who were older, had a higher pre-transplant creatinine or chronically used corticosteroid were less likely to survive to 5-years. Although development of disease specific models did not improve our ability to predict long-term survival, the differences between the models suggest that patient characteristics are different between diagnoses. Understanding these differences may help to develop better models in future refinements.

Finally, although our model has a low predictive ability of 5-year survival (AUC=0.582), our study showed that the post-transplant model in the LAS has a similar AUC for 1-year survival (AUC=0.580). Although the LAS was not designed to predict 5-year survival, we also determined that the post-transplant model variables of the LAS has a poor ability to predict 5-year survival (AUC=0.566). These findings suggest that the LAS does not predict 1- and 5-year survival significantly better than a chance. Findings such as these suggest that the current LAS system needs further refinement and validation in order to improve listing practices and improve post-transplant survival.

There are several limitations of this study. First, development of a good discriminatory predictive model is dependent on the accuracy and completeness of the data. In the ISHLT Registry, accuracy of the severity of illness at the time of transplant may be misrepresented because the data were not collected at the time of transplant and the disease may have progressed since the data were collected. Despite this limitation, no other large registry captures this data better than the ISHLT registry. In addition, there was a significant amount of missing data (Table 1) which could have reduced our ability to predict long-term survival. In order to address this issue, we performed multiple imputations. We believe that this approach was superior to excluding cases that had a significant amount of missing data as this could have drastically reduced the cases available for analysis. Some data was not collected during earlier time periods and therefore could lead to some bias in our imputation. In order to reduce this bias, we did include transplant year as a predictor during the imputation process. In addition, individual centers may not collect certain variables; therefore, some missing data may not be at random. The ISHLT registry does not include center or country identification, therefore we cannot assess this bias. However, the ISHLT database is the largest and the most complete database that exists to use in this study.

A second limitation is that the ISHLT registry may not contain some variables that could be important predictors. This is a more difficult limitation to address, but the ISHLT Registry is clearly the best source of data for this study as it is the largest existing database for lung transplant recipients. A third potential limitation is that errors in the categorization of pre-transplant diagnosis in the ISHLT database could result in too many categories with inadequate data, or, alternatively, be combined into larger categories that results in misclassification. Either error could lead to inaccurate conclusions. We therefore categorized pre-transplant diagnoses into 8 major diagnoses which we defined as having more than 500 subjects transplanted per category over 5-years. Although we will lose some information with this categorization, this categorization is similar to previous studies and it remains more detailed than that used in creation of the LAS.

Finally, there are some limitations in assessing the ability of the post-transplant parameters in the LAS to predict 1- and 5-year survival. First, as with the ISHLT registry, there was a significant amount of missing data in the UNOS database. We eliminated cases that had missing data on key variables which comprised 16% of the database. Although this might cause confounding, we chose to use the same methods as were conducted in the development of the LAS post-transplant parameters(3). Second, the parameters in the LAS post-transplant parameter set were originally selected from data that was collected from 1/1/1995-12/31/1998 for patients with pulmonary vascular disease and from 1/1/1997-12/31/1998 for all other diagnoses. All the eligible data from these time periods were used and there was no validation set created. In our analysis, we used data from 1997 - 2008. Therefore, some of the data that was used to develop the post-transplant LAS model was used in the validation set. This might lead to an overestimation of the AUC. Finally, it has been reported that survival has improved over time. In our validation set, we used patients who were transplanted more recently than those used to develop the post-transplant LAS set. In doing so, we may obtain a lower AUC; however, this will be a fair analysis of how the post-transplant model is currently predicting survival.

In conclusion, we developed and validated a predictive model of 5-year survival on lung transplant patients. We determined that a model developed from pre-transplant characteristics had very little predictive ability and was not significantly improved when we conditioned on 1-year survival or developed diagnosis subgroup models. Finally, we also demonstrated that the post-transplant 1-year survival model of the LAS which was developed from pre-transplant characteristics had a very low predictive ability. These findings suggest that better approaches to predicting short and long-term survival need to be developed in order improve allocation of lungs for transplantation. Databases that have less missing data, are updated with data at the time of transplantation and include more information about the severity and duration of co-morbidities should be constructed in order to develop better predictive models that effectively allocate organs.