The performance of the GAP model in patients with rheumatoid arthritis associated interstitial lung disease

Abstract

Background

Rheumatoid arthritis-associated interstitial lung disease (RA-ILD) is associated with significant morbidity and mortality. Similarities have been observed between patients with idiopathic pulmonary fibrosis (IPF) and the UIP (usual interstitial pneumonia) form of RA-ILD. The GAP (gender, age, physiology) model has been shown to predict mortality in patients with IPF, but its ability to predict mortality in RA-ILD is not known.

Methods

We identified 309 patients with RA-ILD at 4 academic centers with ongoing longitudinal cohorts of patients with ILD. The primary endpoint was mortality. To handle missing data (n=219 subjects with complete dataset), multiple imputation by iterative chained equations was used. Using the GAP model as a baseline, we assessed improvements in mortality risk prediction achieved by incorporating additional variables. Model discrimination was assessed using the c-index, and calibration was checked by comparing observed and expected incidence of death.

Results

Patients had a mean age of 65 years and were predominantly female (54%). The mean forced vital capacity (FVC) % predicted was 73 and the mean diffusing capacity for carbon monoxide (DL_CO) % predicted was 55. Twenty-four percent of the 236 patients with a high-resolution computed tomography scan available for review had a definite UIP pattern. The original GAP model, including gender, age, FVC%, and DL_CO%, had a c-index of 0.746 in our cohort. Calibration of this model was satisfactory at 1, 2 and 3 years. Model discrimination was not meaningfully improved by adding other clinical variables.

Conclusion

The GAP model that was derived for IPF performs similarly as a mortality risk prediction tool in RA-ILD.

Introduction

Rheumatoid arthritis (RA) is a systemic autoimmune disease that is common in the United States with an estimated prevalence of 0.5 – 1%¹. Although the principal manifestation of RA is inflammatory arthritis, extra-articular organ involvement is often observed^2–4. Interstitial lung disease (ILD) constitutes the most frequent pulmonary manifestation of RA². Rheumatoid arthritis-associated interstitial lung disease (RA-ILD) is associated with high morbidity and mortality^5–9.

Previous studies have attempted to identify factors that can predict mortality in RA-ILD^10–15. Variables such as age, gender, diffusion capacity of the lung for carbon monoxide (DLCO), extent of fibrosis, and presence of the usual interstitial pneumonia (UIP) pattern have been shown to be associated with mortality in this population. A recent systematic review highlighted the lack of methodologic quality and the inconsistent results between several of these studies¹⁶.

Clinical and prognostic similarities have been observed between patients with idiopathic pulmonary fibrosis (IPF) and the UIP form of RA-ILD^14,17. The GAP model (gender, age, physiology) is a validated risk prediction model for mortality among patients with IPF²⁰. Its ability to predict mortality among patients with RA-ILD is not known. Given the shared attributes between RA-ILD and IPF, we assessed the ability of the GAP model to predict mortality in a cohort of patients with RA-ILD and explored whether its performance could be improved by adding clinical variables.

Methods

Study population

We used retrospectively collected patient data from longitudinal databases of ILD centers from the Mayo Clinic Rochester (USA), the University of Ulsan, Seoul (Korea), the University of California, San Francisco (USA) and the University of Modena and Reggio Emilia (Italy). To be included in the study, patients were required to have a rheumatologist-confirmed diagnosis of RA ²¹ and a confirmed diagnosis of ILD on high-resolution computed tomography (HRCT) and/or surgical lung biopsy. Patients with no ILD on HRCT were excluded (Figure 1). The final cohort included 309 patients. The institutional review boards at the four sites (Mayo Clinic Institutional Review Board (#12-009206), Institutional Review Board of Asan Medical center (#2013-0433), University of California, San Francisco Institutional Human Subject Review Committee (#10-01592) and the ethics committee of the University of Modena and Reggio Emilia (#2636)) approved the parent studies and patients provided written informed consent.

Figure 1.

Cohort formation

Patients were excluded if they did not have interstitial lung disease on high-resolution computed tomography and/or surgical lung biopsy

Measurements

Baseline demographic data (age, sex, ethnicity, smoking status, oxygen use and body mass index [BMI]) were obtained using structured questionnaires at the initial ILD clinic visit and medical record review. Results of pulmonary function tests (PFTs) performed within 6 months of the first visit were recorded. Medical chart review was used to obtain additional variables such as serologic profile (e.g rheumatoid factor (RF) and anti-cyclic citrullinated peptide antibody (CCP)), surgical lung biopsy results, and immunosuppressive therapy.

HRCTs performed within 1 year of the initial visit in the ILD clinic were reviewed by an experienced thoracic radiologist (BME). Each HRCT was classified as having either a definite UIP, possible UIP or inconsistent UIP pattern, according to the published guidelines²². Time to mortality was the main endpoint. Death dates were obtained using medical chart review and the US. Social Security Death Index or national death registries. Lung transplantation was handled by censoring at the time of transplant.

Statistical analysis

To account for missing values, we used multiple imputation by iterative chained equations (e-Appendix 1). We first looked at the performance of the variables in the continuous GAP model (gender, age, DLCO % predicted and forced vital capacity (FVC) % predicted). The c-index of the GAP model was estimated in our cohort using 10-fold cross-validation in each of the 20 completed datasets, then averaged. A 95% bootstrap percentile confidence interval of this c-index was calculated.

Next, we explored if we could improve the performance of the GAP model by incorporating additional variables. Candidate variables included ethnicity, ever smoking history, BMI, RF, CCP, and definite UIP pattern on HRCT. We evaluated how the combination of the GAP variables with these additional predictors was associated with mortality using Cox models. To identify a prediction model without over-fitting, we exhaustively screened candidate models that included the GAP variables and one to four additional candidate variables. We calculated the c-index using cross-validation for each candidate model. Models were then ranked according to this optimism-corrected measure of discrimination. We then repeated this screening without requiring the inclusion of the four GAP variables.

We calculated the difference between the c-index of the original GAP and the expanded GAP. To better characterize the value of the additional variables, we calculated the p-values to evaluate the impact of these predictors in the Cox model including the original GAP variables. Sensitivity analyses were conducted using complete cases analysis.

Finally, we assessed the calibration of the final model by comparing the model-based estimates to nonparametric estimates of mortality at year 1, 2 and 3 in both the complete cohort (using the imputed data) and the complete case analysis.

All analyses were performed using STATA 14 (Stata Corp, College Station, Texas).

Results

Patient Characteristics

Table 1 presents the baseline characteristics of the complete cohort (n=309). Patients had an average age of 65, were predominantly female (54%), Caucasian (57%) and had a history of smoking (53%). On average, they had moderate physiologic impairment with an FVC % predicted of 73.4% and a DLCO % predicted of 55.1%. The majority of patients had a positive RF (89%) or CCP (71%) antibody. In patients with available results for both serologic tests (n=251), 64% had a positive RF and CCP antibody, while 9% had a negative RF and CCP antibody. Twenty-four percent of the 236 patients with HRCT scans available for review had a definite UIP pattern. Nineteen percent underwent a surgical lung biopsy. Of those, 61% had a UIP pattern on pathology. As noted in Table 1, there was some missing data in some of the baseline characteristics reported. The cohort of patients with complete data included 219 subjects. The baseline characteristics of patients by center are shown in e-Table 1.

Table 1.

Patient Characteristics

Characteristics	n=309
Center
Rochester- n/total (%)	117/309 (38)
Seoul- n/total (%)	104/309 (34)
San Francisco- n/total (%)	57/309 (18)
Modena- n/total (%)	31/309 (10)
Age (SD), years	64.9 (9.7)†
Female – n/total (%)	167/309 (54)
Race and ethnicity
White or Caucasian- n/total (%)	177/309 (57)
Asian- n/total (%)	110/309 (36)
African american- n/total (%)	9/309 (3)
Native American - n/total (%)	3/309 (1)
Hispanic/Latino - n/total (%)	9/309 (3)
Ever smoker - n/total (%)	162/308 (53)
BMI (SD), kg/m2	27.0 (6.2)‡
Rheumatoid factor - n/total (%)	222/266 (89)
Anti-cyclic citrullinated peptide antibody- n/total (%)	186/261 (71)
FVC (SD), % predicted	73.4 (20.1)§
DLCO (SD), % predicted	55.1 (21.3)ll
HRCT - n/total (%)*	236/309 (76)
Definite UIP - n/total (%)	56/236 (24)
Possible UIP – n/total (%)	38/236 (16)
Inconsistent with UIP – n/total (%)	142/236 (60)
Surgical lung biopsy - n/total (%)	59/309 (19)
UIP pattern - n/total (%)	36/59 (61)
NSIP pattern - n/total (%)	13/59 (22)
Others – n/total (%)	10/59 (17)

^*According to the published guidelines

^†n = 309

^‡n = 297

^§n = 300

^lln = 284

Abbreviations: UCSF = University of California, San Francisco; BMI = body mass index ; FVC = forced vital capacity; DLCO = diffusing capacity for carbon monoxide; HRCT = high-resolution computed tomography; UIP = usual interstitial pneumonia; NSIP = non-specific interstitial pneumonia.

Treatment and outcomes are summarized in table 2 and shown by center in e-Table 2. The majority of patients received immunosuppressive therapy (79%). The most frequently reported agents were prednisone (83%), methotrexate (40%), hydroxychloroquine (33%), and azathioprine (23%). Median follow-up time was 3.01 (range 0.03 – 18.8) years. During the follow-up period, 99 subjects died and 3 underwent lung transplantation.

Table 2.

Treatment and outcomes

Characteristics	n=309
Long-term oxygen therapy - n/total (%)	70/302 (23.2)
Immunosuppressive therapy - n/total (%)	243/309 (78.6)
Prednisone - n/total (%)	255/308 (82.8)
Methotrexate - n/total (%)	124/308 (40.3)
Hydroxychloroquine- n/total (%)	98/302 (32.5)
Azathioprine- n/total (%)	70/308 (22.7)
Leflunomide- n/total (%)	43/300 (14.3)
Etanercept - n/total (%)	44/308 (14.3)
Rituximab- n/total (%)	16/309 (5.2)
Mycophenolate Mofetil - n/total (%)	12/306 (3.9)
Deaths - n/total (%)	99/309 (32)
Lung transplant - n/total (%)	3/309 (1)

Original and expanded GAP model (Imputed data)

Averaged over the imputed datasets, the cross-validated c-index of the continuous GAP model was 0.746 (95% CI: 0.733 – 0.756) (Table 3). We then analyzed the effect of incorporating additional predictor variables on model performance. After consideration of 6 new variables and exploration of all potential combination of variables in the candidate models, the model that performed best included a combination of the four GAP and 2 additional variables, definite UIP pattern on HRCT and a positive RF. The c-index of this new model was 0.749 (95% CI: 0.735 – 0.751). The difference between the c-index of the 2 models was 0.0029 (95% CI: 0.0006 – 0.0089). The two additional variables, definite UIP on HRCT (p = 0.06) and positive RF (p=0.10), were only borderline statistically significant in the multivariate Cox model. Last, we performed model screening without requiring the inclusion of the four GAP variables, and could not identify a model that performed better than the GAP model alone.

Table 3.

Mortality risk prediction models

	Imputed datasets (n = 309)	Complete case analysis (n = 219)
	c-index (95% CI)	c-index (95% CI)
GAP	0.746 (0.733 – 0.756)	0.753 (0.668 – 0.800)
Expanded GAP: GAP + Definite UIP + RF	0.749 (0.735 – 0.751)	0.760 (0.678 – 0.812)
Difference between the 2 models	0.0029 (0.0006 – 0.0019)	0.0072 (−0.0233 – 0.0553)

Abbreviations: GAP = gender-age-physiology; UIP = usual interstitial pneumonia; RF = rheumatoid factor

Sensitivity analysis (complete case analysis)

We repeated the same steps using data for patients with complete data. In this dataset, the c-index of the original GAP model was 0.753 (95% CI: 0.668 – 0.800) (Table 3). When looking at the effect of additional variables on the baseline GAP model, the same variables (definite UIP on HRCT and positive RF) were included in the model with the highest cross-validated c-index of 0.760 (0.678 – 0.812). However, these variables did not significantly change the discriminative ability of the model.

Final model choice and model calibration

As illustrated above, the incorporation of additional clinical variables to the GAP model did not substantially improve its ability to predict survival in patients with RA-ILD. Using the original GAP model as the final model, we tested the performance of the GAP index and staging system that categorizes patients into 3 stages according to their 1-year mortality. The calibration of the model was satisfactory in both the imputed datasets and the complete case analysis (Table 4). Figure 2 and e-Figure 1 show the cumulative mortality difference when categorized into the 3 risk groups based on imputed and complete case data, respectively. In addition, the model calibration was similar within each center (e-Table 3).

Table 4.

Model performance by GAP stage

	Imputed data		Complete case analysis
	Predicted	Observed (95% CI)	Predicted	Observed (95% CI)
1-year mortality%
Stage 1	2.1	1.6 (0.2–11.1)	2.0	0.0 (−)
Stage 2	6.6	3.2 (1.2–8.3)	6.4	4.1 (1.7–9.6)
Stage 3	17.9	22.1 (14.8–32.2)	18.9	22.9 (15.4–33.3)
2-year mortality%
Stage 1	4.4	1.6 (0.2–11.1)	4.2	0.0 (−)
Stage 2	13.7	12.7 (7.7–20.6)	13.4	14.1 (8.7–22.3)
Stage 3	34.3	36.4 (27.0–47.7)	36.0	37.5 (27.9–49.0)
3-year mortality%
Stage 1	6.3	3.5 (0.9–13.3)	6.0	1.9 (0.3–12.9)
Stage 2	19.0	17.1 (11.1–25.9)	18.8	17.6 (11.4–26.6)
Stage 3	44.9	47.4 (36.8–59.3)	46.9	50.3 (39.4–62.3)

Figure 2.

Cumulative mortality in the imputed dataset by risk category.

Discussion

In this study, we showed that the continuous GAP model, when applied to a large and diverse cohort of patients with RA-ILD, has a discrimination similar to what has been reported in patients with IPF^20,23. The addition of other clinical variables, such as definite UIP pattern on HRCT and positive RF, did not meaningfully improve the discrimination of the prediction model. The GAP Index and Staging system also had satisfactory calibration in predicting mortality at year 1, 2 and 3 in our cohort of patients with RA-ILD.

The original GAP model was initially derived and validated in a population of IPF patients and had a c-index of 0.695 (95% CI: 0.656 – 0.727)²⁰. It has since been studied in patients with scleroderma related ILD ²⁴ and other chronic ILDs ²⁵. While RA-ILD was included in the connective tissue disease (CTD) associated ILD group of a prior publication ²⁵, their contribution to the study cohort was small (n= 56 out of 281 with CTD-ILD). This study specifically looks at the performance of the GAP model in a substantially larger cohort of RA-ILD patients from several different countries. Nonetheless, the general consistency of these results across different forms of ILD suggest that they may share similar risk factors for death^14,17.

Definite UIP pattern on HRCT or surgical lung biopsy has been shown to be associated with mortality in patients with RA-ILD^19,23,26. The prevalence of a definite UIP pattern on high-resolution computed tomography (HRCT) has been reported to vary between 24 and 41%^12,18,19. When surgical lung biopsy is performed in patients with RA-ILD, UIP pattern is found in up to 56% of cases^14,27. Similar to IPF, radiologic UIP pattern is specific for the finding of UIP pattern on pathology in RA-ILD²⁸. In this study, the incorporation of the definite UIP pattern on HRCT did not significantly improve the c-index of our prediction model over the baseline GAP model, although 73 patients did not have HRCT scans available for review. Similar findings were reported in another smaller cohort of 137 RA-ILD patients. This single-center study demonstrated that UIP pattern was a predictor of survival on univariate analysis, but was not a significant predictor of mortality on multivariate analysis when adjusting for age, sex, smoking history and both baseline and change in FVC % predicted over time¹⁵.

In a previous cohort study of 309 RA patients without known ILD, baseline titers of RF were shown to be associated with mortality in a multivariate Cox model adjusting for age, sex and duration of disease²⁹. Positivity of the RF alone was not a significant predictor of mortality. The authors hypothesized higher titers of RF could be associated with a more severe pathological process. In our study, a positive RF was included in the model with the best c-statistic, although it did not substantially improve the performance of the GAP model. Unfortunately, we could not include RF titer as a candidate variable in our model given the variability in the techniques and reporting of the RF titers across the 4 centers. This difference in RF measurement and titer positivity across centers is a limitation.

The strengths of this study include analysis of patients from 4 different centers in 3 different countries, comprising a diverse, international cohort with one of the largest samples sizes to date among studies investigating RA-ILD mortality. We used multiple imputation to deal with missing data, as well as performing complete case analyses; both yielded similar results, suggesting that the findings are robust. This study does have some limitations. The data was retrospectively analyzed from ongoing longitudinal databases. In addition, all patients were selected from ILD referral centers, and may differ from the general population of patients with RA-ILD, making our findings less generalizable.

In summary, our study suggests that the GAP model can be used as a mortality risk prediction tool in patients with RA-ILD. Although the performance of the model could not be meaningfully improved with the addition of common clinical variables, the role of other biologic markers and/or genetic variants on predicting mortality in RA-ILD should be explored.

• RA-ILD is associated with high morbidity and mortality.

• The GAP model that was derived for mortality risk prediction in IPF performs similarly in patients with RA-ILD.

• The addition of other clinical variables did not meaningfully improve the performance of the model.