Abstract
Objective
Systemic sclerosis (SSc) is associated with a reduction in life expectancy, but there are no validated prognostic models for short-term mortality. The objective of this study was to derive and validate a prediction rule for two-year mortality in patients with early diffuse SSc.
Methods
We used a prospectively enrolled cohort of 387 Caucasian US patients with early diffuse SSc (< 2 years from the first symptom), randomly divided into derivation (n=260) and validation (n=187) cohorts. Predefined baseline predictor variables were placed in a stepwise multivariable logistic regression model to identify factors independently associated with two-year all-cause mortality using a cut-off of p< 0.05. We rounded the beta-weights to the nearest integer and summed to stratify patients into low, moderate and high-risk groups. We then applied this rule to an external validation cohort of 110 Caucasian early diffuse SSc patients from a single UK center and compared stratum-specific mortality using chi-square statistics.
Results
Four independent predictors (with assigned integer values) comprised the model: age at first visit (−1, 0, 1), skin thickness progression rate (0, 1), gastrointestinal tract severity (0, 1, 2) and anemia (0, 2). The model performed well with no significant differences between derivation and US or UK validation cohorts in the low and moderate risk groups of the prediction model.
Conclusion
We have derived and validated in both US and UK cohorts a 4-variable prediction rule to risk stratify two-year mortality in patients with early diffuse SSc.
Keywords: systemic sclerosis, mortality, predictive model, risk prediction
INTRODUCTION
Systemic sclerosis (SSc) is a multisystem autoimmune disease with a variable clinical course and the highest case specific mortality among the rheumatic diseases (1). Published estimates of five-year mortality range from 5–65%, but consistently report a difference in survival between patients with diffuse and limited cutaneous disease (2–11). These studies have identified various demographic features, comorbidities, medications, laboratory findings, organ system involvement and objective test results to predict mortality (1–24).
Patients with diffuse cutaneous SSc tend to accumulate internal organ involvement early in the disease, whereas those with limited cutaneous disease (lcSSc) most often develop organ involvement much later in their disease course. This difference in natural history is very important for two reasons. First, it is likely that important contributors to mortality will differ between limited and diffuse SSc, and thus these subtypes should be considered separately in mortality modeling. Second, it is very important to understand short term mortality and outcomes in the early diffuse SSc population specifically, as they are an at-risk population in the short term. However, there are few studies that have assessed short term mortality in SSc (9), and no validated prediction rule exists for short term mortality in SSc.
Currently, there is no treatment that has been shown to prolong survival or improve other outcomes in diffuse SSc using randomized, controlled trial methodology. The reasons for this may be several-fold, including trial design, the targeted patient population and choice of outcome measures. With the development of clinical trial consortiums to provide the necessary infrastructure to test new therapies in a more rigorous fashion in patients with this disease, there is a critical need to risk stratify patients for short-term outcomes (mortality or organ complications) who would be ideal candidates for therapeutic interventions. A short-term mortality prediction rule could be used to improve clinical trial design by enabling such risk stratification. Additionally, easy-to-use and accurate prediction or risk stratification models for short-term mortality in early diffuse SSc patients could also be directly applied to patient care and education.
The objective of this study was to derive and validate a clinical prediction rule to risk stratify early diffuse SSc patients for two-year mortality that could be easily used at the first patient visit. To address biases in prior studies, we used an inception cohort of patients with early diffuse SSc to develop and internally validate the two-year mortality prediction rule in a US population. We then externally validated the rule in a European SSc cohort.
PATIENTS AND METHODS
Patient Selection
Derivation and internal validation cohort
We prospectively identified an inception cohort of patients with early diffuse SSc presenting within two years from the first symptom attributable to the disease initially evaluated from January 1, 1980 to December 31, 2007 at the University of Pittsburgh Scleroderma Center. We included patients > 16 years old at diagnosis with diffuse skin thickening at the first visit, defined as skin thickening proximal to the elbows and knees. All patients were required to have a complete history, physical exam, and sufficient clinical and laboratory objective testing to accurately assess organ involvement and severity. All data were prospectively collected, with the exception of objective organ system involvement, which may have been documented prior to the first Pittsburgh visit or retrospectively reviewed. To ensure an accurate assessment of vital status, we excluded individuals who did not reside in the United States or were not US citizens. As the overall percentage of non-Caucasians in our sample was < 7%, we restricted the study population to Caucasians as we felt in the future that genetic information could be added to improve model performance. All patients had previously signed an informed consent to participate in our SSc longitudinal clinical research database and serum bank. Patients were randomized to either the derivation or validation cohort.
External validation cohort
We used the SSc population of the Royal Free Hospital in London to externally validate our prognostic model. We applied the same inclusion criteria to identify an inception cohort of adults with early diffuse SSc with an initial visit between 2000 and 2006. All of those patients provided informed consent to participate in a longitudinal SSc research database.
Baseline data
Derviation and internal validation cohort
For all Pittsburgh patients, one of three attending SSc rheumatologists (Drs. Medsger, Steen or Domsic) completed an identical initial visit data collection form, which included demographics, date of symptom onset, date of organ system involvement by objective criteria, medical history, tobacco use, physical examination findings and the following objective test results: chest radiograph or high resolution chest CT, PFTs, echocardiogram, EKG, cine esophagram or esophageal manometry, hemoglobin, ESR, serum creatine phosphokinase (CPK) and SSc-associated autoantibody.
External validation cohort
For all patients an initial visit data collection form was utilized which included age, skin score, objective GI manifestations and laboratory test results. Medical records were reviewed for any missing data.
Candidate predictor variables
We identified candidate predictor variables by reviewing the medical literature to identify factors previously associated with mortality. Candidate predictor variables were grouped into five categories: 1) patient demographics and referral characteristics (i.e., age, gender, referral area, decade of presentation (1–4,6–10,12–20)); 2) medical history (i.e., disease duration, presence of another connective tissue disease (overlap syndrome), tobacco use, hypertension, diabetes mellitus, heart disease, obesity, medication use (9,22)); 3) physical examination findings (i.e., the modified Rodnan skin score (mRss), mRss greater than 20 (25), skin thickness progression rate (STPR) calculated by dividing the mRss by the time since onset of skin thickening in years (26), presence and number of tendon or bursal friction rubs); 4) laboratory findings (anti-RNA polymerase III antibody, anti-topoisomerase I antibody, hemoglobin, erythrocyte sedimentation rate (ESR) (1,4,8–13,18,24); and 5) organ involvement (lung disease, renal crisis, cardiac, gastrointestinal, skeletal muscle, joint/tendon, pulmonary hypertension, peripheral vascular (2–4,7,10,12–16,18–19,21–24).
Organ involvement was quantitated using two classification methods: present or absent (26), or severity (12). Organ system involvement presence was categorized in the following manner: 1) lung (fibrosis on chest x-ray or high resolution chest CT or abnormal PFTs with FVC < 70% with a normal FEV1/FVC ratio); 2) renal (clinical evidence of renal crisis defined as the abrupt onset of accelerated arterial hypertension or rapidly progressive oliguric renal failure); 3) cardiac (pericarditis, myocarditis, arrhythmia requiring treatment or complete heart block); 4) gastrointestinal (distal esophageal dysmotility by esophagram or manometry, evidence of hypomotility of the duodenum or small intestine on imaging or manometry, small bowel bacterial overgrowth requiring antibiotics, gastric antral vascular ectasia, wide-mouthed colonic sacculations, pseudoobstruction, physician judgment of malabsorption syndrome, or heartburn plus distal dysphagia for solid foods); 5) muscular (proximal muscle weakness on physical exam with elevated serum CPK > 2 times normal, myopathic changes on electromyogram or abnormal muscle biopsy); 6) articular (joint swelling or contractures, palpable tendon friction rubs or joint space narrowing or erosions on radiograph); 7) pulmonary arterial hypertension (PAH) defined as mean pulmonary artery pressure of > 25 mmHg on cardiac catheterization; and 8) vascular (Raynaud phenomenon or the presence of digital pitting scars, digital tip ulceration or digital gangrene).
We classified the severity of organ involvement using the Medsger severity scale – a validated scale which grades the severity of 8 organ systems with scores ranging from 0 (no involvement) to 4 (end-stage). We used the “modified” version of this scale which was created by consensus of SSc experts (12). Given the low frequencies of high severity scores for certain organs, we classified severity of organ involvement as none, mild and moderate/severe/end-stage involvement. Lung involvement was classified and analyzed separately as a fibrosis component and a pulmonary hypertension component for two primary reasons. The pathology of these complications is different, as one is primarily fibrotic and the other a vasculopathy, and the natural history of these complications is different, particularly in patients with early diffuse scleroderma (27). Renal disease was classified as no involvement/mild disease and moderate/severe disease as there were was only one individual with mild severity. As there was low transthoracic echocardiogram ascertainment in the early years of the Pittsburgh cohort we did not use the cardiac severity index as a potential candidate variable due to concerns regarding underestimation of cardiac involvement.
Outcomes
We developed our prognostic model to predict all-cause mortality at 2 years from initial presentation at the Scleroderma Center. Vital status was confirmed as of December 31, 2009 with the U.S. Social Security Death Index for the derivation and internal validation cohort. For the external validation cohort, vital status was determined using the UK National Care Record Service.
Model derivation
We randomly divided the study population into a derivation (two-thirds) and an internal validation (one-third) cohort. Differences between the baseline characteristics of the internal validation and derivation cohort were assessed by Fisher’s exact test, chi-square, Wilcoxon or student’s t-test where appropriate. The predefined candidate predictor variables at the first visit were assessed by bivariate analysis for two-year mortality in the derivation cohort. Based upon prior literature (14, 28, 29) we categorized age into five groups (<35, 35–44, 45–54, 55–64, >65 years). All candidate predictor variables were placed into a stepwise multivariable logistic regression model with a p-value ≤ 0.05 required to remain in the model. Regression diagnostics were performed and goodness of fit assessed. To generate a simple integer point score that would be easy to calculate, the logistic regression model coefficients were rounded to the nearest 1.0. A total point score for each patient was then calculated by summing the rounded coefficients. Using the summed scores, patients were assigned one of three risk categories for two-year mortality based on distribution: low (≤ 0 points), moderate (1–2 points) or high (≥3 points). The model derivation was done in accordance with published methodologic standards for clinical risk prediction models (31).
Model validation
Internal validation
To evaluate the diagnostic performance of the initial prediction rule, a receiver operating characteristic (ROC) curve was constructed, with the area under the ROC (AUC) calculated to provide a measure of overall discriminative ability of the model (30). For internal validation, the model developed in the derivation cohort was then applied to the validation cohort and an AUC calculated.
The performance of the total integer score and associated three-level risk stratification was then evaluated using two methods. First, we compared stratum-specific mortality rates in the derivation and validation cohorts within each of the three risk classes using chi-square statistics. Next, we compared the exact AUC for predicting mortality using each of the three risk classes as cut-points in the derivation and validation cohorts.
External validation
Similar analyses were performed in the external validation cohort from the Royal Free Hospital. Multiple data imputations were used for missing data (12% of skin scores, 0.3% of hemoglobin). One hundred imputations were done for each missing data point, and then the mean of these 100 imputations used as the replacement value. Stratum-specific chi-square analysis was then performed. Statistical analysis was completed using SAS version 9.3 software (SAS, Cary, NC).
RESULTS
Derivation and internal validation cohort
Of the 2879 SSc patients first evaluated at the University of Pittsburgh Scleroderma Center between 1980 and 2007, 1304 had diffuse disease, of which 783 presented within two years of symptom onset. Of the 673 Caucasian adults with early diffuse SSc who lived in the US, 387 (58%) had the laboratory and objective testing required to assess organ system involvement at or within the 60 days following their initial visit, including all potential baseline candidate predictor variables. These 387 comprised our final study cohort (Figure 1). When the 387 patients were compared to the 287 without complete objective testing on age, gender, disease duration, skin score, referral area and two-year mortality rate, no significant differences were found. In the final population of 387, there was minimal missing data. All had complete history and physical exam variables recorded, and all had objective testing required to detect and grade the severity of organ involvement. Missing data included: ESR (19%), hemoglobin (3%), and SSc specific autoantibody (13%). We randomly divided these 387 eligible patients into derivation (n=260) and validation (n=127) cohorts.
Figure 1.
Patient population identification
External validation cohort
Of 160 early diffuse SSc patients seen for a first visit between 2000–2006, 110 were Caucasian and formed the external UK validation cohort. The only missing data required for calculation of the risk prediction rule was the skin thickness progression rate for one individual.
Baseline patient characteristics
The overall Pittsburgh cohort of 387 patients was 75% female, had a mean age of 50.6 (± 13.2) years and presented an average 1.0 (± 0.4) years after the first SSc symptom. First SSc symptom was Raynaud in 31% of patients, puffy fingers in 32%, inflammatory joint pain in 12% and carpal tunnel symptoms in 10%.
Sixty-one (16%) were current smokers and 144 (37%) lived within 100 miles of Pittsburgh (our typical referral area). At the first visit, 203 (52%) had evidence of gastrointestinal involvement, 109 (28%), lung involvement, 81 (21%) cardiac involvement and 62 (16%) renal crisis. Overall, 205 (59%) were RNA polymerase III antibody positive, and 90 (23%) were anti-topoisomerase I antibody positive. At two years, a total of 76 (19%) of patients had died.
There were no significant differences in baseline characteristics between the Pittsburgh derivation and internal validation cohort (Table 1). When the Pittsburgh derivation cohort was compared to the Pittsburgh and Royal Free validation cohorts, there were no significant differences in demographics, disease duration, coexistent hypertension or CAD, skin involvement, organ system involvement, hemoglobin or autoantibody serologies among all three patient groups (Table 1). There were significantly lower rates of diabetes in the Royal Free cohort, but slightly increased rates of other connective tissue disease and elevated ESR. Tendon friction rubs, BMI and cardiac involvement based on arrhythmia were not consistently available for the Royal Free cohort, and thus could not be assessed. There was no difference in referral area between the derivation and internal validation cohort at Pittsburgh (p=0.26), or in the decade of diagnosis (p=0.10; data not shown). These did not apply to the Royal Free cohort given their geographic and time distributions.
Table 1.
Comparison of baseline characteristics in the derivation, internal and external validation cohorts
| Characteristics | Pittsburgh Derivation cohort (n=260) |
Pittsburgh Validation cohort (n=127) |
Royal Free Validation cohort (n=110) |
p-value | |
|---|---|---|---|---|---|
| Demographics | |||||
| Mean age in years at the first visit (±SD) | 50.5 ± 13.2 | 50.9 ± 13.3 | 51.2 ± 12.4 | 0.95 | |
| Gender (female) | 195 (75%) | 97 (76%) | (74%) | 0.30 | |
| History | |||||
| Disease duration at first visit in years (IQR) | 0.97 (0.68, 1.33) |
0.88 (0.64, 1.31) |
1.02 (0.78, 1.49) |
0.19 | |
| Hypertension | 11 (4%) | 10 (8%) | 4 (4%) | 0.23 | |
| Coronary artery disease | 4 (2%) | 1 (1%) | 2 (2%) | 0.77 | |
| Diabetes mellitus | 6 (2%) | 8 (6%) | 0 (0%) | 0.01 | |
| Overlapping connective tissue disease | 9 (3%) | 4 (3%) | 11 (10%) | 0.02 | |
| Tobacco use: | none | 125 (48%) | 68 (54%) | 23 (21%)* | 0.01 |
| prior use | 94 (36%) | 38 (30%) | 11 (10%) | ||
| current use | 41 (16%) | 20 (16%) | 19 (17%) | ||
| Physical examination findings | |||||
| Mean modified Rodnan skin score (SD) | 26.8 ± 11.9 | 25.2 ± 11.1 | 26.6 ± 8.1 | 0.40 | |
| Skin thickness progression rate slow(<25) | 79 (31%) | 43 (34%) | 39 (35%) | 0.74 | |
| intermediate (25–45) | 91 (35%) | 36 (29%) | 37 (34%) | ||
| rapid (>45) | 90 (35%) | 47 (37%) | 34 (31%) | ||
| Tendon friction rubs (present or absent) | 154 (59%) | 67 (53%) | 27(21%)* | <0.001 | |
| Number of tendon friction rubs | 0 | 101 (39%) | 58 (46%) | 83 (79%) | 0.30 |
| 1–3 | 79 (30%) | 40 (32%) | 15 (14%) | ||
| > 3 | 80 (31%) | 28(22%) | 7 (6%) | ||
| Mean body mass index | 23.9 ± 4.1 | 24.7 ± 4.0 | ---- | ---- | |
| Internal organ system involvement | |||||
| Pulmonary | 70 (27%) | 39 (31%) | 34 (29%) | 0.64 | |
| Renal crisis | 47 (18%) | 15 (12)% | 17 (16%) | 0.28 | |
| Cardiac | 60 (23%) | 21 (17%) | ---- | ---- | |
| Gastrointestinal | 133 (51%) | 70 (56%) | 57 (50%) | 0.76 | |
| Skeletal muscle | 17 (7%) | 11 (9%) | 5 (4.5%) | 0.45 | |
| Pulmonary hypertension | 3 (1%) | 0 | 0 | 0.25 | |
| Laboratory findings | |||||
| Anti-topoisomerase I antibody | 56 (22%) | 34 (28%) | 28 (27%) | 0.47 | |
| Anti-RNA polymerase III antibody | 147 (63%) | 58 (52%) | 66 (61%) | 0.06 | |
| Elevated erythrocyte sedimentation rate‡ | 113 54%) | 45 (54%) | 82 (74%) | <0.001 | |
| Anemia† | 115 (56%) | 61 (49%) | 42 (38%) | 0.31 | |
43% missing data for tobacco use;
defined as ESR > (age + 10)/2 for females; ESR > (age/2) for males
calculation does not include arrythmia
defined as hgb <12 mg/dL
Survival
There was a significant difference (p=0.04) in the 2-year mortality between the Pittsburgh derivation (22%) and internal validation cohorts (12%) as well as the Pittsburgh derivation (22%) and Royal Free external validation cohorts (7%; p= 0.0005).
Derivation of the prediction rule
In bivariate analyses, we identified 18 baseline variables significantly associated with two-year mortality (Table 2), including two demographic (age, gender), two history (referral area, decade of presentation), four physical exam (mRss, STPR, presence and number of tendon friction rubs), three laboratory (anti-topoisomerase I positivity, anti-RNA polymerase III positivity, anemia), the presence or absence of involvement of four organ systems (lung, renal crisis, cardiac and GI), and severity of involvement of four organ systems (lung, GI, skeletal muscle and skin). The odds ratios for these variables were either protective (OR range 0.43 – 0.78) or indicated increased risk (OR 1.40–4.79).
Table 2.
Bivariate Associations of baseline characteristics and 2-year mortality in the Pittsburgh derivation cohort (N=260).
| Characteristics | Odds Ratio |
95% Confidence Interval |
p-value | ||
|---|---|---|---|---|---|
| Demographics | |||||
| Age | < 0.0001 | ||||
| <35 | 0.75 | 0.22 – 2.50 | |||
| 35–44 | 0.16 | 0.15 – 1.29 | |||
| 45–54 | 1.00 | -- | |||
| 55–64 | 2.08 | 0.88 – 4.86 | |||
| > 65 | 4.63 | 1.99 – 10.77 | |||
| Male | 1.84 | 0.97 – 3.46 | 0.06 | ||
| History | |||||
| Hypertension | 0.77 | 0.16 – 3.65 | 0.74 | ||
| Coronary artery disease | 3.57 | 0.49 – 25.93 | 0.21 | ||
| Diabetes mellitus | 0.69 | 0.08 – 6.04 | 0.74 | ||
| Overlap with another connective tissue disease | 0.99 | 0.20 – 4.92 | 0.99 | ||
| Tobacco use | 0.99 | ||||
| prior vs none | 1.00 | 0.52 – 1.89 | |||
| current vs none | 0.97 | 0.42 – 2.28 | |||
| Prednisone use | never | 0.66 | |||
| prior to first visit | 0.76 | 0.42 – 1.38 | |||
| prescribed at first visit | 0.88 | 0.17 – 4.43 | |||
| Referral area (<100 miles from Pittsburgh) | 2.42 | 1.33 – 4.39 | 0.004 | ||
| Decade of diagnosis | 0.03 | ||||
| 1990s vs 1980s | 0.44 | 0.23 – 0.84 | |||
| 2000s vs 1980s | 0.43 | 0.15 – 1.20 | |||
| Physical examination findings | |||||
| Modified Rodnan skin score | 1.03 | 1.00 – 1.05 | 0.03 | ||
| Skin score > 20 (yes/no) | 1.49 | 0.80 – 2.77 | 0.21 | ||
| Skin thickness progression rate | slow | 0.02 | |||
| intermediate | 0.78 | 0.35 – 1.73 | |||
| rapid | 2.02 | 0.99 – 4.15 | |||
| Tendon friction rubs (present or absent) | 2.95 | 1.50 – 5.81 | 0.001 | ||
| Number of tendon friction rubs | 0.002 | ||||
| 1–3 vs 0 | 2.35 | 1.06 – 5.19 | |||
| > 3 vs 0 | 3.78 | 1.77 – 8.08 | |||
| Mean body mass index | 0.94 | 0.86 – 1.03 | 0.81 | ||
| Laboratory findings | |||||
| Anti-topoisomerase I antibody | 2.06 | 1.06 – 4.01 | 0.03 | ||
| Anti-RNA polymerase III antibody | 0.48 | 0.25 – 0.90 | 0.02 | ||
| Elevated sedimentation rate (ESR)*, | 0.83 | 0.43 – 1.62 | 0.59 | ||
| Anemia (present or absent) | Hgb < 12 mg/dL | 4.79 | 2.50 – 9.15 | < 0.0001 | |
| Degree of anemia | < 0.0001 | ||||
| Hgb > 12 mg/dL | 1.00 | ||||
| Hgb 10–12 mg/dL (mild) | 4.29 | 2.03 – 9.06 | |||
| Hgb < 10 mg/dL (severe) | 4.09 | 1.92 – 8.68 | |||
| Organ System Involvement (present/absent) | |||||
| Lung | 2.39 | 1.29 – 4.44 | 0.006 | ||
| Renal crisis | 1.63 | 0.80 – 3.31 | 0.17 | ||
| Cardiac | 2.93 | 1.55 – 5.53 | 0.0009 | ||
| Gastrointestinal tract | 2.14 | 1.17 – 3.93 | 0.01 | ||
| Joint/tendon | 1.00 | ||||
| Skeletal muscle | 1.49 | 0.50 – 4.43 | 0.47 | ||
| Pulmonary hypertension | 1.76 | 0.16 – 19.71 | 0.65 | ||
| Modified Medsger severity score | |||||
| Interstitial lung disease | none | 0.001 | |||
| mild/moderate | 1.51 | 0.71 – 3.20 | |||
| severe/endstage | 5.44 | 2.17 – 13.61 | |||
| Gastrointestinal tract | none | 0.001 | |||
| mild/moderate | 1.59 | 0.78 – 3.27 | |||
| severe/endstage | 4.07 | 1.92 – 8.62 | |||
| Joint/tendon | none | -- | 0.74 | ||
| mild/moderate | 0.91 | 0.28 – 2.93 | |||
| severe/endstage | 1.23 | 0.34 – 4.33 | |||
| Skeletal muscle | none | < 0.0001 | |||
| mild/moderate | 2.05 | 1.01 – 4.12 | |||
| severe/endstage | 7.57 | 2.96 – 19.34 | |||
| Peripheral vascular | none | 0.66 | |||
| mild | 1.23 | 0.53 – 2.82 | |||
| moderate | 1.36 | 0.65 – 2.83 | |||
| severe/endstage | 1.71 | 0.70 – 4.17 | |||
| Skin | none | 0.16 | |||
| mild | 0.88 | 0.35 – 2.17 | |||
| moderate | 1.17 | 0.43 – 3.21 | |||
| severe | 2.06 | 0.78 – 5.44 | |||
ESR:greater than age/2 for males or greater than age + 10 years/2 for females
Hgb = hemoglobin in mg/dL
In multivariable analysis, four factors were independently associated with two year mortality: (1) age at first visit; (2) STPR; (3) GI severity score; and (4) anemia (Table 3). The AUC for the derivation cohort was 0.82 (95% CI 0.76 – 0.88).
Table 3.
Multivariable associations of baseline characteristics and 2 year mortality in the Pittsburgh derivation cohort (N=260).
| Characteristics | β | Odds Ratio |
95% Confidence Interval |
p-value | Points Assigned |
|
|---|---|---|---|---|---|---|
| Age at first visit (years) | 0.01 | |||||
| < 35 | -0.64 | 0.53 | 0.13 – 2.15 | -1 | ||
| 35–44 | -0.90 | 0.41 | 0.13 – 1.31 | -1 | ||
| 45–54 | -- | 1.00 | 0 | |||
| 55–64 | 0.51 | 1.67 | 0.63 – 4.40 | 1 | ||
| >65 | 0.98 | 2.66 | 1.02 – 6.93 | 1 | ||
| Skin thickness progression rate* | 0.03 | |||||
| slow | -- | 0 | ||||
| intermediate | 0.17 | 1.19 | 0.47 – 3.02 | 0 | ||
| rapid | 1.07 | 2.90 | 1.23 – 6.88 | 1 | ||
| Medsger GI severity† | 0.001 | |||||
| none | -- | 0 | ||||
| mild/moderate | 0.59 | 1.80 | 0.78 – 4.14 | 1 | ||
| severe/endstage | 1.76 | 5.83 | 2.24– 15.20 | 2 | ||
| Anemia | <0.0001 | |||||
| ≥ 12 mg/dL | -- | 0 | ||||
| <12 mg/dL | 1.64 | 5.15 | 2.42 – 10.97 | 2 | ||
Skin thickness progression rate is defined as slow (< 25), intermediate (25–45) and rapid (> 45) based on prior publication (Domsic 2011)
Medsger GI severity: none = no involvement; mild/moderate = distal esophageal hypoperistalsis and/or abnormal small bowel series, and/or antibiotics required for bacterial overgrowth; severe/endstage = malabsoprtion and/or episodes of pseudo-obstruction and/or hyperalimentation required.
To develop a prediction rule that could be used easily in clinical practice, we assigned points by rounding the beta values for the significant predictors as shown in Table 3. When the total points were summed in the derivation population, 63 patients (25%) were low risk (≤0 points) with a mortality rate of 1.6%; 108 (43%) were intermediate risk (1–2 points) with a mortality rate of 14.8%, and 81 (32%) were high risk (≥3 points) with a mortality rate of 49.4% (Table 4).
Table 4.
Comparison of risk class specific 2-year mortality in the derivation and validation cohorts
| Risk Class (sum of points) |
Pittsburgh derivation cohort (n=252) |
Pittsburgh internal validation cohort (n=126) |
p-value | Royal Free external validation cohort (n=110) |
p-value | |||
|---|---|---|---|---|---|---|---|---|
| N | Deceased (%) |
N | Deceased (%) |
N | Deceased (%) |
|||
| Low (≤ 0) | 63 | 1.6 | 28 | 0.0 | 0.50 | 33 | 3.0 | 0.64 |
| Moderate (1–2) | 108 | 14.8 | 50 | 16.0 | 0.85 | 61 | 8.2 | 0.21 |
| High (≥3) | 81 | 49.4 | 48 | 16.6 | 0.0002 | 16 | 12.5 | 0.006 |
Validation of the prediction rule
Internal validation
In stratum-specific analysis comparing the Pittsburgh derivation and internal validation cohorts, there was no difference in mortality between the low (p=0.50) or moderate risk classes (p=0.85), but there was a significant difference in the high risk class (p=0.002). When this three risk class stratification method was used there was no significant difference in the area under the ROC between the derivation cohort (AUC =0.65) and the validation cohort (AUC = 0.63; 95% CI 0.52 – 0.74).
External validation
In the stratum-specific analysis comparing the Pittsburgh derivation cohort and the Royal Free cohort, there was no difference in mortality in the low (p=0.64) or moderate risk classes (p=0.21), but there was a significant difference in the high risk class (p=0.006). There was no significant difference in the area under the ROC curve between the Pittsburgh derivation and the external validation cohort (AUC =0.62, 95% CI 0.44 – 0.88).
DISCUSSION
We derived and internally and externally validated a prediction model to risk-stratify patients with early diffuse SSc for short-term (two-year) mortality. To develop this model, we used an inception cohort of patients, and carefully applied methodologic standards for risk prediction rule development. The result is a simple, four-factor model that can be easily summed to accurately risk stratify patients into low and moderate risk for mortality at the time of their first visit.
In our derivation cohort, the initial prediction rule performed well with a high AUC of 0.82. In an attempt to make the model easy to use by assigning an integer score identifying three distinct risk classes (low, moderate and high risk) there was, as expected, a reduction in the discriminatory performance of the two-year mortality model with an estimated AUC of 0.65 in the derivation cohort. The method of assigning integer scores into three risks classes performed equally well in both validation cohorts with an AUC of 0.64 in the Pittsburgh internal validation and 0.62 in the Royal Free external validation cohort. This validates the discriminative capacity of the model to identify three distinct risk classes for two-year mortality in early diffuse SSc patients.
Closer inspection shows that this model predicts almost identical mortality rates in the derivation and two validation cohorts for the low risk group, and the anticipated higher rates in the moderate and high risk groups. Unfortunately, despite randomization, the baseline rate of mortality was statistically different in the derivation and validation cohorts, with the internal validation cohort having 10% absolute fewer deaths, and the external validation cohort 15% fewer absolute deaths. A difference in the outcome rates between the cohort populations is expected to reduce the performance of the absolute risk prediction model in the validation cohort, as is seen in our results. The phenomenon of validation cohorts having lower rates of the outcomes of interest than derivation cohorts, leading to poorer absolute risk prediction rule performance when applied to the validation cohort, has been demonstrated in prior clinical risk prediction rule assessments (32). Thus, in this case, the lower mortality rate in the validation cohort likely explains the significant difference in the predictive performance of the stratum-specific high risk group between the derivation and validation cohorts. However, it is encouraging that the model performs very well in identifying low and moderate risk individuals for short-term mortality despite the disparate mortality rates in the derivation and two separate validation cohorts. This suggests that the model can be used in populations with variable mortality.
There has been very little information published on short-term mortality in SSc patients. One recent study (19) examined short-term mortality in SSc from the EUSTAR cohort of 5,860 prevalent SSc cases. The cohort had a mean follow-up of 0.9 years, included diffuse and limited SSc patients of widespread disease duration, and reported a 5.4% mortality. Similar to our study, the authors found age to be an independent predictor of mortality, but not gender. Important prognostic variables in their diverse study sample which were not significant in our model included proteinuria, measurements of pulmonary function, presence of PH and skin thickness score. STPR and GI severity, both important in our model, were not tested, and model validation was not reported.
With a prevalent population such as EUSTAR’s there are potential biases to be considered. First, patients who die early in their disease course may not have been included leading to left censorship bias. Second, populations including either limited cutaneous SSc and/or disproportionately longstanding disease are likely to have different predictors for short-term mortality than for early diffuse disease. These differences in methodology may, in part, explain the disparate results in significant predictors of short-term mortality. Two other validated mortality models have focused on longer-term mortality at 5 years (33, 34). Also, in these instances, prevalent populations including both limited and diffuse SSc patients were included, which may have introduced multiple left-censoring biases.
Although we used all prospectively collected cohort data, our derivation and internal validation cohorts contained complete data on demographics, history, examination and objective tests, there are potential limitations. First, the model was developed at a single, tertiary care scleroderma center, and validated at two tertiary care scleroderma centers. This may limit generalizability to an extent, although we have sought to assess the generalizability of the model by choosing a cohort from a different continent with good results. Second, this model was developed in Caucasian SSc patients only, and will need to be evaluated in more racially diverse populations. Third, our final cohorts had somewhat small numbers of early diffuse SSc patients. Although our initial comparisons showed that there was no difference between those with and without objective data available, it is possible that significant unmeasured differences exist in these two patient groups which could have affected model performance.
Conclusions
We derived and validated in an American and European population a simple prediction rule to accurately identify early diffuse SSc patients who are at low to moderate risk for death within two years of their first visit to a scleroderma center. This rule can be used by clinicians at the first evaluation to risk stratify patients for early mortality when they weigh therapy decisions, and by researchers to identify appropriate at-risk populations for inclusion or exclusion criteria for clinical trials. This model holds considerable promise as it is currently the only validated short-term mortality model for early diffuse SSc.
Acknowledgements
Financial support: Dr. Domsic was supported by a National Institutes of Health Award (NIAMS K23 AR057845).
The authors thank Ms. Candi Wills for her assistance with the manuscript.
Footnotes
The authors have no conflicts of interest.
Author Contributions: RTD: conception and design of study, data acquisition, analysis and manuscript writing. SN: data acquisition, analysis and manuscript writing. SW, MJF and CKK: conception, analysis and manuscript writing. MRL: data acquisition. CPD: data acquisition and manuscript writing. TAM: conception, data acquisition and interpretation of results, and manuscript writing.
REFERENCES
- 1.Bryan C, Howard Y, Brennan P, Black C, Silman A. Survival following the onset of scleroderma: results from a retrospective inception cohort study of the UK patient population. Br J Rheumatol. 1996;35:1122–1126. doi: 10.1093/rheumatology/35.11.1122. [DOI] [PubMed] [Google Scholar]
- 2.Barnett AJ, Miller MH, Littlejohn GO. A survival study of patients with scleroderma diagnosed over 30 years (1953–1983): the value of a simple cutaneous classification in the early stages of the disease. J Rheumatol. 1988;15:276–283. [PubMed] [Google Scholar]
- 3.Bennett R, Bluestone R, Holt PJ, Bywaters EG. Survival in scleroderma. Ann Rheum Dis. 1971;30:581–588. doi: 10.1136/ard.30.6.581. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Czirjak L, Kumanovics G, Varju C, Nagy Z, Pakozdi A, Szekanecz Z, et al. Survival and causes of death in 366 Hungarian patients with systemic sclerosis. Ann Rheum Dis. 2008;67:59–63. doi: 10.1136/ard.2006.066340. [DOI] [PubMed] [Google Scholar]
- 5.Ferri C, Valentini G, Cozzi F, Sebastiani M, Michelassi C, La Montagna G, et al. Systemic sclerosis: demographic, clinical, and serologic features and survival in 1,012 Italian patients. Medicine (Baltimore) 2002;81:139–153. doi: 10.1097/00005792-200203000-00004. [DOI] [PubMed] [Google Scholar]
- 6.Hesselstrand R, Scheja A, Akesson A. Mortality and causes of death in a Swedish series of systemic sclerosis patients. Ann Rheum Dis. 1998;57:682–686. doi: 10.1136/ard.57.11.682. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Nagy Z, Czirjak L. Predictors of survival in 171 patients with systemic sclerosis (scleroderma) Clin Rheumatol. 1997;16:454–460. doi: 10.1007/BF02238937. [DOI] [PubMed] [Google Scholar]
- 8.Scussel-Lonzetti L, Joyal F, Raynauld JP, Roussin A, Rich E, Goulet JR, et al. Predicting mortality in systemic sclerosis: analysis of a cohort of 309 French Canadian patients with emphasis on features at diagnosis as predictive factors for survival. Medicine (Baltimore) 2002;81:154–167. doi: 10.1097/00005792-200203000-00005. [DOI] [PubMed] [Google Scholar]
- 9.Wynn J, Fineberg N, Matzer L, Matzer L, Cortada X, Armstrong W, et al. Prediction of survival in progressive systemic sclerosis by multivariate analysis of clinical features. Am Heart J. 1985;110:123–127. doi: 10.1016/0002-8703(85)90525-3. [DOI] [PubMed] [Google Scholar]
- 10.Farmer RG, Gifford RW, Jr, Hines EA., Jr Prognostic significance of Raynaud's phenomenon and other clinical characteristics of systemic scleroderma. A study of 271 cases. Circulation. 1960;21:1088–1095. doi: 10.1161/01.cir.21.6.1088. [DOI] [PubMed] [Google Scholar]
- 11.Tuffanelli DL, Winkelmann RK. Diffuse systemic scleroderma. A comparison with acrosclerosis. Ann Intern Med. 1962;57:198–203. doi: 10.7326/0003-4819-57-2-198. [DOI] [PubMed] [Google Scholar]
- 12.Medsger TA, Jr, Bombardieri S, Czirjak L, Scorza R, Della Rossa A, Bencivelli W. Assessment of disease severity and prognosis. Clin Exp Rheumatol. 2003;21:S42–S46. [PubMed] [Google Scholar]
- 13.Altman RD, Medsger TA, Jr, Bloch DA, Michel BA. Predictors of survival in systemic sclerosis (scleroderma) Arthritis Rheum. 1991;34:403–413. doi: 10.1002/art.1780340405. [DOI] [PubMed] [Google Scholar]
- 14.Jacobsen S, Halberg P, Ullman S. Mortality and causes of death of 344 Danish patients with systemic sclerosis (scleroderma) Br J Rheumatol. 1998;37:750–755. doi: 10.1093/rheumatology/37.7.750. [DOI] [PubMed] [Google Scholar]
- 15.Lee P, Langevitz CA, Alderdice M, Aubrey M, Baer PA, Baron M, et al. Mortality in systemic sclerosis (scleroderma) Q J Med. 1992;82:139–148. [PubMed] [Google Scholar]
- 16.Simeon CP, Armadans L, Fonollosa R, Solans R, Selva A, Villar M, et al. Mortality and prognostic factors in Spanish patients with systemic sclerosis. Rheumatology (Oxford) 2003;42:71–75. doi: 10.1093/rheumatology/keg033. [DOI] [PubMed] [Google Scholar]
- 17.Mayes MD. Scleroderma epidemiology. Rheum Dis Clin North Am. 2003;29:239–254. doi: 10.1016/s0889-857x(03)00022-x. [DOI] [PubMed] [Google Scholar]
- 18.Ioannidis JP, Vlachoyiannopoulos PG, Haidich AB, Haidich AB, Medsger TA, Jr, Lucas M, Michet CJ, et al. Mortality in systemic sclerosis: an international meta-analysis of individual patient data. Am J Med. 2005;118:2–10. doi: 10.1016/j.amjmed.2004.04.031. [DOI] [PubMed] [Google Scholar]
- 19.Tyndall AJ, Bannert B, Vonk M, Airo P, Cozzi F, Carreira PE, et al. Causes and risk factors for death in systemic sclerosis: a study from the EULAR scleroderma trials and research (EUSTAR) database. Ann Rheum Dis. 2010;69:1809–1815. doi: 10.1136/ard.2009.114264. [DOI] [PubMed] [Google Scholar]
- 20.Medsger TA, Jr, Masi AT. Survival with Scleroderma II. A life-table analysis of clinical and demographic factors in 358 male U.S. veteran patients. J Chronic Dis. 1973;26:647–660. doi: 10.1016/0021-9681(73)90054-4. [DOI] [PubMed] [Google Scholar]
- 21.Laing TJ, Gillespie BW, Toth MB, Mayes MD, Gallavan RH, Jr, Burns CJ, et al. Racial differences in scleroderma among women in Michigan. Arthritis Rheum. 1997;40:734–742. doi: 10.1002/art.1780400421. [DOI] [PubMed] [Google Scholar]
- 22.Assassi S, Del Junco D, Sutter K, McNearney TA, Reveille JD, Karnavas A, et al. Clinical and genetic factors predictive of mortality in early systemic sclerosis. Arthritis Rheum. 2009;61:1403–1411. doi: 10.1002/art.24734. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Eason RJ, Tan PL, Gow PJ. Progressive systemic sclerosis in Auckland: a ten year review with emphasis on prognostic features. Aust N Z J Med. 1981;11:657–662. doi: 10.1111/j.1445-5994.1981.tb03542.x. [DOI] [PubMed] [Google Scholar]
- 24.Nishioka K, Katayama I, Kondo H, Shinkai H, Ueki H, Tamaki K, et al. Epidemiologic analysis of prognosis of 496 Japanese patients with progressive systemic sclerosis (SSc). Scleroderma Research Committee Japan. J Dermatol. 1996;23:677–682. doi: 10.1111/j.1346-8138.1996.tb02680.x. [DOI] [PubMed] [Google Scholar]
- 25.Clements PJ, Lachenbruch PA, Seibold JR, Zee B, Steen VD, Brennan P, et al. Skin thickness score in systemic sclerosis: an assessment of interobserver variability in 3 independent studies. J Rheumatol. 1993;20:1892–1896. [PubMed] [Google Scholar]
- 26.Domsic RT, Rodriguez-Reyna T, Lucas M, Fertig N, Medsger TA., Jr Skin thickness progression rate: a predictor of mortality and early internal organ involvement in diffuse scleroderma. Ann Rheum Dis. 2011;70:104–109. doi: 10.1136/ard.2009.127621. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Silver RM, Medsger TA, Jr, Bolster MB. Systemic sclerosis and scleroderma variants: clinical aspects. In: Koopman WJ, Moreland LW, editors. Arthritis and Allied Conditions: A Textbook in Rheumatology. 15th edn. Philadelphia: Lippincott Williams & Wilkins; 2005. pp. 1633–1680. [Google Scholar]
- 28.Silman AJ. Scleroderma and survival. Ann Rheum Dis. 1991;50:267–269. doi: 10.1136/ard.50.4.267. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Krishnan E, Furst DE. Systemic sclerosis mortality in the United States: 1979–1998. Eur J Epidemiol. 2005;20:855–861. doi: 10.1007/s10654-005-2210-5. [DOI] [PubMed] [Google Scholar]
- 30.Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36. doi: 10.1148/radiology.143.1.7063747. [DOI] [PubMed] [Google Scholar]
- 31.Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. JAMA. 1997;277:488–494. [PubMed] [Google Scholar]
- 32.Matheny M, McPheeters ML, Glasser A, Mercaldo N, Weaver RB, McKoy JN, et al. Evidence Synthesis No. 85. Rockville (MD): Agency for Healthcare Research and Quality; 2011. Systemic review of cardiovascular disease risk assessment tools. [PubMed] [Google Scholar]
- 33.Beretta L, Santaniello A, Cappiello F, Chawala NV, Vonk MC, Carreira PE, et al. Development of a five-year mortality model in systemic sclerosis patients by different analytical approaches. Clin Exp Rheumatol. 2010;2:S18–S27. [PubMed] [Google Scholar]
- 34.Fransen J, Popa-Diaconu D, Hesselstrand R, Carreira P, Valentini G, Beretta L, et al. Clinical prediction of 5-year survival in systemic sclerosis: validation of a simple prognostic model in EUSTAR centres. Ann Rheum Dis. 2011;70:1788–1792. doi: 10.1136/ard.2010.144360. [DOI] [PubMed] [Google Scholar]