Defining the optimal disease duration of early diffuse systemic sclerosis for clinical trial design

Abstract

Objectives

Clinical trials in early diffuse cutaneous systemic sclerosis (SSc) using the modified Rodnan skin score (mRSS) as the primary outcome measure have most often been negative. We wanted to assess how the definition of disease onset (first SSc manifestation vs first non-Raynaud manifestation) and varying lengths of disease duration at trial entry as an inclusion criteria functioned. Our objective was to optimize trial inclusion criteria

Methods

We used the prospective, observational University of Pittsburgh Scleroderma Cohort to identify early diffuse SSc patients first evaluated between 1980 and 2015. All had <3 years from first SSc (n = 481) or first non-Raynaud manifestation (n = 514) and three or more mRSS scores. We used descriptive, survival and group-based trajectory analyses to compare the different definitions of disease onset and disease duration as inclusion criteria for clinical trials.

Results

There was no appreciable difference between using first SSc manifestation compared with first non-Raynaud manifestation as the definition of disease onset. Compared with other disease durations, <18 months of disease had >70% of patients fitting into trajectories with worsening cutaneous disease over 6 months of follow-up. Longer disease durations demonstrated the majority of patients with trajectories showing an improvement in mRSS (regression to the mean) over 6 months.

Conclusions

Regardless of whether the first SSc or first non-Raynaud manifestation is used to define disease onset, duration of <18 months at enrolment is preferable. A longer disease duration criterion more frequently results in regression to the mean of the mRSS score, and likely contributes to negative trial outcomes.

Rheumatology key messages

• Data-driven methods were used to optimize early diffuse trial inclusion criteria finding that:

• (i) Either first SSc or non-Raynaud manifestation onset can be used in early diffuse SSc trials.

• (ii) The inclusion criterion <18 months of disease minimizes the placebo mRSS regression to the mean.

Introduction

Systemic sclerosis (SSc) is a multisystem autoimmune disease with the highest case specific mortality of all connective tissue diseases [1]. SSc is divided clinically into diffuse and limited cutaneous disease, based on the presence or absence of skin thickening proximal to the elbows or knees. The cutaneous subtypes have distinct differences in natural history: diffuse SSc patients generally develop rapidly progressive skin thickening with early internal organ involvement, while limited SSc patients have minimal or slower accumulation of skin changes and non-cutaneous manifestations can appear decades after Raynaud’s phenomenon (RP) onset. Patients with early diffuse SSc are at particularly high risk for internal organ complications and mortality [2–5], and hence are a primary focus of clinical trials. Over the years, there have been dozens of agents tested in early diffuse SSc, but none have demonstrated an improvement in the most frequently used primary outcome measure, the modified Rodnan skin score (mRSS) [6]. There have been no medications to receive regulatory approval by the US Food and Drug Administration (FDA) or European Medicines Agency for the management of cutaneous disease. In most trials a reduction in the mRSS has been seen in both the placebo and intervention arms, generally attributed to regression to the mean. There are several factors possibly contributing to this failure to have positive trial results, an important one being flawed or inadequate trial design.

In 1995 a consensus document was developed by an American College of Rheumatology committee on guidelines for SSc clinical trials [7]. For diffuse SSc it was proposed that trials enrol patients with <24 months from the first symptom judged by a physician to be attributable to SSc. Subsequently, for uncertain reasons, the definition of SSc onset as the first non-RP symptom or finding became widely accepted and has been used in almost all randomized controlled trials and published observational studies.

RP is the first symptom in more than half of SSc patients [8]. Theories of SSc pathogenesis suggest that vascular injury and endothelial dysfunction are inciting, early events [9, 10]. In SSc [11, 12], as well as other fibrotic diseases such as idiopathic pulmonary fibrosis [13, 14], there are links between vasculopathy and fibrosis. Therefore, defining disease onset based on the time of the first non-Raynaud manifestation seems counter-intuitive.

It has long been our concern that assessing disease duration from the first non-Raynaud manifestation may introduce a systematic bias into diffuse SSc trials with the mRSS as a primary outcome, effectively enrolling greater numbers of patients at later disease stage when they have already passed their peak mRSS score. Thus, the stage is set to observe a reduction in mRSS score, or ‘regression to the mean’ in both the treatment and the placebo groups. As disease duration is an important, and often the first listed, inclusion criterion for early diffuse SSc clinical trials, determining the effect of different definitions of disease onset and their relationship to mRSS change is critical.

The objectives of this work were to understand how: (i) defining disease onset as first SSc manifestation or first non-Raynaud manifestation may affect mRSS changes observed in groups of patients eligible for clinical trials, and (ii) varying lengths of disease duration as inclusion criterion for clinical trials would impact patterns of mRSS change.

Methods

We used the Pittsburgh Scleroderma Center observational cohort to model different definitions of disease duration as potential clinical trial entry criteria. All data were prospectively collected. We identified all patients with diffuse SSc seen for an initial visit between 1 January 1980 and 28 February 2015. Patients had to have three or more separate visits with a mRSS recorded. One or more of the follow-up visits had to occur within 1 year of the first Pittsburgh visit. All patients had provided written informed consent at the time of enrolment into the observational cohort study, which is approved through the University of Pittsburgh Institutional Review Board and complies with the Declaration of Helsinki. Patients were not directly involved in this research, as it used previously collected data from the cohort.

At the first visit the date of onset of each involved organ system was recorded based on patient history and date of objective testing. The dating is based on a symptom followed by objective manifestation, or an objective manifestation. Definitions of involvement were as previously published [15]. At the first visit and all follow-up visits, skin involvement was quantified using the mRSS by one of only three clinicians. At each visit the interval SSc-specific history and physical examination were recorded. Interval objective imaging and laboratory studies of the cardiac, pulmonary, gastrointestinal and renal function were recorded. Baseline serological profiles were recorded. Serum was analysed by immunoprecipitation or immunodiffusion unless serum was unavailable.

Statistical analysis

We described baseline characteristics of participants according to onset of first SSc manifestation and first non-Raynaud manifestation symptom separately. Numbers and frequencies were reported for categorical variables and continuous variables were reported as means with standard deviations or medians with interquartile ranges, as appropriate. We used two statistical approaches to assess the impact of using different definitions of disease duration (i.e. <18 months from first SSc manifestation and <18 months from first non-Raynaud manifestation) on skin thickness progression. First, we conducted Kaplan–Meier survival analysis to examine the relationship of the two definitions of the disease onset, i.e. first SSc manifestation or first non-Raynaud manifestation symptom onset, to the risk of skin progression, defined as time to reach the peak mRSS. Specifically, we plotted a Kaplan–Meier curve to depict the disease progression and compared the difference in time of reaching peak mRSS using the log-rank test. In addition, we varied the duration of disease onset, i.e. presenting before and after 15 months and before or after 24 months since the first SSc and first non-Raynaud manifestations [16].

Second, we performed group-based trajectory analysis to examine the relationship of disease onset manifestation to the longitudinal changes of mRSS [17, 18]. We created analysis datasets according to 10 different potential clinical trial inclusion criteria for disease duration. Using the first SSc manifestation disease onset definition we had five inclusion criteria analysis sets: (1) <12 months, (2) <15 months, (3) <18 months, (4) <24 months, and (5) <36 months from first SSc manifestation. Using the first non-Raynaud manifestation definition we used similar subgroups: (1a) <12 months, (2a) <15 months, (3a)<18 months, (4a) <24 months, and (5a) <36 months. For each dataset, we identified trajectories of mRSS change by first determining the number of trajectory groups using a group-based trajectory model (PROC TRAJ). For each dataset, we conducted a series of trajectory analyses, i.e. modelling intercept only, linear, quadratic or cubic polynomial terms or varying the number of trajectory groups, until the best-fitting model (i.e. an optimal model) was obtained as indicated by the Bayesian information criterion. We used the posterior probabilities of group membership from each individual to assess the model fit. High probability of membership into a single group represents a good model fit. We repeated the analysis for the remaining datasets.

All statistical analyses were performed using SAS (version 9.3, SAS Institute Inc., Cary, NC, USA).

Results

Our total analysis groups included 481 patients with <3 years of SSc manifestation at first visit, and 514 with <3 years of symptoms from the first non-Raynaud manifestation. As shown in Table 1, the mean age at the first Pittsburgh visit in both groups was 49 years, 74% were female and 90–91% were Caucasian. These patients were typical of the Pittsburgh observational cohort. The median follow-up for both cohorts was >9 years, with a median of 7 and 15 clinic visits, respectively. For the SSc manifestation cohort, the most common first symptom was RP (28%); in the non-Raynaud manifestation cohort, the most common first non-Raynaud symptom was puffy fingers (29%).

Table 1.

Baseline characteristics of first SSc manifestation and first non-Raynaud manifestation symptom onset cohorts

Characteristic	First SSc manifestation <36 months (n = 481)	First non-Raynaud manifestation <36 months (n = 514)
Age at first visit, mean (s.d.), years	49.8 (3.7)	49.4 (13.8)
Female, n (%)	355 (74)	382 (74)
Caucasian, n (%)	435 (90)	468 (91)
Disease duration from first SSc manifestation, median (IQR), years	0.97 (0.66, 1.53)	1.04 (0.70, 1.84)
Follow-up time, median (IQR), years	9.6 (4.3, 16.2)	10.2 (4.5, 16.3)
Number of clinic visits, median (IQR)	7 (4, 14)	14 (8, 23)
Disease characteristics
mRSS at first visit, mean (s.d.)	24.6 (10.7)	24.5 (10.6)
Maximum mRSS during follow-up, mean (s.d.)	31.2 (11.2)	31.1 (11.1)
STPR, n (%)
Rapid	169 (35)	194 (38)
Intermediate	158 (33)	163 (32)
Slow	151 (32)	154 (30)
Tendon friction rubs present, n (%)	219 (46)	231 (46)
Gastrointestinal involvementa, n (%)	215 (45)	235 (46)
Heartburn, n (%)	275 (57)	300 (58)
Fibrosis on chest imagingb, n (%)	95/411 (23)	103/440 (23)
Interstitial lung diseasec, n (%)	126/453 (28)	135/483 (28)
Renal crisis, n (%)	42 (9)	50 (10)
Myositis, n (%)	20 (4)	22 (4)
Pulmonary hypertension, n (%)	9 (2)	9 (2)
Anti-RNA polymerase III positive, n (%)	269 (56)	273 (53)
Anti-Scl-70 positive, n (%)	121 (25)	137 (27)

^aGastrointestinal defined as hypomotility on oesophagram or manometry, small bowel distention on imaging, pseudoobtruction, antibiotics for small bowel bacterial overgrowth or heartburn plus dysphagia for solid foods. ^bPatients in the 1980s had predominantly chest radiographs; in later years they had high resolution chest CT scans. Thus this may underestimate ILD prevalence. ^cInterstitial lung disease by fibrosis on imaging or FVC predicted <70% with FEV1 predicted >80%. IQR: interquartile range; mRSS: modified Rodnan skin score; STPR: skin thickness progression rate.

The mean mRSS was slightly above 24 at presentation and peaked at 31 for both groups. The median peak mRSS occurred at 4 months (0.33 years) after the initial clinic visit. Approximately one-third of patients fell into each of the rapid, intermediate or slow skin thickness progression rate categories at presentation, and 46% had palpable tendon friction rubs [19]. The cohort characteristics of all 10 groups are presented in Supplementary Tables S1 and S2, with a direct comparison of those presenting <18 months and 18–36 months of disease in Supplementary Table S3. Follow-up times are presented in Supplementary Tables S4 and S5, all available at Rheumatology online.

Survival analysis

As recruiting patients with <12 months of disease presents an extraordinarily difficult clinical trial challenge, we examined presentations at <15 month and 15–36 months from first SSc manifestation and first non-Raynaud manifestation (Supplementary Figure S1, available at Rheumatology online) using Kaplan–Meier curves. This was followed by analysis of <12 months, 12–18 months and 18–36 months at presentation (Fig. 1A and B), and <18 months and 18–36 months (Fig. 1C and D). All of these analyses showed a significant difference between the time to mRSS between groups (P < 0.001) from disease onset. The median (IQR) time to peak mRSS was 1.2 (1.0, 1.7) years in those presenting early at <18 months from first SSc manifestation, and 2.5 (2.0, 2.9) years in those presenting with 18–36 months since their first SSc manifestations, indicating that the expected time to peak mRSS from disease onset differs according to the disease duration when presenting to a SSc centre. This indicates that including patient with a large range of disease duration in clinical trials will lead to heterogeneity of expected mRSS behaviour.

Fig. 1.

Kaplan–Meier curves for time to peak skin score by mutually exclusive disease duration groups

mRSS: modified Rodnan skin score.

Trajectory analysis

To further understand the relationship between varying disease duration as inclusion criteria and expected mRSS change, we proceeded to trajectory modelling. We performed latent trajectory modelling using the 10 different criterion groups as detailed under Methods. We modelled skin scores from the first visit through 5 years of follow-up, as <5 years of disease has been frequently used as a trial inclusion criterion.

Depicted in Fig. 2 are the mRSS score trajectory curves. All the models had an average posterior probability of membership assignment >0.8, which suggested good model fit. On first glance it is evident that at each time point the overall pattern of trajectory groups is virtually identical for first SSc manifestation (Fig. 2A) and first non-Raynaud manifestation (Fig. 2B). This suggests the two definitions of disease onset will function similarly when mRSS is the outcome measure.

Fig. 2.

Five-year mRSS trajectory plots from first SSc (A) and first non-Raynaud manifestation (B)

mRSS: modified Rodnan skin score.

All the trajectories showed five groups with similar patterns of change of mRSS over time as shown in Fig. 2. Among patients with low mrSS scores at baseline, some patients’ mrSS score declined (i.e. the low-improver group, coloured red in panels 1, 1a, 2 and 2a, and green in panels 3, 3a, 4 and 4a) and others increased and then plateaued (i.e. the low-progressor group coloured blue in panels 1, 1a, 2 and 2a and purple in panels 3, 3a, 4 and 4a). The low-improver group was the most commonly observed group, containing 26–30% of patients, whereas 10–12% of individuals fit into the low-progressor group. A third group (depicted in green in panels 1, 1a, 2 and 2a, and blue in panels 3, 3a, 4 and 4a) had a moderate mrSS score at baseline with a small increase before rapidly improving (i.e. moderate-rapid improver group) and comprised 23–24% of observed patients. The remaining groups had high mRSS scores at baseline. Of these, a strikingly similar pattern of trajectory groups is seen for <15 or <18 months of disease. The high-rapid improver group (coloured black in all panels) presented with a high mRSS that slightly increased before rapidly improving and accounted for 20–24% of fitted patients. The yellow group in all panels presented with a high mRSS score that increased further before very slowly improving (high-slow improver group) and comprised 15% of patients. This pattern changes when the inclusion criteria are expanded to include those with longer disease duration (<24 and <36 months). What is immediately evident is the appearance of a ‘severe, non-improver’ group shown in red in panels 3, 3a, 4 and 4a of Fig. 2. This group includes 8% of patients using <24 months of disease, and 9% of patients using <36 months as the inclusion criterion, regardless of disease onset definition.

Figure 3 presents a close visual comparison of the 5-year mRSS trajectory plots for the inclusion criterions of <18 and <24 months of disease from the first non-Raynaud manifestation. Detailed data for all eight different definitions of disease duration model are presented in Table 2. When <18 months is the inclusion criterion, 72% of patients fit into a trajectory where a stable or higher mRSS is observed within the first few months of enrolment. At 1 year, roughly half (52%) were in a trajectory pattern where the mRSS falls below their baseline value. This sharply contrasts with what is seen in the trajectory patterns for <24 months as an inclusion criterion. Here only 55% of patients fit into a trajectory where a stable or increase in mRSS is seen at 6 months; and by 12 months over two-thirds (68%) of patients show a mRSS decline below the first visit baseline. Using SSc manifestation as the disease onset definition, an identical pattern is seen at <18 months, where the trajectory patterns show that 70% of patients have a stable or increase in mRSS at 6 months of follow-up. When <24 months from first SSc manifestation is the inclusion criterion, half (48%) have a stable or increased mRSS at 6 months, but by 12 months of follow-up, a notable 71% of patients demonstrated a decline below their baseline mRSS value. To assess the potential effect of treatment on group trajectory placement, regression analysis was performed for the impact of mycophenolate, methotrexate and d-penicillamine as these were the most commonly used medications. The result was non-significant, suggesting that these medication is not a significant predictor of group trajectory.

Fig. 3.

Five-year follow-up mRSS trajectory plots for <18 and <24 months from first non-Raynaud manifestation at presentation

mRSS: modified Rodnan skin score.

Table 2.

Patients with stable or increasing mRSS trajectories at 6 or 12 months of follow-up

	Percentage with a stable or increasing mRSS
Inclusion criterion for disease duration	At 6 months of follow-up	At 12 months of follow-up
First SSc manifestation <15 months	71	48
Non-Raynaud manifestation <15 months	73	42
First SSc manifestation <18 months	70	47
Non-Raynaud manifestation <18 months	72	48
SSc manifestation <24 months	48	28
Non-Raynaud manifestation <24 months	55	32
First SSc manifestation <36 months	49	34
Non-Raynaud manifestation <36 months	43	43

mRSS: modified Rodnan skin score.

The trajectory analyses of observational data suggest that the inclusion criterion for disease duration should be <18 months in order to minimize the effect of regression to the mean.

Discussion

In this study we have used a large SSc observational cohort with detailed assessments of cutaneous disease to examine the effect of using two different definitions of disease onset as inclusion criteria in early diffuse SSc clinical trials. We attempted to determine the definition of disease duration that will minimize an observed decline in mRSS below the baseline value in the placebo group (regression to the mean), while trying to maximize the number of eligible patients. The data presented here and conclusions drawn are pertinent not only to trials using the mRSS as the primary outcome, but also in trials using the Combined Response Index in Systemic Sclerosis (CRISS) [20], as the mRSS contributes a significant portion of the equation used to calculate the CRISS score.

We failed to observe a difference between the first SSc manifestation and first non-Raynaud manifestation definition of disease onset on skin progression using two different modelling techniques. We therefore suggest that either of these disease onset definitions can be used for inclusion criteria when designing clinical trials for early diffuse SSc. We did not analyse the potentially differential impact that these two definitions of disease onset may have in patients with limited SSc, as mRSS is unlikely to be a primary outcome measure for these patients.

Our observations using both time to event and trajectory modelling demonstrate that the behaviour observed in mRSS score changes dramatically when the inclusion criterion is <18 months of disease, as compared with what is observed at <24 and <36 months of disease. If <18 months is used as the inclusion criterion, over 70% of patients fall into a trajectory pattern showing an increase in mRSS at 6 months, corresponding to a common clinical trial end point of change in mRSS at 24 weeks. If the inclusion criterion is extended to <24 or <36 months at enrolment, then less than half of patients would be expected to have an increase in mRSS at 6 months.

More concerning for clinical trial design is that over two-thirds of patients would have a decrease in mRSS below baseline by 12 months, another clinical trial end point. Thus, using <24 months or <36 months as an inclusion criterion would almost certainly ensure significant regression to the mean in the placebo group. Many older trials in SSc enrolled patients of <5 years of disease duration [20–30]. While we did not explore <5 years of disease duration as a specific inclusion criterion in this analysis, we believe that the observations presented here may offer partial explanations regarding the failure of those trials. We consider our results in the context of recently published early diffuse SSc trials. A primarily US-based trial of abatacept in early diffuse SSc (ASSET) enrolled patients with <36 months of disease duration [31]. In the ASSET trial there was a high percentage of RNA polymerase III positive patients (44%) similar to our cohort. However, regression to the mean was noted in the placebo group with a decline in the mRSS. In the RISE-SSc trial exploring the efficacy of riociguat in early diffuse SSc, enrolment was restricted to <18 month of disease [32]. In this trial, no regression to the mean (decline in mRSS score) was noted in the placebo group, in keeping with the results of our analysis. This was observed even in the setting of a slightly different population than our modelling group, as there was a low number of RNA polymerase III positive patients (21%), likely related to the restrictive inclusion criterion of a mRSS score of 10–22.

A question which arises is why the trajectory patterns are different when individuals presenting with >18 months of disease are included. The appearance of the trajectories suggests the interpretation that those who present later may already have an improving mRSS. Our interpretation of this data is that those presenting within 18 months of disease are more frequently experiencing ‘active’ cutaneous disease, and seemingly the most appropriate group in whom to study immunomodulatory therapies. A trajectory appears in 8–9% of patients presenting late with a high mRSS that does not improve over years of follow-up. This persistently high mRSS is not seen as a significant trajectory group in early disease, suggesting that it identifies an atypical variant of disease or represents a lost window of therapeutic opportunity in individuals with significant prolonged cutaneous fibrosis.

We chose to use time-to-event analysis plus trajectory modelling. The latter is mathematically unbiased, allowing examination of longitudinal trends over time, as is appropriate to longitudinal data. Other investigators have used regression modelling techniques to identify high risk groups for skin progression to facilitate cohort enrichment for diffuse SSc clinical trials [16, 33]. A limitation of that approach has been that a single point in time has been used to define whether or not one is a ‘skin progressor’. This method artificially divides patients into two groups, ignoring that patients can experience both an increase and decrease in the mRSS over a period of months. In our analysis we wanted to use all available data to naturally demonstrate the effect of different definitions of disease duration on mRSS patterns over longitudinal follow-up.

The limitations of the study are that it is single-centre in the USA, and that follow-up did not occur at pre-defined time points. Instead, it is purely observational in the real-world setting, with patient follow-up visit lengths being variable. The US location may limit its generalizability to studies performed entirely in Europe. However, it should be applicable to any trial with enrolment in the USA. The strengths of this study include the large number of early diffuse SSc patients with detailed follow-up, the use of only three clinicians (T.A.M., V.S., R.T.D.) performing the mRSS with the same maximum mRSS approach, and the availability of prospectively collected dating of disease onset symptoms with standardized definitions of organ system involvement. The use of two different modelling techniques that demonstrate the same results is reassuring with respect to the conclusions and an additional strength of this study.

In conclusion, there does not appear to be a discernible difference between using the first SSc manifestation as opposed to first non-Raynaud manifestation to define disease onset in early diffuse SSc trials where mRSS is a component of the primary outcome, based on our observational cohort analyses. However, there is a significant impact on the likelihood of observing a decline in mRSS if disease duration is extended beyond 18 months at trial baseline. We thereby propose that the disease duration inclusion criterion for early diffuse SSc should ideally be <18 months. This strategy will minimize the effect of regression to the mean in the mRSS in the placebo group, enhancing the likelihood that a treatment will show significant improvement.

Supplementary data

Supplementary data are available at Rheumatology online.