Predictive model to identify multiple failure to biological therapy in patients with rheumatoid arthritis

Abstract

Background:

Despite advances in the treatment of rheumatoid arthritis (RA) and the wide range of therapies available, there is a percentage of patients whose treatment presents a challenge for clinicians due to lack of response to multiple biologic and target-specific disease-modifying antirheumatic drugs (b/tsDMARDs).

Objective:

To develop and validate an algorithm to predict multiple failure to biological therapy in patients with RA.

Design:

Observational retrospective study involving subjects from a cohort of patients with RA receiving b/tsDMARDs.

Methods:

Based on the number of prior failures to b/tsDMARDs, patients were classified as either multi-refractory (MR) or non-refractory (NR). Patient characteristics were considered in the statistical analysis to design the predictive model, selecting those variables with a predictive capability. A decision algorithm known as ‘classification and regression tree’ (CART) was developed to create a prediction model of multi-drug resistance. Performance of the prediction algorithm was evaluated in an external independent cohort using area under the curve (AUC).

Results:

A total of 136 patients were included: 51 MR and 85 NR. The CART model was able to predict multiple failures to b/tsDMARDs using disease activity score-28 (DAS-28) values at 6 months after the start time of the initial b/tsDMARD, as well as DAS-28 improvement in the first 6 months and baseline DAS-28. The CART model showed a capability to correctly classify 94.1% NR and 87.5% MR patients with a sensitivity = 0.88, a specificity = 0.94, and an AUC = 0.89 (95% CI: 0.74–1.00). In the external validation cohort, 35 MR and 47 NR patients were included. The AUC value for the CART model in this cohort was 0.82 (95% CI: 0.73–0.9).

Conclusion:

Our model correctly classified NR and MR patients based on simple measurements available in routine clinical practice, which provides the possibility to characterize and individualize patient treatments during early stages.

Background

Biologic and target-specific disease-modifying antirheumatic drugs (b/tsDMARDs) have demonstrated their effectiveness in the treatment of rheumatoid arthritis (RA), playing a major role in transforming outcomes in RA, with positive effects on remission rates, joint damage, radiographic progression, and patient’s quality of life.¹

International guidelines/recommendations² are being constantly updated due to the increasing availability of treatments, as well as treat-to-target strategies.³ This allows rheumatologists to optimally manage RA patients, and using the current therapeutic options/strategies available, treatment targets can be achieved in most patients. However, about 20–30% of patients fail to respond to treatment with first tumor necrosis factor inhibitor (TNFi)⁴ during the first 6 months, and at least, 12% of those who receive a second bDMARD discontinue treatment due to inefficacy.⁵

The lack of clinical response in some patients receiving second-line treatments, especially in those in whom the established therapeutic targets repeatedly fail, has led to the concept of difficult-to-treat rheumatoid arthritis (D2TRA). Although an increasing number of studies have tried to define and classify this particular group of patients, data about patients who experience multiple failure to biologics remains scarce. According to recent studies, the prevalence of refractory RA ranges between 5% and 20%, which represents a considerable percentage of patients that must be take into account in the treatment approaches used in daily clinical practice.^6–10

Recently, the European League Against Rheumatism (EULAR) proposed the definition of D2TRA.¹¹ Among the criteria considered in this definition was multi-drug resistance, understood as a failure to at least two bDMARDs or tsDMARDs during the course of the disease. In a previous study,¹⁰ carried out in parallel to the EULAR definition of D2TRA and using other studies in the same field as references,^6–9 approximately, 10% of patients of our RA cohort developed multiple failures to biologics. In the above-mentioned study, some risk factors associated with the development of multi-drug resistance were identified, namely, being younger at bDMARD initiation, having higher baseline disease activity score-28 (DAS-28), the presence of erosions, and poorer early response during the first 6 months of treatment with bDMARDs. Thanks to these results, and noting that these characteristics were eminently clinical and easy to assess in clinical practice, we wanted to further investigate how to not only identify risk factors, but also to advance our knowledge of this phenomenon and provide a framework for the classification and identification of these patients from early stages of treatment with the first b/tsDMARDs. In this sense, the aim of this study was to establish a simple and reproducible tool to predict multiple failure to b/tsDMARDs based on clinical characteristics through more elaborate statistical models, such as a classification and regression tree (CART).

Patients and methods

This study involved subjects with RA from a prospective cohort of patients drawn from the Rheumatoid Arthritis Registry at La Paz University Hospital between January 2000 and December 2020. Ethical approval was obtained from the La Paz Ethics Committee (PI-1155). Written informed consent was obtained from all participants and, in addition, participants’ data are anonymized and their identity cannot be ascertained.

The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement guidelines for reporting observational studies were followed¹² (supplementary material).

The ‘Rheumatoid Arthritis La Paz University Hospital’ (RA-Paz) Registry is a database of all patients who have received, or who are receiving, treatment with bDMARDs and tsDMARDs. This database enables rheumatologists to input clinical information on RA patients from the beginning of their b/tsDMARD treatment and during follow-up, monitoring clinical response, and adverse events every 6 months. For external validation of the results, patients were recruited from Rheumatology Department of Hospital Clínic of Barcelona, in which there is also a specific outpatient clinic for patients receiving b/tsDMARDs where monitoring and assessment of clinical data are performed every 6 months.

For both cohorts, inclusion criteria were as follows: RA patients (⩾ 18 years of age) diagnosed according to the 1987 American College of Rheumatology (ACR) or 2010 ACR/EULAR classification criteria,^13,14 and treated with any b/tsDMARDs. Patients were selectively selected according to the inclusion criteria and classified into two groups according to the number of prior failures to b/tsDMARDs: multi-refractory (MR) and non-refractory (NR) patients. Sample size was not based on data from previous publications because there are few reliable estimates in the literature regarding multiple failure to b/tsDMARDs. Due to the exploratory character of the study, no formal sample size calculation was performed.

Definition for MR and NR patients

Since the publication of D2TRA,¹¹ we classified those who failed to at least two b/tsDMARDs with different mechanisms of action as MR patients. NR patients are defined as those who achieve low-disease activity or remission with the first b/tsDMARD over a long-term follow-up period. We established the cut-off point for long-term follow-up as 5 years based on the definition previously published by our group.¹⁰ This minimum follow-up was established to ensure that patients who later developed secondary inefficacy were not classified as responders.

Patients who discontinued treatment due to adverse events or other reasons unrelated to inefficacy (e.g. pregnancy, sustained remission etc) were excluded. Once the selected patients were under active treatment, we excluded those who did not fulfill pre-established inclusion criteria or those who lacked complete data.

Data collection

For all patients, the following data were collected just prior to starting the first b/tsDMARD: demographic characteristics (age, sex, body mass index and smoking habit), age at diagnosis of RA, age at starting b/tsDMARDs, previous and concomitant treatments (glucocorticoids and conventional synthetic disease-modifying antirheumatic drugs – csDMARDs), comorbidities, presence of bone erosions (as assessed by simple radiography), extra-articular manifestations, and laboratory parameters, such as rheumatoid factor (RF) and anti-citrullinated peptide antibodies (ACPAs), erythrocyte sedimentation rate (ESR), and C-reactive protein (CRP). In addition, health assessment questionnaire (HAQ), pain visual analogue scale (VAS-Pain), and DAS with 28 joint counts (DAS-28-ESR) were assessed at baseline and 6 months after starting the first b/tsDMARD.

Statistical analysis

Descriptive statistics are presented for categorical variables as frequencies and percentages, and for continuous variables as mean and standard deviation (SD). Once exploratory data analyses were performed, this research involved a decision tree learning algorithm known as a ‘CART’, the aim of which was to create a simple and reliable prediction model of multi-drug resistance.^15,16 Among the different learning algorithms for decision trees, the CART strategy has proven to be one of the most successful techniques for classification and regression analysis.¹⁷ CART analyses are based on decision tree algorithms, which consider those input variables most strongly linked to the studied outcome and all subsequent splits of the data to a level at which no further significant splits can be identified, which is called terminal node. In this sense, CART analyses are used in this study to identify sub-groups of patients at increased risk for multi-drug resistance.

As a first step in the CART algorithm to predict MR patients, the RA-Paz dataset was randomly split into two categories: a training dataset (80%) and a testing dataset (20%). This randomization was performed without replacement, meaning that once an observation is selected, it cannot be selected again. First, a ‘seed’ with value 123,487 was set. This ‘seed’ initializes a randomization number generator to facilitate the reproducibility of the results. Then, the functions that performed the random selection were ‘sample_frac’, which are used to select random samples. Hence, this function selects 80% of the initial set for training and 20% for validation data. The functions ‘anti_join’ and ‘select’ are used to create these sets. All the variables that were significant as possible predictors of multiple biologic failure in the univariate analysis (Supplementary Material Table 1) were included for CART development, namely, age at starting the first bDMARD, time between diagnosis and bDMARD initiation, baseline DAS-28, DAS-28 at 6 months, Delta-DAS28, baseline HAQ, HAQ at 6 months, erosions, tender joint count, and swollen joint count. From all of these variables, the CART model selected those that best discriminate between MR and NR patients; then, within these variables, it chooses the optimal cut-off point for classification. In the CART model, at each step, the population is divided into two branches that can become parent nodes. Nodes become more refined with successive divisions. The value at the terminal node represents the observational mode of the training set defining that sub-node. To determine the optimum branching, the Gini index was considered. The Gini index provides an indication of how mixed the training data assigned to each node is; that is, it indicates the model which provides the most information. Tree building and pruning may continue until tree fit, without overfitting, is reached. To avoid overfitting, a maximum tree depth of three was set. Once the model was developed using the training sample, the model performance was assessed by means of a confusion matrix. Predicted probabilities were used for the assessment of the model performance in a testing sample, which was independent of the training/development model, and the accuracy of the model was expressed by receiver-operating characteristic (ROC) curves and area under the curve (AUC).^15–17

Finally, we performed a descriptive analysis of the Clinic Cohort. Features of MR and NR patients in both cohorts were compared using Fisher’s exact or Chi-square tests for categorical variables, and the unpaired t-test or Mann–Whitney U-test for continuous variables, depending on data distribution. An external validation of the CART model was performed using Hospital Clínic Cohort.

R-statistics version 3.6.3 was used to compute all statistical analyses. Random selection without replacement was performed with the R-library ‘dplyr’ version 1.0.7. Rpart v4.1-15 and R part.plot version 3.0.9 libraries were used to develop the CART model and for its external validation.

Results

Patient classification

In total, 629 RA patients in the RA-Paz registry treated with b/tsDMARDs were identified, of whom 216 discontinued treatment due to any one of the following reasons: sustained remission, loss of follow-up, severe infections, malignancies, death, pregnancy, adverse events, including infusion-related reactions, lack of therapeutic adherence, and other causes (Figure 1). Of the 413 patients who remained under treatment at the time of data collection, 197 were excluded due to insufficient follow-up (less than 5 years since the first b/tsDMARD, switching due to other causes aside from inefficacy, and missing data at baseline or 6-month visits). In addition, 80 were excluded because they were refractory to just one bDMARD, not enough to be classified as MR. Ultimately, 136 patients were included in our analysis: 51 MR and 85 NR patients.

Figure 1.

Flowchart for patient selection. NR patients (no refractory) and MR patients (multi-refractory).

Demographic and clinical characteristics of the RA-Paz Cohort

Of the total patients included, 83% were female. Mean age at bDMARD initiation was 52.9 (12.1) years, and at diagnosis, 43.8 (13.2) years. Thus, the mean time between diagnosis and initiation of biologic treatment was 9.1 (8.1) years. Erosions were present in 36.1% of the patients at baseline; 17.6% had extra-articular manifestations and 18.1% were active smokers. In terms of treatment, 36.6% of patients had received ⩾ 3 csDMARDs during the course of their disease and 81% were under concomitant treatment with a csDMARD. In terms of disease activity, the mean baseline DAS-28 was 5.2 (1.1), with a mean improvement of 1.6 (1.2) at month 6 of treatment. Distributions of these demographic and clinical variables for all groups at the start of the first bDMARD and 6 months for all patients are shown in Table 1. All patients included in our cohort started with a bDMARD, generally a TNFi (84.7%), since tsDMARDs were approved for use in Spain beginning in 2017. However, 6% of patients in the MR group received a tsDMARDs during the course of their disease.

Table 1.

Demographic and clinical characteristics of RA-Paz patients included in the study.

Variables	Total n = 136	NR patients N = 85	MR patients N = 51	p-value
Sex (female), n (%)	180 (83.3)	72 (84.7)	41 (80.4)	0.81
Smoking habit, n (%)
Never smoker	75 (55.1)	48 (56.5)	27 (52.9)
Ex-smoker	34 (25.0)	24 (28.2)	10 (19.6)	0.16
Smoker	27 (19.9)	13 (15.3)	14 (27.5)
BMI (kg/m2 ), mean (SD)	26.5 (5.0)	26.1 (4.4)	27.1 (5.8)	0.43
Age mean (SD)
Current	65.1 (11.8)	65.9 (12.0)	64.2 (11.5)	0.44
At diagnosis	44.8 (12.9)	45.5 (13.0)	43.6 (12.9)	0.47
At starting bDMARD	53.4 (11.8)	55.1 (11.7)	50.5 (11.6)	0.03*
Comorbidities, mean (SD)	1.0 (0.9)	1.1 (0.9)	0.9 (0.9)	0.88
Extra-articular manifestations, n (%)	28 (20.6)	15 (17.6)	13 (25.5)	0.17
Immunological parameters, n (%)
Positive ACPA	115 (84.6)	73 (85.9)	42 (82.4)	0.85
Positive RF	118 (86.8)	74 (87.1)	44 (86.3)	0.98
Erosions, n (%)	50 (36.8)	22 (25.9)	28 (54.9)	0.04*
Concomitant csDMARD ref. yes, n (%)	103 (75.0)	60 (70.6)	43 (84.3)	0.01*
Number of previous csDMARD, n (%)
<3	85 (62.5)	64 (75.3)	21 (41.2)	0.01*
⩾3	51 (37.5)	21 (24.7)	30 (58.8)
Disease duration between diagnosis and bDMARDs, mean (SD)	8.5 (7.4)	9.6 (7.8)	6.6 (6.4)	0.04*
DAS-28, mean (SD)	5.3 (1.1)	5.1 (1.0)	5.8 (1.2)	< 0.01*
SJC, mean (SD)	7.9 (4.7)	6.8 (3.4)	9.7 (5.9)	0.02*
TJC, mean (SD)	9.6 (6.9)	8.1 (6.1)	12.2 (7.4)	0.01*
CRP (mg/l), mean (SD)	12.8 (16.9)	10.1 (12.3)	17.4 (22.3)	0.05
ESR (mm/h), mean (SD)	33.3 (20.2)	30.8 (19.6)	37.5 (20.8)	0.11
Pain (VAS), mean (SD)	53.4 (22.2)	50.4 (22.1)	58.5 (21.9)	0.12
HAQ, mean (SD)	1.2 (0.6)	1.1 (0.6)	1.5 (0.6)	0.03*
ΔDAS-28, mean (SD)	1.7 (1.2)	2.0 (1.0)	1.2 (1.3)	0.01*
DAS-28, mean (SD)	3.6 (1.4)	3.0 (1.1)	4.6 (1.5)	< 0.01*
SJC, mean (SD)	3.5 (4.2)	2.1 (2.3)	6.0 (5.4)	< 0.01*
TJC, mean (SD)	4.1 (5.1)	2.1 (2.7)	7.5 (6.4)	< 0.01*
CRP (mg/l), mean (SD)	5.7 (9.7)	3.6 (5.6)	9.1 (13.6)	0.06
ESR (mm/h), mean (SD)	22.5 (17.7)	19.7 (16.5)	27.2 (18.8)	0.04*
Pain (VAS), mean (SD)	30.8 (24.1)	22.3 (19.8)	45.1 (24.2)	< 0.01*
HAQ, mean (SD)	0.9 (0.7)	0.7 (0.7)	1.2 (0.6)	< 0.01*

ACPA, anti-citrullinated peptide antibodies; BMI, body mass index; bDMARD, biologic disease-modifying antirheumatic drugs; CRP, C-reactive protein; csDMARD, conventional synthetic disease-modifying antirheumatic drugs; DAS-28, disease activity score-28; ESR, erythrocyte sedimentation rate; HAQ: health assessment questionnaire; Pain-VAS: pain visual analogue scale; RF, rheumatoid factor; SD, standard deviation; SJC, swollen joint count; TJC, tender joint count.

Results are expressed as the mean (standard deviation) for continuous variables and absolute number (percentage) for categorical variables. Statistical tests applied were chi-square for frequencies; T-test for means

^*Statistically significant (p < 0.05).

Prediction model to identify MR patients

First, all variables collected (see Table 1) were considered in the statistical analysis to assemble the CART predictive model. Those variables with a predictive capability were then selected, including: DAS-28 after 6 months of starting the first bDMARD, Δ-DAS within the first 6 months after the first bDMARD and baseline DAS-28.

Figure 2 shows in detail the cut-off points and the percentage of patients who were classified as NR or MR patients according to the CART model. All patients included in the model yielded an MR probability of 0.37. The predictive model considered the DAS-28 6 months (DAS-28 at 6-month cut-off: 3.7) after starting the first bDMARD (DAS-28 at 6 months). Seventy-two patients (53%) fulfilled this condition, with an MR probability of 0.15. Sixty-four patients (47%) presented a DAS-28 at 6 months ⩾ 3.7, resulting in the application of a second condition to predict MR. At this point, the clinical improvement in those patients with a DAS-28 at 6 months ⩾ 3.7, 6 months after initiating the first bDMARD, were evaluated (ΔDAS-28 at 6–month cut-off: 0.6). Eighteen patients (13%) had a ΔDAS-28 at 6 months < 0.6, showing an MR probability of 0.92. Forty-six patients (34%) recorded a ΔDAS-28 ⩾ 0.6, resulting in the application of a third condition to evaluate the baseline DAS-28 (b-DAS-28 cut-off: 6.1). A b-DAS-28 < 6.1 was observed in 20 patients (15%), with an MR probability of 0.2.

Figure 2.

CART predicting probability of MR patients. Non refractory: NR patients and multi-refractory: MR patients.

Finally, regarding the model’s performance, CART correctly classified 94.1% NR patients and 87.5% MR patients, showing a sensitivity of 0.88, a specificity of 0.94, and an AUC = 0.89 (95% CI: 0.74–1.00). The positive predictive value (PPV) was 0.88 and the negative predictive value (NPV) was 0.94 (Figure 3).

Figure 3.

Receiver-operating characteristic curve (ROC) data of the CART algorithm on the testing dataset for the multi-refractory patient outcomes.

Demographic and clinical characteristics of the RA clinic cohort and compared with the RA-Paz cohort

Of the 480 subjects who underwent b/tsDMARD treatment in the clinic cohort between 2000 and 2021, a total of 82 patients were included in the study, of whom 35 were MR and 47 were NR (Table 2). In the overall sample, we found significant differences in the time-lapse between diagnosis and initiation of a bDMARD, which proved to be shorter in the clinic cohort [4.1 (3.3) years versus 6.6 (4.1) p = 0.04 in MR] and [5.1 (3.9) years versus 9.6 (7.8) p = 0.01 in NR]. In addition, there were differences in the use of corticosteroids, which was lower in both MR and NR patients in the clinic cohort versus the RA-Paz cohort (85.7% versus 100%, p = 0.01 and 98.8% versus 70.5% p = 0.01), respectively.

Table 2.

Comparison between MR and NR patients in the two cohorts (Paz and Clinic).

	MR patients (n = 86)			NR patients (n = 132)
	MR-Paz n = 51	MR-Clinic n = 35	p-value	NR-Paz N = 85	NR-Clinic N = 47	p-value
Sex, female, n(%)	41 (80.4)	33 (94.3)	0.11	72 (84.7)	42 (91.4)	0.21
Smoking status, n(%)
Never smoker	27 (52.9)	18 (51.7)		48 (56.5)	24 (52.2)
Past	10 (19.6)	7 (20.0)	0.91	24 (28.2)	8 (17.4)	0.19
Current	14 (27.5)	10 (28.6)		13 (15.3)	12 (26.1)
BMI (kg/m2) mean (SD)	27.1 (5.8)	25.4 (4.8)	0.16	26.1 (4.4)	24.1 (6.9)	0.05
Age current mean (SD)	65.0 (11.5)	55.4 (13.7)	< 0.01*	66.6 (12.0)	63.0 (13.3)	0.12
At diagnosis	43.8 (12.8)	42.4 (19.4)	0.61	45.5 (12.9)	48.4 (11.8)	0.21
At starting bDMARD	49.9 (11.6)	46.2 (18.1)	0.24	55.1 (11.7)	53.5 (12.4)	0.45
Extra-articular manifestations, n(%)	13 (25.5)	10 (29.6)	0.81	15 (17.6)	2 (4.3)	0.05
Comorbidities mean (SD)	0.9 (0.9)	1.0 (1.1)	0.79	1.1 (0.9)	1.0 (0.9)	0.33
Immunological parameters, n(%)
Positive RF	44 (86.3)	28 (80.5)	0.55	74 (87.1)	31 (67.4)	0.05
Positive ACPA	42 (82.4)	30 (85.7)	0.77	73 (85.4)	37 (80.4)	0.45
Erosions, n(%)	28 (54.9)	20 (57.1)	0.50	22 (25.9)	22 (47.8)	0.02*
Concomitant csDMARD, n(%)	43 (84.4)	31 (88.6)	0.24	60 (70.6)	39 (84.8)	0.08
Number of previous csDMARD(s), n(%)
< 3	42 (39.2)	31 (83.1)	0.21	64 (75.3)	38 (80.8)	0.31
⩾ 3	9 (60.8)	13 (16.9)		21 (24.7)	9 (19.1)
Disease duration between diagnosis and bDMARD mean (SD)	6.6 (6.4)	4.1 (3.3)	0.04*	9.6 (7.8)	5.1 (3.9)	0.01*
Concomitant steroids, n(%)	51 (100)	30 (85.7)	0.01*	84 (98.8)	31 (70.5)	0.01*
First bDMARD n(%)
TNFi	48 (82.4)	28 (80.0)	0.21	65 (76.5)	32 (71.1)	0.21
Non-TNFi	3 (17.8)	7 (20.0)		20 (33.5)	14 (39.9)
Prior to first bDMARD initiation
DAS-28 mean (SD)	5.8 (1.2)	5.5 (1.1)	0.11	5.1 (1.0)	5.1 (1.1)	0.51
TJC mean (SD)	12.3 (7.4)	8.1 (5.7)	0.01*	8.1 (6.1)	7.6 (4.8)	0.66
SJC mean (SD)	9.7 (5.9)	7.7 (5.1)	0.11	6.8 (3.4)	6.7 (3.6)	0.88
HAQ mean (SD)	1.5 (0.6)	0.8 (0.8)	0.02	0.9 (0.6)	0.5 (0.4)	0.05
ESR(mm/h) mean (SD)	37.1 (20.4)	39.6 (33.1)	0.71	30.8 (19.6)	29.2 (21.5)	0.67
CRP (mg/l) mean (SD)	17.4 (22.3)	19.9 (21.2)	0.62	10.1 (12.3)	14.9 (14.6)	0.06
VAS pain mean (SD)	58.5 (21.9)	65.1 (21.9)	0.15	50.4 (22.1)	63.0 (14.5)	0.01*
6 months after first bDMARD
DAS-28 mean (SD)	4.6 (1.5)	4.4 (1.6)	0.55	3.0 (1.1)	2.7 (0.7)	0.09
TJC mean (SD)	7.5 (6.4)	5.2 (5.0)	0.08	2.1 (1.7)	1.1 (1.3)	0.02*
SJC mean (SD)	6.0 (5.1)	4.8 (4.4)	0.28	2.1 (1.4)	1.1 (0.9)	0.02*
ESR (mm/h) mean (SD)	27.2 (18.8)	28.1 (27.5)	0.33	19.7 (16.5)	14.9 (8.6)	0.06
CRP(mg/l) mean(SD)	9.1 (13.2)	9.1 (12.3)	0.97	3.6 (5.6)	2.8 (5.6)	0.42
HAQ mean (SD)	1.1 (0.5)	1.4 (0.8)	0.55	0.7 (0.6)	0.4 (0.4)	0.11
ΔDAS mean (SD)	1.2 (1.3)	1.05 (1.4)	0.48	2.0 (1.0)	2.4 (1.1)	0.02*
VAS pain mean (SD)	45 (24.2)	55.1 (21.2)	0.34	22.3 (19.8)	19.7 (10.4)	0.51

ACPA, anti-citrullinated peptide antibodies; bDMARD, biologic disease-modifying antirheumatic drugs; BMI, body mass index; CRP, C-reactive protein; csDMARD, conventional synthetic disease-modifying antirheumatic drugs; DAS-28, disease activity score-28; ESR, Erythrocyte sedimentation rate; HAQ, health assessment questionnaire; Pain-VAS, pain visual analogue scale; RF, rheumatoid factor; SD, standard deviation; SJC, swollen joint count; TJC, tender joint count.

Results are expressed as the mean (standard deviation) for continuous variables and absolute number (percentage) for categorical variables. Statistical tests applied were chi-square for frequencies; T-student for means.

^*Statistically significant (p < 0.05).

In the group of MR patients, the current age was lower in patients in clinic than in the RA-Paz cohort [55.4 (13.7) versus age 65.0 (11.5) years]. A lower baseline TJC was found in clinic MR patients compared with RA-Paz subjects [8.1 (5.7) versus 12.3 (7.4), p = 0.04]; baseline HAQ was also lower in clinic patients [0.8 (0.8) versus 1.5 (0.6), p = 0.02]. There were no significant differences in the remaining clinical, sociodemographic and laboratory variables, including at baseline and at month 6 of disease activity.

As for NR patients, no statistically significant differences were found in the variables analyzed except for baseline VAS pain, which was higher in clinic NR patients [63.0 (14.5) versus 50.4 (22.1), p < 0.01*] as was clinical response at 6 months as measured by ΔDAS-28. In addition, clinic NR patients showed a greater mean improvement in disease activity [2.4 (1.1) versus 2.0 (1.0), p = 0.02] than NR patients from the RA-Paz cohort.

External validation of the CART model

For external validation of the model, we used the MR and NR data from the clinic cohort and obtained a sensitivity and specificity for the CART model of 0.6 and 0.96, respectively, with an AUC of 0.82 (95% CI: 0.73–0.90) and predictive values of PPV, 0.91 and NPV, 0.75 (Figure 4).

Figure 4.

Receiver-operating curve (ROC) data of the CART algorithm on the external validation cohort.

Discussion

D2TRA patients represent an emerging concern to rheumatologists worldwide for numerous reasons. Within this broad concept of D2TRA, among the points is the failure to different therapies (multi-drug resistance). Therefore, in this study, we aimed to develop a simple and easy-to-use tool to identify these MR cases from the first b/tsDMARDs cycle using clinical data readily available in daily practice. The current CART model is able to predict multiple failures to b/tsDMARDs in patients with RA, using DAS-28 during the early stages of bDMARD initiation. Thus, we determined that response to bDMARDs during the first 6 months, as well as baseline disease activity, may predict future response to treatment in this cohort.

The definition of D2TRA includes treatment failure history as a criterion, taking into account those refractory patients who fail at least two b/tsDMARDs.¹¹ In terms of treatment failure, few observational studies have attempted to establish baseline characteristics and possible risk factors associated with multi-drug resistance. Factors, such as female sex, younger age at start of biologic treatment, shorter disease duration, higher HAQ, smoking, obesity, delay in therapy initiation, and high-disease activity have all been associated with a lack of response to multiple treatments.^6,7 These findings are in the line with our previous study in which we found that being younger at the start of bDMARDs treatment, as well as having higher baseline DAS-28, the presence of erosions, and poorer early response during the first 6 months of treatment with bDMARDs were associated with a classification as MR.¹⁰ Some of these variables have also been associated with poor prognosis in RA (e.g. autoantibody status, smoking, obesity, female sex, erosions). Moreover, while it is important to identify these poor prognostic factors for effective management of patients with RA, there is no evidence linking these features with the development of multi-drug resistance.^18,19

The next question that arises in clinical practice concerns whether early identification of this group of patients is truly important. This issue has not yet been resolved with the current evidence. The true implications that early characterization may have on the therapeutic strategies and clinical outcomes remain unknown. The first steps, recognizing that there is a group of patients with D2TRA for different reasons, and agreeing on a homogeneous definition, remain very important. This will facilitate easier identification of a patient subset that has been challenge in daily clinical practice. In the near future, we will be able to determine whether or not tight control of the disease with personalized therapeutic strategies may improve clinical outcomes in these patients. Moreover, it will be of great interest to investigate whether a ‘window of opportunity’ might alter the course of the disease in those patients more susceptible to multi-drug resistance.

Current evidence regarding this topic is limited, a fact that motivated our group to develop a predictive model to better identify multi-drug resistant patients. Thus, the importance of this model stems from the fact that it uses disease activity as a predictor not only at baseline, but also during changes over time since the onset of biological therapy. Considering that DAS-28 is a composite index encompassing both objective and subjective aspects of the disease, the results it provides in terms of patient characterization are both simple and reliable. It is important to emphasize that, when starting a biologic, the vast majority of patients will most likely present high disease activity; thus, it may be difficult to achieve low-disease activity (even more so, remission) at 6 months. As has been shown in previous studies, higher disease activity (DAS-28 > 5.2) at the start of biological therapy is associated with lower response rates in these patients.^20,21 While achieving disease remission or low-disease activity in patients with a high baseline DAS-28 score could prove more difficult, this does not mean that they cannot achieve substantial improvements in disease activity. In this sense, it seems reasonable to postulate that ΔDAS-28 (with a threshold of 0.6 points) is of particular relevance in classifying future response to treatment in these patients.^22–25

The definition of D2TRA encompasses persistent disease activity/symptoms,¹¹ which may be due not only to the persistence of inflammatory activity, but also to other ‘non-inflammatory’ causes, such as chronic pain syndromes or established structural damage, either of which can lead to persistent symptoms despite controlled disease activity. Although these features were not included in the predictive model, as our aim was to focus on those predictors of multi-drug resistance, they are worth highlighting as indicators of D2TRA, since they are closely related to patients’ clinical perception. Nevertheless, patient-reported results could encourage physicians to focus on the impact of RA on patients, contributing to shared decision-making between patients and rheumatologists, and ultimately leading to a more patient-centered approach and better patient care overall.^26–30

Finally, in recent years, machine learning techniques are increasingly used in medical specialties to classify and identify patients and to predict possible outcomes that can ultimately facilitate making therapeutic decisions and better patient management. An example of these techniques are CART models (as the one we developed) which offer the possibility of using continuous or discrete variables, selecting these variables automatically according to their importance and information contribution. This model based on decision trees, provides a simple and easy-to-use approach to patients classification, as has been demonstrated in other areas of biology and medicine, for example, Su et al³¹. developed a CART model that provided a simple classification by age and bone mineral density to estimate a clinical risk of bone fracture, and this could be easily applied by clinicians in practice.^32–33

For all of the above-mentioned reasons, the main strength of this study is the development of a method for classifying patients at the start of b/tsDMARD treatment. In addition, the external validation of the model supports this tool as a simple and widely applicable predictor of MR and NR.

This study is not without limitations. First, the two cohorts are not strictly homogeneous. This fact may be due, on one hand, to the lack of consensus until relatively recently on the subject of refractory or difficult-to-treat patients, which may lead to a percentage of patients being misclassified when retrospectively reviewing a registry. As well as the intrinsic differences in the two populations and the variability in the clinical practice of the different rheumatology units, both samples had fairly similar and comparable sociodemographic and clinical characteristics. In addition, the sample size is not very large, so these results should be interpreted with caution and it would be very useful to validate them in other cohorts. Thus, we are confident that increasing knowledge in this area will yield more homogeneous cohorts, and that further studies can be performed in the near future to corroborate our data.

On the other hand, although the sensitivity of the model obtained in the external validation decreases with respect to the internal validation, specificity, the predictive values, and the overall accuracy remained good, and we obtained a model with an adequate discriminative capacity between MR and NR.³⁴

Conclusion

Our tool is capable of correctly classifying NR patients and MR patients using data available in routine clinical practice, which is highly applicable and simple to use. In this way, we could better facilitate early characterization of those patients who constitute significant treatment challenges. With a few simple measurements done at the beginning of treatment, we may stratify those patients most likely to be multi-drug resistant, possibly carrying out further tests to fully characterize these patients and more effectively tailor their treatments. These findings would hopefully lead to further studies, in which early identification employing simple tools now available in clinical routine practice, will improve patient care.