Abstract
Background
Simplified measures to quantify rheumatoid arthritis (RA) disease activity are increasingly used. The minimally clinically important differences (MCID) for some measures, such as the clinical disease activity index (CDAI), have not been well-defined in real-world clinic settings, especially for early RA patients with low/moderate disease activity.
Methods
Data from Canadian Early Arthritis Cohort patients were used to examine absolute change in CDAI in the first year after enrollment, stratified by disease activity. MCID cutpoints were derived to optimize the sum of sensitivity and specificity versus the gold standard of patient self-reported improvement or worsening. Specificity, positive predictive value and negative predictive values were calculated against patient self-reported improvement (gold standard) and for change in pain, HAQ and DAS28 improvement. Discrimination was examined using area under receiver operator curves (ROC). Similar methods were used to evaluate MCIDs for worsening for patients who achieved low disease activity.
Results
A total of 578 patients (mean (SD) age 54.1 (15.3) years; 75% women, median (IQR) disease duration 5.3 (3.3, 8.0) months) contributed 1169 visit pairs to the improvement analysis. The MCID cutpoints for improvement were 12 (patients starting in high disease activity, CDAI>22), 6 (moderate, CDAI 10–22), and 1 (low disease activity, CDAI <10). Performance characteristics were acceptable using these cutpoints for pain, HAQ, and DAS28. The MCID for CDAI worsening among patients who achieved low disease activity was 2 units.
Conclusions
These minimally important absolute differences in CDAI can be used to evaluate improvement and worsening and increase the utility of CDAI in clinical practice.
Keywords: early inflammatory arthritis, early rheumatoid arthritis, clinical disease activity index (CDAI), treatment response, minimal clinically important difference (MCID), minimal important difference (MID)
Background
Guidelines recommend quantitative and longitudinal assessment of rheumatoid arthritis (RA) disease activity (1, 2). A variety of measures to quantify disease activity exist, including the Disease Activity Score (DAS) (3). Many, but not all measures of RA disease activity have defined a minimally important difference (MID) for improvement, or when patient reported outcome (PRO) data are used to determine cutoffs, a minimal clinically important difference (MCID). The intent is to quantify the smallest change that exceeds measurement error (e.g. MID) and that is relevant to patients (MCID) (4). Failure to improve beyond the improvement MID of the DAS28, typically defined as 1.2 units, at twelve weeks has been shown to be highly correlated with future response to anti-TNF therapy at 12 months (5). Exceeding the MIDs in the DAS28 has been shown to correlate with improvement in a variety of other outcomes (6).
Despite the common use of the DAS28 in RA clinical trials, it requires real-time results from an acute phase reactant lab test, and a complex calculation. For that reason, simpler measures have been developed, such as the Simplified Disease Activity Index (SDAI) and the Clinical Disease Activity Index (CDAI) (7). A relative change from baseline in the SDAI, derived from clinical trial data, has suggested that an improvement of 50% (minor response), 70% (moderate response), or 85% (major response) correlates well with the American College of Rheumatology (ACR) response rates for the ACR20, ACR50, and ACR70 (8). These relative improvement cut points were able to discriminate between active treatment and placebo in 2 anti-TNF clinical trials. Although these thresholds were defined for SDAI, the same relative improvement values for the change in CDAI were similarly correlated with ACR responses.
While these findings may be useful for comparing treatments in clinical trials, these cut points were intended to categorize responses as minor, moderate, or major, thus transforming a continuous measure of disease activity to a categorical response variable. Using a clinical disease activity measure to guide decision making in clinical practice, however, relies on understanding the absolute amount of change in the continuous variable that is relevant for individual patients. Additionally, the amount of relevant change may be importantly influenced by the starting level of disease activity of the patient. For example, patients who start in moderate disease activity might have a smaller magnitude change in CDAI that is perceived as important compared to patients who started in high disease activity. Moreover, defining the MCID in the direction of both improvement and worsening is needed to guide clinical care as these may differ (9, 10). In this study we determined the amount of change in CDAI that represented improvement and worsening MCIDs in a usual care setting using a large observational cohort of patients with early rheumatoid arthritis (ERA). These MCIDs were then internally and externally validated against a variety of other patient-reported outcomes (PROs) and composite clinical outcomes in observational data and a clinical trial.
Methods
CATCH cohort and study visits
We studied patients from the Canadian early ArThritis CoHort (CATCH) (11, 12) from July 2003 to April 2013, with data from the first three years being contributed by the first pilot site for CATCH in Toronto ON Canada, and additional sites providing data beginning November 2006. CATCH participants have rheumatologist-diagnosed RA as diagnosed with disease < 1 year since symptom onset at time of enrollment. To maximize generalizability, participants were not required to meet formal ACR/EULAR criteria for RA. Patients provided explicit consent to participate in the study and the study was approved by all site’s ethics boards. The cohort data were used to identify pairs of consecutive CATCH visits for each patient. These visits had to occur 3 months apart, and all visit pairs used in this analysis were restricted to those occurring within 12 months of cohort entry. For the improvement analysis, patients could therefore contribute up to 4 visit pairs, occurring at 0 to 3, 3 to 6, 6 to 9, and 9 to 12 months. Sensitivity analyses applied restrictions such that patients contributed only a single visit pair of data, described below. In our initial analysis, we excluded data from individuals who had been identified by their provider as having concomitant fibromyalgia (9%), due to concerns that fibromyalgia may attenuate the magnitude of CDAI improvement or accentuate worsening, and this restriction was also relaxed as part of a sensitivity analysis.
CDAI cut point derivation and cross-classification against other PROs
The proposed CDAI cut points were derived by comparing change in CDAI between two visits with a patient-reported health transition question using a 7-point Likert scale(“Since your last visit would you rate your arthritis as much better, better, a little better, the same or slightly worse, worse, or much worse”) as a gold standard. Patients who reported themselves as much better, better, or slightly better were compared with those who reported themselves the same or worse for the analyses of improvement; for worsening patients reporting themselves as slightly, somewhat, or much worse were compared with those reporting themselves the same or any level of better. The CDAI cutpoints were selected to maximize Youden’s J index (13), calculated as the sum of (sensitivity + specificity − 1). These CDAI cutpoints were rounded to the nearest integer and were cross-classified compared to the published MCIDs commonly used for patient pain (>1 vs. ≤ 1, and >2 vs. ≤2 on a 0–10 visual analog scale) and individual level (rather than group-level) changes in HAQ (≥ 0.31 vs. < 0.31) (14–17). Performance of the CDAI cut points were examined using Sensitivity (Se), Specificity (Sp), positive predictive value (PPV) and negative predictive value (NPV).
Given strong expectations that the CDAI cutpoints would differ according to starting disease activity, the improvement results were stratified by starting disease activity state (low disease activity, CDAI 3 – 10; moderate, CDAI 10 – 22; high, CDAI > 22) (7). We did not evaluate patients starting in remission because of a potential floor effect to detect improvement, although these patients were included in the worsening analysis. The distribution of change in CDAI was plotted visually against improvement in the DAS28ESR using a threshold for DAS28 improvement of twice the measurement error of 0.6 units. This 1.2 unit change typically is considered the MID for the DAS28ESR (4, 18). Similar methods as were used in the improvement analysis were used to derive MCIDs for worsening among patients who were able to achieve LDA, comparing patients who said that they were slightly worse, worse or much worse compared to those who said that they were the same or any magnitude improved. For this ‘worsening’ analysis, data were restricted to visits occurring between 6 and 12 months so as to select a group of improved patients with potentially more stable disease, in whom worsening presumably would be more relevant.
External Validation
As a further validation step, we evaluated the proposed CDAI cut point for MCID for improvement compared to patients’ perceived response to RA therapies in the TEAR trial (19). As in the CATCH cohort, the 755 patients in TEAR also had early RA (median duration of disease 4 months). They were randomized to treatment arms including MTX monotherapy or combination treatment with etanercept or triple therapy, as well as to immediate versus step up therapy to etanercept, or triple therapy. Given that a majority of TEAR patients were in high disease activity at baseline (mean starting CDAI 25, SD=15), analyses were confined to that population who began the trial with CDAI > 22, and restricted to patients beginning MTX, to select the group most relevant to real-world practice. Patients were asked, “Please circle the number that most accurately reflects how your arthritis is responding to the study medication”. They were asked to rate their response on a 1–10 visual analog scale (VAS), where 1 represented an ‘excellent ideal response’ and 10 was ‘none, no change, ineffective”. Comparisons were made between baseline and month 6 to examine the mean difference in CDAI compared to self-reported improvement.
Statistical analysis
Discrimination using the CDAI cut points compared to DAS improvement of > 1.2 units versus non-improvement was assessed using an area under the receiver operator curve (AUROC) (20). Descriptive statistics were used to evaluate the performance characteristics of the CDAI cut points compared to patient self-reported improvement (compared to the last study visit), patient self-reported pain, and HAQ with confidence intervals calculated using a binomial distribution. Finally, the proposed CDAI cut points were examined in the TEAR data using box and whisker plots according to patients’ self-reported improvement. A ROC was calculated using the TEAR trial data for the proposed CDAI cut points derived using the CATCH cohort data. All missing data were handled by case wise deletion.
Two sensitivity analyses were conducted. The first limited each patient to contribute only a single visit pair, the first visit pair for the improvement analysis, and the last visit pair between months 6–12 for the worsening analysis. A second sensitivity analysis added back data from the originally excluded patients with fibromyalgia. All analyses were conducted in SAS 9.4.
Results
CATCH patients’ data used for this analysis were selected as described in Figures 1a and 1b. From 2128 enrolled CATCH patients, we excluded those starting in remission (CDAI < 3), those who had only 1 visit within the first year, and those with fibromyalgia, yielding 578 patients who were included for the improvement analyses (Table 1). Mean age was 54 years, 75% were women, and approximately 80% were Caucasian. Overall, approximately 60% of patients were seropositive. In the improvement analysis, mean baseline disease activity was high as assessed by CDAI and on the cusp between moderate and high by DAS28ESR, with at mean of 5.1 units.
Figure 1.
a: Flow chart for Improvement analysis from CATCH RA Patients
b: Flow chart for Worsening analysis from CATCH RA patients
Table 1.
Baseline characteristics of Eligible CATCH Patients used to study Improvement and Worsening
| Characteristics | Mean ± SD, or % (Improving: N=578) |
Mean ± SD, or % (Worsening: N=553) |
|---|---|---|
| Age, year | 54.1±15.3 | 53.9±15.2 |
| Female sex, % | 75 | 74 |
| Caucasian Ethnicity, % | 84 | 85 |
| Education completed, % | ||
| High school | 33 | 32 |
| College | 56 | 58 |
| Other | 10 | 10 |
| Annual Income $, % | ||
| < 20k | 10 | 9 |
| 20–50k | 27 | 26 |
| > 50k | 31 | 31 |
| Declined to answer | 32 | 34 |
| Number of comorbidities, % | ||
| None | 12 | 14 |
| 1–2 | 46 | 43 |
| >= 3 | 42 | 43 |
| Smoker***, % | ||
| Current | 16 | 17 |
| Past | 37 | 37 |
| Never | 45 | 45 |
| RA symptom duration, days* | 159.0 (97.5,240.5) | 157.0 (99.0,239.0) |
| CDAI (0–76) | 26.4±13.4 | 25.1±13.2 |
| DAS28ESR | 5.1±1.3 | 4.9±1.4 |
| DAS28CRP | 4.7±1.2 | 4.6±1.2 |
| HAQ (0–3)* | 0.875 (0.375,1.5) | 0.75 (0.375,1.375) |
| Patient Global (0–10) | 5.9±2.9 | 5.7±2.9 |
| Physician global (0–10) | 4.9±2.4 | 4.8±2.4 |
| Serologic status**, % | ||
| Positive | 41 | 45 |
| Negative | 14 | 14 |
| Missing | 45 | 41 |
| Rheumatoid factor | ||
| Positive | 38 | 42 |
| Negative | 14 | 15 |
| Missing | 48 | 43 |
| anti-CCP antibody | ||
| Positive | 22 | 24 |
| Negative | 14 | 14 |
| Missing | 64 | 62 |
| CRP (mg/L)* | 7.65 (2.7, 20) | 7.4 (2.3,19.9) |
| Erosive disease, % | ||
| Yes | 17 | 18 |
| No | 61 | 61 |
| Unknown | 22 | 21 |
shown as median (IQR)
rheumatoid factor or anti-CCP antibody
The absolute change in CDAI associated with improvement was > 12 for patients starting in CDAI high disease activity, > 6 for patients starting in moderate disease activity, and > 1 for patients starting in low disease activity (Table 2a). The sensitivity of these cutpoints ranged between approximately 60–85% using the reported MCIDs for PROs including 2 cut points for change in pain and change in HAQ, and depending on starting disease activity. The PPVs of the three MCID cutpoints fell between 74–81% compared to patient’s self-reported improvement (Table 2a, first row), although varied more for the other measures.
Table 2.
| a: Performance Characteristics of CDAI Cut points versus Patient Self-reported Improvement and Pain in CATCH (n=578 unique patients, 1169 visit pairs) | ||||||
|---|---|---|---|---|---|---|
| Variables Associated with Improvement |
CDAI Cutpoint for Improvement (Starting Disease Activity) |
N* | Sensitivity ** (95% CI) |
Specificity ** (95% CI) |
Positive Predictive Value** (95% CI) |
Negative Predictive Value** (95% CI) |
| Patient self-reported improvement (better vs. same/worse) | 1 (low) | 377 | 65 (60,70) | 70 (65,74) | 74 (70,79) | 60 (55,65) |
| 6 (moderate) | 436 | 59 (54,64) | 77 (73,81) | 80 (76,84) | 54 (49,59) | |
| 12 (high) | 356 | 84 (80,88) | 60 (55,65) | 81 (77,85) | 64 (59,69) | |
| Pain (>1 vs. ≤1) | 1 (low) | 349 | 84 (80,88) | 59 (54,64) | 36 (31,41) | 93 (90,96) |
| 6 (moderate) | 410 | 65 (61,70) | 68 (63,72) | 56 (51,61) | 76 (72,80) | |
| 12 (high) | 326 | 86 (82,90) | 57 (52,62) | 74 (69,79) | 74 (69,79) | |
| Pain (>2 vs. ≤ 2) | 1 (low) | 349 | 84 (80,88) | 54 (49,59) | 18 (14,22) | 97 (95,98) |
| 6 (moderate) | 410 | 69 (64,73) | 64 (59,69) | 38 (33,43) | 86 (82,89) | |
| 12 (high) | 326 | 88 (84,91) | 53 (47,58) | 66 (61,71) | 81 (76,85) | |
| HAQ (>0.31 vs. ≤ 0.31) | 1 (low) | 351 | 62 (57,67) | 52 (47,57) | 16 (12,20) | 90 (87,93) |
| 6 (moderate) | 410 | 65 (60,69) | 63 (59,68) | 43 (38,48) | 80 (77,84) | |
| 12 (high) | 325 | 81 (77,86) | 51 (45,56) | 69 (64,74) | 67 (62,72) | |
| b: Performance Characteristics of CDAI Cut points versus Patient Self-reported Worsening and Pain in CATCH for patients with low disease activity (n=381 unique patients, 627 visit pairs) | ||||||
|---|---|---|---|---|---|---|
| Worsening in | CDAI Cutpoint for Worsening |
N* | Sensitivity ** (95% CI) |
Specificity ** (95% CI) |
Positive Predictive Value** (95% CI) |
Negative Predictive Value** (95% CI) |
| Patient self-reported worsening (worse vs. same/better) | 2 | 627 | 58 (77,84) | 82 (80,85) | 38 (34,41) | 92 (89,94) |
| Pain (>1 vs. ≤ 1) | 2 | 581 | 66 (62,70) | 84 (81,87) | 45 (41,49) | 93 (90,95) |
| Pain (>2 vs. ≤ 2) | 2 | 581 | 71 (67,75) | 81 (78,84) | 31 (27,35) | 96 (94,98) |
| HAQ (> 0.31 vs. ≤ 0.31) | 2 | 582 | 66 (62,69) | 80 (77,84) | 28 (25,32) | 95 (93,97) |
n’s slightly vary across rows due to missing data for each outcome measure; rows for pain and HAQ are subsets of the first row for patient self-reported improvement;
results shown as % (95% Confidence Interval)
n’s slightly vary across rows due to missing data for each outcome measure; rows for pain and HAQ are subsets of the first row for patient self-reported improvement;
results shown as % (95% Confidence Interval)
Discrimination for the CDAI MCIDs was good compared to DAS28 improvement of 1.2 units, as shown in Figures 2a and 2b. The AUROC was 0.82 for patients starting in moderate disease activity and 0.83 for patients starting in high disease activity. Results from the sensitivity analysis that limited patients to contribute only a single visit pair were consistent with the main results (Appendix Table 1). Results remained consistent when adding back the 9% of patients originally excluded with concomitant RA and fibromyalgia (Appendix Table 2).
Figure 2.
a: Distribution of the change in CDAI for RA patients starting in moderate disease activity (CDAI between 10 and 22) and corresponding change in the DAS28ESR of 1.2 units (n=212 unique CATCH patients, 288 visit pairs after excluding observations with missing DAS28ESR)
The solid black line indicates the threshold value for MCID −6 unit and includes patients who start in moderate disease activity. Light grey bars indicate overlap between the black and white curves. AUROC = 0.82.
b: Distribution of the change in CDAI in starting in high disease activity (CDAI greater than 22) and corresponding change in the DAS28ESR of 1.2 units (n=200 unique CATCH patients w/ 234 pairs after excluding observations with missing DAS28ESR)
The solid black line indicates the threshold value for MCID −12 units and includes patients who start in high disease activity. Grey bars indicate overlap between the black and white curves. AUROC = 0.83.
Appendix 1.
| a: Performance Characteristics of CDAI Cut points versus Patient Self-reported Improvement and Pain in CATCH (keep 1st pair of the visits, n=578 unique patients contributing only a single observation) | ||||||
|---|---|---|---|---|---|---|
| Variables Improved | CDAI Cutpoint for Improvement (Starting Disease Activity) |
N* | Sensitivity (95% CI)** |
Specificity (95% CI)** |
Positive Predictive Value (95% CI)** |
Negative Predictive Value (95% CI)** |
| Patient self-reported improvement (better vs. same/worse) | 1 (low) | 110 | 73 (64,81) | 66 (57,75) | 76 (68,84) | 62 (53,71) |
| 6 (moderate) | 192 | 63 (56,70) | 63 (56,70) | 78 (72,84) | 45 (38,52) | |
| 12 (high) | 276 | 86 (82,90) | 55 (49,60) | 83 (79,87) | 60 (54,66) | |
| Pain (>1 vs. ≤1) | 1 (low) | 103 | 88 (82,94) | 51 (42,61) | 37 (27,46) | 93 (88,98) |
| 6 (moderate) | 181 | 75 (69,81) | 62 (55,69) | 61 (54,68) | 76 (70,82) | |
| 12 (high) | 250 | 89 (85,93) | 53 (47,59) | 76 (71,81) | 74 (69,80) | |
| Pain (>2 vs. ≤ 2) | 1 (low) | 103 | 91 (85,96) | 46 (36,55) | 17 (9,24) | 98 (95,100) |
| 6 (moderate) | 181 | 75 (68,81) | 55 (48,62) | 42 (35,49) | 83 (78,89) | |
| 12 (high) | 250 | 92 (89,95) | 50 (43,56) | 70 (64,75) | 83 (79,88) | |
| HAQ (>0.31 vs. ≤ 0.31) | 1 (low) | 104 | 77 (69,85) | 45 (35,55) | 17 (10,24) | 93 (88,98) |
| 6 (moderate) | 181 | 70 (64,77) | 54 (47,61) | 45 (38,53) | 77 (71,83) | |
| 12 (high) | 249 | 85 (81,90) | 47 (41,53) | 73 (67,78) | 66 (60,72) | |
| b: Performance Characteristics of CDAI Cut points versus Patient Self-reported Improvement and Pain in CATCH (keep the last pair of the visits, n=381 unique patients contributing only a single observation) | ||||||
|---|---|---|---|---|---|---|
| Variables with Worsening |
CDAI Cutpoint for Worsening |
N* | Sensitivity (95% CI)** |
Specificity (95% CI)** |
Positive Predictive Value (95% CI)** |
Negative Predictive Value (95% CI)** |
| Patient self-reported worsening (worse vs. same/better) | 2 | 381 | 69 (65,74) | 78 (74,82) | 37 (32,41) | 93 (91,96) |
| Pain (>1 vs. ≤ 1) | 2 | 353 | 71 (67,76) | 79 (75,83) | 42 (37,48) | 93 (90,95) |
| Pain (>2 vs. ≤ 2) | 2 | 353 | 78 (73,82) | 76 (72,80) | 29 (25,34) | 96 (94,98) |
| HAQ (> 0.31 vs.≤ 0.31) | 2 | 356 | 75 (71,79) | 76 (71,80) | 28 (23,33) | 96 (94,98) |
n’s slightly vary across rows due to missing data for each outcome measure; rows for pain and HAQ are subsets of the first row for patient self-reported improvement;
results shown as % (95% Confidence Interval)
n’s slightly vary across rows due to missing data for each outcome measure; rows for pain and HAQ are subsets of the first row for patient self-reported improvement;
results shown as % (95% Confidence Interval)
Appendix 2.
| a: Performance Characteristics of CDAI Cut points versus Patient Self-reported Improvement and Pain in CATCH including fibromyalgia patients (n=611 unique patients contributing 1231 pairs) | ||||||
|---|---|---|---|---|---|---|
| Variables Improved | CDAI Cutpoint for Improvement (Starting Disease Activity) |
N* | Sensitivity (95% CI)** |
Specificity (95% CI)** |
Positive Predictive Value (95% CI)** |
Negative Predictive Value (95% CI)** |
| Patient self-reported improvement (better vs. same/worse) | 1 (low) | 390 | 65 (60,69) | 70 (65,74) | 74 (69,78) | 60 (55,65) |
| 6 (moderate) | 467 | 57 (53,62) | 78 (74,81) | 80 (76,83) | 54 (49,58) | |
| 12 (high) | 374 | 84 (80,88) | 58 (53,63) | 80 (76,84) | 65 (60,69) | |
| Pain (>1 vs. ≤ 1) | 1 (low) | 362 | 84 (80,87) | 60 (55,65) | 37 (32,42) | 93 (90,96) |
| 6 (moderate) | 439 | 63 (59,68) | 69 (65,73) | 55 (51,60) | 76 (71,80) | |
| 12 (high) | 343 | 86 (82,89) | 55 (50,60) | 72 (67,77) | 74 (69,79) | |
| Pain (>2 vs. ≤ 2) | 1 (low) | 362 | 83 (79,86) | 55 (50,60) | 18 (14,22) | 96 (94,98) |
| 6 (moderate) | 439 | 67 (63,72) | 64 (60,69) | 38 (33,42) | 86 (83,89) | |
| 12 (high) | 343 | 88 (84,91) | 51 (45,56) | 64 (59,69) | 81 (76,85) | |
| HAQ (> 0.31 vs. ≤ 0.31) | 1 (low) | 364 | 61 (56,66) | 53 (48,58) | 17 (13,21) | 90 (87,93) |
| 6 (moderate) | 439 | 61 (56,65) | 64 (60,69) | 42 (38,47) | 79 (75,83) | |
| 12 (high) | 342 | 81 (77,85) | 49 (43,54) | 67 (62,72) | 66 (61,71) | |
| b: Performance Characteristics of CDAI Cut points versus Patient Self-reported Improvement and Pain in CATCH, including fibromyalgia patients (n=392 unique patients contributing 641 pairs) | ||||||
|---|---|---|---|---|---|---|
| Worsening in | CDAI Cutpoint for Worsening |
N* | Sensitivity( 95% CI)** |
Specificity (95% CI)** |
Positive Predictive Value (95% CI)** |
Negative Predictive Value (95% CI)** |
| Patient self-reported worsening (worse vs. same/better) | 2 | 641 | 57 (54,61) | 82 (79,85) | 38 (34,41) | 91 (89,93) |
| Pain (>1 vs. ≤1) | 2 | 595 | 66 (63,70) | 84 (81,87) | 46 (42,50) | 92 (90,95) |
| Pain (>2 vs. ≤2) | 2 | 595 | 71 (67,74) | 81 (78,84) | 32 (28,35) | 96 (94,97) |
| HAQ (> 0.31 vs. ≤0.31) | 2 | 596 | 66 (62,69) | 80 (77,83) | 29 (25,32) | 95 (93,97) |
n’s slightly vary across rows due to missing data for each outcome measure; rows for pain and HAQ are subsets of the first row for patient self-reported improvement;
results shown as % (95% Confidence Interval)
n’s slightly vary across rows due to missing data for each outcome measure; rows for pain and HAQ are subsets of the first row for patient self-reported improvement;
results shown as % (95% Confidence Interval)
In the worsening analysis using data from 553 eligible patients at 6–12 months after cohort entry (Figure 1b), baseline characteristics in Table 1 were similar to the patients represented in the improvement analysis. After selecting the 381 unique patients (69%) who were in low disease activity at or beyond 6 months the MCID cutpoint for worsening was > 2 units (Table 2b). Sensitivity was in the 60–70% range, and specificity was 80% or greater for all measures.
Validation
For patients starting MTX monotherapy in TEAR and who began the trial in high disease activity with evaluable CDAI at baseline and month 6 (n=282), their distribution of CDAI at month 6 according to self-reported response on a 1–10 VAS is shown in Figure 3. In total, 231 (82%) patients reported some or a good response (1–6 on VAS). The median change in CDAI for these patients was −20 units, which was significantly higher (p < 0.001) than the median change in CDAI of −9 units for the 18% of patients reporting limited or no response (7–10 on VAS). Similar to the MCID for improvement in CATCH, a cut point of −12 units maximized sensitivity and specificity compared to other possible CDAI cut points (Figure 4), with an AUROC of 0.75.
Figure 3.
Distribution of Change in CDAI in the TEAR trial between 0 and 6 months according to Patients’ Self-reported Response to Therapy on a 1 (Most Improvement) to 10 (No Improvement) Visual Analog Scale (n=282 patients initiating MTX monotherapy)
Note: solid line drawn at CDAI cutpoint of −12
Figure 4.
Receiver Operator Curve examining Various CDAI Cutpoints, Comparing TEAR Trial Participants with Minimal to No response to Therapy to Those with Some or a Good Response (n=282 patients)
“Minimal to No Response” defined as 7–10 on a visual analog scale; “Some or Good Response” defined as 1–6 on visual analog scale
Note: numbers in the figure reference various CDAI cutpoints ranging from 9 to 17
Discussion
In this study, which included data from a large cohort of early RA patients participating in a long term observational study with DMARD initiation and data from a clinical trial, we derived, assessed and validated the performance characteristics of the absolute change in CDAI that defined MCIDs for improvement and worsening. As suggested previously for some PROs (21), but not hitherto shown for CDAI, we found that the MCID for the CDAI was highly dependent on patients’ starting level of disease activity. For patients who began in high disease activity, we determined that the MCID for improvement in the CDAI was > 12 units. For those starting in moderate disease activity, the MCID was >6 units, and for low disease activity, > 1 unit.
Previous research has defined response criteria for the SDAI and CDAI in relation to relative improvement from a prior disease state (8). Improvement of 50%, 70%, and 85% in the SDAI and CDAI defined a ‘minor’, ‘moderate’ and ‘major’ response. Patients in that analysis began with high disease activity on average, with CDAIs ranging from 25–43 across the trials and registries examined. In that study, separate MCIDs for patients starting in moderate or low disease activity was not calculated. A 50% improvement in the CDAI, classified as a ‘minor response’, therefore represents an absolute change of approximately 12 – 21 CDAI units. This magnitude of change is consistent with a more than 12 unit absolute change in the CDAI that we propose as the MCID for patients beginning in high disease activity, based on our analysis.
Our results also are consistent with a previously published study that examined change in CDAI in relation to other outcomes. A study using data from the Consortium of Rheumatology Researchers of North America (CORRONA) examined patients initiated on anti-TNF therapy (22). Change in CDAI was compared between patients continuing on therapy through 1 year to patients switching their treatment regimens. For patients starting in high disease activity, the mean change in CDAI in the switchers was −10 units, compared to the −20 unit change in patients continuing therapy. For patients starting in moderate disease activity, mean change in CDAI was −1 (switchers) vs. −7 (continuers); and for patients in low disease activity, mean change was +2 units (switchers) vs. −2 units (continuers). Each of the MCIDs obtained in the current analysis was within the range for the 3 disease activity groups. Our results are also compatible with an earlier analysis using CORRONA that found that the CDAI threshold corresponding to an improvement in DAS28(ESR) of 1.2 was −10 units (23) with an AUROC of 0.84, although those results were not stratified by starting disease activity. Results from a smaller study of 223 early RA patients with starting mean DAS28ESR of 6.1 units and mean CDAI of 36, showed a somewhat larger change in CDAI (16 units) that corresponded with a DAS28 change of 1.2 units (24).
We derived an MCID for worsening of the CDAI among patients who had achieved low disease activity or remission. We initially considered the possibility of deriving CDAIs for worsening stratified by initial disease activity just as we did for improvement. However, we found that many patients reporting that they were ‘worse’ or experiencing RA flare had no concomitant worsening in their CDAI, or even a small improvement (e.g. 1–2 units). Our conclusion from these initial findings was that patients with essentially stable but very active RA may have difficulty distinguishing ‘worsening’ or RA ‘flare’ from their ongoing moderate or high disease activity state. Thus, we derived the CDAI cutpoints for worsening only among those who attained low disease activity.
Strengths of our study include representation of RA patients across the spectrum of RA disease activity, and inclusion of data from both a large observational cohort and a clinical trial. However, our results must be interpreted in light of our study design. We recognize that multiple methods exist to derive a MCID (25, 26), or smallest detectable difference (SDD), often using distribution-based methods involving the measurement error. In our study, we considered anchor-based approaches based on a patient-reported health transition question and changes in other patient reported and clinical indices rather than consideration of the measurement error.
We also recognize that the generalizability of such an analysis is important. We used data from early RA patients, but it is theoretically possible that response thresholds might differ for patients with more established disease or based upon other characteristics (e.g. age, sex, comorbidity burden). While results could differ for RA patients with concomitant chronic pain syndromes that were excluded in this analysis, including the 9% of the total cohort who had fibromyalgia, the performance of our CDAI cutpoints when these patients were included was consistent with our main analysis. Also, while we optimized sensitivity and specificity in order to establish our cutpoints, we found that PPVs and NPVs varied appreciably across other PROs. Unlike sensitivity and specificity, PPVs and NPVs are dependent on the prevalence of a condition, so investigators should consider this and the consequences of potential misclassification when applying these findings to their own RA populations. Finally, MCIDs have been suggested to differ for patients with other conditions. (27, 28). Thus, we caution applying these MCID values for change in CDAI for early RA disease activity improvement and worsening to assume that they are the same in other settings, such as for RA patients with more established disease
In conclusion, in this cohort of early RA patients, the MCID of the CDAI for patients starting in high disease activity was a change of greater than 12 units. For moderate disease patients, the MCID was change more than 6 units, and for low disease activity, it was a change of more than 1 unit. Thresholds for MCID that relate to patients’ starting disease activity should be used, rather than a single threshold that ignores baseline disease activity. The MCID of the CDAI for worsening in patients having achieved at least low disease activity is a change of more than 2 units. These MCIDs for absolute change in CDAI can be used to evaluate improvement and increase the usefulness of CDAI as measured in settings of usual care.
Significance and Innovation.
The absolute minimal clinically important difference (MCID) for CDAI has not yet been defined, especially in early RA
Data from CATCH (Canadian Early Arthritis Cohort) patients were used to derive and validate the absolute change in CDAI representing the MCID, examining sensitivity, specificity, positive and negative predictive values that maximized performance and discrimination compared to other PROs and clinical indices
The optimal cut point for the MCID for CDAI improvement stratified for baseline disease activity was change of 12 (high disease activity), 6 (moderate disease activity), and 1 (low disease activity).
The MCID for CDAI worsening among RA patients who achieved low disease activity was 2 units.
Acknowledgments
Competing Interests and Disclosures: The CATCH study was designed and implemented by the investigators. The authors have received unrestricted grants from: Amgen and Pfizer Canada - Founding sponsor since 2007, Hoffmann-LaRoche, UCB Canada, Bristol-Myers Squibb Canada, AbbVie Corporation and Janssen Biotech since 2011. This analysis was funded in part by National Institutes of Health (AR064172) and AHRQ (HS018517) grants awarded to Dr. Curtis.
Dr. Curtis has grants and/or consulting relationships with Roche/Genentech, UCB, Janssen, CORRONA, Amgen, Pfizer, BMS, Crescendo, AbbVie. Dr. Bykerk receives funding from the Cedar Hill Foundation; NIH grant (1UH2AR067691-01); unrestricted research grants and/or has consulting relationships with Amgen, Pfizer, BMS, Crescendo, AbbVie, Antares, Janssen, Roche/Genentech, UCB.
Dr. Bingham has been funded through a Pilot Project Award from the Patient Centered Outcomes Research Institute (PCORI), IP2-PI000737. He has received research grants and/or served as a consultant to Abbvie, Amgen, BMS, Celgene, CORRONA, Genentech/Roche, Lilly, Janssen, Pfizer, UCB. He is a member of the executive leadership committee for OMERACT (Outcome Measures in Rheumatology), an international academic collaborative group that receives hands-off funding from 23 pharmaceutical and research organizations over the last 3 years.
CATCH Investigators: Dr. Vivian Bykerk, Dr Boulos Haraoui, Dr. Janet Pope, Dr. Gilles Boire, Dr. Murray Baron, Dr. Susan Bartlett, Dr Carter Thorne, Dr Carol Hitchon, Dr. Edward C Keystone, Dr. William Bensen, Dr. Lawrence Rubin, Dr. Louis Bessette, Dr. Bindee Kuriya, Dr. Pooneh Akhavan, Dr. Maggie Larche, Dr. Ines Colmegna, Dr. Vandana Ahluwahlia, Dr. Christopher Penney, Dr. Christopher Lyddell, Dr. Alice Klinkhoff, Dr. Shahin Jamal, Dr. Michel Zummer
Allied Health Representative: Ms Diane Tin
Management Team: Ms. Franci Sniderman, Ms Diane Tin
Patient Representative: Dawn Richards
Statistical Support: Daming Lin, Ye Sun, Jim Wang
References
- 1.Smolen JS, Landewe R, Breedveld FC, Buch M, Burmester G, Dougados M, et al. EULAR recommendations for the management of rheumatoid arthritis with synthetic and biological disease-modifying antirheumatic drugs: 2013 update. Ann Rheum Dis. 2014;73(3):492–509. doi: 10.1136/annrheumdis-2013-204573. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Singh JA, Furst DE, Bharat A, Curtis JR, Kavanaugh AF, Kremer JM, et al. 2012 update of the 2008 American College of Rheumatology recommendations for the use of disease-modifying antirheumatic drugs and biologic agents in the treatment of rheumatoid arthritis. Arthritis Care Res (Hoboken) 2012;64(5):625–639. doi: 10.1002/acr.21641. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Prevoo ML, van 't Hof MA, Kuper HH, van Leeuwen MA, van de Putte LB, van Riel PL. Modified disease activity scores that include twenty-eight-joint counts. Development and validation in a prospective longitudinal study of patients with rheumatoid arthritis. Arthritis Rheum. 1995;38(1):44–48. doi: 10.1002/art.1780380107. [DOI] [PubMed] [Google Scholar]
- 4.van Gestel AM, Prevoo ML, van 't Hof MA, van Rijswijk MH, van de Putte LB, van Riel PL. Development and validation of the European League Against Rheumatism response criteria for rheumatoid arthritis. Comparison with the preliminary American College of Rheumatology and the World Health Organization/International League Against Rheumatism Criteria. Arthritis Rheum. 1996;39(1):34–40. doi: 10.1002/art.1780390105. [DOI] [PubMed] [Google Scholar]
- 5.van der Heijde D, Keystone EC, Curtis JR, Landewe RB, Schiff MH, Khanna D, et al. Timing and magnitude of initial change in disease activity score 28 predicts the likelihood of achieving low disease activity at 1 year in rheumatoid arthritis patients treated with certolizumab pegol: a post-hoc analysis of the RAPID 1 trial. J Rheumatol. 2012;39(7):1326–1333. doi: 10.3899/jrheum.111171. [DOI] [PubMed] [Google Scholar]
- 6.Uhlig T, Kvien TK, Pincus T. Test-retest reliability of disease activity core set measures and indices in rheumatoid arthritis. Ann Rheum Dis. 2009;68(6):972–975. doi: 10.1136/ard.2008.097345. [DOI] [PubMed] [Google Scholar]
- 7.Aletaha D, Nell VP, Stamm T, Uffmann M, Pflugbeil S, Machold K, et al. Acute phase reactants add little to composite disease activity indices for rheumatoid arthritis: validation of a clinical activity score. Arthritis Res Ther. 2005;7(4):R796–R806. doi: 10.1186/ar1740. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Aletaha D, Martinez-Avila J, Kvien TK, Smolen JS. Definition of treatment response in rheumatoid arthritis based on the simplified and the clinical disease activity index. Ann Rheum Dis. 2012;71(7):1190–1196. doi: 10.1136/annrheumdis-2012-201491. [DOI] [PubMed] [Google Scholar]
- 9.Alten R, Pohl C, Choy EH, Christensen R, Furst DE, Hewlett SE, et al. Developing a construct to evaluate flares in rheumatoid arthritis: a conceptual report of the OMERACT RA Flare Definition Working Group. J Rheumatol. 2011;38(8):1745–1750. doi: 10.3899/jrheum.110400. [DOI] [PubMed] [Google Scholar]
- 10.Bingham CO, 3rd, Pohl C, Woodworth TG, Hewlett SE, May JE, Rahman MU, et al. Developing a standardized definition for disease "flare" in rheumatoid arthritis (OMERACT 9 Special Interest Group) J Rheumatol. 2009;36(10):2335–2341. doi: 10.3899/jrheum.090369. [DOI] [PubMed] [Google Scholar]
- 11.Bykerk VP, Jamal S, Boire G, Hitchon CA, Haraoui B, Pope JE, et al. The Canadian Early Arthritis Cohort (CATCH): patients with new-onset synovitis meeting the 2010 ACR/EULAR classification criteria but not the 1987 ACR classification criteria present with less severe disease activity. J Rheumatol. 2012;39(11):2071–2080. doi: 10.3899/jrheum.120029. [DOI] [PubMed] [Google Scholar]
- 12.Lee YC, Lu B, Boire G, Haraoui BP, Hitchon CA, Pope JE, et al. Incidence and predictors of secondary fibromyalgia in an early arthritis cohort. Ann Rheum Dis. 2013;72(6):949–954. doi: 10.1136/annrheumdis-2012-201506. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Youden W. Index for rating diagnostic tests. Cancer. 1950;3(32–35) doi: 10.1002/1097-0142(1950)3:1<32::aid-cncr2820030106>3.0.co;2-3. [DOI] [PubMed] [Google Scholar]
- 14.Tubach F, Ravaud P, Baron G, Falissard B, Logeart I, Bellamy N, et al. Evaluation of clinically relevant changes in patient reported outcomes in knee and hip osteoarthritis: the minimal clinically important improvement. Ann Rheum Dis. 2005;64(1):29–33. doi: 10.1136/ard.2004.022905. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Wells GA, Tugwell P, Kraag GR, Baker PR, Groh J, Redelmeier DA. Minimum important difference between patients with rheumatoid arthritis: the patient's perspective. J Rheumatol. 1993;20(3):557–560. [PubMed] [Google Scholar]
- 16.Greenwood MC, Doyle DV, Ensor M. Does the Stanford Health Assessment Questionnaire have potential as a monitoring tool for subjects with rheumatoid arthritis? Ann Rheum Dis. 2001;60(4):344–348. doi: 10.1136/ard.60.4.344. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Bruce B, Fries JF. The Stanford Health Assessment Questionnaire: a review of its history, issues, progress, and documentation. J Rheumatol. 2003;30(1):167–178. [PubMed] [Google Scholar]
- 18.van Gestel AM, Haagsma CJ, van Riel PL. Validation of rheumatoid arthritis improvement criteria that include simplified joint counts. Arthritis Rheum. 1998;41(10):1845–1850. doi: 10.1002/1529-0131(199810)41:10<1845::AID-ART17>3.0.CO;2-K. [DOI] [PubMed] [Google Scholar]
- 19.Moreland LW, O'Dell JR, Paulus HE, Curtis JR, Bathon JM, St Clair EW, et al. A randomized comparative effectiveness study of oral triple therapy versus etanercept plus methotrexate in early aggressive rheumatoid arthritis: the treatment of Early Aggressive Rheumatoid Arthritis Trial. Arthritis Rheum. 2012;64(9):2824–2835. doi: 10.1002/art.34498. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.McNeil BJ, Hanley JA. Statistical approaches to the analysis of receiver operating characteristic (ROC) curves. Med Decis Making. 1984;4(2):137–150. doi: 10.1177/0272989X8400400203. [DOI] [PubMed] [Google Scholar]
- 21.Aletaha D, Funovits J, Ward MM, Smolen JS, Kvien TK. Perception of improvement in patients with rheumatoid arthritis varies with disease activity levels at baseline. Arthritis Rheum. 2009;61(3):313–320. doi: 10.1002/art.24282. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Zhang J, Shan Y, Reed G, Kremer J, Greenberg JD, Baumgartner S, et al. Thresholds in disease activity for switching biologics in rheumatoid arthritis patients: experience from a large U.S. cohort. Arthritis Care Res (Hoboken) 2011;63(12):1672–1679. doi: 10.1002/acr.20643. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Greenberg JD, Harrold LR, Bentley MJ, Kremer J, Reed G, Strand V. Evaluation of composite measures of treatment response without acute-phase reactants in patients with rheumatoid arthritis. Rheumatology (Oxford) 2009;48(6):686–690. doi: 10.1093/rheumatology/kep054. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Ranganath VK, Yoon J, Khanna D, Park GS, Furst DE, Elashoff DA, et al. Comparison of composite measures of disease activity in an early seropositive rheumatoid arthritis cohort. Ann Rheum Dis. 2007;66(12):1633–1640. doi: 10.1136/ard.2006.065839. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Wells G, Beaton D, Shea B, Boers M, Simon L, Strand V, et al. Minimal clinically important differences: review of methods. J Rheumatol. 2001;28(2):406–412. [PubMed] [Google Scholar]
- 26.Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol. 2008;61(2):102–109. doi: 10.1016/j.jclinepi.2007.03.012. [DOI] [PubMed] [Google Scholar]
- 27.Colangelo KJ, Pope JE, Peschken C. The minimally important difference for patient reported outcomes in systemic lupus erythematosus including the HAQ-DI, pain, fatigue, and SF-36. J Rheumatol. 2009;36(10):2231–2237. doi: 10.3899/jrheum.090193. [DOI] [PubMed] [Google Scholar]
- 28.Wheaton L, Pope J. The minimally important difference for patient-reported outcomes in spondyloarthropathies including pain, fatigue, sleep, and Health Assessment Questionnaire. J Rheumatol. 2010;37(4):816–822. doi: 10.3899/jrheum.090086. [DOI] [PubMed] [Google Scholar]