Abstract
Objective
To evaluate the Rheumatoid Arthritis Observation of Biologic Therapy (RABBIT) Risk Score for serious infections in patients with rheumatoid arthritis (RA).
Methods
The RABBIT Risk Score for serious infections was developed in 2011 on a cohort of RA patients enrolled in the German biologics register RABBIT between 2001 and 2007. To evaluate this score, we used data from patients enrolled in RABBIT after 1 January 2009. Expected numbers of serious infections and expected numbers of patients with at least one serious infection per year were calculated by means of the RABBIT Risk Score and compared with observed numbers in the evaluation sample.
Results
The evaluation of the score in an independent cohort of 1522 RA patients treated with tumour necrosis factor α (TNFα) inhibitors and 1468 patients treated with non-biological disease-modifying antirheumatic drugs (DMARDs) showed excellent agreement between observed and expected rates of serious infections. For patients exposed to TNF inhibitors, expected as well as observed numbers of serious infections were 3.0 per 100 patient-years (PY). For patients on non-biological DMARDs the expected and observed numbers were 1.5/100 PY and 1.8/100 PY, respectively. The score was highly predictive in groups of patients with low as well as with high infection risk.
Conclusions
The RABBIT Risk Score is a reliable instrument which determines the risk of serious infection in individual patients based on clinical and treatment information. It helps the rheumatologist to balance benefits and risks of treatment, to avoid high-risk treatment combinations and thus to make informed clinical decisions.
Keywords: Rheumatoid Arthritis, Infections, Anti-TNF
Introduction
Serious infections are a major concern in patients treated with cytokine inhibitors or other biological agents. Randomised clinical trials as well as observational cohort studies have shown that there is an increased risk of serious infection in patients treated with tumour necrosis factor α (TNFα) inhibitors. The risk, however, seems to decrease over time: at the start of treatment, an up to 4.5-fold risk was reported,5 whereas after 1 year of treatment no increase in risk over conventional therapy was seen in observational cohort studies.1 5 6 When interpreting data from observational studies, it is important to distinguish the changes in overall risk in the cohorts from the changing risks occurring in individual patients. We showed previously that the decline in risk over time observed in cohorts can be attributed to (i) the increased loss to follow-up of patients with higher susceptibility to infections, as well as (ii) a risk decline in individual patients attributable to improvement in clinical status and reduced glucocorticoid (GC) intake. Taking baseline characteristics and time-dependent changes in the clinical status and treatment of individual patients into account, we were able to calculate the expected infection risk for specific patient profiles and for individual therapies at any time point during the course of treatment.1 This risk estimate allows rheumatologists to align their therapy to the expected risk of individual patients.
However, the risk calculation was based on one single cohort. The aim of this analysis was to evaluate the score with a new cohort of patients not included in its development.
We used data from the German biologics register RABBIT, which is the German acronym for ‘Rheumatoid Arthritis (RA) Observation of Biologic Therapy’.
Methods
The RABBIT Risk Score was developed on patients enrolled in RABBIT before 1 January 2007.1 For the evaluation of the risk score, we used an independent cohort of patients who were enrolled in RABBIT between January 2009 and January 2012 at start of treatment with a TNFα inhibitor or a non-biological disease-modifying antirheumatic drug (nbDMARD). Using the RABBIT Risk Score, we calculated for each patient the expected likelihood of a serious infection based on the individual risk profile.
In our previous analysis we had estimated the number of serious infections per 100 patient-years by means of generalised estimation equations with a Poisson link function. The Poisson link is appropriate for counts such as numbers of infections. A more intuitive measure, however, is the percentage of patients with at least one infection per year. We therefore calculated in addition a modified version of the risk score using the development sample of patients enrolled before 1 January 2007. A complementary log–log link function was applied to estimate the probability of at least one serious infection per year per patient. Both versions of the RABBIT Risk Score contain the following risk factors which had proven predictive of the infection risk in our previous analysis: age (up to or above 60), functional status (assessed by the Hannover Functional Status Questionnaire, Funktionsfragebogen Hannover, FFbH), specific co-morbidities (chronic renal or lung disease), treatment with GCs (≤7.5 mg/day, 7.5–15 mg/day or above), number of previous DMARD treatment failures, previous serious infection, and current treatment with a TNFα inhibitor or an nbDMARD. The expected numbers of serious infections as well as the expected numbers of patients with at least one infection were calculated according to the two versions of the RABBIT Risk Score and then compared to the observed numbers in the cohort or within defined subgroups of patients.
Table 1 shows the risk factors included in the score, their weights and the formula for the calculation of both versions of the score. Since the German instrument FFbH for the measurement of functional capacity can be transformed into Health Assessment Questionnaire (HAQ) scores using a formula published by Lautenschlager et al,7 the RABBIT Risk Score can also be calculated with HAQ values.
Table 1.
Calculation of the RABBIT Risk Score
| Risk factors | V.1: Number of serious infections per 100 PYs | V.2: Percentage of patients with at least one infection per year | |
|---|---|---|---|
| Intercept | Always add | −3.996 | −4.191 |
| Age | If age >60 add | 0.479 | 0.470 |
| Function (FFbH) | Add | −0.01014*FFbH | −0.00865*FFbH |
| Alternatively: HAQ | Add | 0.362(HAQ-3.16) | 0.309(HAQ-3.16) |
| Chronic lung disease | If yes add | 0.522 | 0.484 |
| Chronic renal disease | If yes add | 0.441 | 0.415 |
| Previous serious infection | If yes add | 0.748 | 0.992 |
| Number of treatment failures | If >5 add | 0.443 | 0.397 |
| Mean glucocorticoid dose | If 7.5–14 mg/day add | 0.756 | 0.782 |
| Mean glucocorticoid dose | If ≥ 15 mg/day add | 1.554 | 1.355 |
| Treatment with TNF inhibitor | If yes (last 3 months) add | 0.593 | 0.589 |
| Calculate the sum of the corresponding values | Sum1 | Sum2 | |
| Rabbit Risk Score | Calculate |  |
 |
FFbH, Hannover Functional Status Questionnaire, Funktionsfragebogen Hannover; HAQ, Health Assessment Questionnaire; PY, patient-years; RABBIT, Rheumatoid Arthritis Observation of Biologic Therapy; TNF, tumour necrosis factor.
To evaluate the agreement between observed and expected event rates, we applied the Hosmer–Lemshow test to subgroups of patients was well as to 10 groups established according to the expected risk (figure 1).
Figure 1.
Expected and observed rates of serious infections per 100 patient-years by deciles of expected rates.
Results
Data on 1522 RA patients treated with TNFα inhibitors and 1468 patients treated with nbDMARDs were available. The median observation time was 1.6 years. Table 2 compares baseline characteristics of patients enrolled in the development cohort to those enrolled in the evaluation cohort. The differences in patient characteristics, co-medication and previous treatments between the original sample and the evaluation cohort indicate the changes in treatment strategies for RA in Germany. Patients enrolled in the register between 2001 and 2007 had higher disease activity in both the anti-TNF and the nbDMARD cohorts, they had lower functional capacity, more previous DMARD failures, were more often rheumatoid factor positive and were more often treated with GCs ≥7.5 mg/day than patients enrolled between 2009 and 2012. However, there was no difference in the prevalence of co-morbid chronic lung or renal disease.
Table 2.
Baseline characteristics of patients
| Original sample | Evaluation sample | |||
|---|---|---|---|---|
| Anti-TNF | nbDMARD | Anti-TNF | nbDMARD | |
| N | 3271 | 1773 | 1522 | 1468 |
| Female, n (%) | 2556 (78.1) | 1394 (78.6) | 1143 (75.1%) | 1106 (75.3%) |
| Age | 53.8 (12.3) | 56.2 (11.5) | 55.8 (12.9) | 58.2 (12.5) |
| Disease duration (years), median (IQR) | 9 (5, 16) | 6 (3, 12) | 7 (3, 14) | 4 (2, 9) |
| Follow-up time (years), median (IQR) | 3.1 (2.1, 4.9) | 3.3 (2.5, 5.0) | 1.5 (0.6, 2.1) | 1.6 (0.9, 2.4) |
| Rheumatoid factor positive, n (%) | 2624 (80.2) | 1271 (71.7) | 1095 (72.8) | 866 (59.0) |
| DAS28 | 5.7 (1.2) | 5.1 (1.3) | 5.1 (1.3) | 4.6 (1.3) |
| FFbH | 57.0 (23.0) | 66.6 (21.5) | 64.8 (22.8) | 71.2 (21.8) |
| Smoking never, n (%) | 1027 (47.0) | 585 (45.6) | 667 (44.5) | 693 (48.0) |
| No. of previous DMARDs | 3.3 (1.3) | 1.8 (1.0) | 2.5 (1.1) | 1.6 (0.9) |
| No. of previous biologicals | 0.23 (0.6) | 0.01 (0.1) | 0.24 (0.57) | 0.07 (0.34) |
| Glucocorticoids 7.5–14 mg/day, n (%) | 1027 (31.4) | 386 (21.8) | 383 (25.2) | 171 (11.6) |
| Glucocorticoids ≥15 mg/day, n (%) | 491 (15.0) | 147 (8.3) | 117 (7.7) | 50 (3.4) |
| Chronic lung disease, n (%) | 246 (7.5) | 112 (6.3) | 111 (7.3) | 88 (6.0) |
| Chronic renal disease, n (%) | 139 (4.3) | 31 (1.8) | 57 (3.8) | 27 (1.8) |
Values are means and SDs if not otherwise specified.
; nbDMARD, non-biological disease-modifying antirheumatic drug; TNF, tumour necrosis factor.
We applied the original RABBIT Risk Score based on the number of infections per 100 patient-years to the evaluation cohort (table 3, columns 4–7). According to the score, 69.6 infections were expected under TNF inhibitors. With 69 infections observed in the evaluation cohort (corresponding to a rate of 3.0 per 100 patient-years) the agreement with the estimate was high. This was also the case in subgroups of patients with specific risk factors where we found a high concordance between expected and observed rates (p=0.32 for disagreement). When we applied the second version of the score which gives the percentage of patients affected by at least one infection per year, thus including the first infection per patient only, we again found high agreement between numbers of observed and expected cases (p=0.24 for disagreement). In the patient groups with higher risks, the observed numbers of infections were insignificantly higher than expected.
Table 3.
Expected and observed numbers and rates of serious infections for V.1 (per 100 patient-years) and V.2 (% of patients with at least one serious infection during 12 months)
| n | PY | V.1: Number of serious infections per 100 PY (CI) | V.2: Number and percentage of patients affected by at least one serious infection per year (CI) | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Exp. n | Obs. n | Exp. rate/100 PY | Obs. rate/100 PY | Exp. n | Obs. n | Exp. per year (%) | Obs. per year (%) | |||
| TNFα inhibitor, no risk factor | 764 | 1059 | 16.8 | 16 | 1.6 | 1.5 (0.9–2.5) | 15.3 | 16 | 1.4 | 1.5 (0.9–2.5) |
| nbDMARD, no risk factor | 632 | 816 | 7.0 | 6 | 0.9 | 0.7 (0.3–1.6) | 6.4 | 6 | 0.8 | 0.7 (0.3–1.6) |
| TNFα inhibitor, ≥1 risk factor*, no GC | 635 | 871 | 28.9 | 37 | 3.3 | 4.3 (3.0–5.9) | 25.4 | 33 | 2.9 | 3.8 (2.6–5.3) |
| nbDMARD, ≥1 risk factor*, no GC | 674 | 939 | 15.5 | 19 | 1.7 | 2.0 (1.2–3.2) | 14.0 | 19 | 1.5 | 2.0 (1.2–3.2) |
| TNFα inhibitor + GC, no other risk factor | 225 | 196 | 8.6 | 5 | 4.4 | 2.6 (0.8–6.0) | 7.4 | 4 | 3.8 | 2.0 (0.6–5.2) |
| nbDMARD + GC, no other risk factor | 128 | 86 | 1.9 | 2 | 2.2 | 2.3 (0.3–8.4) | 1.7 | 2 | 2.0 | 2.3 (0.3–8.4) |
| TNFα inhibitor, ≥1 risk factor* + GC | 206 | 160 | 15.3 | 11 | 9.6 | 6.9 (3.4–12.3) | 12.8 | 11 | 8.0 | 6.9 (3.4–12.3) |
| nbDMARD, ≥1 risk factor* + GC | 141 | 96 | 5.1 | 8 | 5.3 | 8.3 (3.6–16.4) | 4.4 | 8 | 4.6 | 8.3 (3.6–16.4) |
| TNFα inhibitor total | 1830 | 2286 | 69.6 | 69 | 3.0 | 3.0 (2.3–3.8) | 60.9 | 64 | 2.7 | 2.8 (2.2–3.7) |
| nbDMARD total | 1575 | 1937 | 29.6 | 35 | 1.5 | 1.8 (1.2–2.5) | 26.6 | 35 | 1.4 | 1.8 (1.2–2.5) |
Bold: The confidence intervals of the observed rates overlap with the expected rates. There is no other test for insignificant difference necessary or sensible.
*At least one of: chronic lung disease, chronic renal disease, age above 60 years, previous serious infection, high number of DMARD failures. GC: treatment with glucocorticoids ≥7.5 mg/day prednisolone equivalent.
exp, expected; obs, observed; nbDMARD, non-biological disease-modifying antirheumatic drug; PY, patient-years; TNF, tumour necrosis factor.
The comparison of expected and observed event rates in deciles of risk scores using the Hosmer–Lemshow test resulted in a high agreement as shown in figure 1. The p values for difference between observed and expected were 0.85 for all patients, 0.29 for patients exposed to TNF inhibitors and 0.49 for those exposed to nbDMARDs.
We developed an online risk score calculator which can be accessed via the RABBIT website (http://www.biologika-register.de/risk score). The following example shows how the score can be used in daily practice to balance risks from different treatments: if we consider a patient aged 65 with RA and chronic obstructive pulmonary disease, this patient's estimated risk under methotrexate and 7.5 mg/day GCs is 4.7%. If the GC dose is increased to 15 mg/day, the risk increases to 8.2%. Adding a TNF inhibitor would further increase the risk to 14.3%. However, if therapy is effective and the GC dose can be tapered down to 5 mg/day, the risk declines to 3.9%.
The calculator also contains weights for the more recently licensed substances rituximab, tocilizumab and abatacept, based on data from 1343, 825 and 444 patients exposed to the respective agents since 2007. Using the formula for TNF inhibitors, we calculated the difference between expected and observed incidences of at least one infection per year. The observed incidences were very similar to those expected (factors compared to TNF inhibitors: 1.15 for tocilizumab, 0.92 for rituximab and 0.82 for abatacept). The weights for the newer substances need further evaluation in a replication cohort.
Discussion
In daily clinical practice, choices have to be made between treatment strategies with different benefit–risk ratios. Scoring systems that enable calculation of absolute numbers of expected adverse events per year or per 100 patient-years help the treating rheumatologist to make informed decisions, taking the overall risk resulting from treatments and their alternatives as well as specific risks resulting from the demographic and clinical situation of individual patients into account.
Using data from the Rochester cohort, Doran et al8 showed in 2002 that, among others, RA severity, functional status and co-morbidity were predictors of serious infection. This group recently developed a risk score for serious infection, based on 584 patients enrolled in the Rochester cohort between 1955 and 1994, and evaluated the score in 410 patients enrolled between 1995 and 2007.9 The contribution of specific risk factors such as age, previous infection, co-morbidity or GC dose was similar as in our data. They found an HR of 3.6 for more than 10 mg GCs/day while we found a fourfold increased risk for >15 mg/day compared to <7.5 mg/day. Each co-morbidity doubled the risk which is comparable to our data. A direct comparison of the two scores is not possible since the Rochester score does not include treatment with biological or non-biological DMARDs and our score does not include erythrocyte sedimentation rate and extra-articular manifestations of RA.
A second infection risk score was developed very recently by Curtis et al10 using two administrative databases in the USA. Predictors of serious infection were, as in our data, older age, co-morbid conditions, higher dosages of GCs and previous serious infection. Diabetes mellitus was also associated with a moderately increased risk in this study. We did not include diabetes into our model due to low predictive capacity. Diabetes as a co-morbid condition may play a greater role in the USA: in the Curtis study, its prevalence was >30% in the older age group and >10% in the younger one, whereas only 8% of our patients had diabetes. The risk score is also not fully comparable to ours since parameters of RA disease activity or severity were not available in the US study. On the other hand, the Curtis score includes a number of items such as cancer screening or treatments for other chronic diseases that are not available to us.
A strength of our risk score is that it includes time-varying parameters of disease activity and disability which results in more precise risk estimates on the individual level than treatment and sociodemographic factors only. The limitation is that a risk score is not generalisable to populations with substantially different risk distributions. Therefore, we consider our risk score valid for the Caucasian population with moderate to severe RA in Germany and probably also in Western Europe. Re-evaluation of the score in other RA populations would be useful to test its reliability beyond the RABBIT register.
Footnotes
Correction notice: This article has been corrected since it was published Online First. The last row of table 1 has been corrected.
Acknowledgements: The authors acknowledge the invaluable contributions of all participating consultant rheumatologists. In particular, we would like to thank those who enrolled the highest numbers of patients: Thilo Klopsch, MD, Neubrandenburg; Andreas Krause, MD, Constanze Richter, MD, Stuttgart; Karin Babinsky, MD, and Anke Liebhaber, MD, Halle; Hans Joachim Berghausen, MD, Duisburg; Arnold Bussmann, MD, Geilenkirchen; Hans Peter Tony, MD, Katja Richter, MD, Universitätsklinikum Carl Gustav Carus Dresden; Andreas Kapelle, MD, Hoyerswerda; Karin Rockwitz, MD, Goslar; Rainer Dockhorn, MD, Weener; Siegfried Wassenberg, MD, Ratingen; Brigitte Krummel-Lorenz, MD, Frankfurt am Main; Anett Gräßler, MD, Pirna; Elke Wilden, MD, Köln; Michael Hammer, MD, Sendenhorst; Edmund Edelmann, MD, Bad Aibling; Winfried Demary, MD, Hildesheim; Martin Aringer, MD, Dresden. We also acknowledge the significant contributions of Matthias Schneider, MD, Düsseldorf, Joern Kekow, MD, Magdeburg and Peter Herzer, MD, Munich as members of the advisory board.
Contributors: All authors were involved in drafting the manuscript or revising it critically for important intellectual content. All authors approved the final version of the article. JL had access to all of the data in the study and takes full responsibility for the integrity of the data and the accuracy of the data analysis. Study conception and design: AZ, JL and AS. Acquisition of data: BM, JK, CE and AK. Analysis and interpretation of data: AZ, JL and AS.
Funding: Supported by a joint, unconditional grant from Abbott, Amgen/Biovitrum, Bristol Myers Squibb, MSD, Pfizer, Roche and UCB. The principal investigators and their team had full academic freedom in study design and conduct, data analysis and publication of results.
Competing interests: None.
Patient consent: Obtained.
Ethics approval: Ethical Committee of the Charite Medical School.
Provenance and peer review: Not commissioned; externally peer reviewed.
References
- 1.Strangfeld A, Eveslage M, Schneider M, et al. Treatment benefit or survival of the fittest: what drives the time-dependent decrease in serious infection rates under TNF inhibition and what does this imply for the individual patient? Ann Rheum Dis 2011;70:1914–20 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Martin-Mola E, Balsa A. Infectious complications of biologic agents. Rheum Dis Clin North Am 2009;35:183–99 [DOI] [PubMed] [Google Scholar]
- 3.Bongartz T, Sutton AJ, Sweeting MJ, et al. Anti-TNF antibody therapy in rheumatoid arthritis and the risk of serious infections and malignancies: systematic review and meta-analysis of rare harmful effects in randomized controlled trials. JAMA 2006;295:2275–85 [DOI] [PubMed] [Google Scholar]
- 4.Leombruno JP, Einarson TR, Keystone EC. The safety of anti-tumour necrosis factor treatments in rheumatoid arthritis: meta and exposure-adjusted pooled analyses of serious adverse events. Ann Rheum Dis 2009;68:1136–45 [DOI] [PubMed] [Google Scholar]
- 5.Galloway JB, Hyrich KL, Mercer LK, et al. Anti-TNF therapy is associated with an increased risk of serious infections in patients with rheumatoid arthritis especially in the first 6 months of treatment: updated results from the British Society for Rheumatology Biologics Register with special emphasis on risks in the elderly. Rheumatology (Oxford) 2010;50:124–31 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Askling J, Fored CM, Brandt L, et al. Time-dependent increase in risk of hospitalisation with infection among Swedish RA-patients treated with TNF-antagonists. Ann Rheum Dis 2007;66:1339–44 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Lautenschlaeger J, Mau W, Kohlmann T, et al. Comparative evaluation of a German version of the Health Assessment Questionnaire and the Hannover Functional Capacity Questionnaire. Z Rheumatol 1997;56:144–55 [DOI] [PubMed] [Google Scholar]
- 8.Doran MF, Crowson CS, Pond GR, et al. Predictors of infection in rheumatoid arthritis. Arthritis Rheum 2002;46:2294–300 [DOI] [PubMed] [Google Scholar]
- 9.Crowson CS, Hoganson DD, Fitz-Gibbon PD, et al. Development and validation of a risk score for serious infections in patients with rheumatoid arthritis. Arthritis Rheum 2012;64:2847–55 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Curtis JR, Xie F, Chen L, et al. Use of a disease risk score to compare serious infections associated with anti-tumor necrosis factor therapy among high- versus lower-risk rheumatoid arthritis patients. Arthritis Care Res (Hoboken) 2012;64:1480–9 [DOI] [PMC free article] [PubMed] [Google Scholar]