Correlation of the Multi-Biomarker Disease Activity Score with Rheumatoid Arthritis Disease Activity Measures: A Systematic Review and Meta-Analysis

Abstract

Objective:

There are conflicting reports on the validity of the multi-biomarker disease activity (MBDA) score to assess RA disease activity. We performed a systematic review of the MBDA and meta-analysis of the correlation between the MBDA and other RA disease activity measures.

Methods:

A systematic review was performed, searching MEDLINE, EMBASE, Scopus, Google Scholar, and the Cochrane Library from inception to March 7, 2017. Study details, MBDA performance, and study quality were assessed by independent reviewers. Correlations of the MBDA with composite RA disease activity measures were pooled using random-effects meta-analyses.

Results:

Twenty-two studies were identified in the systematic review, of which 8 (n=3,242 assays) reported correlations of the MBDA with RA disease activity measures. Pooling results from these eight studies in the meta-analysis, the MBDA demonstrated modest correlations with DAS28-CRP (r = 0.41, 95% CI 0.36–0.46) and DAS28-ESR (r = 0.48, 95% CI 0.38–0.58) with weaker correlations observed with SDAI (r = 0.35, 95% CI 0.26–0.43), CDAI (r = 0.26, 95% CI 0.19–0.33), and RAPID3 (r = 0.23, 95% CI 0.19–0.27). Correlations between change in MBDA and change in disease activity measures ranged from r = 0.53 (DAS28-ESR) to r = 0.26 (CDAI).

Conclusion:

The MBDA demonstrates moderate convergent validity with DAS28-CRP and DAS28-ESR, but weaker correlations with SDAI, CDAI, and RAPID3. While it appears to complement existing RA disease activity measures, further assessment of the MBDA’s performance characteristics is warranted.

INTRODUCTION

Rheumatoid arthritis (RA) is an autoimmune inflammatory arthritis characterized by synovitis, progressive damage, functional disability, extra-articular manifestations, and premature mortality (1). The 2015 American College of Rheumatology (ACR) and 2016 European Union League Against Rheumatism (EULAR) treatment guidelines recommend early treatment with disease modifying anti-rheumatic drug (DMARD) therapy in a treat-to-target strategy to obtain sustained remission or low disease activity (2, 3). This approach has been shown to improve clinical outcomes, decrease cost, and limit radiographic progression (4). By definition, adhering to a treat-to-target strategy for RA management requires regular assessment of RA disease activity.

Numerous RA disease activity measures have been developed that include various patient-reported measures, provider assessments, and laboratory measurements of inflammation (5). Following a critical review of the literature (including psychometric properties, feasibility, and cost), survey of practicing rheumatologists, and expert opinion, the ACR has recommended the use of the Disease Activity Score with 28-joint count (DAS28), Simplified Disease Activity Index (SDAI), Clinical Disease Activity Index (CDAI), Routine Assessment of Patient Index Data with 3 measures (RAPID3), Patient Activity Scale (PAS), or PAS-II (6). All of these measures incorporate subjective assessments of disease activity by the patient or provider, which can be influenced by factors other than RA disease activity, such as non-inflammatory pain (7). Laboratory measures such as erythrocyte sedimentation rate (ESR) and C-reactive protein (CRP) are objective, but nonspecific, insensitive, and may only measure one domain of disease activity (8). Given the aforementioned limitations in RA disease activity assessment, there remains a need for the development of improved measures of RA disease activity.

The Multi-Biomarker Disease Activity (MBDA) score is a novel, commercially available, blood test developed to assess RA disease activity (9). The algorithm used to calculate the MBDA was initially derived to predict simultaneously collected DAS28-CRP scores. The algorithm combines 12 individual serum biomarkers involved in the pathogenesis of RA (interleukin-6 [IL-6], tumor necrosis factor receptor type I [TNFRI], vascular cell adhesion molecule 1 [VCAM-1], epidermal growth factor [EGF], vascular endothelial growth factor A [VEGF-A], YKL-40, matrix metalloproteinase 1 [MMP-1], MMP-3, CRP, serum amyloid A [SAA], leptin, and resistin) to produce a disease activity score with values ranging from 0 to 100. As an objective measure of disease activity, the MBDA score may be useful in routine clinical practice or complement clinical disease activity assessment in the challenging comorbid patient. However, there have been concerns raised regarding the validity of the MBDA in measuring RA disease activity (10, 11).

Given the discrepant validity reported for the MBDA, the purpose of our study was to perform a systematic review of the MBDA score in RA and determine the convergent validity, or degree to which two measures that are measuring the same construct agree, of the MBDA with ACR-endorsed RA disease activity measures through a meta-analysis.

METHODS

We conducted a systematic review and meta-analysis following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (12) and registered the protocol with PROSPERO (ID: CRD42017060181), an international prospective registry of systematic reviews.

Search Strategy

A full description of the systematic literature review search strategy is available in the supplementary materials (Appendix 1). Briefly, led by a medical librarian (CMS), we searched MEDLINE, EMBASE, Scopus, Google Scholar, and the Cochrane library from the inception of each database through March 7, 2017 using combinations of terms for the MBDA score and RA.

Study Selection

For the systematic review, we included all published manuscripts in the English language reporting original observations relevant to MBDA performance in RA. To contribute to the meta-analysis, studies were required to report (or have the necessary data to report) a correlation between the MBDA and an ACR-endorsed RA disease activity measure (DAS28, SDAI, CDAI, PAS, PAS-II or RAPID3). Although included in the initial protocol, study heterogeneity in evaluating radiographic progression (study population, cut-off values for radiographic progression, duration of study follow-up, modeling of the MBDA score, and statistical analyses utilized), precluded meaningful meta-analysis of radiographic progression. Two authors (BRE, KAR) separately reviewed titles, abstracts, and full text to determine eligibility for inclusion. There was perfect agreement between reviewers for study inclusion in both the systematic review and meta-analysis.

Data Extraction

Two authors (TMJ, KAR) independently extracted study data in duplicate, including patient characteristics (age, sex, disease activity, and serologic status), study characteristics (study design, country of origin, inclusion/exclusion criteria, sample size, funding source, sample handling, and duration of follow-up), and study outcomes. Items extracted for the meta-analysis included correlation of MBDA with composite RA disease activity measures and correlation of change in MBDA with change in RA disease activity measures. Corresponding authors were contacted to provide missing data or to provide overall correlations for cohorts that were reported in multiple studies, to avoid duplication. In instances where the corresponding authors could not provide the requested data, we requested data directly from study sponsors.

Quality Assessment

Study quality was independently assessed by two authors (TMJ, KAR) using a 13-item assessment tool adapted from tumor biomarker reporting guidelines (13, 14). There was 96.6% agreement in the assessment of study quality items, with differences settled by third author review (BRE). Quality scores were reported as a percentage of quality items fulfilled.

Statistical Analysis

We calculated pooled correlation coefficients (r) with 95% confidence intervals (CI) of the MBDA with RA disease activity measures by performing a random-effects meta-analysis using the DerSimonian-Laird model (15). Studies that assessed MBDA performance in multiple cohorts or time points (e.g. baseline and follow-up) were modeled separately in the meta-analysis. In sensitivity analysis, we transformed correlation coefficients to Fisher’s Z scores prior to meta-analysis. Results were consistent with using untransformed correlation coefficients (data not shown). Stratified analyses were completed based on study time-point (baseline vs. follow-up), with baseline analyses also restricting each study cohort to one contribution in the meta-analysis. Finally, we performed analyses stratified by study design (observational vs. randomized controlled trial). Heterogeneity across studies was assessed using I². Publication bias was assessed using a funnel plot for correlation with DAS28-CRP, since this was the most frequently reported disease activity measure in our analyses. All analyses were performed using Stata v14.0 software (StataCorp).

RESULTS

Study Selection

Our search strategy identified 718 studies with 470 remaining after excluding duplicates (Figure 1). Full text review of 121 studies was completed with exclusion of those reported only in abstract form (n=98) or without original data (n=1), resulting in 22 articles included in the systematic review (9–11),(16–34). Eight studies reported correlations with RA disease activity measures and were included in meta-analyses (10, 11, 16, 17, 19, 22, 23, 25).

Figure 1. Flow diagram of study selection.

Search strategy identified 718 articles with 470 remaining after removing duplicates. After title and abstract review, 121 full text manuscripts were reviewed with 22 fulfilling criteria for inclusion into the systematic review and 8 further fulfilling criteria for inclusion in meta-analysis.

Study Characteristics from Systematic Review

Study and patient characteristics for all cohorts included in the systematic review are detailed in Table 1 and Table 2. Most studies (n=22) identified were secondary analyses, with original investigations including both observational studies and randomized controlled trials. Patients in the systematic review were predominantly female (60–91%) in the fifth to sixth decade (mean/median age 51/61 years), seropositive (61–97% rheumatoid factor [RF]; 55–98% anti-cyclic citrullinated peptide antibody [anti-CCP]), and had moderate to high disease activity (DAS28 ranged from 3.2–6.0, except for one study of patients in remission). Patients also included in the meta-analysis were similar (67–84% female; 51–61 years; 62–93% RF positive; 57–88% anti-CCP positive; DAS28 3.5–5.7). Sample sizes from individual studies ranged from 24 to 524 patients with assessments of 94 to 558 serum samples. Crescendo Bioscience (South San Francisco, CA) was the most frequently noted funding source, providing funding or support (ranging from MBDA measurement to full study support) in 18 of 22 studies. Specific contributions of study sponsors are listed in Table 1.

Table 1.

Characteristics of studies reporting on the multi-biomarker disease activity score in rheumatoid arthritis

Study, Year	Country	Original Study Design(s)	Sample Handling Sample Assay(s)	Funding Source(s)	Contributions of Study Sponsor(s)	Quality Score‡
Studies included in both systematic review and meta-analysis
Bakker, 2012	Netherlands	POS	Standard separator tubes, frozen after collection, stored at −20°C until analysis Meso Scale Discovery, ELISA.	Crescendo Bioscience	Conception and design, data collection, analysis, manuscript preparation	100
Curtis, 2012	Multiple	POS	Serum separator tubes, processed at study site, shipped overnight using NanoCool shippers (2–8°C) Meso Scale Discovery	Crescendo Bioscience Biogen Idec NIH Agency for Healthcare Research and Quality	Conception and design, data collection, analysis, manuscript preparation	100
Hirata, 2013	Netherlands	RCT	Serum separated, dispensed, and stored at −70°C Meso Scale Discovery	Crescendo Bioscience	Provided study support	77
Hambardzumyan, 2015	Sweden	RCT	Not reported Meso Scale Discovery	Crescendo Bioscience Swedish Rheumatism Association Stockholm County (ALF funds) Schering-Plough Sweden	MBDA score analysis provided at no cost	92
Hirata, 2015	Japan	ROS	Samples stored at −40°C after collection and −70°C after transport to site of analysis Meso Scale Discovery	Crescendo Bioscience Ministry of Health, Labor, and Welfare of Japan Ministry of Education, Culture, Sports, Science, and Technology of Japan University of Occupational and Environmental Health	Shipment of samples and MBDA analysis provided at no cost.	85
Fleischmann, 2016	Multiple	RCT	Not reported	Bristol-Meyers Squibb	Study design, manuscript review	77
Reiss, 2016	Multiple	RCT	Not reported Used reported Vectra® DA algorithm	Genentech F. Hoffmann-La Roche Ltd.	Study design, data collection and analysis, manuscript preparation	77
Krabbe, 2017	Denmark	POS	Samples stored at −80°C Same reagents and immunoassay instruments as the Vectra® DA test	Not reported	Not reported.	85
Studies included only in systematic review
Eastman, 2012	USA, Canada	POS	Aliquoted into single-use vials, stored at −80 °C until thawed for assay Meso Scale Discovery	Crescendo Bioscience	Conducted study	100
Centola, 2013	USA, UK	POS RCT	Serum separator tubes maintained at 2-8°C until frozen at −80°C. BRASS cohort shipped samples at ambient temperature prior to serum separation Luminex-based assays, Meso Scale Discovery, ELISA	Crescendo Bioscience Biogen Idec	Conception and design, data collection, analysis, manuscript preparation	100
Li, 2013	USA	POS	Not applicable	Crescendo Bioscience	Conducted study	85
Peabody, 2013	Germany	RCT	Not applicable	Crescendo Bioscience	Study design, manuscript preparation	91
van der Helm-van Mil, 2013	Netherlands	POS	Not reported Same reagents and immunoassay as Vectra® DA	Netherlands Organization for Health Research and Development Dutch Arthritis Association Crescendo Bioscience	Conception and design, data collection, analysis, manuscript preparation	77
Markusse, 2014	Netherlands	RCT	Samples stored at −70°C Meso Scale Discovery	Dutch Insurance Companies Schering-Plough B.V. Janssen B.V.	Provided study support	100
Michaud, 2015	USA	ROS	Not applicable	Crescendo Bioscience	Provided study support	100
van Vollenhoven, 2015	Sweden	RCT	De-identified, frozen serum samples. Not reported	Crescendo Bioscience Schering-Plough Sweden	Editorial, graphic, and statistical support; data analysis, manuscript preparation	77
Hambardzumyan, 2016	Sweden	RCT	Not reported Meso Scale Discovery	Crescendo Bioscience Swedish Rheumatism Association Stockholm County (ALF funds) Schering-Plough Sweden	MBDA score analysis provided at no cost	85
Hirata, 2016	Japan	POS	Samples stored at –40°C until transfer to Crescendo Bioscience then stored at −70°C Meso Scale Discovery	Crescendo Bioscience Research Grant-In-Aid for Scientific Research by the Ministry of Health, Labor and Welfare of Japan The Ministry of Education, Culture, Sports, Science and Technology of Japan The University of Occupational and Environmental Health, Japan.	Biomarker analysis and statistical support	92
Lee, 2016	USA	POS	De-identified, frozen, serum samples Meso Scale Discovery	Crescendo Bioscience	Generation of biomarker data, statistical analysis, manuscript formatting	100
Li, 2016	Netherlands	POS	De-identified, frozen, serum samples Meso Scale Discovery	Crescendo Bioscience	Funded sample handling, generation of biomarker data and statistical analysis	100
Rech, 2016	Germany	RCT	Serum stored at –80°C Meso Scale Discovery	Multiple*	MBDA score analysis provided at no cost.	100
Hambardzumyan, 2017	Sweden	ROS	Not reported Meso Scale Discovery	Crescendo Bioscience Swedish Rheumatism Association Stockholm County (ALF funds) Schering-Plough Sweden	MBDA score analysis provided at no cost.	77

Abbreviations: ROS, retrospective observational study; POS, prospective observational study; RCT, randomized controlled trial

^‡Quality score = percentage of 13 quality items reported

^*This study was supported by Crescendo Bioscience, the Deutsche Forschungsgemeinschaft (SPP1468-IMMUNOBONE), the Bundesministerium für Bildung und Forschung (BMBF; project METARTHROS), the Marie Curie project OSTEOIMMUNE, the TEAM and MASTERSWITCH projects of the European Union and the IMI funded project BTCure.

Table 2.

Baseline characteristics of patients in studies reporting on the multi-biomarker disease activity score in rheumatoid arthritis^*

Study / Year	Country	Original Study Design(s)	Baseline n (Patients/samples) Total n (Patients/Samples)**	Age Mean/Median(SD or IQR)	Sex, %F	DAS28-CRP Mean/Median (SD or IQR)	RF/anti-CCP+ (%)
Studies included in both systematic review and meta-analysis
Bakker, 2012	Netherlands	POS	72 / 72 74 / 120	53 (15)	70	5.6 (1.0)‡	68 / ----
Curtis, 2012	Multiple	POS
SP validation			230 / ----	58 (48-66)	77	4.1 (2.3-5.8)	93 / 88
SN validation			141 / ----	57 (46-65)	82	3.7 (2.4-4.9)	0
SN performance			141 / ----	58 (49-65)	79	3.5 (2.4-4.7)	0
Tx response			45 / 45 45 / 144	54 (39-64)	84	5.5 (4.9-6.4)	73 / ----
Hirata, 2013	Netherlands	RCT	91 / 91 125 / 179	53 (14)	74	5.5 (0.9)	62 / 57
Hambardzumyan, 2015	Sweden	RCT	235 / 235		72	5.4 (1.0)	65 / 57
Hirata, 2015	Japan	ROS	147 / 147 147 / 378	60 (50-68)	84	5.0 (4.3-5.7)	86 / ----
Fleischmann, 2016	Multiple	RCT	496 / 496 524 / ----	Ref	Ref	5.5 ( - )	Ref
Reiss, 2016	Multiple	RCT	48 / 48 78 / 107	51 (14)	82	5.7 (0.9)
Krabbe, 2017	Denmark	POS	50 / 50 50 / 284	61 (50-70)	67	4.9 (4.2-5.6)‡
Studies included only in systematic review
Eastman, 2012	USA, Canada	POS	512 / ----	60 (20-91)‡‡	76	3.2 (1.1-8.2)	76 / ----
Centola, 2013	USA, UK	POS RCT
Study I			128 / 128	60 (13)	82	5.8 (4.7-6.5)	83 / 63
Study II			320 / 320	59 (14)	80	4.0 (2.9-5.3)	83 / 62
Study III			85 / 255	59 (13)	91	3.8 (2.7-5.0)	64 / 62
Study IV			119 / 119	60 (14)	77	5.2 (4.1-6.2)	97 / 61
PoC Study			24 / 107	56 (13)	75	3.3 (2.2-4.4)
Li, 2013	USA	POS	101 / 101	62 (13)	82		63 / 45
Peabody, 2013	Germany	RCT
van der Helm-van Mil, 2013	Netherlands	POS	163 / 271	55 (14)	67		65 / 66
Markusse, 2014	Netherlands	RCT	91 / 91 125 / 180	53 (14)	75	5.8 (1.0)	62 / 56
Michaud, 2015	USA	ROS	Ref	Ref	Ref	Ref	Ref
van Vollenhoven, 2015	Sweden	RCT	347 / 347 347 / 474
MTX naïve cohort			220 / 220 220 / 220	Ref	Ref	5.7 ( - )‡	Ref
MTX-IR cohort			127 / 127 127 / 254	Ref	Ref	4.9 ( - )‡	Ref
Hambardzumyan, 2016	Sweden	RCT	220 / 220 220 / 558		71	5.7 (1.0)‡	65 / 57
Hirata, 2016	Japan	POS	83 / 83 83 / 249	59 (14)	84	5.7 (1.2)‡	87 / ----
Lee, 2016	USA	POS	198 / 198	58 (11)	85		63 / 62
Li, 2016	Netherlands	POS	163 / 271	55 (14)	67	3.3 (2.3-4.3)	66 / 67
Rech, 2016	Germany	RCT	94 / 94	55 (19)	60	1.9 (0.8)‡	61 / 56
Hambardzumyan, 2017	Sweden	ROS	157 / 157		80	6.0 (1.0)‡	62 / 55

Abbreviations: RF, rheumatoid factor; anti-CCP, anti-cyclic citrullinated peptide antibody; SP, seropositive; SN, seronegative; Tx, treatment; MTX-IR, methotrexate incomplete response; Ref, authors referred to original study

^*If left blank, study did not report item.

^**If single row of data provided, only total patients / samples available for report.

^‡DAS28 without modifier reported

^‡‡Full age range reported

Quality Assessment

After assessment of 13 quality items, all studies included in the systematic review fulfilled >75% of quality items (range 77–100%, Table 1). Complete details of study quality assessment are provided in Supplementary Table 1. The most common quality items missing were descriptions of sample handling (i.e. collection, preservation, and storage; n = 7), descriptions of confounding variables considered (n = 10), and assessments of biomarker performance in both univariate and multivariate analyses (n = 10).

Outcomes of Studies Identified in Systematic Review

Of the 22 studies, eight reported on MBDA correlations with a composite RA disease activity measure as a primary or secondary outcome with six identifying significant positive correlations (Table 3) (16, 17, 19, 22, 23, 25). The strength of correlations of MBDA score with RA disease activity measures are detailed below under ‘meta-analysis’. The MBDA discriminated between low and moderate-high disease activity categories based on DAS28-CRP (area under the receiver operating characteristic curve [AUROC] 0.70–0.86) (16, 17) and ACR/EULAR remission (AUROC 0.83) (22). There were conflicting results specific to the validity of the MBDA following DMARD initiation. The MBDA did not correlate with DAS28-CRP, CDAI, SDAI, or RAPID3 over two-years of follow-up in patients with active RA treated with adalimumab or abatacept in a randomized controlled trial (10). Correlations of the MBDA with DAS28-CRP decreased over 24 weeks of treatment with tocilizumab (r = 0.50 at baseline, r = 0.19–0.33 between weeks 4–24), as did agreement in disease activity categories (77.1% at baseline, 23.7% at week 24) (11). In contrast, change in the MBDA correlated with change in DAS28-CRP or DAS28-ESR after initiation of TNF inhibitors in a Japanese cohort (22).

Table 3.

Results of published literature on the multi-biomarker disease activity score in rheumatoid arthritis

Study (reference)	Primary Aim	Secondary Aim(s)	Summary of Results Reported
Studies included in both systematic review and meta-analysis
Bakker, 2012	Correlation with disease activity measures	Contribution of non-CRP biomarkers, MBDA response to treatment, ability to predict RP	MBDA correlated with DAS28-CRP and discriminated between remission/low and moderate/high DAS28-CRP disease activity categories (AUROC 0.86) Non-CRP biomarkers were independently associated with SJC28, TJC28, and VAS-GH. MBDA decreased with 6 months of treatment (53[18] -> 39[16]), more significantly in intensive treatment arm. MBDA did not predict radiographic progression.
Curtis, 2012	Establish criterion and discriminant validity	Contribution of non-CRP MBDA biomarkers Characterize performance in seropositive vs. seronegative patients	MBDA correlated with DAS28-CRP, CDAI, SDAI, and RAPID3, and discriminated low vs. mod/high DAS28-CRP (AUROC 0.70-0.77 across cohorts studied). ∆MBDA correlated with ∆DAS28-CRP, ACR response criteria and discriminated clinical response (DAS28-CRP AUROC 0.77; ACR50 AUROC 0.69). MBDA better correlated with disease activity measures in seropositive (vs. seronegative) patients; SDAI (r 0.55 vs. 0.29), CDAI (r 0.48 vs. 0.21), RAPID3 (r 0.47 vs. 0.26). Non-CRP MBDA biomarkers predicted DAS28-CRP.
Hirata, 2013	Correlation with disease activity measures	Ability to discriminate EULAR disease activity categories	MBDA significantly correlated with DAS28-ESR, SDAI, CDAI, and HAQ-DI. MBDA correlated with change in DAS28-ESR and SDAI (not CDAI) over one-year follow-up. Remission by MBDA associated with ACR/EULAR (AUROC 0.83), DAS28-ESR, CDAI, and SDAI remission criteria.
Hambardzumyan, 2015	Ability of baseline MBDA to predict radiographic progression (∆SHS>5)		Baseline MBDA higher (p<0.001) in patients with RP. MBDA independent predictor of RP as continuous (OR 1.05, 95% CI 1.02-1.08) or categorical variable (OR 3.86, 95% CI 1.04-14.26 for high vs. low/mod MBDA).
Hirata, 2015	Correlation with change in disease activity measures	Comparison between anti-TNF therapies	∆MBDA correlated with ∆DAS28-ESR and ∆DAS28-CRP, but not ∆CDAI or ∆SDAI. No difference in correlations between anti-TNF therapies.
Fleischmann, 2016	Correlation with disease activity measures	Correlation with radiographic progression	Not associated with DAS28-CRP, CDAI, SDAI, RAPID3, or radiographic progression over 2-year follow up.
Reiss, 2016	Effect of TCZ on correlation of MBDA with disease activity	Effect of TCZ on individual biomarkers in MBDA	Correlation of MBDA with DAS28-CRP decreased (Spearman’s p=0.50 at baseline -> p 0.19-0.33) between weeks 4-24, and agreement between low/mod/high MBDA and DAS28-CRP categories decreased (77.1% -> 23.7%) with 24 weeks of TCZ treatment. Individual analyte changes following TCZ treatment included an increase in IL-6 and a decrease in CRP and serum amyloid A.
Krabbe, 2017	Correlation with imaging measures of inflammation	Correlation with DAS28-CRP	MBDA did not correlate with imaging inflammation at baseline or week 52, and in general did not predict change in imaging inflammation. Correlated modestly with MRI synovitis (r=0.43), MRI bone marrow edema (r=0.36), and ultrasound (US) power Doppler score (r=0.35) at week 26. MRI/US were concordant with MBDA in detecting disease activity for patients in DAS28-CRP remission. MBDA correlated with DAS28-CRP at baseline and week 26. ∆MBDA correlated with ∆DAS28-CRP from baseline to 26 weeks, but not baseline to 52 weeks.
Studies included only in systematic review
Eastman, 2012	Analytical performance of MBDA multiplex assay.		MBDA biomarker assays were precise, with minimal interference or cross-reactivity
Centola, 2013	Development of MBDA score	Impact of comorbidities on MBDA	MBDA algorithm developed through biomarker screening, feasibility studies, and assay optimization. Co-morbidities assessed (hypertension, osteoarthritis, osteoporotic bone fractures, degenerative joint disease, diabetes, asthma) were not associated with the MBDA.
Li, 2013	Effect on provider treatment choices	Effect on overall drug use, correlation with PGA	Treatment plans changed in 38% of patients with MBDA. No effect on overall drug use. Modest correlation with PrGA (r=0.35).
Peabody, 2013	Impact on quality scores using Clinical Performance and Value vignettes	Appropriate use of DMARDs, number of labs or imaging tests ordered, use of other resources	Quality scores improved 12% with MBDA. Appropriate use of DMARDS improved with comorbid patients. No effect on number of labs or imaging tests ordered, or use of health care resources.
van der Helm-van Mil, 2013	Frequency of radiographic progression (∆SHS>3) in MBDA remission	Detection of subclinical disease activity	Greater rate of non-progression in MBDA remission vs. non-remission (93% vs 70%). +LR of non-progression in MBDA remission 4.73 (95% CI 1.67-15.0). High MBDA score in DAS28-CRP remission increased risk of radiographic progression (RR 2.28, 95% CI 1.13-3.68).
Markusse, 2014	Ability to predict radiographic progression (∆SHS>5)		MBDA at baseline discriminates radiographic progressors vs. non-progressors better than DAS (AUROC 0.767, 95% CI 0.639–0.896) and predicts RP based on MBDA at baseline (RR 1.039, 95% CI 1.018–1.059) and 1 year (RR 1.037, 95% CI 1.009–1.065) associated with increased RP.
Michaud, 2015	Outcomes and cost when used in RA management		Decreased HAQ scores (0.09 in 1 year, 0.02 over 10 years), increase quality-adjusted life years 0.08, and decreased overall cost US$457.
van Vollenhoven, 2015	Impact on recruitment to clinical trials based on data from SWEFOT trial		High MBDA (>44) enhanced recruitment in low CRP (<10) patients – additional 24% MTX-naïve patients and 47% MTX-incomplete responders included.
Hambardzumyan, 2016	Ability of MBDA at multiple time points to predict radiographic progression (∆SHS>5)	Ability of MBDA to predict RP in Triple Therapy (TT) vs. anti-TNF treated patients	Persistently low/mod MBDA was predictive of less RP. MBDA was numerically (but not statistically) superior to CRP, ESR, and DAS28 for identifying RP. Patients with high MBDA scores on TT had increased risk of RP compared to anti-TNF therapy (45% vs. 25% at baseline and 57% vs. 32% at month 3).
Hirata, 2016	Correlation with radiographic progression		MBDA correlated with ∆SHS (r=0.47 at week 24; AUROC 0.44 over 52 weeks). High MBDA increased risk of ∆SHS >3 (RR 14.3[2.5-85.5]) at week 24 compared to low MBDA. In patients with low or mod/high DAS28, MBDA further discriminated risk of radiographic progression.
Lee, 2016	Correlation with disease activity measures	Utility in RA patients with fibromyalgia (FM).	MBDA correlated with CRP in RA patients with (r=0.89) or without (r=0.73) concomitant FM. Composite indices (DAS28-CRP, SDAI, CDAI, RAPID3) all greater in patients with concomitant FM, though no difference in MBDA between these groups.
Li, 2016	Correlation with radiographic progression		High MBDA increased risk of ∆SHS>3 (RR 3.4 if MBDA 45-51; 4.3 if MBDA 52-59; 5.2 if MBDA ≥60) and ∆SHS>5 (RR 12.4, 12.0, and 17.4) compared with low MBDA. MBDA independent risk factor for radiographic progression after adjustment for SJC28, DAS28-CRP, CRP, and pre-existing joint damage.
Rech, 2016	Ability to predict disease relapse in patients tapering DMARDs		Baseline MBDA scores significantly (p=0.0001) higher in patients with subsequent relapse. MBDA and Anti-CCP independent predictors of disease relapse. Able to predict >80% of relapses using anti-CCP plus MBDA.
Hambardzumyan, 2017	Predicting response to Triple therapy (TT) vs. anti-TNF		More patients with low MBDA responded to TT vs. anti-TNF (88% vs 18%); more patients with high MBDA responded to anti-TNF (35% vs 58%).

Abbreviations: ACR50, American College of Rheumatology response criteria - 50% improvement; AUROC, area under the receiver operating characteristic curve; CDAI, clinical disease activity index; HAQ-DI, health assessment questionnaire without disability index; IL-6, interleukin-6; MTX, methotrexate; PrGA, provider global assessment of disease activity; RAPID3, routine assessment of patient index data 3; RF, rheumatoid factor; RP, radiographic progression; RR, relative risk; SDAI, simple disease activity index; SHS, sharp score as modified by van der Heijde; SJC28, swollen 28-joint count; TJC28, tender 28-joint count; TCZ, tocilizumab; VAS-GH, visual analogue scale of patients’ assessment of general health; +LR, positive likelihood ratio.

In contrast to other composite ACR-recommended RA disease activity measures, MBDA scores were not influenced by comorbid fibromyalgia in one study (26). Additional comorbidities including hypertension, osteoarthritis, degenerative joint disease, osteoporotic bone fractures, diabetes, and asthma also did not affect MBDA performance (9). Exclusion of CRP, a common component between the MBDA score and the DAS28-CRP and SDAI, did not attenuate MBDA score performance in two independent studies (16, 17).

Nine studies investigated the ability of the MBDA to predict radiographic progression (Table 3). In secondary analyses of both observational studies and randomized controlled trials, the MBDA predicted radiographic progression in six studies (19, 20, 24, 28, 29, 33). Sharp van der Heijde Score (SHS) cut-offs, analytic methods, and resulting effect sizes were highly variable. Relative to patients with low disease activity, patients with high disease activity by MBDA score had a relative risk (RR) of radiographic progression range of 1.04–14.30, and an odds of radiographic progression range of 1.03–3.86. RA patients in MBDA remission were less likely to demonstrate radiographic progression with a positive likelihood ratio for non-progression (change SHS <3) of 4.73 (33). MBDA more effectively discriminated radiographic progressors from non-progressors than 44-joint DAS (AUROC 0.767 vs. 0.521) (29) as well as CRP and ESR (20). Additionally, among those in low or moderate/high disease activity by DAS28, MBDA scores further discriminated risk of radiographic progression (24). In contrast, two studies demonstrated less capability of the MBDA score to predict radiographic progression (10, 16). In an observational study of RA patients treated with MTX, MBDA was not predictive of radiographic progression (OR 1.033, 95% CI 0.995–1.072) (16). Likewise, in a randomized controlled trial comparing abatacept and adalimumab in MTX-inadequate responders, MBDA categories were not associated with radiographic non-progression (10). Finally, the MBDA did not demonstrate consistent correlation with novel imaging measures of inflammation such as magnetic resonance imaging (MRI)-based synovitis/bone marrow edema or ultrasound power doppler score (25).

Initial studies of the MBDA on quality of care in RA have yielded favorable results. A decision analysis projecting cost-effectiveness estimated an overall decrease in Health Assessment Questionnaire (HAQ) scores, increase in quality adjusted life years, and decreased net total cost (savings in labor force participation and work productivity offsetting an increase in direct medical costs) accompanying use of the MBDA score (30). These effects were largely driven by the effect of the MBDA on provider treatment choices, utilizing data from a previous study showing that treatment plans changed in 38% of patients when an MBDA score was provided (27). Overall quality scores improved on clinical performance and value vignettes in providers receiving MBDA scores (31).

An emerging area of research is whether the MBDA predicts treatment response. Among RA patients with inadequate response to MTX in the Swedish Farmacotherapy trial (SWEFOT), more patients with low MBDA responded to Triple Therapy (MTX, hydroxychloroquine, and sulfasalazine) than the addition of infliximab (88% vs. 18%; p = 0.006), defined as achieving a DAS28 <3.2 or EULAR good response. Patients with high MBDA scores responded more frequently to the addition of infliximab rather than Triple Therapy (58% vs. 35%; p = 0.040) (21). In a separate analysis from the SWEFOT trial, patients with high MBDA scores receiving Triple Therapy were more likely to have RP after two years of follow up than patients on MTX and infliximab (20).

Additional outcomes reported on the MBDA included its potential to increase recruitment to clinical trials (i.e. identifying patients with low CRP who would otherwise be excluded) (34), as well as its ability to predict disease relapse (32).

Meta-analysis

Correlations of the MBDA with composite RA disease activity measures were available from eight studies (Figure 2; Supplementary Table 2). Using a random-effects meta-analysis with 3,242 samples from six studies (8 cohorts and 17 time points), the MBDA score was moderately correlated with DAS28-CRP (r = 0.41, 95% CI 0.36–0.46) with low-to-moderate heterogeneity (I² = 39.59%, p = 0.07). Performance was stronger at baseline (r = 0.48, 95% CI 0.39–0.56) compared to follow-up time points (r = 0.36, 95% CI 0.31–0.40). Performance was also numerically superior in observational studies (r 0.49, 95% CI 0.38–0.61) compared to randomized controlled trials (r 0.38, 95% CI 0.34–0.42), but there was substantially greater heterogeneity detected in observational studies (I² 54.7%, p = 0.03) than randomized controlled trials (I² 0.0%, p = 0.57). The MBDA score performed similarly in its correlation with DAS28-ESR (r = 0.48, 95% CI 0.38–0.58), though this was reported in fewer studies (3 studies with 3 cohorts, 5 different time points, n = 1,367) and with greater heterogeneity (I² = 67.81%, p = 0.01). In four studies (5 cohorts, 12 time points, n = 2,664), the MBDA score demonstrated a low-to-moderate correlation with SDAI (r = 0.35, 95% CI 0.26–0.43) with high heterogeneity (I²=74.58%, p < 0.001). Correlation with CDAI was less robust (r = 0.26, 95% CI 0.19–0.33), with moderate heterogeneity (I²=66.91%, p < 0.001) among the five contributing studies (6 cohorts, 13 time points, n = 2,719). The MBDA score was weakly correlated with the RAPID-3 (r = 0.23, 95% CI 0.19–0.27) in two studies (3 cohorts, 9 time points, n=2,416) with very low study heterogeneity (I²=0.00%, p = 0.92).

Figure 2. Correlation of the multi-biomarker disease activity score with rheumatoid arthritis disease activity measures.

Forest plots demonstrating the correlation of the multi-biomarker disease activity score with RA disease activity measures including (A) DAS28-CRP, (B) SDAI, (C) CDAI, and (D) RAPID3. Studies assessing correlation at multiple time points are modeled separately for meta-analysis.

*Patients with initial follow-up at 6 weeks and again at 12 weeks if adequate treatment response not obtained in the BRASS and NESTED cohorts (Curtis 2012).

Correlation of MBDA change with change in RA disease activity was also determined with random-effects meta-analysis (Figure 3). Correlations of MBDA change with change in DAS28-CRP were reported in five studies (6 cohorts, n = 1857), with a moderate correlation observed (r = 0.42, 95% CI 0.37–0.48). Change in MBDA score also moderately correlated with change in DAS28-ESR (r = 0.53, 95% CI 0.46–0.60) in 3 studies (3 cohorts and 5 time points, n = 825). Weaker correlations were seen with change in SDAI (r = 0.33, 95% CI 0.26–0.40) in 4 studies (5 cohorts, 9 time points, n = 1,710), CDAI (r = 0.26, 95% CI 0.20–0.33) in 4 studies (5 cohorts, 9 time points, n = 1,718) and RAPID-3 (r = 0.31, 95% CI 0.25–0.38) in 3 studies (3 cohorts, 7 time points, n = 1,617). Heterogeneity observed for comparisons of change in MBDA score with change in RA disease activity was low-to-moderate (I² 0.00 – 43.44%, p 0.08–0.77).

Figure 3. Correlation of the change in multi-biomarker disease activity score with change in rheumatoid arthritis disease activity measures over time.

Forest plots demonstrating correlation of change in multi-biomarker disease activity score with change in RA disease activity measures including (A) DAS28-CRP, (B) SDAI, (C) CDAI, (D) RAPID3. Studies assessing correlation at multiple follow up points are modeled separately for meta-analysis.

*Patients followed up initially at 6 weeks, and again at 12 weeks if adequate treatment response not obtained in the BRASS and NESTED cohorts (Curtis 2012).

Publication bias was assessed by funnel plot analysis with DAS28-CRP, the most frequently reported RA disease activity measure (Supplementary Figure 1). Study distribution was largely symmetric suggesting no substantial publication bias among studies included in the meta-analysis.

DISCUSSION

The MBDA score, a composite score of 12 serum biomarkers that was initially derived to predict DAS28-CRP, provides an objective measure of RA disease activity by eliminating subjective assessments from the patient or provider, a potentially useful tool when caring for RA patients. However, there is conflicting data regarding its convergent validity with other RA disease activity measures. In this study, we report the first systematic review assessing the performance of the MBDA score in RA across multiple outcomes and the first meta-analysis of the convergent validity with ACR-endorsed RA disease activity measures.

The DAS28 using either ESR or CRP is often considered the “gold standard” RA disease activity measure and is frequently used for disease activity measurement in clinical trials. Using a random-effects meta-analysis including 3,242 MBDA measurements, we found moderate correlations between the MBDA score with both DAS28-CRP (r = 0.41) and DAS28-ESR (r = 0.48). These were the strongest correlations observed between the MBDA and composite RA disease activity measures, perhaps anticipated given the MBDA score was derived to predict DAS28-CRP (9). SDAI, another ACR endorsed RA disease activity measure that includes patient, physician, and acute phase reactant components, demonstrated weaker correlation with the MBDA (r = 0.35) compared to DAS28 measures, but stronger than those with CDAI (r = 0.26) and RAPID-3 (r = 0.23). Because the CDAI differs from the SDAI only by the exclusion of CRP, this suggests that the common component of acute phase reactants is unlikely to account entirely for its performance in measuring RA disease activity. Further supporting this are two studies identified in our systematic review where MBDA scores were associated with RA disease activity after exclusion of CRP (16, 17).

Variable heterogeneity was observed in the meta-analyses of MBDA scores with RA disease activity measures. There was moderate-to-high heterogeneity for cross-sectional correlations examining MBDA correlations with DAS28-ESR, SDAI, and CDAI. The majority of this heterogeneity appeared to be related to exceptionally strong performance in the BeSt study (22). Exclusion of this study reduced variability (I²) by 40% for DAS28-ESR, 24% for SDAI, and 34% for CDAI, respectively. The remaining cross sectional and longitudinal correlations between MBDA scores and RA disease activity measures had low-to-moderate heterogeneity by I². Though there was only moderate heterogeneity for correlations between the MBDA scores and DAS28-CRP, sensitivity analyses identified this heterogeneity to be limited to observational studies (I² 54.7% in observational studies) and minimal heterogeneity to exist in studies from randomized controlled trials (I² 0.0%).

Our systematic review identified additional studies characterizing the performance of the MBDA score in assessing RA disease outcomes. The MBDA score was able to discriminate between low vs. moderate/high disease activity in three studies (16, 17, 22), an important characteristic with the goal of treating to remission or low disease activity (2). MBDA scores were also predictive of radiographic progression in several studies and independent cohorts, though study heterogeneity precluded a formal meta-analysis of this performance characteristic. The ability of this tool to predict a well-established complication of active RA further supports the validity of the MBDA as a measure of RA disease activity. Non-inflammatory pain such as that resulting from comorbid fibromyalgia can influence traditional RA disease activity measures, complicating disease activity measurement. However, as an objective serum biomarker measurement, MBDA scores were not influenced by fibromyalgia (26).

As with any test being implemented in clinical use, the cost of testing is an important consideration to patients, providers, and the healthcare system. Our systematic review identified limited study of cost-effectiveness, with a simulated analysis suggesting cost-effectiveness when balancing improvement in quality of life and increased labor force participation (30). Further analysis of its cost-effectiveness from actual patient data is needed to confirm these findings.

Our study has limitations. Only 22 studies of the MBDA score in RA were published during the search period, and only 8 of these fulfilled eligibility criteria to be included in the meta-analysis. We included only published manuscripts because patient cohorts were often used in multiple studies and including results from “gray literature” would have increased the probability of duplicate inclusion of subjects/samples. Additionally, the MBDA manufacturer supported 18 of the 22 included studies, though in 5 of these, support was only providing MBDA measurement in the absence of any further involvement in study design or analysis. Details of how serum samples were collected, processed, and stored prior to analysis were inconsistently reported. Because sample handling may affect biomarker measurement (35), this represents an unknown confounder in this meta-analysis. Finally, a novel scoring algorithm for the MBDA score has been recently developed which accounts for age and body mass index (36). Because of our search dates, studies using this revised score were not included.

There are a number of strengths to our study. We conducted a systematic review of the MBDA score in RA, searching five databases for eligible studies. Rigorous methodology was used with duplicate assessment of study eligibility, data abstraction, and quality assessment. Moreover, study quality assessment was completed using an adapted tool from tumor biomarker reporting guidelines (13, 14). Finally, corresponding authors and companies were contacted to provide additional data on studies that did not initially report correlations with composite RA disease activity measures and to prevent duplication of subjects/samples in meta-analysis.

In summary, this is the first systematic review and meta-analysis to examine the performance of the MBDA score in RA. The MBDA score demonstrates moderate convergent validity with DAS28-CRP and DAS28-ESR, less robust correlation with SDAI, and weak convergent validity with CDAI and RAPID3, composite measures lacking acute phase reactants. It also appears to predict radiographic progression and influence provider decision-making, though these findings need further validation in light of high levels of variability and low effect sizes observed across studies. While the MBDA score represents another tool to measure RA disease activity, further assessment of its ability to improve RA management (such as the ability to predict treatment response or comparisons of patient outcomes for individuals treated to target with the MBDA score vs. other RA disease activity measures), validation of its performance characteristics, evaluation of a recently proposed scoring modification, as well as appraisal through independently funded efforts is necessary.

SIGNIFICANCE AND INNOVATION.

• Through a systematic review and meta-analysis, the multi-biomarker disease activity score demonstrated moderate convergent validity with DAS28-CRP and DAS28-ESR, but weaker correlations with other rheumatoid arthritis disease activity measures.

• Additional performance characteristics of the multi-biomarker disease activity score, such as predicting radiographic changes, discriminating disease activity states, and predicting treatment response, are summarized through a systematic review.