Multicentre study to improve clinical interpretation of rheumatoid factor and anti-citrullinated protein/peptide antibodies test results

Abstract

Background

Rheumatoid factor (RF) and anti-citrullinated protein/peptide antibodies (ACPA) are important biomarkers for diagnosis of rheumatoid arthritis (RA). However, there is poor harmonisation of RF and ACPA assays. The aim of this study was to refine RF and ACPA interpretation across commercial assays.

Materials and methods

Six total RF isotype-non-specific assays, 3 RF IgM isotype-specific assays and 9 ACPA immunoglobulin G assays of 13 different companies were evaluated using 398 diagnostic samples from patients with RA and 1073 disease controls.

Results

Using cut-offs proposed by the manufacturer, there was a large variability in diagnostic sensitivity and specificity between assays. Thresholds of antibody levels were determined based on predefined specificities and used to define test result intervals. Test result interval-specific likelihood ratios (LRs) were concordant across the different RF and ACPA assays. For all assays, the LR for RA increased with increasing antibody level. Higher LRs were found for ACPA than for RF. ACPA levels associated with LRs >80 were found in a substantial fraction (>22%) of patients with RA.

Conclusion

Defining thresholds for antibody levels and assigning test result interval-specific LRs allows alignment of clinical interpretation for all RF and ACPA assays.

Key messages.

What is already known about this subject?

• There is poor harmonisation of rheumatoid factor (RF) and anti-citrullinated protein/peptide antibodies (ACPA) assays with large variability in sensitivity and specificity.

What does this study add?

• Variability in diagnostic sensitivity between RF and ACPA assays can be reduced by defining thresholds that are based on predefined specificity.

• Thresholds of antibody levels based on predefined specificities were used to define test result intervals.

• Test result interval-specific likelihood ratios (LRs) were concordant across RF and ACPA assays.

How might this impact on clinical practice or further developments?

• Defining thresholds for antibody levels and assigning test result interval-specific LRs aligns clinical interpretation of RF/ACPA assays.

Introduction

Rheumatoid arthritis (RA) is a chronic, systemic autoimmune disease that affects approximately 1% of the population. It is characterised by joint inflammation that can lead to cartilage and bone damage if left untreated and can potentially result in disability, reduction of quality of life and increased mortality.¹ Early identification of RA (even in the preclinical phase) is critical as early intervention results in better long-term disease control and can prevent joint damage.^2–8 Identifying a person with RA can be challenging, even for expert physicians.^{9 10} Clinical signs and symptoms alone are usually insufficient to diagnose RA. The presence of rheumatoid factor (RF) and anti-cyclic citrullinated protein/peptide antibodies (ACPA) in serum supports the diagnosis of RA.¹¹ However, there is poor agreement among the currently available RF and ACPA assays, which may have an impact on RA diagnosis and classification of a patient.^12–14 This poor agreement not only relates to heterogeneity in test characteristics (nature of antigen, assay set up, affinity/avidity and antibody detection methodology),¹⁵ but also, and largely, to heterogeneity in the way the cut-off is set. Differences in cut-off values among assays were clearly demonstrated by the fact that the international standard for RF (W1066) scored either negative, equivocal or positive, depending on the assay used.¹³ Thus, there is an evident need to harmonise clinical interpretation of autoantibody tests used for RA diagnosis.

The aim of this international multicentre study is to harmonise interpretation of commonly used commercially available RF and ACPA assays. In order to overcome differences in the arbitrary units of the different assays, we applied the concept of likelihood ratio (LR), which is a unit-independent method to express test results.^16–18 The LR is defined as the fraction of patients with a particular test result divided by the fraction of controls with the same test result. For example, a test result with an LR of 10 indicates that this test result is 10 times more likely to be found in patients with the disease than in (disease) controls, whereas a test result with an LR of 0.1 is 10 times less likely to be found in patients with the disease than in (disease) controls. The LR allows to convey immediately clinically relevant information related to the antibody level. The higher the antibody level, the higher the LR for disease.^{13 19–21} Based on the LR, the post-test probability of disease can be estimated (post-test odds=pre-test odds×LR).²⁰

Materials and methods

Patients and samples

Serum samples were obtained from 11 European hospitals: Division of Rheumatology, Medical University of Vienna (Austria), University Hospital of Leuven (Belgium), University Hospital of Ghent (Belgium), OLV Hospital of Aalst (Belgium), National Institute of Rheumatology and Physiotherapy of Budapest (Hungary), Centre Hospitalier de Luxembourg (Luxembourg), University Medical Centre of Ljubljana (Slovenia), Sahlgrenska Academy Hospital of Gothenburg (Sweden), University Hospital of Linköping (Sweden), University Hospital of Basel (Switzerland) and Kantonsspital of Aarau (Switzerland). An overview of the patients with RA and controls recruited at each institute is given in online supplemental table 1.

The RA cohort (n=398) consisted of consecutive newly diagnosed patients with RA. A patient with synovitis was considered to have RA when no alternative diagnosis could better explain the symptoms, and when the treating rheumatologist initiated methotrexate treatment or other disease-modifying antirheumatic drugs. Demographic data (age and sex) and the clinical data included in the ACR/EULAR RA classification criteria (number and type of joints affected, duration of symptoms, C reactive protein and erythrocyte sedimentation rate) were recorded. Descriptive characteristics of the RA cohort are provided in online supplemental table 2.

The disease control cohort (n=1073) consisted of (a) a rheumatologic disease control group (RDCG) (ie, consecutive patients consulting a rheumatology clinic for the first time but in whom RA was eventually excluded; the exact composition is listed in online supplemental table 1) (n=656); (b) specific disease control groups (DCG) (ie, patients with established diagnoses^22–30 of antineutrophil cytoplasmic antibody (ANCA) associated vasculitis (n=24) with arthritis, osteoarthritis (n=25), psoriatic arthritis (n=25), reactive arthritis (n=20), spondyloarthritis (n=25), systemic lupus erythematosus (SLE) (n=50) and primary Sjögren’s syndrome (pSS)) (n=48) and (c) a healthy control (HC) group (n=200) (anonymised blood donors from the Red Cross-Flanders (2008) (n=50) and healthy individuals recruited by OLV Hospital Aalst (n=150)). Sample collection complied with the World Medical Association’s Declaration of Helsinki (as revised in 2013). Due to the unavailability of reagents or an instrument in Belgium at the timepoint of the study, the analyses of, respectively, ACPA Bio-Rad and RF/ACPA Siemens Healthineers (hereafter referred by Siemens) were performed after the 19 analyses batches of the other analysis runs. These analyses were performed on a smaller cohort of samples consisting of 362 patients with RA, 594 RDCG, 206 DCG and 196 HC samples. An overview of the demographic features of the different study cohorts (RA and controls) is included in online supplemental table 3.

RF and ACPA assays

All samples were analysed with six total RF isotype-non-specific assays, three RF IgM isotype-specific assays and nine ACPA immunoglobulin (IgG) assays. An overview of the different RF and ACPA assays and their specific test characteristics is provided in online supplemental table 4. The tests were performed by the OLV Hospital Aalst according to the manufacturers’ instructions. Results were expressed in the arbitrary units of the respective assay.

For RF, the RF EliA IgM on Phadia 250 (Thermo Fisher Scientific, Sweden), RF‐II (Roche Diagnostics, Germany) and Diagam RF (Diagam, Belgium) on a cobas c 501 analyser (Roche Diagnostics), AUTOZYME RF IgM (Cambridge Life Science, UK) on a QUANTA-Lyser two instrument (Inova Diagnostics, San Diego, California, USA), Alegria RF IgM on Alegria instrument (Orgentec, Germany), RF on Alinity c (Abbott, Germany), RF on Vitros 4600 (Ortho-Clinical diagnostics, Raritan, New York, USA), RF on AU680 (Beckman Coulter, Brea, California, USA), and RF on Atellica CH-930 (Siemens, UK) were evaluated.

For ACPA, CCP EliA IgG on Phadia 250, anti-CCP IgG on a cobas e 601 analyser, IMMUNOSCAN CCPlus (Svar Life Science, Sweden) on a QUANTA-Lyser 2 instrument, CCP IgG on IDS-iSYS (Immunodiagnostic Systems, UK), anti-CCP hs on Alegria (Orgentec), anti-CCP IgG on Alinity i, anti-CCP ELISA (IgG) on Euroimmun Analyzer I-2P (Euroimmun, Germany), anti-CCP IgG on BioPlex 2200 (Bio-Rad Laboratories, Hercules, California, USA) and anti-CCP IgG on Atellica IM 1300 (Siemens, UK) were included. All included ACPA assays were CCP2 tests (online supplemental table 4).

Statistical analysis

Diagnostic performance was evaluated by sensitivity, specificity and (test result interval-specific) LR. Statistics were performed using MEDCALC (V.17.1, Ostend, Belgium) and Analyse-it for Microsoft Excel (V.4.90).

Results

Performance characteristics of RF and ACPA

For all methods, RF and ACPA levels were significantly higher in patients with RA than in controls, except for RF in pSS (online supplemental table 5, supplemental figure 1A, B, supplemental table 6, supplemental figure 2A, B).

Based on the cut-off values proposed by the manufacturer, the sensitivity of the RF assays ranged from 51.8% to 74.4%, the specificity from 72.4% to 93.9% and the LR for RA from 2.7 to 9.0 (inter-manufacturer variation (coefficient of variation, CV%) in LR for RA was 29.5%; table 1). The sensitivity of the RF IgM assays (range 51.8%–62.3%) was comparable to the sensitivity of total RF assays (range: 51.8%–74.4%).

Table 1.

Performance characteristics of RF assays at the cut-off defined by the manufacturer. 95% CIs are presented within brackets

	Thermo Fisher RF IgM	Cambridge RF IgM	Orgentec RF IgM	Roche RF	Diagam RF	Abbott RF	Ortho RF	Beckman RF	Siemens RF
Unit	IU/mL	U/mL	U/mL	IU/mL	kIU/mL	IU/mL	IU/mL	IU/mL	IU/mL
Measuring range	0.4–200	1.1–600	1.0–500	10–130	2.2–120	20–200	8.6–120	10–120	3.5–90
Manufacturer’s cut-off	5	15.3	20	14	20	30	12	14	14
Sensitivity (%)	62.3 (57.3 to 67.1)	53.3 (48.2 to 58.3)	51.8 (46.7 to 56.8)	64.3 (59.4 to 69.0)	54.5 (49.5 to 59.5)	51.8 (46.7 to 56.8)	74.4 (69.8 to 78.6)	65.8 (60.9 to 70.5)	68.2 (63.2 to 73.0)
Specificity (%)	89.8 (87.8 to 91.5)	91.7 (89.9 to 93.3)	91.7 (89.9 to 93.3)	89.8 (87.8 to 91.5)	93.9 (92.3 to 95.3)	93.9 (92.2 to 95.2)	72.4 (69.6 to 75.1)	88.4 (86.4 to 90.3)	86.5 (84.2 to 88.5)
LR+	6.1 (5.0 to 7.4)	6.4 (5.1 to 8.0)	6.2 (5.0 to 8.2)	6.3 (5.2 to 7.6)	9.0 (7.0 to 11.6)	8.4 (6.5 to 10.8)	2.7 (3.4 to 3.0)	5.7 (4.8 to 6.8)	5.0 (4.2 to 6.0)
LR−	0.4 (0.4 to 0.5)	0.5 (0.5 to 0.6)	0.5 (0.5 to 0.6)	0.4 (0.3 to 0.4)	0.5 (0.4 to 0.5)	0.5 (0.5 to 0.6)	0.4 (0.3 to 0.4)	0.4 (0.3 to 0.4)	0.4 (0.3 to 0.4)

LR+, positive likelihood ratio (sensitivity/1−specificity); LR−, negative likelihood ratio (1−sensitivity/specificity); RF, rheumatoid factor.

Based on the cut-off values proposed by the manufacturer, the sensitivity of the ACPA assays ranged from 57.8% to 64.6%, the specificity from 94.9% to 97.8% and the LR for RA from 12.6 to 26.6 (CV% of LR for RA was 21.3%; table 2).

Table 2.

Performance characteristics of ACPA assays at the cut-off defined by the manufacturer. 95% CIs are presented between brackets

	Thermo Fisher ACPA	Roche ACPA	Svar ACPA	IDS ACPA	Orgentec ACPA	Abbott ACPA	Euroimmun ACPA	BioRad ACPA	Siemens ACPA
Unit	U/mL	U/mL	U/mL	AU/mL	U/mL	U/mL	RU/mL	U/ml	U/mL
Measuring range	0.4–340	7–500	1.6–3200	1.2–320	1–1000	0.5–195.6	1–200	0.5–300	0.54–200
Manufacturer’s cut-off	10	17	25	5	20	5	5	3	5
Sensitivity	63.1 (58.1 to 67.8)	62.6 (57.6 to 67.3)	61.3 (56.3 to 66.1)	62.6 (57.6 to 67.3)	57.8 (52.8 to 62.7)	62.1 (57.1 to 66.8)	64.6 (59.7 to 69.3)	60.8 (55.5 to 65.8)	63.5 (58.3 to 68.5)
Specificity	97.6 (96.5 to 98.4)	96.7 (95.5 to 97.7)	97.4 (96.3 to 98.3)	96.4 (95.5 to 97.7)	97.8 (96.7 to 98.6)	97.7 (96.6 to 98.5)	94.9 (93.4 to 96.1)	96.8 (95.5 to 97.8)	97.3 (96.1 to 98.2)
LR+	26.3 (17.7 to 38.3)	19.2 (13.7 to 26.8)	23.5 (16.2 to 34.1)	19.2 (13.7 to 26.8)	25.8 (17.2 to 38.7)	26.6 (17.0 to 39.5)	12.6 (9.6 to 16.5)	18.9 (13.3 to 26.9)	23.4 (16.0 to 34.4)
LR−	0.4 (0.3 to 0.4)	0.4 (0.3 to 0.4)	0.4 (0.4 to 0.4)	0.4 (0.3 to 0.4)	0.4 (0.4 to 0.5)	0.4 (0.3 to 0.4)	0.4 (0.3 to 0.4)	0.4 (0.4 to 0.5)	0.4 (0.4 to 0.5)

ACPA, anti-citrullinated protein/peptide antibody; LR−, negative likelihood ratio (1−sensitivity/specificity); LR+, positive likelihood ratio (sensitivity/1−specificity).

At a threshold corresponding to a specificity of 92.5% and 97.5%, the sensitivity of the RF tests ranged from 49.5% to 61.6% and 22.6% to 39.2%, respectively, with a corresponding range in LR from, respectively, 6.7 to 8.2 (CV% of LR for RA was 7.5%) and 9.0 to 15.6 (CV% of LR for RA was 15.7%) (online supplemental table 7A). A substantial fraction (>22%) of the patients with RA had total RF and RF IgM results exceeding the 97.5% specificity threshold.

At a threshold corresponding to a specificity of 97.5% and 99.0%, the sensitivity of ACPA tests ranged from 58.8% to 63.6% and 41.5% to 50.5%, respectively, with a corresponding range in LR from, respectively, 23.4 to 25.3 (CV% of LR for RA was 2.4%) and 40.4 to 49.3 (CV% of LR for RA was 7.5%) (online supplemental table 7B). A substantial fraction of the patients with RA (41.5%–51.7%) had ACPA values exceeding the 99% specificity threshold.

The range of sensitivities associated with a predefined specificity was narrower than the range of sensitivities associated with the cut-offs proposed by the manufacturer.

Test result interval-specific LR

Next, we evaluated how the LR for RA increases with increasing antibody level. In order to determine test result intervals in a consistent manner across assays, we defined intervals that are delimited by thresholds that correspond to predefined specificities (90.0%, 92.5%, 95.0% and 97.5% for RF and 90.0%, 97.5%, 99.0% and 99.8% for ACPA). Online supplemental table 7 shows the sensitivities and LR associated with the various predefined specificities. Online supplemental table 8 shows the sensitivities and LRs after recalculations based on different compositions of the control population. The thresholds that correspond to a high specificity (eg, 97.5%) were lower for an HC population compared with a diseased control population.

For each test result interval, the interval-specific LR for RA was calculated, as well as the fraction of patients and controls that had a result within this interval. The results are presented for RF and ACPA in, respectively, tables 3 and 4 and illustrated for one RF and ACPA assay in, respectively, figures 1 and 2.

Table 3.

RF test result-specific LRs

	Interval	Fraction of controls	Fraction of patients	LR	95% CI
Thermo Fisher RF IgM CO=10 IU/mL	<5.0	0.897	0.377	0.42	0.37 to 0.48
	5.0–7.5	0.027	0.058	2.14	1.25 to 3.65
	7.5–15.0	0.027	0.085	3.16	1.95 to 5.12
	15.0–45.0	0.025	0.188	7.49	4.90 to 11.45
	45.0–≥200	0.023	0.291	12.51	8.25 to 18.97
Cambridge RF IgM CO=15.3 U/mL	<12.3	0.899	0.427	0.47	0.42 to 0.53
	12.3–17.2	0.025	0.055	2.20	1.27 to 3.81
	17.2–26.5	0.025	0.058	2.30	1.33 to 3.96
	26.5–126.0	0.025	0.234	9.29	6.15 to 14.03
	126.0–≥600.0	0.025	0.226	8.99	5.94 to 13.60
Orgentec RF IgM CO=20 IU/mL	<15.6	0.901	0.457	0.51	0.46 to 0.57
	15.6–22.8	0.025	0.048	1.90	1.07 to 3.37
	22.8–33.5	0.023	0.065	2.80	1.64 to 4.80
	33.5–95.8	0.025	0.123	4.89	3.10 to 7.72
	95.8–≥500.0	0.025	0.307	12.18	8.16 to 18.19
Roche RF CO=14 IU/mL	<14.6	0.900	0.362	0.40	0.35 to 0.46
	14.6–17.3	0.024	0.023	0.93	0.44 to 1.97
	17.3–25.5	0.025	0.095	3.79	2.35 to 6.13
	25.5–57.2	0.025	0.128	5.09	3.24 to 8.00
	57.2–≥130.0	0.025	0.392	15.58	10.53 to 23.05
Diagam RF CO=20 kIU/L	<10.9	0.900	0.352	0.39	0.34 to 0.45
	10.9–16.9	0.024	0.070	2.90	1.72 to 4.89
	16.9–24.6	0.025	0.065	2.60	1.53 to 4.39
	24.6–57.9	0.025	0.158	6.29	4.07 to 9.73
	57.9–≥120.0	0.025	0.354	14.08	9.48 to 20.91
Abbott RF CO=30 IU/mL	<10.0	0.915	0.427	0.47	0.42 to 0.52
	10.0–24.6	0.009	0.035	3.77	1.69 to 8.43
	24.6–45.4	0.025	0.078	3.10	1.87 to 5.12
	45.4–107.6	0.025	0.188	7.49	4.90 to 11.45
	107.6–≥200	0.025	0.271	10.78	7.19 to 16.18
Ortho RF CO=12 IU/mL	<16.3	0.900	0.352	0.39	0.34 to 0.45
	16.3–19.7	0.025	0.048	1.90	1.07 to 3.37
	19.7–28.4	0.024	0.101	4.15	2.57 to 6.70
	28.4–60.0	0.025	0.126	4.99	3.17 to 7.86
	60.0–≥120.0	0.025	0.374	14.88	10.04 to 22.05
Beckman RF CO=14 IU/mL	<16.8	0.899	0.354	0.39	0.34 to 0.45
	16.8–21.0	0.028	0.055	1.98	1.15 to 3.39
	21.0–30.0	0.023	0.085	3.67	2.22 to 6.07
	30.0–69.8	0.024	0.148	6.12	3.91 to 9.56
	69.8–≥120.0	0.025	0.357	14.18	9.55 to 21.05
Siemens RF CO=14 IU/mL	<18.0	0.903	0.362	0.40	0.35 to 0.46
	18.0–22.0	0.024	0.047	1.95	1.06 to 3.59
	22.0–32.0	0.025	0.091	3.63	2.19 to 6.02
	32.0–75.1	0.023	0.138	5.98	3.71 to 9.66
	75.1–≥90.0	0.025	0.362	14.42	9.56 to 21.74

CO, cut-off; IgM, immunoglobulin M; LRs, likelihood ratios; RF, rheumatoid factor.

Table 4.

ACPA test result-specific LRs

	Interval	Fraction of controls	Fraction of patients	LR	95% CI
Thermo Fisher ACPA CO=10 U/mL	<3.3	0.902	0.299	0.33	0.29 to 0.39
	3.3–9.5	0.073	0.065	0.90	0.49 to 1.29
	9.5–79.8	0.015	0.133	8.93	1.65 to 48.42
	79.8–324.6	0.007	0.204	27.30	2.71 to 271.40
	324.6–≥340.0	0.003	0.299	106.90	2.58 to 4425.96
Roche ACPA CO=17 U/mL	<3.5	0.955	0.362	0.38	0.33 to 0.43
	3.5–36.1	0.020	0.030	1.54	0.86 to 2.62
	36.1–230.4	0.015	0.193	12.97	7.67 to 21.60
	230.4–≥500.0	0.010	0.415	40.44	22.21 to 73.64
Svar ACPA CO=25 U/mL	<4.3	0.898	0.352	0.39	0.34 to 0.45
	4.3–26.6	0.076	0.035	0.46	0.31 to 0.68
	26.6–147.8	0.015	0.113	7.58	4.43 to 12.82
	147.8–398.0	0.007	0.226	30.33	14.85 to 61.93
	398.0–≥3200.0	0.003	0.273	97.95	31.29 to 306.68
IDS ACPA CO=5 AU/mL	<2.0	0.904	0.281	0.31	0.27 to 0.37
	2.0–7.7	0.071	0.100	1.41	1.05 to 1.89
	7.7–43.4	0.015	0.121	8.08	4.75 to 13.70
	43.4–236.8	0.007	0.273	36.73	18.08 to 74.62
	236.8–≥320.0	0.003	0.224	79.98	25.46 to 251.27
Orgentec ACPA CO=20 U/mL	<5.8	0.900	0.314	0.35	0.30 to 0.40
	5.8–14.7	0.075	0.080	1.08	0.79 to 1.44
	14.7–137.7	0.015	0.156	10.45	6.17 to 17.54
	137.7–926.0	0.007	0.178	23.93	11.63 to 49.25
	926.0–≥1000.0	0.003	0.271	97.06	31.00 to 303.91
Abbott ACPA CO=5 U/mL	<1.0	0.913	0.314	0.34	0.30 to 0.40
	2.7–4.5	0.062	0.063	1.02	0.72 to 1.43
	4.5–53.1	0.015	0.186	12.47	7.38 to 20.83
	53.1–188.2	0.007	0.186	24.94	12.13 to 51.25
	188.2–≥196.0	0.003	0.251	89.87	28.66 to 281.74
Euroimmun ACPA CO=5 RU/mL	<3.0	0.901	0.319	0.35	0.31 to 0.41
	3.0–9.9	0.074	0.070	0.96	0.69 to 1.30
	9.9–37.5	0.015	0.106	7.08	4.15 to 12.05
	37.5–187.7	0.007	0.276	37.07	18.25 to 75.28
	187.7–≥200.0	0.003	0.229	81.78	26.04 to 256.81
BioRad ACPA CO=3 RU/mL	<1.5	0.912	0.334	0.37	0.32 to 0.43
	1.5–5.5	0.063	0.077	1.22	0.87 to 1.67
	5.5–127.5	0.015	0.160	10.63	6.33 to 17.98
	127.5–≥300.0	0.010	0.428	42.65	22.76 to 79.92
Siemens ACPA CO=5 U/mL	<1.1	0.904	0.273	0.30	0.26 to 0.36
	1.1–5.6	0.071	0.094	1.32	0.99 to 1.78
	5.6–57.8	0.015	0.155	10.27	6.13 to 17.43
	57.8–≥200.0	0.010	0.478	47.60	25.45 to 89.02

ACPA, anti-citrullinated protein/peptide antibodies; CO, cut-off; LRs, likelihood ratios.

Figure 1.

Box whisker plots of the likelihood ratio of serum rheumatoid factor (RF) for the different test result-specific intervals, delimited by thresholds that correspond to predefined specificities (90.0%, 92.5%, 95.0% and 97.5%). See table 3 for further details. Results are shown for the Beckman Coulter RF assay, but similar results were obtained for the other RF assays included in the study.

Figure 2.

Box whisker plots of the likelihood ratio of serum anti-citrullinated protein/peptide antibodies (ACPA) for the different test result-specific intervals, delimited by thresholds that correspond to predefined specificities (90.0%, 97.5%, 99.0% and 99.8%). See table 4 for further details. Results are shown for the Orgentec ACPA assay, but similar results were obtained for the other ACPA assays included in the study.

The LR for RF test results that were below the 90.0% specificity threshold varied between 0.39 and 0.51, depending on the assay. The LRs of the RF test result interval delimited by thresholds corresponding to a specificity of 90.0% and 92.5% (found in 2.3%–7.8% of patients with RA) ranged from 0.93 to 3.77 (which is close to 1; most of the corresponding 95% CIs included 1). The LRs further increased with increasing antibody levels (figures 1 and 2). It ranged from 2.30 to 4.15 and from 4.89 to 9.29 for the test result intervals delimited by thresholds corresponding to 92.5%–95.0% specificity and 95.0%–97.5% specificity, respectively. A substantial fraction (>22% for IgM or total RF) of the patients with RA had RF results exceeding the 97.5% specificity threshold, with LRs ranging from 8.99 to 15.58.

The LR for ACPA test results that were below the 90.0% specificity threshold varied between 0.30 and 0.39, depending on the assay. The LR of the ACPA test result interval delimited by thresholds corresponding to specificities of 97.5% (found in 3%–10% of the patients) ranged from 0.46 to 1.54 (which is close to 1; 95% CI included 1). The LR increased with increasing ACPA antibody levels (figures 1 and 2). It ranged from 7.08 to 12.97, from 23.93 to 37.07 and from 79.98 to 106.90 for the ACPA test result intervals delimited by thresholds corresponding to 97.5%–99.0% specificity, 99.0%–99.8% specificity and >99.8% specificity, respectively. For some assays (Roche ACPA, Bio-Rad ACPA and Siemens ACPA), the 99.8% specificity threshold could not be determined and the LR for values exceeding the 99.0% specificity threshold ranged from 40.44 to 47.60. A substantial fraction of the patients with RA (41.5%–51.7%) had ACPA values exceeding the 99.0% specificity threshold.

The LR depends on the composition of the patient and control population included. We recalculated the test result interval-specific LRs for different compositions of the control population (HCs; consecutive rheumatologic disease controls; and all control groups excluding HCs, pSS and SLE groups). The results are summarised in online supplemental tables 9 and 10 for, respectively, RF and ACPA.

Discussion

Even though many studies have attempted to discover and validate novel biomarkers for RA,⁶ ACPA and RF remain the most well-established and widely available RA markers for diagnosis and classification.^{31 32} However, there is great variability among commercially available RF and ACPA assays.^12–14 This is mainly related to differences in the way companies define cut-off values and to the differences in the patient and control populations used to establish the cut-off values. Importantly, the variability among RF and ACPA assays affects patient RA classification according to 2010 ACR/EULAR criteria^{13 33} and jeopardises RA diagnosis in routine rheumatological practice.¹¹

In order to overcome these shortcomings, we established an international cohort of samples obtained from RA patients at diagnosis and from controls and defined test result-specific LRs at predefined specificity thresholds. In that way, we could align interpretation across companies. Test result-specific LRs bear more clinically relevant information than the dichotomous information associated with a single cut-off value. Our study demonstrated remarkably similar LRs across the different companies at predefined specificity thresholds (table 4). Reporting test resul-specific LRs helps with the interpretation of autoantibody tests (and thus creates added value).^{16 17} A test result with an LR of 1 indicates that this test result is equally likely to be found in (disease) controls as in patients with the disease. Generally, an LR >10 is considered to support a diagnosis and an LR <0.1 to rule out a diagnosis.³⁴ It is important that laboratory professionals and clinicians become more familiar with the concept of LRs and that they develop an intuitive feeling for the clinical relevance of an LR.¹⁸ There is increasing awareness that efforts should be undertaken to harmonise interpretation of RF and ACPA test results. Rönnelid et al recently stated that ‘complementing harmonised cut-offs with information about test result-specific likelihood ratios … will increase the richness and information value of autoantibody data …’.³⁵ The concept of test result-specific LRs has recently also been proposed to harmonise interpretation of proteinase-3 and myeloperoxidase ANCA.^36–38

The companies that participated in the study support the concept of test result-specific LRs and will evaluate options for providing that information to their customers, for example, by the organisation of scientific seminars, white papers or product flyers. Laboratory professionals are encouraged to report the LR associated with a particular test result obtained with a specific assay (manufacturer). Data presented in online supplemental tables 9 and 10 allow to report the LR that best fits with the patient population served by the laboratory (eg, tertiary centre vs primary care) (through selection of the LR derived from the most appropriate control group).

When test result-specific LRs are applied, relevant differences in clinical significance between low antibody levels versus higher antibody levels become apparent. The higher the antibody level, the higher the LR and the higher the likelihood for disease.^{20 21} Moreover, when test result-specific LRs are applied, differences in the clinical value between high levels of RF and ACPA become apparent as well. For example, a high antibody level, that is, a value exceeding the 97.5% specificity (for RF) and 99% specificity (for ACPA) threshold was associated with, respectively, an LR of 7.0–15.6 for RF and an LR of 40.4–49.3 for ACPA, depending on the test kit used. Cut-off values for RF and ACPA are not aligned across manufacturers. Relatedly, cut-offs defining ‘high antibody positivity’ based on three times the manufacturer’s cut-off (as applied in the 2010 classification criteria¹¹) are arbitrarily chosen and not aligned as well.¹³ In RA diagnosis, serology plays a crucial role in the diagnostic assessment of patients with less typical symptoms, that is, with a lower pre-test probability.¹¹ Using test result-specific LRs on a nomogram, one can calculate the post-test probability of disease.^{11 32 34} Consequently, the use of harmonised LRs directly impacts clinical decision-making.

In a subsequent study, we show the impact of using specificity-based thresholds on RA classification. In that study, we also show that RA classification can be improved by taking into account the nature of the antibody (RF vs ACPA), the antibody level and single or double positivity (manuscript in preparation).

In conclusion, defining thresholds for antibody levels and assigning test result-specific LRs helps to harmonise interpretation of RF and ACPA immunoassays in routine clinical practice. The LR is dependent on the antibody level, and the antibody type (higher LR for ACPA than for RF). Applying the same criteria on the same cohort of patients and controls is a powerful tool to align interpretation across assays from different companies. It is the shared societal responsibility of laboratory professionals, in vitro diagnostics and clinicians to harmonise interpretation of laboratory test results such that they can be reliably applied in routine diagnostic protocols.³⁹

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

This study involves human participants and the study was approved by the institutional ethical boards of the University Hospitals Leuven (B322201939877) and OLV Ziekenhuis Aalst. Participants gave informed consent to participate in the study before taking part.