Analytical performance of commercial myositis-specific autoantibody tests evaluated against immunoprecipitation assays as a reference standard: a systematic review and meta-analysis

Abstract

Objective:

To evaluate the analytical performance of commercial myositis-specific autoantibody (MSA) assays against immunoprecipitation (IP) assays.

Methods:

A systematic literature search was conducted in PubMed, Web of Science, and Scopus through July 2024. Data were extracted on study design, participant characteristics, index tests, and 2 × 2 contingency tables for diagnostic performance. Study quality was assessed using the QUADAS-2 tool. Sensitivity and specificity were calculated for each dataset and presented as paired forest plots and summary receiver operating characteristic (SROC) curves. A hierarchical SROC model was used to estimate pooled sensitivity and specificity for meta-analysis.

Results:

Of 3,156 articles, 23 met inclusion criteria and were judged to have low risk of bias across all QUADAS-2 domains. The most frequently evaluated commercial assay was the line blot assay (LBA; 16 studies), followed by enzyme immunoassay (EIA; 9 studies). In the meta-analyses, the highest pooled sensitivity was observed for anti-MDA5 with EIA (95.7%), followed by anti-SAE with LBA (88.3%), anti-PL-12 with LBA (87.2%), and anti-Jo-1 and anti-MDA5 with LBA (82.8%). Lower sensitivities were observed for anti-Mi-2 (67.4%), anti-NXP2 (69.7%), and anti-TIF1-γ (63.8%) with LBA. Pooled specificity ranged from 94.7% to 99.3% across MSA assays, but a false-positive result was a common concern for LBA, except for anti-EJ.

Conclusion:

Among commercially available MSA assays, LBA is the most commonly used in the literature. However, potential false-positive or false-negative results pose a clinical challenge.

Graphical Abstract:

Introduction

Myositis-specific autoantibodies (MSAs) are detected in over 60% of patients with idiopathic inflammatory myopathies (IIMs), which is a heterogenous group of diseases characterized by inflammation and damage of skeletal muscle in association with autoimmunity (1). Patients with IIMs also have extra-muscular manifestations, such as skin rashes, arthritis, interstitial lung disease (ILD), cardiomyopathy, and gastrointestinal dysmotility. Clinical presentation, treatment response, and outcomes are highly variable among IIM patients. The discovery and characterization of MSAs have fundamentally improved management of IIM patients (1). Individual MSAs are useful adjunctive laboratory tests in the diagnosis of IIMs and are closely associated with different clinical manifestations of IIMs. Thus, accurate assessment of MSAs is pivotal to support diagnosis, predict prognosis, and formulate therapeutic strategies, leading to a personalized medical approach for IIM patients (2).

Developments in immunoassay technology over the past two decades has further advanced the field of MSA detection (1).Virtually all MSAs were first identified by immunoprecipitation (IP) assay, and their utility in diagnosis and disease subgrouping were established using this method. The IP assay is considered as the ‘gold-standard’ test for the detection of MSAs, but this assay requires specialized skills and poses inherent risks to laboratory personnel, such as the handling of cultured cells and radioisotope in laboratories and the procedure is complicated and time-consuming. Thus, convenient assay systems for detecting MSAs have been developed for clinical use, and include enzyme immunoassay (EIA), immunoblots such as line blot assay (LBA), and multiplex bead-based assay. However, the accessibility, cost and reliability of MSA testing depend on individual assays employed. Nevertheless, commercial assays, such as EIA and LBA, have become available as a tool for detection of MSAs in clinical practice across the world. In 2019, the Myositis Autoantibody Scientific Interest Group (SIG) of the International Myositis Assessment and Clinical Studies (IMACS) Group conducted an online survey to better understand perceived concerns from the international myositis community regarding the reliability of commercial MSA assays and how they are being used (3). Commercial EIA and LBA systems are popular assays used worldwide, but the commonly used assays were different among regions and countries. Nevertheless, over 80% of the respondents had concerns about the reliability of MSA testing in terms of false-positive and false-negative results. Despite this concerning issue, over 80% reported that the MSA results influenced their diagnostic confidence and their decisions on treatment in clinical practice. This survey urged us to develop evidence-based guidance for use of commercial MSA assays in clinical practice. For this purpose, the SIG assessed the performance of commercial MSA assays against the reference test, IP assays, by means of a systematic review (SR) and meta-analysis.

Methods

Protocol registration and search strategy

Our protocol for SR was created according to the PRISMA-diagnostic test accuracy statement (4). The protocol was registered on the PROSPERO database in September 2021 (registration number #189663). A systematic literature search through PubMed, Web of Science and Scopus were performed using terms for the target condition (myositis, myopathy, polymyositis, dermatomyositis), antibodies, and targeted antigens by MSAs from January 1946 through July 2024. Full details of our search strategy are shown in the supplemental Table S1.

Eligibility criteria for SR

Diagnostic cohort/cross sectional studies or diagnostic case-control studies that fulfill the following criteria were eligible for inclusion: 1) populations, children or adults (regardless of gender, ages, and races) with IIM or myositis spectrum disease, including anti-synthetase syndrome (ASyS). The study also included comparator patient cohorts comprising other connective tissue diseases, non-inflammatory myopathies and/or healthy subjects; 2) Index test, MSAs determined by commercial assays for routine diagnostic use; 3) Reference standard, RNA/protein IP assay; 4) Target condition: whether MSAs are positive or negative, as measured by an assay. Studies that fulfilled the above criteria were also considered eligible for inclusion for the secondary objective if they incorporated ≥2 commercial assays for detection of MSAs within the same population. Case report, review, and editorial articles were excluded. In addition, studies regarding laboratory/research-based assays were also excluded from SR.

Study selection

We conducted SR based on stratification by the following individual MSAs: anti-aminoacyl-tRNA synthetase (ARS), anti-3-hydroxy-3-methylglutaryl-coenzyme A reductase (HMCGR), anti-melanoma differentiation–associated gene 5 (MDA5), anti-Mi-2, anti-nuclear matrix protein 2 (NXP2), anti-small ubiquitin-like modifier-1 activating enzyme (SAE), anti-signal recognition particle (SRP), and anti-transcriptional intermediary factor 1-γ (TIF1-γ) which are available for detection in commercial assays. We organized 5 SR subgroups composed of a group leader and two reviewers: 1) anti-ARS, 2) anti-HMGCR/SRP, 3) anti-MDA5, 4) anti-Mi-2/NXP2, and 5) anti-SAE/anti-TIF1-γ subgroups. At first, representative investigators independently (NN, LG, TG) screened all the search results for title/abstract review, and then two reviewers of each subgroup independently reviewed the full article texts assigned to their respective MSAs within the SR. Any discordant results were solved by discussing between them at first. If necessary, the subgroup leader served as the third reviewer, and consensus was achieved.

Data extraction and quality assessment

Two reviewers independently extracted data on participants, study design, index tests as well as 2×2 data on test performance. The extracted details are shown in supplemental Table S2. The output was checked for accuracy and consistency by the corresponding subgroup leader. Study quality was assessed using the QUADAS-2 tool (5). This tool includes the following 4 domains: 1) patient selection, 2) index test, 3) reference standards and 4) flow and timing. The risk of bias and concerns regarding applicability are judged as “low”, “high” or “unclear” for each domain. Two reviewers independently assessed study quality. The grading was checked for consistency by the corresponding subgroup leader. If there were discordant results, these were discussed among the members of the subgroup and consensus was reached.

Data synthesis and meta-analysis

Sensitivity and specificity were calculated for each set of 2×2 data and plotted in summary receiver operating characteristics curves (SROC) if there were ≥2 studies, and demonstrated as paired forest plots using RevMan 5.4.1 (Cochrane Collaboration, Oxford, UK). Heterogeneity was assessed visually using paired forest plots and SROC plots according to previously published guidelines (6). If plots in observed studies ended up close to the SROC curve, it was determined that the presence of heterogeneity was due to variation in threshold among studies. To consider presence of the heterogeneity regardless of degree, the hierarchical SROC (HSROC) model was used to estimate pooled sensitivity and specificity with 95% confidence intervals (CIs) and confidence regions around the summary points for meta-analyses involving ≥5 studies according to previously published guidelines (6). These analyses were performed using R version 4.2.1 (R Foundation for Statistical Computing, Vienna, Austria). Two reviewers in each subgroup evaluated the overall quality of evidence for meta-analysis data on MSA tests using the GRADE system, based on the previous definitions (7) as “high quality”, “moderate quality”, “low quality”, or “very low quality”. If discordant results were found, these were discussed with the corresponding subgroup leader and consensus was reached.

For the second objective, we calculated the positive percent agreement (PPA), negative percent agreement (NPA), and total positive agreement (TPA) between individual commercial assays, and evaluated the consistency of the results between those individual commercial assays statistically by Cohen’s kappa coefficient: if the kappa is less than 0, “no agreement”; 0–0.2, “slight agreement”; 0.2–0.4, “fair agreement”; 0.4–0.6, “moderate agreement”; 0.6–0.8, “substantial agreement”; and 0.8–1.0, “almost perfect agreement” according to previously published criteria (8).

Results

Study selection and study characteristics

A total of 3,156 articles were identified according to the search strategy. We removed 1,156 articles before the title/abstract screening due to duplicate records. Out of the remaining 2,000 articles, we reviewed 572 articles for the full-text screening. Ultimately, 23 articles were selected for SR (Figure 1) (9–31).

Figure 1. PRISMA flow diagram.

Of 23 articles, 18 were cohort studies (9–26) and five were cross-sectional studies (27–31). Non-IIM patients and/or healthy subjects were also included in 13 articles (Table 1). The LBA, with the product name EUROLINE (EUROIMMUN, Lübeck, Germany), was the most commonly used commercial assay for detecting MSAs in 16 studies, followed by the EIA, with the product name MESACUP (Medical and Biological Laboratories, Tokyo, Japan) in 9, and the immunoblot assay, with the product name BlueDiver Dot (D-tek Mons, Belgium) in one study (Table1). No other commercial assays were used in the selected articles.

Table 1.

Characteristics of selected 23 studies for systematic review

Author, year (ref)	Country	Population (no)	Age, mean, (SD); female, %	Assay: Index test	Measurement of myositis-specific autoantibodies
					ARS*	Jo-1	EJ	PL-7	PL-12	MDA5	Mi-2	NXP2	SAE	SRP	TIF1-γ
Nakashima, 2014 (26)	Japan	IIM (250), Other CTD (276), IIP (168), HC (30)	N.A	EIA	○
Kuwana, 2023 (27)	Japan	DM (116)	50 (16), 89	EIA	○					○	○				○
Shinoda, 2022 (8)	Japan	IIM (24), other CTD (13), IIP (7)	58 (12), 80	EIA, LBA	○	○	○	○	○
Ghirardello, 2010 (9)	Italy, Sweden	IIM (208), non-IIM (34), other CTD (146), HC (50)	49 (17), 77 for IIM	LBA		○
Loganathan, 2022 (10)	UK	IIM (109), HC (225)	N.A	EIA		○	○	○	○	○	○
Tansley, 2020 (11)	United Kingdom	IIM (461)	N.A	IBA, LBA		○	○	○	○	○	○	○	○	○	○
Cavazzana, 2016 (12)	Italy	IIM (57)	N.A	LBA		○	○	○	○	○	○	○	○	○	○
Chang, 2023 (13)	South Korea	IIM (153), HC (79)	51 (14), 72 for IIM, 45 (14), 49 for HC	LBA		○	○	○	○	○	○	○		○	○
Espinosa-Ortega, 2019 (14)	Sweden	IIM (110), HC (60)	N.A	LBA		○	○	○	○	○	○	○	○	○	○
Hamaguchi, 2020 (15)	Japan	IIM (80), other CTD (220)	N.A	LBA		○	○	○	○		○			○
Loganathan, 2024 (16)	United Kingdom	IIM (147)	37 (23–48)**, 75	LBA		○	○	○	○	○	○	○	○	○	○
Mecoli, 2020 (17)	United States	IIM (281)	52 (14), 71	LBA		○	○	○	○		○			○
Cavazzana, 2019 (18)	Italy	IIM (54)	N.A	LBA		○	○			○	○	○		○	○
Mahler, 2019 (19)	United Kingdom	IIM (175)	53 (4–83)***, 70	LBA			○	○	○	○	○	○		○	○
Gono, 2019 (28)	Japan	DM/CADM (70)	N.A	EIA						○
Sato, 2016 (29)	Japan	IIM (242), other CTD (190), IIP (154), HC (123)	55 (15), 70	EIA						○
Fiorentino, 2019 (20)	United States	DM	47 (17), 74	EIA, LBA						○	○	○	○		○
Fujimoto, 2016 (21)	Japan	IIM (242), other CTD (190), IIP (154), HC (123)	55 (15), 70	EIA							○
Pinal-Fernandez, 2020 (30)	United States	IIM and non-IIM (666)	N.A	LBA							○
Albayda, 2021 (22)	United States	IIM (1844), non-IIM (283)	N.A	LBA									○
Peterson, 2018 (23)	United States	Disease (194), HC (67)	N.A	LBA									○
Mulhearn, 2022 (24)	United Kingdom	IIM (42), HC (68)	N.A	EIA											○
Westerdahl, 2021 (25)	United States	DM (26)	53 (18), 62	LBA											○

^*Mixed antigens including Jo-1, EJ, PL-7, PL-12, and KS.

^**median (interquartile range).

^***median (range)

SD, standard deviation; DM, dermatomyositis; IIM, idiopathic inflammatory myopathy; CTD, connective tissue disease, IIP, idiopathic interstitial pneumonia; HC, healthy control; CADM, clinically amyopathic DM; N.A, not available; EIA, enzyme immunoassay (MESACUP, Medical and Biological Laboratories, Tokyo, Japan); LBA, line blot assay (EUROLINE, EUROIMMUN, Lubeck, Germany); IBA, immunoblot assay (BlueDiver Dot, D-tek, Mons, Belgium).

Risk of bias in individual studies

We assessed risk of bias and applicability concerns using QUADAS-2 in the 23 selected articles (Figure 2A, B). Most of the selected studies had a low risk of bias in patient selection, index test, reference test, and flow and timing, although there are several articles which had high or unclear risk. In terms of applicability concerns, almost all of the articles had low concerns (Figure 2A, B).

Figure 2. Assessment of risk of bias by QUADAS-2.

Results in individual selected articles (A) and integrated results of all 23 selected articles were shown (B).

Sensitivity and specificity in individual studies

Sensitivity and specificity were shown as paired forest plots for individual studies in each MSA (Figure 3 for MSA assays involved in ≥5 studies and available for meta-analysis and supplemental Figure S1 for MSA assays involved in <5 studies). Sensitivity was variable in most results, except for anti-PL-12 with the LBA and anti-MDA5 with the EIA. Specificity consistently exceeded 90% for all MSAs with the LBA or EIA. In addition, sensitivity and specificity were plotted in SROC for individual MSAs by each assay (supplemental Figure S2). Regarding variable sensitivity due to heterogeneity among the studies, the observed plots were visually close to the SROC curve in the LBA for all MSAs, except for anti-Jo-1, anti-MDA5, and anti-Mi-2, and the EIA for all MSAs: anti-ARS, anti-Mi-2, anti-MDA5, and anti-TIF1-γ (supplemental Figure S2).

Figure 3. Paired forest plots for individual studies in each MSA.

LBA, line blot assay; EIA, enzyme immunoassay

Meta-analysis findings

We were able to conduct 11 meta-analyses using data obtained from 19 articles (Table 2 and Figure 4). These included 10 performed with LBA for detection of various MSAs, and one conducted with EIA for detection of anti-MDA5. The highest pooled sensitivity was observed for anti-MDA5 with the EIA (95.7%), followed by anti-SAE with the LBA (88.3%), anti-PL-12 with the LBA (87.2%), and anti-Jo-1 (82.8%) and anti-MDA5 (82.8%) with the LBA (Table 2). Of particular note, pooled sensitivity was particularly low for anti-Mi-2 (67.4%), anti-NXP2 (69.7%), and anti-TIF1-γ (63.8%) with the LBA. On the other hand, pooled specificity ranged from 94.7% to 99.3% across MSA assays, but false-positive results were commonly seen in LBA-based MSA assays, except for anti-EJ (Figure 4). Anti-MDA5 EIA represented the best pooled sensitivity of 95.7% and best pooled specificity of 99.3% with the best AUC of 0.99. Plots for sensitivity and specificity in HSROC were scattered outside the confidence region in all MSA assays, except for EIA measuring anti-MDA5-autoantibodies (Figure 4). Overall quality of evidence evaluated by the GRADE was high in anti-MDA5 EIA and anti-SAE LBA, and moderate in the remaining MSA measurements (Table 2).

Table 2.

Meta-analysis findings from 19 articles

MSA	Commercial Assay	Number of Studies	Number of participants	Pooled sensitivity (95%CI)	Pooled specificity (95%CI)	AUC	Overall of quality of evidence*
Anti-Jo-1	LBA	10	1,749	82.8 (65.8 to 92.4)	95.9 (91.1 to 98.2)	0.963	Moderate
Anti-EJ	LBA	10	1,411	70.5 (55.4 to 82.1)	99.1 (98.3 to 99.5)	0.896	Moderate
Anti-PL-7	LBA	9	1,643	77.2 (64.6 to 86.2),	97.2 (95.3 to 98.4)	0.893	Moderate
Anti-PL-12	LBA	9	1,644	87.2 (75.6 to 93.7)	97.0 (95.0 to 98.2)	0.876	Moderate
Anti-MDA5	EIA	5	771	95.7 (84.8 to 98.7)	99.3 (98.1 to 99.8)	0.99	High
Anti-MDA5	LBA	8	1,335	82.8 (74.2 to 88.9)	96.3 (91.2 to 98.5)	0.896	Moderate
Anti-Mi-2	LBA	11	2,580	67.4 (49.9 to 81.2)	94.7 (91.9 to 96.6)	0.941	Moderate
Anti-NXP2	LBA	8	1,224	69.7 (56.9 to 80.1)	97.4 (95.4 to 96.5)	0.946	Moderate
Anti-SAE	LBA	7	1,206	88.3 (62.8 to 97.1)	95.8 (75.7 to 96.4)	0.959	High
Anti-SRP	LBA	9	1,642	74.4 (50.3 to 89.3)	95.0 (92.2 to 96.9)	0.957	Moderate
Anti-TIF1-γ	LBA	9	1,344	63.8 (47.8 to 77.3)	96.5 (93.2 to 98.3)	0.921	Moderate

^*The quality was evaluated by GRADE (6).

MSA, myositis-specific autoantibody; CI, confidence interval; AUC, area under the curve; LBA, line blot assay (EUROLINE, EUROIMMUN, Lübeck, Germany), EIA, enzyme immunoassay (MESACUP, Medical and Biological Laboratories, Tokyo, Japan).

Figure 4. Hierarchical summary receiver operating characteristic analyses for studies included in meta-analysis.

LBA, line blot assay; EIA, enzyme immunoassay; SROC, summary receiver operating characteristic; conf. region; 95% confidence region.

PPA, NPA, and TPA between individual commercial assays

Only one study compared the performance of MSA detection between the EIA and LBA in reference to results by IP assay (Table 3). The kappa value between EIA and LBA was high for anti-MDA5 (0.92) and anti-Mi-2 (0.81), while low for anti-TIF1-γ (0.30).

Table 3.

Comparison of MSA results between individual commercial assays

Author, year (ref)	MSA (no)	Commercial assays	Positive percentage agreement	Negative percentage agreement	Total percentage agreement	Cohen’s kappa	Grade of agreement*
Fiorentino, 2019 (19)	MDA5 (n=261)	LBA vs EIA	0.90	0.99	0.98	0.92	Almost perfect
Fiorentino, 2019 (19)	Mi-2 (n=261)	LBA vs EIA	0.75	0.99	0.96	0.81	Almost perfect
Fiorentino, 2019 (19)	TIF1-γ (n=261)	LBA vs EIA	0.31	1	0.65	0.30	Fair agreement

^*The grade of the agreement was defined based on the previous manner (7).

MSAs, myositis specific autoantibodies; LBA, line blot assay (EUROLINE, EUROIMMUN, Lubeck, Germany), EIA, enzyme immunoassay (MESACUP, Medical and Biological Laboratories, Tokyo, Japan) for anti-MDA5, anti-Mi-2, and anti- TIF1-γ.

Discussion

This is the first SR and meta-analysis to provide comprehensive information on the performance of commercial MSA assays used in routine clinical practice, compared to the reference standard IP assays. The most commonly used assay platform identified in the literature was the LBA, particularly the EUROLINE system. In fact, many of the individual meta-analyses for LBA-based assays incorporated large sample sizes, exceeding 1,000 participants. In contrast, only one meta-analysis could be conducted for an EIA-based test—specifically the MESACUP assay for detecting anti-MDA5 autoantibodies. The pooled sensitivity of LBA-based assays ranged from 63.8% to 88.3%, indicating relatively low sensitivity overall. While pooled specificity was >94%, this was likely overestimated due to a high proportion of samples negative for each specific MSA. More studies are needed in populations at higher risk - e.g. patients attending clinic with symptoms prompting investigations for myositis. In addition, false-positive results were a notable concern for all LBA-based assays. Conversely, the EIA-based MESACUP anti-MDA5 assay demonstrated the highest pooled sensitivity and specificity, although the number of participants in these analyses (n = 771) was smaller compared to those for LBA-based assays. This study provides valuable insights for clinicians managing IIM patients, as well as researchers utilizing MSAs in clinical or translational studies, by clarifying the analytical performance of widely used commercial MSA assays.

Regarding study characteristics, most included studies were cohort studies comparing the performance of LBA or EIA to IP assays. The majority were assessed as having a low risk of bias based on the QUADAS-2 tool. Among the 23 selected studies, the EUROLINE LBA platform was used in 16 studies (7 from Europe, 6 from the United States, and 3 from Asia). A recent online survey indicated that 74% of myositis researchers in Europe use LBA, largely due to EUROLINE’s market dominance (1). On the other hand, the MESACUP EIA was used in 9 studies (2 from Europe, 1 from the United States, and 6 from Japan). These regional differences may have influenced both the number of studies available and the conclusions drawn from this review.

Based on QUADAS-2 assessments, the LBA was considered a useful tool for detecting MSAs, with an overall evidence quality rated as high to moderate. However, false-negative results were common across LBA-based assays, especially for anti-Mi-2, anti-NXP2, and anti-TIF1-γ autoantibodies, all of which had detection rates below 70% compared to IP assays. This may be due to reduced antigenicity caused by denatured protein conformation on LBA strips (32). Additionally, false-positive results were seen across all LBA-based assays. This is consistent with a recent study conducted in alliance with the CLASS project, which aimed to establish classification criteria for anti-synthetase syndrome (33). The false-positive results were common, resulting in a low positive predictive value, in commercial assays for detection of non-Jo-1 anti-synthetase autoantibodies. These limitations are clinically important, as inaccurate results from commercial assays can influence key medical decisions including diagnosis, disease classification, and treatment strategy.

While EIA-based assays may offer better diagnostic accuracy than LBA-based platforms, our meta-analysis included only the anti-MDA5 MESACUP assay. Although there are several other EIA-based commercial tests, insufficient data prevented their inclusion in this SR and meta-analysis. Therefore, the apparent superiority of the MESACUP EIA over LBA-based assays may reflect the specific characteristics of the MDA5 antigen, rather than a broader advantage of EIA technique.

This systematic review and meta-analysis have several limitations. First, some included studies had high risk of bias in domains such as patient selection, index or reference tests, and timing—e.g., studies involving serum samples from healthy controls, or interpretation of commercial assay results with prior knowledge of IP results. Second, cutoff values for LBA varied among studies, introducing heterogeneity and affecting sensitivity and specificity. In LBA assays, it is often difficult to establish appropriate cutoff thresholds that balance sensitivity and specificity (34, 35). Third, our analysis was limited to studies using LBA-based EUROLINE and EIA-based MESACUP assays; we excluded in-house ELISAs and research-use-only assays such as bead-based assays, which were beyond the scope of this review. Finally, we could not analyze commercial assay performance for anti-HMGCR and anti-cN1A autoantibodies due to a lack of comparative studies using both IP and commercial assays.

In conclusion, this systematic review and meta-analysis highlight that false-positive and false-negative results remain a significant challenge in the use of commercial MSA assays. These limitations should be carefully considered when interpreting assay results in clinical practice.

Supplementary Material

Supplementary material associated with this article can be found, in the online version.