The reliability of immunoassays to detect autoantibodies in patients with myositis is dependent on autoantibody specificity

Abstract

Objectives

In order to address the reliability of commercial assays to identify myositis-specific and -associated autoantibodies, we aimed to compare the results of two commercial immunoassays with the results obtained by protein immunoprecipitation.

Methods

Autoantibody status was determined using radio-labelled protein immunoprecipitation for patients referred to our laboratory for myositis autoantibody characterization. For each autoantibody of interest, the sera from 25 different patients were analysed by line blot (Euroline Myositis Antigen Profile 4, EuroImmun, Lübeck, Germany) and dot blot (D-Tek BlueDiver, Diagnostic Technology, Belrose, NSW, Australia). Sera from 134 adult healthy controls were analysed.

Results

Overall commercial assays performed reasonably well, with high agreement (Cohen’s κ >0.8). Notable exceptions were the detection of rarer anti-synthetases with κ < 0.2 and detection of anti-TIF1γ, where κ was 0.70 for the line blot and 0.31 for dot blot. Further analysis suggested that the proportion of patients with anti-TIF1γ may recognize a conformational epitope, limiting the ability of blotting-based assays that utilize denatured antigen to detect this clinically important autoantibody. A false-positive result occurred in 13.7% of samples analysed by line blot and 12.1% analysed by dot blot.

Conclusion

The assays analysed do not perform well for all myositis-specific and -associated autoantibodies and overall false positives are relatively common. It is crucial that clinicians are aware of the limitations of the methods used by their local laboratory. Results must be interpreted within the clinical context and immunoprecipitation should still be considered in selected cases, such as apparently autoantibody-negative patients where anti-synthetase syndrome is suspected.

Rheumatology key messages

• The assays tested do not reliably detect anti-TIF1γ.

• The assays tested do not reliably detect rarer anti-synthetase autoantibodies.

• False positive results are relatively common.

Introduction

Myositis-specific and -associated autoantibodies (MSAAs) can be identified in ∼60% of adults and children with myositis [1, 2]. They are widely acknowledged to be clinically useful and can aid diagnosis, inform prognosis and guide further investigations and treatment [1–3]. There are a number of different established laboratory methods for the detection of MSAAs in patient sera, each with their own advantages and limitations. To date, immunoprecipitation is considered the reference standard method. It has been used to identify novel MSAAs, and MSAA specificity can subsequently be confirmed using immunoprecipitation blotting or mass spectrometry [4, 5]. However, immunoprecipitation is an impractical method for widespread diagnostic use, as it is relatively expensive, has low throughput and requires specialist facilities along with staff expertise. As a result, the availability of immunoprecipitation for diagnostic purposes is limited to a handful of specialist centres worldwide.

Commercially available immunoassays offer the rapid detection of MSAAs at low cost and without the need for specialist expertise. As such, they allow the widespread use of MSAA testing in order to take advantage of the enhanced prognostic information MSAAs provide. A number of different commercial immunoassays are now available, but validation has been limited to using small cohorts, with the majority of the MSAA specificities underpowered for significant analysis. Concerns have been raised regarding the validity of these alternative assays, particularly with regard to certain MSAA specificities [6], and false positive rates may also be unacceptably high, with one study reporting antibody positivity in 22% of healthy controls (17% if anti-Ro52 was excluded) [7].

We aimed to determine the ability of two different commercially available testing methods to detect MSAAs, using immunoprecipitation as the reference standard.

Methods

Sample selection

Serum samples were selected from >3000 cases of myositis previously analysed by immunoprecipitation in our laboratory (for research or diagnostic purposes) and reported as containing the MSAA of interest [1, 2]. Where possible, 25 samples of each autoantibody were analysed, although, due to the rarity of some MSAAs, smaller numbers of samples were used. Sera was stored at −20°C prior to analysis and the same sample was used for all analyses. A total of 134 adult healthy controls were also tested by dot blot and 76 by line blot. Ethical approval was not required for this study.

IIF

IIF was performed on HEp-2 cells (Nova-lite, Inova, CA, USA) according to the manufacturer’s instructions. Samples were diluted to 1 in 100.

Immunoprecipitation

Sera (10 μl) was mixed with 2 mg protein-A-Sepharose beads (Merck, Gilligham, Dorset) in immunoprecipitation buffer (10 mM Tris-Cl pH 8.0, 500 mM NaCl, 0.1% v/v Igepal) at room temperature for 30 min. Beads were washed in immunoprecipitation buffer prior to the addition of 120 μl (35S)methionine-labelled K562 cell extract in immunoprecipitation buffer. Samples were mixed at 4°C for 2 h. Beads were washed in IP buffer and Tris-buffered saline (10 mM Tris-Cl pH 7.4, 150 mM NaCl) before being resuspended in 50 μl SDS sample buffer (Sigma-Aldrich, UK). After heating, proteins were fractionated by 9% SDS-PAGE gels, enhanced, fixed and dried. Labelled proteins were analysed by autoradiography. Standard controls were included on each gel (healthy control sera, anti-Jo-1, anti-U1RNP, anti-NXP2, anti-PM/Scl, anti-Ro, anti-La, anti-AMA, anti-PL12, anti-PL7, anti-Zo, anti-Mi-2, anti-Ku, anti-MDA5, anti-RNAPI/III, anti-SAE, anti-U3RNP, anti-TIF1γ, anti-Scl70 and anti-SRP). Patients with bands at 140 kDa were identified as anti-NXP2, anti-MDA5 or neither by ELISA [8, 9]. Anti-HMGCR samples were confirmed by ELISA as described in Tansley et al. [10].

Line blot

Line blots were performed according to manufacturer’s instructions (Euroline Myositis Antigen Profile 4, EuroImmun, Lübeck, Germany).

Dot blot

Dot blots were performed according to manufacturer’s instructions (D-Tek, BlueDiver, Diagnostic Technology, Belrose, NSW, Australia).

ELISA

In-house ELISAs were performed as detailed above to confirm autoantibody specificity in particular circumstances following immunoprecipitation.

Immunoprecipitation blotting

Immunoprecipitation was performed as described above using a non-radiolabelled K562 cell extract. Immunoprecipitated proteins were fractionated by 9% SDS-PAGE gels and transferred to nitrocellulose membrane. The membrane was then probed with either patient sera or a commercially available antibody (Sigma-Aldrich).

Data analysis

Statistical analysis was performed using R Studio 0.99.903 (R Foundation, Vienna, Austria) [11]. The level of agreement was assessed using weighted Cohen’s κ.

Results

In total, 461 serum samples were analysed. Of these, 321 serum samples were analysed using both commercial assays (25 anti-Jo1, 25 anti-Mi2, 25 anti-NXP2, 25 anti-MDA5, 25 anti-SAE, 25 anti-SRP, 25 anti-TIF1γ, 21 anti-PL7, 20 anti-PL12, 14 anti-OJ, 10 anti-EJ, 9 anti-Zo, 3 anti-KS, 1 anti-Ha and 68 healthy controls). An additional 74 serum samples were tested by line blot alone (25 anti-HMGCR, 24 anti-Ku and 25 anti-PM/Scl) and 66 healthy controls were tested by dot blot alone.

Data on the presence of anti-Ro52 is not shown, as this autoantigen is not detected by immunoprecipitation.

Sensitivity, specificity and level of agreement with the reference test

Sensitivity varied considerably between MSAA specificities for both assays. Interestingly, both assays performed poorly in detecting the rarer anti-synthetase autoantibodies as well as anti-TIF1γ. KS and Ha are included on the dot blot but were not analysed further due to very small numbers of immunoprecipitation positive sera available. It is noteworthy that of the three anti-KS samples and one anti-Ha sample available, none tested positive on the dot blot assay. The sensitivity of line blot and dot blot to detect MSAA previously identified by immunoprecipitation is shown in Tables 1 and 2, respectively.

Table 1.

The performance of a commercial line blot compared with immunoprecipitation

Autoantibody (n)	Line blot result		Number of samples with index MSAA as false positivea	Specificity	Sensitivity	Cohen’s κ
	False negative	True positive
Mi2a (25)	9	16	0	1	0.64	0.77
Mi2b (25)	17	8	7	0.98	0.32	0.37
Mi2a or Mi2b (25)	7	18	7	0.98	0.72	0.70
TIF1γ (25)	10	15	2	0.99	0.6	0.70
MDA5 (25)	3	22	4	0.99	0.88	0.85
NXP2 (25)	4	21	0	1	0.84	0.91
SAE (25)	1	24	2	0.99	0.96	0.94
Ku (24)	0	24	12	0.97	0.97	0.78
PM/Scl-75 (25)	11	14	10	0.98	0.56	0.56
PM/Scl- 100 (25)	5	20	3	0.99	0.8	0.82
PM/Scl-75 or PM/Scl-100 (25)	4	21	13	96.2	0.84	0.69
Jo-1 (25)	3	22	4	0.98	0.82	0.85
SRP (25)	2	23	6	0.98	0.92	0.84
PL-7 (21)	5	16	5	0.99	0.64	0.75
PL-12 (20)	2	18	5	0.99	0.9	0.83
EJ (10)	4	6	0	1	0.6	0.76
OJ (14)	14	0	1	1	0	0
Zo (9)	–	–	–	–	–	–
KS (3)	–	–	–	–	–	–
Ha (1)	–	–	–	–	–	–
HMGCR (25)	–	–	–	–	–	–
Healthy controls (68)	0	0	11b	–	–	–
Total (395)	54	199c	54d	0.62	0.78	0.41

The sensitivity, specificity and Cohen’s κ coefficient for each assay are shown. Tests with κ < 0.8 are highlighted in bold.

^aSome samples contained more than one false positive result. In total, 54 samples contained at least one false positive result.

^bOne sample was positive for anti-Ro52. The 11 listed were positive for myositis-specific autoantibodies. Anti-Ro52 were excluded from specificity calculations, as immunoprecipitation is unable to detect this autoantibody, which can be found in healthy individuals.

^cSamples that contained a true positive result and no false positive result.

^dNumber of samples analysed containing at least one false positive result.

Table 2.

The performance of a commercial dot blot compared with immunoprecipitation

Autoantibody (n)	Dot blot result			Number of times index MSA occurred as false positiveb	Specificity	Sensitivity	Cohen’s κ
	False negative	‘Unresolved’a	True positive
Mi2 (25)	0	6	19	2	0.99	0.76	0.83
TIF1γ (25)	18	1	6	5	0.99	0.24	0.31
MDA5 (25)	6	0	19	7	0.98	0.76	0.73
NXP2 (25)	2	2	21	5	0.99	0.84	0.81
SAE1 (25)	1	2	22	4	0.98	0.88	0.85
SAE2 (25)	10	2	13	5	0.98	0.52	0.58
SAE 1 or SAE2 (25)	1	2	22	8	0.98	0.88	0.78
Jo-1 (25)	4	0	21	2	0.99	0.84	0.87
SRP (25)	3	0	22	10	0.97	0.88	0.75
PL-7 (21)	0	1	20	2	0.99	0.95	0.93
PL-12 (20)	1	1	18	2	0.99	0.90	0.81
EJ (10)	1	0	9	3	0.99	0.90	0.77
OJ (14)	11	2	1	1	1	0.07	0.12
Zo (9)	9	0	0	2	0.99	0	−0.02
KS (3)	3	0	0	2
Ha (1)	1	0	0	1
Healthy controls (134)	0	11	0	13c	–	–	–
Total (387)	54	26	151d	46e	0.73	0.74f	0.48 f

The sensitivity, specificity and Cohen’s κ coefficient for each assay are shown. Tests with κ < 0.8 are highlighted in bold. KS and Ha are included on the dot blot assay but were not analysed further due to very small numbers of immunoprecipitation positive sera available. It is noteworthy that of the three anti-KS samples and one anti-Ha sample available, none tested positive on the assay.

^aIn total, 15 patient samples and 11 healthy control samples were ‘unresolved’ by the assay and a result was unavailable. These were counted as negative for the purposes of sensitivity, specificity and κ calculations.

^bSome samples contained more than one false positive result. In total, 46 samples contained at least one false positive result.

^cAn additional five samples were positive for anti-Ro52. The 13 listed were positive for myositis-specific autoantibodies. Anti-Ro52 were excluded from specificity calculations because immunoprecipitation is unable to detect this autoantibody, which can be found in healthy individuals.

^dSamples that contained a true positive result and no false-positive result.

^eNumber of samples analysed containing at least one false positive result.

^fIf ‘unresolved’ results are considered to be false positives, specificity would be reduced to 0.60 and Cohen’s κ to 0.34.

Specificity for each MSAA is shown in Tables 1 (line blot) and 2 (dot blot), and was generally high. Overall, 13.7% of samples analysed by line blot and 12.1% by dot blot produced a false positive result. An additional 6% of samples analysed by dot blot produced an ‘unresolved’ result, meaning that they were labelled as neither positive nor negative and could not be analysed further. This occurred across autoantibody specificities but was particularly common in anti-Mi2 sera, where 24% of samples were ‘unresolved’.

False positives

A total of 16.1% of healthy controls analysed by line blot tested positive for an MSAA, excluding anti-Ro52. While this is high, we note that positive results were generally low level (10 of 11 false positive result in healthy controls were 1+ only) and 4 of the 11 were anti-PM/Scl-75 positive in isolation (rather than the anticipated PM/Scl-75 and PM/Scl-100). It is likely therefore that these results would be treated with a degree of suspicion in the clinical context. On dot blot, the reported level of false positive results in healthy controls was variable, ranging from 7 to 100 (positive range 1–100).

Looking at all false positive results, anti-Ku appeared as a false positive most commonly on line blot [12 samples (3%)] and anti-SRP on dot blot [10 samples (2.5%)]. Multiple autoantibody positivity was a common feature of samples containing a false positive result: 55% of samples containing a false positive result by line blot and 57% by dot blot were reported as having more than one MSAA.

IIF was performed as described in all samples where immunoprecipitation results differed from line blot and/or dot blot findings. To better understand how the commercial assay results might be interpreted in a non-research setting, we analysed whether the corresponding ANA pattern was consistent or inconsistent with that reported by line/dot blot results.

In those samples where an MSAA had been identified by immunoprecipitation, the ANA pattern could often be interpreted as consistent with the line/dot blot result. This was usually because the immunoprecipitation result and the line/dot blot result would be expected to produce the same ANA pattern. For example, in a patient with anti-Jo1 on immunoprecipitation and a cytoplasmic staining pattern on IIF, the pattern could be interpreted as consistent with anti-SRP or other anti-synthetases where these were identified on commercial immunoassays. For samples ‘false positive’ for at least one known MSAA on line blot, IIF was consistent with this result in 40% of cases. For samples ‘false positive’ for at least one known MSAA on dot blot, IIF was consistent with the result in 75% of cases.

For healthy control samples, which were largely ANA negative, IIF was less likely to be consistent with the MSA identified by the commercial assay. For healthy control samples ‘false positive’ for at least one MSAA by line blot, no samples had an ANA pattern that could further support this result. For samples ‘false positive’ for at least one MSAA by dot blot, 25% of samples had an ANA pattern that could be consistent with this result (all had a non-specific fine speckle nuclear staining pattern).

Anti-TIF1γ

Immunoprecipitation blotting was performed on 16 samples reported as containing anti-TIF1γ by immunoprecipitation but negative on other assays. For 11 (69%) of these samples, a prototype anti-TIF1γ serum or commercial anti-TIF1γ was able to detect the immunoprecipitated antigen by blotting, confirming the presence of the anti-TIF1γ in the original sample. However, these 11 samples did not identify antigen immunoprecipitated by the prototype anti-TIF1γ serum or commercial anti-TIF1γ, suggesting that they target conformational epitopes that are not recognized in a denatured state (see Supplementary Fig. S1, available at Rheumatology online).

Discussion

Commercial assays to detect MSAAs are now in widespread use worldwide. Limited validation, particularly for rarer autoantibodies, has been a growing concern among the myositis community [6, 12, 13]. Previous studies in this area have been small, with low numbers of sera containing individual MSAA specificities [6, 14, 15]. They have highlighted that the level of agreement of line blot with immunoprecipitation is highly dependent on autoantibody specificity [6, 14, 15]. By selecting sera based on autoantibody specificity rather than analysing a cohort, where some MSAA specificities may be very low prevalence or absent, we have been able to clearly demonstrate that the two commercially available assays tested do not perform well for all MSAA specificities. ‘Problem’ MSAAs in both assays assessed are the rarer anti-synthetase autoantibodies, where manufacturers may have limited access to patient sera for assay development and validation, and anti-TIF1γ. With the exception of anti-MDA5, which we found to be reasonably reliable, the ‘problem’ MSA specificities identified in our study are similar to those reported by Espinosa-Ortega et al. [6]. The difference with anti-MDA5 may be due to the number of sera analysed, as the previous study included just three patients with anti-MDA5. While we found a higher number of MSAAs in control sera than reported by Espinosa-Ortega et al., our findings are comparable to those of Bundell et al. [7].

Anti-TIF1γ is arguably one of the most clinically important MSAAs. It is the most common MSAA in those with juvenile-onset myositis in the UK and USA, where it is present in 20–30% of affected children, and in adults it is strongly associated with the presence of an underlying malignancy [1, 2, 16–19]. Anti-TIF1γ is present in ∼7% of European adults with myositis and if this group is to be appropriately targeted for malignancy screening, it is crucial that anti-TIF1γ can be accurately identified [1]. Anti-TIF1γ false negatives, which occurred in 40% of samples analysed by line blot and 76% by dot blot, could result in less rigorous malignancy screening in adults. Furthermore, a significant proportion of patients with juvenile-onset myositis would be incorrectly labelled as autoantibody negative. Our data are in keeping with the earlier findings of Espinosa-Ortega et al. [6] and suggest that clinicians should not be reassured by a negative result obtained by these methods. As previously reported by Targoff et al. [17] in 2006, we have shown that anti-TIF1γ frequently targets a conformational epitope, thus limiting the utility of blotting-based assays that use denatured antigen. In our experience, ELISA appears to be more sensitive (in-house assay data not shown) and a recent paper reports a high sensitivity and specificity for a commercial ELISA to detect anti-TIF1γ compared with immunoprecipitation [20]. These assays are disadvantaged by not being multiplex, but may be necessary as an adjunct to other testing methods in order to identify those patients with anti-TIF1γ accurately. False negative results are important because, not only will they influence prognostic information provided to patients and potentially the approach to further investigation, but in the UK, NHS England requires the presence of a myositis-relevant autoantibody for affected adults to access rituximab in the context of treatment-resistant disease.

Multiplex assays such as those analysed in this study simultaneously test for a number of key MSAAs. While increasing efficiency, multiple testing has the disadvantage of increasing the likelihood of false positive results. While assays were all highly specific for individual MSAAs, the overall false positive rate was high. MSAAs are generally mutually exclusive and true multiple positives are exceptionally rare [1]. Multiple MSAA positivity can be an important clue to false positive results. We would also recommend ensuring that the ANA pattern as determined by IIF is consistent with that expected for the MSAA identified, as any discrepancy may point to a false positive result. For those patients with negative results by immunoassay, multiple MSAA positivity or where the clinical picture does not fit with results obtained, other methods including immunoprecipitation should be considered to confirm MSAA status.

The limitations of this study include that we were unable to determine the interday or interlaboratory variability of the assays tested, which was beyond our scope. This does not detract from the concerns raised nor change recommendations for when additional testing should be considered.

In conclusion, myositis is a complex and heterogeneous disease and MSAAs provide an opportunity to stratify patients and provide more personalized prognostic information. Multiplex assays to detect MSAAs, including those analysed in this study, are already part of routine clinical practice. This study demonstrates the limitations of such techniques, an understanding of which is crucial in order for clinicians to interpret results in the clinical context and in light of additional investigations.

Funding: This work was supported by funding from CureJM and the Bath Institute of Rheumatic Diseases.

Disclosure statement: The authors have declared no conflicts of interest.