Determination of diagnostic threshold in harmonization and comparison of clinical utility for five major antiphospholipid antibody assays used in Japan

Abstract

Background

Anticardiolipin antibodies (aCL) and anti‐β₂‐glycoprotein I antibodies (aβ₂GPI) are essential in diagnosing antiphospholipid syndrome (APS) according to the international APS guideline. Five commercial assays for aCL and aβ₂GPI are available in Japan, but their test results are quite discordant. For harmonization of diagnosing APS, upper reference limit (URL) and diagnostic accuracy of each assay were evaluated and compared by testing common sets of specimens across all assays.

Methods

We evaluated two manual and three automated assays for aCL and aβ₂GPI of IgG‐ and IgM classes. 99%URL (the upper limit of reference interval: as per guideline) together with 97.5%URL were determined by testing sera from 198 to 400 well‐defined healthy subjects. Both URLs were compared with the cutoff values, which were determined based on ROC analysis by testing 50 each of plasma specimens from patients with/without APS. Diagnostic accuracy was evaluated as area under curve (AUC) of the ROC curve.

Results

A variable degree of discrepancy between URLs and the cutoff values was observed, which was partly attributable to between‐year assay variability. 97.5%URLs were set lower and closer to the cutoff values than 99%URLs. For all assays, diagnostic accuracies of both aβ₂GPI‐IgG and aCL‐IgG were generally high (AUC: 0.84−0.93); whereas those for IgM‐class assays were low (AUC: 0.57−0.67), implicating its utility is limited to rare IgG negative APS cases.

Conclusion

To ensure harmonized APS diagnosis, the diagnostic thresholds of the five assays were evaluated by common procedures. Contrary to the guideline, 97.5%URL is rather recommended for diagnosing APS, which showed a closer match to the cutoff value.

Anticardiolipin antibodies (aCL) and anti‐β₂‐glycoprotein I antibodies (aβ₂GPI) are required for diagnosing antiphospholipid syndrome (APS). Five commercial assays for aCL and aβ₂GPI of IgG/IgM classes are used in Japan, but test results are quite discordant. For harmonized diagnosis of APS, 99% upper reference limits were derived for each assay using sera from healthy subjects in common. Diagnostic performance was found comparable across all IgG class assays, but IgM class assays generally exhibited low diagnostic utility.

1. INTRODUCTION

Antiphospholipid antibodies (aPLs) are a heterogeneous group of autoantibodies that appear in a variety of autoimmune diseases. The presence of aPLs is associated with various clinical events, such as arterial and/or venous thrombosis and recurrent fetal loss. ¹ In general, the diagnosis of antiphospholipid syndrome (APS) is based on those clinical manifestations and laboratory evidence of persistent aPLs. ² As per the revised criteria of the International Classification Criteria for APS, the items required for testing of aPL include anti‐cardiolipin antibodies (aCL) and anti‐β₂‐glycoprotein I antibodies (aβ₂GPI), along with lupus anticoagulant (LA) activity. ³ LA activity is currently detected in terms of the inhibitory effect of aPLs on certain in vitro phospholipid‐dependent coagulation reactions. Particularly, LA activity assay involves cumbersome specimen preparation and a series of complicated procedures for measurements. While assays for aCL and aβ₂GPI can be performed easily using conventional enzyme‐linked immunosorbent assays. In recent years, automated assays for aPLs became available, which allow simultaneous detection of multiple types of aPLs for improved diagnosis of APS.

In Japan, five major assay systems, composed of two manual and three automated assays, are currently available to detect one or two types of aPLs (aCL, aβ₂GPI) for both IgG and IgM‐class of antibodies. Despite their popularity, test results differ greatly among the assays. Besides, the upper reference limits (URLs) provided by manufacturers are quite discordant with each other. It is obviously attributed to undisclosed specification of samples and statistical methods used for the determination. This unharmonized diagnostic threshold arouses concern over the possibility of mismatched interpretation of the test results across the assays.

With these backgrounds, the first objective of this study was to determine the diagnostic threshold in a harmonized way across the assay systems. In the guideline, the diagnostic threshold is specified as “99^th percentile of controls”, ³ and thus we derived it as 99% upper reference limit (99%URL) of test results of healthy individuals. For comparison, 97.5%URL was also derived, which corresponds to the upper limit of conventional reference interval. A common set of serum specimens and statistical procedures were used across the assays for harmonization. To evaluate the appropriateness of derived URLs, they were compared with conventional cutoff values, which were determined, by a scheme of case‐control study, using a common set of clinical specimens from patients with/without APS.

The second objective was to evaluate the clinical utility across the assays based on test results of the two patients' groups. The diagnostic accuracy of assays for both IgG‐ and IgM‐class of antibodies was compared by using the ROC analysis. The associations of test results were also compared in a pairwise format among the five assays. Further, the clinical utility was compared between the two types of aPL assays (aCL vs. aβ₂GPI).

2. MATERIALS AND METHODS

2.1. Study subjects

2.1.1. Sera from healthy volunteers for determination of URL

Sera from healthy volunteers were obtained during the East‐Southeast Asian multicenter study conducted in 2009 to establish reference intervals ⁴ for 82 major clinical laboratory tests, including serum proteins, lipids, enzymes, electrolytes, hormones, vitamins, and tumor markers. The exclusion criteria adopted in recruiting the volunteers have been described elsewhere. ⁴ In brief, (1) body mass index (BMI) ≥28 kg/m², (2) consumption of ethanol ≥75 g/day, (3) ≥20 cigarettes/day, (4) under drug therapy, and (5) pregnancy or ≤1 year after childbirth. In total, 1986 Japanese individuals (879 males; 1107 females) with age range of 20–65 years (average 38.2) were included in that study. From serum aliquots stored at −80°C, 400 samples with no abnormal results in any of 25 major chemistry tests (electrolytes, lipids, enzymes, major proteins in serum) were randomly chosen as “fully normal” and were used to determine the URLs for aPLs.

2.1.2. Clinical specimens used for the assessment of diagnostic performance

To evaluate the clinical utility of aPL assays, we used 100 well‐defined clinical specimens obtained from the Rheumatology Department of Hokkaido University Hospital. They comprised plasma specimens from 20 patients with primary APS (18 females, 2 males; aged 14–70 years, mean 46.0), 30 patients with APS secondary to SLE (24 females, 6 males; aged 15–67 years, mean 44.3), 10 SLE patients without APS (10 females; aged 16–56 years, mean 33.7), and 40 patients suspected of collagen diseases of miscellaneous varieties but ruled out of both APS and SLE (23 females, 17 males; aged 18–86 years, mean 59.3).

The diagnosis of SLE was made based on the revised criteria of the American College of Rheumatology Criteria for Classification of Systemic Lupus Erythematosus (SLE), ⁵ and that of APS was made according to the classification criteria for definite antiphospholipid syndrome. ³

This study was reviewed and approved by the medical research ethics committee of Yamaguchi University Graduate School of Medicine, Faculty of Health Sciences (approval no. 363), and informed consent was obtained from all patients and control subjects.

2.2. Measurements

2.2.1. Measurements of antiphospholipid antibodies using ELISA kits

MESACUP‐2 test

The assay kit (MESACUP‐2 test cardiolipin IgG/IgM) was provided by Medical & Biological Laboratories Co. Cardiolipin antigen is immobilized to the “microcup” well. The IgG and IgM classes of aCL in patient sera bind to the immobilized cardiolipin (that is to be activated by addition of β₂GPI). For detection, a peroxidase‐labeled anti‐human IgG monoclonal antibody or IgM polyclonal antibody is added, and the reaction is quantitated by the addition of hydrogen peroxide substrate and measurements of color at 450 nm by use of Multiskan FC (Thermo Fisher Scientific Inc.).

QUANTA Lite^®

The assay kit (QUANTA Lite^® ACA IgG/IgM and QUANTA Lite^® β₂GPI IgG/IgM ELISA) was provided by INOVA Diagnostics. The kit uses purified cardiolipin or β₂GPI antigen immobilized to the microplate wells. IgG and IgM classes of aCL and aβ₂GPI in patient sera bind to the immobilized antigen. Added enzyme‐labeled anti‐human IgG or IgM conjugate binds to patient antibodies. The enzyme activity was measured by adding a chromogenic substrate and the intensity of its color was detected by use of Multiskan FC (Thermo Fisher Scientific Inc.).

2.2.2. Measurements of antiphospholipid antibodies by automated analyzers

QUANTA Flash^®

The reagents used to test the IgG and IgM classes of aCL and aβ₂GPI, QUANTA Flash ^® (INOVA Diagnostics), were provided by IL Japan. They were measured on the platform of an ACL AcuStar autoanalyzer (Instrumentation Laboratory). The principle of the assay is chemiluminescent immunoassay with reagents composed of magnetic particles coated with bovine cardiolipin and human purified β₂GPI, and isoluminol‐labeled mouse monoclonal anti‐human IgG or IgM antibodies. aPLs in patients' specimens that react with the antigens on the magnetic particle are captured by the isoluminol‐labeled antibodies and quantitated by the addition of a reagent that triggers a luminescent reaction. The light emitted from the labeled antibody is measured using a luminometer, as described elsewhere. ⁶

EliA

The reagents used to test the IgG and IgM classes of aCL and aβ₂GPI (EliA Cardiolipin IgG/IgM, EliA β₂‐GlycoproteinI IgG/IgM) were provided by Thermo Fisher Scientific Inc. They were measured on the platform of a Phadia‐100 autoanalyzer (Thermo Fisher Scientific Inc.). The principle of the assay is a fluorescence enzyme immunoassay with reagents composed of antigen‐coated microwell and anti‐human IgG or IgM monoclonal antibodies that are labeled with β‐galactosidase to be detected by a fluorescent substrate (4‐methylumveriferyl‐β‐D‐galactopyranoside lactose hydrate). The EliA wells are coated with specific antigens: a complex of bovine cardiolipin and bovine β₂GPI for the aCL assay, or a purified human β₂GPI for the β₂GPI assay. The aPLs in patient specimens bound to the coated antigens are captured by enzyme‐labeled antibodies. The enzyme activity is quantitated as fluorescent intensity. The calibration curve is common to both IgG and IgM classes, and the measured value is calculated by multiplying a unit conversion factor set for each reagent lot. ⁷

MEBLux test

The reagents used to test the IgG and IgM classes of aβ₂GPI (STACIA MEBLux test β₂GPI IgG/IgM) were provided by Medical & Biological Laboratories Co. They were measured on the platform of a STACIA autoanalyzer (LSI Medience Corporation). The principle of the assay is chemiluminescent enzyme immunoassay with reagents composed of β₂GPI‐immobilized magnetic beads and alkaline phosphatase‐labeled anti‐human IgG or IgM antibodies. The antigen‐antibody complexes formed on the magnetic beads are collected by magnetic force. After washing and removing unreacted moieties, the luminescence reagents are added, and the luminescence intensity is measured. ⁸

As a common quality control measures in the collaborating laboratory, we tested the same lot of positive control specimens provided by the kit manufacturer in each run of the assay. The analytical specifications of the assays were summarized in Table S1.

2.3. Method for determination of the URL

The number of healthy volunteers' specimens that were measured for the determination of the URL was different from one assay system to another because of the limited serum volume in some specimens: 400 for ELISA kits, 276 for QUANTA Flash^®, and 264 for the MEBLux. For the last two assays, the sera were randomly chosen to that number from the 400 specimens. For EliA, 198 sera were also randomly chosen  and measured in common in three laboratories (Yamaguchi University, Health Science University of Hokkaido, and Kanazawa University) using the same type of the analyzer. Therefore, the averages test results of each specimen from the three laboratories were used for determination of the URL after confirmation of almost perfect correlations of test results among them; i.e., correlation coefficient >0.98 and slope between 0.89 and 1.12 for all pairwise linear regression analyses.

The diagnostic thresholds, 99%URL and 97.5%URL were derived as test results corresponding to the upper 99^th and 97.5^th percentile, respectively. In this study, we basically relied on the parametric method to determine both URLs through Gaussian transformation of test results using the following Box‐Cox formula ⁹ :

$$\left\{\begin{array}{c}X=\frac{{\left(x‐a\right)}^p‐1}{p}\cdots \left(p\ne 0.0\right)\\ X=\log \left(x\right)\cdots \left(p=0.0\right)\end{array}\right.$$

where p, a, x, and X represent the power, the origin of transformation (shift), the test result, and the transformed test result, respectively.

The power “p” was changed incrementally from 0.0 to 1.0 at a step of 0.1. The optimal “p” value was determined using a probability plot, the X‐axis of which conformed to the specified power. The transformation origin “a” was adjusted after the selection of “p” for improved fitting. The primary criterion of the goodness‐of‐fit of the power transformation was linearity of the segment that corresponds to a cumulative frequency (CF) of 10%–90% on the Y‐axis of the probability plot. Using an optimal set of “p” and “a”, mean and SD of transformed test results were calculated. The upper 99% limit under the transformed scale (URL^T) computed as mean±2.326×SD was then reverse‐transformed to obtain the 99%URL of the original scale (URL) as

$$\left\{\begin{array}{c}URL={\left(1+p\times UR{L}^T\right)}^{1/p}+a\cdots \left(p\ne 0.0\right)\\ URL=\exp UR{L}^T+a\cdots \left(p=0.0\right)\end{array}\right.$$

However, when the linearity was not attained for the 10%–90% CF segment, the URL was determined non‐parametrically as the 99th percentile after sorting all test results in ascending order.

2.4. Methods for prediction of the cutoff value and diagnostic accuracy

The conventional cutoff value of each assay was determined, from test results of APS and non‐APS patients, as a threshold at which the sensitivity and the specificity of diagnosing APS patients are equal. The diagnostic accuracy of the aPL test results was evaluated by the ROC analysis as the area under the curve (AUC), which represents the degree of separation of the two groups.

3. RESULTS

3.1. Determination of URLs

For each antibody assay, the linearity of test results after the Box‐Cox transformation was evaluated using a probability plot as shown in Figure 1. Judging from the linearity for the CF segment of 10%−90%, p = 0.0 that corresponds to logarithmic transformation gave the best fit in almost all antibody assays. The exceptions were for the QUANTA Lite^® (aCL IgG) and QUANTA Flash^® (aβ₂GPI IgM) assays, which required values of p = 0.5 and p = 0.3, respectively, to attain the best linearity.

FIGURE 1.

Determination of the 99%URL for each antibody based on probability plot. This figure illustrates the procedure for determining 99% upper reference limits (99%URLs) from test results of healthy subjects for 16 antiphospholipid assays evaluated in this study. Shown on top of each panel are histograms of test results before and after Box‐Cox power transformation. The probability plot (Q–Q plot) shown underneath was constructed by plotting cumulative frequencies on the Y‐axis and test results on the X‐axis (power‐transformed scale) to evaluate the appropriateness of the power transformation from the linearity of the cumulative frequency curve. Test results were plotted using the power (p), in a stepwise manner from 0.0 to 1.0, at an increment of 0.1. In case sufficient linearity was not attained, the origin (a) was adjusted empirically. When linearity was attained for a cumulative frequency segment between 10 and 90% by the optimal “p” and “a” shown inside the Q–Q plot, the 99%URL was determined parametrically (p) as a point (blue closed circle) where the linear curve intersected with the horizontal blue line (99th percentile) on the upper end. If the curve failed to attain sufficient linearity, a point (red closed circle) where the upper end of the Q–Q plot intersects with the 99th percentile line was adopted as the nonparametric (NP) estimate of 99%URL. As an optional reference limit, 97.5%URL was determined as a point where the linear curve intersected with the horizontal red line of the 97.5th percentile. The use of the parametric method was judged to be valid for the following assays from the linearity of the CF 10%−90% segment: MESACUP‐2 (aCL IgG/IgM), QUANTA Lite^® (aCL IgG/IgM), QUANTA Flash^® (aCL IgG/IgM), MEBLux (aβ₂GPI IgG and IgM), EliA (aCL IgG/IgM), and EliA (aβ₂GPI IgG and IgM). Thus, the URLs were determined for those assays by extending the linear segment as a value corresponding to CF = 99%, as indicated by the blue point located at the upper end of the probability plot and also by the vertical blue solid line. In contrast, linearity of the 10%−90% CF segment was not attained in other assays, namely QUANTA Lite^® (aβ₂GPI IgG/IgM) and QUANTA Flash^® (aβ₂GPI IgG/IgM). In all of these assays, the insufficient linearity was attributable to the fact that a sizable number of values were below the detection limits, which resulted in a conspicuous vertical segment at the lower end of the plot. Therefore, a nonparametric method was used to determine the URL, which was indicated by the red point located at the upper end of the probability plot and also by the vertical blue solid line

As a result, we derived both 99%URL and 97.5%URL as listed in Table 1. Their locations were, respectively, indicated by the solid and broken vertical line that intersect with the cumulative frequency of 99% (the blue upper limit of the graph frame) and 97.5% (the red horizontal line). The manufacturer's URL provided in the instructions for use (IFU) are also listed in the table.

TABLE 1.

Anti‐phospholipid antibody assay systems evaluated in this study, and a list of URLs and cutoff values derived

Reagent name	Unit	URL in IFU	99%URL	97.5%URL	Cutoff value
MESACUP‐2 test cardiolipin IgG	U/mL	<10.0	12	9	6.5
MESACUP‐2 test cardiolipin IgM	U/mL	<8	10.9	8.1	NA
QUANTA Lite® ACA IgG	GPL	<15	7	6.2	8
QUANTA Lite® ACA IgM	MPL	<12.5	20.4	16.7	NA
QUANTA Lite® β2GPI IgG ELISA	SGU	≤20	2.6	1.8	1.5
QUANTA Lite® β2GPI IgM ELISA	SMU	≤20	18	12.9	NA
MEBLux test β2GPI IgG	U/mL	<1.0	0.21	0.17	0.05
MEBLux test β2GPI IgM	U/mL	<21.2	15.7	10.7	NA
QUANTA Flash® aCL IgG	U/mL	<20	24.9	17.8	10.6
QUANTA Flash® aCL IgM	U/mL	<20	17.8	13.6	NA
QUANTA Flash® β2GPI IgG	U/mL	<20	14.4	13.2	10.4
QUANTA Flash® β2GPI IgM	U/mL	<20	8.8	6.3	NA
EliA Cardiolipin IgG	GPL‐U/mL	<40	56.5	34.9	3.8
EliA Cardiolipin IgM	MPL‐U/mL	<40	28.1	19.8	NA
EliA β2‐Glycoprotein I IgG	U/mL	<10	3	2.5	1
EliA β2‐Glycoprotein I IgM	U/mL	<10	5	3.7	NA

Abbreviations: IFU, instructions for use; URL, upper reference limit.

3.2. Determination of cutoff values with their comparison to the URLs

The cutoff values determined for IgG‐class aCL and aβ₂GPI were shown as the red vertical line in Figure 2 where the test results of APS and non‐APS patients were compared by a one‐dimensional scattergram. In each graph, both the 99%URL and 97.5%URL just derived are also indicated as the upper borders of the blue and light‐blue‐shaded regions, respectively, which represent the range of values for healthy individuals. The location of the manufacturer's URL (IFU‐URL) is indicated by an inverted black triangle.

FIGURE 2.

Diagnostic utility of IgG‐class antiphospholipid assay in distinguishing patients with/without APS. The diagnostic utility was evaluated for eight antiphospholipid assays of IgG class by testing plasma specimens of 50 APS and 50 non‐APS cases. The three assays placed on the top correspond to manual assays, whereas the five on the bottom are automated assays. The left panel of each assay represents a one‐dimensional scattergram of the two groups. The 99%URL is indicated by an inverted blue triangle and vertical blue line at the upper border of blue shade, whereas the 97.5%URL is indicated by an inverted light‐blue triangle and vertical blue broken line at the upper border of light‐blue‐shaded region. The inverted black triangle on the top border indicates the URL provided by the reagent manufacturer (IFU‐URL). The red vertical line indicates the cutoff value determined by the ROC analysis as the point of the test result at which sensitivity and specificity for detecting APS cases were equal. The right panel represents the ROC curve. The degree of distinction of the two groups was calculated as the area under the curve (AUC) of the ROC curve

It is noteworthy that 99%URLs in most assays are located higher than the corresponding cutoff values, while 97.5%URLs are generally placed closer to the cutoff values. The exception was the cutoff value of EliA for IgG‐class aCL, which was markedly set lower than both the URLs [refer to possible reasons for the gap described in the Discussion]. Whereas the manufacturers' URLs (IFU‐URLs) indicated by the black inverted triangles were generally located higher than both URLs, except IFU‐URLs of the aCL by MESACUP‐2, QUANTA Flash^®, and EliA, which were located in‐between the 97.5%URL and 99%URL.

3.3. Diagnostic accuracy in distinguishing APS from non‐APS cases

For the four aCL IgG assays, AUCs were generally high with values ranged 0.858−0.933. Using a ROC analysis, the optimal cutoff value was predicted as the test result at which the sensitivity equaled the specificity for distinguishing APS from non‐APS cases.

In Figure [Link], [Link], the results of the same analyses are shown for the IgM‐class assays. It became obvious that the AUCs of all IgM assays were all low with values ranging from 0.571 to 0.661, which implies that none of these assays were practical for use in the diagnosis of APS.

3.4. Correlation of test results between automated and manual assays

The correlations of the test results among the antibody assays were evaluated pairwise between one of the three automated assays and either of the two manual assays, as shown in Figure 3. The scales of both axes were transformed using the power adopted in determining their URL. The data points representing the APS and non‐APS cases are distinguished by crosses and open circles, respectively. Spearman's correlation coefficient (r_S) was calculated separately for APS and non‐APS cases and is shown in that order above each graph. The URLs of the respective assays are indicated as the boundaries of the light‐blue‐shaded region representing the range of healthy subjects.

FIGURE 3.

Correlation of test results between automated and manual assays. All pairwise comparisons of test results between manual and automated assays were performed by drawing a two‐dimensional scattergram. Results are arranged in four types of antibody assays. The red cross and black open circle in each panel represent data points from 50 APS and 50 non‐APS cases, respectively. Spearman's correlation coefficients were calculated for the respective groups and are shown above each graph

Regarding aβ₂GPI assays, when test results were limited to APS cases (cross), the correlations of test results between the QUANTA Lite^® manual assay and any of the three automated assays were very high for IgG‐class assays (r_S: 0.919−0.946) and moderately high for IgM‐class assays (r_S: 0.735−0.797). In contrast, for aCL assays, the correlations between either of the manual assays and any of the two automated assays (QUANTA Flash^® and EliA) were not so high (r_S: 0.672−0.792 for both IgG and IgM classes).

However, when the test results were limited to non‐APS cases (open circle), these correlations were generally weak for IgG‐class assays (r_S: 0.237−0.472) and weak or moderate for IgM‐class assays (r_S: 0.334−0.773).

3.5. Correlation of test results between automated assays

All pairwise correlations between the three automated assays are as shown in Figure 4. When limited to APS cases, for aβ₂GPI assays, the r_S values for IgG [IgM]‐class assays were high between QUANTA Flash^® and EliA (0.927 [0.883]), between QUANTA Flash^® and MEBLux (0.965 [0.795]), and between EliA and MEBLux (0.877 [0.864]). In contrast, for aCL assays, the r_S values for IgG [IgM] class assays were 0.816 [0.758] between QUANTA Flash^® and EliA. However, when data were limited to non‐APS cases, the correlations between the three automated assays were markedly reduced for IgG‐class (r_S: 0.311−0.510) and IgM‐class (r_S: 0.482−0.692) assays.

FIGURE 4.

Correlation of test results between automated assays. (A) Pairwise comparisons of test results were performed between different automated assay systems for IgG‐class antibodies (the panels in the top row) and for IgM‐class antibodies (those in the second row). (B) Comparison of test results between aCL and aβ₂GPI assays of the same manufacturer was performed for IgG‐class antibodies (the panels in the top row) and for IgM‐class antibodies (the second row). All correlation graphs were prepared by plotting data points of 50 APS cases in red cross and 50 non‐APS cases in black open circles. The upper border of the light‐blue shade represents the 99%URL just determined for each assay. Spearman correlation coefficients calculated separately for the APS and non‐APS cases are shown on top of each graph in that order

3.6. Correlation between the test results of aβ₂GPI and aCL within each assay system

For assay systems capable of testing for both aβ₂GPI and aCL, the correlations of their test results were examined, as shown in Figure 4. Extremely high correlations between aβ₂GPI and aCL for IgG‐ and IgM‐class assays were observed in QUANTA Flash^® assays (r_S: 0.975 and 0.917, respectively). In contrast, the correlations were less prominent in EliA assays (r_S: 0.771 and 0.798, respectively) and QUANTA Lite^® assays (r_S: 0.734 and 0.643, respectively).

4. DISCUSSION

According to the international classification criteria for the definite diagnosis of APS, the diagnostic threshold of aPL assays is specified as the 99^th percentile of “controls”. ³ In the present study, we derived the threshold as 99%URL as well as 97.5%URL from test results of healthy individuals by using the parametric method, which relies on Gaussian transformation of the test results, using the Box‐Cox power transformation formula. Importantly, a vast majority of the test results obtained from healthy volunteers were above the detectable level of the respective assays. In contrast, aβ₂GPI assays for the IgG‐ and IgM‐class of antibodies, provided by QUANTA Lite^® and QUANTA Flash^®, featured a sizable number of test results below the detection limit. For the assays with sufficient samples above the detection limit, the distributions of the test results were made Gaussian by power transformation, and thus both URLs were successfully calculated parametrically. Otherwise, the URLs were determined non‐parametrically as the 99^th and 97.5^th percentiles of all results.

The appropriateness of the newly determined URLs was assessed in comparison to the corresponding cutoff values that represent a threshold for distinguishing between APS and non‐APS cases.

Unexpectedly, the URLs did not match well with the corresponding cutoff values. This was partly attributable to the differences in the time periods of measurements in determining the URLs (in 2019) and the cutoff values (in 2014) as discussed below for EliA assay and in the Limitation. Even setting aside of the between‐year bias, the 99%URLs are generally located above the cut‐off values, whereas the 97.5%URLs lie a little closer to the cut‐off values. Consequently, as a practical threshold in diagnosing APS, we rather recommend 97.5%URL, which allows higher sensitivity with a smaller false negative rate and is statistically more reproducible compared to the 99%URL: that is, theoretical width of the 90% confidence interval for the 99^th percentile point is 15% wider than that of 97.5^th percentile based on binominal distribution theory.

As for quite a large gap between URLs and cutoff values observed for EliA assays, between‐lot differences in test results are obvious, but the reagent manufacturer denied of between‐lot variability without disclosing their QC monitoring. Therefore, it is assumed that the URLs and/or the cut‐off values were not appropriate for the assay, and a repeat study must be conducted with simultaneous measurements of specimens from both patients and healthy individuals.

As an additional objective of this study, we evaluated the diagnostic equivalence of automated assays to manual assays by using the common set of specimens from APS and non‐APS patients with the following results.

For the aβ₂GPI assays, the correlation of the test results between the manual assay (QUANTA Lite^®) and any of the three automated assays was remarkably high for IgG‐class assays (r_S: 0.919−0.946). The diagnostic utility of the three automated assays in terms of AUC ranged 0.890−0.933, which was comparable to the AUC of the manual assay (0.858).

For the aCL assays, the correlation of test results for IgG‐class antibodies between either of the two manual assays (MESACUP‐2, QUANTA Lite^®) and either of the two automated assays (QUANTA Flash^® and EliA) was generally low with an r_S in the range of 0.672−0.760. However, AUCs of the four IgG‐class aCL assays ranged from 0.844 and 0.918, which were comparable to AUCs of the IgG‐class aβ₂GPI assays.

Overall, the diagnostic utility (AUC) of IgG‐class aβ₂GPI and aCL assays is quite comparable with each other, but the correlation of the test results between the four aCL assays was weak as compared to those observed between the four aβ₂GPI assays.

Contrastingly, we revealed that the diagnostic utility of the IgM‐class assays in distinguishing APS from non‐APS cases was quite poor as evident from the low AUC of ~0.6, regardless of the assay system used. Nonetheless, this study also identified a small number of cases wherein only IgM‐class antibodies were detected, with no increase in IgG‐class antibodies. Among 50 patients with APS, 1 case (2%) was only detected by QUANTA Flash^® IgM assay and 4 cases (8%) were only detected by EliA test for IgM. Therefore, IgM‐class assays might be of practical use in diagnosing infrequent APS patients without any increase in IgG‐class antibodies.

For the cross‐correlation of aCL and aβ₂GPI test results measured by the same assay system, the degree of correlation was mixed among the three assays. For the IgG (IgM)‐class assay by EliA and QUANTA Lite^®, the r_S values were 0.771 (0.798) and 0.734 (0.643), respectively, whereas the IgG (IgM)‐class assay by QUANTA Flash^® showed extremely high r_S values of 0.975 (0.917).

All aCL assays followed a common assay principle, wherein the antibodies in patients are supposed to bind to the cardiolipin‐β₂GPI complex formed on the surface of the solid‐phase. However, it is assumed that “non‐specific” antibodies do exist, which interact directly with cardiolipin unassociated with β₂GPI, but are unrelated to the thrombotic complications of APS. ¹⁰ In contrast, such interference by cross‐reacting antibodies is unlikely in aβ₂GPI assays owing to the absence of cardiolipin on the surface of the solid‐phase. The test results of aβ₂GPI in previous studies showed a close clinical association with thrombotic events. ¹¹ , ¹²

From this distinction of the assay principles between aCL and aβ₂GPI assays, our observation of the suppressed correlation of test results between aCL and aβ₂GPI in the EliA and QUANTA Lite^® assays is understandable. Whereas the close correlation between the aCL and aβ₂GPI test results observed in the QUANTA Flash^® assay may imply that patient sera reacted identically in both the aCL and aβ₂GPI assays, and thus the specificity of aCL results for the assay system must be re‐evaluated.

Patients with APS are known to have a higher risk of thrombus formation due to the presence of a mixture of various antibodies at high‐titer. The incidence and rate of recurrence of thrombosis were reported to be remarkably high among patients who were simultaneously positive for three tests (aCL, aβ₂GPI, and LA activities). ¹³  Thus, all three tests are necessary to predict the risk of thrombosis. However, the LA assay involves complicated multi‐step procedures, thus it is difficult to perform it as a routine examination. Besides, plasma factors associated with LA activity are heterogeneous, which include autoantibodies against two or more phospholipid‐bound plasma proteins. ¹⁰ , ¹⁴  Therefore, it is worthwhile to measure anti‐phosphatidylserine/prothrombin antibodies (aPS/PT) as an auxiliary test for the LA assay. In fact, aPS/PT has been reported to be strongly correlated with the clinical symptoms of patients with APS possessing LA activity. ¹⁵ , ¹⁶ , ¹⁷ Although aPS/PT is not included in the current classification criteria for definite APS, we think the aPS/PT assay should be included in automated analyzers to replace cumbersome LA assays.

5. LIMITATIONS

To establish URLs for five APS, assays systems, sera from healthy subjects were measured between 2014 and 2018, whereas the plasma samples of patients with/without APS were measured for predicting cut‐off values in 2019. Therefore, the observed differences between the URL and cut‐off values might be partly attributed to the between‐year differences in the test results. Unfortunately, the between‐year coefficient of variation could not be monitored for individual assays. As another limitation of this study, the sample size of 50 each for APS and non‐APS cases was insufficient for reliable prediction of cut‐off values when compared to 198–400 specimens used for determining the URLs. Thus, some of the gaps observed between URLs and cut‐off values could be attributed to higher imprecision in predicting the cut‐off values.

In the end, it is important to note that the 97.5/99% URLs derived from healthy volunteers and the cutoff values predicted from patients with/without APS cannot be compared across different assays because of the lack of harmonization of test results among them despite the use of the same sets of specimens. Both URLs and cutoff values are only valid in clinical use of respective assay. Nonetheless, the clinical utility expressed as AUC or correlation coefficient between different assays are valid because both measures are independent of test bias or differences in unit among the assays compared.

6. CONCLUSION

Both URLs were generally well below the IFU‐URLs but set slightly higher than the cutoff values, partly due to the time difference in measurements. Despite the specification of 99^th percentile of “controls” in the guideline that corresponds to 99%URL of healthy individuals' values, we rather recommend 97.5%URL as a diagnostic threshold for practical use because it is statistically more reproducible and a little closer to the cutoff value (i.e., implicating a smaller false negativity) than the 99%URL.

The comparison of the test results between APS and non‐APS groups revealed an excellent diagnostic accuracy of both IgG‐class aCL and aβ₂GPI assays with very high AUCs. However, the AUCs for the corresponding IgM‐class assays were low (AUC: 0.57–0.67) indicating that the diagnostic utility of IgM is limited to exceptional APS cases that are solitarily positive for IgM‐class antibodies.

The comparison of the automated and manual assays showed that the diagnostic performances were quite comparable, with high correlations of test results among the assays.

CONFLICT OF INTEREST

The authors declare no conflict of interest.

AUTHOR CONTRIBUTION

JN, MI, and EM conceived and designed the study. KI offered the serum specimens from healthy individuals. KO collected patient samples. RK, YM, and MY performed the measurements. RK and KI analyzed the data and wrote the original manuscript. All authors were involved in reviewing the draft and approved to submit the final manuscript.

Supporting information

Fig S1

Fig S1

Tab S1

Kaneshige R, Motoki Y, Yoshida M, et al. Determination of diagnostic threshold in harmonization and comparison of clinical utility for five major antiphospholipid antibody assays used in Japan. J Clin Lab Anal. 2022;36:e24340. doi: 10.1002/jcla.24340