Is the Enzyme-linked Immunosorbent Assay More Accurate Than the Lateral Flow Alpha Defensin Test for Diagnosing Periprosthetic Joint Infection?

Abstract

Background

Alpha defensin was proposed as a new biomarker in synovial fluid for the diagnostic workup of failed joint prostheses. To our knowledge, no comparative study of the performance of the quantitative enzyme-linked immunosorbent assay (ELISA) and qualitative lateral flow alpha defensin test has been reported.

Questions/purposes

(1) Using the proposed European Bone and Joint Infection Society (EBJIS) criteria for defining periprosthetic joint infection (PJI), is there a difference in the diagnostic accuracy of quantitative ELISA and qualitative lateral flow alpha defensin tests? (2) Is there a difference in the performance of the two alpha defensin tests when using three definition classification systems (Musculoskeletal Infection Society [MSIS], Infectious Diseases Society of America [IDSA], and proposed EBJIS)?

Methods

In this retrospective study of samples collected earlier as part of a related longitudinal study, we included patients in whom aspiration of the prosthetic hip or knee was performed as routine investigation before every revision arthroplasty. Between October 2016 and April 2017, a total of 73 patients were eligible for inclusion. As a result of an insufficient fluid volume for analysis (< 5 mL), two patients were excluded. Among the 71 patients in the final analysis, 54 had a knee and 17 a hip arthroplasty. Using the proposed EBJIS criteria, PJI was diagnosed in 22 patients (31%) and aseptic failure in 49 (69%). The alpha defensin ELISA and lateral flow tests were performed in synovial fluid. Patients were classified as having PJI or aseptic failure using the MSIS, the IDSA, and the proposed EBJIS criteria. Sensitivity and specificity of ELISA and the lateral flow alpha defensin test were calculated. Based on receiver operating characteristic analysis, area under the curve values were compared.

Results

When measured against the proposed EBJIS criteria, the sensitivity of alpha defensin ELISA and the lateral flow test was low and not different from one another with the numbers available at 50% (95% confidence interval [CI], 31%-69%) and 46% (95% CI, 27%-65%; p = 0.857), respectively, whereas both methods showed high specificity (98% [95% CI, 88%-100%]; p = 1.000). For sensitivity, the highest values were seen when compared against the MSIS criteria (ELISA: 85% [95% CI, 56%-97%], lateral flow: 77% [95% CI]; p = 0.871), intermediate with IDSA criteria (ELISA: 73% [95% CI, 48%-89%], lateral flow: 67% [95% CI]; p = 0.867), and lowest with proposed EBJIS criteria (ELISA: 50% [95% CI, 31%-69%], lateral flow: 46% [95% CI]; p = 0.763). Specificity, however, was high regardless of the criteria used, where ELISA and lateral flow produced results that were not different (MSIS: 98% [95% CI, 90%-100%], IDSA: 98% [95% CI, 90%-100%], EBJIS: 98% [95% CI, 88%-100%]; p = 1.000). The area under the curve of alpha defensin ELISA and the lateral flow test was similar, regardless of the definition criteria used (EBJIS: p = 0.566; IDSA: p = 0.425; MSIS: p = 0.339).

Conclusions

There is no difference between the quantitative and qualitative alpha defensin test for confirmation of PJI, irrespective of applied definition criteria. Having the advantage of providing results within 10 minutes without the need for a laboratory facility, the qualitative test may be of interest in the intraoperative setting, however, at a cost of higher test expense.

Level of Evidence

Level I, diagnostic study.

Introduction

The diagnosis of periprosthetic joint infections (PJI) is challenging, especially in chronic infections with low-grade inflammation [4]. High sensitivity is paramount for diagnostic tests as a missed PJI diagnosis often leads to infection relapse. Measuring the leukocyte count and differential in synovial fluid is one of the most accurate and widely used diagnostic tests [8]. However, its specificity is limited in noninfectious inflammatory conditions [22, 25]. Furthermore, there is a lack of validated, defined criteria to uniformly diagnose PJI in scientific and clinical practice. Among the three currently available classification systems, the Musculoskeletal Infection Society (MSIS) criteria [16] and the Infectious Diseases Society of America (IDSA) criteria [15] are widely used in the United States. In September 2017, the European Bone and Joint Infection Society (EBJIS) proposed criteria as a working PJI definition. In contrast to the MSIS and IDSA criteria, the proposed EBJIS criteria also consider sonication of the removed implant in the diagnosis and use lower cutoff values for synovial fluid leukocyte count, allowing for better detection of low-grade PJI. The application of an accurate classification system is not only of paramount relevance in clinical practice, but also for the evaluation of the performance of any novel diagnostic test.

Novel diagnostic methods using synovial fluid have been proposed for rapid and accurate PJI diagnosis [7, 12, 24]. Among biomarkers, alpha defensin has been well investigated and has been proposed as an alternative to synovial fluid leukocyte count by several authors [3, 10]. This antimicrobial peptide is released by neutrophils and induces depolarization of the microbial cell membrane causing rapid microorganism death. Quantitative determination of alpha defensin using an enzyme-linked immunosorbent assay (ELISA) showed high sensitivity (97%-100%) and specificity (95%-100%) in previous studies [1, 2, 5-7, 9]. A qualitative bedside immunoassay test (alpha defensin lateral flow test) was designed for simple use and rapid results (within 10 minutes). With this advantage, its potential intraoperative use as a decision aid regarding surgical procedure gained attention. However, some studies suggest that the sensitivity of this alpha defensin lateral flow test was lower (67%-69%) with similar specificity (93%-94%) compared to previously reported results of the ELISA test [13, 21]. We recently also found this to be the case, and in addition determined that the test's diagnostic properties depended upon how PJI was defined [20]. Furthermore, the alpha defensin lateral flow test is considerably more expensive [personal communication with manufacturers of both tests]. To date, no comparative study of the performance of the quantitative (ELISA) and qualitative lateral flow alpha defensin has been reported.

The aim of this study was to answer the following questions in a subset of our earlier patient cohort [20] to ensure comparable results: (1) Using the proposed EBJIS criteria for defining PJI, is there a difference in the diagnostic accuracy of quantitative ELISA and qualitative lateral flow alpha defensin tests? (2) Is there a difference in the performance of the two alpha defensin tests using three different infections classification systems (MSIS, IDSA, and proposed EBJIS)?

Patients and Methods

We conducted this retrospective diagnostic study on samples collected earlier as part of a longitudinal study [20] in a tertiary healthcare center between October 2016 and April 2017. The current study was performed to expand on the earlier report, with the specific intention of comparing the ELISA and the alpha defensin lateral flow approaches to measuring alpha defensin. We obtained approval of the institutional review board and informed consent from all patients before participation (public trial identification: www.clinicaltrials.gov, NCT02530229). The study was done in accordance with the Declaration of Helsinki. The results were not communicated to the treating physician and did not influence treatment decisions.

In our earlier prospective study [20], we collected synovial fluid on all patients who underwent revision arthroplasty of the hip or knee between April 2016 and May 2017. After prespecified exclusions were applied [20], we had samples on 212 patients (151 TKAs, 71%, and 61 THAs, 29%). The current study analyzed the subpopulation of those patients included from October 2016 to April 2017, in whom sufficient synovial fluid remained after the earlier analyses to perform the ELISA test (n=71). In our surgery center that specializes in the treatment of musculoskeletal infections, aspiration of prosthetic joints is routinely performed before every revision surgery. Patients in whom the joint was aspirated with a cement spacer in place or resection arthroplasty were not eligible. In the current study, we relied on the results of the alpha defensin lateral flow and other conventional tests as performed in the earlier study [20] and did not repeat them.

A total of 71 patients with a median age of 70 years (range, 41-85 years) were included; 33 (46%) were women. In 54 patients (76%), a knee prosthesis and in 17 patients (24%) a hip prosthesis was involved.

Test performance was assessed applying three classification systems in all patients to confirm infection (Table 1): the MSIS criteria [16, 17], the IDSA criteria [16], and the proposed EBJIS criteria [20]. For descriptive statistics and specific analyses, the proposed EBJIS criteria were used, as they are used in many European institutions. According to the proposed EBJIS criteria, 22 patients (31%) were diagnosed with PJI and 49 (69%) with aseptic failure (Table 2). Among patients diagnosed with PJI, the infection was classified as chronic in 19 patients (86%) and acute (< 6 weeks) in three patients (14%). Four patients (19%) presented with sinus tract. Using MSIS criteria, 13 patients (18%) and, using IDSA criteria, 15 patients (21%) were diagnosed with PJI. No patient received antibiotic treatment before surgery. No active follow-up to confirm infection-free status was performed in patients classified as aseptic failures.

Table 1.

Classification systems for diagnosing PJI

Table 2.

Patient demographic data

An orthopaedic surgeon aspirated synovial fluid under sterile conditions either preoperatively in the outpatient setting or intraoperatively before incision of the joint capsule. One milliliter of the collected synovial fluid was put in a vial containing EDTA for quantification of the erythrocyte and leukocyte count as well as granulocyte percentage in an automatic manner. Clotted specimens were treated with 10 µL hyaluronidase (Sigma-Aldrich Chemie, Munich, Germany) for 10 minutes at room temperature before analysis.

One milliliter was inoculated in a native vial for aerobic and anaerobic culture. Then, 0.1 mL of the preprocessed sonicate fluid was inoculated onto aerobic and anaerobic plates (sheep blood agar, chocolate agar, and Schädler anaerobic agar) and 1 mL was inoculated in thioglycolate broth. The agar plates were incubated at 37° C in aerobic and anaerobic atmospheres for 2 days (aerobic cultures) or 14 days (anaerobic cultures); they were inspected daily for microbial growth. Microorganisms were enumerated (ie, number of colony-forming units/mL sonication fluid) and species identification was performed using Vitek® 2 (bioMérieux, Nürtingen, Germany) or a matrix-associated laser desorption/ionization-time of flight mass spectrometer (Vitek® MS (bioMérieux, Nürtingen, Germany)). Another milliliter of synovial fluid was introduced into a pediatric blood culture bottle (BacTec™ PedsPlus™/F; Beckton Dickinson and Co, Shannon, County Clare, Ireland) and incubated at 36 ± 1° C for 14 days or until positive growth was observed. The remaining joint fluid was introduced in thioglycolate broth for enrichment. The colonies of each microorganism morphology were identified by standard microbiologic methods using automated system Vitek® 2 (bioMérieux, Marcy L'Etoile, France). One milliliter was sent for polarization microscopy analysis for urate and pyrophosphate crystal detection. Another milliliter was transferred to a native vial for determination of alpha defensin and performance of a leukocyte esterase test strip (Combur-Test®; Roche Diagnostics, Mannheim, Germany). All aspirates were stored in a biobank at -80° C.

Performance of Conventional Diagnostic Tests

Serum C-reactive protein (CRP) was elevated (> 10 mg/L) in 12 of 21 patients (57%). Twelve PJIs were culture-positive (55%). The most common isolated microorganisms were coagulase-negative staphylococci (n = 8) followed by Enterobacteriaceae (n = 4), Enterococcus spp (n = 3), Staphylococcus aureus (n = 1), and Campylobacter coli (n = 1). In patients with PJI, synovial fluid leukocyte count (absolute or percentage of granulocytes) was positive in 18 of 22 patients (82%), the leukocyte esterase test strip in five of 14 patients (36%; eight patients had an inconclusive test), the periprosthetic histopathology in 10 of 14 patients (71%), and the synovial fluid culture in eight of 22 patients (36%). The erythrocyte sedimentation rate (ESR) was replaced by CRP and was therefore not determined in any included patient.

Determination of Alpha Defensin

A trained physician (NR, KY, IKS) performed the qualitative lateral flow test (Synovasure™; Zimmer Biomet, Winterthur, Switzerland) according to the manufacturer’s instructions. A volume of 250 to 500 µL of synovial fluid was processed and the qualitative result (ie, infection yes or no) was read after 10 minutes. For the quantitative ELISA test, a 1.5-mL sample of the collected synovial fluid was centrifuged at 2700 rpm for 10 minutes (Microcentrifuge 5427R; Eppendorf, Wesseling-Berzdorf, Germany) to remove all cellular and particulate content. The supernatant (at least 1.0 mL) was transferred to a new aliquot and stored at -80° C until further processing in an external laboratory (Labor Dr Fenner und Kollegen, Hamburg, Germany) [5]. A standard ELISA test (Synovasure™; CD Diagnostics, Claymont, DE, USA) was used for quantitative determination of human alpha defensin 1-3 peptide. The results were given as standardized signal relative to a tolerance limit value (interpretation values: < 0.9 aseptic, 0.9–0.99 unspecific, ≥ 1.0 septic).

Intraoperative Tests

Irrespective of the revision reason, all patients received antibiotic prophylaxis 30 to 60 minutes before incision. In patients undergoing revision surgery, at least three periprosthetic tissue samples (deep capsular or pseudocapsular tissue, synovial lining along the bone/implant and/or cement-implant interface, and intramedullary tissue) were obtained and sent for microbiological and histopathological analysis. The removed implant was transported to the microbiology laboratory in a sterile airtight container (Lock & Lock, Frankfurt am Main, Germany) and sonication was performed as previously described [24].

Statistical Analysis

Statistical analysis is based on the receiver operating characteristic (ROC). Sensitivity, specificity, accuracy, positive predictive value and negative predictive value, positive likelihood ratio (LR+) and negative likelihood ratio (LR-), and the area under the ROC curve (AUC) with corresponding 95% confidence intervals (95% CIs) were calculated. For comparison between alpha defensin ELISA and lateral flow tests, the AUC values were compared using the z-test. The sample size was calculated on the following assumptions: evaluation of the performance of both tests using the proposed EBJIS criteria, assuming no difference margin of < 10%, power 80%, and α-risk 5%. The calculated sample size was 70. The significance level for the tests is p < 0.05. Statistical analyses were done in XLSTATPM (version 2017; XLSTAT; Addinsoft, New York, NY, USA).

Results

Performance of Alpha Defensin Using the Proposed EBJIS Criteria

When measured against the proposed EBJIS criteria, the sensitivity of alpha defensin ELISA and lateral flow test was low and not different from one another with the numbers available at 50% (95% CI, 31%-69%) and 46% (95% CI, 27%-65%; p = 0.857), respectively, whereas both methods showed high specificity (98% [95% CI, 88%-100%]; p = 1.000; Table 3). Additionally, there was no difference in the AUC between alpha defensin ELISA (0.74 [95% CI, 0.63-0.85]) and lateral flow (0.72 [95% CI, 0.61-0.83]) tests (p = 0.566). Analysis of the discordant results revealed an ELISA false-positive rate of one of 49 (2%), lateral flow false-positive rate of one of 49 (2%; Table 4), ELISA false-negative rate of 11 of 22 (50%), and lateral flow false-negative rate of 12 of 22 (55%; Table 5). In one patient (man, 83 years old), the lateral flow test was false-negative, the ELISA test was positive, and the leukocyte count (19,131 mg/µL), the granulocyte percentage (86.9%), and the serum CRP (40.7 mg/L) were elevated. Hence, PJI was diagnosed. In the 11 patients with concordant false-negative alpha defensin tests, six patients had elevated synovial fluid leukocyte count as the only diagnostic criterion present. In one patient with a communicating sinus tract, positive histopathology (type II [14]), positive culture (two of seven positive tissue cultures: Escherichia coli, Bacteroides fragilis; one of seven positive tissue cultures: Enterococcus faecalis, Enterococcus faecium, Proteus mirabilis, Klebsiella oxytoca), and elevated serum CRP (50.5 mg/L), both methods were negative. In the remaining three patients with false-negative alpha defensin lateral flow tests, infection was identified based on positive histopathology (type II or III).

Table 3.

Performance of ELISA and lateral flow alpha defensin test

Table 4.

False-positive alpha defensin tests based on proposed EBJIS criteria

Table 5.

False-negative alpha defensin tests based on proposed EBJIS criteria

Performance of Alpha Defensin Using MSIS and IDSA Criteria

The sensitivity when comparing ELISA and the lateral flow test against the MSIS criteria was 85% (95% CI, 56%-97%) and 77% (95% CI, 49%-92%; p = 0.871), respectively, against IDSA criteria at 73% (95% CI, 48%-89%) and 67% (95% CI, 42%-85%; p = 0.867), respectively, and against proposed EBJIS criteria at 50% (95% CI, 31%-69%) and 46% (95% CI, 27%-65%; p = 0.857), respectively (Fig. 1). Specificity, however, was high regardless of the criteria used, where ELISA and lateral flow test were not different with the numbers available (MSIS: 98% [95% CI, 90%-100%], IDSA: 98% [95% CI, 90%-100%], proposed EBJIS: 98% [95% CI, 88%-100%]; p = 1.000). We found no differences between both AUCs when comparing ELISA versus lateral flow test (MSIS: p = 0.339; IDSA: p = 0.425; proposed EBJIS: p = 0.566; Fig. 2).

Fig. 1.

The graph shows sensitivity of quantitative ELISA and the qualitative lateral flow alpha defensin test according to the MSIS, IDSA, and proposed EBJIS definition criteria. The p values give the comparison between the sensitivities of both methods.

Fig. 2.

ROC curves for diagnostic accuracy of PJI based on the qualitative alpha defensin lateral flow test and quantitative alpha defensin ELISA test are shown.

Discussion

In clinical practice, there is a need for a reliable and feasible test to discriminate between septic and aseptic prosthesis failure. Where preoperative diagnostic workup is not conclusive, a rapid intraoperative test is needed. The quantitative ELISA test is not suitable as a rapid intraoperative test, because it requires laboratory infrastructure. Therefore, the qualitative lateral flow test was developed to provide more rapid results, but at the cost of approximately sevenfold higher cost [personal communication with test manufacturer]. We earlier found the alpha defensin lateral flow test to be specific but not sensitive [20]; in the current study, which evaluated a subset of that earlier population, we compared the performance of the quantitative ELISA and the qualitative alpha defensin lateral flow test. We found the quantitative ELISA test to perform fairly similarly to the alpha defensin lateral flow test. Both tests are suitable as confirmatory tests, but not screening tests. The ELISA test will be of greater use in the outpatient setting rather than during surgery, since it requires a laboratory and results will not be immediately available in the operating room.

This study has several limitations including the small sample size. However, we believe that with a larger sample, no differences between the two tests would have been detected because the error would apply equally to both tests, although it could result in changes in sensitivity or specificity. Importantly, the size of some subgroups (such as hip and knee prostheses) was small, and so the results should be considered preliminary. Another limitation of this study is that the sample size calculation had not taken into consideration the large spectrum of clinical presentation (including patient comorbidities, infecting pathogen, anatomic location, previous antibiotic treatment, and surgical interventions) and so there is a possibility of false positives or false negatives with respect to determining the presence or absence of infection. However, we do not believe this is a severe limitation, given the high sensitivity of the proposed EBJIS criteria for diagnosing PJI. Nevertheless, the results should be interpreted with caution when used in routine clinical practice, where all of these factors may influence the performance of diagnostic tests. Future studies will need to validate these findings, especially for different joints, microbiology, and clinical presentations. We also note that the study population is a subset of patients included in an earlier study on a related topic [20]. That being so, the findings in the current report should be considered somewhat preliminary, and so we hope that future studies will seek to confirm the observations we have made here. In addition, we caution future meta-analysts that the 71 patients whose data were analyzed here were a subset of those presented in our earlier study on the alpha defensin lateral flow test, and so the alpha defensin lateral flow results of that study and this one should not be pooled as though they are independent observations, since they came from the same patients.

A further drawback is that some parameters used in the three definition criteria were not available for all patients, which is the reality in clinical routine [2, 21]. For example, ESR is not routinely performed any more in many institutions including our department and was therefore not available for patients included in this cohort. Nevertheless, we do not believe that the test performance would be influenced by this systemic marker of inflammation, because it has low sensitivity and specificity for PJI [18]. Another limitation is the lack of data on the most appropriate criteria to use to define PJI. Whereas MSIS (and to a lesser extent IDSA) criteria may miss some patients with PJI (false-negative) owing to the high threshold for confirmation of infection, the proposed EBJIS criteria may be prone to misdiagnose aseptic patients as PJI (false-positive), leading to unnecessary surgical interventions and antimicrobial treatments, because only one fulfilled criterion is sufficient to diagnose PJI (eg, increased synovial fluid leukocyte count). To prevent overdiagnosing PJI, additional tissue specimens are collected during revision surgery, which are analyzed microbiologically and histopathologically to definitely confirm or exclude the diagnosis of PJI and thereby guide further treatment.

In our study, the ELISA test did not outperform the lateral flow test, regardless of the defining classification system. This contrasts with previous reports, in which the performance of the ELISA test was superior to the lateral flow alpha defensin test [13, 19, 21]. We cannot explain with certainty why our results considerably differ from those of previous studies. However, none of the previous studies directly compared the performance of both diagnostic tests in the same patient population. This apparent difference in our findings and the ones of others highlights the importance of cautiously interpreting results deriving from different studies.

The sensitivity of both diagnostic tests was considerably lower when using the proposed EBJIS criteria (45%-50%) compared with reported values by authors using MSIS criteria [2, 13, 21]. This discrepancy reflects the fact that by using the proposed EBJIS criteria, more patients with PJI, especially chronic low-grade infections, were identified than by other definition criteria, as has recently been observed by our group [20]. Especially those infections are challenging and often misdiagnosed as aseptic failures. As a result of its poor sensitivity when applying proposed EBJIS criteria, the alpha defensin diagnostic test (neither ELISA nor lateral flow) does not appear to be an appropriate PJI screening test, especially for low-grade PJI. The preoperative joint aspiration with determination of the leukocyte count and differential remains the most accurate test, demonstrating > 90% sensitivity for diagnosing infection [11, 23, 26]. However, in situations in which the high synovial fluid leukocyte count may be false-positive as a result of an underlying noninfectious inflammatory condition of the joint (such as rheumatic disease, periprosthetic fracture, dislocation), or in the early postoperative period (within the first 6 weeks after surgery), the alpha defensin test may be used as a confirmatory test. Both the quantitative and qualitative tests have high specificity and are useful to rule in infection in situations in which leukocyte count is elevated as a result of underlying inflammatory conditions. As a result of the easy and rapid test setup, the lateral flow test is more suitable for the intraoperative setting, however at the cost of higher expenses.

Similar to our earlier report on the qualitative ALDF test [20], here we found that the quantitative (ELISA) test for alpha defensin was specific but not sufficiently sensitive for diagnostic use. The alpha defensin lateral flow test provides results within 10 minutes without the need for a laboratory, and so may be more applicable for the intraoperative setting, but it is much more expensive than the ELISA test that we studied here. The ELISA may be advantageous as a cost-saving approach where it is available as a preoperative test. In addition, the proposed EBJIS criteria diagnosed considerably more patients with PJI than other available criteria (MSIS and IDSA), consequently lowering the performance of both alpha defensin tests. Future research is needed to define the best definition criteria for PJI to avoid over- and underdiagnosing PJI and guide the most appropriate treatment strategy for each individual patient.