Evaluation of a sequential enzyme immunoassay testing algorithm for Lyme disease demonstrates lack of test independence but high diagnostic specificity

Abstract

To diagnose Lyme disease, a two-tier testing algorithm is used in which supplemental IgM and IgG immunoblots to detect antibody to Borrelia burgdorferi are reflexively performed if a first-tier assay, such as a whole-cell sonicate-based enzyme immunoassay (WCS EIA), is reactive. Recent data suggest that equal specificity is found by substituting the C6 peptide EIA for immunoblots. In this study using 3956 control sera, we demonstrated that although this two-tier testing algorithm does significantly improve diagnostic specificity compared with each of the EIAs individually, the WCS EIA and the C6 peptide EIA are not independent tests. Therefore, when the C6 peptide EIA is used as the second-tier test, it should be regarded as a supplemental rather than a confirmatory test.

Since 1995, two-tier sequential serologic testing to detect antibody to Borrelia burgdorferi has been the recommended testing strategy to diagnose United States patients with extracutaneous manifestations of Lyme disease (CDC, 1995). In this testing protocol, separate IgM and IgG immunoblots are performed on serum samples found to be seroreactive by either an enzyme immunoassay (EIA) or an immunofluorescence assay.

Often, for the first tier of the two-tier protocol, laboratories use whole-cell sonicate (WCS) EIAs, which are first-generation EIAs prepared from protein extracts of cultured strains of B. burgdorferi. Similarly, the most commonly used second-tier immunoblot reagents are prepared from protein extracts of cultured strains of B. burgdorferi. Not unexpectedly, therefore, WCS EIAs and immunoblot assays are not independent tests (Wormser et al., 2000). Furthermore, immunoblot testing has other drawbacks. One is subjectivity in interpretation, leading to false-positive test results because of overreading of weak bands, especially with respect to the IgM immunoblot, which only requires the presence of two bands to be interpreted as positive (CDC, 1995; Seriburi et al., 2012). Another is the inability of some hospital laboratories to perform immunoblot assays, which results in a time delay in obtaining the results, as the serum specimens must be transported to another testing site. An additional consideration is the added financial cost of immunoblot testing in comparison to the simpler EIA format of first-tier tests (Wormser et al., 2013a). To address many of these limitations, alternative two-tier testing strategies are being evaluated. One promising approach, called the 2-EIA protocol, uses the C6 peptide EIA as the second-tier test on serum samples that are reactive by a WCS-based EIA (Branda et al., 2011).

Two large-scale studies have shown essentially equivalent specificity using this approach compared with the standard two-tier algorithm using immunoblots (Branda et al., 2011; Wormser et al., 2013a). However, whether WCS EIAs and the C6 peptide EIA are independent tests (Shen et al., 2001) has not been systematically evaluated. In the current study, we addressed this question by utilizing over 3900 control serum samples.

1. Methods

Unpublished data collected in four previously reported studies that evaluated the specificity of WCS EIAs and the C6 peptide EIA were analyzed (Branda et al., 2011; Lipsett et al., 2016; Molins et al., 2016; Wormser et al., 2013b). Two analyses were performed: one that compared the specificity of the C6 peptide EIA with WCS EIA reactive versus WCS EIA negative control sera, and a second that excluded those control serum samples that were reactive by a WCS EIA and also positive by an IgG immunoblot assay, with the IgG immunoblot assay interpreted using the criteria recommended by the Centers for Disease Control and Prevention (CDC, 1995). A positive IgG immunoblot requires reactivity against at least 5 out of 10 diagnostic antigens. The second analysis was performed based on the assumption that individuals with IgG immunoblot positive results may have had a prior B. burgdorferi infection since donors of control serum samples were not interviewed to exclude those with a history of Lyme disease. The study was considered exempt by the institutional review board at New York Medical College.

1.1. Statistical methods

Two-tailed Fisher’s exact test was used for bivariate comparisons of proportions. Some data were analyzed using a two-sample t test, assuming unequal variances. However, in comparing the rate of falsely reactive results for serum samples that were reactive by both the C6 peptide EIA and the WCS EIA with the rate of reactivity by either assay alone, a modification of the one-tailed McNemar test was performed. The modification accounts for the association induced when comparing the joint outcome of the C6 peptide EIA and WCS EIA to each one by itself. For the latter comparisons, one-sided P values were used because the circumstance in which both of the tests were reactive would only reduce misclassification of either test separately, not increase it. A P value < 0.05 was considered statistically significant. Confidence intervals were calculated using Newcombe’s method #10 without the continuity correction (Newcombe, 1998).

2. Results

To determine if WCS EIAs and the C6 peptide EIA are independent tests, we compared the specificity of the C6 peptide EIA in control serum samples separated into those that were WCS EIA negative and those that were WCS EIA reactive (i.e., positive or equivocal). If the tests were independent, there should be no significant difference in the specificity of the C6 peptide EIA in these two groups. However, the results demonstrate that the specificity of the C6 peptide EIA was significantly lower when assessed using sera regarded as falsely reactive by a WCS EIA, compared with control sera that tested negative by the same WCS EIA (Table 1) (P ≤ 0.0002). The reduction in specificity was approximately 16% (95% CI: 10.1%–23.7%) using sera from healthy controls and 7.2% (95% CI: 3.0%–14.6%) using sera from disease controls, defined as serum donors who had illnesses other than Lyme disease. The overall reduction in specificity of the C6 peptide EIA in the total group of controls was 11.9% (95% CI: 8.0%–17.2%), with reductions of 18.9% (Branda et al., 2011), 11.7% (Lipsett et al., 2016), 9.8% (Wormser et al., 2013b), and 9.3% (Molins et al., 2016) in the four individual studies.

Table 1.

Comparisons of C6 Peptide EIA test specificity

Categories of controls (references)	Specificity of the C6 peptide EIA in control sera that are negative by WCS EIA	Specificity of the C6 peptide EIA in control sera that are reactivea by WCS EIA	Specificity of the C6 peptide EIA in control sera that are reactivea by WCS EIA but IgG immunoblot negative	P value columns 2 versus 3	P value columns 2 versus 4
Healthy controls from four studies (Branda et al., 2011; Lipsett et al., 2016; Molins et al., 2016; Wormser et al., 2013b)	3235/3275 (98.8%)	97/117 (82.9%)	97/107 (90.7%)	<0.0001	<0.0001
Disease controls from three studies (Branda et al., 2011; Molins et al., 2016; Wormser et al., 2013b)	471/474(99.4%)	83/90 (92.2%)	82/87 (94.3%)	0.0002	0.0031
Total controls (healthy plus disease) from four studies (Branda et al., 2011; Lipsettet al., 2016; Molins et al., 2016; Wormser et al., 2013b))	3706/3749 (98.9%)	180/207 (87.0%)	179/194 (92.3%)	<0.0001	<0.0001

^aReactivity of either the WCS EIA or the C6 peptide EIA means that the EIA results were positive or equivocal.

Even after excluding WCS EIA serum samples that were IgG immunoblot positive, which may be more likely than other samples to have been collected from control subjects with past exposure to B. burgdorferi, the C6 peptide EIA had significantly lower specificity in the WCS EIA falsely reactive samples compared with the WCS EIA-negative samples (P < 0.0001). The specificity of the C6 peptide EIA was about 5% higher in the combined group of controls after excluding these samples, but this difference was not statistically significant (P = 0.12).

To further explore whether the analysis could be confounded by past B. burgdorferi exposure among some control subjects, we determined the number of IgG immunoblot bands in serum samples that were IgG immunoblot negative and thus had fewer than five bands. The number of IgG bands was significantly greater for serum samples that were reactive by both the WCS EIA and the C6 peptide EIA (number of samples evaluable = 15) compared with sera that were only reactive by the WCS EIA and not by the C6 peptide EIA (number of samples evaluable = 179): mean number of bands ± SD was 1.53 ± 1.45 versus 0.665 ± 0.883, P = 0.038. A similar comparison for serum samples that were reactive by both the WCS EIA and the C6 peptide EIA and those that were only reactive by the C6 peptide EIA and not by the WCS EIA (number of samples evaluable = 42) also showed a significant difference in the mean number of IgG bands (1.53 ± 1.45 versus 0.619 ± 0.615, P = 0.034).

Although the WCS EIAs and the C6 peptide EIA are not independent tests, the frequency of simultaneous reactivity with both EIAs (27/3956 [0.7%]) differed significantly from the frequency of seroreactivity by the WCS EIA (207/3956 [5.2%], P < 0.0001) or by the C6 peptide EIA (70/3956 [1.8%], P< 0 .0001) when these tests were considered individually. This observation establishes that the 2-EIA protocol significantly improves diagnostic specificity when compared with the individual EIAs.

3. Discussion

This study, which included more than 3900 control serum samples, demonstrated that the WCS EIA and the C6 peptide EIA for antibody to B. burgdorferi are not independent tests. Independence between tests is usually attributable to the use of different testing approaches, such as serology versus culture, or, in the case of two serologic tests, to the use of different antigenic constituents (Shen et al., 2001). Although the WCS and the C6 peptide EIAs are not independent tests, our study findings indicate that a two-tier testing algorithm with these EIAs significantly improves diagnostic specificity compared with either EIA when used alone.

The relative infrequency of concordant reactivity between these EIAs might seem counterintuitive since, in theory, the C6 peptide would be expected to be included among the proteins and peptides present in a protein extract (sonicate) of B. burgdorferi cells. The most likely explanation for this is that the vlsE gene, which encodes the VlsE protein from which the peptide used in the C6 peptide EIA is derived, is minimally expressed, or not expressed at all, by in vitro cultures of B. burgdorferi (Crother et al., 2003; Wormser et al., 2013b), which are typically used to prepare WCS EIAs. In addition, even if expressed, the epitopes of the C6 peptide recognized in patients with Lyme disease may not be consistently exposed on the molecular surface of the vlsE protein in vitro (Embers et al., 2007). Thus, the antigenic constituents of the two assays probably differ in that the VlsE protein and the relevant C6 peptide epitopes are not well represented in whole-cell lysates, and the C6 peptide EIA does not incorporate other B. burgdorferi protein antigens. However, the lack of complete discordance between reactivity on these EIAs may imply that there is some degree of VlsE protein expression by the cultured B. burgdorferi cells used to manufacture WCS EIAs (Lawrenz et al., 1999).

Alternatively, concordant reactivity between the two tests could relate to vulnerabilities of the EIA platform itself. It is conceivable, for example, that individual serum samples may contain broadly cross-reactive antibodies capable of causing false-positive results in EIAs with different antigenic constituents. Furthermore, false-positive results can be caused by antibody reactivity against EIA components other than the target antigens; such components (e.g., bovine serum albumin or milk proteins used as blocking agents) can be common to many EIAs regardless of their target antigens (Guven et al., 2014). Finally, some of the control sera may have been obtained from persons previously infected with B. burgdorferi. The improved specificity of the C6 peptide EIA in the combined group of control serum samples that were falsely reactive using a WCS EIA, after excluding IgG immunoblot-positive samples, is consistent with the hypothesis that at least some of the observed concordance is due to true seroreactivity against B. burgdorferi. It is also likely that removal of additional serum samples representing true seroreactivity but meeting less stringent evidence of a Lyme disease history would have further reduced the concordance between these EIAs. ln support of this hypothesis, we demonstrated that serum samples that were negative by IgG immunoblot criteria (and thus had fewer than five bands) but were reactive by both the WCS EIA and the C6 peptide EIA had significantly more IgG bands compared with serum samples that were reactive by only one of the two EIAs (P < 0.04). Serologic testing, however, is primarily intended to diagnose active Lyme disease rather than to assess prior exposures or immune status (Wormser et al., 2000).

ln conclusion, as previously shown for the IgM immunoblot and the WCS EIA (Wormser et al., 2000), the WCS EIA and the C6 peptide EIA are also not independent tests for detection of antibodies to B. burgdorferi. As with immunoblot testing, when using the 2-EIA protocol of a WCS EIA with reflex to the C6 peptide EIA, the C6 peptide EIA should not be regarded as a confirmatory test but instead as a supplemental test.