Evaluation of Modified Two-Tiered Testing Algorithms for Lyme Disease Laboratory Diagnosis Using Well-Characterized Serum Samples

Standard two-tiered testing (STTT) is the recommended algorithm for laboratory diagnosis of Lyme disease (LD). Several limitations are associated with STTT that include low sensitivity in the early stages of disease, as well as technical complexity and subjectivity associated with second-tier immunoblotting; therefore, modified two-tiered testing (MTTT) algorithms that utilize two sequential first-tier tests and eliminate immunoblotting have been evaluated.

ABSTRACT

Standard two-tiered testing (STTT) is the recommended algorithm for laboratory diagnosis of Lyme disease (LD). Several limitations are associated with STTT that include low sensitivity in the early stages of disease, as well as technical complexity and subjectivity associated with second-tier immunoblotting; therefore, modified two-tiered testing (MTTT) algorithms that utilize two sequential first-tier tests and eliminate immunoblotting have been evaluated. Recently, a novel MTTT that uses a VlsE chemiluminescence immunoassay followed by a C6 enzyme immunoassay has been proposed. The purpose of this study was to evaluate the performance of the VlsE/C6 MTTT using well-characterized serum samples. Serum samples from the CDC Lyme Serum Repository were tested using three MTTTs, VlsE/C6, whole-cell sonicate (WCS)/C6, and WCS/VlsE, and three STTTs (immunoblotting preceded by three different first-tier assays: VlsE, C6, and WCS). Significant differences were not observed between the results of the MTTTs assessed; however, the VlsE/C6 MTTT resulted in the highest specificity (100%) when other diseases were tested and the lowest sensitivity (75%) for LD samples. Significant differences were present between the results for various MTTTs and STTTs evaluated. Specifically, all MTTTs resulted in higher sensitivities than the STTTs for all LD groups combined and were significantly more accurate (i.e., higher proportion of correct classifications) for this group, with the exception of the WCS/ViraStripe STTT. Additionally, when other diseases were tested, only the results of the VlsE/C6 MTTT differed significantly from those of the WCS/ViraStripe STTT, with the VlsE/C6 MTTT resulting in a 6.2% higher accuracy. Overall, the VlsE/C6 MTTT offers an additional laboratory testing algorithm for LD with equivalent or enhanced performance compared to that of the other MTTTs and STTTs evaluated in this study.

INTRODUCTION

Current laboratory testing for Lyme disease (LD) in the United States is serology based. Specifically, assays detect antibody responses to Borrelia burgdorferi, the most common etiologic agent responsible for causing LD in the United States (1, 2). In 1995, a standardized two-tiered testing (STTT) algorithm was adopted and continues to be recommended by the Centers for Disease Control and Prevention (CDC) for laboratory diagnosis of LD (3). STTT utilizes a first-tier enzyme immunoassay (EIA) or immunofluorescence assay (IFA) that, if positive or equivocal, is followed by a second-tier assay consisting of an IgM and/or IgG immunoblot (IB). Both tiers must be considered positive for laboratory support of an LD infection, and the second tier is positive if either an IgM and/or IgG IB is positive, with IgM IB testing evaluated only on samples from patients with a duration of illness of ≤30 days.

Several limitations are associated with STTT; these include low sensitivity (29 to 40%) in early stages of infection when an erythema migrans (EM) skin lesion is present, difficulties associated with the subjectivity and interpretation in reading IBs, the inability to differentiate past from present infection, and the improper use of the recommended algorithm (4). Two modified two-tiered testing (MTTT) algorithms have been proposed with the intention of simplifying and improving test performance for LD laboratory testing (5, 6). The two proposed MTTT algorithms utilize Food and Drug Administration (FDA)-cleared first-tier assays and omit the use of IBs. A wide variety of first-tier FDA-cleared assays is available and includes EIAs and chemiluminescence immunoassays (CLIAs) that use either B. burgdorferi whole-cell sonicate (WCS), whole protein(s), or a single peptide (2).

The first MTTT algorithm was suggested by Branda et al. and uses a WCS EIA as the first test that, if positive or equivocal, is followed by the C6 EIA (5). More recently, a novel MTTT algorithm was proposed by Branda et al. that uses a whole-protein VlsE (variable major protein-like sequence, expressed) CLIA that, if positive or equivocal, is followed by the C6 EIA (6). For a patient to be considered positive for LD when tested by either of the MTTT algorithms, both tests need to be equivocal or positive.

The C6 EIA, which is used in both MTTT algorithms, is based on a single peptide that reproduces the sequence of the invariable region 6 (IR6) of the surface lipoprotein VlsE (7). Both the C6 peptide and VlsE lipoprotein are highly immunogenic (7–10). VlsE elicits a stronger IgM response than C6; however, C6 elicits an IgG response earlier than that observed for VlsE (10). Although observed differences in the responses produced by individual patients between VlsE and C6 exist, the overall diagnostic sensitivities of these antigens for patient groups are similar (1, 2).

The sensitivity of the WCS/C6 MTTT algorithm has been shown to be increased slightly compared to that of the STTT, particularly for acute- and convalescent-phase samples from early LD patients with EM (5, 11, 12). For most non-LD groups, the specificity of STTT is retained by this MTTT algorithm (5, 12). Additional studies to better assess the performance of the VlsE/C6 MTTT algorithm have not been performed. Therefore, the goal of the current study was to test the performance of the proposed VlsE/C6 and other MTTT algorithms using well-characterized and widely available serum samples and to compare the results to those with STTT algorithms. The algorithms reported here used FDA-cleared assays that are currently available for LD laboratory testing and have not been tested previously against serum samples available through the CDC Lyme Serum Repository (LSR) (with the exception of the C6/ViraStripe STTT) (11, 12). Our results corroborated the findings by Branda et al. (6) and demonstrated that the VlsE/C6 MTTT algorithm is slightly more sensitive than the STTT approaches tested in this study and retains equal specificity. Additionally, our results show that the VlsE/C6 MTTT algorithm achieves sensitivities and specificities for diagnosing LD similar to those of other MTTTs.

MATERIALS AND METHODS

Patient samples.

The sera used in this study have been previously described in other studies (11–13) and are from the CDC LSR (11). The LSR contains serum samples from LD-positive patients with various manifestations and stages of LD, as well as from LD-negative groups comprised of healthy controls and patients with other diseases. The sera from LD patients are from individuals who had early LD with a characteristic EM rash (acute- and convalescent-phase serum samples; n = 78; 38 paired samples), Lyme neuroborreliosis (n = 10), Lyme carditis (n = 7), or Lyme arthritis (n = 29) at the time of collection. All but two Lyme neuroborreliosis patients and three Lyme carditis patients had known durations of illness of ≤30 days when the serum samples were collected. The LD samples were not tested for coinfections, including infection with Borrelia miyamotoi. The control sera (n = 347) were collected from patients with fibromyalgia (n = 31), infectious mononucleosis (n = 30), multiple sclerosis (n = 22), rheumatoid arthritis (n = 21), severe periodontitis (n = 20), or syphilis (n = 20) and from healthy donors from regions of endemicity (n = 101) or nonendemicity (n = 102) for LD. For LD positive-control samples and samples from patients with other disease, IgM immunoblot testing was considered only if the duration of illness was ≤30 days. This criterion was not applied to healthy controls. The intended purpose of the CDC LSR is to provide samples that can be used by investigators for serologic test comparisons. Informed consent and Institutional Review Board approval were granted for the testing of these samples.

Serological testing performed.

The LSR samples were tested blindly at the ARUP Laboratories using the Liaison Borrelia burgdorferi CLIA (Stillwater, MN) that detects IgM and IgG antibodies against VlsE and the Captia Borrelia burgdorferi IgG/IgM EIA (Trinity Biotech, Jamestown, NY) that detects IgM and IgG antibodies against whole-cell lysate. It should be noted that the VlsE used in the Liaison CLIA is manufactured with both B. burgdorferi sensu stricto B31 and Borrelia garinii strain PBi because increased sensitivity has been observed with the combination of these antigens compared to that with VlsE manufactured from B31 alone (14). All assays were performed according to the manufacturers' instructions. Previously collected and published data on the samples used in this study included data for the C6 B. burgdorferi (Lyme) EIA (Immunetics, Boston, MA) and the ViraStripe IgM and IgG IBs (Viramed, Biotech Ag, Germany) (12). STTT criteria were applied to all samples (3). Specifically, 2 of 3 and 5 of 10 bands were required for IgM and IgG positivity, respectively. All IBs were read using densitometry and were considered positive when the intensity of the band was ≥60% of the cutoff control for IgM and ≥85% of the cutoff control band for IgG (12). At least one IB had to be positive for a second-tier positive result. Additionally, IgM criteria were applied only when the patient's duration of illness was ≤30 days. All algorithms evaluated in this study are summarized in Fig. 1.

FIG 1.

MTTT and STTT algorithms evaluated in this study.

Statistical methods.

Kappa statistics and their associated confidence intervals were computed for all serum groups combined to estimate agreement among test results, as previously described (11–13). McNemar's test was used to test for difference in percent correct values for the various testing strategies evaluated, and this was also performed as previously described (11–13). Asymptotic Wald intervals were used to estimate the magnitude of the difference.

RESULTS

First-tier tests alone.

The results for the first-tier tests are summarized in Table 1. When samples from patients with acute LD and EM were tested, the WCS gave higher sensitivity (73%) than VlsE and C6, both of which had sensitivities of 58%. A similar trend was observed with convalescent-phase samples from LD patients with EM. All assays performed well when samples for patients with Lyme neuroborreliosis were tested, with slight differences observed when Lyme carditis samples were evaluated. It should be noted that the numbers of samples tested for Lyme neuroborreliosis and Lyme carditis were low. The overall sensitivities for all LD groups combined were 78%, 81%, and 89% for VlsE, C6, and WCS, respectively, and the increase observed when WCS was used was due to an increase in sensitivity for samples from acute- and convalescent-phase LD patients with an EM. It is important to consider that the sensitivity of antibody-based tests increases with duration of illness and if dissemination of spirochetes occurs (1). Given that a higher number of early LD samples than samples obtained from patients with late-stage LD were used in this study, the overall sensitivities for all tests and algorithms being evaluated are likely lower than if equal numbers of samples from each stage were being tested. Therefore, we present all LD data combined in the manuscript only as a means to provide an overall value for each testing method to simplify comparisons among the same LD patients tested using different assays.

TABLE 1.

Results of first-tier tests, MTTTs, and STTTs

Sample type	No. of samples	Test resulta (no. of positives [%])
		First-tier test			MTTT algorithm			STTT algorithmb
		VlsE	C6	WCS	VlsE/C6	WCS/C6	WCS/VlsE	VlsE/ViraStripe	C6/ViraStripe	WCS/ViraStripe
Lyme disease	124
Early Lyme disease with EMc
Acute phase	40	23 (58)	23 (58)	29 (73)	20 (50)	22 (55)	23 (58)	17 (43)	17 (43)	20 (50)
Convalescent phase	38	30 (79)	32 (84)	35 (92)	29 (76)	30 (79)	29 (76)	23 (61)	23 (61)	24 (63)
Early Lyme neuroborreliosis or Lyme carditis
Lyme neuroborreliosis	10	10 (100)	10 (100)	10 (100)	10 (100)	10 (100)	10 (100)	9 (90)	9 (90)	9 (90)
Lyme carditis	7	5 (71)	6 (86)	7 (100)	5 (71)	6 (86)	5 (71)	5 (71)	6 (86)	7 (100)
Late Lyme disease
Lyme arthritis	29	29 (100)	29 (100)	29 (100)	29 (100)	29 (100)	29 (100)	28 (97)	28 (97)	28 (97)
Total for Lyme disease		97 (78)	100 (81)	110 (89)	93 (75)	97 (78)	96 (77)	82 (66)	83 (67)	88 (71)
Controls	347
Other diseases	144
Fibromyalgia	31	0 (0)	0 (0)	9 (29)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)
Severe periodontitis	20	0 (0)	0 (0)	1 (5)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)
Rheumatoid arthritis	21	0 (0)	0 (0)	9 (43)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)
Syphilis	20	2 (10)	2 (10)	18 (90)	0 (0)	2 (10)	2 (10)	0 (0)	1 (5)	0 (0)
Multiple sclerosis	22	0 (0)	1 (5)	8 (36)	0 (0)	1 (5)	0 (0)	0 (0)	0 (0)	0 (0)
Infectious mononucleosis	30	1 (3)	4 (13)	19 (63)	0 (0)	2 (7)	1 (3)	0 (0)	1 (3)	9 (30)
Total for other diseases		3 (2)	7 (5)	64 (44)	0 (0)	5 (3)	3 (2)	0 (0)	2 (1)	9 (6)
Healthy controls	203
Region of endemicity	101	3 (3)	1 (1)	24 (24)	1 (1)	1 (1)	1 (1)	1 (1)	1 (1)	5 (5)
Region of nonendemicity	102	1 (1)	4 (4)	23 (23)	1 (1)	2 (2)	0 (0)	0 (0)	1 (1)	1 (1)
Total for healthy controls		4 (2)	5 (2)	47 (23)	2 (1)	3 (1)	1 (0)	1 (0)	2 (1)	6 (3)
Total for negative controls		7 (2)	12 (3)	111 (32)	2 (1)	8 (2)	4 (1)	1 (0)d	4 (1)	15 (4)

^aMTTT, modified two-tiered testing; STTT, standard two-tiered testing; WCS, whole-cell sonicate.

^bFor STTT algorithms, IgM criteria were applied only when the duration of illness was ≤30 days. Duration of illness was considered only for other diseases as it does not apply to healthy controls.

^cEM, erythema migrans. The following numbers of acute/convalescent pairs were initially positive and then negative by the following tests: VlsE, 1 pair; C6, 1 pair; VlsE/C6, 3 pairs; WCS/C6, 4 pairs; WCS/VlsE, 4 pairs, VlsE/ViraStripe, 1 pair; C6/ViraStripe, 1 pair, WCS/ViraStripe, 1 pair. Three pairs were the same for all three MTTTs where the acute sample was initially positive and the convalescent sample was negative. All other pairs where this occurred for all other assays tested were different pairs.

^dActual percent positive is 0.29%.

When samples from patients with other diseases were tested, VlsE cross-reacted with three samples: two were from patients with syphilis, and one was from a patient with infectious mononucleosis. When C6 was used, cross-reactivity was observed for samples from patients with syphilis, multiple sclerosis, and infectious mononucleosis. WCS did not perform as well as VlsE or C6 when other diseases were tested, and cross-reactivity was observed with all of the other disease groups tested. The overall specificities for other diseases for the first-tier tests evaluated were 98%, 95%, and 56% for VlsE, C6, and WCS, respectively. Healthy controls tested had overall specificities of 98%, 98%, and 77% for VlsE, C6, and WCS, respectively (Table 1). VlsE had higher cross-reactivity than C6 for samples from healthy controls from regions of endemicity; however, C6 had higher cross-reactivity when samples from healthy controls from regions of nonendemicity were tested. WCS had similar cross-reactivity with samples from healthy controls from regions of both endemicity and nonendemicity.

Pairwise comparisons of the three assays used as first-tier tests (VlsE versus WCS, VlsE versus C6, and WCS versus C6) indicated that when all samples were tested, there were significant differences (as defined by P < 0.05) in results between VlsE and WCS (P < 0.01) and between C6 and WCS (P < 0.01) (Table 2). When samples were analyzed by groups (LD, other diseases, and healthy controls), VlsE and C6 had significantly (P < 0.01) higher accuracies than WCS when other diseases and healthy controls were tested; however, WCS had higher accuracy than VlsE and C6 when LD samples were tested. Overall, the percent agreement between the three first-tier assays when all samples were included was 71% (see Table S1 in the supplemental material), and the percent agreement between VlsE and C6 for all samples tested was 94% (Table 2).

TABLE 2.

Pairwise comparisons of proportions of correct results between different testing strategies

Test type and assays used	Comparison of correct test results by sample typea
	All			Lyme disease			Other diseases			Healthy controls
	% Difference	95% CI	P value	% Difference	95% CI	P value	% Difference	95% CI	P value	% Difference	95% CI	P value
First-tier tests
VlsE vs WCS	19.3	14.8, 23.8	<0.01	−10.5	−17.1, −3.8	<0.01	42.4	33.6, 51.1	<0.01	21.2	14.6, 27.8	<0.01
C6 vs WCS	18.9	14.4, 23.4	<0.01	−8.1	−15, −1.1	0.02	39.6	30.5, 48.7	<0.01	20.7	14.3, 27.1	<0.01
VlsE vs C6a	0.4	−1.9, 2.8	0.85	−2.4	−8.5, 3.6	0.55	2.8	−2.2, 7.8	0.34	0.5	−2.2, 3.1	1
Two-tiered tests
MTTTs
VlsE/C6 vs WCS/C6	0.4	−1.3, 2.2	0.79	−3.2	−7.9, 1.4	0.22	3.5	−0.2, 7.2	0.07	0.5	−1.7, 2.7	1
VlsE/C6 vs WCS/VlsE	−0.2	−1.7, 1.2	1	−3.2	−6.7, 1.9	0.37	2.1	−0.9, 5.1	0.25	−0.5	−1.9, 1.0	1
WCS/C6 vs WCS/VlsE	−0.6	−2.7, 1.4	0.65	0.8	−4.7, 6.4	1	−1.4	−5.9, 3.1	0.72	−1.0	−2.8, 0.9	0.48
STTTs
VlsE/ViraStripe vs WCS/ViraStripe	1.7	0, 3.8	0.12	−4.8	−9.4, −0.3	0.04	6.2	1.6, 10.9	0.01	2.5	−0.2, 5.1	0.07
C6/ViraStripe vs WCS/ViraStripe	1.7	0, 3.7	0.10	−3.2	−7.9, 1.4	0.22	4.9	1.0, 9.0	0.02	2.5	−0.2, 5.1	0.07
VlsE/ViraStripe vs C6/ViraStripe	0	−1.2, 1.2	1	−1.6	−5.6, 2.3	0.62	1.4	−1.2, 4.0	0.48	0	−0.5, 0.5	1

^aThe overall percent agreement between VlsE and C6 results was calculated to be 94%.

MTTTs.

Three MTTTs were tested: (i) VlsE/C6, (ii) WCS/C6, and (iii) WCS/VlsE (Fig. 1); the results are summarized in Table 1. The sensitivities for acute- and convalescent-phase LD samples from patients with EM were similar at 50%, 55%, and 58% for acute-phase samples and 76%, 79%, and 76% for convalescent-phase samples for the VlsE/C6, WCS/C6, and WCS/VlsE MTTTs, respectively (Table 1). The sensitivities for all LD samples combined were 75% when the VlsE/C6 MTTT was used, 78% with the WCS/C6 MTTT, and 77% with the WCS/VlsE MTTT.

When samples from patients with other diseases were tested, the results of the three MTTTs evaluated were not significantly different. Specificities were 100%, 97%, and 98% for the VlsE/C6, WCS/C6, and WCS/VlsE MTTTs, respectively (Table 1). Both the WCS/C6 and WCS/VlsE MTTTs cross-reacted with samples from patients with syphilis (specificities were 90% for both MTTTs) and infectious mononucleosis (specificities were 93% and 97% for the WCS/C6 and WCS/VlsE MTTTs, respectively). Additionally, the WCS/C6 MTTT cross-reacted with one multiple sclerosis sample (specificity for this group was 95%).

The VlsE/C6 MTTT was cross-reactive with samples from two healthy controls, one from a region of endemicity and one from a region of nonendemicity. The overall specificity for this MTTT for healthy controls was 99% (Table 1). The WCS/C6 MTTT cross-reacted with one and two samples from healthy controls from regions of endemicity and nonendemicity, respectively, and had a specificity of 99%. The WCS/VlsE MTTT was cross-reactive with a sample from only one healthy control from a region of endemicity, resulting in a rounded specificity of 100%. The overall specificities for all negative-control samples (samples from patients with other diseases and those from healthy controls combined) were 98% for the WCS/C6 MTTT and 99% for both the VlsE/C6 and WCS/VlsE MTTTs.

Pairwise comparisons of the proportion of results called correct between the MTTTs was performed (Table 2). No significant differences between the proportions of correct results for the three MTTT combinations (VlsE/C6 versus WCS/C6, VlsE/C6 versus WCS/VlsE, and WCS/C6 versus WCS/VlsE) were observed for comparisons among all samples, LD samples, samples from patients with other diseases and healthy controls. The percent agreement between the three MTTTs for all samples evaluated was 96% (Table S1).

STTTs.

Three different STTT combinations were tested: (i) VlsE/ViraStripe, (ii) C6/ViraStripe, and (iii) WCS/ViraStripe. The results for IgM and IgG ViraStripe immunoblot testing are summarized in Table S2 in the supplemental material. The sensitivities for acute- and convalescent-phase samples for LD patients with EM were similar at 43%, 43%, and 50% for acute-phase samples and 61%, 61%, and 63% for convalescent-phase samples for VlsE/ViraStripe, C6/ViraStripe, and WCS/ViraStripe STTTs, respectively (Table 1). The overall sensitivities for the three STTT combinations tested with all LD samples were 66% for the VlsE/ViraStripe STTT, 67% for the C6/ViraStripe STTT, and 71% for the WCS/ViraStripe STTT.

When samples from patients with other diseases were tested, specificities of 100%, 99%, and 94% were obtained for the VlsE/ViraStripe, C6/ViraStripe, and WCS/ViraStripe STTTs, respectively (Table 1). Both the C6/ViraStripe STTT and WCS/ViraStripe STTT cross-reacted with samples from patients diagnosed with infectious mononucleosis, with a lower specificity observed when the WCS/ViraStripe STTT was used (specificities were 97% for the C6/ViraStripe STTT and 70% for the WCS/ViraStripe STTT for this group). The overall specificities for all negative controls (samples from patients with other diseases and from healthy controls) were 100% (one healthy control from a region of endemicity was found to be false positive), 99%, and 96% for the VlsE/ViraStripe, C6/ViraStripe, and WCS/ViraStripe STTTs, respectively.

Pairwise comparisons were also performed between the STTTs evaluated in this study (VlsE/ViraStripe versus WCS/ViraStripe, C6/ViraStripe versus WCS/ViraStripe, and VlsE/ViraStripe versus C6/ViraStripe), and results are summarized in Table 2. A significant difference was observed between results for the VlsE/ViraStripe STTT and those of the WCS/ViraStripe STTT (P = 0.04) when LD samples were tested (a 4.8% higher accuracy was observed for the WCS/ViraStripe STTT; confidence interval [CI] of 0.3 to 9.4%), the results for VlsE/ViraStripe STTT and those of the WCS/ViraStripe STTT (P = 0.01) when samples from patients with other diseases were tested (a 6.2% higher accuracy was observed for the VlsE/ViraStripe STTT; CI of 1.6 to 10.9%), and results for the C6/ViraStripe STTT and those of the WCS/ViraStripe STTT (P = 0.02) when samples from patients with other diseases were tested (a 4.9% higher accuracy was observed for the C6/ViraStripe STTT; CI of 1.0 to 9.0%).

The difference in the proportions of samples classified correctly between STTTs using the VlsE/ViraStripe STTT and those using the WCS/ViraStripe STTT when all samples were considered was 1.7% (95% CI of 0 to 3.8%; P = 0.12), indicating that the former is slightly more accurate than the latter, on average (Table 2). When the results of the VlsE/ViraStripe STTT were compared to those of the C6/ViraStripe STTT, no differences were observed between the two (0% difference with a 95% CI of −1.2 to 1.2%; P = 1.0). When only LD samples were evaluated, the WCS/ViraStripe STTT had only a 3.2% (95% CI of −1.4 to 7.9) higher accuracy than the C6/ViraStripe STTT; no significant difference was observed between the results of these two STTTs (P = 0.22). The percent agreement between the results of the three STTTs tested was 95% (Table S1).

MTTTs versus STTTs.

When pairwise comparisons were performed between results of MTTTs and those of STTTs, significant differences between the two approaches were primarily observed when all samples or when only LD samples were evaluated (Table 3). Only two comparisons were not significantly different when all samples were evaluated. The first was a comparison of the results of the WCS/C6 MTTT to those of the VlsE/ViraStripe STTT (P = 0.15), and the second was a comparison of the results of the WCS/C6 MTTT to those of the C6/ViraStripe STTT (P = 0.12). All other comparisons were significantly different (P < 0.05). When results of the MTTTs and STTTs were compared for LD samples, the comparisons that were not significantly different were the results of the VlsE/C6 MTTT compared to those of the WCS/ViraStripe STTT (P = 0.36), the results of the WCS/C6 MTTT compared to those of the WCS/ViraStripe STTT (P = 0.07), and the results of the WCS/VlsE MTTT compared to those of the WCS/ViraStripe STTT (P = 0.12). For samples from patients with other diseases, only results of the VlsE/C6 MTTT and those of the WCS/ViraStripe STTT differed significantly (P = 0.01), with a 6.2% higher accuracy (CI of 1.6 to 11%) observed for the VlsE/C6 MTTT. There were no significant differences observed between the results of any of the MTTTs compared to those of the STTTs for healthy-control samples.

TABLE 3.

Pairwise comparisons of MTTT and STTT results

Test and compared assays	Comparison of correct test results by sample type
	All			Lyme disease			Other diseases			Healthy controls
	% Difference	95% CI	P value	% Difference	95% CI	P value	% Difference	95% CI	P value	% Difference	95% CI	P value
VlsE/C6 vs:
VlsE/ViraStripe	2.1	0.4, 4.0	0.02	9.0	2.6, 15.2	0.01	0	−0.7, 0.7	1	0	−0.7, 0.7	1
WCS/ViraStripe	3.8	1.2, 6.4	<0.01	4.0	−3.6, 11.7	0.36	6.2	1.6, 11.0	0.01	2.0	−0.9, 4.8	0.22
C6/ViraStripe	2.1	0.2, 4.1	0.03	7.0	0.5, 14.1	0.04	1.4	−1.2, 4.0	0.48	−0.5	−1.9, 1.0	1
WCS/C6 vs:
VlsE/ViraStripe	1.7	−0.5, 3.9	0.15	12.0	5.1, 19.1	<0.01	−3.5	−7.2, 0.2	0.07	−1.0	−2.8, 0.9	0.48
WCS/ViraStripe	3.4	1.0, 6.1	0.01	7.0	−0.3, 14.8	0.07	2.8	−2.2, 7.8	0.34	1.5	−1.6, 4.5	0.45
C6/ViraStripe	1.7	−0.4, 3.8	0.12	11.0	3.8, 17.1	<0.01	−2.1	−5.1, 0.9	0.25	−1.0	−2.8, 0.9	0.48
WCS/VlsE vs:
VlsE/ViraStripe	2.3	0.4, 4.3	0.02	11.0	4.9, 17.7	<0.01	−2.1	−5.1, 0.9	0.25	0	−0.5, 0.5	1
WCS/ViraStripe	4.0	1.3, 6.8	<0.01	7.0	−1.3, 14.2	0.12	4.2	−1.2, 9.5	0.18	2.5	−0.2, 5.1	0.07
C6/ViraStripe	2.3	0.1, 4.5	0.04	10.0	2.4, 17.0	0.01	−0.7	−4.4, 3.0	1	0	−0.5, 0.5	1

DISCUSSION

In this study, we used well-characterized serum samples from the CDC LSR to evaluate the recently proposed VlsE/C6 MTTT algorithm (6, 11). This algorithm was compared to two other MTTT algorithms, one that uses a WCS EIA followed by a C6 EIA and another that uses a WCS EIA followed by a VlsE CLIA. Additionally, results from all MTTTs were compared to those from several STTTs (VlsE/ViraStripe, C6/ViraStripe, and WCS/ViraStripe). Overall, the findings of our study corroborate the findings of Branda et al. (6). Specifically, an increase in sensitivity was observed for acute- and convalescent-phase samples from early LD patients with an EM when any of the three MTTT algorithms was used and compared to results with the STTT algorithm. Of the three MTTT algorithms, the VlsE/C6 MTTT had the lowest sensitivity when acute-phase early LD samples were tested. This result differed slightly from results reported by Branda et al. and is likely due to the different whole-cell lysate tests used in the two studies. The same trend as that observed with acute- and convalescent-phase samples from patients with EM was observed when all LD samples were evaluated as a group. The highest sensitivity for all LD samples combined was observed when the WCS/C6 MTTT algorithm was used; however, this approach had a slightly lower specificity than the WCS/VlsE and VlsE/C6 MTTT algorithms.

Of the three first-tier tests used (VlsE, C6, and WCS), the WCS had the lowest specificity, at only 56% when samples from patients with other diseases were tested; however, when this assay was followed by C6 or VlsE as second-tier tests, specificity increased to 97% and 98%, respectively, for this test group. When WCS was followed by IB, specificity was much lower (94%) for samples from patients with other diseases than when VlsE or C6 was used as a first-tier test followed by IB (100% and 99% specificity, respectively). The highest specificities when samples from patients with other diseases were tested occurred when any of the algorithms used VlsE as a first-tier or as a second-tier test. Among the samples tested from patients with other diseases, false-positive results with the various MTTT algorithms (with exception of VlsE/C6 MTTT) and STTT algorithms (with exception of the VlsE/ViraStripe STTT) occurred in samples from patients with syphilis, multiple sclerosis, and infectious mononucleosis. Interestingly, cross-reactivity when all three first-tier tests were used was observed in samples from patients with infectious mononucleosis. The highest false positivity was observed for this group when WCS (37% specificity) was used, followed by C6 (87% specificity) and then VlsE (97% specificity). These data support the continued need of confirmatory testing, showing that no first-tier test alone performs as well as any of the two-tiered algorithms. Among the two-tiered algorithms tested, the MTTT and STTT algorithms, the only testing algorithms that resulted in 100% specificity for infectious mononucleosis samples were the VlsE/C6 MTTT and the VlsE/ViraStripe STTT algorithms.

Previously, we used the same samples tested in this study to evaluate the original MTTT algorithm (WCS/C6) based on the Vidas WCS EIA (LYT; bioMérieux, Inc., Durham, NC) followed by the C6 EIA (Immunetics) (12). In the current study, we tested this MTTT algorithm using the Captia Borrelia burgdorferi IgG/IgM EIA (Trinity Biotech) WCS assay. Our results for this MTTT algorithm were comparable to those of the two different WCS assays. Specifically, the sensitivities for all LD samples tested were 76% and 78%, respectively, when the Vidas WCS/C6 and the Captia WCS/C6-based MTTT algorithms were used. The specificities attained when samples from all controls (healthy subjects and patients with other diseases) were tested was 98% for both of these MTTT algorithm configurations. It should be noted that the Vidas WCS EIA is no longer commercially available. The LYT assay was replaced by the manufacturer in favor of two isotype-specific EIAs (LYM and LYG). We previously showed equivalence between the two assay formats (13).

When results of the MTTT algorithms were compared to those of the STTT algorithms, the percent difference in proportion of samples correctly classified was always in favor of the former when all samples from patients with LD were considered. This was not the case when the results of the MTTT and STTT algorithms were compared for samples from patients with other diseases and healthy controls. For these two groups, the use of an MTTT algorithm did not always result in proportions of correctly classified samples significantly higher than those with use of an STTT algorithm. Although the performance of the MTTT algorithms was not inferior to that of the STTT algorithms, there was an overall trend for better specificity when results of all MTTTs were compared to those of all STTTs. In particular, the STTT that incorporated WCS as the first-tier test did not perform as well as the two MTTTs that used WCS.

Additional studies are required to further understand the performance of the MTTT algorithms and to understand the interdependence of the combined assays. Additionally, the order of tests applied for first- and second-tier assays needs to be evaluated to establish the most beneficial test combination. If the VlsE/C6 combination proposed by Branda et al. (6) was performed in the opposite order (C6 followed by VlsE), the same overall results would occur, given that both tests need to be considered positive for an overall positive result. However, using our data set (Table 1), if the VlsE CLIA is used as a first-tier test, 104 second-tier C6 EIAs are needed, whereas if the C6 EIA is used as the first-tier test, 112 second-tier VlsE CLIAs are required. Traditionally, the less specific, more sensitive test has been applied as a first-tier test that is followed by the more specific, less sensitive second-tier test. With the samples tested in this study, the C6 appears to be the more appropriate screening assay, and VlsE would be the more appropriate confirmatory assay. However, as a first-tier test VlsE requires fewer confirmatory tests than C6, and there was no statistical difference in the results observed whether VlsE was employed as a first- or second-tier test in either the VlsE/C6 or WCS/VlsE MTTTs. The minor differences in the MTTT results could be better realized when the algorithm is applied to demographics with drastically disparate local incidences of LD with high testing volumes. For instance, in a region with a high incidence of LD, beginning the MTTT with the C6 would ensure the most sensitive detection of true infections in the first phase of testing, while maintaining very high sensitivity and specificity in the second tier with VlsE. Alternatively, in regions with very low prevalence of LD (and therefore low pretest probability of LD), starting the MTTT with VlsE would reduce false positives caused by more prevalent conditions such as infectious mononucleosis.

To further understand the optimal sequence of the tests in various MTTT algorithms, additional testing of patients with LD as well as testing larger numbers of controls/control groups is required. Control groups should include samples from conditions that clinically resemble LD and/or result in a rash similar to the EM of LD (e.g., southern tick-associated rash illness) as well as samples from patients with other tick-borne illnesses known to be serologically cross-reactive (e.g., tick-borne relapsing fever), albeit these infections are rare, and samples are difficult to obtain (15–19). FDA clearance is not currently available for the MTTT algorithms evaluated in this study; however, when STTT recommendations were made, provisions were included for the acceptance of new tests that displayed equivalent or improved sensitivity and specificity for replacement of one or both components of the STTT algorithm (3). Recently, the IgM and IgG ViraChip (Viramed, Inc., Oceanside, CA), a second-tier test platform differing from Western and striped immunoblotting, has received FDA clearance, paving the way for FDA clearance of other second-tier tests and prospective MTTT approaches.