A Multisite Trial Comparing Two Cytomegalovirus (CMV) pp65 Antigenemia Test Kits, Biotest CMV Brite and Bartels/Argene CMV Antigenemia

Abstract

A total of 513 blood specimens, predominantly from organ transplant recipients, human immunodeficiency virus-positive patients, and bone marrow transplant recipients, were tested for cytomegalovirus (CMV) by culture and pp65 antigenemia across four test sites. Peripheral blood leukocytes were examined by using both the Biotest CMV Brite and the Bartels/Argene CMV Antigenemia kits. A total of 109 specimens were positive for CMV, 106 (97%) were positive by antigenemia, and 34 (31%) were positive by culture. According to the manufacturers' instructions, 150,000 cells were applied per slide for the Biotest kit and 200,000 cells per slide for the Bartels kit. A total of 93 specimens (88%) were positive by the Biotest kit, and 86 (81%) were positive by the Bartels kit. In specimens found to be positive by only one kit, the positive cell counts were low (median, 1; range, 1 to 7). When the data from all four sites were combined and analyzed, there was no statistical difference between the performance of the two kits; the Biotest and Bartels kits were found to be equivalent in sensitivity, specificity, and positive and negative predictive values for the detection of CMV pp65 antigenemia.

Cytomegalovirus (CMV) continues to be an important pathogen in transplant recipients, human immunodeficiency virus (HIV)-infected patients, and other immunosuppressed individuals. In recent years, it has been found to be preferable to treat CMV infection in high-risk patients prior to the onset of clinical disease (1, 4). Culture methods, including shell vial centrifugation cultures, lack sufficient sensitivity to detect CMV infection prior to the development of clinical disease (1, 2, 4).

The CMV antigenemia test has proven to be a rapid and sensitive assay with which to detect infection. It utilizes immunofluorescence methodology that can be readily implemented in routine clinical virology laboratories. In addition, the antigenemia assay is quantitative and has been useful for estimating the likelihood of disease progression, as well as for monitoring the response to therapy (1, 7).

There are few commercially available kits for CMV antigenemia testing and few studies comparing them with sample sizes large enough to guarantee reasonable statistical power. In this report we present the findings of a multisite trial comparing two commercially available kits for CMV antigenemia testing: the Biotest CMV Brite and Bartels/Argene CMV Antigenemia tests.

MATERIALS AND METHODS

Trial sites.

This study was performed in the clinical virology laboratories at four sites: the University of Pittsburgh Medical Center (UPMC), Pittsburgh, Pa.; North Shore University Hospital-NYU School of Medicine (NSUH), Manhasset, N.Y.; Yale New Haven Hospital (YNHH), New Haven, Conn.; and Integris Baptist Medical Center (IBMC), Oklahoma City, Okla. All sites had between 2 and 5 years of experience in antigenemia testing at the time of the study: UPMC and NSUH using Argene antibodies and YNHH and IBMC using Biotest antibodies.

Patient samples.

During the study period, June to November 1997, 513 blood specimens were received for CMV testing across the four sites: 299 (58%) specimens from organ transplant recipients, 10 (2%) from bone marrow transplant recipients, 149 (29%) from HIV-positive patients, and 55 (11%) from patients with other or unknown reasons for testing.

Processing and testing.

Blood specimens were separated into three aliquots: one aliquot was processed and tested using the Biotest CMV Brite Kit (Biotest Diagnostics, Denville, N.J.), one aliquot was tested with the Bartels/Argene CMV Antigenemia Kit (Intracel Corp., Issaquah, Wash.), and one aliquot was cultured by the shell vial assay (UPMC and IBMC) or standard tube culture (NSUH and YNHH), according to the standard of practice in each laboratory (5). Sample processing and testing for each kit were performed according to the manufacturer's instructions. In brief, peripheral blood leukocytes (PBLs) were separated by dextran sedimentation, erythrocytes were lysed in NH₄Cl solution, and PBLs were then washed and counted. Cells were applied to slides by cytocentrifugation. For the Biotest kit slides of 150,000 cells each were prepared, while for the Bartels/Argene kit slides with 200,000 cells per slide were prepared. All slides were then fixed in formaldehyde, permeabilized in NP-40 solution, and stained with a pool of antibodies to CMV pp65. The antibodies used were C10/C11 for the Biotest and 1C3 and AYM-1 for the Bartels/Argene test. After incubation with fluorescein-labeled conjugate, slides were examined under an epifluorescence microscope, and positive cells were enumerated. The sites differed in the number of slides prepared per specimen. Two sites (YNHH and IBMC) prepared two Biotest slides and one Bartels slide per specimen, one site (NSUH) prepared two slides with each kit, and the fourth site (UPMC) prepared one slide of each. All numbers reported here and used for statistical analysis are based on positive cell counts per slide.

In addition to the different antibody reagents and different numbers of cells per slide, differences between the kits included vortexing of the cells in the Biotest kit and its prohibition in the Bartels kit, the higher concentration of NH₄Cl in the Biotest erythrocyte lysis buffer and the higher concentration of Evan's blue counterstain in the Bartels/Argene kit. In addition, more reagents were provided at working dilutions in the Biotest kit.

Discrepant results.

Discrepant results, defined as specimens testing positive by either or both antigenemia kits but negative by culture, were investigated by assessment of other laboratory data and review of patient histories for evidence of CMV infection or disease. Results were considered true positives if CMV was isolated from other cultures of blood, urine, nasopharynx, saliva, bronchoalveolar lavage fluid, or tissue, if histopathology was diagnostic for CMV infection, if seroconversion to CMV or anti-CMV immunoglobulin M antibodies were detected, or if CMV retinitis was diagnosed. In some patients, no additional samples were submitted for testing; these patient results were omitted from the final analysis.

Data analysis.

Results were analyzed separately by trial site and also collectively. Comparisons of positive cell counts were performed with the Wilcoxon signed rank test, and comparisons of sensitivity, specificity, positive and negative predictive values were made with the chi-square and exact tests. All statistical analyses were performed with StatXact 3 software (Cytel, Cambridge, Mass.).

RESULTS

Patient populations.

There were significant site variations in distribution of patient types. A total of 90% of UPMC and 81% of IBMC samples were from organ transplant recipients. In contrast, 78% of NSUH samples were from HIV-infected patients. YNHH had equal numbers of samples from organ transplant and HIV-infected patients. Of 109 positive samples, 76 were from organ transplant recipients, 25 were from HIV-infected patients, 8 were from other or unknown CMV risk groups, and none were from bone marrow transplant recipients.

Comparison of culture and antigenemia.

As shown in Table 1, CMV antigenemia detected significantly more positives than either shell vial or tube culture at all four sites. Cultures detected only 31% and the antigenemia method detected only 97% of the positives. Across all trial sites there were only three culture-positive, antigenemia-negative samples: two from NSUH and one from IBMC. Conventional culture detected 27 of 54 positives (50%) inoculated into tubes, but the shell vial method detected only 7 of 55 positives (13%) tested.

TABLE 1.

Comparison of CMV culture methods and CMV antigenemia

Trial site	Total no. of specimens tested	No. tube culture positive	No. shell vial positive	No. antigenemia positivea	Total no. positiveb
UPMC	180	NDc	0	30	30
NSUH	130	6	ND	11	12
YNHH	102	21	ND	42	42
IBMC	101	ND	7	23	25
Total	513	27	7	106	109d

^aAntigenemia positive by either or both kits. 

^bTotal specimens testing positive by any test. 

^cND, not done. 

^dAcross all trial sites, only three specimens were positive by culture and negative by antigenemia. 

Comparison of Biotest and Bartels antigenemia test kits at each test site.

In Table 2, the performance of the Biotest and Bartels kits at each site is shown. At UPMC and YNHH more positive samples were detected with the Biotest kit, at IBMC more positive specimens were detected with the Bartels kit, and at NSUH equal numbers of positive specimens were detected with both kits. When Wilcoxon signed rank analysis was applied to the positive cell counts, no statistical difference was seen between the two kits for data from UPMC, NSUH, or YNHH (P = 0.721, 0.834, and 0.103, respectively). The number of antigenemia method-positive cells was significantly greater for the Bartels kit at IBMC (P = 0.002), and when data from all four sites were combined and analyzed together the Bartels kit showed significantly higher counts (P = 0.02).

TABLE 2.

Comparison of Biotest and Bartels kits for positive antigenemia (AG) results

Trial site	Total no. CMV AG-positive samples	Biotest kit positivea		Bartels kit positiveb		Pd
		No. of specimens	No. of positive cellsc (median [range])	No. of specimens	No. of positive cellsc (median [range])
UPMC	30	25	5 (1–287)	23	7 (1–90)	0.721
NSUH	11	9	2 (1–>100)	9	2 (1–>100)	0.834
YNHH	42	40	8 (1–1,421)	33	10 (1–1,760)	0.103
IBMC	23	19	13 (1–178)	21	43 (1–673)	0.002
Total	106	93	3 (1–1,421)	86	7 (1–1,760)	0.020

^aPositive cell count per 150,000 leukocytes. 

^bPositive cell count per 200,000 leukocytes. 

^cOne positive cell per two slides given as one cell. 

^dAs determined by Wilcoxon signed rank test. 

Analysis of discrepant antigenemia results.

Of 106 antigenemia-positive samples, 33 were positive by only one of the two antigenemia test kits. As shown in Table 3, all of these samples had low positive cell counts (median, 1; range, 1 to 7). CMV infection was confirmed for 27 of these 33 samples from patient histories and other laboratory tests. For four samples found to be positive by the Biotest kit only, insufficient information was available, and these samples were eliminated from the final analysis in Table 4. Three samples were classified as false positives, since all other laboratory tests for CMV were negative, and there were no clinical signs or symptoms of infection. Two of these were positive by the Biotest kit, and one was positive by the Bartels kit. Additionally, the three specimens that were positive by culture but negative by antigenemia were classified as false negatives for both kits.

TABLE 3.

Analysis of discrepant CMV antigenemia results^a

Trial site	No. of samples positive by both kits	Biotest kit positive onlyb		Bartels kit positive onlyc
		No. of specimens	No. of positive cells (median [range])	No. of specimens	No. of positive cells (median [range])
UPMC	18	7	1 (1–2)	5	1 (1–7)
NSUH	7	2	1.5 (1–2)	2	1.5 (1–2)
YNHH	31	9	1 (1–2)	2	1 (1)
IBMC	17	2	1.5 (1–2)	4	2.5 (1–4)
Total	73	20	1 (1–2)	13	1 (1–7)

^aDefined as positive by one kit only. 

^bCount per 150,000 leukocytes. 

^cCount per 200,000 leukocytes. 

TABLE 4.

Performance characteristics of Biotest and Bartels antigenemia test kits

Kit	No. of samples				% Sensitivityc	% Specificityd	% PPVe	% NPVf
	True positive	True negative	False positivea	False negativeb
Biotest	87	405	2	15	85.3	99.5	97.8	96.4
Bartels	85	406	1	17	83.3	99.8	98.8	96.0

^aNo other CMV-positive samples and absence of CMV disease. 

^bIncludes two specimens from IBMC and one specimen from NSUH that were CMV positive by culture but negative by both antigenemia kits. 

^cNo significant difference between the two kits: χ²_{(1 df)} = 0.148, P = 0.700. 

^dNo significant difference between the two kits: exact test (two-sided), P = 0.376. 

^eNo significant difference between the two kits: exact test (two-sided), P = 0.384. PPV, positive predictive value. 

^fNo significant difference between the two kits: χ²_{(1 df)} = 0.116, P = 0.734. NPV, negative predictive value. 

Performance characteristics of Biotest and Bartels CMV antigenemia kits.

The corrected sensitivity, specificity, and positive and negative predictive values for both kits are given in Table 4. There were no statistical differences between the two kits for any of these parameters.

DISCUSSION

CMV antigenemia has proved very useful in the rapid diagnosis of CMV infection and in monitoring response to therapy. The original antibodies developed were the C10/C11 pool first reported by The and colleagues in 1988 (12), and these are the antibodies used in the Biotest kit. Gerna et al. subsequently improved on the antigenemia technique by demonstrating the superior results obtained with formaldehyde fixation over methanol-acetone and immunofluorescence over immunoperoxidase staining (3). Gerna et al. also reported that a pool of three monoclonal antibodies to pp65 (2A6, 1C3, and 4C1) detected a higher number of positive cells than C10/C11, and showed a greater sensitivity in detecting low levels of viremia (<5 infected cells) (3).

The CINA antibody pool used in the Bartels/Argene kit contains one of the antibodies reported by Gerna, 1C3, and an additional antibody, AYM-1. St. George et al. (10) previously compared the 1C3 and C10/C11 antibodies, but not the complete kits, and reported better results with the 1C3 antibody including, as reported by Gerna et al. (3), improvement in the detection of low-level positive specimens. However, in the present study, the same laboratory at UPMC found no significant difference in sensitivity between the Biotest and Bartels kits.

Most discrepancies between the two kits were from samples with only one positive cell. Such very low-level positive specimens are generally only clinically significant in high-risk bone marrow transplant recipients (1) and possibly high-risk solid organ transplant recipients (4). Moreover, discrepancies at such a low level can easily be explained by sample-to-sample variation within a specimen.

Despite equivalent performance, some differences between the kits were evident. Most importantly, the number of PBLs examined was different. At two sites a greater total number of cells were examined with the Biotest kit (two slides of 150,000 cells each, or 300,000 total) than with the Bartels kit (one slide of 200,000 cells), while at the other two sites a greater number were examined with the Bartels kit (two slides of each at NSUH and one of each at UPMC). The number of positive cells reported, however, was given per 150,000 cells for the Biotest and per 200,000 for Bartels regardless of how many slides were examined. Since the denominator was 33% higher per slide for Bartels, the number of positive cells reported for Bartels also tended to be higher. The optimal number of PBLs to examine has not been carefully studied. Certainly, when a greater number of cells are examined, more low-level positives will be detected. Interestingly, the latest version of the Biotest kit recommends preparing cell spots for CMV antigenemia testing with 200,000 cells per slide. Although CMV end-organ disease is usually associated with higher numbers of CMV-positive cells (3, 7, 9), early detection of low-level antigenemia can be important in high-risk transplant recipients (1, 4).

The Bartels protocol was also different in that it recommended gentler manipulations of cells, utilized a weaker erythrocyte lysing buffer, prohibited vortexing, and required a refrigerated centrifuge. Whether these steps resulted in better preservation of cells was not specifically evaluated in the present study. Furthermore, the Evan's blue reagent as routinely diluted in the Bartels kit is more concentrated, raising the concern that weakly staining positive cells might be masked by the intense red counterstain. While the instructions provided with the kit suggested using a 1/100 dilution of Evan's blue, it was also stated that dilutions up to 1/500 could be used if desired. The laboratory that reported the best results with the Bartels kit, IBMC, used a more dilute Evan's blue, which might have contributed to their superior results. The Biotest kit provided more reagents at working dilutions, which saved technical time. Another difference was the time to sample processing, which was given as 6 to 8 h for Biotest and 5 h for Bartels. In the latter, the manufacturer's instructions also state that blood can be held at room temperature for up to 24 hours with rocking. The impact of such a processing delay on antigenemia results was not evaluated in this study (8).

Importantly, all laboratories had from 2 to 5 years of experience in CMV antigenemia testing. Thus, any interlaboratory differences in performance of the tests could not be attributed to inexperience. Two laboratories were experienced in use of the Biotest kit, and two laboratories were experienced in use of the Argene/Biosoft antibodies. The only laboratory that had a statistically better performance for the Bartels kit, IBMC, actually used the Biotest kit routinely.

It should be noted that although three antigenemia-positive samples were designated as false positives based on lack of other CMV-positive tests or evidence of CMV disease, these samples were likely to be true positives from cases of low-level asymptomatic CMV infection or reactivation. Many immunocompromised hosts may have transient low-level antigenemia and in the face of fever or bacterial sepsis, CMV antigenemia may occur (6).

Across all the laboratories, CMV antigenemia was much more sensitive than culture. Shell vials, which detected only 7 of 55 positives tested, were much less sensitive than conventional culture, which detected 27 of 54 positives. Only three samples were positive by culture only, whereas 75 samples were positive by antigenemia only. Culture-negative antigenemia-positive samples are most commonly from patients on antiviral therapy and/or patients who have low-level antigenemia (7, 11). Since the comparison of two antigenemia kits in this study resulted in more slides examined per sample, it could be argued that the antigenemia results were artificially high. However, at the time this study was performed, the Biotest kit had already been approved by the Food and Drug Administration for the diagnosis of CMV infection. If only the results of the Biotest kit are considered for comparison with culture, antigenemia detected 93 of 96 (97%), conventional culture detected 27 of 50 (54%), and the shell vial method detected 7 of 46 (15%) positives tested. These results are not significantly different from the combined data from both kits.

In summary, the Biotest and the Bartels CMV antigenemia kits were essentially equivalent, showing no statistical difference in either sensitivity, specificity, or positive or negative predictive values. Thus, both kits can be highly recommended for clinical use. The statistical analyses on individual site data and collective data highlight the necessity for large sample sizes, and preferably multiple trial sites, for these kinds of kit comparisons.