Results of Human Papillomavirus DNA Testing with the Hybrid Capture 2 Assay Are Reproducible

Abstract

Reproducibility of the Hybrid Capture 2 Test (HC 2) for human papillomavirus (HPV) DNA detection was evaluated by assaying frozen cervical specimens in 1997 and again in 2001 from 1,775 women with normal cervical cytology. Using a cutoff point of 1.0 pg of HPV DNA/ml between a negative and a positive test result, the result of the kappa test for agreement was 0.72 (a kappa value of >0.60 is considered good agreement). Using cutoff points of 1.0 and 10.0 pg/ml between negative and low positive and between low positive and high positive, respectively, the kappa was 0.68 and the linear-weighted kappa was 0.76. The results of this study indicate that HC 2 testing is reproducible even among cytologically normal women with low test values.

The Hybrid Capture 2 Test (HC 2) (Digene Corporation, Gaithersburg, Md.) (5) is a commercially available assay approved by the Food and Drug Administration for the DNA detection of human papillomavirus (HPV) infections, the primary cause of cervical cancer (1, 6, 11). HC 2 detects a group of 13 cancer-associated HPV types: 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, and 68. Previous studies have demonstrated that HC 2 is a sensitive and specific assay for detecting HPV DNA from cervical specimens (7), perhaps comparable to PCR assays (2), and that HPV DNA detection by HC 2 might be a useful triage for women with a community diagnosis of atypical squamous cells of undetermined significance (9). Understanding the reproducibility of assays based on testing different specimen aliquots or repeat testing over time is important in epidemiologic investigations, especially cohort studies, in which HC 2 is currently being used.

The present analysis compares HPV testing results for 1,775 specimens that were tested in 1997 (test 1) and in 2001 (test 2). The specimens were selected from a collection of 21,145 frozen cervicovaginal lavage specimens obtained from eligible, consenting women between 1989 and 1990 as part of enrollment in a cohort study examining the natural history of HPV and cervical cancer at Kaiser Permanente, Northwest (Portland, Oreg.) (4, 8). Lavages were performed by rinsing the cervical os with 10 ml of sterile physiologic saline using a syringe fitted with an intracatheter extender and then collecting the pooled fluid from the vaginal fornix using the same device (8).

Lavage specimens were refrigerated within 1 h of collection and shipped to a central laboratory for processing. A 1-ml aliquot was removed for HPV DNA testing by PCR as described elsewhere (4, 8). The remaining volume was split into two aliquots of approximately equal volume. Aliquots were either centrifuged at 400 × g for 5 min (4°C) or remained unprocessed, and the resulting cell pellets or unprocessed lavage aliquots were then frozen at −70°C.

A sample of 1,775 specimens from women in the incidence cohort (no known current or past cervical neoplasia) (4, 8) was chosen for testing. For test 1, cell pellets were used. For test 2, 1,733 of the 1,775 tests were performed on the second aliquot of the lavage stored either as a frozen cell pellet (n = 803) or as an unprocessed (liquid) lavage specimen (n = 930); 42 specimens were tested on the same lysed aliquot used in 1997 after one additional freeze-thaw cycle.

In test 1, cell pellets were thawed, suspended in 1 ml of specimen transport medium (STM; Digene), and mixed vigorously. Following incubation for 45 min at 65°C with 0.5 ml of the denaturation reagent, a 75-μl aliquot was removed for the HC 2 test, performed according to the package insert. We subsequently discovered that specimens stored for extended periods in the alkaline solution might be unstable for DNA quality. Thus, for long-term storage of DNA, the alkalinizing step was excluded from processing of aliquots for test 2. Unprocessed lavage aliquots were first pelleted, and then the freshly pelleted cells were treated similarly to aliquots stored as frozen cell pellets, both of which were resuspended in 1 ml of STM and incubated at ambient temperature for 48 to 72 h. After incubation, the exfoliated cervical cells were lysed and a 100-μl aliquot of the cell lysate (instead of the entire lysed specimen) was denatured using 50 μl of denaturant in a 96-well plate on a plate heater for 45 min at 65°C. This volume was sufficient for two HC 2 tests.

Results were expressed in terms of relative light units of the specimen divided by the mean relative light units of three controls positive for HPV type 16 (HPV-16) DNA (1.0 pg/ml) to estimate the test value in picograms per milliliter. The estimated values in picograms per milliliter were significantly nonnormal even after log transformation (P < 0.001). Therefore, to compare the nonnormal continuous in picograms-per-milliliter values, the Spearman (rank) correlation coefficient was determined for all (untransformed) values. Also, because of the preponderance of HPV DNA negative specimens with test values below the level of assay accuracy, Spearman correlation coefficients were computed for values above 1.0 pg/ml.

Kappa values (3, 10) (which test for agreement while taking chance agreement into account) were calculated based on a cutoff of 1.0 pg/ml (∼5,000 genomic equivalents of HPV DNA) between a negative and a positive test result, which was previously demonstrated to optimize assay performance (7). With the addition of a second cutoff point at 10.0 pg/ml, data were also categorized as negative (<0.995 pg/ml), low positive (1 to 9.995 pg/ml), and high positive (≥9.995 pg/ml) and evaluated using both kappa and linearly weighted kappa statistics, which linearly weight disagreement inversely proportional to the number of categories of difference between the two assays (3, 10) (i.e., exact agreement is given a weight of 1, a difference of one category is given a weight of 0.5, and a difference of two categories is given a weight of 0).

All HPV testing was performed independently, with previous test results masked. In test 1, a subset of 200 randomly chosen specimens was assayed in duplicate in a masked fashion for quality control purposes. The overall Spearman coefficient of the picograms-per-milliliter values for the duplicate testing was 0.66; the Spearman coefficient among those specimens with values above the 1.0-pg/ml positive control cutoff point was 0.98. For a negative-versus-positive test based on the 1.0-pg/ml cutoff point, there was 97% exact agreement and a kappa of 0.87 (95% confidence interval [CI] = 0.73 to 1.0) for the duplicates.

Of the specimens tested, there were 1,734 specimens with valid test results at both testing periods; 5 and 36 specimens had invalid or missing results for test 1 and test 2, respectively. For test 1, the median was 0.3 pg/ml, the mean was 23.7 pg/ml, the standard deviation was 134.3 pg/ml, and the range was 0.1 to 2,223 pg/ml. For test 2, the median was 0.4 pg/ml, the mean was 18.2 pg/ml, the standard deviation was 105.3 pg/ml, and the range was 0.01 to 1,956 pg/ml.

A comparison of the picograms-per-milliliter values for test 1 and test 2 is shown in Fig. 1. The overall Spearman correlation coefficient of the picograms-per-milliliter values was 0.45; the Spearman coefficient among those specimens with values above the 1.0-pg/ml positive control cutoff was 0.82. For the negative-versus-positive test (Table 1), there was 92% exact agreement and kappa was 0.72 (95% CI = 0.67 to 0.76) for the comparison of test 1 and test 2. For the results of the test with negative, low-positive, and high-positive cutoff points (Table 2), kappa was 0.68 (95% CI = 0.63 to 0.72) and the linearly weighted kappa was 0.76 (95% CI = 0.73 to 0.80). There were no differences in the reproducibility of HC 2 when the results were stratified by storage conditions (lysate, frozen pellet, or unprocessed lavage) of specimens used in test 2, and therefore only the combined data are reported.

FIG. 1.

Plot of test 1 (1997) results versus test 2 (2001) results.

TABLE 1.

Results of HC 2 testing on specimens tested in test 1 (1997) and in test 2 (2001) using a 1.0-pg/ml cutoff between negative and positive results

Result by test 1	No. of specimens with indicated result by test 2		Total no. for row
	Negative	Positive
Negative	1,360	71	1,431
Positive	71	232	303
Total	1,431	303	1,734

TABLE 2.

Results of HC 2 testing on specimens tested in test 1 (1997) and in test 2 (2001) using a 1.0-pg/ml cutoff between negative and positive and a 10.0-pg/ml cutoff between low positive and high positive

Result by test 1	No. of specimens with indicated result by test 2			Total no. for row
	Negative	Low positive	High positive
Negative	1,360	63	8	1,431
Low positive	61	66	13	140
High positive	10	16	137	163
Total	1,431	145	158	1,734

These results suggest that HC 2 testing is reproducible and robust with a 4-year interval between tests. Although there were apparent slight decreases in the average test values (e.g., mean and range) in test 2 compared to test 1, these differences were nonsignificant. Furthermore, the discordance between the tests was symmetrical (McNemar test, P = 1.0, for dichotomous outcome; symmetry χ² test, P = 0.90, for trichotomous outcome), suggesting that there was no bias at one time point versus the other. Of the 142 discordant tests (of 1,734 tests, or 8%), 124 were negative in one test and gave a low-positive result in the other. Therefore, most disagreements were not extreme, as reflected by the weighted kappa being greater than the unweighted statistic. Given that these women were mostly without disease and therefore had a low average viral load, the crude method of dividing of the split specimen employed in this study into aliquots (which likely resulted in variability in cell numbers between pellets), the different specimen preparation used in test 2, and the different cell processing methods for the different HC 2 testing periods, it is perhaps not surprising that a small percentage of tests fluctuated between a negative and a low-positive test result. For 92 of 124 (74%) borderline specimens, HPV DNA testing was also performed by PCR with L1 consensus primers MY09/11 (4); 68% of these specimens were HPV DNA negative by PCR for the same HPV types tested for by HC 2, suggesting that most of these specimens might truly be HPV DNA negative or have very low viral loads.

In summary, HC 2 demonstrated good reproducibility despite testing among women who often had viral loads near the minimum threshold of accurate detection. Our findings suggest that HC 2 can be used on archival specimens for either epidemiologic studies or for retrospective testing in the clinical management of equivocal cytology such as atypical squamous cells of undetermined significance (9), where higher viral loads in these cervical specimens may result in even better assay performance.