Performance of 4 Point‐of‐Care Screening Tests for Feline Leukemia Virus and Feline Immunodeficiency Virus

Abstract

Background

More than 3 million cats in the United States are infected with FeLV or FIV. The cornerstone of control is identification and segregation of infected cats.

Hypothesis/Objectives

To compare test performance with well‐characterized clinical samples of currently available FeLV antigen/FIV antibody combination test kits.

Animals

Surplus serum and plasma from diagnostic samples submitted by animal shelters, diagnostic laboratories, veterinary clinics, and cat research colonies. None of the cats had been vaccinated against FIV. The final sample set included 146 FeLV+, 154 FeLV−, 94 FIV+, and 97 FIV− samples.

Methods

Prospective, blind comparison to a gold standard: Samples were evaluated in 4 different point‐of‐care tests by ELISA antigen plate tests (FeLV) and virus isolation (FIV) as the reference standards. All test results were visually read by 2 blinded observers.

Results

Sensitivity and specificity, respectively, for FeLV were SNAP ^® (100%/100%), WITNESS ^® (89.0%/95.5%), Anigen^® (91.8%/95.5%), and VetScan^® (85.6%/85.7%). Sensitivity and specificity for FIV were SNAP ^® (97.9%/99.0%), WITNESS ^® (94.7%/100%), Anigen^® (96.8%/99.0%), and VetScan^® (91.5%/99.0%).

Conclusions and Clinical Importance

The SNAP ^® test had the best performance for FeLV, but there were no significant differences for FIV. In typical cat populations with seroprevalence of 1–5%, a majority of positive results reported by most point‐of‐care test devices would be false‐positives. This could result in unnecessary segregation or even euthanasia.

Abbreviations

CI

confidence interval

ELISA

enzyme‐linked immunosorbent assay

FeLV

feline leukemia virus

FIV

feline immunodeficiency virus

IFA

immunofluorescence assay

NPV

negative predictive value

PPV

positive predictive value

More than 3% of cats in the United States are infected with FeLV or FIV, contagious retroviruses that predispose to bone marrow suppression, cancer, chronic inflammatory conditions, immune dysfunction, and wasting syndromes.1 This equates to more than 3 million cats affected by these preventable viruses and at risk for disease and premature death. Although vaccination plays a role in prevention, the cornerstone of control is identification and segregation of infected cats.2

The American Association of Feline Practitioners recommends screening all cats for infection at the time they are first acquired, before initial vaccination against FeLV or FIV, after potential exposure to infected cats, and during illness.2 Point‐of‐care screening tests for FeLV antigen and FIV antibodies using whole blood, serum, or plasma are commonly used for diagnosis in veterinary clinics and animal shelters.

It has been recommended that positive screening tests be followed up with confirmatory testing, such as PCR, IFA, Western blot, or different point‐of‐care tests.2 However, what the most accurate confirmatory testing protocols are is a topic of debate fueled by a lack of evidence, emerging information about the pathobiology of both viruses, and inaccuracies intrinsic to each of the available testing modalities. The costs, logistics, and uncertainties inherent in confirmatory testing result in poor compliance with confirmation guidelines. As a result, diagnosis often rests solely on a single point‐of‐care screening test. Although the AAFP recommends against routine culling of infected cats, it is not uncommon for cats to be euthanized as a result of a positive screening test, regardless of their clinical condition. Underutilization of confirmatory tests and life or death decisions based on a single test result make the use of point‐of‐care screening tests a high‐stakes event.

Recently, new point‐of‐care screening tests have become available in the United States, but independent comparison of test performance with well‐characterized clinical samples is needed. The purpose of this study was to determine the diagnostic performance of currently available FeLV antigen/FIV antibody combination test kits.

Materials and Methods

Sample Collection and Characterization

The sample pool for this study was compiled from surplus sample volumes left over from diagnostic samples submitted by animal shelters, veterinary clinics, and cat research colonies. Different sample sets were used for the FeLV and FIV testing components of this study for the purpose of enriching the sample set with positive samples to increase statistical power. For FIV testing, whole‐blood samples were submitted by veterinary clinics (n = 82), cat research colonies (n = 90), and animal shelters (n = 19), in EDTA and refrigerated until centrifuged for harvesting of peripheral blood mononuclear cells for FIV culture within 24 hours after collection. After centrifugation, plasma was stored at −80°C. For FeLV testing, plasma (n = 98) and serum (n = 202) samples submitted by veterinary clinics (n = 224) or animal shelters (n = 76) were refrigerated for up to 1 week at the point of collection before freezing at −20°C. All samples were coded to assure blinding of the test observers. Samples were thawed at room temperature on the day the tests were performed. The use of surplus samples was approved by the University of Florida IACUC.

The true status of samples tested for FeLV antigen was established through the use of 2 different microtiter plate ELISAs for the detection of FeLV p27 antigen, and only those samples giving concordant results on both tests were included in this study. ViraCHEK^®/FeLV¹ was performed according to the manufacturer's instructions. The manufacturer reports this test has a sensitivity of ≥94.9% and a specificity of ≥98.4% defined by combined results of other p27 antigen test brands and virus isolation. PetChek^® FeLV 15 ELISA² was performed with sequential screening and confirmatory protocols, including a neutralization step to reduce nonspecific reactivity as described.3 The sensitivity and specificity of this modified assay were both reported to be 100% defined by the combined results of FeLV PCR and ViraCHEK^®/FeLV.3 A total of 10 samples were discarded because they were positive on PetChek^® and negative on ViraCHEK^®. The final sample set included 146 FeLV antigen‐positive samples and 154 antigen‐negative samples.

The true status of samples tested for FIV antibodies was determined by virus isolation using peripheral blood mononuclear cells as described.4 In addition, the viral subtype was determined for a subset of positive samples by sequence analysis of PCR products as described, to assure a spectrum of FIV types.4 Medical histories from pet owners and research laboratories indicated that none of the samples came from cats that had been vaccinated against FIV. The final sample set included 94 FIV antibody‐positive samples (28 subtype A, 31 subtype B, 6 subtype C, 5 subtype A/B, 1 A/C, 23 unknown subtype) and 97 FIV antibody‐negative samples

Test Performance

Four different commonly used point‐of‐care screening tests for FeLV antigen and FIV antibodies were selected for this study: SNAP^® Combo FeLV Ag/FIV Ab Test³ (bidirectional flow ELISA), WITNESS^® FeLV‐FIV Test Kit⁴ (lateral flow immunochromatography with colloidal gold), Anigen^® Rapid FIV Ab/FeLV Ag Test Kit⁵ (lateral flow immunochromatography with colloidal gold), and VetScan^® Feline FeLV/FIV Rapid Test⁶ (lateral flow immunochromatography with colloidal gold). All point‐of‐care tests were performed according to their manufacturers’ instructions. All 4 tests were performed simultaneously in small batches of 6 randomized samples at a time to assure equivalent testing conditions among the different tests. Test devices were visually assessed by 2 independent observers who were blinded to the true status of the samples and to the other observer's interpretation. In situations in which the observers disagreed, a third observer provided an additional interpretation and the test result was assigned the outcome selected by two of 3 observers. All completed test devices were photographed for a permanent record.

Statistical Analysis

A power analysis determined that detection of a 10% difference in test performance at P < .05 and power of 0.8 would require minimum group sizes of 93;⁷ all of the groups exceeded this size. Sensitivity for each test device was calculated by the formula: number of true positives divided by the number of true positives plus false negatives. Specificity for each test device was calculated by the formula: number of true negatives divided by the number of true negatives plus false positives. Confidence intervals and predictive values were calculated,⁸ and the Fisher's exact test was used to determine significant differences between tests.⁹

Results

All of the test devices provided valid results indicated by color development at the positive control marker. Among FeLV tests, there were 6 instances (1 SNAP^®, 3 WITNESS^®, 2 VetScan^®) in which the 2 observers disagreed on the result, and the opinion of the 3rd observer was recorded. Among FIV tests, there were 4 instances (2 WITNESS^®, 2 VetScan^®) in which the 2 observers disagreed on the result, and the opinion of the 3rd observer was recorded.¹⁰

SNAP^® was significantly more sensitive and specific than the 3 other tests for detection of FeLV antigen and correctly identified the status of all 300 samples (Table 1). WITNESS^® and Anigen^® were less specific than SNAP^®, but more specific than VetScan^®. False‐negative results occurred in 16 WITNESS^®, 12 Anigen^®, and 21 VetScan^® samples. False‐positive results occurred in 7 WITNESS^®, 7 Anigen^®, and 22 VetScan^® samples.

Table 1.

Sensitivity and specificity of 4 point‐of‐care tests for FeLV antigen and FIV antibodies in cats

	SNAP®	WITNESS®	Anigen®	VetScan®
FeLV Sensitivity (95% CI)	100%a (96.9–100)	89.0% (82.8–93.2)	91.8% (86.1–95.4)	85.6% (78.9–90.5)
FeLV Specificity (95% CI)	100%b (97.1–100)	95.5%c (90.8–98.0)	95.5%c (90.8–98.0)	85.7% (79.3–90.5)
FIV Sensitivity (95% CI)	97.9% (92.1–99.9)	94.7% (87.9–98.0)	96.8% (90.6–99.3)	91.5% (83.9–95.8)
FIV Specificity (95% CI)	99.0% (93.8–100)	100% (95.4–100)	99.0% (93.8–100)	99.0% (93.8–100)

^aSNAP^® is significantly more sensitive for the detection of FeLV antigen than the 3 other tests (P < .001).

^bSNAP^® is significantly more specific for the detection of FeLV antigen than WITNESS^® (P = .02), Anigen^® (P = .02), and VetScan^® (P < .001).

^cWITNESS^® and Anigen^® are significantly more specific for the detection of FeLV antigen than VetScan^® (P = .005).

There were no significant differences in test performance for detection of FIV antibodies (Table 1). False‐negative results occurred in 2 SNAP^®, 5 WITNESS^®, 3 Anigen^®, and 8 VetScan^® samples. False‐positive results occurred in 1 SNAP^®, 0 WITNESS^®, 1 Anigen^®, and 1 VetScan^® sample.

The predictive values for different hypothetical infection prevalences are shown for each test in Table 2. For FeLV, with the exception of SNAP^®, 83–94% of positive test results from a hypothetical low‐risk population of cats with a 1% seroprevalence for FeLV would be predicted to be false‐positives. Diagnostic accuracy of these tests remained low even in a higher risk theoretic cat population with 5% FeLV prevalence, in which 49–76% of positive results would be predicted to be erroneous. The predictive value of a positive test for FIV was better, but, with the exception of WITNESS^®, approximately half of positive results would be predicted to be erroneous in a hypothetical low‐risk population of cats with 1% FIV seroprevalence. The predictive value of negative tests for both FeLV and FIV was high and declined only slightly when theoretic seroprevalence reached 25% or greater.

Table 2.

Calculated positive and negative predictive values for 4 point‐of‐care screening tests in hypothetical populations of cats with low risk (1% prevalence) to high risk (50% prevalence) of FeLV or FIV infection

	1% prevalence		5% prevalence		10% prevalence		25% prevalence		50% prevalence
	PPV	NPV	PPV	NPV	PPV	NPV	PPV	NPV	PPV	NPV
FeLV antigen test
SNAP®	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%
WITNESS®	17%	100%	51%	99%	69%	99%	87%	96%	95%	90%
Anigen®	17%	100%	52%	100%	69%	99%	87%	97%	95%	92%
VetScan®	6%	100%	24%	99%	40%	98%	67%	95%	86%	86%
FIV antibody test
SNAP®	50%	100%	84%	100%	92%	100%	97%	99%	99%	98%
WITNESS®	100%	100%	100%	100%	100%	99%	100%	98%	100%	95%
Anigen®	49%	100%	84%	100%	92%	100%	97%	99%	99%	97%
VetScan®	48%	100%	83%	100%	91%	99%	97%	97%	99%	92%

PPV, Positive predictive value; NPV, Negative predictive value.

Discussion

The SNAP^® test had the overall best performance for diagnosis of FeLV, but there were no significant differences in performance for diagnosis of FIV. For some tests, most positive results obtained by testing low‐risk populations would be false‐positives. This could trigger inappropriate decisions to isolate or euthanize cats or create a need to perform additional confirmatory testing.

It is difficult to compare the results of the current study with previously published results or with data provided by manufacturers because of differences in study design, particularly in the rigor with which reference standards are established. In addition, tests with similar names may have proprietary differences in various countries or may undergo design changes over time, making it difficult to know which historical reports are appropriate for comparison.

Establishing the true retroviral infection status of cats can be difficult, even when exhaustive confirmatory testing is attempted. Virus isolation is commonly used as a reference standard in research, but is not routinely available for clinical samples. In addition, virus isolation can be negative in some infected cats because of low levels of circulating virus, inhibitory factors, and inactivation after sample collection, processing, and storage.5 PCR is also commonly used as a reference standard. Although PCR is more widely available in commercial diagnostic laboratories than virus isolation, some infections are missed, possibly because of viral sequence variation and low copy numbers in circulation.^{9 ,} 5, 6, 7, 8 In addition, accuracy of PCR varies among different diagnostic laboratories.9 Because of these issues, there are no universally accepted or perfect “gold standard” tests for retroviral infections in cats.

Diagnosis of FeLV is complicated by regressive infection, an incompletely characterized but common condition in which infected cats cease producing productive virus and circulating p27 antigen, but still carry provirus detectable by PCR.10, 11, 12 These cats initially test positive for soluble p27 antigen and then revert to antigen‐negative status. Small studies of laboratory cats suggest that cats with regressive infection are likely to remain in this status for their lifetime, do not shed infectious virus, and are less likely to develop FeLV‐associated diseases.12, 13 In contrast, cats with progressive infection remain persistently antigen‐positive, are infectious to other cats, and are more likely to suffer from clinical conditions and a reduced life span. Even though cats with regressive infection have negative screening tests, they can still transmit infection via blood transfusions14 and may occasionally relapse into a productive infection and illness.12, 13

This emerging understanding of FeLV pathogenesis challenges traditional definitions of FeLV infection, simplistic diagnostic protocols, and criteria for defining a diagnostic gold standard. The selection of the most appropriate confirmatory tests and reference standards depends to some degree on the clinical importance assigned to the outcome. For most pet situations, identification of cats with progressive FeLV infection by detection of circulating p27 over time is a priority because of their risk of transmission to other cats and of developing disease. Blood donor cats should be tested by PCR in addition to reduce the potential for transfusion‐related transmission of FeLV from cats with regressive infections.14, 15 However, it could be argued that ideal screening tests would not report the common condition of regressive infection as FeLV‐positive, because such cats are unlikely to pose a threat to other cats or to develop disease.

Because the purpose of this study was to determine the performance of point‐of‐care screening tests for detecting clinically more important productive infections, concordant p27 detection by 2 different microtiter ELISA tests was used as a reference standard to define an unambiguous collection of positive and negative samples. The only other test capable of detecting circulating p27 antigen, the immunofluorescent antibody test (IFA), is notably less sensitive than other diagnostic tests.16 PCR detects not only progressive but also regressive infections, and thus, results are not comparable.17 Virus culture for FeLV must be performed within a short time after sample collection and has been associated with a higher false‐negative rate than for FIV culture.6, 18, 19

In previous studies using virus isolation as the reference standard for detection of FeLV, the sensitivity and specificity of SNAP^® were reported to be 91.3 and 98.218 and 92.3 and 97.3%,19 respectively. The sensitivity and specificity of WITNESS^® were previously reported to be 94.5 and 99.419 and 66.6 and 98.7%.18 The WITNESS^® test was produced by a different company at the time of the previous studies than at the time of the current study. In a study using PCR as the reference standard for FeLV, the sensitivity and specificity reported for SNAP^® were 53.3 and 100% and for Anigen^® were 40.0 and 100%, respectively.17 The high rate of “false negatives” reported by the point‐of‐care tests for FeLV can be attributed to PCR detection of regressive FeLV infections that failed to produce detectable circulating p27 antigens. No previously published independent studies reporting the performance of VetScan^® were identified.

Diagnosis of FIV infection is more straightforward than diagnosis of FeLV infection, because it is believed that cats remain infected for life. Most infected cats produce anti‐FIV antibodies, which form the basis for current point‐of‐care screening tests.7, 8 Diagnosis of FIV is complicated in regions in which FIV vaccines are used, because some tests cannot discriminate between antibodies induced by infection and those induced by vaccination.^{11 ,} 20 In this study, virus isolation with detection of FIV antigen in cell culture supernatants was used as the reference standard to determine the true status of samples in the study.

In studies using Western blot as the reference test for detection of FIV antibodies, the sensitivity and specificity of SNAP^® were reported to be 88.9 and 100%,17 100 and 99.6%,19 and 86.1 and 98.6%,18 respectively. The sensitivity and specificity of WITNESS^® were reported to be 95.5 and 99.7%,18 and 94.5 and 99.4%,19 and for Anigen^® to be 88.9 and 99.7%.17

When tests lack sensitivity, infected cats may escape detection and remain at risk for infecting other cats. When tests lack specificity, uninfected cats may be unnecessarily segregated or even euthanized. Even small differences in test performance can result in a large number of diagnostic mistakes when the prevalence of infection is low. Prevalence for FeLV and FIV in the United States can be as low as 1% for healthy young cats living indoors and increases with risk factors including advancing age, male sex, lack of neutering, outdoor exposure, illness, and animal hoarding conditions.1, 21 Infection rates have been documented to reach as high as 15% for FeLV and 23% for FIV in free‐roaming adult male cats with bite wounds.22

This study has several limitations, the chief being that a true “gold standard” does not exist for confidently defining the true infection status of cats. Virus culture is the most accurate assay for FIV, but can yield false‐negative results if the sample is mishandled before processing. Virus culture is often used as the reference standard for FeLV as well, but is more problematic.17 In 1 study, 25–30% of samples that were positive for FeLV antigen by point‐of‐care tests were negative by virus culture, but when a subset of the samples with discordant results were tested by PCR, 86% of the culture‐negative samples were positive.6 This indicates that the cats were probably infected, and that at least some of the virus culture results were false‐negatives, a concern that has been echoed in other reports.18, 19 We attempted to mitigate this issue by using 2 different plate ELISA tests, and excluding any samples that were not positive in both assays. One of the plate ELISAs is validated against FeLV PCR and incorporates both a screening protocol and a confirmatory neutralization protocol for positive samples.3 It is conceivable that using a combination of 2 ELISAs as the reference standard could provide an advantage to the only point‐of‐care test that utilized ELISA technology (SNAP^®) for p27 detection over those that used immunochromatography with colloidal gold (WITNESS^®, Anigen^®, VetScan^®). However, there are no alternatives for accurate identification of productive FeLV infections. Another limitation of this study is that only plasma and serum were used, that samples were shipped to a testing center, and that samples were stored frozen before testing. In practice, point‐of‐care tests are usually performed on‐site shortly after sample collection and often with anticoagulated whole blood. In this study, samples were collected in a single location and tested in batches under standardized conditions as was the case in similar studies.18, 19 The impact of sample shipping, freezing, thawing, and storage on test performance is unknown. Samples were not selected in a random manner, but instead were collected as excess volumes of diagnostic samples collected for other purposes. The proportion of positive samples in this study, approximately 50%, does not reflect common prevalence rates, but was selected to increase the statistical power and to narrow the confidence intervals to detect subtle differences between the point‐of‐care tests. The predictive values of each test at various different population prevalences are provided in Table 2. Finally, this study excluded cats with a history of FIV vaccination, which is known to cause false‐positive results by some tests.20, 23

Because most infection prevalences for FeLV and FIV generally fall below 25%, the accuracy (predictive value) of any 1 test is most affected by the specificity of the test system. In high‐stakes testing, point‐of‐care screening test kits for FeLV and FIV should be selected for both high sensitivity and specificity. Positive test results, especially for cats in low‐risk categories, should be confirmed with alternative testing modalities, such as a different type of point‐of‐care test, PCR, or, in the case of FeLV, IFA. However, all available diagnostic tests for FeLV and FIV have some intrinsic level of inaccuracy, making it difficult to resolve discordant results between screening and confirmatory tests with certainty.