Performance of Five Serological Assays for Diagnosis of Rhodococcus equi Pneumonia in Foals

Abstract

The performance of four enzyme-linked immunosorbent assays (ELISAs) (ELISA-6939, ELISA-33701, ELISA-VapA, and ELISA-California) and an agar gel immunodiffusion test for diagnosis of Rhodococcus equi pneumonia in foals was evaluated. Antibody concentrations of foals with culture-confirmed R. equi pneumonia (n = 41) were compared to those of age-matched pasturemates that remained clinically healthy during the entire breeding season (n = 24). For each serological assay evaluated, selection of a low cutoff resulted in high sensitivity but low specificity. Increasing the cutoff value resulted in better specificity but to the detriment of sensitivity. The best diagnostic performance was achieved with ELISA-California at a cutoff of 40%, resulting in a sensitivity of 59% and a specificity of 88%. We conclude that current serological assays do not differentiate between diseased and clinically healthy foals.

Rhodococcus equi, a gram-positive facultative intracellular pathogen, is the most devastating cause of pneumonia in foals between 3 weeks and 5 months of age. Other less common clinical manifestations of R. equi infections in foals include ulcerative enterocolitis, colonic or mesenteric lymphadenopathy, immune-mediated synovitis and uveitis, osteomyelitis, and septic arthritis (9). R. equi, a saprophytic inhabitant of soil, is widespread in the environment of horse-breeding farms. Although all horse farms are likely infected to various degrees, the clinical disease is unrecognized or sporadic on some farms but enzootic and devastating on others, with morbidity rates sometimes exceeding 40% (15). On farms where the disease is enzootic, costs associated with veterinary care, early diagnosis, long-term therapy, and mortality of foals may be very high. In addition to significant immediate costs, R. equi pneumonia has a long-term detrimental effect on the equine industry, because foals that recover from the disease are less likely to race as adults (1).

The farm-to-farm variation in the incidence of the disease likely reflects differences in environmental and management conditions as well as differences in the virulence of isolates. Unlike most environmental R. equi, isolates from pneumonic foals typically contain an 80- to 90-kb plasmid encoding a family of seven closely related virulence-associated proteins, designated VapA and VapC to VapH (4, 20, 23-26). Plasmid-cured derivatives of virulent R. equi strains lose their ability to replicate and survive in macrophages and fail to induce pneumonia in foals, confirming that the large plasmid is required for the virulence of R. equi (8, 27).

R. equi pneumonia of foals has traditionally been diagnosed based on culture or PCR amplification of the microorganism from a tracheobronchial aspirate. However, obtaining a tracheobronchial aspirate is not recommended for foals that present with severe respiratory distress, and the technique is not practical for routine diagnosis on large farms where the disease is enzootic and several foals must be sampled. There is therefore a need for a less invasive and more practical means of diagnosis. Although several assays detecting antibody to R. equi have been developed and are commonly used by practicing veterinarians for the diagnosis of R. equi infections, the diagnostic value of these assays has never been assessed in a clinical setting of heavy natural challenge. The objective of this study was to assess the performance of four currently available enzyme-linked immunosorbent assays (ELISAs) and an agar gel immunodiffusion test for diagnosis of R. equi pneumonia in foals.

MATERIALS AND METHODS

Foals and sample collection.

Foals between 3 weeks and 6 months of age presented to the Veterinary Medical Teaching Hospital of the University of Florida for diagnostic evaluation of bronchopneumonia (n = 39) and foals born on a thoroughbred farm with a history of R. equi infections (n = 80) located in Marion County, Fla., were initially considered for inclusion in the study. Foals on the farm were monitored daily for clinical signs of illness by experienced farm personnel. Foals with clinical signs of bronchopneumonia such as cough, bilateral nasal discharge, tachypnea, or fever received a complete physical examination, including thorough auscultation of the lungs, by a veterinarian. In addition to receiving a complete physical examination, foals presented to the University of Florida were given lung radiographs and thoracic ultrasonography.

A tracheobronchial aspirate for bacterial culture and serum sample were obtained from each foal with clinical signs or radiographic or ultrasonographic evidence of pneumonia. For each foal diagnosed with bronchopneumonia, a serum sample was also obtained from the most closely age-matched clinically healthy pasturemate. Serum samples were frozen at −20°C until the end of the calendar year. Only serum samples from foals that remained clinically healthy throughout the entire breeding season were used as samples from clinically healthy foals (n = 24). If a foal bled as a healthy pasturemate subsequently developed respiratory, gastrointestinal, or joint disease, the sample was discarded.

Tracheobronchial aspiration was performed with the transtracheal approach or by passing a sterile, double-guarded aspiration catheter through the biopsy channel of a 1.2-m fiber optic endoscope, as described before (5). Bacteriologic culture of tracheobronchial aspirates obtained by use of this technique correlates closely with the results from the transtracheal approach (5). The tracheobronchial aspirate fluid was kept refrigerated until submitted to a local laboratory for aerobic bacteriologic culture between 5 min and 5 h after collection. Tracheobronchial aspirate fluid was inoculated onto standard bacteriologic culture media (Trypticase soy agar with 5% sheep blood and MacConkey agar) as well as on R equi-selective nalidixic acid-novobiocin-actidione (cycloheximide)-potassium tellurate medium (28). Bacterial pathogens were identified by use of standard identification procedures.

Serology by ELISA.

Anti-R. equi antibody concentrations were measured with four ELISAs (ELISA-6939, ELISA-33701, ELISA-VapA, and ELISA-California) as previously described (11, 16, 21, 22). Antigens used were Tween 20 extract from a virulent plasmid-containing (ELISA-33701) or an avirulent (ELISA-6939) R. equi strain, supernatant antigen from clinical isolates (ELISA-California), and recombinant VapA (VapA-ELISA). Results for ELISA-6939 and ELISA-33701 are reported as optical density values (21, 22). ELISA-California was performed by the California Animal Health and Food Safety Laboratory System (Davis, Calif.),and results are reported as the ratio of positive serum or test serum absorbance to the negative control absorbance value (11). Results of the ELISA-VapA are reported as the last serum dilution giving a reading twice the optical density of a negative control serum diluted 1:10 (14, 16). Serum samples were coded so that the health status and culture results of the foals were unknown to the laboratory personnel.

Agar gel immunodiffusion.

Fresh isolates of virulent R. equi were grown in dialysates of brain heart infusion broth for 30 days at 37°C. Bacteria were removed by high-speed centrifugation, and the resultant supernatant consisted of a concentrated composite of soluble R. equi antigens, including exoenzymes, cell wall components, capsular polysaccharides, and virulence-associated proteins, that served as a polyvalent antigen in the assay (6). Positive control serum samples were collected from naturally occurring microbiologically confirmed cases of R. equi-induced bronchopneumonia (3, 7).

Wells (4 mm) were punched into borate-buffered agar in plastic petri dishes. Serum samples (50 μl) from foals to be tested were added to wells arranged in hexagonal patterns around central wells that contained antigen (35 μl) so that they alternated with wells containing positive control serum (45 μl). Up to three serum samples could be tested per individual hexagonal pattern. The test wells were incubated in a humidified chamber at room temperature for 48 h, and development of precipitin lines was then assessed by viewing with indirect light against a dark background. The control serum samples produced up to four to five precipitin lines with the antigen, and the test serum samples formed lines of identity of varying intensity. Test serum samples were assessed as negative when the adjacent positive control precipitin line(s) passed directly into the test serum well.

A weakly positive reaction was defined as a line of identity of the test serum with the control serum line(s) that formed immediately in front of the test serum well. A positive reaction was defined as a line of identity of the test serum with the control serum line(s) that was roughly equidistant between the test serum and antigen wells. A strongly positive reaction was defined as a line of identity of the test serum with the adjacent positive control serum line(s) that formed closer to the antigen well than did the positive control serum line(s) (Fig. 1). Serum samples were coded so that the health status and culture results of the foals were unknown to the laboratory personnel.

FIG. 1.

Interpretation of agar gel immunodiffusion assay. AG, antigen well; SP, strongly positive; P, positive; WP, weakly positive; N, negative.

Data analysis.

The diagnostic performance of each ELISA was assessed by receiver-operating characteristic curve analysis (10). Culture of R. equi from a tracheobronchial aspirate in pneumonic foals was used as the reference standard test. Antibody concentrations of foals with R. equi pneumonia were compared to those of age-matched pasturemates that remained clinically healthy during the entire breeding season.

The dependence of the diagnostic sensitivity and specificity on selected cutoff values was considered for test evaluation and comparison. The area under the curve (AUC) is a summary statistic of overall diagnostic performance. Receiver-operating characteristic plots for diagnostic tests with perfect discrimination between negative and positive reference samples have an AUC of 1.00 (100% sensitivity and 100% specificity). One can distinguish between a noninformative (AUC = 0.50), less accurate (AUC = 0.50 to 0.70), moderately accurate (AUC = 0.71 to 0.90), and perfect (AUC = 1.00) test (10). Comparison of the AUC between ELISAs was conducted with computer software from MedCalc (Mariakerke, Belgium).

While sensitivity and specificity are fixed characteristics of a test, the predictive values of positive and negative tests will vary with the prevalence of the disease. To simulate the diagnostic performance of each assay, the predictive values of positive and negative test results were estimated based on prevalences of R. equi pneumonia of 10% and 40%. Comparison of ELISA results between foals with R. equi pneumonia and clinically healthy foals was done with the Mann-Whitney U test. Results were considered significant if the value of P was <0.05.

For the agar gel immunodiffusion assay, sensitivity, specificity, and predictive values were also calculated at each cutoff point (weakly positive, positive, and strongly positive) with culture of R. equi from a tracheobronchial aspirate as the reference standard test. Predictive values of positive and negative test results were also estimated based on prevalences of R. equi pneumonia of 10% and 40%.

RESULTS

R. equi was cultured from 17 of 39 (43.6%) pneumonic foals presented to the University of Florida and from 24 of 32 (75%) foals that developed clinical signs of bronchopneumonia on the study farm. Streptococcus equi subsp. zooepidemicus (n = 7) and a Klebsiella sp. (n = 1) were cultured from the eight pneumonic foals with negative R. equi cultures. A total of 41 foals classified as having R. equi pneumonia based on clinical signs and culture of the microorganism from a tracheobronchial aspirate were included in the analysis and compared to 24 age-matched foals that remained clinically healthy during the entire breeding season. The median age at time of diagnosis was 38 days (range, 28 to 94 days).

With receiver-operating characteristic analysis, the overall diagnostic performance of ELISA-California (AUC = 0.68; 95% confidence intervals = 0.56 to 0.79) was significantly higher (P = 0.03) than that of ELISA-VapA (AUC = 0.55; 95% confidence intervals = 0.44 to 0.65) for diagnosis of R. equi pneumonia (Table 1). Differences in the AUC of the receiver-operating characteristic analysis between other ELISAs were not statistically significant.

TABLE 1.

Area under the receiver-operating characteristic curve, sensitivity, specificity, and predictive values of five serological assays for the diagnosis of R. equi pneumonia in foals^a

Assay	Area under ROC curve (95% CI)	Selected cut-off	Sensitivity (%)	Specificity (%)	Predictive value (%)
					Prevalence 10%		Prevalence 40%
					Positive	Negative	Positive	Negative
ELISA-VapA	0.55a (0.44-0.65)	1:40	81	4	9	66	36	24
		1:80	71	13	8	79	36	40
		1:160	61	30	9	87	37	53
		1:320	56	50	12	41	43	64
		1:640	37	63	10	89	39	60
		1:1,280	24	84	14	90	50	62
		1:2,560	14	96	28	91	69	62
		1:5,120	10	86	7	89	31	59
ELISA-California	0.68b (0.56-0.79)	20%	90	8	10	88	40	56
		30%	83	46	15	97	50	46
		40%	59	88	34	95	76	76
		50%	34	92	31	93	73	68
		60%	7	92	10	90	37	60
ELISA-33701	0.67a,b (0.54-0.78)	0.2	66	4	7	53	31	16
		0.4	44	48	9	88	36	57
		0.7	15	65	4	87	31	53
		1.0	10	83	6	89	27	57
		1.2	5	83	3	89	16	56
ELISA-6939	0.67a,b (0.54-0.78)	0.2	78	61	18	96	58	80
		0.4	39	83	20	92	60	68
		0.7	17	91	18	91	57	62
		1.0	5	96	11	91	43	60
		1.2	5	100	100	90	100	61
Agar gel immunodiffusion	NA	Weak positive	61	58	13	93	49	69
		Positive	44	71	14	91	50	65
		Strong positive	10	92	12	90	45	60

^aWithin a column, assay results with different superscripts were significantly different (P < 0.05). NA, not applicable. CI, confidence interval.

For each serological assay evaluated, selection of a low cutoff resulted in high sensitivity but low specificity (Table 1). Increasing the cutoff value resulted in better specificity but at the detriment of sensitivity (Table 1). The best diagnostic performance was achieved with ELISA-California at a cutoff of 40%, resulting in a sensitivity of 59% and a specificity of 88% (Table 1). Antibody concentrations in foals with R. equi pneumonia were significantly higher than in healthy controls by ELISA-California (P = 0.01) and ELISA-6939 (P = 0.02) (Table 2). In contrast, antibody concentrations with ELISA-33701 were significantly lower (P = 0.02) in foals with R. equi pneumonia compared to healthy controls (Table 2).

TABLE 2.

Comparison of anti-R. equi antibody in foals with R. equi pneumonia and clinically healthy age-matched controls by various ELISAs^a

Assay	Anti-R. equi antibody
	R. equi pneumonia (n = 41)		Clinically healthy controls (n = 24)
	Median	Range	Median	Range
ELISA-VapA (titer)	640a	0-10,240	480a	40-10,240
ELISA-California (%)	43.2a	11.0-74.9	34.05b	19.4-87.3
ELISA-6939 (OD)	0.301a	0.096-1.544	0.154b	0.040-1.177
ELISA-33701 (OD)	0.288a	0.050-1.617	0.430b	0.067-2.47

^aDifferent superscripts within each row indicate significantly different antibody concentrations between foals with R. equi pneumonia and clinically healthy controls (P < 0.05). OD, optical density.

DISCUSSION

A serological assay that permitted differentiation between foals developing R. equi pneumonia and clinically healthy foals would be a considerable benefit on farms with enzootic R. equi infections. Early recognition of foals with R. equi pneumonia prior to development of clinical signs would likely reduce losses and limit the costs associated with long-term therapy of severely affected animals.

Before investigating the value of serology for early identification of foals with R. equi pneumonia on farms where the disease is enzootic, the first step was to determine if such an assay could differentiate foals with clinical R. equi pneumonia from apparently healthy animals. In the present study, it is likely that many foals used as clinically healthy controls may have had a subclinical infection with R. equi. In one study conducted on a farm with enzootic R. equi pneumonia, 77 (35%) of 216 foals sampled had tracheobronchial aspirate cultures positive for R. equi but no signs of respiratory disease (2). For this reason, bacteriologic culture or PCR amplification of R. equi from a tracheobronchial aspirate should always be interpreted in the context of clinical findings, and although they are considered gold standards for diagnosis in pneumonic animals, these assays should never be used as screening tests for clinically healthy animals.

Although anti-R. equi antibody concentrations in foals with R. equi pneumonia were significantly higher than those of healthy controls with ELISA-California and ELISA-6939, the extensive overlap in individual titers between groups indicated the limited value of these assays for diagnostic purposes (Table 2). To evaluate the diagnostic performance of each assay, sensitivity, specificity, and predictive values were calculated at various cutoffs. For most ELISAs, selection of a low cutoff resulted in good sensitivity but poor specificity (Table 1). Increasing the cutoff value resulted in better specificity but at the detriment of sensitivity (Table 1).

The choice of the most appropriate cutoff for a diagnostic test should include a consideration of (i) the distribution of results in two different populations, normal animals and animals with the disease (Table 1), (ii) the prevalence of disease in the population to be tested, and (iii) the consequences of false-positive and false-negative tests results (17). On a farm with enzootic R. equi infections, the consequence of missing the diagnosis of an infected animal (false-negative) may be, in the worst-case scenario, the death of the infected animal. Therefore, a very sensitive test is essential.

In the present study, at low cutoffs providing adequate sensitivity (>80%), specificity was less than 50% regardless of the assay considered, resulting in poor predictive value of a positive test. At such low cutoff values, the best negative predictive values on a farm where the prevalence of the disease is high were obtained with ELISA-California and ELISA-6939 (Table 1). The best diagnostic performance was achieved with ELISA-California at a cutoff of 40%, resulting in a sensitivity of 59% and a specificity of 88%. At a prevalence of 40%, this would result in positive and negative predictive values of 76%.

In one study, serological surveillance at 30 and 45 days of age with ELISA-6939 and a cutoff value of 0.3 was proposed for early detection of R. equi-infected foals on farms with enzootic infections (13). In another study with ELISA-6939, 20 (95%) of 21 foals with culture-confirmed R. equi pneumonia had an optical density value of 0.3 or greater, indicating good sensitivity of the assay (12). However, the diagnostic performance of the assay (sensitivity, specificity, and predictive values) on a farm where R. equi infections are enzootic was not determined. In the present study, ELISA-6939 with a cutoff of 0.3 gave a sensitivity of 41.5% and a specificity of 69.6%, resulting in poor positive and negative predictive values.

The poor diagnostic performance of serological assays in the present study is likely because widespread exposure of foals to R. equi at a young age leads to antibody production without necessarily inducing clinical disease (11). In addition, maternally derived antibody causes positive reactions with ELISAs, which further confounds their interpretation (11, 16).

Even on farms where the disease is enzootic, most R. equi isolates in the environment lack the VapA-encoding plasmid and are avirulent (18). In one study, foals experimentally infected with virulent R. equi seroconverted with ELISA-VapA 2 weeks postinfection, whereas foals administered the avirulent plasmid-cured derivative did not, suggesting that a diagnostic advantage of ELISA-VapA might be the failure to detect exposure to nonvirulent isolates (9). In the present study, the 95% confidence intervals of the area under the receiver-operating characteristic curve for ELISA-VapA included 0.5, indicating that the assay's performance was no better than flipping a coin. The poor diagnostic performance of ELISA-VapA in the present study confirms that many clinically healthy foals have antibody to virulent R. equi (19).

Agar gel immunodiffusion assays, such as the one used in the present study, are less sensitive than ELISA (21). As a result, adult horses are negative for precipitating antibody on agar gel immunodiffusion tests, and there are no positive reactions from colostral transfer of anti-R. equi antibody (7). In the present study, the agar gel immunodiffusion test with a strongly positive result as the cutoff resulted in poor sensitivity but good specificity.

In conclusion, the present study shows that currently available serological assays do not permit distinction between foals with R. equi pneumonia and clinically healthy pasturemates when performed on a single sample. Using these serological assays for screening or diagnosis of R. equi pneumonia in foals would result either in unnecessary therapy of many foals or in missing several infected foals, depending on the cutoff value used.