Evaluation of a Computer‐aided Lung Auscultation System for Diagnosis of Bovine Respiratory Disease in Feedlot Cattle

Abstract

Background

A computer‐aided lung auscultation (CALA) system was recently developed to diagnose bovine respiratory disease (BRD) in feedlot cattle.

Objectives

To determine, in a case–control study, the level of agreement between CALA and veterinary lung auscultation and to evaluate the sensitivity (Se) and specificity (Sp) of CALA to diagnose BRD in feedlot cattle.

Animals

A total of 561 Angus cross‐steers (initial body weight = 246 ± 45 kg) were observed during the first 50 day after entry to a feedlot.

Methods

Case–control study. Steers with visual signs of BRD identified by pen checkers were examined by a veterinarian, including lung auscultation using a conventional stethoscope and CALA that produced a lung score from 1 (normal) to 5 (chronic). For each steer examined for BRD, 1 apparently healthy steer was selected as control and similarly examined. Agreement between CALA and veterinary auscultation was assessed by kappa statistic. CALA's Se and Sp were estimated using Bayesian latent class analysis.

Results

Of the 561 steers, 35 were identified with visual signs of BRD and 35 were selected as controls. Comparison of veterinary auscultation and CALA (using a CALA score ≥2 as a cut off) revealed a substantial agreement (kappa = 0.77). Using latent class analysis, CALA had a relatively high Se (92.9%; 95% credible interval [CI] = 0.71–0.99) and Sp (89.6%; 95% CI = 0.64–0.99) for diagnosing BRD compared with pen checking.

Conclusions

CALA had good diagnostic accuracy (albeit with a relatively wide CI). Its use in feedlots could increase the proportion of cattle accurately diagnosed with BRD.

Abbreviations

BRD

bovine respiratory disease

CALA

computer‐aided lung auscultation

CI

credible interval

Hap

haptoglobin

SD

standard deviation

Se

sensitivity

Sp

specificity

Accurate diagnosis of bovine respiratory disease (BRD) in feedlot cattle is crucial for effective treatment and implementation of prevention strategies.1 Furthermore, because BRD treatment relies mainly on use of antimicrobials, an accurate BRD diagnosis should promote prudent use of antimicrobials by reducing unnecessary treatments. Unfortunately, current diagnostic methods to identify feedlot cattle affected with BRD are not always accurate.2 Indeed, these methods, based on visual inspection by pen checkers, are highly subjective, even when combined with measurement of rectal temperature.3 Based on a latent class analysis using clinical inspection throughout the feeding period and presence of lung lesions at slaughter as tests for BRD diagnosis, the sensitivity (Se) and specificity (Sp) of clinical inspection were 62 and 63%, respectively.2

Several methods including lung ultrasonography, radiographs, lung auscultation, determination of serum haptoglobin concentration have been used to improve accuracy of BRD diagnosis.3 Among these methods, lung sound auscultation is inexpensive, can be conducted chute side and is highly specific in dairy calves compared with ultrasonographic assessment of lung lesions.4 Unfortunately, lung auscultation is also subjective and requires a well‐trained person with good acoustic abilities to correctly recognize abnormal sounds. To overcome these drawbacks, a computer‐aided lung auscultation (CALA) system¹ has been developed. By automatically classifying acoustic patterns in lung scores, this system could increase accuracy of BRD diagnosis. However, to be useful its accuracy to diagnose BRD must be critically evaluated in a case–control study.

The objectives were to: (1) determine the level of agreement between CALA and lung auscultation by an experienced veterinarian and; (2) evaluate using Bayesian latent class analysis the diagnostic accuracy (Se, Sp) of CALA for BRD in feedlot cattle. We hypothesized that a moderate to substantial agreement exists between CALA and veterinary auscultation and that CALA is an accurate method to diagnose BRD.

Materials and Methods

Animals

All management and procedures were reviewed and approved by the University of Calgary Animal Care Committee (AC13‐0212) and were in accordance with guidelines of the Canadian Council on Animal Care.5

Angus cross‐steers (n = 561; initial body weight = 246 ± 45 kg) at high risk of developing BRD because of recent weaning, commingling and being auction‐market derived were studied during the first 50 day after their arrival at a commercial feedlot in Western Canada. Upon arrival, steers were allowed to rest for at least 12 h (with ad libitum access to hay and water) before processing. At processing, steers received a subcutaneous injection of a long‐acting macrolide² and were vaccinated against infectious bovine herpes virus‐1,³ bovine viral diarrhea virus (types I and II),³ bovine parainfluenza‐3,³ bovine respiratory syncytial virus,³ Mannheimia haemolytica,⁴ Histophilus somni,⁵ and clostridial pathogens.⁵ Steers were also dewormed with pour‐on ivermectin solution.⁶

Steers were fed in 2 large outdoor dirt‐floor pens (67 × 61 m with a 64‐m fence‐line concrete feed bunk) with approximately 280 steers per pen. Steers were fed twice daily, at 0630 and 1430 hours, a 55–63% concentrate receiving/growing diet formulated to meet or exceed nutrient requirements.6 Each morning before feeding, bunks were visually evaluated and feed deliveries were adjusted to ensure that sufficient feed was available for ad libitum consumption. On day 50, steers were revaccinated⁷ and implanted.⁸

Study Design: Case–Control Study

During the study period, steers were observed daily by experienced pen checkers for detection of clinical illness. Steers with visual signs of BRD including one or more of depression, nasal or ocular discharge, cough, increased respiratory rate, and labored breathing were removed from the pen by pen checkers and examined by an experienced veterinarian. For each steer suspected of having BRD, 1 apparently healthy steer with no visual signs of BRD or other disease was conveniently selected based on proximity to the gate or apparently sick animal as pen‐matched contemporary control and similarly examined. Clinical examination included measurement of respiratory rate using a stopwatch and rectal temperature, complete lung auscultation using a conventional stethoscope⁹ to detect abnormal lung sounds including increased bronchial sounds, crackles and wheezes,7 and focused lung auscultation using the CALA system. The veterinarian who performed the clinical examinations did not know which animals were pulled as BRD cases or control and veterinary auscultation was always performed before CALA to avoid potential bias (i.e., human auscultation blinded to CALA results).

Steers with visual signs of BRD and a rectal temperature ≥40°C received flunixin meglumine and florfenicol SC.¹⁰

Computer‐aided Lung Auscultation

Computer‐aided lung auscultation consisted of holding the diaphragm of an electronic stethoscope¹ over the 5th intercostal space of the right thoracic wall, approximately 10 cm above the elbow and recording lung sounds for 8 s (as per manufacturer's instructions). Recorded lung sounds were then automatically transmitted wirelessly to a computer located within 3 m of the stethoscope and analyzed by software provided by the manufacturer.¹ This program: (1) displayed spectrogram of recorded sounds; (2) preprocessed lung sounds to remove heart sounds and potential interference from the environment (chute noise, etc.); and (3) classified acoustic patterns in lung scores ranging from 1 to 5 (1 = normal, 2 = mild acute, 3 = moderate acute, 4 = severe acute, and 5 = chronic). Lung scores were transmitted back to the stethoscope and displayed.

Serum Haptoglobin Determination

In addition to clinical examination, a blood sample was collected from each steer to detect inflammation by measurement of serum haptoglobin (Hap) concentration. Serum haptoglobin concentrations were determined in duplicate using a commercial kit.^{11 ,} 8

Data Analysis

Clinical findings (rectal temperature, respiratory rate per minute, serum Hap concentrations) between cattle examined for BRD and cattle selected as controls by pen checker were compared using nonparametric (Mann–Whitney U‐test) and parametric tests (Student's t‐test).1

The level of agreement between lung auscultation by an experienced veterinarian and CALA (using a CALA score ≥2 as a cut off) was compared using Kappa statistic.¹² The strength of agreement for the Kappa coefficient was interpreted using the scale of Landis and Koch9: ≤0 = poor, 0.01–0.20 = low, 0.21–0.40 = fair, 0.41–0.60 = moderate, 0.61–0.80 = substantial, and 0.81–1 = almost perfect.

Because of the absence of a reference test to identify the true BRD status of cattle (i.e., no gold standard), Bayesian latent class analysis was used to evaluate the Se and Sp of CALA for BRD diagnosis in feedlot cattle.10 For this analysis, results of CALA were compared with pen checker classification. A CALA score ≥2 was considered positive for BRD, whereas a CALA score = 1 was considered negative. Pen checker classification and accuracy were based on a previous study,2 with cattle detected with visual BRD signs defined as BRD positive and cattle with no visual BRD signs defined as BRD negative (i.e., cattle selected as controls in this study).

Prior probability distributions of tests' Se and Sp and BRD prevalence used for the Bayesian analysis are shown (Table 1). Because no prior information on CALA's Se and Sp (Se_CALA and Sp_CALA) was available, uninformative prior probabilities in the shape of uniform distribution between zero and one (modeled using a Beta (1,1) distribution) were chosen for Se_CALA and Sp_CALA. Prior probability distributions for pen checkers' Se and Sp (Se_p and Sp_p) were chosen based on a previous study.2 Prior probability distributions chosen for BRD prevalence were fairly noninformative (ranging from 30 to 70%, with a best guess of 50%, because of the case–control design).

Table 1.

Prior distributions and caudal latent class estimates (median and 95% credibility interval [CI]) of bovine respiratory disease (BRD) prevalence and test sensitivity (Se) and specificity (Sp) of pen checkers and computer‐aided lung auscultation (CALA) for BRD diagnosis

	Prior distribution			Caudal estimates
	Median (%)	95% CI (%)	Betaa	Median (%)	95% CI (%)
BRD prevalence	50.0	30.0–70.0	8, 8	50.7	35.6–64.1
Se pen checkers	62.0	56.0–68.0	164.4, 101.1	63.5	57.9–68.8
Sp pen checkers	63.0	60.0–66.0	646.9, 380.3	63.5	60.5–66.4
Se CALA	–b	–b	1, 1	92.9	71.1–99.7
Sp CALA	–b	–b	1, 1	89.6	64.1–99.5

^aBeta parameters were calculated from 95% CI using free online software. (Epitools, Sergeant, ESG, 2013. AusVet Animal Health Services and Australian Biosecurity Cooperative Research Centre for Emerging Infectious Disease. Available at: http://epitools.ausvet.com.au).

^bA uniform probability over the range 0–100 was used for the priors of CALA's Se and Sp.

The final model used 2 tests and 1 population and assumed conditionally independence of tests.10 Visual appraisal by pen checker and CALA were considered conditionally independent, as they were not based on similar biological principles. Notwithstanding, independence between these 2 tests was nevertheless confirmed by demonstrating that covariances in healthy steers (11.6%; 95% credible intervals [CI], −0.4 to 20.6) and steers with BRD (8.6%; 95% CI, −2.7 to 20.6) crossed the value of 0 using Markov Chain Monte Carlo methods with Gibbs sampler.^{13 ,} 11

Bayesian computations were implemented using free software.¹³ The first 5,000 iterations were discarded as burn‐in, whereas the next 100,000 were used to obtain caudal distributions. Convergence of the model was assessed by visual inspection of the time series plots of selected variables and Gelman–Rubin diagnostic plots (after running multiple chains with various starting values). The caudal distributions of tests sensitivities and specificities and disease prevalence were reported as medians and corresponding 95% CI.

Results

Of the 561 steers, 35 (6.2%) were detected with visual BRD signs and 35 were selected as pen‐matched controls. All steers with visual signs of BRD had abnormal lung sounds including one or more of increased bronchial sounds, crackles, and wheezes detected by auscultation by a veterinarian. Interestingly, 9 steers selected as controls had also abnormal lung sounds. Rectal temperatures, respiratory rates per minute, and serum Hap concentrations differed (P < .05) between steers detected with visual signs of BRD and those selected as controls (Table 2).

Table 2.

Health data (rectal temperature, respiratory rate per minute, serum haptoglobin concentration) in feedlot steers selected as bovine respiratory disease (BRD) cases (n = 35) or pen‐matched healthy controls (n = 35) by pen checkers

	BRD (n = 35)		Control (n = 35)
	Mean (±SD)	Median (Q1–Q3)	Mean (±SD)	Median (Q1–Q3)
Rectal temperature (°F)	105.3a (0.9)	105.3 (104.9–106.2)	102.7b (1.1)	102.8 (102.2–103.4)
Respiratory rate/min.	46 (5)	46a (42–50)	34 (5)	32b (30–37)
Serum haptoglobin (g/L)	1.49 (0.68)	1.47a (1.03–1.89)	0.37 (0.48)	0.18b (0.13–0.34)

Q1, 25th percentile; Q3, 75th percentile; SD, Standard deviation.

Within a row, means or medians without a common superscript differed (P < .05).

A CALA score was obtained from all examined steers (n = 70), with scores ranging from 1 to 5 (Fig. 1). Comparison of CALA results with auscultation by a veterinarian (using a CALA score ≥2 as a cut off) revealed a substantial agreement (kappa = 0.77; 95% CI, 0.62–0.92), with 62 concordant results out of the 70 clinical examinations (Table 3). The 8 discordant results were attributed to the presence of abnormal lung sounds detected by auscultation by a veterinary, but not by CALA.

Figure 1.

Repartition of lung scores (1 = normal, 2 = mild acute, 3 = moderate acute, 4 = severe acute and 5 = chronic) obtained after computer‐aided lung auscultation¹ in a population of steers identified with visual signs of bovine respiratory disease (BRD) by pen checkers (n = 35) and in a population of steers selected as healthy controls (n = 35).

Table 3.

Agreement between lung auscultation by an experienced veterinarian using a conventional stethoscope⁹ and computer‐aided lung auscultation (CALA) for detection of abnormal lung sounds (e.g., increased bronchial sounds, crackles, and wheezes)7 in feedlot cattle (kappa = 0.77; 95% CI = 0.62–0.92)

	Veterinary auscultation		Total
	+	−
CALAa
+	36	0	36
−	8	26	34
Total	44	26	70

^aCattle with a CALA score ≥2 were considered BRD positive (+), whereas cattle with a CALA score = 1 were considered BRD negative (−).

Pen checker classifications and CALA results were crossed classified into a 2 × 2 table (Table 4), which was used for the Bayesian latent class analysis. Caudal estimates (median and 95% CI) for Se_CALA, Sp_CALA, Se_p, Sp_p, and prevalence of BRD are shown (Table 1). Computer‐aided lung auscultation had good diagnostic accuracy with relatively wide CI with Se_CALA and Sp_CALA estimated at, respectively, 92.9% (95% CI, 0.71–0.99) and 89.6% (95% CI, 0.64–0.99). Compared with CALA, pen checker's accuracy was lower with Se_p and Sp_p estimated at, respectively, 63.5% (95% CI, 0.58–0.69) and 63.5% (95% CI, 0.60–0.66).

Table 4.

Two by two table comparing diagnosis of bovine respiratory disease (BRD) by pen checkers with BRD diagnosis by a computer‐aided lung auscultation (CALA) system

	Pen checker		Total
	+	−
CALAa
+	29	7	36
−	6	28	34
Total	35	35	70

^aCattle with a CALA score ≥2 were considered BRD positive (+) whereas those with a CALA score = 1 were considered BRD negative (−).

Discussion

In this study, there was a substantial level of agreement between CALA and lung auscultation performed by an experienced veterinarian. Compared with pen checking using Bayesian latent class analysis, CALA also had a relatively high Se (92.9%; 95% CI = 0.71–0.99) and Sp (89.6%; 95% CI = 0.64–0.99) for diagnosing BRD in feedlot cattle.

The substantial agreement between CALA and veterinary auscultation was expected as CALA's algorithm was initially trained to correctly classify abnormal lung sounds detected by experienced veterinarians (R. Geissler, personal communication). In this study, veterinary auscultation nevertheless detected abnormal lung sounds more often than CALA. This finding could be explained by a higher sensitivity of veterinary auscultation. Indeed, moderate sensitivity is a common drawback of computerized lung sounds analysis. In a meta‐analysis,12 algorithms for classification of lung sounds had an overall Se of 80% (95% CI = 72–86%) for detection of abnormal lung sounds (wheezes and crackles) in humans when compared with auscultation by a trained person. However, further research is needed to confirm this hypothesis, as Se of auscultation by a veterinarian was not calculated in this study.

In the absence of a perfect reference test (gold standard), the use of latent class analysis is considered to be the best method to estimate the accuracy of a new diagnostic test.13 Indeed, latent class analysis refers to the idea that true disease status of animals is unknown and needs to be estimated from the data. If classification errors in the reference test are ignored, serious bias can be introduced in assessment of the accuracy of the new test. For example, in a case of a reference test with a Se <100% (as pen checking, which has a Se estimated at 62.0%),2 samples which are falsely classified as negative by this imperfect test could be correctly detected as positive by a more sensitive new test, thus leading to a biased estimate of Sp (in this case, too low) of the new test.

Furthermore, the Bayesian model used in this study allowed for incorporation of prior scientific information on variables to estimate (test accuracies and disease prevalence). However, because we had a relatively small sample size, we choose noninformative prior probability distributions for Se_CALA and Sp_CALA. Although the use of noninformative prior distributions allows caudal densities to be impacted more by the data than by the prior distributions, this could also explain why the 95% CI for Se_CALA and Sp_CALA were relatively wide. Further research is therefore needed to narrow the CI around CALA's Se and Sp and consequently have more confidence in the results provided by this technology.

It is noteworthy that the prior probability distributions chosen for pen checkers’ Se and Sp were based on a previous study and thus might not represent the Se and Sp of the pen checkers involved in this study, which could influence the accuracy of CALA. However, additional analyses were conducted using modified prior distributions and similar results for the CALA's Se and Sp were obtained. Indeed, by using a pen checker's Se and Sp ranging from 50 to 100% with a best guess of 75% (i.e., beta distribution 9.63–3.88), we obtained a CALA's Se and Sp of 91.9% (95% CI, 74.0–99.6) and 90.3% (95% CI, 71.1–99.5), respectively (data not show). Therefore, the authors are confident that the choice of prior probability distributions based on a previous study did not bias the findings of this study.

The sensitivity obtained in this study for CALA was higher than anticipated. In a recent study on dairy calves, Se of lung auscultation to diagnose BRD (defined as lung consolidation detected with ultrasonography) was only 5.9% (range, 0–16.7%). This difference in Se can be explained by the fact that CALA's algorithm included increased bronchial breath sounds for calculation of lung scores, whereas in this previous study, only crackles, wheezes or absence of respiratory sounds was interpreted as abnormal. Indeed, in this previous study, investigators did not interpret bronchial breath sounds (although highly Se to diagnose BRD)7 as these sounds were considered too subjective. The main advantage of CALA resides in its algorithm that can provide an objective lung score and thereby minimize bias.

On the basis of the higher specificity of CALA compared with pen checker, we inferred that this technology has the potential to decrease the proportion of cattle falsely diagnosed with BRD and thus, could promote prudent use of antimicrobials in commercial feedlots by reducing unnecessary treatments. Interpretation of CALA results in cattle previously identified by pen checkers as BRD‐affected which is serial interpretation scheme with conditional independence, could increase the overall Sp of BRD diagnosis in feedlot cattle (Sp_p+CALA = Sp_p + Sp_CALA – (Sp_p*Sp_CALA) = 96.1%) compared with pen checking alone (Sp_p = 63.0%).2 Furthermore, CALA does not require experience in lung auscultation and therefore could be easily used by the feedlot employees who have primary responsibility for diagnosis and treatment of BRD.

In conclusion, this study showed that CALA was a promising technology to improve accuracy of BRD diagnosis in feedlots. Its use could increase the proportion of cattle accurately diagnosed with BRD by a reduction in false‐positive diagnoses.