Skip to main content
NCBI home page
Clinical and Diagnostic Laboratory Immunology logoLink to Clinical and Diagnostic Laboratory Immunology
letter
. 2003 Jul;10(4):729–730. doi: 10.1128/CDLI.10.4.729-730.2003

Calculation Method for Likelihood Ratios Dictates Interpretation

Alecia Larew Naugle 1,2, William P Shulaw 1,2, William J A Saville 1,2, Thomas E Wittum 1,2, Beverly Byrum 1,2
PMCID: PMC164275  PMID: 12853414

Recently, likelihood ratios were published for use as an aid in the clinical interpretation of test results obtained from a commercially available enzyme-linked immunosorbent assay (ELISA) for the detection of antibodies against Mycobacterium avium subsp. paratuberculosis in cattle (2). As described by Dr. M. T. Collins, likelihood ratios can provide diagnostic information for multiple levels of a diagnostic test result when results are reported on a continuous scale rather than as a dichotomous (positive and negative) outcome. Unfortunately, the author did not take advantage of this characteristic of likelihood ratios.

Because the formula used to calculate likelihood ratios presented in this article incorporates sensitivity and specificity estimates for the ELISA at several arbitrary cutoff points, the resulting comparisons remain dichotomous in nature. Further, the interpretation of these likelihood ratios as calculated is incorrect. For example, the likelihood ratio calculated at the 0.10 level indicates that cows with S/P ratios of ≥0.10 are five times more likely to be infected than noninfected. With this approach, all cows with ELISA S/P ratios of ≥0.10 are included in the comparison. Multiple likelihood ratios that allow for comparisons between different levels of S/P ratios were not calculated. Because the ELISA S/P ratios were not stratified into multiple categories, the likelihood ratios presented in Table 1 of Collins's article (2) are overestimates resulting from heterogeneity within the defined groups of interest.

TABLE 1.

Calculation of likelihood ratios using a multilevel approach

Range of ELISA S/P ratios No. of infected cows No. of noninfected cows Likelihood ratioa
0.00-<0.10 39 1,359 0.5
0.10-<0.25 14 454 0.5
0.25-<0.40 8 45 2.9
0.40-<1.00 14 43 5.3
≥1.00 41 3 224.3
Total 116 1,904
Open in a new tab
a

Indicates odds that a cow with an ELISA S/P ratio in a specific range will be infected, calculated as [(number of ELISA S/P ratios from infected cows in stratum)/(total number of ELISA S/P ratios from infected cows)]/[(number of ELISA S/P ratios from noninfected cows in stratum)/(total number of ELISA S/P ratios from noninfected cows)].

Table 1 provides likelihood ratios based on the multiple-level approach proposed by Sackett et al. (3) and calculated from the data presented by Dr. Collins in his Table 1 (2).

The likelihood ratios calculated in this manner differ considerably from those presented by Dr. Collins, and differences in their subsequent interpretations may have significant clinical and economic consequences. For example, cows with ELISA S/P ratios of 0.10 to <0.25 are 0.5 times as likely (i.e., half as likely) to be infected than noninfected, not “5 to 15 times” as likely to be infected, as Dr. Collins indicated in his Table 3 (2). Likelihood ratios demonstrating less-than-eightfold differences between comparison groups have been suggested to provide weak statistical evidence that a test result is indicative of a defined outcome (1).

Additionally, the exclusion of ELISA S/P ratios of <0.00 from this analysis is a matter for concern. ELISA results from 1,097 animals, over one-third of the available results, were not considered. As this report indicates, ELISA S/P ratios of <0.00 may constitute a significant proportion of the results obtained during herd screening. These results could have been incorporated into the analysis had the multiple-level approach for likelihood ratio calculations been utilized.

Likelihood ratios can provide additional diagnostic information for use in the clinical interpretation of the ELISA for M. avium subsp. paratuberculosis in cattle. However, care must be taken in order to interpret these values correctly and to make correct management recommendations based on them. These data do not support the interpretation of ELISA S/P ratios presented by Dr. Collins.

REFERENCES

  • 1.Blume, J. 2002. Likelihood methods for measuring statistical evidence. Stat. Med. 21:2563-2599. [DOI] [PubMed] [Google Scholar]
  • 2.Collins, M. T. 2002. Interpretation of a commercial bovine paratuberculosis enzyme-linked immunosorbent assay by using likelihood ratios. Clin. Diagn. Lab. Immunol. 9:1367-1371. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Sackett, D., R. Haynes, G. Guyatt, et al. 1991. Clinical epidemiology: a basic science for clinical medicine, 2nd ed., p. 119-139. Lippincott, Williams and Wilkins, Philadelphia, Pa.
Clin Diagn Lab Immunol. 2003 Jul;10(4):729–730.

Author's Reply

M T Collins 1

Naugle et al. are correct in stating that Table 1 of the original paper (1) presents likelihood ratios (LRs) based on dichotomous interpretations of the ELISA results at multiple cutoffs and that this method of interpretation is different from a true multilevel LR analysis. That this method was used is indicated in the table by the term “ELISA S/P cutoff value” in the heading of the first column and by the footnote for “LR” in the last column, which gives the LR formula based on sensitivity and specificity.

LRs based on the same data used for the original article, including ELISA S/P results of <0.00, and calculated by using a multilevel approach are shown in Table 1 presented here.

TABLE 1.

LRs for for accurate diagnosis of M. paratuberculosis in cattle from ELISA S/P ratiosa

ELISA S/P ratio range No. (%) of cows tested for M. paratuberculosis that were:
 LR
Infected Noninfected
<0.10b 66 (46.15) 2,429 (81.67) 0.57
0.10-0.24 14 (9.79) 454 (15.27) 0.64
0.25-0.39 8 (5.59) 45 (1.51) 3.07
0.40-0.99 14 (9.79) 43 (1.45) 6.75
≥1.00 41 (28.67) 3 (0.10) 286.70
Total 143 2,974
Open in a new tab
a

Calculation of the LRs is based on multilevel approach and uses the same data as the original study, including ELISA S/P results of <0.00.

b

The value <0.10 includes results for all animals with S/P values of <0.10, not just those within the limits of the 0.00 to 0.10 range, thereby accounting for the data perceived by Naugle et al. as excluded.

The small number of infected cattle in the midrange categories limits the precision of the estimated LR. The negative association of ELISA S/P values with Mycobacterium paratuberculosis infection status in the 0.10 to 0.24 range is a function of as-yet-unexplained differences between U.S. and Dutch cattle.

To improve the precision of LR estimates, in the original analysis (1) I tried to include cattle from the full spectrum of infection (those shedding M. paratuberculosis in feces and those not shedding the bacterium) and as large a population of controls from paratuberculosis-free herds as possible. This necessitated the use of sera from Dutch cattle, since The Netherlands had more readily available certified paratuberculosis-free herds than did the United States. The choice of cases and controls significantly affected the five-level LRs, as shown in Table 2.

TABLE 2.

Effects of different case-control combinations on the accuracy of LRs for cattle diagnosed as positive for M. paratuberculosis by ELISA S/P ratios

ELISA S/P ratio range LR for:
Case Aa- control Ac Case Bb- control A Case A- control Bd Case B- control B
<0.10 0.56 0.35 0.48 0.29
0.10-0.24 0.64 0.56 5.73 5.03
0.25-0.39 3.70 2.45 7.08 4.68
0.40-0.99 6.75 8.17 37.65 47.50
≥1.00 286.70 469.10 110.27 180.42
Open in a new tab
a

Case A, all cattle determined to be infected in original study (n = 143).

b

Case B, only fecal-culture-positive cattle (n = 81).

c

Control A, all noninfected cattle nonresident in the United States and The Netherlands (n = 2,974).

d

Control B, only noninfected U.S. cattle (n = 760).

These results argue for geographically specific LRs. Paratuberculosis ELISA specificity differences were also reported by Van Maanen et al. (Abstr. 7th Int. Colloq. Paratuberculosis, abstr. 139, 2002). More data are required to evaluate the necessity of geographically specific LRs.

I elected to use an S/P ratio of 0.25 as the cutoff because it is the one recommended by the kit manufacturer to define positive results and because I felt it was more pragmatic to use this cutoff than to alter it. Instead, I tried to divide the S/P values below the cutoff into two categories and those above the cutoff into three categories. With U.S. cattle in certified paratuberculosis-free herds used as the comparison control group, the two right-hand columns in Table 2, which includes five levels of LRs, show that dairy cattle in the S/P range of 0.10 to 0.24 are over five times more likely to be infected with M. paratuberculosis. This justifies the proposed low-cost intervention strategies to control the potential spread of paratuberculosis from animals such as those presented in Table 3 of the original paper (1). Based on the same definitions for cases and controls and Blume's criteria for strength of evidence based on LRs, dairy cattle with ELISA results in the 0.40 to 0.99 range show strong evidence of M. paratuberculosis infection.

While the statistical concerns raised by Naugle et al. are valid, the relationship between ELISA S/P ratios and the likelihood of M. paratuberculosis infection in dairy cattle (and the correlation of ELISA S/P positive results with those of other tests for M. paratuberculosis infection, shown in Table 2 of the original paper [1]) remains apparent, regardless of choice of cases and controls. The purpose of the study was to create a simple system for decision making by dairy producers and veterinary practitioners based on M. paratuberculosis ELISA S/P values that was founded on the principles of LR analysis. Feedback from end users about this system is favorable; it allows for the management of this infectious disease without excessive cost, particularly in comparison to the heretofore recommended “test-and-cull” programs for bovine paratuberculosis using tests with only positive and negative interpretations.

Modeling economic decision analysis will require precise, multilevel LRs based on a larger number of well-characterized cases of bovine paratuberculosis and appropriate controls, and it may be necessary to define them for specific geographic regions. One must also keep in mind the effects of biological variation in host response to exposure or infection with mycobacteria as well as technical variations in assay performance and not become too enamored with statistical precision when describing diagnostic test outcomes.

I thank my Ohio colleagues for pointing out the important difference between the dichotomous LRs I provided in Table 1 of the original paper (1) and the five-level LRs calculated according to the method of Sackett et al. Our exchange of ideas and the resulting expanded data analysis have shed more light on the calculation and use of LRs in bovine paratuberculosis ELISA interpretation.

REFERENCE

  • 1.Collins, M. T. 2002. Interpretation of a commercial bovine paratuberculosis enzyme-linked immunosorbent assay by using likelihood ratios. Clin. Diagn. Lab. Immunol. 9:1367-1371. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Clinical and Diagnostic Laboratory Immunology are provided here courtesy of American Society for Microbiology (ASM)

RESOURCES