Concerns about what constitutes clinical evidence

We read with great interest the article by Soltani and Moayyeri¹ in the November 2005 issue of Canadian Family Physician. We fully agree with the authors that probabilistic reasoning (as opposed to a deterministic approach) is the optimal use of available evidence in estimating the likelihood of a diagnosis.

Nevertheless, we have serious hesitations about the validity of what the authors define as the dynamic properties of likelihood ratios (LRs) and about their argument that LRs can be easily used in a sequence of tests. An important advantage of LRs is that they can combine the results of multiple tests, in which the LR of the whole set of findings is the product of the LR of each individual test.² A necessary assumption for this approach, however, is the conditional independence between tests.³ Two tests are independent if knowing the result of one test does not change the probability of the result of the other one. This condition is often not met in reality, constituting an obstacle against using sequential LRs without proper adjustment for test dependency.

In the example given in the article, the authors have multiplied 5 LRs attributable to 5 items in the medical history, physical examination, and paraclinical evaluation of a hypothetical patient. This approach resulted in a change in the probability of cancer from 0.7% to around 20%. The problem of dependence, however, arises. In this instance, for example, history of cancer and age are very likely to be correlated because patients with previous history of cancer are generally older than patients without such history. The combined LR of these 2 findings for the diagnosis of cancer is, therefore, different from the product of individual LRs.

Similar arguments hold for several other combinations of tests in Table 1 of the article (eg, dependence between the duration of pain and weight loss and between the history of cancer and radiographic findings). Consequently, the estimated posterior probability of 20% is probably inaccurate. No further refinement of this estimation is possible unless some measure of conditional dependence is at hand.

Dealing with test dependency is a statistical issue. Some methods have been proposed to account for test dependence,^4,5 none of which are simple enough to be used in routine clinical practice.

Notwithstanding all the advantages of evidence-based reasoning, physicians should be aware of the pitfalls involved in implementing such approaches without considering the underlying assumptions and limitations.