Rejoinder to “On the robustness of latent class models for diagnostic testing with no gold standard”

We thank both Dendukuri and Albert for their insightful commentaries on our article. They have provided food for thought for us (the authors), as well as the wider biostatistical community. We highlight some of the important points that were raised and offer our perspective.

Dendukuri highlights the value of working collaboratively within an interdisciplinary group. They discuss how this can help in the development of a realistic depiction of how the tests operate in the context of the disease in question, making it more likely that we specify and fit statistical models that are fit for purpose and can be applied by practitioners. Using the leptospirosis example in Schofield et al¹ they discuss how this could involve the latent classes being defined in terms of levels of IgM and IgG antibodies.

We agree that interdisciplinary collaborations (such as ours) are of tremendous value but note the importance of considering not only the model but also the underlying clinical question. We need to ensure that our models are not providing an “answer to the wrong question” (paraphrasing Tukey²). In the leptospirosis examples in Schofield et al,¹ we note the rapid tests were being evaluated for clinical utility in diagnosing leptospirosis disease. Therefore, an ability to interpret the latent classes as “disease (leptospirosis)” and “no disease (no leptospirosis)” is critical. Considering a model where we can interpret the latent classes in terms of IgM and IgG antibodies is of little to no clinical interest.

Dendukuri is critical of the simple model we have evaluated noting it is “unrealistic.” Implicit is the notion that understanding such models is of little value in applied research. While we agree with Dendukuri that the model is often unrealistic for real data, we disagree that exploration of a two-state conditionally independent LCM model is of little value. That idealized latent variables (disease status) may not correspond to the fitted latent variables is a fundamental difficulty for any model involving latent features. We believe that demonstrating potential problems with the interpretation of latent states in simple scenarios can be a powerful tool for understanding limitations of a wider model class.

Albert describes the similarity between our results and those for more complex LCMs, in particular Albert and Dodd.³ We agree with Albert’s summary and appreciate the clarity their comments have brought to the discussion. They also outline possible approaches for alleviating the identified problems with lack of robustness of LCMs.^4,5 In particular, they suggest the use of a gold standard, even if only for a subset of our observations. We agree that if possible, consideration should be given in the design of the study to use of a gold-standard test on a subset of participants. It is likely to add expense and complexity, but it could provide confidence in the reliable interpretation of the latent classes as disease and disease free, and thus reliable estimates of sensitivity and specificity. Unfortunately this is not possible for leptospirosis as no gold standard is available. What may be possible is a silver standard,⁶ a test that has perfect specificity but unknown sensitivity. It is unclear the extent to which a silver standard can alleviate the lack of robustness that we and others have identified. Understanding the properties of such models is an area of future research.