Bayesian latent class models to estimate diagnostic test accuracies of COVID‐19 tests

To the Editor,

With great interest we read the article from Dramé et al ¹ “Should RT‐PCR be considered as gold standard in the diagnosis of COVID‐19” questioning the conclusions from Cassaniti et al. ² We agree with the argument that considering reverse transcription‐polymerase chain reaction (RT‐PCR) as gold standard when evaluating a new test will inevitably lead to biased test accuracy estimates of the new test. ¹ However, we disagree that it is impossible to “demonstrate that a new test might be better than the old‐test” and propose using Bayesian latent class models (BLCMs) to estimate the sensitivity and specificity in the absence of a gold standard. “Latent” means here that the true disease/infection status is not observed, but can be estimated from the data. BLCMs have been applied widely, reporting guidelines have been developed, ³ and currently an European Union‐funded initiative aims to expand their application in the evaluation of diagnostic tests (www.harmony-net.eu).

For showcasing BLCMs, we used the data from Cassaniti et al, ² although the keys to validity in diagnostic test studies are not respected, that is, “a consecutive series of patients suspected (but not known) to have the target disorder,” ⁴ which precludes the generalizability of the findings. We considered a model with three tests for the diagnosis of coronavirus disease‐2019 (COVID‐19) in one population (110 patients), and extracted the following data of cross‐classified test results (RT‐PCR, IgG, and IgM) ² : +++ 28, ++− 1, +−+ 3, +−− 36, −++ 0, −+− 0, −−+ 1, and −−− 41. We aim to estimate the prevalence, the sensitivities, and the specificities of three tests, and potential conditional dependencies (correlations) between tests. Since the maximum number of estimable parameters for three tests in one population is seven, ⁵ this model is not identifiable. Therefore, in a Bayesian framework, we rely on prior information for the specificities of all three tests similar to a previous analysis with two tests and four populations. ⁶ We assume that they are close to 100%. Further technical details about the model and Markov Chain Monte Carlo simulation, including the code to reproduce the results, are available on (https://github.com/shartn/BLCM-COVID19). In Table 1 the posteriors resulting from a model with conditional dependency between the IgG and IgM test sensitivities are displayed.

Table 1.

Posterior medians and 95% probability intervals for the sensitivities and specificities of three tests, prevalence and the correlation between the sensitivities of the IgM and the IgG test resulting from BLCM analysis

Parameter	Posterior median (95% probability interval)
Prevalence	87.9 (72.2‐99.0)
RT‐PCR sensitivity	55.2 (49.7‐65.6)
RT‐PCR specificity	99.0 (97.7‐99.7)
IgG sensitivity	31.3 (21.9‐42.4)
IgG specificity	98.2 (94.8‐99.6)
IgM sensitivity	33.2 (23.8‐44.1)
IgM specificity	98.1 (94.4‐99.6)
Correlation IgG/IgM	8.6 (6.3‐10.4)

Abbreviations: BLCM, Bayesian latent class model; Ig, immunoglobulin; RT‐PCR, reverse transcription‐polymerase chain reaction.

In contrast to the previous analysis, ² which used the RT‐PCR as a gold standard, implicitly assuming 100% sensitivity and specificity, the RT‐PCR sensitivity based on our BLCM analysis is lower, while the specificity of all three tests is close to 100%. The estimated sensitivity of the combined IgM/IgG in the study from Cassaniti et al ² was 18.4%, being outside the 95% probability interval of our BLCM estimates for both the IgM and IgG. Although, based on our BLCM analysis, we cannot state “that the new test might be better,” ¹ we are confident that BLCMs allow estimation of sensitivities and specificities in an unbiased way without needing a gold standard. Since this data set—with the exception of the 50 emergency patients—does not comply with the key aspects of validity, ⁴ it is not possible to generalize the results of our BLCM beyond the study population of the 110 patients.

Despite enormous COVID‐19 research activities, the application of BLCMs and suitable data sets are virtually absent. For the preparedness of future pandemics, we suggest to include the design of appropriate diagnostic test accuracy studies, which will also be suitable for BLCMs.

DATA AVAILABILITY STATEMENT

We used for our BLCM analysis data which have been published by the journal, and extracted the data directly from this publication: https://doi.org/10.1002/jmv.25800.