BAYESIAN NESTED LATENT CLASS MODELS FOR CAUSE-OF-DEATH ASSIGNMENT USING VERBAL AUTOPSIES ACROSS MULTIPLE DOMAINS

Abstract

Understanding cause-specific mortality rates is crucial for monitoring population health and designing public health interventions. Worldwide, two-thirds of deaths do not have a cause assigned. Verbal autopsy (VA) is a well-established tool to collect information describing deaths outside of hospitals by conducting surveys to caregivers of a deceased person. It is routinely implemented in many low- and middle-income countries. Statistical algorithms to assign cause of death using VAs are typically vulnerable to the distribution shift between the data used to train the model and the target population. This presents a major challenge for analyzing VAs, as labeled data are usually unavailable in the target population. This article proposes a latent class model framework for VA data (LCVA) that jointly models VAs collected over multiple heterogeneous domains, assigns causes of death for out-of-domain observations and estimates cause-specific mortality fractions for a new domain. We introduce a parsimonious representation of the joint distribution of the collected symptoms using nested latent class models and develop a computationally efficient algorithm for posterior inference. We demonstrate that LCVA outperforms existing methods in predictive performance and scalability. Supplementary Material and reproducible analysis codes are available online. The R package LCVA implementing the method is available on GitHub (https://github.com/richardli/LCVA).

1. Introduction.

Data describing cause of death is an essential component for understanding the burden of disease, emerging health needs, and effect of public health interventions. Few low- and middle-income countries (LMIC) have systems that produce high-quality cause of death statistics. Only about two-thirds of the deaths worldwide are registered and up to half of these deaths are either not assigned a cause or assigned only an ill-defined cause (World Health Organization (2021)). In many populations not served by official medical certification of causes, a technique known as verbal autopsy (VA) has been routinely used to infer causes of death. VA is a survey-based method whereby a structured questionnaire is conducted to a family member or a caregiver of a recently deceased person after a suitable mourning period. The VA interview collects information about the circumstances, signs, and symptoms leading up to the death. It is routinely used by researchers in Health and Demographic Surveillance System (HDSS) including the INDEPTH network (Sankoh and Byass (2012)) and the ALPHA network (Maher et al. (2010)), multicountry research projects (Breiman et al. (2021), Nkengasong et al. (2020)), and national scale surveys in many LMICs.

Data collected by VAs are analyzed either by a panel of physicians or with statistical algorithms. The physician review approach can be effective if resource permits (Lozano et al. (2011)), but it is expensive and rarely possible for timely monitoring of cause-specific mortality. On the other hand, statistical methods have been widely used to automate the cause-of-death assignment from VA data (e.g., Byass, Huong and Van Minh (2003), McCormick et al. (2016), Miasnikof et al. (2015), Serina et al. (2015)). These methods aim to construct a high-dimensional classifier for cause-of-death assignment and obtain population-level cause-specific mortality fraction (CSMF) estimates, that is, the distribution of deaths due to each possible cause.

There are many methodological challenges for cause-of-death assignment algorithms to be successfully deployed in practice. First, all existing VA algorithms assume that there is a single training dataset or domain knowledge base from which the relationship between symptoms and causes can be learned. In most situations where VA is used, local deaths from the target population with known causes are extremely rare. High-quality VA data with reference causes are usually only available outside of the target population and may consist of data collected from several distinct study populations. This leads to the data shift problem, where the source and target domain data may have different marginal and conditional distributions (Moreno-Torres et al. (2012)), and it could significantly bias the cause-of-death assignment and CSMF quantification. The issue of data shift received little discussion in the midst of developments of automated VA cause-of-death assignment algorithms in the last decade. The recent work of Datta et al. (2021) and Fiksel et al. (2021) are the first to address this issue by calibrating model predictions to additional local validation data with known causes of death. When no validation data exist, however, all existing VA methods are vulnerable to data shift. To our knowledge, there is no literature that characterizes the heterogeneous joint distributions of symptoms and causes across different domains or leverages such information for cause-of-death assignment.

Second, existing VA methods used by practitioners typically assume that symptoms are conditionally independent given the underlying cause of death as it reduces model complexity and computation cost significantly (Byass, Huong and Van Minh (2003), McCormick et al. (2016), Miasnikof et al. (2015)). More recently, several approaches have been proposed to account for symptom dependence using latent Gaussian models with sparse or low-rank representations (Kunihama et al. (2020), Li, McComick and Clark (2020), Moran et al. (2021)). It has been shown that incorporating symptom dependence usually improves the accuracy of both individual cause-of-death assignment and the population-level CSMF estimation. However, the inferred dependence relationship among the continuous latent variables is difficult to interpret, as they do not correspond to the dependence of the observed binary symptoms (Li, McComick and Clark (2020)). In addition, the computation required to estimate the high-dimensional latent Gaussian mixtures can take hours or days to complete, which makes these methods difficult for routine use in low-resource settings.

In this paper we address these challenges by developing a novel latent class model framework for VA data (LCVA). A key feature of LCVA is that we estimate a common collection of latent symptom profiles across different populations. This leads to an explicit domain adaptation strategy for classifying out-of-domain deaths that takes into account the similarity between the target population and the existing training datasets. We introduce a sparse parameterization to further reduce the dimensionality of the latent symptom profiles that also facilitates direct interpretations of the conditional independence relationship among the observed symptoms. Furthermore, our inference procedure is an order of magnitude faster than the existing methods using latent Gaussian models to capture symptom dependence.

The rest of the paper is organized as follows. Section 2 reviews the background of existing VA methods and their implications under data shift. Section 3 proposes the nested latent class model approach for cause-of-death assignment. Section 3.1 discusses the model with one training dataset. Section 3.2 extends the model to the scenario where training data consist of deaths from multiple domains. Section 4 develops an efficient Markov chain Monte Carlo (MCMC) algorithm for posterior sampling. Section 5 demonstrates the proposed model under various realistic scenarios using the Population Health Metrics Research Consortium (PHMRC) gold-standard VA dataset. Section 6 concludes the paper and discusses future directions.

2. Domain adaption in cause-of-death assignment.

The domain adaptation problem refers to the situation where a statistical learning model trained on one labeled dataset needs to be generalized to the target dataset, or target domain, drawn from a different distribution and with insufficient labeled data (Daumé III and Marcu (2006)). In the context of cause-of-death assignment using VA, the discussion of cross-domain generalizability goes back to the early work of King and Lu (2008), where the authors developed a constrained least square model trained on hospital deaths with known causes and subsequently applied to classifying community deaths. Let $X_i$ denote the vector of symptoms for individual $i$ and $Y_i$ denote the corresponding cause of death. For simplicity, let us assume for now that there are two datasets: a training dataset $𝒯$, with both $X_i$ and $Y_i$ known, and a target dataset ${𝒯}_0$ where only $X_i$ is observed. King and Lu (2008) makes the assumption that $p_{{𝒯}_0}(X\mid Y)=p_{𝒯}(X\mid Y)$. This assumption implies that the difference in the joint distribution of $p(X,Y)$ between the two domains can be fully explained by the difference in $p(Y)$, since $p(X,Y)=p(Y)p(X\mid Y)$. Prediction tasks under this decomposition are also termed anticausal prediction (Schölkopf et al. (2012)), and this type of data shift with domain-specific $p(Y)$ and domain-invariant $p(X\mid Y)$ is known as label shift or prior shift (Storkey (2009)).

A large collection of VA algorithms, however, took a more generic classification perspective. They first learn a model $p_{𝒯}(Y\mid X)$ with the training data and then apply the classifier to observations from the target domain for cause-of-death assignment. The generalizability of this approach relies on the assumption that $p_{{𝒯}_0}(Y\mid X)=p_{𝒯}(Y\mid X)$. That is, the difference in the joint distribution of $p(X,Y)$ across domains are fully explained by different symptom distribution $p(X)$. This type of data shift is also known as covariate shift (Shimodaira (2000)). Under this assumption the target CSMF can be estimated by simple aggregation of individual classifications, that is,

$${\hat{p}}_{{𝒯}_0}(Y)={\int }_x{\hat{p}}_{{𝒯}_0}(Y\mid X=x)p_{{𝒯}_0}(x)dx={\int }_x{\hat{p}}_{𝒯}(Y\mid X=x)p_{{𝒯}_0}(x)dx\approx \frac{1}{n}\sum _{i=1}^n{\hat{p}}_{𝒯}(Y\mid X_i).$$

Algorithms that make the implicit assumption on covariate shift include some of the most widely adopted VA models, such as InterVA (Byass et al. (2012, 2019)), Tariff (Serina et al. (2015)), Naive Bayes Classifier (Miasnikof et al. (2015)), as well as many others in the literature applying off-the-shelf classification models on VA data (e.g., Blanco et al. (2020), Flaxman et al. (2011)). Models assuming covariate shift are easy to design and implement but difficult to justify in the context of VA. As pointed out by King and Lu (2008), symptoms collected by VA are usually the consequences of the underlying causes of death. Thus, the covariate shift assumption is almost always violated as $p_{{𝒯}_0}(Y\mid X)$ and $p_{𝒯}(Y\mid X)$ are different, even when $p(X\mid Y)$ is the same across domains.

The recent advancement of Bayesian modeling of VA data again returned to the formulation using the more natural decomposition of $p(X,Y)=p(Y)p(X\mid Y)$ to describe the data generating process. The first Bayesian hierarchical model for cause-of-death assignment, InSilicoVA (McCormick et al. (2016)), proposed to use noninformative priors to model $p_{{𝒯}_0}(Y)$ in the target population and strong informative priors to shrink $p_{{𝒯}_0}(X\mid Y)$ to the observed $p_{𝒯}(X\mid Y)$ calculated from training data or provided by domain experts. This framework was later extended to include more flexible characterizations of $p(X\mid Y)$, using latent Gaussian graphical models (Li, McComick and Clark (2020)) and factor models (Kunihama et al. (2020), Moran et al. (2021)). While Kunihama et al. (2020) and Moran et al. (2021) treated the cause of death of all observations as drawing from the same distribution, they can be readily adapted to have different $p(Y)$ in different populations.

The prior shift assumption, however, can still be overly simplistic in practice. Figure 1 illustrates an example from the PHMRC gold-standard VA dataset (Murray et al. (2011a)) discussed in Section 5. It shows the proportion of deaths with a “yes” response in two symptoms, having trouble breathing and drinking alcohol, conditional on the 10 causes of death with most observations, for each of the six study sites in the dataset. While it only illustrates the conditional distribution of two symptoms, strong domain heterogeneity can be observed from these site-specific empirical response probabilities $p(X\mid Y)$. Some of these reporting patterns may be explained by the presence of confounding factors (Moran et al. (2021)). However, we usually do not have enough data to identify or adjust for all potential confounders, many of which may be unobservable, or quantify the similarity of these distributions without reference cause of death. When $p(X\mid Y)$ vary across domains, the choice of training data could have strong implications for out-of-domain predictions.

Fig. 1.

Empirical proportion of “yes” response in two symptoms, drinking alcohol (left) and having trouble breathing (right), among deaths due to different causes of death in the six study sites in the PHMRC gold-standard dataset. Gray color indicates no collected death due to a given cause in a given site.

A line of research closely related to this paper is the calibration models developed in Datta et al. (2021) and Fiksel et al. (2021). The focus of these methods are fundamentally different from our work. They consider a different scenario where the cause of death is known for a small sample of observations ${𝒯}_0^{\prime }$ from the target population and assume that $p_{{𝒯}_0^{\prime }}(X\mid Y)=p_{{𝒯}_0}(X\mid Y)$, that is, only prior shift may exist between the labeled and unlabeled deaths from the same population, with potentially different $p_{{𝒯}_0^{\prime }}(Y)$ and $p_{{𝒯}_0}(Y)$. It can be shown that this assumption is sufficient to de-bias a classifier trained on $𝒯$ and applied to ${𝒯}_0$, using the labeled target population data ${𝒯}_0^{\prime }$. The advantage of such calibration methods is that they circumvent the need to identify and parameterize transportable component of the joint distribution from the training domains to the target domain. The cost, however, is the requirement for additional local data with known causes, which is usually impractical in resource-constrained settings. Moreover, the estimation of the error transition matrix of the predicted cause of death is challenging when there are many potential causes of death. For a small number of local labeled data, Datta et al. (2021) and Fiksel et al. (2021) recommend grouping the causes of death into a handful of broad categories in the analysis. It remains an open problem to estimate the CSMF for longer cause lists with limited calibration data.

The focus of this paper is to develop a general framework to characterize domain heterogeneity and leverage such information when assigning causes of death for out-of-domain observations without additional calibration data. Unlike any existing VA models, we allow both $p(Y)$ and $p(X\mid Y)$ to vary across domains. The key to our modeling approach is a latent class model representation (Goodman (1974)) that assumes source component shift for data across different domains (Storkey (2009)). That is, we assume that there is a latent variable $Z$ that renders $p_{𝒯}(X\mid Y,Z)=p_{{\tau }_0}(X\mid Y,Z)$. This assumption can be considered as an extension of prior shift on the latent space. It allows us to combine data from multiple domains with domain-specific $p(X,Y)$ efficiently when assigning cause of death for unlabeled deaths in a new domain. We discuss our proposed model in more details in the next section.

3. Nested latent class models for VA data.

In this section we present the nested latent class model for VA data, LCVA. We first introduce a single-domain latent class model that parsimoniously models the joint distribution of symptoms and causes and then extend it to modeling data from multiple domains.

3.1. The single-domain model.

Let $Y_i\in \{1,\dots ,C\}$ denote the categorical cause of death for the $i$th death and $X_i\in \{\mathrm{0,1}{\}}^p$ denote the $p$-dimensional reported binary symptom vector. The standard practice of VA usually involves preprocessing any continuous variables into binary indicators (Li, McComick and Clark (2020)). Thus, we focus our discussion on binary symptoms here, but the model can easily incorporate categorical symptoms of more than two levels, thus mitigating the potential loss of information in the preprocessing step. We begin by assuming that each cause of death can be further divided into $K$ subcategories and each individual is associated with a latent class $Z_i\in \{1,\dots ,K\}$. We assume reported symptoms are independent draws from a Bernoulli distribution, given both the cause of death $Y_i$ and the latent class membership $Z_i$, that is,

$$X_{ij}\mid Y_i=c,Z_i=k\sim \text{Bern}({\theta }_{ckj}),j=1,\dots ,p.$$

We treat the cause of death $Y=\left(Y_1,\dots ,Y_n\right)$ and latent class indicator $Z=\left(Z_1,\dots ,Z_n\right)$ as random variables and let, for $i=1,\dots ,n$,

$$\begin{gathered} Y_i\sim \text{Cat}(\mathit{\pi }), \\ {Z}_i\mid Y_i=c\sim \text{Cat}({\mathbf{\lambda }}_c). \end{gathered}$$

For deaths due to each cause of death, $p\left(X_i\mid Y_i=c\right)={\sum }_{k=1}^K{\mathrm{\lambda }}_{ck}{\prod }_{j=1}^p{\theta }_{ckj}^{x_{ij}}{\left(1-{\theta }_{ckj}\right)}^{1-x_{ij}}$. Conditional dependence among symptoms are introduced by marginalizing out the latent class indicators. With sufficiently many latent classes, this representation is flexible enough to represent any multivariate discrete distribution (Dunson and Xing (2009)). We treat the latent class $Z$ as defined within each cause of death, and both latent indicators $Z$ and $Y$ need to be estimated for a new death. This nested formulation allows the model to be sufficiently expressive for deaths from each cause.

The naive factorization of the joint distribution of $(X,Y,Z)$ requires $CK(p+1)-1$ parameters. This can be computationally challenging for VA data, as the number of symptoms $p$ and the size of cause list $C$ are usually large while sample size is typically small to moderate. We adopt a sparsity-inducing prior to model the response probabilities ${\theta }_{ckj}$ proposed by Zhou et al. (2015). We let

$$\begin{gathered} {\theta }_{ckj}={\delta }_{ckj}{\varphi }_{ckj}+(1-{\delta }_{ckj}){\gamma }_{cj},{\delta }_{ckj}\sim \text{Bern}({\tau }_c), \\ {\varphi }_{ckj}\sim \text{Beta}(1,v_{\varphi }),{\gamma }_{cj}\sim \text{Beta}(a_{\gamma },b_{\gamma }),{\tau }_c\sim \text{Beta}(1,v_{\tau }), \end{gathered}$$

where ${\delta }_{ckj}$ is a binary indicator specifying whether $p\left(X_{ij}\mid Y_i=c,Z_i=k\right)$ is affected by the latent class membership or takes the baseline response probability ${\gamma }_{cj}$. For each cause $c$, this prior encourages the associated latent classes $k=1,\dots ,K$ to have similar response profiles that differ only in a subset of symptoms where ${\delta }_{ckj}\ne 0$. Thus, it significantly improves the efficiency of the latent representation when $K$ is large. In the original proposal of Zhou et al. (2015), the baseline response vector is set to fixed values in advance. Here we model the baseline response probabilities ${\gamma }_{cj}$ in a data adaptive fashion, as the baseline distribution of $p\left(X_{ij}\mid Y_i=c\right)$ is generally unknown and hard to specify. In certain situations we may have external information on such conditional distributions in the form of physician provided domain knowledge (Li, McComick and Clark (2020), McCormick et al. (2016)), which may be used to construct informative priors on ${\gamma }_{cj}$. Here we use noninformative priors with ${v_{\varphi }=a}_{\gamma }=b_{\gamma }=1$.

We also expect deaths due to different causes to exhibit varying levels of complexity in the reported symptoms. For example, distribution of symptoms among deaths from external causes (e.g., drowning) might require fewer parameters to characterize than those from deaths due to infectious diseases. Instead of specifying different latent class size $K$ for each cause, the complexity of the latent symptom profiles are governed by the cause-specific sparsity level ${\tau }_c$. We use a noninformative prior on ${\tau }_c$ with $v_{\tau }=1$. We note that the sparsity induced by the mixture prior on $\mathit{\theta }$ does not perform variable selection but rather it aims to reduce the number of parameters by identifying symptoms that are conditionally independent of others. When $\left\{{\delta }_{c1j},\dots ,{\delta }_{cKj}\right\}$ are all 0’s for symptom $j$ and cause $c$, it implies that symptom $j$ is independent of all other symptoms, given cause $c$. The conditional independence model assumed by the InSilicoVA algorithm can be thought of as the special case where $\mathit{\delta }=0$.

To complete the hierarchical specification with priors for the latent parameters, we consider the stick-breaking prior for the mixing weights $\mathbf{\lambda }$ and a Dirichlet prior on $\mathit{\pi }$, that is,

$$\begin{array}{l}\begin{array}{cc}{\mathrm{\lambda }}_{ck}=V_{ck}\prod _{l<k}(1-V_{cl}),& V_{ck}\sim \text{Beta}(1,{\omega }_c),\end{array}\\ \begin{array}{cc}{\omega }_c\sim \text{Gamma}(a_{\omega },b_{\omega }),& \mathit{\pi }\sim \text{Dirichlet}(\mathit{\alpha }).\end{array}\end{array}$$

With a training dataset $𝒯=\left\{\left({\mathit{x}}_1,y_1\right),\dots ,\left({\mathit{x}}_n,y_n\right)\right\}$, the posterior predictive distribution for the cause of death of a single new data point $\left(X^{(0)},Y^{(0)}\right)$ from the same population can be readily computed by

$$p\left(Y^{(0)}=c\mid X^{(0)},𝒯\right)=\int p\left(Y^{(0)}=c\mid X^{(0)},\mathit{\pi },\mathbf{\lambda },\mathit{\theta }\right)p(\mathit{\pi },\mathbf{\lambda },\mathit{\theta }\mid 𝒯)d(\mathit{\pi },\mathbf{\lambda },\mathit{\theta }).$$

VAs are most commonly used to determine the CSMF of a target population. Consider a target dataset ${𝒯}_0=\left\{{\mathit{x}}_1^{(0)},\dots ,{\mathit{x}}_{n^{(0)}}^{(0)}\right\}$ with unknown cause of death; we assume the same data generating process with target-specific distributions of causes of death and latent classes. For $i=1,\dots ,n^{(0)}$, we let

$$\begin{gathered} Y_i^{(0)}\sim \text{Cat}\left({\mathit{\pi }}^{(0)}\right), \\ {Z}_i^{(0)}\mid Y_i^{(0)}=c\sim \text{Cat}\left({\mathbf{\lambda }}_c^{(0)}\right), \\ {X}_{ij}^{(0)}\mid Y_i^{(0)}=c,Z_i^{(0)}=k\sim \text{Bern}({\theta }_{ckj}). \end{gathered}$$

For the prior on the target CSMF, we let ${\mathit{\pi }}^{(0)}\sim \mathrm{Dirichlet}\left({\mathit{\alpha }}^{(0)}\right)$. We consider two different specifications for $p_{{𝒯}_0}(X\mid Y)$:

1. Constant weights: The most restrictive parameterization is to assume only prior shift exists across the domains. That is, we let $p_{{𝒯}_0}(X\mid Y)=p_{𝒯}(X\mid Y)$, or equivalently, ${\mathbf{\lambda }}^{(0)}=\mathbf{\lambda }$.

2. New weights: We may relax the prior shift assumption by allowing the target dataset to have a different composition over the latent classes learned in the training domain. In other words, we assume $p_{{𝒯}_0}(X\mid Y,Z)=p_{𝒯}(X\mid Y,Z)$ only. The new mixing weights can be estimated with the same prior as in the first stage, that is,

$${\mathbf{\lambda }}^{(0)}\sim V_{ck}^{(0)}\prod _{l<k}\left(1-V_{cl}^{(0)}\right),V_{ck}^{(0)}\sim \text{Beta}\left(1,{\omega }_c^{(0)}\right),{\omega }_c^{(0)}\sim \text{Gamma}(a_{\omega },b_{\omega }).$$

We note that the additional flexibility of the second predictive strategy may sometimes suffer from overfitting in the target domain when $K$ is large. With data from multiple domains, we could further estimate ${\mathbf{\lambda }}^{(0)}$ in a more robust fashion, as described in the next subsection.

3.2. The multidomain model.

When training data are collected from a single domain, only one set of mixing weights can be inferred. Thus, the mixing weights in the target domain are treated either as the same as those from the training domain or as independent random variables. When the training data consist of observations from multiple domains, however, we can leverage the estimated collection of mixing weights to improve out-of-domain prediction. Consider a training dataset consisting of observations from $G$ distinct domains. We use $D_i$ to denote the domain membership for the $i$th observation so that $D_i\in \{0,\dots ,G\}$ with $D_i=0$ indicating the target domain. The unified data generating process for all domains can be expressed as

$$\begin{gathered} Y_i\mid D_i=g\sim \text{Cat}\left({\mathit{\pi }}^{(g)}\right), \\ {Z}_i\mid Y_i=c,D_i=g\sim \text{Cat}\left({\mathrm{\lambda }}_c^{(g)}\right), \\ {X}_{ij}\mid Y_i=c,Z_i=k\sim \text{Bern}({\theta }_{ckj}). \end{gathered}$$

That is, we assume there are $K$ latent symptom profiles nested within each of the $C$ causes across all domains, but the distribution of the causes and latent classes are domain-specific. The domain-specific distributions of $p(X\mid Y)$ illustrated earlier can then be explained by different mixing over latent classes. Similar to the single-domain model, we put independent stick breaking priors on the domain-specific weights and Dirichlet priors for the CSMF, that is, for $g=1,\dots ,G$,

$$\begin{array}{l}\begin{array}{cc}{\mathrm{\lambda }}_{ck}^{(g)}=V_{ck}^{(g)}\prod _{l<k}\left(1-V_{cl}^{(g)}\right),& V_{ck}^{(g)}\sim \text{Beta}\left(1,{\omega }_c^{(g)}\right),\end{array}\\ \begin{array}{cc}{\omega }_c^{(g)}\sim \text{Gamma}(a_{\omega },b_{\omega }),& {\mathit{\pi }}^{(g)}\sim \text{Dirichlet}\left({\mathit{\alpha }}^{(g)}\right).\end{array}\end{array}$$

When performing cause-of-death assignment for a target domain without labeled death, the estimation of the mixing weights ${\mathbf{\lambda }}^{(0)}$ should ideally borrow information from the collection of mixing weights ${\left\{{\mathbf{\lambda }}^{(g)}\right\}}_{g=1,\dots ,G}$ in the training domains. For the target domain, we let ${\mathrm{\lambda }}_{ck}^{(0)}={\sum }_{g=1}^G{\eta }_{cg}{\mathrm{\lambda }}_{ck}^{(g)}$, that is, we assume the deaths due to each cause from the new domain can be modeled as a mixture of deaths due to the same cause from the observed domains. Compared to putting independent priors on ${\mathbf{\lambda }}^{(0)}$ and estimating it without constraints, this parameterization allows the cause-of-death classification to be less sensitive to small and noisy target domain data. We note that this modeling assumption can be relaxed if domain-level covariates are known and can be used to model domain similarities. Here we consider the more common case where no additional information about the target domain exists and leave covariate-based modeling to future work. Specifically, we consider the following two strategies to parameterize ${\mathbf{\lambda }}^{(0)}$, and a graphical model representation of the model is summarized in Figure 2:

Fig. 2.

Graphical model of our Bayesian nested latent model. Boxes indicate replication of random variables and shaded nodes the observations. The middle block correspond to observed and latent variables for each death. The right block correspond to the shared parameters describing the symptom profile collections. The left block correspond to parameters describing the site-level mixing of the latent symptom profiles.

1. Domain-level mixture: We fix ${\mathit{\eta }}_c=\mathit{\eta }$ to be the same for each cause $c=1,\dots ,C$ and let $\mathit{\eta }\sim \mathrm{Dirichlet}\left({\mathit{\alpha }}_{\eta }\right)$. That is, the relationship between the new mixing weights ${\mathbf{\lambda }}^{(0)}$ and the existing weights $\left\{{\mathbf{\lambda }}^{(1)},\dots ,{\mathbf{\lambda }}^{(G)}\right\}$ do not vary across causes of death.

2. Domain-cause-level mixture: The simple domain-level mixture formulation can be too restrictive when the cause-of-death distributions are unbalanced, as the symptom distributions in the target domain are assumed to be similar to the domains with large $\eta$ conditional on any cause, including those rarely observed in these domains. Instead, we can perform cause-specific partial pooling by letting ${\mathit{\eta }}_c\sim \mathrm{Dirichlet}\left({\mathit{\alpha }}_{\eta }^{(c)}\right)$. We specify the concentration parameters to be ${\mathit{\alpha }}_{\eta }^{(c)}={\mathit{\alpha }}_{\eta }{m}_{cg}$, where $m_{cg}$ is the fraction of deaths from domain $g$ among all deaths due to cause $c$. This domain-cause-level mixture encourages ${\mathbf{\lambda }}_c^{(0)}$ to be similar to the corresponding weights in domains with larger sample sizes for cause $c$. When domain-level covariates exist, ${\mathit{\alpha }}_{\eta }^{(c)}$ can also be parameterized as proportional to other domain-level similarity measures.

With a sufficient number of latent classes, both the single-domain and multidomain models can capture the data distribution in the training data equally well. The advantage of the multidomain model, however, is in the out-of-domain prediction, as ${\mathbf{\lambda }}^{(0)}$ can be estimated with more flexibility. The similarity parameter, $\mathit{\eta }$, quantifies the similarity between the target and existing training domains in terms of the decomposition of latent classes within causes, as illustrated in Section 5.

3.3. Practical considerations.

In the context of VA, $\mathit{\theta }$ serves as the role of symptom-cause-information (SCI) (Clark, Li and McCormick (2018)) on the latent space and is assumed to be shared across domains. In our implementation we fix the distribution of $\mathit{\theta }$ to its posterior distribution, given the training data $𝒯$ when modeling the VAs collected in the target domain. This modular procedure has several advantages. First, it prevents the information on $\mathit{\theta }$ from flowing from the target domain to the training domains, as the target domain typically have few or no labeled deaths. This type of modularized procedure is known as “cutting the feedback” (Plummer (2015)) and has been studied extensively in different applications (Cucala et al. (2009), Jacob et al. (2017), Lunn et al. (2009), Pompe and Jacob (2021), Zigler et al. (2013), e.g.). In addition, a naive joint model likely will lead to the unlabeled deaths forming their own mixtures away from the latent classes in the training domains (Adams and Ghahramani (2009)). Second, as the information from the training domains can be summarized in the posterior distributions of $(\mathit{\theta },\mathbf{\lambda })$, users of the algorithm do not need to have access to the entire training dataset, which could be subject to data sharing restrictions. Third, the modular approach allows us to improve the exploration of parameter space by running multiple chains in parallel in the training stage, leading to a better approximation of $p(\mathit{\theta }\mid 𝒯)$ and reduced computational cost for practitioners.

Lastly, data collected through VA inevitably contain a high proportion of missing responses due to the respondents not knowing whether certain symptoms exist or not being willing to talk to the interviewer about them. Some of the missing symptoms can be deterministically inferred by the logic of the questionnaire, whereas most of them are difficult to impute systematically. We assume these missing responses are missing at random, that is, the probability of the missing data mechanism depends on observed data but not on missing values. This is a standard assumption made by existing VA algorithms (Kunihama et al. (2020), McCormick et al. (2016), Moran et al. (2021)) and allows us to avoid building another imputation model, which is extremely challenging to specify and validate in practice. Under the missing at random assumption, we can conduct inference on model parameters using only the observed data. More discussions on this assumption can be found in Kunihama et al. (2020).

3.4. Related work outside of VA.

Learning from data collected in different domains is an active area of research in machine learning and has been explored in various applications including natural language processing (Ramponi and Plank (2020)), visual classification (Wang and Deng (2018)), sentiment prediction (Glorot, Bordes and Bengio (2011)), and more recently in prediction problems in public health and clinical settings (Laparra, Bethard and Miller (2020), Mhasawade, Rehman and Chunara (2020), Rehman et al. (2018)). The task we consider in this paper is usually categorized as unsupervised domain adaptation (UDA), where no labeled data exist in the target domain. In particular, our estimation framework can be seen as a domain generalization method (Muandet, Balduzzi and Schölkopf (2013)), as the training stage does not involve the target domain data. A comprehensive review of UDA can be found in Wilson and Cook (2020). Most of the UDA methods rely on creating domain-invariant representations of the features (e.g., Shen et al. (2022), Tzeng et al. (2014)). There is extensive literature focusing on using deep learning to extract such representation in computer vision and natural language processing, but these methods are less appropriate in the context of VA, as they require extensive training data (Oquab et al. (2014)). Instead, we take a flexible parametric approach and learn a parsimonious and interpretable symptom representation that is shared across domains.

Bayesian hierarchical models are routinely used to model heterogeneous, multidomain data in many scientific fields, though less common in the domain adaptation literature. Bruzzone and Prieto (2001) and Raghuram, Miller and Kesidis (2012) proposed Gaussian mixture models to learn the data representation of training domains and apply to target domains in remote sensing. Wood and Teh (2009) proposed nonparametric Bayesian methods for domain adaptation in language models. Hajiramezanali et al. (2018) and Boluki, Qian and Dougherty (2021) developed Bayesian hierarchical models for modeling RNA sequencing data from multiple domains. While being tailored for different applications, the common conceptual framework is to model the data distribution in different domains as mixtures of shared latent clusters, an insight that we also use in building our proposed latent class model.

4. Posterior inference.

We have the following posterior joint distribution, given the training dataset $𝒯=\left\{\left({\mathit{x}}_1,y_1,d_1\right),\dots ,\left({\mathit{x}}_n,y_n,d_n\right)\right\}$:

$$\begin{gathered} p(\mathit{\pi },\mathbf{\lambda },\mathit{\varphi },\mathit{\gamma },\mathit{\omega }\mid 𝒯)\propto \prod _{i=1}^{n}p\left(y_i\mid {\mathit{\pi }}^{(d_i)}\right)\sum _{z_i=1}^K\left(p\left(z_i\mid y_i,{\mathbf{\lambda }}^{(d_i)}\right)\prod _{j=1}^{p}p(x_{ij}\mid y_i,z_i,\mathit{\gamma },\mathit{\varphi },\mathit{\delta })\right) \\ \times \prod _{c=1}^C\prod _{j=1}^{p}p({\gamma }_{cj})\prod _{c=1}^C\prod _{k=1}^K\prod _{j=1}^{p}p({\varphi }_{ckj})p({\delta }_{ckj}\mid {\tau }_c)\prod _{c=1}^{C}p({\tau }_c) \\ \times \prod _{g=1}^{G}p\left({\mathit{\pi }}^{(g)}\right)p\left({\mathbf{\lambda }}^{(g)}\mid {\mathit{\omega }}^{(g)}\right)p\left({\mathit{\omega }}^{(g)}\right). \end{gathered}$$

The posterior distribution of the single-domain model is a special case with $G=1$ and thus omitted. In the target domain, the modularized joint posterior distribution is

$$\begin{gathered} p(Y^{(0)},{\mathit{\pi }}^{(0)}\mid X^{(0)},𝒯)\propto \int \prod _{i=1}^{n^{(0)}}p\left(y_i^{(0)}\mid {\mathit{\pi }}^{(0)}\right)\sum _{z_i^{(0)}=1}^K\left(p\left(z_i^{(0)}\mid y_i^{(0)},{\mathbf{\lambda }}^{(0)}\right)\prod _{j=1}^{p}p(x_{ij}^{(0)}\mid y_i^{(0)},z_i^{(0)},\mathit{\theta })\right) \\ \times p\left({\mathit{\pi }}^{(0)}\right)p\left({\mathbf{\lambda }}^{(0)}\mid {\mathbf{\lambda }}^{(1)},\dots ,{\mathbf{\lambda }}^{(G)},\mathit{\eta }\right)p(\mathit{\eta }) \\ \times p\left({\mathbf{\lambda }}^{(1)},\dots ,{\mathbf{\lambda }}^{(G)},\mathit{\theta }\mid 𝒯\right)d\left(\mathit{\eta },{\mathbf{\lambda }}^{(0)},\dots ,{\mathbf{\lambda }}^{(G)},\mathit{\theta }\right). \end{gathered}$$

The posterior distribution of the model parameters is not available in closed form, but we can easily draw posterior samples from a Gibbs sampler. The detailed procedure of fitting the model is described next.

4.1. Training stage.

We first infer the parameters associated with the training data using the following steps:

1. Sample the latent class membership $Z_i$ for $i\in 𝒯$ with

$$p(Z_i=k\mid Y_i=c,D_i=g,\mathit{\theta },\mathbf{\lambda })\propto {\mathrm{\lambda }}_{ck}^{(g)}\prod _j{\theta }_{ckj}^{x_{ij}}{(1-{\theta }_{ckj})}^{1-x_{ij}}.$$

2. Sample the stick-breaking parameters ${\mathit{V}}^{(g)}$ and ${\omega }^{(g)}$ for $g=1,\dots ,G$. Denote $n_{ck}^{(g)}={\sum }_i{1}_{Y_i=c,Z_i=k,D_i=g}$. The full conditional is

$$\begin{gathered} V_{ck}^{(g)}\mid Y,Z,{\mathit{\omega }}^{(g)}\sim \text{Beta}\left(1+n_{ck}^{(g)},{\omega }_c^{(g)}+\sum _{l>k}{n}_{cl}^{(g)}\right),c=1,\dots ,C;k=1,\dots ,K-1, \\ {\omega }_c^{(g)}\mid {\mathit{V}}^{(g)}\sim \text{Gamma}\left(a_{\omega }+K-1,b_{\omega }-\sum _{k=1}^{K-1}\text{log}\left(1-V_{ck}^{(g)}\right)\right),c=1,\dots ,C. \end{gathered}$$

3. Sample the response probabilities $\mathit{\theta }$ after integrating out the latent binary indicators $\mathit{\delta }$. The posterior conditional of ${\theta }_{ckj}$ is a mixture of Beta distribution and point mass at the baseline vector ${\gamma }_{cj}$. We let $n_{ckj1}={\sum }_i{1}_{Y_i=c,Z_i=k,X_{ij}=1}$ and $n_{ckj0}={\sum }_i{1}_{Y_i=c,Z_i=k,X_{ij}=0}$. The posterior conditional of ${\theta }_{ckj}$ is

$$p({\theta }_{ckj}\mid \cdot )={\tilde{\pi }}_{ckj}\text{Beta}(1+n_{ckj1},v_{\varphi }+n_{ckj0})+(1-{\tilde{\pi }}_{ckj})I({\gamma }_{cj}),$$

where $I\left({\gamma }_{cj}\right)$ is the point mass at ${\gamma }_{cj}$. The mixing probability is

$${\tilde{\pi }}_{ckj}=\frac{{\tau }_{c}B(1+n_{ckj1},v_{\varphi }+n_{ckj0})}{{\tau }_{c}B(1+n_{ckj1},v_{\varphi }+n_{ckj0})+(1-{\tau }_c)B(1,v_{\varphi }){\gamma }_{cj}^{n_{ckj1}}{(1-{\gamma }_{cj})}^{n_{ckj0}}}.$$

4. Sample the latent binary indicators $\delta$ from its full conditional,

$${\delta }_{ckj}\sim \text{Bern}({\tilde{\pi }}_{ckj}).$$

5. Sample the sparsity level parameter ${\tau }_c$,

$${\tau }_c\sim \text{Beta}\left(1+\sum _j\sum _k{\delta }_{ckj},v_{\tau }+\sum _j\sum _k(1-{\delta }_{ckj})\right).$$

6. Sample the baseline vector $\mathit{\gamma }$ from its full conditionals. Let $O_j=\{i,x_{ij}\text{not missing}{\}}_{i=1,\dots ,n}$, we have

$${\gamma }_{cj}\mid X,Y,Z,\mathit{\delta }\sim \text{Beta}\left(a_{\gamma }+\sum _{i\in O_j}{x}_{ij}(1-{\delta }_{y_i{z}_i}j),b_{\gamma }+\sum _{i\in O_j}(1-x_{ij})(1-{\delta }_{y_i{z}_{i}j})\right).$$

7. Update the CSMF vector in each domain ${\mathit{\pi }}^{(g)}\mid Y$. Let $n_c^{(g)}={\sum }_i{1}_{Y_i=c,D_i=g}$,

$${\mathit{\pi }}^{(g)}\mid Y\sim \text{Dirichlet}\left({\alpha }_{\pi ,1}+n_1^{(g)},\dots ,{\alpha }_{\pi ,C}+n_C^{(g)}\right).$$

When there exist observations from the training domains with unknown labels, the above posterior sampling procedure can be easily modified by updating the cause-of-death assignment and latent class indicator for each unlabeled data with

$$p(Y_i=c,Z_i=k\mid X_i,D_i=g)\propto {\pi }_c^{(g)}{\mathrm{\lambda }}_{ck}^{(g)}\prod _j{({\theta }_{ckj})}^{x_{ij}}{(1-{\theta }_{ckj})}^{1-x_{ij}}.$$

4.2. Prediction stage.

When performing cause-of-death assignment for the target dataset, we take the modular approach described in Section 3.3 by plugging in the posterior draws of $\mathit{\theta }$ and $\mathbf{\lambda }$ from the training stage model. We describe the sampler for the other parameters in the multidomain model below and leave the case of single-domain model to the Supplementary Material (Li et al. (2023)):

1. Sample the cause-of-death assignments $Y_i$ and latent class indicator $Z_i$. To facilitate easier computation, we augment the data with latent indicator ${\tilde{D}}_i\in \{1,\dots ,G\}$ for observations from the target population and sample all latent indicators jointly with

$$p(Y_i=c,Z_i=k,{\tilde{D}}_i=g\mid X_i,D_i=0,\mathbf{\lambda },\mathit{\theta })\propto {\pi }_c^{(0)}{\eta }_{cg}{\mathrm{\lambda }}_{ck}^{(g)}\prod _j{({\theta }_{ckj})}^{x_{ij}}{(1-{\theta }_{ckj})}^{1-x_{ij}}.$$

2. For the multidomain model with domain-level mixture, update the domain similarity weights $\mathit{\eta }$ with

$$\mathit{\eta }\mid Y,\tilde{D}\sim \text{Dirichlet}\left({\alpha }_{\eta ,1}+\sum _i{1}_{{\tilde{D}}_i=1},\dots ,{\alpha }_{\eta ,G}+\sum _i{1}_{{\tilde{D}}_i=G}\right).$$

For the multidomain model with domain-cause-level mixture,

$${\mathit{\eta }}_c\mid Y,\tilde{D}\sim \text{Dirichlet}\left({\alpha }_{\eta ,1}{m}_{c1}+\sum _i{1}_{Y_i=c,{\tilde{D}}_i=1},\dots ,{\alpha }_{\eta ,G}{m}_{cG}+\sum _i{1}_{Y_i=c,{\tilde{D}}_i=G}\right).$$

3. Update the CSMF vector in the target domain ${\mathit{\pi }}^{(0)}\mid Y$ with

$${\mathit{\pi }}^{(0)}\mid Y\sim \text{Dirichlet}\left({\alpha }_{\pi ,1}^{(0)}+n_1^{(0)},\dots ,{\alpha }_{\pi ,C}^{(0)}+n_C^{(0)}\right),$$

where $n_c^{(0)}={\sum }_i{1}_{Y_i=c,D_i=0}$.

4.3. Computational consideration.

The latent class model formulation leads to a multimodal posterior distribution that could be difficult to explore using MCMC in practice. One of the reasons is likelihood invariance to permutation of class labels. When we model the mixing weights in the target domain as a weighted average of the weights in the training domains, however, the model parameters for the target site do not depend on the labeling of latent classes, making the computation in the prediction stage invariant to the latent class labels. This allows us to implement the training stage sampler multiple times in parallel with different starting values and combine the posterior draws of $(\mathit{\theta },\mathit{\lambda })$ afterward. To further improve the predictive performance, we adopt the stacking approach proposed by Yao, Vehtari and Gelman (2022) to combine the multiple MCMC chains.

Finally, we note that the sampler proposed in this section enjoys significant advantage over models that use latent Gaussian representations to characterize symptom dependence (Li, McComick and Clark (2020), Kunihama et al. (2020), Moran et al. (2021)). The classification stage of LCVA essentially reduces to allocating observations to CK or CKG latent classes with conditional independent response probabilities, which has the same order of computational complexity as the InSilicoVA algorithm (McCormick et al. (2016)).

5. Results.

In this section we present results of our model evaluated on the PHMRC gold-standard VA dataset (Murray et al. (2011a)). The PHMRC dataset has been used extensively in validating and comparing VA cause-of-death assignment methods (Clark, Li and McCormick (2018), Kunihama et al. (2020), McCormick et al. (2016), Moran et al. (2021)). It consists of 7841 adult deaths collected from six study sites (Andhra Pradesh, India; Bohol, Philippines; Dar es Salaam, Tanzania; Mexico City, Mexico; Pemba Island, Tanzania; and Uttar Pradesh, India). All deaths occurred in health facilities and gold-standard causes are determined based on laboratory, pathology, and medical imaging findings. The cause of death is coded into 34 categories, and we preprocess the raw dataset into 168 binary symptoms, as described in McCormick et al. (2016). The data processing steps are implemented using the openVA package (Li et al. (2023)).

We present three sets of experiments in this section. We first evaluate the out-of-domain predictive performance of different models for a leave-one-site-out experiment. To further explore how these methods behave under more extreme data shift, we then construct resampled datasets based on each target domain, and evaluate the models on these replicated datasets. Finally, we consider the case where some local labeled data exist in the target domain and compare different approaches to incorporate the labeled data. In addition to the evaluation of predictive performance, we also compare computational time of different methods and discuss the estimated latent symptom profiles. We also perform comparisons on synthetic data and results of these simulations are included in the Supplementary Material (Li et al. (2023)).

5.1. Prediction for a new study site.

For our first evaluation, we consider the task of out-of-domain prediction for each of the six study sites. We iteratively take one site as the target domain and the rest of the five sites as training domains. A summary of the true CSMFs for each site is included in the Supplementary Material (Li et al. (2023)).

Throughout all the experiments in this paper, we use the same six sets of training data, each including five sites. Since our goal is to develop models that are robust to unseen target domains, we should choose $K$ based solely on the training data. To determine the value of $K$, we first fit the multidomain model with a large $K$ and truncate the latent class rarely occupied. For each experiment we first let $K=15$ and monitor the number of latent classes occupied in the posterior samples of $Z_i$. Figure 3 plots the fraction of posterior samples where each latent class is occupied for each cause of death in each experiment. The latent classes after the largest 10 classes are occupied in less than 5% of posterior samples across almost all causes and experiments. Therefore, we let $K=10$ in all our subsequent analysis of the PHMRC data. The performance of LCVA does not vary much from different choices of $K$. Additional sensitivity analysis are provided in the Supplementary Material (Li et al. (2023)).

Fig. 3.

The fraction of latent classes utilized for each experiment. The latent classes are sorted by the fraction of posterior draws where they are occupied. Each line represents one cause of death, and each panel represents a training dataset where the target site in the label is excluded.

We consider both the multidomain (LCVA-M) models in Section 3.2 and the single-domain (LCVA-S) models in Section 3.1 by ignoring the domain indicators in the training data. For the concentration parameters, we let ${\mathit{\alpha }}_{\pi }={\mathit{\alpha }}_{\pi }^{(0)}={\mathit{\alpha }}_{\eta }=1$. In the training stage, we ran six parallel MCMC chains with different starting values for 4000 iterations and discarded the first 1000 posterior draws as burn-in. For LCVA-S with new weights, the posterior draws from a single chain were used in the prediction stage. For the other models, we reweighted the six chains using stacking (Yao, Vehtari and Gelman (2022)) and generated 4000 posterior samples from the combined posterior distribution. Finally, the first half of the chain in the prediction stage were discarded as burn-in.

We evaluated the performance of the proposed model based on the classification accuracy of the predicted top cause of death and the so-called “CSMF accuracy” (Murray et al. (2011b)), a widely used metric to compare the estimated CSMF vector with the truth. The CSMF accuracy metric is a form of the normalized absolute error (González et al. (2017)). It measures the $L_1$ distance between the estimated and true CSMF and is normalized to range between 0 (worst) and 1 (best). It is defined as ${\mathrm{C}\mathrm{S}\mathrm{M}\mathrm{F}}_{acc}(\hat{\pi })=1-\frac{{\sum }_{c=1}^C\left|{\hat{\pi }}_c-{\pi }_c\right|}{2\left(1-{\mathrm{m}\mathrm{i}\mathrm{n}}_c{\pi }_c\right)}$, where ${\pi }_c$ is the true CSMF calculated by the empirical distribution in the test data. Unlike the top cause accuracy that focuses on individual level performance, CSMF accuracy evaluates the task of prevalence estimation, which is more informative for VA practitioners, especially in unbalanced datasets.

We benchmarked the performance of the proposed model to the InSilicoVA algorithm. InSilicoVA is one of the most widely adopted VA methods used in practice and has been shown to have better or comparable performance compared to other VA models in use (Clark, Li and McCormick (2018), McCormick et al. (2016)). We also compared with the Bayesian factor model (Kunihama et al. (2020)) and the FARVA model (Moran et al. (2021)). Both models approximate the joint distribution of symptoms using a latent Gaussian model and have been shown to significantly improve the predictive performance compared to InSilicoVA. For the FARVA model, we included a binary covariate indicating age at death exceeding 65. The inclusion of covariates did not change the performance metric of FARVA by much in our experiments. We follow the recommendation by the authors of these methods in setting the tuning parameters and length of MCMC iterations. For the InSilicoVA algorithm, we ran the MCMC for 4000 iterations. For the Bayesian factor model, we used six latent factors and obtained 1000 posterior samples after discarding the first 500 samples as burn-in and saving every fifth sample. Far FARVA we used 10 latent factors and obtained 500 posterior samples after discarding the first 2000 samples as burn-in and saving every tenth sample.

Figure 4 summarizes the top cause accuracy and CSMF accuracy for the different models evaluated on the six sites, respectively. All models show significantly improved top cause accuracy compared to InSilicoVA. The CSMF accuracy of the three classes of models are also comparable and improves from the InSilicoVA results, except in two cases (FARVA in Bohol and LCVA in Andhra Pradesh). There is no model that dominates either predictive metrics in all six experiments.

Fig. 4.

Individual-level top cause prediction accuracy (left) and CSMF accuracy (right) of different models. The cells are colored by the relative difference from the InSilicoVA results, where blue cells indicate higher accuracy and red cells indicate lower accuracy. All variations of LCVA demonstrate good performance that is significantly higher than InSilicoVA, except in one experiment. Bayesian factor model and FARVA also show comparable improvements.

We note, however, that an important caveat in this experiment is that the CSMFs in most sites are similar in the PHMRC dataset, except for Pemba. A comparison of the empirical CSMFs are included in the Supplementary Material (Li et al. (2023)). Both the Bayesian factor model and FARVA assume a common data generating process for causes of deaths in both training and target datasets with a shared parameter $\mathit{\pi }$ and thus enjoy stronger regularization from the prior compared to LCVA. However, such model formulation can negatively affect the predictive performance in domains with a very different cause-of-death distribution, compared to the training domains such as Pemba in this experiment. We further evaluate the performance of the cause-of-death assignment models under more extreme scenarios in the next subsection.

5.2. Prediction for a new study site under more extreme data shift.

In order to evaluate the models under more extreme data shift, for each leave-one-site-out experiment, we created 50 synthetic target dataset by first generating ${\pi }^{(0)}\sim \mathrm{Dirichlet}(0.1,\dots ,0.1)$ and resampled deaths in the held-out site with replacement to match the generated target CSMF. This data generating process leads to the target CSMFs being concentrated on a small number of causes and thus very different from the overall prevalence in the training domains. This reflects more closely the cause-of-death distributions in small- and moderate-scale VA studies in practice. We again compare our methods to InSilicoVA, Bayesian factor model, and FARVA. The top cause accuracy and CSMF accuracy metrics vary significantly across replications due to the high variability of the target domain CSMF. Thus, we focus on the relative performances of the models. Figures 5 and 6 show the pairwise comparison of all models in terms of the fraction of times one model outperform the other in the two metrics across the six collection of target datasets. The simplest model, InSilicoVA, led to worse top cause accuracy estimation in almost all pairwise comparisons. However, in terms of the CSMF accuracy, which is more of interest for domains with unbalanced cause-of-death distribution, both Bayesian factor model and FARVA performed worse than InSilicoVA on more than half of the resampled datasets in four out of six sites. LCVA-M with domain-cause-level mixture is the only method that outperformed InSilicoVA in both metrics on more than half of the resampled datasets across all six sites, and it outperforms all three existing methods in CSMF accuracy on more than half of the resampled datasets. LCVA-M with domain-level mixture also perform very similarly. This demonstrates the robustness of the proposed models to extreme data shift.

Fig. 5.

Pairwise comparison of all models in terms of top cause accuracy. The color of each cell represents the proportion of target datasets where the model on the left achieves higher top cause accuracy than the model on the bottom.

Fig. 6.

Pairwise comparison of all models in terms of CSMF accuracy. The color of each cell represents the proportion of target datasets where the model on the left achieves higher CSMF accuracy than the model on the bottom.

We further take a closer examination of the scale of the relative improvements of the proposed models. Figure 7 shows the percentage improvement of each model $m$ compared to the Bayesian factor model, that is, $\left({\mathrm{A}\mathrm{c}\mathrm{c}}_m-{\mathrm{A}\mathrm{c}\mathrm{c}}_{\mathrm{B}\mathrm{F}}\right)/{\mathrm{A}\mathrm{c}\mathrm{c}}_{\mathrm{B}\mathrm{F}}$ for both metrics. In Dar es Salaam and Pemba island, LCVA-M had similar performances compared to the Bayesian factor model. In the other four sites, LCVA-M models achieved around 10% to 40% in median improvements for top cause accuracy and 10% to 20% for CSMF accuracy. FARVA also outperformed the Bayesian factor model in five sites in terms of individual-level prediction, but showed worse CSMF accuracy across all sites. Additional comparisons are included in the Supplementary Material (Li et al. (2023)).

Fig. 7.

Boxplot of improvement over the Bayesian factor model (Kunihama et al. (2020)) in top cause accuracy (left) and CSMF accuracy (right) by different models on 50 resampled target domain datasets based on deaths from each of the six sites.

Based on these two set of experiments and the synthetic data evaluations in the Supplementary Material (Li et al. (2023)), LCVA-M with domain-level mixture was shown to be more robust to different data shift scenarios while enjoying the more parsimonious parameterization than the domain-cause-level mixture model. Thus, we focus our attention on this model in the rest of the paper.

5.3. Computation time.

The proposed model is highly efficient in computation compared to the Bayesian factor model and FARVA. Across the six leave-one-site-out prediction experiments, the InSilicoVA algorithm is the fastest to implement, as the symptoms are treated as conditionally independent. It required 20 seconds to run 1000 iterations of MCMC.

For the LCVA with domain-level mixture, every 1000 iterations of MCMC required 142 seconds in the training stage and 42 seconds in the prediction stage. In comparison, for the same tasks and 1000 iterations of the MCMC, the Bayesian factor model used 4275 seconds, and the FARVA model used 17,422 seconds. Data analysis using these two methods takes hours or days to finish, which is clearly infeasible as routine tools in resource-constrained settings. The LCVA requires the least additional computational cost among the methods that take into account symptom dependence. The computation time are all evaluated on a MacBook Pro with 2.6 GHz 6-Core Intel Core i7 processor and 32 GB memory.

5.4. Interpreting the latent symptom profiles.

We now turn to the estimated latent classes and domain similarity parameters. As an example, we examine the experiment treating Pemba island as the target domain and the other five larger sites as training domains. As an illustration we focus on the multidomain LCVA with domain-level mixture model. It is worth noting that the estimated latent classes are subject to permutation and further research is needed to relate latent classes with observable characteristics of the causes of death. Nevertheless, they still reveal interesting latent structures in these data. The top panel of Figure 8 shows the estimated conditional probabilities, $\mathit{\theta }$. We sort the symptoms by their largest conditional probabilities given a latent class, that is, ${\mathrm{m}\mathrm{a}\mathrm{x}}_k{\theta }_{ckj}$, and show the conditional probabilities of the top 20 symptoms for the six most prevalent causes in Pemba. Within each cause of death, most of the symptoms have similar conditional probabilities across latent classes, as encouraged by the prior on $\theta$. Taking deaths due to “other Non-Communicable Diseases (NCD)” as an example, the latent symptom profiles differ most significantly on the response probabilities for the sex of the respondent, injury, and specific symptoms such as convulsion, chest pain, having trouble breathing, and cough. The corresponding mixing weights are shown in the lower left panel of Figure 8. Mexico City and Dar es Salaam have more different latent class decomposition than the other sites. For example, comparing deaths due to other NCD in Dar es Salaam with the other sites, the first and third latent classes were utilized less often, compared to the second and fourth, indicating that symptoms, such as having trouble breathing, are less likely to be present for deaths due to otherNCDs compared to other sites. We note, however, that such differences could be due to either different decomposition of specific NCDs or differential reporting across sites, and more studies are needed to understand their underlying mechanism. We also observe that for simpler causes, such as drowning and falling, the latent symptom profiles are heavily shrunk toward the baseline values, indicating that the conditional independence simplification being a useful approximation for these causes. The shrinkage protects the model from overfitting on the noise in the training domains when $K$ is large for a relatively simpler causes. Finally, the estimated $\mathit{\eta }$ in the lower right panel of Figure 8 shows how ${\mathbf{\lambda }}^{(0)}$ is related to the existing mixing weights through the similarity parameter $\mathit{\eta }$. The domain with the most similar mixing weights to Pemba is estimated to be Dar es Salaam, which is as expected given both sites are in Tanzania.

Fig. 8.

Top: Posterior means of the conditional probabilities of observing the selected subset of symptoms given the top six causes of death in Pemba and latent class membership. Bottom: Posterior means of the mixing weights of latent classes by domain (left) and the posterior distribution of the similarity parameter $\mathit{\eta }$. Pemba island is the target domain. The mixing weights corresponding to drowning in Mexico city are removed since there is no drowning deaths in Mexico city in this dataset.

5.5. Incorporating labeled data from the target domain.

Finally, we consider the situation where a small subset of labeled data is available in the target domain. In this case the target population ${𝒯}_0=ℒ\cup 𝒰$, where $ℒ$ are the data with labels and $𝒰$ are the data without labels. The LCVA model can incorporate such information naturally by modeling all deaths in ${𝒯}_0$ and skipping the sampling of $Y_i$ for the deaths in $ℒ$. This leads to a joint modeling approach in the prediction stage in a semisupervised fashion. An alternative strategy is to calibrate the estimated CSMF to the known labels after fitting a model on only the training data and use $ℒ$ to estimate the misclassification matrix and de-bias the estimated $p_{{𝒯}_0}(Y)$ (Fiksel et al. (2021)).

Here we again consider the six experiments of out-of-domain prediction in Section 5.1. For the calibration step, we took the posterior means of the estimated individual cause-of-death probabilities from InSilicoVA and LCVA-M with domain-level mixture as the input to the calibration procedure in Fiksel et al. (2021) and randomly selected 30% of deaths in the target domain to reveal the true labels.

For each randomly selected set of labeled deaths, we compared the three estimates of the CSMFs: calibrated InSilicoVA, calibrated LCVA, and LCVA with partial labels treated as known in the prediction stage. The results over 50 randomly sampled $ℒ$ are summarized in Figure 9. With the additional information of the known labels, the CSMF accuracy improves for both InSilicoVA and LCVA. By directly incorporating the deaths with known causes in the joint modeling approach, we were able to obtain the highest CSMF accuracy in all six sites using LCVA.

Fig. 9.

Comparison of LCVA-M with domain-level mixture and InSilicoVA when the cause of death of 30% of data in the target domain are known. The horizontal line indicates the CSMF accuracy of the two methods before calibrating to the labeled deaths. The box plot shows the distribution of the CSMF accuracy over 50 resampled labeled dataset.

We note, however, that the calibration approach in Fiksel et al. (2021) were developed for a much smaller cause list. The calibration procedure is less effective in this analysis due to the need of estimating a $34\times 34$ dimensional error transition matrix using a small number of labeled data. In applications where only broader cause categories are of interest, the calibration procedure can sometimes perform similarly well compared to the joint modeling approach. We experimented with calibration using aggregated 11 and five broader causes in the Supplementary Material (Li et al. (2023)).

6. Discussion.

Quantifying the population distribution of causes of death and assigning causes to specific deaths using VAs are challenging, especially when using information derived from nonlocal data sources. In this paper we proposed a statistical framework for modeling VA data collected over multiple domains and quantifying the cause-of-death distributions for deaths in a new domain. The proposed model framework uses a parsimonious nested latent class model to characterize the joint distributions of symptoms, given causes, which allows direct interpretation of the latent structure and faster computation relative to existing methods. We demonstrated how this framework can efficiently borrow information from multiple domains and improve out-of-domain predictive performance significantly, with both synthetic and real data, and recommend a parsimonious prediction framework where we treat the mixing weights of the target domain to be a weighted average of the those in the training domains (i.e., LCVA-M with domain-level mixture).

The proposed method could be extended in a few different ways. First, we have treated the domains as independent datasets in this paper. In many situations data are collected over different subnational regions and over time. Structural information about spatial and temporal correlation as well as domain-level similarities could further improve the modeling of domain-specific distributions, as cause of death and latent class distributions are likely to vary smoothly across similar domains. Summarizing domain heterogeneity and similarities using the latent class model can also be of scientific interest to understand the underlying population. In addition, symptom distributions conditional on similar causes of death may also show similarities. Tree-based methods for domain adaptation can be a fruitful direction to explore such additional information (Wu et al. (2021)). Second, it is an important question to evaluate the discriminative power of VA questions in assigning causes and to identify the questions that do no contribute to cause assignments across different domains. This can facilitate the design of shorter VA questionnaires in the future. The proposed framework can be extended to perform symptom selection by further shrinking the response probabilities across different causes of death to the same values. Third, while our model focuses on the prediction task, the discovery and inference of interpretable latent classes shared across different populations alone is a topic of significant practical interest (De Vito et al. (2019)). It is a challenging task in the VA context, as the collected symptoms could be measured inconsistently across domains and subject to reporting biases. This is related to the discussion of transportability in the causal inference literature (Ackerman, Siddique and Stuart (2019), Wu et al. (2019)).

Finally, we conclude by highlighting two additional open questions. First, the proposed model relies on reference deaths with known causes of death from multiple training domains. A common practice of collecting labeled VA data is by having physician experts read and assign a cause best describing the death. The assigned causes of death are treated as the ground truth. However, many factors could contribute to the misclassification of deaths and lead to noise in these training labels. In particular, when a new disease emerges and limited information about the disease is known, such as during the early periods of the COVID-19 pandemic, higher rate of misclassification may be expected from the training labels. A key challenge, therefore, will be to identify latent classes that correspond to misclassified or unseen causes. Methods dealing with label noises in classification tasks (Liu (2021), Liu and Guo (2020)) could potentially improve cause-of-death assignment models and produce more realistic characterization of uncertainties. Second, a grand challenge with VA is to embed the analysis framework into the mortality surveillance system to detect changes and abnormalities. Modeling the temporal drift of the data distribution is highly useful for surveillance purposes. Covariate-dependent modeling (Moran et al. (2021)) is also important in this context, as additional information from both the population mortality surveillance and individual deaths could further help understand transferability and explain changes in mortality profile. We leave these topics for future work.