Direct-Assisted Bayesian Unit-level Modeling for Small Area Estimation of Rare Event Prevalence

Abstract

Small area estimation using survey data can be achieved by using either a design-based or a model-based inferential approach. Design-based direct estimators are generally preferable because of their consistency, asymptotic normality, and reliance on fewer assumptions. However, when data are sparse at the desired area level, as is often the case when measuring rare events, these direct estimators can have extremely large uncertainty, making a model-based approach preferable. A model-based approach with a random spatial effect borrows information from surrounding areas at the cost of inducing shrinkage. As a result, estimates may be over-smoothed and inconsistent with design-based estimates at higher area levels when aggregated. We propose two unit-level Bayesian models for small area estimation of rare event prevalence which use design-based direct estimates at a higher area level to increase consistency in aggregation. This model framework is designed to accommodate sparse data obtained from two-stage stratified cluster sampling, which is particularly relevant to applications in low- and middle-income countries. After introducing the model framework and its implementation, we conduct a simulation study to evaluate its properties and apply it to the estimation of the neonatal mortality rate in Zambia, using 2014 Demographic Health Surveys data.

1. INTRODUCTION

In terms of simplicity and fewest assumptions, the ideal method for obtaining estimates from complex surveys is a design-based weighted estimator, such as the Horvitz-Thompson (Horvitz and Thompson, 1952) or Hájek estimator (Hájek, 1964). These estimates use sampling weights which incorporate the sampling probability, and possibly non-response and post-stratification adjustments. While these design-based weighted estimators are consistent and asymptotically normal (Breidt and Opsomer, 2017), they also produce very large design-based variance estimates when data are sparse. In small area estimation (SAE) problems there is often insufficient data to produce design-based estimates with reasonable precision, making it necessary to use a model-based approach with random effects. By borrowing information across areas, precision is increased at the cost of introducing some shrinkage bias (Datta and Ghosh, 2012; Wakefield et al., 2020). The bias introduced by these model-based approaches often makes higher level combinations of these small area estimates inconsistent in aggregate. For example, the aggregation of state-level model-based estimates may not be consistent with the national design-based estimate.

There have been many procedures introduced in the SAE literature which ensure aggregation accuracy of small area estimates to more reliable, higher-level estimates, referred to as benchmarks. There is a vast literature on this topic and we highlight a relevant subset here. One early development is the use of nested error linear regression models for estimating small area means (Pfeffermann and Barnard, 1991; You and Rao, 2002). Wang et al. (2008) derived a unique best linear unbiased estimator for small area means from augmented area-level linear models under a single benchmarking constraint, and Bell et al. (2013) extended this result to accommodate multiple benchmarking constraints. Datta et al. (2011) developed a class of benchmarked Bayes estimators under area-level generalized linear models. All of these methods modify the best linear unbiased predictor of the small area means to achieve the benchmarking constraint, which leads to increased variance under the model (Bell et al., 2013). Berg and Fuller (2018) proposed two benchmarking procedures for nonlinear models: one which uses a linear additive adjustment, and the other which uses an augmented model for the expectation function. All of the aforementioned methods consider benchmark constraints which are estimated from the same data. There is also a body of work exploring the use of posterior projections to conduct Bayesian inference in constrained parameter spaces (Dunson and Neelon, 2003; Astfalck et al., 2018; De Nicolò et al., 2024). As Astfalck et al. (2018) notes, this framework is quite sensitive to misspecification of constraints, which implies that it would not provide reliable estimates when using design-based weighted estimates as constraints because of their large uncertainty. Zhang and Bryant (2020) and Nandram and Sayit (2011) proposed inexact Bayesian benchmarking methods which can incorporate the uncertainty of the benchmarks into the posterior distribution via a soft constraint. However, both of these frameworks require that the benchmarks and small area estimates are produced from separate data sources. Violating this assumption would yield benchmarked small area estimates that underestimate uncertainty (Okonek and Wakefield, 2024).

The contribution of this paper is a pair of unit-level Bayesian mixed effects models with a likelihood modified to incorporate higher-level design-based direct estimates obtained from the same data source. The set of models we will introduce use design-based estimates as constraints while accounting for the fact that these constraints are estimated from the same data. As Stefan and Hidiroglou (2021) note, “statistical agencies favor an overall agreement between the sum of the model-based small area estimates and the direct estimate at a higher level that corresponds to the union of the small areas”. The two models differ in that one enforces a hard constraint, while the other uses a soft constraint, accounting for the uncertainty of the benchmark. The goal of the latter is fundamentally different from exact benchmarking procedures, because its primary goal is to use higher-level estimates to encourage consistency in aggregation while taking the uncertainty of the benchmark into account, as opposed to imposing a hard benchmark constraint. Consistency in aggregation with design-based estimates is not only desirable for logistical reasons, but also because these estimates take the survey design into account, which model-based estimates typically do not. We propose two variations of our model because while taking the uncertainty of the benchmark into account is the more principled approach, there may be practical cases in which exact benchmarking is preferred by the practitioner. We will focus on the estimation of rare event prevalence, as this is a setting in which issues of data sparsity tend to be most severe, making this direct-assisted unit-level model necessary. Examples of commonly measured outcomes that can be considered rare events, depending on the study population, include neonatal mortality, vaccination (or no vaccination) status, and HIV status. While the monitoring of these rare events at subnational levels is of substantive importance, the exceedingly small number of sampled events poses unique challenges in prevalence estimation.

In section 2 we will introduce the context of SAE in low- and middle-income countries (LMICs), which motivates the new model. In section 3 we will present a standard Bayesian unit-level model for rare events and in section 4 we will extend this model and introduce the soft and hard direct-assisted Bayesian unit-level (DABUL) models and discuss its implementation. In section 5 we conduct a simulation study comparing the DABUL models to a standard unit-level Bayesian model. Section 6 presents an application of the DABUL models to estimation of the neonatal mortality rate (NMR) in Zambia and a corresponding cross-validation study. We conclude with a discussion in Section 7.

2. SMALL AREA ESTIMATION IN LOW- AND MIDDLE-INCOME COUNTRIES

It is common to utilize nationally representative samples collected by the Demographic and Health Surveys (DHS) (Corsi et al., 2012) and Multiple Indicator Cluster Surveys (MICS) (UNICEF et al., 2015) to obtain estimates in LMICs at the level of the first or second administrative area. DHS and/or MICS carry out surveys in the majority of LMICs, often as frequently as every five years. The DHS and MICS are conducted using a two-stage stratified cluster sampling design, where the sampling strata are defined by urban/rural crossed with the first (or for some countries, second) administrative area. In the first stage, a selection of enumeration areas (EAs) are sampled from each strata, where the probability a given EA will be selected is proportional to the number of households in that EA, relative to the others in its strata. In the second stage, a fixed number of households are sampled with equal probability from each EA. Under this design, a ‘cluster’ refers to either an EA or a segment of an EA.

The NMR is one of a multitude of demographic and health outcomes that is often estimated from these surveys. Accurate and precise estimation of the NMR at subnational levels is particularly important because it allows government officials at the country or regional level to evaluate which regions need more targeted interventions. The Sustainable Development Goals (SDGs) provide a target of no more than 12 deaths per 1000 live births by 2030 (https://sdgs.un.org/2030agenda). As previously stated, using a design-based weighted estimator to estimate the subnational NMR would be ideal, but is often not feasible due to small area-level sample sizes and the relative rarity of neonatal death. The sampling frame for most DHS and MICS are powered at the first administrative level, so estimates at smaller area levels using design-based methods, or even area-level models, such as the Fay-Herriot model, are often not reliable. However, estimates at the second administrative level are desired because this is often the level at which health interventions are administered. As a result of this sampling design, it is often helpful to use unit-level models to obtain small area estimates (Rao and Molina, 2015), where the units are the sampled clusters. Because the households in each cluster are recorded with the same location and sampling weight, it is natural to combine data within the same cluster and model these summed counts.

The methods of this paper are motivated by estimation of the NMR in cases where design-based estimates are reliable at the first, but not second, administrative level. The model can easily be simplified to accommodate a case where the design-based estimates are reliable at the national level, but not the first administrative level. We will use NMR-specific terminology throughout (i.e., births and neonatal deaths), but our proposed model can be applied to prevalence estimation of any rare event using complex survey data. For example, if HIV status is the indicator of interest, ‘births’ can be replaced by ‘individuals’ and ‘neonatal deaths’ replaced by ‘individuals with positive HIV status’.

3. BAYESIAN UNIT-LEVEL MODELS FOR RARE EVENT PREVALENCE

3.1. Sampling Model

Suppose we seek to estimate the prevalence of a rare event, neonatal mortality, at the second administrative level using sparse survey data. It is common to fit such a model under the assumption that the number of events follows an overdispersed Poisson distribution (Diggle and Giorgi, 2019), where the overdispersion accounts for the within-cluster dependence in outcomes. Specifically, let $n_c$ and $Z_c$ represent the number of births and neonatal deaths in a fixed time period in the sampled households in sampled cluster $c$, respectively, where each cluster is contained in one of $m_2$ second administrative areas. Then, for neonatal mortality rate in cluster $c$ (i.e., probability of death in the neonatal population in cluster $c$), $r_c$, and overdispersion parameter $\lambda$, we assume that for each cluster, $c$,

$$Z_c\mid r_c,\lambda \sim \text{Negative}-\text{binomial}(n_c{r}_c,\lambda )$$ (1)

with the parameterization

$$P(Z_c\mid n_c{r}_c,\lambda )=\frac{\Gamma (Z_c+n_c{r}_c∕\lambda )}{\Gamma (Z_c+1)\Gamma (n_c{r}_c∕\lambda )}{\lambda }^{Z_c}{(1+\lambda )}^{-(Z_c+n_c{r}_c∕\lambda )}.$$ (2)

Under this parameterization, $\mathbb{E}[Z_c\mid n_c,r_c,\lambda ]=n_c{r}_c$ and $\mathrm{Var}(Z_c\mid n_c,r_c,\lambda )=(1+\lambda )n_c{r}_c$. We link this distribution with the regression model,

$$\log (r_c)=\alpha +b_j{1}_{c\in {\delta }_2(j)}$$ (3)

where $b_j$ is a second administrative level random effect (independent and identically distributed or with spatial structure) and ${\delta }_k(j)$ is the set of clusters in $k^{\mathit{th}}$ administrative area $j$. Appendix A provides a summary of definitions and notation introduced throughout this paper. When this model is fit with sparse data there is often a large amount of shrinkage because information in each second administrative area is very limited. One way to mitigate this shrinkage effect is to use a regression model with nested spatial effects which includes a set of fixed effects at the first administrative level and a set of random effects at the second administrative level, i.e., replacing (3) with

$$\log (r_c)=\alpha +{\beta }_i{1}_{c\in {\delta }_1(i)}+b_j{1}_{c\in {\delta }_2(j)}$$ (4)

where ${\beta }_i$ is a first administrative level fixed effect and ${\beta }_1$ is fixed at 0 to preserve identifiability.

3.2. Motivating example: NMR in Zambia

To demonstrate the difference between regression models (3) and (4), consider the example of NMR estimation in Zambia at the second administrative level using all births between 2009 and 2013 sampled in the 2014 Zambia DHS (ICF International, 2014). Zambia has 10 first administrative areas and 115 second administrative areas. A summary of these data is displayed in figure 1. The numbers of observed births and neonatal deaths in each first administrative area are sufficiently large to use an area-level model, such as Fay-Herriot (Fay and Herriot, 1979), with an average of 1319 births and 32 deaths observed in each area. However, the number of observed births and neonatal deaths in each second administrative area are prohibitively small, with an average of 115 births and 2.8 deaths observed in each area, and 21.7% of areas observing no neonatal deaths at all. At this level of data sparsity, a unit level model is useful for estimation at the second administrative level (Wakefield et al., 2025).

Figure 1. Summary of births and neonatal deaths between 2009 and 2013 recorded in the 2014 Zambia DHS.

The top row provides maps of the total number of observed births in each first and second administrative area. The bottom row provides maps of the total number of neonatal deaths in each first and second administrative area. Each map has a different color scale to make differences between areas more clear.

We must account for the stratified sampling design of the DHS survey (first administrative area crossed with urban/rural), so instead of estimating a global intercept, we include one urban and one rural intercept as follows,

$$\log (r_c)={\alpha }_U{1}_{c\in {\delta }_U}+{\alpha }_R{1}_{c\in {\delta }_R}+b_j{1}_{c\in {\delta }_2(j)}$$ (5)

$$\log (r_c)={\alpha }_U{1}_{c\in {\delta }_U}+{\alpha }_R{1}_{c\in {\delta }_R}+{\beta }_i{1}_{c\in {\delta }_1(i)}+b_j{1}_{c\in {\delta }_2(j)},$$ (6)

where ${\delta }_U$ and ${\delta }_R$ are the sets of urban and rural clusters, respectively, and ${\beta }_1$ is fixed at 0 to preserve identifiability.

We estimate the NMR at the second administrative level under regression models (5) and (6) using the Stan software (Stan Development Team, 2024). While a variety of random effect models at the second administrative level could be employed, we use a BYM2 spatial effect. Introduced by Riebler et al. (2016), the BYM2 model is a re-parameterized version of the Besag-York-Mollié (BYM) model (Besag et al., 1991), which includes both unstructured, independent and identically distributed (IID), spatial effects and structured, intrinsically conditional autoregressive (ICAR), spatial effects, (Besag, 1974). The BYM2 model has two parameters: ${\sigma }^2$, which indicates the total variance of the random effects, and $\varphi$, which indicates the proportion of this variation that is explained by the structured component. The structured component has a sum-to-zero constraint to ensure identifiability. More specifically, in (6), the structured component of the spatial effect has a separate sum-to-zero constraint for the areas within each first adimistrative area, while in (5) there is only one global sum-to-zero constraint. In Appendix B we compare NMR estimates and uncertainty under model (6) using IID and BYM2 spatial effects, which shows that, in this example, choice of spatial effect model makes little difference, so we choose BYM2 as it provides the option of more structure (by including an ICAR component in addition to an IID component) and has the added benefit of allowing more spatially informative estimation in areas without observed data. For hyperpriors, we set $\varphi \sim \mathit{Beta}(0.5,0.5)$ and use a penalized complexity (PC) prior for ${\sigma }^2$ with hyperparameters $U=1$ and $\alpha =0.01$, which corresponds to the prior belief that $P(\sigma >1)=0.01$ (Simpson et al., 2017). We place diffuse priors on the overdispersion and regression parameters: $\lambda \sim \mathrm{Exp}(1)$, $({\alpha }_U,{\alpha }_R)\stackrel{\mathrm{iid}}{\sim }𝒩(-3.5,3^2)$ and ${\beta }_i\sim 𝒩(0,1)$. The priors on the urban and rural intercepts are centered at −3.5 to reflect the prior belief that neonatal death is a rare event $(e^{-3.5}\approx 0.03)$. Neonatal mortality rate estimates in many Sub-Saharan African countries, including Zambia, which are available on the UN-IGME Child Mortality Estimation dashboard (childmortality.org), confirm that this is a reasonable prior mean for the overall level. The priors on the regression parameters (${\alpha }_U$, ${\alpha }_R$, $\beta$) are chosen with the goal of being relatively diffuse on the exponential scale, while still accounting for the prior belief that the outcome measure is a rare event. More information regarding these prior choices is presented in Appendix C. We aggregate the urban and rural estimates within each second administrative area using urban/rural population fractions estimated using the method in Wu and Wakefield (2024). This method is a logistic classification model which uses pixel-level population density and defines a threshold for classifying pixels as urban or rural. This threshold is calibrated by urban/rural composition at the first administrative level which is reported by DHS (Corsi et al., 2012). The pixel-level population density is obtained from WorldPop (Tatem, 2017), a large-scale producer of high-resolution age-specific population distributions for a wide array of countries.

In figure 2 we map the NMR (deaths per 1000 live births) estimates under non-nested, nested, and direct methods. We observe that including a set of fixed effects at the first administrative level in the nested model significantly reduces shrinkage, by inducing shrinkage at a more local level. The nested model estimates, mapped on the top right, have a range of 18 – 33 deaths per 1000 live births, while the estimates from the non-nested model, mapped on the top left, are nearly homogeneous with a range of 22 – 26 deaths per 1000 live births. In the bottom right panel of figure 2, we also include a map of the second administrative level direct estimates (for areas where at least 1 neonatal death is observed) to illustrate the magnitude of shrinkage, but we caution that they are estimated from very small sample sizes and have large uncertainty (of the 87 out of 115 areas which have valid estimates, 70% of areas have coefficient of variation greater than 50% and 23% have coefficient of variation greater than 100%). In Appendix D we provide maps of the coefficients of variation for the estimates in figure 2. To further compare the model-based and direct estimates, we aggregate the second administrative level model-based estimates to the first administrative level with population weights calculated using WorldPop data (Tatem, 2017). In figure 3, observe that the aggregation of the estimates to the first administrative level can be quite far from the consistent design-based estimate. This figure further highlights the overwhelming shrinkage of estimates under the non-nested model.

Figure 2. Map of NMR estimates in Zambia (2009-2013) under various models.

The maps on the top row display results from unit-level Bayesian negative binomial models with BYM2 spatial effects at the second administrative level, while the maps on the bottom row display Hájek direct estimates at the first and second administrative level, respectively. The map on the top left displays the estimates obtained from the model with spatial effects only at the second administrative level (5), while the map on the top right displays the estimates obtained from the model with nested spatial effects (6). Note that data at the second administrative level is sparse, so the bottom right map is only included to visualize the magnitude of shrinkage.

Figure 3. NMR estimates at the second administrative level in Zambia (2009-2013) under various models.

The colored circles denote the second administrative level estimates using unit-level Bayesian negative binomial models with BYM2 spatial effects at the second administrative level and the colored diamonds denote their aggregations to the first administrative level. The black diamonds denote Hájek direct estimates at the first administrative level.

In some first administrative areas, such as the Eastern, Northern, Luapala, and Central regions, aggregated estimates from the nested model are in close agreement with their corresponding design-based estimates. However, for other areas, such as the Lusaka, Muchinga, Western, and Copperbelt regions, aggregated estimates from the nested model are quite different from their corresponding design-based estimates. While employing nested spatial effects can mitigate shrinkage in unit-level models, it is not sufficient to achieve small area estimates which agree in aggregation with higher-level design-based estimates. In fact, there is no guarantee that aggregated model-based estimates will be at all similar to their design-based counterparts. Because design-based estimators are consistent, it is desirable to consider a model which will encourage consistency with them in aggregation.

4. DIRECT-ASSISTED BAYESIAN UNIT-LEVEL MODEL

In the following section we introduce the DABUL models: an extension of the standard unit-level Bayesian model with nested random effects which incorporates direct design-based estimates to encourage consistency with design-based estimates in aggregation.

First, consider a model for a full enumeration of the population in which all births in all clusters are observed. Let $\mathbf{Y}$ be the vector containing the total number of neonatal deaths in all households, $Y_c$, in each cluster $c$, so that the vector has length equal to the total number of clusters in the population. Let ${\mathbf{Y}}_{\mathbf{i}}$ be the sub-vector containing the elements of $\mathbf{Y}$ corresponding to all clusters in the first administrative area $i$, and $Y_{i+}$ be the sum of the elements in this sub-vector. Similarly, let $\mathbf{N}$ be the vector containing the total number of births in all households, $N_c$, in each cluster $c$, so that the sum of the elements of $\mathbf{N}$ is the total number of births in the population. Let ${\mathbf{N}}_{\mathbf{i}}$ be the sub-vector containing the elements of $\mathbf{N}$ corresponding to all clusters in the first administrative area $i$, and $N_{i+}$ be the sum of the elements in this sub-vector. Also, let $\mathbf{r}$ be the vector containing elements $r_c$, and ${\mathbf{r}}_{\mathbf{i}}$ be the sub-vector containing the elements of $\mathbf{r}$ corresponding to clusters in the first administrative area, $i$. Let $\odot$ denote the Hadamard product (elementwise multiplication; see Appendix A for more details). Then an equivalent way to define a negative binomial distribution is

$$P({\mathbf{Y}}_{\mathbf{i}}\mid {\mathbf{r}}_{\mathbf{i}},\lambda )=P({\mathbf{Y}}_{\mathbf{i}},Y_{i+1}\mid {\mathbf{r}}_{\mathbf{i}},\lambda )=P({\mathbf{Y}}_{\mathbf{i}}\mid Y_{i+},{\mathbf{r}}_{\mathbf{i}},\lambda )\times P(Y_{i+}\mid {\mathbf{r}}_{\mathbf{i}},\lambda )$$ (7)

for clusters in first administrative area, $i$, where

$${\mathbf{Y}}_{\mathbf{i}}\mid Y_{i+},{\mathbf{r}}_{\mathbf{i}},\lambda \sim \text{DCM}(Y_{i+},{\mathbf{N}}_{\mathbf{i}}\odot {\mathbf{r}}_{\mathbf{i}}∕\lambda )$$ (8)

and

$$Y_{i+1}\mid {\mathbf{r}}_{\mathbf{i}},\lambda \sim \text{Negative}-\text{binomial}\left(\sum _{c\in {\delta }_1(i)}{N}_c{r}_c,\lambda \right).$$ (9)

Here DCM abbreviates the Dirichlet compound multinomial distribution (Mosimann, 1962), also referred to as the multivariate Pólya distribution, which is defined by the probability mass function,

$$P({\mathbf{Y}}_{\mathbf{i}}\mid Y_{i+},{\mathbf{r}}_{\mathbf{i}},\lambda )=\frac{\Gamma (Y_{i+}+1)\Gamma (\frac{1}{\lambda }{\sum }_{c\in {\delta }_1(i)}{N}_c{r}_c)}{\Gamma (Y_{i+}+\frac{1}{\lambda }{\sum }_{c\in {\delta }_1(i)}{N}_c{r}_c)}\prod _{c\in {\delta }_1(i)}\frac{\Gamma (Y_c+N_c{r}_c∕\lambda )}{\Gamma (N_c{r}_c∕\lambda )\Gamma (Y_c+1)}.$$

This alternative expression follows directly from the additive property of negative binomial random variables and the relationship between the negative binomial and DCM distributions. Specifically, if $\mathbf{x}=(x_1,\dots ,x_n)$ are independent random variables, each following a negative binomial distribution with mean parameter $\theta =({\theta }_1,\dots ,{\theta }_n)$, respectively, and a common overdispersion parameter $\lambda$, then $x_{+}≔{\sum }_{i=1}^n{x}_i$ follows a negative binomial distribution with mean ${\sum }_{i=1}^n{\theta }_i$, and overdispersion parameter $\lambda$. It also holds that $\mathbf{x}\mid x_{+},\theta ,\lambda$ follows a DCM distribution with parameter vector $\theta ∕\lambda$.

When all births and neonatal deaths in each cluster are observed (i.e. $\mathbf{Y}$ and $Y_{i+}$ is known), this alternative parameterization of the negative binomial distribution is equivalent to the parameterization presented in (2). We will now consider a survey sampling context in which 1) we use two-stage sampling, so that only some clusters are sampled and only some of the births and corresponding neonatal deaths are observed in each of those sampled clusters, i.e., $\mathbf{Y}$ is not known, and 2) each $Y_{i+}$ can be estimated through a design-based estimate.

4.1. Accounting for survey sampling

We first assume $Y_{i+}$ is known for all $i$, so that only point 1) needs to be considered. Suppose the birth and death observations are collected using the two-stage cluster sampling design described in section 2. In this situation, we must incorporate the fact that not all births, and corresponding deaths, are observed in each sampled cluster. For each cluster $c$, let ${\gamma }_c$ be an indicator variable denoting whether the cluster was sampled and let $n_c$ and $Z_c$ be the number of births and neonatal deaths across all of the sampled households in that cluster. Also, let the vectors $\gamma$, ${\gamma }_i$, $\mathbf{Z}$ and ${\mathbf{Z}}_{\mathbf{i}}$ be defined analogously to $\mathbf{Y}$ and ${\mathbf{Y}}_{\mathbf{i}}$. Note that if ${\gamma }_c=0$ for a cluster $c$, this implies $n_c=Z_c=0$ because that cluster was not sampled. Observe then, for each cluster $c$,

$$Z_c\mid Y_c,{\gamma }_c=1\sim \text{Hypergeometric}(N_c,Y_c,n_c),P(Z_c\mid {\gamma }_c=0)=I(Z_c=0).$$ (10)

4.2. Incorporating higher level design-based estimates

As alluded to in the previous section, in this survey sampling context each $Y_{i+}$ is not known, and must be estimated. We have established that the DABUL models pertain to the situation in which there is sufficient data to use design-based estimates at the first administrative level, though not at the second administrative level. We will denote the design-based prevalence estimate for a first administrative area $i$ as $r_i^{D_1}$. The only difference between the two DABUL models is how the design-based estimates are incorporated into the likelihood. For the hard DABUL model, we can simply set $Y_{i+}=r_i^{D_1}\times N_{i+}$, to apply a hard benchmarking constraint. For the soft DABUL model, we account for the uncertainty of the design-based estimate by using the property that design-based estimators are asymptotically normal (Breidt and Opsomer, 2017, Section 3). On the logit scale, we can specify

$$\mathrm{logit}(r_i^{D_1})\mid Y_{i+1}\sim 𝒩(\mathrm{logit}(Y_{i+}∕N_{i+}),V_i^{D_1})$$ (11)

where $V_i^{D_1}$ is the design-based variance of $\mathrm{logit}(r_i^{D_1})$, which accounts for the uncertainty induced by the sampling design. This variance can be estimated using the svydesign function in the survey package (Lumley, 2024), to estimate the prevalence, which is then transformed to the logit scale using the delta method. If this method does not produce stable variance estimates, a variance smoothing method may be employed (e.g., using generalized variance functions described in chapter 7 of Wolter (2007)), but we will not consider this option here.

Then, the joint distribution of $(\mathbf{Z},{\mathbf{r}}^{{\mathbf{D}}_1})$ conditional on ($\mathbf{Y}$, ${\mathbf{Y}}_{+}$, $\gamma$) under the hard DABUL model can be expressed as,

$$\begin{gathered} P(\mathbf{Z},{\mathbf{r}}^{{\mathbf{D}}_1}\mid \mathbf{Y},{\mathbf{Y}}_{+},\gamma )=P(\mathbf{Z}\mid {\mathbf{r}}^{{\mathbf{D}}_1},\mathbf{Y},\gamma )P({\mathbf{r}}^{{\mathbf{D}}_1}\mid {\mathbf{Y}}_{+}) \\ \propto \prod _{i=1}^{m_1}P({\mathbf{Z}}_{\mathbf{i}}\mid r_i^{D_1},{\mathbf{Y}}_{\mathbf{i}},{\gamma }_i)I(r_i^{D_1}=Y_{i+}∕N_{i+}). \end{gathered}$$ (12)

Similarly, the joint distribution of ($\mathbf{Z}$, ${\mathbf{r}}^{{\mathbf{D}}_1}$) conditional on ($\mathbf{Y}$, ${\mathbf{Y}}_{+}$, $\gamma$) under the soft DABUL model can be expressed as,

$$\begin{gathered} P(\mathbf{Z},{\mathbf{r}}^{{\mathbf{D}}_1}\mid \mathbf{Y},{\mathbf{Y}}_{+},\gamma )=P(\mathbf{Z}\mid {\mathbf{r}}^{{\mathbf{D}}_1},\mathbf{Y},\gamma )P({\mathbf{r}}^{{\mathbf{D}}_1}\mid {\mathbf{Y}}_{+}) \\ =\prod _{i=1}^{m_1}P({\mathbf{Z}}_{\mathbf{i}}\mid r_i^{D_1},{\mathbf{Y}}_{\mathbf{i}},{\gamma }_i)P(r_i^{D_1}\mid Y_{i+}). \end{gathered}$$ (13)

By definition, $r_i^{D_1}$ is a weighted sum of ${\mathbf{Z}}_{\mathbf{i}}$, so it follows that $P({\mathbf{Z}}_{\mathbf{i}}\mid r_i^{D_1},{\mathbf{Y}}_{\mathbf{i}},{\gamma }_i)$ is a product of Hypergeometric distributions conditional on the survey-weighted sum of ${\mathbf{Z}}_{\mathbf{i}}$, $g_i^{D_1}({\mathbf{Z}}_{\mathbf{i}})$, being equal to $r_i^{D_1}$. More explicitly, for first administrative area $i$,

$$\begin{gathered} P({\mathbf{Z}}_{\mathbf{i}}\mid r_i^{D_1},{\mathbf{Y}}_{\mathbf{i}},{\gamma }_i)=\frac{P({\mathbf{Z}}_{\mathbf{i}},g_i^{D_1}({\mathbf{Z}}_{\mathbf{i}})=r_i^{D_1}\mid {\mathbf{Y}}_{\mathbf{i}},{\gamma }_i)}{P(g_i^{D_1}({\mathbf{Z}}_{\mathbf{i}})=r_i^{D_1}\mid {\mathbf{Y}}_{\mathbf{i}},{\gamma }_i)} \\ \propto \frac{I(g_i^{D_1}({\mathbf{Z}}_{\mathbf{i}})=r_i^{D_1})}{P(g_i^{D_1}({\mathbf{Z}}_{\mathbf{i}})=r_i^{D_1}\mid {\mathbf{Y}}_{\mathbf{i}},{\gamma }_i)}\underset{\text{Hypergeometric}(10)}{\underbrace{\left[\prod _{\begin{array}{c}\hfill c\in {\delta }_1(i):\hfill \\ \hfill {\gamma }_c=1\hfill \end{array}}P(Z_c\mid Y_c,{\gamma }_c=1)\right]}}. \end{gathered}$$ (14)

Note that $g_i^{D_1}({\mathbf{Z}}_{\mathbf{i}})=r_i^{D_1}\mid {\mathbf{Y}}_{\mathbf{i}},{\gamma }_i$ follows the distribution of a weighted sum of Hypergeometric random variables.

4.3. Combining likelihood components

The joint distribution of ($\mathbf{Z}$, ${\mathbf{r}}^{{\mathbf{D}}_1}$, ${\mathbf{Y}}_{+}$, $\mathbf{Y}$) conditional on ($\gamma$, $\mathbf{r}$, $\lambda$) can be expressed as,

$$\begin{gathered} P(\mathbf{Z},{\mathbf{r}}^{{\mathbf{D}}_1},{\mathbf{Y}}_{+},\mathbf{Y}\mid \gamma ,\mathbf{r},\lambda )\propto \prod _{i=1}^{m_1}P(\mathbf{Z},{\mathbf{r}}^{{\mathbf{D}}_1}\mid \mathbf{Y},{\mathbf{Y}}_{+},\gamma )P(Y_{i+},{\mathbf{Y}}_{\mathbf{i}}\mid {\mathbf{r}}_{\mathbf{i}},\lambda ) \\ \propto \prod _{i=1}^{m_1}\underset{(12)\text{or}(13)}{\underbrace{P(\mathbf{Z},{\mathbf{r}}^{{\mathbf{D}}_1}\mid \mathbf{Y},{\mathbf{Y}}_{+},\gamma )}}\underset{\text{DCM}(8)}{\underbrace{P({\mathbf{Y}}_{\mathbf{i}}\mid Y_{i+},{\mathbf{r}}_{\mathbf{i}},\lambda )}}\underset{\text{Neg}-\text{bin}(9)}{\underbrace{P(Y_{i+}\mid {\mathbf{r}}_{\mathbf{i}},\lambda }} \end{gathered}$$ (15)

where ${\mathbf{Y}}^{(\mathbf{s})}$ is the sub-vector of $\mathbf{Y}$ containing the elements corresponding to sampled clusters and $m_1$ is the number of first administrative areas.

Observe that in this joint distribution, the latent variables corresponding to the unobserved clusters (i.e., $\mathbf{Y}\setminus {\mathbf{Y}}^{(\mathbf{s})}$) are only included in the DCM distribution. Estimation of these latent variables will cause an identifiability issue because there is no data for these clusters, so the total neonatal death counts from all of the unobserved clusters in a first administrative area can be collapsed into one group. Because we can estimate ${\mathbf{Y}}^{(s)}$ and ${\mathbf{Y}}_{+}$, this collapsing ensures identifiability. Hence, we replace (8) with

$$({\mathbf{Y}}_{\mathbf{i}}^{(\mathrm{s})},Y_{i+}-Y_{i+}^{(s)})\mid Y_{i+},{\mathbf{r}}_{\mathbf{i}},\lambda \sim \text{DCM}\left(Y_{i+}\left(({\mathbf{N}}_{\mathbf{i}}^{(\mathrm{s})}\odot {\mathbf{r}}_{\mathbf{i}}^{(\mathrm{s})})∕\lambda ,\left(\sum _{\begin{array}{c}\hfill c\in {\delta }_1(i)\hfill \\ \hfill {\gamma }_c=0\hfill \end{array}}{N}_c{r}_c\right)∕\lambda \right)\right)$$ (16)

where $Y_{i+}^{(s)}$ is the sum of the elements in ${\mathbf{Y}}_{\mathbf{i}}^{(\mathbf{s})}$, and ${\mathbf{N}}_i^{(s)}$ and ${\mathbf{r}}_i^{(s)}$ are the sub-vectors of ${\mathbf{N}}_{\mathbf{i}}$ and ${\mathbf{r}}_{\mathbf{i}}$ containing elements corresponding to the observed clusters in first administrative area $i$. Note that this distribution has dimension equal to the total number of observed clusters plus one, as does the corresponding probability vector. By making this adjustment we do not lose any information and reduce the dimension of the latent variable vector from the total number of clusters, to the total number of observed clusters, which is of significantly smaller magnitude. This reduction increases numerical stability and computational efficiency.

This completes the derivation of the posterior distribution under the DABUL models, which can be expressed as follows,

$$\pi (\mathbf{r},\lambda ,\kappa ,{\mathbf{Y}}^{(\mathrm{s})},{\mathbf{Y}}_{+}\mid \mathbf{Z},{\mathbf{r}}^{{\mathbf{D}}_1},\gamma )\propto \underset{(15)}{\underbrace{P(\mathbf{Z},{\mathbf{r}}^{{\mathbf{D}}_1},{\mathbf{Y}}_{i+},\mathbf{Y}\mid \gamma ,\mathbf{r},\lambda )}}P(\mathbf{r}\mid \kappa )\pi (\lambda ,\kappa )$$ (17)

where $\kappa$ is the set of regression parameters and hyperparameters. Figure 4 depicts the components of this model in a visual form and compares it to the standard unit-level Bayesian model with nested spatial effects. Observe the only difference between the hard and soft DABUL models is how ${\mathbf{r}}^{{\mathbf{D}}_1}$ is used to estimate ${\mathbf{Y}}_{+}$.

Figure 4. Comparison of DABUL models with the standard unit-level Bayesian model.

The cluster-length vector $\mathbf{Y}$, indexed by $c$, denotes the total number of neonatal deaths in each cluster. The vector ${\mathbf{Y}}^{\mathbf{s}}$ is the sub-vector of $\mathbf{Y}$ containing the elements corresponding to sampled clusters and $\mathbf{Z}$ is the vector of observed deaths in each sampled cluster, indexed by $c$.

4.4. Implementation

Estimation of the posterior distribution (17) is not straightforward because most Bayesian inference software, including Stan (Stan Development Team, 2024) and the INLA package in R (Rue et al., 2009), are not built to accommodate discrete latent variables that cannot be marginalized out. As a result, we implement a NUTS-within-Gibbs sampler, described in Algorithms 1 and 2, which uses a No-U-Turn Sampler (NUTS) to update all continuous parameters and inverse transform sampling to update the discrete parameters. Introduced by Hoffman et al. (2014), NUTS is an efficient extension of Hamiltonian Monte Carlo and is the computational framework for Stan. We implemented the NUTS step of our algorithm in R according to Algorithm 3 in Hoffman et al. (2014), with assistance from the code provided by Märtens (2017). We modified the algorithm to include a scaling matrix, denoted $\Sigma$, in the proposal density to better account for correlation between parameters, as is done in Stan. The gradient of the log-likelihood with respect to all continuous parameters was derived analytically to avoid costly numerical approximations at each step of the NUTS sampler. The tuning parameter, $ϵ$, is chosen to optimize the step size of the random walk.

All components of the posterior distribution (17) have a closed form except for $P(g_i^{D_1}({\mathbf{Z}}_{\mathbf{i}})=r_i^{D_1}\mid {\mathbf{Y}}_{\mathbf{i}},{\gamma }_i)$, the probability mass function of a weighted sum of independent Hypergeometric random variables. Although $g_i^{D_1}({\mathbf{Z}}_{\mathbf{i}}){\mathbf{Y}}_{\mathbf{i}},{\gamma }_i$ follows a discrete distribution, the domain contains all the possible weighted sums of ${\mathbf{Z}}_{\mathbf{i}}$, which is very dense because each $Z_c$ has a different corresponding weight. Therefore, in practice, we can treat $P(g_i^{D_1}({\mathbf{Z}}_{\mathbf{i}})=r_i^{D_1}\mid {\mathbf{Y}}_{\mathbf{i}},{\gamma }_i)$ as a probability density function (with continuous domain). At each iteration of the algorithm this component must be approximated for the inverse transform sampling of ${\mathbf{Y}}^{(s)}$ step. We approximate this term by drawing 2000 multivariate samples from the independent Hypergeometric distributions implied by the proposed and current states and taking the weighted sum of each sample vector, resulting in 2000 draws from the distribution of $g_i^{D_1}({\mathbf{Z}}_{\mathbf{i}}){\mathbf{Y}}_{\mathbf{i}},{\gamma }_i$. Gaussian kernel density estimation is then applied to this empirical distribution, using the density function in R, to approximate $P(g_i^{D_1}({\mathbf{Z}}_{\mathbf{i}})=r_i^{D_1}\mid {\mathbf{Y}}_{\mathbf{i}},{\gamma }_i)$. The bandwidth is chosen using Silverman’s rule-of-thumb method (Silverman, 2018).

Algorithm 1 NUTS-within-Gibbs sampler for soft DABUL model

$$\begin{array}{cc}\hfill & \mathrm{Input}:(\mathbf{Z},\gamma ,\mathbf{n},\mathbf{N});({\mathbf{r}}^{{\mathbf{D}}_1},{\mathbf{V}}^{{\mathbf{D}}_1});ϵ,\Sigma ,M,{\ell }_{REG}≔\log \pi (\alpha ,\beta ,\mathbf{b},{\sigma }^2,\varphi ,\lambda \mid {\mathbf{Y}}^{(\mathrm{s})},{\mathbf{Y}}_{+},\mathbf{Z},\gamma )\hfill \\ \hfill & \mathbf{for}m\text{in}1:M\mathbf{do}\hfill \\ \hfill & {(\alpha ,\beta ,\mathbf{b},{\sigma }^2,\varphi ,\lambda )}^{(m)}\leftarrow NUTSOneStep({(\alpha ,\beta ,\mathbf{b},{\sigma }^2,\varphi ,\lambda )}^{(m-1)},{\ell }_{REG},{({\mathbf{Y}}^{(\mathrm{s})},{\mathbf{Y}}_{+})}^{(m-1)},ϵ,\Sigma )\hfill \\ \hfill & \mathbf{for}\mathbf{each}\text{cluster},c\mathbf{do}\hfill \\ \hfill & r_c^{(m)}\leftarrow exp(\alpha +{\beta }_i{1}_{c\in {\delta }_1(i)}+b_j{1}_{c\in {\delta }_2(j)})\hfill \\ \hfill & \mathbf{end}\mathbf{for}\hfill \\ \hfill & \mathbf{for}\mathbf{each}\text{first administrative area},i\mathbf{do}\hfill \\ \hfill & \text{Draw}{Y}_{i+}^{(m)}\sim P(Y_{i+}\mid {{\mathbf{Y}}^{(\mathrm{s})}}^{(m-1)},\gamma ,{\mathbf{r}}^{(m)},{\lambda }^{(m)},r_i^{D_1},V_i^{D_1})\text{using inverse transform sam}-\hfill \\ \hfill & \text{pling}\hfill \\ \hfill & \mathbf{end}\mathbf{for}\hfill \\ \hfill & \mathbf{for}\mathbf{each}\text{observed cluster}c\mathbf{do}\hfill \\ \hfill & \text{Draw}{Y}_c^{(m)}\sim P(Y_c\mid {{\mathbf{Y}}_{1:(\mathbf{c}-1)}^{(\mathrm{s})}}^{(m-1)},{{\mathbf{Y}}_{(\mathbf{c}+1){\mathbf{n}}_{\mathbf{clusters}}}^{(\mathrm{s})}}^{(m-1)},Y_{i+}^m,Z_c,{\gamma }_c,r_c^{(m)},{\lambda }^{(m)})\text{using in}-\hfill \\ \hfill & \text{verse transform sampling}\hfill \\ \hfill & \mathbf{end}\mathbf{for}\hfill \\ \hfill & \mathbf{end}\mathbf{for}\hfill \\ \hfill & \mathrm{Output}:\alpha ,\beta ,\mathbf{b},\lambda ,{\sigma }^2,\varphi ,{\mathbf{Y}}^{(\mathrm{s})},{\mathbf{Y}}_{+}\hfill \end{array}$$

Algorithm 2 NUTS-within-Gibbs Sampler for hard DABUL model

$$\begin{array}{cc}\hfill & \mathrm{Input}:(\mathbf{Z},\gamma ,\mathbf{n},\mathbf{N});{\mathbf{r}}^{{\mathbf{D}}_1};ϵ,\Sigma ,M,{\ell }_{REG}≔\log \pi (\alpha ,\beta ,\mathbf{b},{\sigma }^2,\varphi ,\lambda \mid {\mathbf{Y}}^{(\mathrm{s})},{\mathbf{Y}}_{+},\mathbf{Z},\gamma )\hfill \\ \hfill & Y_{i+}=r_i^{D_1}{N}_{i+}\text{for each first administrative area}i\hfill \\ \hfill & \mathbf{for}m\text{in}1:M\mathbf{do}\hfill \\ \hfill & {(\alpha ,\beta ,\mathbf{b},{\sigma }^2,\varphi ,\lambda )}^{(m)}\leftarrow NUTSOneStep({(\alpha ,\beta ,\mathbf{b},{\sigma }^2,\varphi ,\lambda )}^{(m-1)},{\ell }_{REG},{({\mathbf{Y}}^{(\mathrm{s})})}^{(m-1)},{\mathbf{Y}}_{+},ϵ,\Sigma )\hfill \\ \hfill & \mathbf{for}\mathbf{each}\text{sample cluster},c\mathbf{do}\hfill \\ \hfill & r_c^{(m)}\leftarrow exp(\alpha +{\beta }_i{1}_{c\in {\delta }_1(i)}+b_j{1}_{c\in {\delta }_2(j)})\hfill \\ \hfill & \text{Draw}{Y}_c^{(m)}\sim P(Y_c\mid {{\mathbf{Y}}_{1:(\mathbf{c}-1)}^{(\mathrm{s})}}^{(m-1)},{{\mathbf{Y}}_{(\mathbf{c}+1){\mathbf{n}}_{\mathbf{clusters}}}^{(\mathrm{s})}}^{(m-1)},Y_{i+},Z_c,{\gamma }_c,r_c^{(m)},{\lambda }^{(m)})\text{using in}-\hfill \\ \hfill & \text{verse transform sampling}\hfill \\ \hfill & \mathbf{end}\mathbf{for}\hfill \\ \hfill & \mathbf{end}\mathbf{for}\hfill \\ \hfill & \mathrm{Output}:\alpha ,\beta ,\mathbf{b},\lambda ,{\sigma }^2,\varphi ,{\mathbf{Y}}^{(\mathrm{s})}\hfill \end{array}$$

5. SIMULATION STUDY

In this study we will compare the performance of the proposed DABUL models to an analogous, standard unit-level Bayesian model with nested spatial effects. We start by defining a neighborhood structure with 8 first administrative areas and 159 second administrative areas. There is a minimum number of 13 second administrative areas in each first administrative area and a maximum of 27. A map of these areas is provided in figure 5. This neighborhood structure and map is modified from that of Angola. The total number of urban and rural clusters in each first administrative area is drawn from $𝒩(700,{100}^2)$ and $𝒩(800,{100}^2)$ distributions, respectively, and the total number of births in each urban cluster and each rural cluster is drawn from $𝒩(100,{10}^2)$ and $𝒩(125,{10}^2)$ distributions, respectively. All draws are rounded to the nearest natural number. These values are calibrated to be comparable to the sampling frames in LMICs used by DHS and MICS. Within each first administrative area, the clusters are distributed evenly across second administrative areas so that the number of births and clusters in a second administrative area are similar to other second administrative areas within its first administrative area.

Figure 5. Second administrative area neighborhood structure for simulation study.

The total number of neonatal deaths in each cluster, $Y_c$, is drawn from a negative binomial distribution with mean $N_c{r}_c$ and overdispersion parameter $\lambda =0.25$, where $r_c$ is defined as

$$\log (r_c)={\alpha }_U{1}_{c\in {\delta }_U}+{\alpha }_R{1}_{c\in {\delta }_R}+{\beta }_i{1}_{c\in {\delta }_1(i)}+b_j{1}_{c\in {\delta }_2(j)}$$ (18)

where ${\alpha }_U=\log (0.02)$, ${\alpha }_R=\log (0.025)$ and $\beta =(-0.2,-0.1,-0.05,-0.025,0.025,0.05,0.1,0.5)$. Separate urban and rural intercepts are specified to replicate real-world conditions in which prevalence are often different between urban and rural areas. This corresponds to a neonatal mortality rate between 16 and 41 per 1000 live births, which figure 2 shows is a reasonable range. Additionally neonatal mortality rate estimates in many Sub-Saharan African countries, including Zambia, which are available on the UN-IGME Child Mortality Estimation dashboard (childmortality.org), confirm that this is a reasonable choice. The choice of overdispersion parameter $d=0.25$ is also reasonable compared to the overdispersion parameter estimate in the motivating Zambia example, 0.19 (90% credible interval: [0.08, 0.32]). The random spatial effects $\mathbf{b}$ are drawn from a BYM2 model with parameters ${\sigma }^2$ and $\varphi$, which is equivalent to a mean-zero multivariate normal distribution with covariance matrix, ${\sigma }^2((1-\varphi )\mathbf{I}+\varphi {\mathbf{Q}}_{\ast }^{-})$, where ${\mathbf{Q}}_{\ast }^{-}$ is the unique Moore-Penrose generalized inverse of the scaled structure matrix, $Q_{\ast }$, as described in Riebler et al. (2016). The values of ${\sigma }^2$ and $\varphi$ vary over three different hyperparameter settings: in the first, ${\sigma }^2={0.15}^2$ and $\varphi =0.25$; in the second, ${\sigma }^2={0.05}^2$ and $\varphi =0.25$; and in the third ${\sigma }^2={0.05}^2$ and $\varphi =0.7$. The choices of $\sigma$ are reasonable compared to the $\sigma$ parameter estimate in the motivating Zambia example, 0.12 (90% credible interval: [0.03, 0.31]). For the choice of $\varphi$, anywhere in the range of (0, 1) is reasonable, so we pick two distinct values in that range. Through these settings, we can observe whether the relative performance of the DABUL model is affected by spatial precision or dependence.

For each simulation setting, 500 datasets are independently sampled from a single generated risk surface using two-stage stratified cluster sampling (the same method which is used by DHS and MICS, except that in the second stage we directly sample births instead of households). A summary of the four simulation settings is provided in table 1. A proportion of urban and rural clusters are sampled from each first administrative area. For the majority of simulation settings, 8% and 5% of urban and rural clusters are sampled, respectively, but for one setting, only 5% and 3% are sampled so that the effect of sample size can be observed. Note that urban clusters are oversampled as this is often the case for DHS and MICS. Similarly, note that clusters are sampled at the first administrative level to mimic DHS and MICS, which are powered at the first administrative level. Due to this sampling design, there may be second administrative areas which have very small samples, or no sampled clusters at all.

Table 1.

Summary of simulation parameters and sampled datasets

	Simulation setting
	1	2	3	1a
Spatial variance: ${\sigma }^2$	0.152	0.052	0.052	0.152
Proportion of variation explained by structured effect: $\varphi$	0.25	0.25	0.7	0.25
Percentage urban clusters sampled	8%	8%	8%	5%
Percentage rural clusters sampled	5%	5%	5%	3%
Avg. # of births per 1st admin area	1640	1640	1640	1007
Avg. # of neonatal deaths per 1st admin area	40	39	39	23
Avg. # of births per 2nd admin area	83	83	83	51
Avg. # of neonatal deaths per 2nd admin area	2.0	2.0	2.0	1.2
Pct. 2nd admin areas w/ no observed neonatal deaths	23.2	22.9	23.0	39.7

For each sampled urban cluster, the number of births sampled is drawn from $\mathbf{N}(15,4)$ and for each sampled rural cluster, the number of births sampled is drawn from $\mathbf{N}(20,4)$. Each birth in a cluster has equal probability of being sampled. A summary of the resulting datasets (i.e., average numbers of observed births and deaths per administrative area) is displayed in table 1. For each of these 2000 sample datasets (4 settings × 500 samples), we obtain the following:

1. Hájek direct estimates and standard errors at the first administrative level using the SUMMER (Li et al., 2024) and survey (Lumley, 2024) packages in R.

2. Second administrative level prevalence estimates from a standard unit-level Bayesian model using Stan (Stan Development Team, 2024), by fitting the nested BYM2 regression model specified in (18) under the assumption that each $Y_c$ follows a negative binomial distribution with mean $N_c{r}_c$ and overdispersion parameter $\alpha$.

3. Second administrative level prevalence estimates from the soft DABUL model with the regression equation specified in (18) using Algorithm 1 for 1000 iterations, after 1000 iterations of burn-in, for 4 chains.

4. Second administrative level prevalence estimates from the hard DABUL model with the regression equation specified in (18) using Algorithm 2 for 1000 iterations, after 1000 iterations of burn-in, for 4 chains.

As in the motivating example, we aggregate the urban and rural estimates within each second administrative area using urban/rural population fractions. The run time of each DABUL model was approximately 130 minutes, while the run time of each standard unit-level model in Stan was approximately 2 minutes.

We compare the performance of the second administrative level prevalence estimates using four metrics. First, we quantify the discrepancy, or absolute difference, between the first administrative level aggregated estimates and the direct estimates, averaged across all simulations. For first administrative area $i$, the average discrepancy is calculated by

$$\frac{1}{500}\sum _{k=1}^{500}\mid \sum _{j\in \zeta (i)}{w}_j{\widehat{r}}_j^{(k)}-r_i^{D_1(k)}\mid$$

where ${\widehat{r}}_j^{(k)}$ is the second administrative level prevalence estimate for the $k^{\mathit{th}}$ sample dataset, $w_j$ is the population weight for second administrative area $j$, and $\zeta (i)$ is the set of second administrative areas in first administrative area $i$. Second, we evaluate the absolute error of the second administrative level estimates, averaged across all simulations, calculated by $\frac{1}{500}{\sum }_{k=1}^{500}\mid {\widehat{r}}_j^{(k)}-r_j\mid$, where $r_j$ is the population prevalence for second administrative area $j$. Then, we compare the coverage and average width of the second administrative level 90% credible intervals, calculated by

$$\frac{1}{500}\sum _{k=1}^{500}I(r_j\in [Q_j^{(k)}(5),Q_j^{(k)}(95)])$$

and

$$\frac{1}{500}\sum _{k=1}^{500}(Q_j^{(k)}(95)-Q_j^{(k)}(5)),$$

respectively, where $Q_j^{(k)}(q)$ is the $q^{\mathit{th}}$ quantile of the posterior distribution for second administrative area $j$ and sample dataset $k$. Additionally, coefficients of variation of the second administrative level estimates, calculated by $\frac{1}{500}{\sum }_{k=1}^{500}100\times \frac{S{D}_j^{(k)}}{{\widehat{r}}_j^{(k)}}$, are presented in Appendix E, where $S{D}_j^{(k)}$ is the standard deviation of the posterior distribution for second administrative area $j$ and sampled dataset $k$.

Figure 6 displays the discrepancies between aggregated second administrative level model-based estimates and first administrative level direct estimates. The top panel depicts the average discrepancies across simulations for each first administrative area, model, and simulation setting. From this figure we observe that discrepancies are uniformly lower among the aggregated soft DABUL estimates, compared to the aggregated standard unit-level model estimates. As expected, the discrepancies among the aggregated hard DABUL estimates are consistently near zero, as a result of enforcing a hard benchmarking constraint. Both of these observations are consistent across all four simulation settings. The bottom panel depicts the distribution of the average percent decrease in discrepancy of the soft DABUL estimates for each first administrative area, relative to the standard unit-level estimates, across all simulations, for each simulation setting.

Figure 6. Discrepancy between aggregated model-based second administrative level estimates and design-based first administrative level estimates, across 500 simulations for each of 4 settings.

We observe that the aggregated soft DABUL estimates have 58.4-63.9% lower discrepancy with the first administrative level direct estimates, on average, compared to aggregated estimates from the standard unit-level model. Percentage decrease in discrepancy is larger when the standard unit-level model has larger discrepancy with the direct estimates (scenario 1a). Figure S6 presented in Appendix E further shows this pattern.

Figure 7 displays the absolute error of the second administrative level estimates, i.e., the absolute value of the difference between the prevalence estimate and the true prevalence in that area. The top panel depicts the average absolute error across simulations for each second administrative area, model, and simulation setting. The bottom panel depicts the distribution of the average absolute error of each second administrative area across all simulations, for each model and simulation setting. This figure suggests that absolute error is comparable across all models and simulation settings, with a slightly lower average absolute error for standard unit-level model estimates.

Figure 7. Absolute error of second administrative level estimates, across 500 simulations for each of 4 settings.

Figures 8 and 9 compare the coverage and width of the 90% credible intervals for the second administrative level estimates. Note that coverage of area-specific Bayesian credible intervals in unit-level models (and that of frequentist confidence intervals, for that matter) are expected to attain the desired coverage on average, but the interval for each specific area does not necessarily have the target coverage (Yu and Hoff, 2018). Moreover, in an extensive simulation study, Osgood-Zimmerman and Wakefield (2023) found that coverage of area-specific credible intervals from unit-level models did not obtain the nominal level in a range of scenarios. From figures 8 and 9, we observe that the DABUL estimates have wider credible intervals and more consistent coverage at or above the target threshold, compared to the standard unit level model. This higher variability is due to the estimation of the additional parameters $\mathbf{Y}$ (and ${\mathbf{Y}}_{+}$ for the soft DABUL model). We observe that the coverage and width of the hard DABUL credible intervals are slightly lower than the soft DABUL credible intervals as a result of not accounting for the uncertainty of the benchmarks. While the credible intervals from the standard unit level model are narrower and have a marginal rate that is closer to the target rate, there are also significantly more areas for which the credible intervals have considerable undercoverage. From these results we observe that DABUL estimates provide more conservative credible intervals, with the benefit that a much higher proportion of areas do not have undercoverage.

Figure 8. Coverage of 90% credible intervals for second administrative level estimates, across 500 simulations for each of 4 settings.

Figure 9. Width of 90% credible intervals (in deaths per 1000 births) for second administrative level estimates, across 500 simulations for each of 4 settings.

6. APPLICATION TO ZAMBIA DHS DATA

We return to the example of NMR estimation in Zambia and compare the performance of the DABUL models to the standard unit-level model with nested spatial effects. We use the same priors and hyperpriors as in the motivating example. We use WorldPop population estimates (Tatem, 2017) and sampling frame information from the 2014 Zambia DHS (ICF International, 2014) to obtain population counts. The run time of each of the DABUL models was approximately 90 minutes, while the run time of the standard unit-level model in Stan was approximately 1 minute.

Figure 10 provides a map of the NMR estimates resulting from each of the three models and figure 11 compares the credible interval width for each second administrative area. A scatter plot comparing the NMR estimates from each of the DABUL models to the standard nested model is presented in Appendix F. From these figures, we observe that the NMR point estimates are fairly similar between models, with the DABUL estimates having less shrinkage towards the first administrative level. The credible interval widths from the DABUL estimates are uniformly wider than those resulting from the standard unit-level model, which is in agreement with the simulation results.

Figure 10. Map of NMR estimates at the second administrative level in Zambia (2009-2013) using the DABUL models, as compared to a standard unit-level Bayesian model, both with BYM2 spatial effects at the second administrative level nested based on first administrative level.

Figure 11. Width of 90% credible intervals of NMR estimates (in deaths per 1000 live births) at the second administrative level in Zambia (2009-2013) using the DABUL models, as compared to a standard unit-level Bayesian model.

In figure 12 we evaluate whether the aggregated soft DABUL model estimates have less discrepancy with the direct estimates than the aggregated standard model estimates do. For 6 out of the 10 regions, the DABUL model estimates have similar or less discrepancy with the direct estimates, while 5 of these regions have a significant reduction in discrepancy. Two of the regions for which discrepancy increases (Luapala and Central) have substantially higher uncertainty, with coefficients of variation of 30% and 31%, respectively, compared to the other first administrative area level direct estimates, which have coefficients of variation between 18% and 23%. This is an expected consequence of taking the uncertainty of the benchmarks into account: when uncertainty is higher, the estimates are less likely to adhere to the benchmark. Additionally, the DABUL models produce results with reduced shrinkage, so if the more populous second administrative areas happen to be more extreme areas, the aggregated estimate will be pulled in that direction. If estimates under the soft DABUL model do not have sufficient agreement in aggregation for a particular dataset, the practitioner may instead choose to use the hard DABUL model, with the warning that uncertainty of the benchmarks will not be accounted for.

Figure 12. NMR estimates at the second administrative level in Zambia (2009-2013) using the soft DABUL model, as compared to a standard unit-level nested Bayesian model.

The colored circles denote the second administrative level estimates and the colored diamonds denote their aggregations to the first administrative level. The black diamonds denote Hájek direct estimates at the first administrative level.

6.1. Model comparison

To compare performance of the DABUL models to the standard unit-level model in this data example, we use a leave-one-area-out cross-validation procedure. While there are cross-validation procedures for count data (Czado et al., 2009), these methods are difficult to use in the context of rare events because the resulting predictive distribution is generally too noisy to meaningfully distinguish predictive ability across the narrow relevant domain. Therefore, instead of leaving out one cluster at a time, we leave out all of the clusters in one second administrative area at a time, and then predict the logit transformed design-based estimate for that second administrative area (Wakefield et al., 2025). By using this approach, we avoid using the predictive distribution of a count variable, in favor of the predictive distribution of a Gaussian variable, which is much more stable. We systematically leave one second administrative area out at a time, and use each of the 3 models to predict the design-based estimate for that second administrative area. This means we can only evaluate predictive ability for second administrative areas which have valid design-based estimates and variances. This condition is met for 75% of second administrative areas (87 out of 115), as the remaining 25% have no observed deaths, so design-based estimates cannot be ascertained. While cross-validation is one of several ways we assess model performance, we emphasize that these results are presented with the caveat that there are valid criticisms of using cross-validation for spatially dependent data and a consensus on best practices has not yet been reached (Bürkner et al., 2020; Adin et al., 2024).

We use three metrics to evaluate predictive ability. First, we calculate absolute error, which is defined as $\mid {\widehat{r}}_{-j}-r_j^{D_2}\mid$, where ${\widehat{r}}_{-j}$ is the second administrative level estimate for area $j$, when area $j$ data is excluded from the model, and $r_j^{D_2}$ is the design-based estimate for second administrative area $j$. Note that this metric only evaluates accuracy of the point estimate prediction with respect to the design-based point estimate. We present this standard metric with caution because it may not be the best description of model performance, as it does not take the high uncertainty of $r_j^{D_2}$ into account. Second, we approximate the log score (or log-CPO) on the logit scale, which is defined as $\log p_{-j}(\mathrm{logit}(r_j^{D_2}))$, where $p_{-j}$ is the predictive density based on the model which excludes area $j$ data. This density does not have a closed form, but we can approximate it by

$$p_{-j}(\mathrm{logit}(r_j^{D_2}))\approx \frac{1}{B}\sum _{b=1}^B\varphi (\mathrm{logit}(r_j^{D_2})\mid \mathrm{logit}({\widehat{r}}_{-j}^{(b)}),V_j^{D_2})$$

where $\varphi (\cdot \mid \mathrm{logit}({\widehat{r}}_{-j}^{(b)}),V_j^{D_2})$ is the density of a normal random variable with mean, $\mathrm{logit}({\widehat{r}}_{-j}^{(b)})$, and variance, $V_j^{D_2}$, and ${\widehat{r}}_{-j}^{(b)}$ is the $b^{\mathit{th}}$ draw from the posterior distribution of second administrative level prevalence in area $j$, based on the model which excludes area $j$ data. This metric also evaluates the accuracy of the point estimate, but with respect to the whole distribution of the design-based estimate. Lastly, we calculate the interval score, a proper scoring rule (Gneiting and Raftery, 2007) which favors narrow intervals containing the true parameter value. Let $(l_{-j},u_{-j})$ denote the 90% credible interval for second administrative level prevalence in area $j$, based on the model which excludes area $j$ data. Then we define the interval score for administrative area $j$ as

$$u_{-j}-l_{-j}+\frac{2}{\alpha }\max \{0,l_{-j}-\mathrm{logit}({\widehat{r}}_{-j})\}+\frac{2}{\alpha }\max \{0,\mathrm{logit}({\widehat{r}}_{-j})-u_{-j,}\}$$

where we take $\alpha =0.1$. We average each of these 3 metrics over the 87 second administrative areas and compare them across models in table 2. The mean absolute error is very similar across all models, with the soft DABUL model have the lowest value (0.0122). The average log score is also quite similar across models, with the standard unit-level model having the highest value (0.440). The average interval score was lowest for the soft DABUL model (3.1) and highest for the standard unit-level model (4.2). This result shows that although the DABUL intervals are more conservative, this is offset by having better coverage, so, according to this proper scoring rule, the DABUL intervals are preferred over those of the standard unit-level model.

Table 2.

Summary of cross-validation metrics averaged over second administrative areas with valid design-based estimates (87 out of 115). The best scores are in bold.

	Standard unit-level (nested)	Soft DABUL	Hard DABUL
Mean absolute error	0.0125	0.0122	0.0126
Average log score	0.440	0.424	0.421
Average interval score	4.2	3.1	3.9

7. DISCUSSION

We first observe that in the case of rare events and small sample size, when there are multiple nested areas, using a model with a nested structure can help reduce overshrinkage. We have demonstrated the value of a nested unit-level Bayesian model for estimating rare event prevalence which utilizes design-based estimates at a higher aggregation level. We presented two models: the hard DABUL model which imposes a hard benchmarking constraint, and the soft DABUL model which takes uncertainty of the benchmark into account, thus imposing a soft constraint. In our simulation study and application to Zambia DHS data, we observe that the DABUL models produce conservative credible intervals with much lower frequency of undercoverage, compared to the standard unit-level model. Although the hard DABUL model slightly underestimates the variance by not taking the uncertainty of the direct estimates into account, the resulting credible intervals are still fairly conservative. For this reason, and for the obvious benefit of exact agreement in aggregation, our results suggest that the hard DABUL model may be preferable over the soft DABUL model in some cases. However, our cross-validation study suggests, via average interval scores, that while both DABUL models provide credible intervals which are superior to that of the standard unit-level model, the soft DABUL model provides the credible intervals with the best performance. Further study is necessary to determine the costs and benefits of choosing between the two DABUL models, however, the soft DABUL model remains the more conservative, principled approach because of its proper accounting for the large uncertainty of the benchmarks.

One criticism of the use of generalized linear unit-level models for complex survey data, is that they do not directly take sample design into account. Solutions have been developed for the case of linear models (You and Rao, 2002), but they do not extend to the generalized linear case (Wakefield et al., 2025). While the DABUL models do not completely solve this open problem, they are an improvement in the sense that they take design-based estimates at a higher aggregation level into account.

The DABUL models could also be extended to include covariates, which is encouraged whenever covariates with predictive power are available. We do not include covariates here because previous work has shown that available covariates in LMICs have little effect on predictive power in estimating NMR when spatial random effects are already included in the model (Golding et al., 2017,Wakefield et al., 2019). It is straightforward to include covariates in the DABUL models, though the aggregation step can introduce additional error.

Because the DABUL models, as derived, use a Negative Binomial sampling model, they require the outcome to be a rare event. While this model could theoretically be extended to estimate prevalence of non-rare events by using a Binomial or Beta-Binomial sampling model, these distributions do not enjoy the same additive properties as the Poisson and Negative Binomial distributions. As a result, the distributions of $Y_{i+}$ and ${\tilde{Y}}_i\mid Y_{i+}$ quickly become unwieldy as they require summing over a complete enumeration of the state space. Conversely, this model could easily be extended to continuous outcomes because the Normal distribution does share these additive properties. The only necessarily changes to the model would be to replace (8) and (9) with the normal distributions implied by the Gaussian likelihood.

Statement of significance:

This work contributes to the literature regarding small area estimation of rare event prevalence using sparse complex survey data - such applications are extremely important in low and middle income countries in particular. We introduce a Bayesian model which integrates design-based estimators into a unit-level hierarchical model, which encourages agreement between aggregated small area estimates and design-based estimates at a higher area level, which will be advantageous in global and public health settings.

Data Availability Statement

The processed 2014 Zambia DHS data used in this article are available in this article’s online supplementary material.