An elastic net penalized small area model combining unit- and area-level data for regional hypertension prevalence estimation

Abstract

Hypertension is a highly prevalent cardiovascular disease. It marks a considerable cost factor to many national health systems. Despite its prevalence, regional disease distributions are often unknown and must be estimated from survey data. However, health surveys frequently lack in regional observations due to limited resources. Obtained prevalence estimates suffer from unacceptably large sampling variances and are not reliable. Small area estimation solves this problem by linking auxiliary data from multiple regions in suitable regression models. Typically, either unit- or area-level observations are considered for this purpose. But with respect to hypertension, both levels should be used. Hypertension has characteristic comorbidities and is strongly related to lifestyle features, which are unit-level information. It is also correlated with socioeconomic indicators that are usually measured on the area-level. But the level combination is challenging as it requires multi-level model parameter estimation from small samples. We use a multi-level small area model with level-specific penalization to overcome this issue. Model parameter estimation is performed via stochastic coordinate gradient descent. A jackknife estimator of the mean squared error is presented. The methodology is applied to combine health survey data and administrative records to estimate regional hypertension prevalence in Germany.

1. Introduction

Hypertension is the most common cardiovascular disease worldwide. It is estimated that approximately 1.13 billion people aged 18 and older suffer from raised blood pressure globally [30]. Thus, it represents a major cost factor in the majority of national health systems that needs to be monitored closely. However, despite its high prevalence, the distribution of hypertension on regional levels is often unknown. Corresponding figures are rarely recorded in registries and must be estimated from suitable survey data instead. In order to obtain accurate estimates on regional levels, corresponding survey samples must be exhaustive and need to contain considerable geographic detail. But as a matter of fact, national health surveys often contain only a few observations per region (area) due to significant sampling efforts and limited resources. Direct estimators that only consider the observations within a given area then fail to provide prevalence estimates with sufficient accuracy as a result of large sampling variances.

Small area estimation (SAE) solves this problem by combining data from multiple areas in regression models. Estimation efficiency over a direct estimator is improved by exploiting the functional relation between an area statistic of interest and suitable auxiliary data. Area statistic estimates are obtained as predictions from the model [21]. Thereby, the estimates depend on the aggregation level of the auxiliary data. The most prominent SAE techniques are the area-level estimator by [11] and the unit-level estimator by [1]. Both approaches consider auxiliary data on a single aggregation level. But with respect to our application, it makes sense to consider both levels simultaneously. On the one hand, hypertension has characteristic comorbidities, such as type 2 diabetes [25], and is closely related to lifestyle variables, like alcohol consumption [20]. These features are typically unit-level information that are recorded in health surveys. On the other hand, hypertension is associated with socioeconomic indicators [26]. This information is often only available on the area-level, for example in terms of administrative records from official statistics. The joint usage of unit- and area-level data for SAE is not as well-established in the literature.

Some SAE methods use unit-level data while accounting for heterogeneity in fixed effects on the area-level, as for example proposed by [29]. This marks an important generalization of the nested error structure in the original unit-level approach by [1], which models differences between areas only via random intercept. Yet, it does not allow for the direct combination of unit- and area-level data sets as area-level heterogeneity is assumed to be due to random deviations from unit-level fixed effects. Twigg et al. [38] presented a multi-level approach to include both individual and ecological components to predict small area health-related behavior as a binary response. Still, the approach relies on a sequential procedure to calibrate a model on one data set first, and then use it in conjunction with another, which requires model specification to be very simple. Ghosh and Steorts [15] developed a two-stage benchmarking approach that combines unit- and area-level in a weighted loss function while benchmarking weighted means at both levels.

Using unit- and area-level data jointly encloses some methodological problems. Firstly, it requires model parameter estimation on both levels. In the presence of small samples, the increased number of parameters may lead to considerably high model parameter estimate variances due to the lack in degrees of freedom. In this case, model-based small area estimates also suffer from high variance. Secondly, unit- and area-level data have different distributional characteristics and correlation structures due to different degrees of aggregation [7]. As a result, the levels should not be treated equally in terms variable selection or model parameter estimation. Thirdly, unit- and area-level data is usually subject to different kinds of measurement errors. As ignoring measurement errors leads to suboptimal area statistic estimates, the researcher should account for this [27]. And finally, unit- and area-level data usually differs in terms of availability. Unit-level data is often rare due to privacy issues, whereas area-level data is less sensitive and easier to access, for example, from registries. Accordingly, an approach must deal with situations where there are a lot of variables on one level while there are only few on the other.

In the light of regional hypertension prevalence estimation, we develop a multi-level small area model with level-specific penalization to combine unit- and area-level data. Level-specific penalization refers to penalized maximum likelihood estimation of the model parameters where the fixed effect sets on each level are penalized individually. For this purpose, the elastic net [39] with level-specific weights on the included ${\ell }_1$-norm and squared ${\ell }_2$-norm is considered. Using level-specific penalization solves the methodological problems mentioned before. Firstly, it allows for high-dimensional inference. Hence, even if the number of model parameters surpasses the number of observations, the underlying optimization problem for model parameter estimation is still well-posed. This is particularly attractive in the presence of small samples. Secondly, level-specific penalization marks an intuitive way to treat unit- and area-level data differently for model parameter estimation. The penalties can be defined dependent on the distributional characteristics of the corresponding auxiliary data. Due to the sparsity-inducing effect of the ${\ell }_1$-norm, an automatic level-specific variable selection is conducted. Thirdly, penalized model parameter estimation implies a robustification against measurement errors in the auxiliary data, as shown by [2] as well as [4]. Accordingly, level-specific penalization allows for different measurement errors on each level. And finally, the amount of shrinkage on each level can be altered depending on the number of variables available for prediction.

Penalized maximum likelihood estimation of the model parameters is performed via stochastic coordinate gradient descent (SCGD) using insights from [34,37]. Random effect prediction is done using a maximum a posteriori estimation, as shown by [31,34]. Estimation of the mean squared prediction error is performed via a modified jackknife approach drawing from [4,22]. The methodology is applied to estimate federal state-level hypertension prevalence in Germany. We combine unit-level data from the German health survey Gesundheit in Deutschland Aktuell (GEDA) [33] with area-level data from microcensus records [18] to estimate regional hypertension prevalence. The remainder of the paper is organized as follows. In Section 2, the methodology is explained. Section 3 contains a small simulation study and the application to regional health measurement. Section 4 closes with an outlook on future research. Please note that the presented article contains insights from a related working paper by [5].

2. Methodology

2.1. Penalized multi-level model

The subsequent description is slightly more generic to allow for more generality with respect to potential other applications. Let $\mathcal{U}=\{1,\dots ,N\}$ be a finite population of $|\mathcal{U}|=N$ individuals indexed by $\iota =1,\dots ,N$. Assume that the population is segmented into m areas of size $|{\mathcal{U}}_i|=N_i$ such that $\mathcal{U}=\bigcup _{i=1}^m{\mathcal{U}}_i$ with ${\mathcal{U}}_i,{\mathcal{U}}_k$ pairwise disjoint for $i\ne k$ and $\sum _{i=1}^m{N}_i=N$. Let $\mathcal{S}\subset \mathcal{U}$ be a random sample of $|\mathcal{S}|=n\ge 2m$ individuals. Assume that sampling procedure is such that $\mathcal{S}=\bigcup _{i=1}^m{\mathcal{S}}_i$ where ${\mathcal{S}}_i\subset {\mathcal{U}}_i$ and $|{\mathcal{S}}_i|=n_i\ge 2$. Let ${\mathbf{y}}_i\in {\mathbb{R}}^{n_i\times 1}$ be a vector containing observations $y_{i\iota }\in \mathbb{R}$ of the response variable Y from which the area statistic of interest in area i, let us say the mean ${\overline{Y}}_i=N_i^{-1}\sum _{\iota \in {\mathcal{U}}_i}{y}_{i\iota }$, is calculated. Let ${\mathbf{X}}_{ui}\in {\mathbb{R}}^{n_i\times p_u}$ be the fixed effect design matrix in area i containing unit-level auxiliary data for the description of ${\mathbf{y}}_i$. For notational convenience, denote ${\mathbf{X}}_{ai}=({\mathbf{x}}_{ai}^{\mathrm{\prime }},\dots ,{\mathbf{x}}_{ai}^{\mathrm{\prime }}{)}^{\mathrm{\prime }}\in {\mathbb{R}}^{n_i\times p_a}$ as the fixed effect design matrix resulting from an expansion of the vector ${\mathbf{x}}_{ai}\in {\mathbb{R}}^{1\times p_a}$ containing area-level auxiliary data. Note that $(p_u+p_a):=p>n$ is allowed. Let ${\mathbf{Z}}_i\in {\mathbb{R}}^{n_i\times q}$ be the random effect design matrix in area i with $q\le p$. In the majority of SAE models, the random effect structure is limited to an area-specific random intercept. However, as they are typically special cases of linear mixed models [31], we allow for a more general area-specific random effect structure. The multi-level model combining unit- and area-level data is given by

$${\mathbf{y}}_i={\mathbf{X}}_{ui}{\mathit{\beta }}_u+{\mathbf{X}}_{ai}{\mathit{\beta }}_a+{\mathbf{Z}}_i{\mathit{b}}_i+{\mathbf{e}}_i\mathrm{\forall }i=1,\dots ,m,$$ (1)

where ${\mathit{\beta }}_u\in {\mathbb{R}}^{p_u\times 1}$, ${\mathit{\beta }}_a\in {\mathbb{R}}^{p_a\times 1}$ are the fixed effect coefficient vectors for each level and ${\mathbf{b}}_i\sim MVN(\mathbf{0},\mathbf{\Psi })$ denotes the random effect coefficient vector under multivariate normality with some general positive-definite covariance matrix $\mathbf{\Psi }$. ${\mathbf{e}}_i\sim MVN(\mathbf{0},{\sigma }^2{\mathbf{I}}_{n_i})$ is a vector of independent and identically distributed random errors with model variance parameter ${\sigma }^2$. Note that ${\mathit{b}}_1,\dots ,{\mathit{b}}_m$, ${\mathbf{e}}_1,\dots ,{\mathbf{e}}_m$ are assumed to be stochastically independent. The response vector is multivariate normal under the model

$${\mathbf{y}}_i\sim MVN({\mathbf{X}}_{ui}{\mathit{\beta }}_u+{\mathbf{X}}_{ai}{\mathit{\beta }}_a,{\mathbf{V}}_i({\sigma }^2,\mathit{\psi }))\mathrm{\forall }i=1,\dots ,m,$$ (2)

with ${\mathbf{V}}_i({\sigma }^2,\mathit{\psi })={\sigma }^2{\mathbf{I}}_{n_i}+{\mathbf{Z}}_i\mathbf{\Psi }{\mathbf{Z}}_i^{\mathrm{\prime }}$, where the random effect covariance matrix $\mathbf{\Psi }$ is parameterized by a vector $\mathit{\psi }\in {\mathbb{R}}^{q^{\ast }}$, for example, resulting from a Cholesky decomposition. We assume that $q^{\ast }<n$. Restating the model over all areas obtains

$$\mathbf{y}={\mathbf{X}}_u{\mathit{\beta }}_u+{\mathbf{X}}_a{\mathit{\beta }}_a+\mathbf{Z}\mathit{b}+\mathbf{e},$$ (3)

with ${\mathbf{X}}_u=({\mathbf{X}}_{u1}^{\mathrm{\prime }},\dots ,{\mathbf{X}}_{um}^{\mathrm{\prime }}{)}^{\mathrm{\prime }}$, ${\mathbf{X}}_a=({\mathbf{X}}_{a1}^{\mathrm{\prime }},\dots ,{\mathbf{X}}_{am}^{\mathrm{\prime }}{)}^{\mathrm{\prime }}$, $\mathbf{Z}=diag({\mathbf{Z}}_1,\dots ,{\mathbf{Z}}_m)$ as stacked matrices and $\mathbf{y}=({\mathbf{y}}_1^{\mathrm{\prime }},\dots ,{\mathbf{y}}_m^{\mathrm{\prime }}{)}^{\mathrm{\prime }}$, $\mathbf{b}=({\mathbf{b}}_1^{\mathrm{\prime }},\dots ,{\mathbf{b}}_m^{\mathrm{\prime }}{)}^{\mathrm{\prime }}$, $\mathbf{e}=({\mathbf{e}}_1^{\mathrm{\prime }},\dots ,{\mathbf{e}}_m^{\mathrm{\prime }}{)}^{\mathrm{\prime }}$ as stacked vectors. Define $\mathit{\theta }:=({\mathit{\beta }}_u^{\mathrm{\prime }},{\mathit{\beta }}_a^{\mathrm{\prime }},{\mathit{\psi }}^{\mathrm{\prime }},{\sigma }^2{)}^{\mathrm{\prime }}\in {\mathbb{R}}^{p_u+p_a+q^{\ast }+1}$ as the full parameter vector. The negative log-likelihood function of (3) is given by

$$\begin{array}{rl}-\mathcal{L}(\mathit{\theta })& =\frac{1}{2}[n\log (2\pi )+\log (\det (\mathbf{V}))\\ & +{(\mathbf{y}-{\mathbf{X}}_u{\mathit{\beta }}_u-{\mathbf{X}}_a{\mathit{\beta }}_a)}^{\mathrm{\prime }}{\mathbf{V}}^{-1}(\mathbf{y}-{\mathbf{X}}_u{\mathit{\beta }}_u-{\mathbf{X}}_a{\mathit{\beta }}_a)],\end{array}$$ (4)

with $\mathbf{V}=diag({\mathbf{V}}_1,\dots ,{\mathbf{V}}_m)$ and $\det (\mathbf{V})$ denoting the determinant of $\mathbf{V}$. In a standard maximum likelihood framework, model parameter estimation is performed according to $\hat{\mathit{\theta }}=\mathrm{argmin}-\mathcal{L}(\mathit{\theta })$. However, in the light of the methodological issues mentioned in Section 1, we expand the negative log-likelihood by level-specific penalties. For this, the elastic net penalty introduced by [39] is used. It can be viewed as a compromise between the LASSO [36] and ridge regression [19]. The LASSO component contains the ${\ell }_1$-norm of the fixed effect coefficients. It conducts an automatic variable selection by favoring sparse solutions to a given estimation problem. The ridge component contains the squared ${\ell }_2$-norm. It stabilizes model parameter estimates in the presence of strong correlation within the covariates by favoring smooth solutions. The elastic net is a linear convex combination of the two components incorporating both of these properties. The optimization problem for model parameter estimation is stated as

$$\hat{\mathit{\theta }}=\underset{{\mathit{\beta }}_u,{\mathit{\beta }}_a,\mathit{\psi },{\sigma }^2>0}{\mathrm{argmin}}\{\mathcal{Q}(\mathit{\theta },{\lambda }_u,{\lambda }_a,{\alpha }_u,{\alpha }_a)=-\mathcal{L}(\mathit{\theta })+{\lambda }_u\mathcal{P}({\alpha }_u,{\mathit{\beta }}_u)+{\lambda }_a\mathcal{P}({\alpha }_a,{\mathit{\beta }}_a)\},$$ (5)

where ${\lambda }_u,{\lambda }_a>0$ are predefined penalty parameters that regulate the effect of the penalty on the solutions for each level. They are typically determined by some form of cross validation [34]. In addition to that

$$\mathcal{P}({\alpha }_l,{\mathit{\beta }}_l)=(1-{\alpha }_l)\Vert {\mathit{\beta }}_l{\Vert }_2^2+{\alpha }_l\Vert {\mathit{\beta }}_l{\Vert }_1,l\in \{u,a\}$$ (6)

is the elastic net penalty with ${\alpha }_l\in (0,1)$ as predefined hyper parameter controlling the contribution of the ${\ell }_1$-norm and ${\ell }_2$-norm to the overall penalization on a level. In the light of our application, using the elastic net with level-specific penalty and hyper parameters is attractive. It allows to adjust the penalization to the data situation on each level individually. Recall that the response variable has been observed on the unit-level. With respect to the unit-level covariates, the health survey data may contain insights on comorbidities of hypertension, but clearly also a variety of records regarding unrelated health issues. Thus, it is useful to choose ${\alpha }_u>0.5$ and put a higher weight on the LASSO component to induce more sparsity. Regarding the area-level covariates, the socioeconomic indicators are likely to have stronger covariance structures as they are typically calculated from the same set of variables. Hence, it makes sense to choose ${\alpha }_a<0.5$ and put a higher weight on the ridge component to obtain a smooth solution for the corresponding parameter estimates.

2.2. Model parameter estimation

A SCGD algorithm is used for model parameter estimation. We draw from [34,37] and modify their (block) coordinate gradient descent (CGD) method by a randomized cycling order. This improves the convergence probability in the light of the non-convexity of (5), as an unfortunate series of coordinates less likely to occur [3]. CGD implies that the value of the objective function is minimized gradually by updating a single element of the target parameter vector $\mathit{\theta }$ at a time while keeping the others fixed. Thereafter, the remaining elements are updated accordingly such that a cyclic movement through all coordinates of $\mathit{\theta }$ is achieved. This approach is particularly useful for the proposed multi-level model, as it allows for easy implementation of level-specific penalization in the estimation process.

Due to the unknown variance parameters ${\sigma }^2$, $\mathit{\psi }$ in the negative log-likelihood  (4), the minimization problem (5) is non-convex. This complicates model parameter estimation significantly, as the algorithm is not guaranteed to achieve the global minimum. The non-convexity of the objective function $\mathcal{Q}(\cdot )$ favors the existence of local minima, which implies that the resulting model parameter estimates may be sensitive to starting values. However, the minimization with respect to ${\mathit{\beta }}_u,{\mathit{\beta }}_a$ is convex under fixed variance parameters. Following [34], this can be exploited in the estimation process. For $s^{[t]}=1,2,\dots$, let ${\mathcal{R}}_{s^{[t]}}$ be the index cycling through the coordinates $\{1\},\{2\},\dots ,\{p+q^{\ast }+1\}$ in the tth iteration of the algorithm. Note that the order of the coordinates changes randomly after each iteration. Let ${\theta }_{{\mathcal{R}}_{s^{[t]}}}^{[t]}$ denote the sth element of ${\mathit{\theta }}^{[t]}$, where ${\mathit{\theta }}^{[t]}$ is the full parameter vector in the tth iteration. In general, an update is given by

$${\theta }_{{\mathcal{R}}_{s^{[t]}}}^{[t+1]}={\theta }_{{\mathcal{R}}_{s^{[t]}}}^{[t]}-a_{s^{[t]}}^{[t]}{d}_{s^{[t]}}^{[t]},$$ (7)

where $d_{s^{[t]}}^{[t]}$ is the descent direction and $a_{s^{[t]}}^{[t]}$ is the step size. A common choice would be $d_{s^{[t]}}^{[t]}:=\mathrm{\partial }\mathcal{Q}({\mathit{\theta }}^{[t]})/\mathrm{\partial }{\theta }_{\mathcal{R}{s}^{[t]}}$ and $a_{s^{[t]}}^{[t]}:=({\mathrm{\partial }}^2\mathcal{Q}({\mathit{\theta }}^{[t]})/\mathrm{\partial }{\theta }_{\mathcal{R}{s}^{[t]}}^2{)}^{-1}$. However, if ${\theta }_{{\mathcal{R}}_{s^{[t]}}}^{[t]}$ is subject to elastic net penalization, the first and second partial derivatives do not exist. In that case, $d_{s^{[t]}}^{[t]}$ and $a_{s^{[t]}}^{[t]}$ have to be determined differently. Let $h_{s^{[t]}}^{[t]}$ be an approximation of ${\mathrm{\partial }}^2\mathcal{Q}({\mathit{\theta }}^{[t]})/\mathrm{\partial }{\theta }_{\mathcal{R}{s}^{[t]}}^2$. Following [37], we set $h_{s^{[t]}}^{[t]}:=min(max(\mathcal{I}({\mathit{\theta }}^{[t]}{)}_{{\mathcal{R}}^{s^{[t]}}{\mathcal{R}}^{s^{[t]}}},{10}^{-6}),{10}^8)$, where $\mathcal{I}({\mathit{\theta }}^{[t]}{)}_{{\mathcal{R}}_{s^{[t]}}{\mathcal{R}}_{s^{[t]}}}$ is the main diagonal element of the Fisher information matrix corresponding to ${\theta }_{{\mathcal{R}}_{s^{[t]}}}^{[t]}$. If $\mathcal{I}({\mathit{\theta }}^{[t]}{)}_{{\mathcal{R}}_{s^{[t]}}{\mathcal{R}}_{s^{[t]}}}$ is not truncated, we can built upon an analytic update proposed by [34] for the LASSO that exploits $\mathcal{L}(\mathit{\theta })$ being quadratic with respect to the fixed effect coefficients. For the elastic net penalty, we induce additional shrinkage to the LASSO update through dividing by $1+{\lambda }_l(1-{\alpha }_l)$, as suggested by [12]. If $\mathcal{I}({\mathit{\theta }}^{[t]}{)}_{{\mathcal{R}}_{s^{[t]}}{\mathcal{R}}_{s^{[t]}}}$ is truncated, $d_{s^{[t]}}^{[t]}$ is obtained according to [37] and $a_{s^{[t]}}^{[t]}$ is determined via the Armijo Rule. We further use an active-set strategy [12,34]. This implies that ${\theta }_{{\mathcal{R}}_{s^{[t]}}}^{[t]}$ is only updated when ${\theta }_{{\mathcal{R}}_{s^{[t]}}}^{[t-1]}\ne 0$. Let ${\tilde{\mathbf{y}}}_{s^{[t]}}^{[t]}$ be the prediction of $\mathbf{y}$ excluding the contribution of ${\theta }_{{\mathcal{R}}_{s^{[t]}}}^{[t]}$. The complete SGCD procedure is stated hereafter.

2.3. Random effect prediction and area statistic estimates

From Section 2.1, we can conclude that the conditional distribution of ${\mathbf{y}}_i$ given ${\mathbf{b}}_i$ is

$${\mathbf{y}}_i|{\mathbf{b}}_i\sim MVN({\mathbf{X}}_{ui}{\mathit{\beta }}_u+{\mathbf{X}}_{ai}{\mathit{\beta }}_a+{\mathbf{Z}}_i{\mathbf{b}}_i,{\sigma }^2{\mathbf{I}}_{n_i})\mathrm{\forall }i=1,\dots ,m.$$ (8)

In order to predict the random effect realizations, the conditional distribution of the random effects given the response variable realizations ${\mathbf{b}}_i|{\mathbf{y}}_i$ must be quantified. The mode of this distribution is the best predictor (BP) for ${\mathbf{b}}_i$. This is often referred to as maximum a posteriori estimation. Using the Bayes theorem, we obtain [34]

$$\begin{array}{rl}{\hat{\mathbf{b}}}_i^{BP}& =\underset{{\mathbf{b}}_i}{\mathrm{argmax}}\varphi ({\mathbf{b}}_i|{\mathbf{y}}_1,\dots ,{\mathbf{y}}_m,{\mathit{\beta }}_u,{\mathit{\beta }}_a,\mathit{\psi },{\sigma }^2)\\ & =\underset{{\mathbf{b}}_i}{\mathrm{argmax}}\varphi ({\mathbf{b}}_i|{\mathbf{y}}_i,{\mathit{\beta }}_u,{\mathit{\beta }}_a,\mathit{\psi },{\sigma }^2)\\ & =\underset{{\mathbf{b}}_i}{\mathrm{argmax}}{\displaystyle \frac{\varphi ({\mathbf{y}}_i|{\mathbf{b}}_i,{\mathit{\beta }}_u,{\mathit{\beta }}_a,\mathit{\psi },{\sigma }^2)\cdot \varphi ({\mathbf{b}}_i|\mathit{\psi })}{\varphi ({\mathbf{y}}_i|{\mathit{\beta }}_u,{\mathit{\beta }}_a,\mathit{\psi },{\sigma }^2)}}\\ & =\underset{{\mathbf{b}}_i}{\mathrm{argmin}}{\displaystyle \frac{1}{{\sigma }^2}}\Vert {\mathbf{y}}_i-{\mathbf{X}}_{ui}{\mathit{\beta }}_u-{\mathbf{X}}_{ai}{\mathit{\beta }}_a-{\mathbf{Z}}_i{\mathit{b}}_i{\Vert }_2^2+{\mathbf{b}}_i^{\mathrm{\prime }}{\mathbf{\Psi }}^{-1}{\mathbf{b}}_i,\end{array}$$ (9)

with φ as normal probability density. The minimization problem in (9) has a closed-form solution under the model assumptions, which is given by

$${\hat{\mathbf{b}}}_i^{BP}=E({\mathbf{b}}_i|{\mathbf{y}}_i)={({\mathbf{Z}}_i^{\mathrm{\prime }}{\mathbf{Z}}_i+{\sigma }^2{\mathbf{\Psi }}^{-1})}^{-1}{\mathbf{Z}}_i^{\mathrm{\prime }}({\mathbf{y}}_i-{\mathbf{X}}_{ui}{\mathit{\beta }}_u-{\mathbf{X}}_{ai}{\mathit{\beta }}_a).$$ (10)

Since in practice the model parameters $\mathit{\theta }$ are unknown, we use the empirical BP

$${\hat{\mathbf{b}}}_i^{EBP}={({\mathbf{Z}}_i^{\mathrm{\prime }}{\mathbf{Z}}_i+{\hat{\sigma }}^2{\hat{\mathbf{\Psi }}}^{-1})}^{-1}{\mathbf{Z}}_i^{\mathrm{\prime }}({\mathbf{y}}_i-{\mathbf{X}}_{ui}{\hat{\mathit{\beta }}}_u-{\mathbf{X}}_{ai}{\hat{\mathit{\beta }}}_a),$$ (11)

with the estimates obtained from the minimization of $\mathcal{Q}(\cdot )$. For the estimation of the area statistic ${\overline{Y}}_i$, predictions from the multi-level model have to be generated. The BP of ${\overline{Y}}_i$ under the model is given by

$${\hat{\overline{Y}}}_i^{BP}=E({\overline{Y}}_i|{\mathbf{y}}_i)=\frac{1}{N_i}[\sum _{\iota \in {\mathcal{S}}_i}{y}_{i\iota }+\sum _{\iota \in {\mathcal{U}}_i\setminus {\mathcal{S}}_i}({\mathbf{x}}_{ui\iota }{\mathit{\beta }}_u+{\mathbf{x}}_{ai}{\mathit{\beta }}_a+{\mathbf{z}}_{i\iota }{\hat{\mathbf{b}}}_i^{BP})].$$ (12)

However, (12) demands that the unit-level vector ${\mathbf{x}}_{ui\iota }$ to be observed for all $\iota \in \mathcal{U}$, which can be unrealistic in practice. An alternative arises from exploiting the SAE setting $n_i/N_i\approx 0$ by means of the small sample approximation proposed by [1]. We substitute the population-based area statistic by a model-based quantity

$${\overline{Y}}_i\approx {\mu }_i={\overline{\mathbf{x}}}_{ui}{\mathit{\beta }}_u+{\mathbf{x}}_{ai}{\mathit{\beta }}_a+{\mathbf{z}}_i{\mathbf{b}}_i\mathrm{\forall }i=1,\dots ,m,$$ (13)

where ${\overline{\mathbf{x}}}_{ui}$ is the area-specific mean of ${\mathbf{x}}_{ui\iota }$. Using the model parameter estimation obtained before, the empirical BP is then given by

$${\hat{\mu }}_i^{EBP}={\overline{\mathbf{x}}}_{ui}{\hat{\mathit{\beta }}}_u+{\mathbf{x}}_{ai}{\hat{\mathit{\beta }}}_a+{\mathbf{z}}_i{\hat{\mathbf{b}}}_i^{EBP}\mathrm{\forall }i=1,\dots ,m.$$ (14)

2.4. Mean squared error estimation

Based on the small sample approximation (13), we now address mean squared error (MSE) estimation. Generally, the MSE of the empirical BP is

$$MSE({\hat{\mu }}_i^{EBP})=E[({\hat{\mu }}_i^{EBP}-{\mu }_i{)}^2]\mathrm{\forall }i=1,\dots ,m.$$ (15)

The estimation of (15) is a difficult task due to the nonlinearity of the predictor. In SAE, depending on the model, second-order approximations to analytical solutions for the MSE may be available. Corresponding contributions were provided by for example [9,23,32], as well as [8]. However, these approximations are not applicable for the penalized multi-level model for several reasons. On the one hand, they do not consider penalization and the resulting shrinkage of the regression coefficients. On the other hand, they are not suitable for high-dimensional settings and they don't consider multi-level data. As far as we know, the derivation of an analytical MSE estimator in this context is still subject to ongoing research. A common alternative to analytical MSE approximations is resampling methods. Famous techniques are the bootstrap or the jackknife [10,22]. Corresponding contributions to SAE were provided by, for example, [17,24], as well as [6]. These methods seek to approximate the distribution of $({\hat{\mu }}_i^{EBP}-{\mu }_i{)}^2$ via resampling and have also been applied in SAE models with penalization. On that note, Burgard et al. [4] presented a modified jackknife approach for MSE estimation in a penalized area-level model. The basic idea is to first derive the conditional MSE of the best predictor under known model parameters given the design matrix. Afterwards, a delete-1-jackknife procedure is applied to account for the additional uncertainty resulting from penalized model parameter estimation. In the following, we extend this approach in order to apply it to the multi-level model in Section 2.1. The conditional MSE can be characterized according to [28]

$$\begin{array}{rl}MSE({\hat{\mu }}_i^{BP}|{\overline{\mathbf{x}}}_{ui},{\mathbf{x}}_{ai})& =E[{({\hat{\mu }}_i^{BP}-{\mu }_i)}^2|{\overline{\mathbf{x}}}_{ui},{\mathbf{x}}_{ai}]\\ & =E\left[{({\hat{\mathbf{b}}}_i^{BP}-{\mathbf{b}}_i)}^2\right]\\ & =E\left[E\left[{({\hat{\mathbf{b}}}_i^{BP}-{\mathbf{b}}_i)}^2|{\mathbf{b}}_i\right]\right]\\ & =E[Var\left({\hat{\mathbf{b}}}_i^{BP}-\mathbf{b}|{\mathbf{b}}_i\right)+E{\left[({\hat{\mathbf{b}}}_i^{BP}-{\mathbf{b}}_i)|{\mathbf{b}}_i\right]}^2]\\ & =E[Var\left({\hat{\mathbf{b}}}_i^{BP}|{\mathbf{b}}_i\right)+{(E\left[{\hat{\mathbf{b}}}_i^{BP}|{\mathbf{b}}_i\right]-{\mathbf{b}}_i)}^2].\end{array}$$ (16)

Naturally, the representation of the conditional MSE is determined by the random effect structure. For simplicity, we assume that the latter is limited to a random intercept. Hence ${\mathbf{b}}_i=b_i$ is a random scalar with $b_i\sim N(0,\psi )$ and ${\mathbf{z}}_i$ is a vector of 1s. See [32] for representations with other random effect structures. In our setting, we obtain

$$\begin{array}{rl}MSE({\hat{\mu }}_i^{BP}|{\overline{\mathbf{x}}}_{ui},{\mathbf{x}}_{ai})& =E[{\left(\frac{\psi }{\psi +{\sigma }^2/n_i}\right)}^2\frac{{\sigma }^2}{n_i}+{(\frac{\psi }{\psi +{\sigma }^2/n_i}{b}_i-b_i)}^2]\\ & ={\left(\frac{\psi }{\psi +{\sigma }^2/n_i}\right)}^2\frac{{\sigma }^2}{n_i}+{(\frac{\psi }{\psi +{\sigma }^2/n_i}-1)}^2\psi \\ & =\frac{\psi {\sigma }^2/n_i}{\psi +{\sigma }^2/n_i}.\end{array}$$ (17)

Define $\mathit{\beta }:=({\mathit{\beta }}_u^{\mathrm{\prime }},{\mathit{\beta }}_a^{\mathrm{\prime }}{)}^{\mathrm{\prime }}$ and $\mathit{\eta }:=(\psi ,{\sigma }^2{)}^{\mathrm{\prime }}$. Let

$${\gamma }_i:={\gamma }_i\left(\hat{\mathit{\eta }}\right)=\frac{\hat{\psi }{\hat{\sigma }}^2/n_i}{\hat{\psi }+{\hat{\sigma }}^2/n_i}.$$ (18)

Further, recall that ${\hat{\mu }}_i^{EBP}={\hat{\mu }}_i^{EBP}(\hat{\mathit{\beta }},{\hat{b}}_i)$ and ${\hat{b}}_i={\hat{b}}_i(\hat{\mathit{\beta }},\hat{\mathit{\eta }})$, where $\hat{\mathit{\beta }}$ and $\hat{\mathit{\eta }}$ are estimated from the sample observations of all m areas. Let ${\hat{\mathit{\beta }}}_{-i}$ and ${\hat{\mathit{\eta }}}_{-i}$ denote estimates of $\mathit{\beta }$ and $\mathit{\eta }$ that have been calculated from all areas except area i. The jackknife is sketched in Algorithm 2. After completing the algorithm, the jackknife estimator for the unconditional MSE of the empirical BP is estimated by

$$\begin{array}{rl}\widehat{MSE}\left({\hat{\mu }}_i^{EBP}\right)& ={\gamma }_i(\hat{\mathit{\eta }})-\frac{m-1}{m}\sum _{k=1}^m[{\gamma }_i({\hat{\mathit{\eta }}}_{-k})-{\gamma }_i(\hat{\mathit{\eta }})]\\ & +\frac{m-1}{m}\sum _{k=1}^m{[{\hat{\mu }}_i^{EBP}({\hat{\mathit{\beta }}}_{-k},{\hat{b}}_i({\hat{\mathit{\beta }}}_{-k},{\hat{\mathit{\eta }}}_{-k}))-{\hat{\mu }}_i^{EBP}(\hat{\mathit{\beta }},{\hat{b}}_i(\hat{\mathit{\beta }},\hat{\mathit{\eta }}))]}^2.\end{array}$$ (19)

3. Simulation and application

The subsequent section is divided into two parts. In the first part, we provide a small simulation study to demonstrate the effectiveness of the penalized multi-level model for SAE. We further test the presented jackknife approach for MSE estimation. In the second part, the multi-level small area model from Section 2.1 is used to provide point estimates of the regional hypertension prevalence in Germany.

3.1. Simulation study

3.1.1. Set up

A Monte Carlo simulation with R = 500 iterations ( $r=1,\dots ,R$) is conducted. We create a synthetic population of $N=\mathrm{30,000}$ individuals in m = 100 areas of size $N_i=300$. The population is generated once and held fixed for the subsequent simulation. In each iteration, a stratified random sample of size n = 300 with strata sample size $n_i=3$ is drawn from the synthetic population. Here, each stratum corresponds to one area of the synthetic population, which implies a sampling fraction of 1 $\mathrm{\%}$ per stratum. For unit-level auxiliary data, a total of 40 variables with a weak internal correlation structure is drawn from a multivariate normal with area-specific means. For area-level auxiliary data, 100 variables with a strong internal correlation structure is drawn from a multivariate normal. The response variable is created on the unit-level according to

$$y_{i\iota }=100+{\beta }_{u1}{x}_{u1i\iota }+{\beta }_{u2}{x}_{u2i\iota }+{\beta }_{a1}{x}_{a1i}+{\beta }_{a2}{x}_{a2i}+b_i+e_{i\iota },$$ (20)

where $b_i\sim N(0,{200}^2)$ is a random area intercept and $e_{i\iota }\sim N(0,{100}^2)$ is a unit-level error term. Note that from the unit-level and area-level auxiliary data sets described above, only 2 variables per set is relevant for the functional description of Y. This is done in order to include variable selection aspects in the simulation study. The area statistic of interest is the area-specific mean of the response variable: ${\overline{Y}}_i=N_i^{-1}\sum _{\iota \in {\mathcal{U}}_i}^{N_i}{y}_{i\iota }$. In order to estimate ${\overline{Y}}_i$, the following predictors are considered:

• LMM.Oracle: EBP under the true multi-level model (18) with known covariates for all units of the population.

• FH.Oracle: Fay-Herriot EBP considering the true auxiliary variables, but using the true area-specific means of $x_{u1i\iota },x_{u2i\iota }$ as substitute for unit-level data.

• LMM.Select: EBP under a multi-level model where variable selection is performed via the corrected conditional AIC proposed by [16].

• Multi.EN: Prediction from the penalized multi-level model.

The EBP under the original unit-level model proposed by [1] is not included since it exclusively considers unit-level variables. It cannot provide any reasonable results given the way the response variable is generated. However, the EBP under the Fay–Herriot model is included, as the unit-level variables can be aggregated in order to use them on the area-level. Point estimation performance is evaluated via the relative root mean squared error (relative RRMSE) over all areas and Monte Carlo iterations. We further look at the relative bias and the coefficient of variation. The performance of the MSE estimation is measured via relative RRMSE and relative bias.

3.1.2. Results

We start with the point estimation. Table 1 shows the performance of the considered predictors. It can be seen that the penalized multi-level model obtains efficient results compared the standard SAE approaches. Its relative RRMSE is the lowest among all predictors except for the LMM.Oracle. The latter is the most efficient predictor, which was expected since it serves as a reference within the simulation. It has perfect information by knowing the true model and the covariate values for all individuals in each area. LMM.Select is slightly less efficient than FH.Oracle despite using information from both the unit- and the area-level. This is due to additional uncertainty resulting from multi-level covariate selection with a single-level information criterion, as LMM.Select does not know the true model. An interesting aspect is that Multi.EN has the highest relative bias of all predictors. This is due to the fact that penalized maximum likelihood introduces bias to model parameter estimation. On the other hand, it stabilizes model predictions, as can be seen by the coefficient of variation. The penalized multi-level model has the lowest variation among all predictors, including LMM.Oracle. This makes it ultimately more efficient than the standard predictors.

Table 1. Point estimation results.

Predictor	Relative Bias	Coeff.Variation	Relative RMSE
LMM.Oracle	0.00063	0.01977	0.01667
FH.Oracle	0.00094	0.02606	0.02313
LMM.Select	0.00026	0.02938	0.02368
Multi.EN	0.00128	0.01951	0.01960

We continue with the MSE estimation results obtained from the jackknife procedure in Section 2.4. They are visualized in Figure 1. The density of the relative MSE estimate deviations over all areas and Monte Carlo iterations are depicted. They are given by $(\widehat{\mathrm{MSE}}({\hat{\mu }}_i{)}^{[r]}-\mathrm{MSE}({\hat{\mu }}_i{)}^{[r]})/\mathrm{MSE}({\hat{\mu }}_i{)}^{[r]}$. We see that the jackknife obtains reasonable estimates on average. The mean of the distribution is close to zero. However, it has a slight tendency to overestimation. The overall relative bias is 0.040, whereas the relative RRMSE is 0.217. Further, some right-skewness is clearly evident. This is mainly due to estimation outliers. On the one hand, the methodology is sensitive to certain data constellations as a result of the non-convexity of the underlying optimization problem for model parameter estimation. And on the other hand, the ${\ell }_1$-norm induces different degrees of sparsity in the resampling process, which results in different sets of active predictors. Since in the jackknife procedure the squared deviations to the original predictions are added up, this can lead to large MSE estimates.

Figure 1.

MSE estimation results.

3.2. Application

The objective is to estimate the hypertension prevalence for the population of age 18+ on the federal state level. The definition of the disease profile is adapted from [33]. We combine two different data sources for this purpose. The first data source is the German health survey Gesundheit in Deutschland Aktuell (GEDA) from 2010. It is a national health survey of roughly 20,000 participants of age 18+ that are interviewed via CATI within a national representative telephone sample. The survey contains medical and lifestyle-related information that are used as unit-level data source within our study. For further information on the survey as well as its respective sampling design and response rates, we refer to [33], pp. 173. The second data source is administrative records on the federal state level that are obtained from the German microcensus 2010. The microcensus is a large-scale survey that covers 1 $\mathrm{\%}$-sample of the German population with a single-stage stratified cluster sampling design. The data is collected via CAPI. The microcensus contains (among others) sociodemographic and economic information that we use to maximize the explanatory power for hypertension prevalence estimation. For deeper insights into the survey as well as its sampling design and data collection procedures, see [13].

Regarding the level-specific penalization, the hyper parameters ${\alpha }_u=0.75,{\alpha }_a=0.25$ are used. This choice is in accordance with Section 2.1, where we addressed the effects of the elastic net's components. From GEDA, we have an extended amount of variables that cover different aspects of health. In order to identify the variables that are relevant for hypertension, we choose ${\alpha }_u$ closer to put emphasis on variable selection. From the microcensus, we have socioeconomic indicators that have strong internal correlation. Therefore, we choose ${\alpha }_a$ close to zero in order to obtain smooth coefficient estimates for them. The tuning parameters ${\lambda }_u,{\lambda }_a$ are determined by k-fold cross-validation with a bivariate grid search. Hereafter, we provide a brief overview of selected covariates for hypertension prevalence estimation. For further insights on the statistical properties of variable selection via penalized maximum likelihood, see [14]. We also increase the determined level-specific tuning parameter values gradually in order to assess the relevance of the selected fixed effect for the regional hypertension prevalence within the underlying regression model. With this, we obtain a rough measure of significance for the selected covariates. We distinguish three levels of significance: strong (***), medium (**), weak (*). From GEDA, exemplary unit-level variables are

• Demography: sex***, age group***.

• Comorbidity: having other cardiovascular diseases***.

• Lifestyle: smoking or drinking***, sport activities***.

• Medical care: visits to the doctor**, health insurance membership*.

• Living conditions: degree of urbanisation**.

From the microcensus, variables were selected on the area-level. Examples are

• Socioeconomy: income distribution***, education structure***.

• Labour market: share of industrial sectors**, unemployment**.

• Population structure: foreign nationalities*.

Using the mentioned variables for prevalence estimation obtains the following results.

Figure 2 is a heat map of Germany in which the estimated hypertension prevalence per federal state are displayed. The nationwide hypertension prevalence is at 26.8 $\mathrm{\%}$. This is consistent with the results of the [33], which calculated a survey-based 95 $\mathrm{\%}$-confidence interval of [25.9 $\mathrm{\%}$; 27.6 $\mathrm{\%}$]. By looking at the federal state estimates, one can see that the lowest prevalence is located in the south of the country, which consists of the federal states Baden–Württemberg and Bavaria. The highest prevalence can be found in the east of the country, which is the former territory of the German Democratic Republic. The estimated distribution is plausible, as in past studies similar distributions of related diseases like type 2 diabetes mellitus have been found [35].

Figure 2.

Estimated hypertension prevalence.

4. Conclusion and discussion

A penalized multi-level model for regional hypertension prevalence estimation was presented. It allows for the efficient combination of unit- and area-level data via level-specific elastic net penalties. With this feature, it contributes to multi-level modelling in the context of SAE. This is particularly attractive for official statistics and especially public health reporting, where more and more data sources are considered due to phenomena like the digitization. However, there is still need for further methodological research. While the model-based properties of penalized maximum likelihood have been recently established, for instance by [14], its design-based properties are still unclear. In particular, the consideration of survey weights and corresponding design-consistency in this context have not been sufficiently addressed yet. As long as these issues remain, researchers have to carefully check whether obtained estimates are sensitive with respect to survey weights.

Further, MSE estimation for penalized multi-level models is still subject to ongoing research. The presented jackknife procedure allows for decent MSE estimates on average. However, some results are unstable as the procedure is sensitive to outliers. As an analytical approach to MSE estimation in this setting does not seem to be in grasp, further research must be done on adequate resampling methods that take into account the effects of penalization on the estimation procedure. A more practical question for future research is how to properly determine the sets of covariates in a given context. Depending on the field of application, there may be data regarding a specific covariate on both the unit- and area-level. Due to the automatic variable selection caused by the sparsity inducing penalizations, it cannot be predicted in advance whether the methodology will consider both or only one of the levels in that case. In the light of known phenomena like ecological fallacy, this is an interesting topic for future studies.

Funding Statement

This work was supported by RIFOSS.

Disclosure statement

No potential conflict of interest was reported by the author(s).