INCORPORATING DESIGN WEIGHTS AND HISTORICAL DATA INTO MODEL-BASED SMALL-AREA ESTIMATION

Abstract

Bayesian hierarchical regression (BHR) is often used in small area estimation (SAE). BHR conditions on the samples. Therefore, when data are from a complex sample survey, neither survey sampling design nor survey weights are used. This can introduce bias and/or cause large variance. Further, if non-informative priors are used, BHR often requires the combination of multiple years of data to produce sample sizes that yield adequate precision; this can result in poor timeliness and can obscure trends. To address bias and variance, we propose a design assisted model-based approach for SAE by integrating adjusted sample weights. To address timeliness, we use historical data to define informative priors (power prior); this allows estimates to be derived from a single year of data. Using American Community Survey data for validation, we applied the proposed method to Behavioral Risk Factor Surveillance System data. We estimated the prevalence of disability for all U.S. counties. We show that our method can produce estimates that are both more timely than those arising from widely-used alternatives and are closer to ACS’ direct estimates, particularly for low-data counties. Our method can be generalized to estimate the county-level prevalence of other health related measurements.

1. Introduction

National and state-level surveys are crucial to public health surveillance in the United States. These are typically designed to provide direct estimates. However, direct estimates are, due to small sample sizes, often impractical at the county level (a unit of local government in the United States). Some U.S. states use other names for these units of government, such as ‘parish’. Since they function like counties, we will call them all counties.

Small area estimation (SAE), through modeling, provides estimates for counties that do not have large enough samples for direct estimation. In brief, by borrowing “strength” from the entire domain (i.e., whole counties across state-wide or/and nation-wide) as well as auxiliary variables from other survey studies (Erciulescu, A., 2019), county level estimates can often be derived (Ghosh, M. and Rao, J.N.K., 1994).

Frequentist model-based methods, such as those of Das, K. et al., (2004) and Pierannunzi C. et al., (2016) can provide SAE-based estimates for more counties than can direct estimation. However, such methods can also fail for counties with small sample sizes. Bayesian hierarchical regression (BHR) models can provide estimates for counties with small sample sizes (Planck, N. R. V. et al., 2017). However, BHR methods typically condition on the samples and the parameters of interest. That is, the sampling design and survey weights are not used (Pfeffermann, D., 2013). Thus, BHR can be vulnerable to model misspecification, resulting in both bias and large variance (Kish, L. and Frankel, M., 1974; Hansen, M. et al., 1983).

Additionally, BHR requires specification of a prior. Specifying an informative prior often requires more information than is readily available. Further, informative priors are inherently subjective. Non-informative priors avoid this difficulty. However, for small sample size counties, the posterior distributions are heavily influenced by the subjective priors. Avic (2017) suggested obtaining an informative prior, when sample sizes are small, via meta-analysis of multiple published data sets.

The power prior (Ibrahim, J. G. et al., 2015) is a compromise between informative and uninformative priors – it uses historical data (if available) to modify an uninformative prior to make it more ‘informative’. In brief, the power prior is derived by raising the historical likelihood to a power α₀ ∈ (0,1) and combining it with the uninformative ‘flat’ prior. This results in a proper posterior distribution. Many studies (Neelon, B. et al., 2010; De Santis, F., 2007; Congdon, P., 2008) have shown that the power prior can improve the precision and accuracy of posterior estimates.

Here, we propose an approach, which we call the power prior log-weights estimates (PLOW). The PLOW incorporates adjusted design-based sampling weights and uses a power prior. As an example, we apply the PLOW to obtain estimates of county level prevalence of impaired vision from 2015 Behavioral Risk Factor Surveillance System (BRFSS) data. We provide estimates for all 3142 counties in the United States.We validate the estimates using both simulations and data from the American Community Survey (ACS), a survey large enough to provide direct estimates for many counties.

2. Survey Data

Behavioral Risk Factor Surveillance System (BRFSS) is an annual state-level telephone surveillance system conducted by the Centers for Disease Control Prevention (CDC). It collects data on risk behaviors, preventive health practices and health-related conditions in the non-institutionalized adult household population with ages 18 years and older. Details on BRFSS have been previously published (Cadwell, B.L. et al., 2010). ACS, conducted by the US Census Bureau, is also an ongoing annual survey, using internet, mail, telephone, and person visits to collect data (Gettens, J. et al., 2015). ACS has more than eight times as many respondents as BRFSS.

In 2013, five survey questions concerning disability were used in common by both BRFSS and ACS. Table 1 displays those disabilities and the definitions used in ACS and survey questions in BRFSS. Possible responses for both surveys were yes, no, and no answer.

Table 1.

The descriptions of five disabilities listed in ACS and BRFSS

Disability	ACS Disability Definitions	BRFSS Disability Questions
Vision	Blind or having serious difficulty seeing, even when wearing glasses	Are you blind or do you have serious difficulty seeing, even when wearing glasses?
Cognitive	Because of a physical, mental, or emotional problem, having difficulty remembering, concentrating, or making decisions	Because of a physical, mental, or emotional condition, do you have serious difficulty concentrating, remembering, or making decisions?
Ambulatory	Having serious difficulty walking or climbing stairs	Do you have serious difficulty walking or climbing stairs?
Self-care	Having difficulty bathing or dressing	Do you have difficulty dressing or bathing?
Independent	Difficulty because of a physical, mental, or emotional problem, having difficulty doing errands alone such as visiting a doctor’s office or shopping	Because of a physical, mental, or emotional condition, do you have difficulty doing errands alone such as visiting a doctor’s office or shopping?

3. Methods

3.1. Notation

Auxiliary variables are variables other than the variable of interest used to construct models for SAE. We use the county-level covariates age, sex and race/ethnicity as auxiliary variables. Similar to Barker, L. E. et al. (2013), sampled persons in each county are cross-classified by age-groups (20-44, 45-64, 65 years plus), sex (male and female) and race/ethnicity (non-Hispanic white and all others; in some counties, numbers of people other than non-Hispanic whites were too small to stratify further). This resulted in 12 classes.

Let y_ijk be the binary disability outcome of survey participant k th in the county i (k=1,…, n_ij, i =1 to 3142) and class j (k =1 to 12), where n_ij is the sample size of county i and class j. We denote $y_{ij}={\sum }_{j=1}^{n_i}{y}_{ijk}$ as the total case counts and N_ij as the total population size in county i and class j, respectively. The BRFSS uses design weights and raking weights. We use w_ijk to denote the sampling raking weight attached to kth sample in county i and class j. County-level data y_ijk and w_ijk were derived from 2015 BRFSS while N_ij were from Census Bureau Center county-level population projections (Barker, L. E. et al., 2013).

3.2. Adjustment of sampling weights

The most common design-based inference for the SAE is the Horvitz-Thompson (HT) estimator (Horvitz, D.G. and Thompson, D.J., 1952). Let ${\widehat{p}}_{ij}$ be the direct estimate probability in county i and class j. The HT estimate is ${\widehat{p}}_{ij}={\sum }_{k=1}^{n_{ij}}{w}_{ijk}{y}_{ijk}∕{\sum }_{k=1}^{n_{ij}}{w}_{ijk}$. The HT estimate is impossible in counties with no survey respondents, and yield large variance in counties with small sample sizes. If the sampling weights have a distribution with a heavy right tail, some estimates can have very large variances (Beaumont, J. and Rivest, L., 2009). As can be seen in Table 2, BRFSS sampling weights have a heavy right tail (the ‘adjusted’ sampling weights are explained later).

Table 2.

The distribution of BRFSS sampling weights and the adjusted sampling weights (N = 426218). GF= max W_ijk/min W_ijk is the Gelman Factor, a measure of the dispersion of sampling weights.

Weighs	min	max	median	skewness	GF
Raw	1.18	36700	231.28	5.74	31102
Adjusted	1.24	1612	68.16	2.49	1300

Meng, X. L. et al. (2010) presented a power transformation to deal with weights with right tails. However, their approach did not work well with the most extreme weights. The trade-off of bias-variance relied on correlation between variable and weights (Chen, C. N. et al., 2006). Instead of what was done earlier, we propose a “log-weight” transformed method to adjust the sampling weights. Here, we use τ as an index of transformation in the range 0 to 1, in other words, τ∈ [0.1], is a tuning parameter; in Tukey’s ladder of transformations (Tukey, J. W., 1977), a logarithmic transformation corresponds to an asymptotically zero exponent. We let ${\widehat{p}}_{ij}^{\tau }$ denote the weighted direct estimate probability. This modifies the HT estimator as:

$${\widehat{\mathit{p}}}_{\mathit{ij}}^{\tau }=\{\begin{array}{cc}\frac{\sum {\mathit{y}}_{\mathit{ij}\mathit{k}}}{{\mathit{N}}_{\mathit{ij}}}\hfill & \mathit{if}\tau =0\hfill \\ \frac{\sum (\mathit{log}({\mathit{w}}_{\mathit{ij}\mathit{k}}{)}^{\tau }){\mathit{y}}_{\mathit{ij}\mathit{k}}}{\sum \mathit{log}({\mathit{w}}_{\mathit{ij}\mathit{k}}{)}^{\tau }}\hfill & \mathit{if}0<\tau \le 1\hfill \end{array}\phantom{\}}$$ (3.1)

In particular, τ=0 corresponds to the unweighted adjustment while τ=1 to the fully-weighted adjustment. To target the “optimal” τ value, we calculated the mean squared error (MSE) between ${\widehat{p}}_{ij}$ and ${\widehat{p}}_{ij}^{\tau }$ as follows (Cox, B. G. and McGrath, D. S., 1981):

$$\begin{gathered} \widehat{\mathit{MSE}}({\widehat{\mathit{p}}}_{\mathit{ij}},{\widehat{\mathit{p}}}_{\mathit{ij}}^{\tau })=({\widehat{\mathit{p}}}_{\mathit{ij}}-{\widehat{\mathit{p}}}_{\mathit{ij}}^{\tau }{)}^2+\mathit{var}({\widehat{\mathit{p}}}_{\mathit{ij}}^{\tau }) \\ =({\widehat{\mathit{p}}}_{\mathit{ij}}-{\widehat{\mathit{p}}}_{\mathit{ij}}^{\tau }{)}^2+\frac{1}{{\mathit{n}}_{\mathit{ij}}}(1-\frac{{\mathit{n}}_{\mathit{ij}}}{{\mathit{N}}_{\mathit{ij}}})\frac{1}{{\mathit{n}}_{\mathit{ij}}-1}\sum {\mathit{w}}_{\mathit{ij}}^2({\mathit{y}}_{\mathit{ij}\mathit{k}}-{\widehat{\mathit{p}}}_{\mathit{ij}}^{\tau }{)}^2 \end{gathered}$$

In general, the bias ${\widehat{p}}_{ij}-{\widehat{p}}_{ij}^{\tau }$ increases and (${\widehat{p}}_{ij}^{\tau }$) decreases as τ increases, and vice versa (Appendix. Figure 1). The “optimal” τ is then selected by the cutoff associated with minimal MSE value (Potter, F.J., 1988). Using the “optimal” cutoff of τ, we can calculate the “effective” number of cases, named as $y_{ij}^e$, as the product of raw sampling cases and the weighted probability (3.1):

$${\mathit{y}}_{\mathit{ij}}^{\mathit{e}}={\mathit{y}}_{\mathit{ij}}\times {\widehat{\mathit{p}}}_{\mathit{ij}}^{\tau }$$ (3.2)

3.3. Power prior using historical survey data

When historical data are available, it is possible to ‘borrow’ strength from it. Let α₀ ∈(0,1) be the power parameter (defined later) which controls how much the historical data impacts the prior. Let π₀(p_ij) be the prior for p_ij from before the historical data are observed (Ibrahim, J. G. and Chen, M.H., 2000).We denote the historical data Y₀ and the likelihood function of Y₀ is L(p∣Y₀). The power prior for p_ij is then defined as:

$$\pi ({\mathit{p}}_{\mathit{ij}}\mid {\mathit{Y}}_0)\propto \mathit{L}({\mathit{p}}_{\mathit{ij}}\mid {\mathit{Y}}_0{)}^{{\alpha }_0}{\pi }_0({\mathit{p}}_{\mathit{ij}})$$

From Bayes’ theorem, the posterior distribution of $p_{ij}\mid y_i^e$, Y₀ can be re-written as:

$$\begin{gathered} f(p_{ij}\mid y_i^e,Y_0)=\frac{L(y_i^e\mid p_{ij})\pi (p_{ij}\mid Y_0)}{\int L(y_i^e\mid p_{ij})\pi (p_{ij}\mid Y_0)d(p_{ij}\mid Y_0)} \\ \propto L(y_i^e\mid p_{ij})\pi (p_{ij}\mid Y_0)=L(y_i^e\mid p_{ij})\frac{L(Y_0\mid p_{ij}{)}^{{\alpha }_0}{\pi }_0(p_{ij})}{\int L(Y_0\mid p_{ij}{)}^{{\alpha }_0}{\pi }_0(p_{ij})d(p_{ij})} \\ \propto L(y_i^e\mid p_{ij})L(Y_0\mid p_{ij}{)}^{{\alpha }_0}{\pi }_0(p_{ij}) \end{gathered}$$ (3.3)

For notational convenience, we combine $L(y_i^e\mid p_{ij})L(Y_0\mid p_{ij}{)}^{{\alpha }_0}$ into ${\mathit{L}}^{\prime }(y_i^e,Y_0\mid p_{ij})$ and call it the “power likelihood”.

3.4. Estimating disability prevalence using the PLOW approach

We employ the BHR model for county-level disability prevalence estimates. Let p_ij be the prevalence of disability in county i and class j. The “effective” case count $y_{ij}^e$ (3.2) is assumed to follow a binomial distribution:

$${\mathit{y}}_{\mathit{ij}}^{\mathit{e}}\mid {\mathit{p}}_{\mathit{i}}\sim \mathit{Binomial}({\mathit{n}}_{\mathit{ij}},{\mathit{p}}_{\mathit{ij}}).$$

Let α_ij be the overall mean effect for county i and class j; X^T be the vector of auxiliary variables; and β_ij be the associated vector of fixed effect in county i and class j. Under this framework, $U_i\mid {\sigma }_i^2\sim (0,{\sigma }_i^2)$ and $V_{s(i)}\mid {\sigma }_{s(i)}^2\sim (0,{\sigma }_{s(i)}^2)$ are county-specific and state-specific random effects, respectively. We assume U_i and V_s(i) are independent. Borrowing “strength” (Rao, J. N. K. and Molina I., 2015) from county and state variabilities, the logit function of p_ij in county i and class j can be modeled as:

$$\mathit{log}\mathit{it}({\mathit{p}}_{\mathit{ij}})={\alpha }_{\mathit{i}}+{\mathit{X}}^{\mathit{T}}{\beta }_{\mathit{ij}}+{\mathit{U}}_{\mathit{i}}+{\mathit{V}}_{\mathit{s}(\mathit{i})}.$$

We apply the power prior, as described earlier. For example, to estimate the prevalence of disability in 2015, we use 2015 BRFSS data (most current available) and 2013 and 2014 BRFSS data (historical). We integrate them with current data through use of the power prior (3.3). Let p_i be the model-based estimated prevalence of disability in county i. Then the estimated prevalence in county i is:

$${\mathit{p}}_{\mathit{i}}=\frac{{\sum }_{\mathit{j}=1}^{12}{\mathit{p}}_{\mathit{ij}}{\mathit{N}}_{\mathit{ij}}}{{\sum }_{\mathit{j}=1}^{12}{\mathit{N}}_{\mathit{ij}}}.$$

3.5. Validation of PLOW estimates

The ACS releases single-year disability data for the 835 large counties (population>65000). For those counties, validation is done via direct comparison of HT (hereafter, direct estimates), PLOW estimates, and the unweighted, non-informative prior estimates of Cadwell et al. (2010) (hereafter, ‘Cadwell estimates’).

For counties for which single year data are not released, direct comparison is not possible. For those counties, we validate through a simulation. We create fictional small sample size ‘pseudo-counties’ by sampling from actual counties for which ACS data are available. That is, we:

1. Randomly select 200 counties from 835 counties covered by ACS 1-year data

2. Randomly select 1%~5% survey samples from each selected county to form pseudo-counties; some pseudo-counties may have no data

3. Apply PLOW to estimate the prevalence of disability of each pseudo-county

4. Repeat steps 1 to 3 for 500 times and average the estimates

5. Validate simulation results with the real ACS 1-year results for each county

4. Results

4.1. Shrinkage of sampling weights

Weightings heavily influence the estimates, particularly in small population counties where the weights can be large. The weights tend to have a highly right-skewed distribution. To limit this effect, we rescale weights by a logarithmic transformation. Table 2 describes the distribution of sampling weights before and after transformation. The transformation substantially reduced the skewness of the weights. The Gelman Factor (Burgard, et al. 2014) is the ratio of the largest to smallest sampling weights, which reflects the dispersion of weights. In general, the Gelman Factor of survey design should not exceed 1000 (Meng, X. L. et al., 2010). With the adjustment, the Gelman factor is reduced by over 95%.

4.2. Validation with ACS for large counties

Table 3 presents the MSE of the disability questions for direct estimates, Cadwell estimates, and PLOW estimates. For all of these, the MSE of the Cadwell estimate is smaller than it is for the direct estimate. Similarly, the MSE of the PLOW estimate is smaller (sometimes much smaller) than that of the Cadwell estimate.

Table 3.

Mean squared error (MSE) of HT direct estimates, unweighted model estimates and new approach (weighted model estimates) for estimation of disabilities (Vision, Cognitive, Ambulatory, Self-care and Independent living) prevalence

Method	Mean Square Error (MSE)×104 (cases/10000 persons)
	Vision Difficulty	Cognitive Difficulty	Ambulatory Difficulty	Self-care Difficulty	Independent living
HT direct estimates	7.12	34.65	43.53	3.36	6.32
unweighted model estimates (Cadwell estimates)	3.52	26.17	34.80	1.52	3.25
weighted model estimates (PLOW estimates)	1.02	2.61	4.05	0.88	2.07

Validation, expressed as scatter plots, appears in Figure 1. There, we plot ACS direct estimates (x-axis) vs BRFSS direct estimates, Cadwell estimates, and PLOW estimates (y-axes). Were all estimates unbiased, one would expect to see random scatter around the diagonals. A systematic departure indicates bias. Figure 1 suggests that direct and Cadwell estimates both have a positive bias, compared with ACS estimates. In contrast, the PLOW estimates are generally similar to ACS estimates. This finding indicates that inappropriately using sampling weights (direct estimate) or not using sampling weights (Cadwell estimates) can introduce bias.

Figure1.

Scatter plots showing validation of the disability (Vision, Cognitive, Ambulatory, Self-care and Independent living) prevalence estimates through HT direct estimate, Cadwell estimate and PLOW estimate using ACS 1-year results for “large” counties

4.3. Validation with ACS 5-year county-level and ACS state-level data

Next, we plot PLOW results against the ACS 5-year county-level data. ACS 5-year county-level data were collected in the range of five years, e.g., survey data from 2011 to 2015 were aggregated as the 2015 reports for all 3142 counties. Based on the Census Bureau guideline, we classify 3142 counties into three groups: large size with population>65000, (“large”, hereafter), medium size with population>20000 and <65000 (“medium”, hereafter) and small size with population<20000 (“small”, hereafter). Using Cognitive Difficulty for the example, Figure 2 shows the agreement of results between PLOW and ACS 5-year decreases from “large” to “small” counties.

Figure 2.

Scatter plots of prevalence estimates of Cognitive Difficulty using PLOW against ACS 5-year county-level results by county size (“large”, “medium” and “small”)

Additionally, we compared the results of the three methods at the state-level. We aggregated the county-level estimations into state-level and compared them with ACS state-level results. Figure 2 (suppl.) presents the state-level Cognitive Difficulty prevalence with PLOW and the other two methods. PLOW estimates are closer to ACS estimates than are direct and Cadwell estimates.

4.4. Validation for small counties

Figure 3 displays estimates in pseudo-counties of prevalence of cognitive difficulty using PLOW (red triangle), BRFSS direct (black dot) and Cadwell estimates (open square); results for other disabilities are similar and appear in Figure 3 (suppl.) for reference. The PLOW estimates have less dispersion and lower bias.

Figure 3.

Validation plot for comparison of results of ACS 1-year data with results of HT direct estimator (black circle), Cadwell estimates (black square) and PLOW (red triangle) for the small sizes counties using “pseudo-county” data. The diagonal reference line (dash line) represents the estimates results are same to ACS 1-year county-level results

4.5. Power prior vs flat prior

To compare the power prior with flat prior implemented in the Bayesian hierarchical model, we applied PLOW to estimate prevalence of Vision Difficulty with these two types of prior, respectively, using BRFSS 2015 data. The difference is that we use BRFSS 2013 and BRFSS 2014 data as “historical” data in power prior modeling, whereas we chose the inverse Gamma distribution as the non-informative prior in the flat prior modeling. Figure 4 describes the distribution of prevalence of 3142 counties modeling with power and flat prior, respectively. The flat prior tends to generate larger estimates, with more outliers, than the power prior in each quantile, accordingly: 0.023 vs. 0.016 (1st quartile), 0.027 vs. 0.019 (median) and 0.033 vs. 0.023 (3rd quartile).

Figure 4.

Box-plot comparison of Bayesian hierarchical model implemented with power prior and flat prior to estimate the Vision Difficulty prevalence

5. Discussion

Previous work has been done on SAE for county-level prevalence estimates using BRFSS survey data. Cadwell, et al. (2010) used area-level Bayesian hierarchical regression model to estimate the diabetes prevalence rates in 3141 counties. Zhang, et al. (2011) estimated county-level obesity prevalence in Mississippi using multilevel logistic model and synthetic estimation techniques. The model borrowed the “strength” from the county-specific demographic variables, geographic and socioeconomic variables and auxiliary random effect from counties. Other examples are found in Xie, et al. (2007), Goodman (2010), Olives (2013) and Schneider (2009).

We have introduced PLOW, a design assisted model-based approach, and used it to estimate county-level disabilities prevalence. This approach has several advantages over model based estimates that do not use weights and design based estimates that are directly derived from weights. We demonstrate that PLOW can produce estimates with less bias and variance than other approaches.

First, PLOW calculates “effective” disability case counts using the adjusted sampling weights before applying the Bayesian hierarchical model. Smaller skewness for the weights reduces bias and variance in the downstream model-based estimates (Beaumont, J. and Rivest, L., 2009). Secondly, PLOW uses a power prior, which shows promise in counties with little or no data. Estimates based on non-informative priors tend to be close to each other and, for examples consider here, were much higher than ACS reports. Inverse gamma is widely used as a flat prior for the unknown variance of normal distribution. Gelmen, A. (2006) demonstrated the inverse gamma distribution with small shape and scale produced large reductions of parameter space, especially for a domain with few observations. Informative priors require prior elicitation and are inherently subjective. Power priors, while remaining objective, avoid prior elicitation by borrowing strength from historical data.

Perhaps most importantly, PLOW makes possible annual county-level prevalence estimates for all counties, using single-year data. Other methods are typically limited to counties with larger samples sizes or combine multiple years of data. Combining multiple years of data obscures secular trends and makes timely detection of rapid changes difficult or impossible. For example, Cadwell estimates, used by CDC for estimating county level prevalence of diabetes, obesity, and regular physical activity (Gregg, E.W., et al., 2009), combines three years of data. This introduces bias if there are secular trends in prevalence. ACS states “single-year and multiple-year estimates are not expected to be the same” because in counties with smaller population, small changes in numbers of people who are members of a given demographic class can result in large percentage changes.

In short, PLOW, when applied to survey data for which historical data are available, can provide prevalence estimates that are both more useful and more timely. While we have provided estimates for disability prevalence, the PLOW method does not depend on any characteristic that is unique to disability. It can be used to provide county-level estimates of any measure with historical data.

APPENDIX

Optimal “Shrinkage” Factors

The weights “shrinkage” index τ can be optimized by the MSE$({\widehat{\mathrm{p}}}_{\text{ij}},{\widehat{\mathrm{p}}}_{\text{ij}}^{\tau })$ which consists of two parts: $({\widehat{p}}_{ij}-{\widehat{p}}_{ij}^{\tau }{)}^2$ and var$\text{var}({\widehat{p}}_{ij}^t)$. Suppl. Figure 2 shows the relationship between τ and MSE${\widehat{p}}_{ij,{\widehat{p}}_{ij}^{\tau }}$. Two dash lines represent $({\widehat{p}}_{ij}-{\widehat{p}}_{ij}^{\tau }{)}^2$ and var $\text{var}({\widehat{p}}_{ij}^{\tau })$ changing across τ, respectively. The solid represents mean square error which is the sum of those two parts. The joint of these two dish lines points to the minimal of MSE which corresponds to the optimal τ. Since the MSE is relied on the sample sizes and case counts, the optimal τ varies: Self-care Difficulty has the smallest τ=0.81 while Ambulatory Difficulty has the largest τ=0.89.

Figure 1 (supple.).

MSE$({\widehat{\mathrm{p}}}_{\text{ij}},{\widehat{\mathrm{p}}}_{\text{ij}}^{\tau })$ changes across the weights transformed index τ.