Bayesian Probabilistic Projections of Life Expectancy for All Countries

Abstract

We propose a Bayesian hierarchical model for producing probabilistic forecasts of male period life expectancy at birth for all the countries of the world from the present to 2100. Such forecasts would be an input to the production of probabilistic population projections for all countries, which is currently being considered by the United Nations. To evaluate the method, we did an out-of-sample cross-validation experiment, fitting the model to the data from 1950–1995, and using the estimated model to forecast for the subsequent ten years. The ten-year predictions had a mean absolute error of about 1 year, about 40% less than the current UN methodology. The probabilistic forecasts were calibrated, in the sense that (for example) the 80% prediction intervals contained the truth about 80% of the time. We illustrate our method with results from Madagascar (a typical country with steadily improving life expectancy), Latvia (a country that has had a mortality crisis), and Japan (a leading country). We also show aggregated results for South Asia, a region with eight countries. Free publicly available R software packages called bayesLife and bayesDem are available to implement the method.

INTRODUCTION

Every two years, the United Nations Population Division (UN) publishes an updated edition of the World Population Prospects (WPP), which has included projections of the populations of all the countries in the world to 2050, broken down by age and sex. The UN issues several variants of its population projections, of which the most important is the Medium variant, viewed as the best projection of future population trends. It also issues High and Low variants, obtained by increasing and decreasing the total fertility rate (TFR) by half a child, respectively. The High and Low variants are scenarios designed to indicate what would happen if the assumptions underlying the Medium variant were violated in various ways; they are not probabilistic projections. The UN does not issue variants indicating the likely impact of differences in future mortality rates. Largely because of the need to project climate change over the next century, the UN has just extended its population projections to 2100.

Fully probabilistic population projections are an alternative to scenarios, which may be preferable because, unlike scenarios, they indicate the likely range of future population outcomes (Bongaarts & Bulatao, 2000). To produce probabilistic population projections for all the world's countries, one needs probabilistic projections of the main demographic processes affecting national populations: fertility, mortality and international migration.

This article is part of a research program aiming to do this. The UN produces many of its current (deterministic) projections by projecting broad summaries of population processes, and then breaking them down into age-specific rates using model schedules and relational models, to yield the age- and sex- specific fertility, mortality and migration rates that are required by the standard cohort-component population projection method. Our research therefore focuses on probabilistic projection of summary population measures. Fully probabilistic projections of the total fertility rate for all countries have already been produced (Alkema et al., 2008; Raftery et al., 2009; Alkema et al., 2011).

The best known approach to probabilistic projection of mortality is that of Lee & Carter (1992), which uses past age-specific death rates for each year over time in a country. They use a log-linear model for the age-specific mortality rate at age x in year t, log(m_xt) = a_x + k_tb_x + ε_xt, where k_t is the mortality index. They project k_t into the future using a random walk model with constant drift. The Lee-Carter method has been found to perform well for some developed countries (Booth et al., 2005; Bell, 1997). When forecasting a group of countries simultaneously, a common age parameter is fixed to ensure consistent forecasts of multiple countries (Li & Lee, 2005).

However, the Lee-Carter method requires that age-specific death rates be available for at least three time periods, which is not the case for many developing countries (Li et al., 2004). Also, while the Lee-Carter model specifies changes in age-specific mortality rates to have a constant distribution on the logarithmic scale, White (2002) found that linear changes in life expectancy gave a better fit to data from 21 industrialized countries. Further, Lee & Miller (2001) found that the forecasts of the Lee-Carter method for the United States, Japan, Canada, Sweden, and France systematically underestimated future life expectancy.

Girosi & King (2008) proposed a Bayesian method based on smoothing age-specific mortality rates over age and time. They argued that this would outperform other existing methods when informative covariates are observed. In their Section 12.1, they applied their method to all-cause mortality for males for 48 countries, using tobacco use and GDP as covariates, and showed that it outperformed the Lee-Carter method. Most of these 48 countries are among the 39% of countries that have good vital registration and other data. Our goal, however, is to provide forecasts for all the countries of the world, for many of which covariate and age-specific mortality data of the kind and quality used by Girosi & King (2008, chap. 12) are not available.

The approaches of both Lee & Carter (1992) and Girosi & King (2008) are based on the assumption that the rate of change of age-specific mortality remains constant over time. This does not hold for more than a few decades into the future (Li & Gerland, 2011). However, the UN’s projections are for 90 years into the future, and these methods can be problematic for such a long projection horizon. The choice of the reference period can have a substantial influence on the results, and reversals of historical mortality trends as in the former Soviet Union can lead to implausible projections.

Lutz and colleagues at the International Institute for Applied Systems Analysis (IIASA) have taken a different approach, producing probabilistic projections from expert opinion. They addressed data limitations by aggregating countries into regions and forecasting regional life expectancy based on expert-based probabilistic projections (Lutz et al., 2004).

We propose an approach that is adapted to the UN’s task of projecting populations for all the countries in the world, many of which have data that are patchy and of variable quality. We project period life expectancy directly, using a random walk model with a nonconstant drift. The drift term is a nonlinear function of current life expectancy and reflects the fact that life expectancy tends to improve more slowly for the countries with the lowest and highest life expectancies, and more quickly for the countries in the middle. Also, the overall rate of improvement varies by country. We use a Bayesian hierarchical model (BHM), which allows us to estimate the rate of improvement in life expectancy for a country using past data from that country, and also taking account of the observed past patterns in all other countries.

We develop a one-sex model for males and discuss potential extensions to a two-sex model in the closing discussion section.

The article is organized as follows. We first discuss the UN’s current projection methodology. We then describe our proposed model, which is a natural extension of the UN’s current practices from deterministic to probabilistic projections. We assess our model by holding back the last ten years of data (1995–2005), reestimating our model using only the data from 1950 to 1995, generating probabilistic projections for 1995 to 2005, and comparing them with what was actually observed.

We then present probabilistic projections of life expectancy to 2100 in three countries with widely different current life expectancies, Madagascar, Latvia, and Japan, each of which presents different forecasting challenges. Life expectancy in Madagascar has been improving steadily but is still in the lowest quartile. Latvia has experienced a mortality crisis in the past generation, with both declining and increasing life expectancy. Japan is a leading country, with one of the highest life expectancies. For each country, we present out-of-sample projections for 1995–2005. Lastly, we discuss aggregation of country-specific projections. Comparisons are made with the regional projections for South Asia by IIASA.

METHODOLOGY

Data

We use the estimates of male period life expectancy at birth from the UN World Population Prospects (WPP) 2008 Revision from 1950 through 2005 (United Nations, 2009). Period life expectancy refers to the life expectancy of a hypothetical cohort subjected to current mortality rates throughout its life (Preston et al., 2001, Section 3.1). Because of the significant impact of the HIV/AIDS epidemic on mortality rates, we do not include countries with a generalized HIV/AIDS epidemic in this analysis. We base our results on data from 158 countries, comprising about 90% of the world’s population.

The UN produces estimates of age-specific mortality and period life expectancy at birth for 230 countries and areas, updated every two years. The data available vary widely between countries, with only 89 countries (39%) having good vital registration data allowing direct and accurate estimation of age-specific mortality rates. Other countries have incomplete vital registration data (32 countries), summary estimates of child and adult mortality (38 countries), or estimates of child mortality only (49 countries), based on surveys, censuses or administrative records. Twenty-two small areas have no recent data at all.

There are many problems with the available data, particularly in the majority of countries without good vital registration data. These include the absence of age and sex breakdowns of census data, highly questionable census counts, incomplete geographical coverage and major divergences between mortality estimates from different sources. The UN adjusts the available data in light of knowledge about the biases and quality of the different data sources. About half the countries lack age-specific mortality data, and in these cases the UN uses model life tables and relational models to estimate life expectancy from the available summary information about child and adult mortality.

There is considerable regional variation in the availability of reliable data. Of the 50 countries in Asia, 56% have “reliable” or “fairly reliable” vital statistics, whereas 95% of the countries in Europe and North America have reliable statistics. This number decreases dramatically in Africa where only five of the 54 countries (9%) maintain “reliable” or “fairly reliable” vital statistics (United Nations, 2006).

Current UN Population Projection Methodology

Currently, the UN projects life expectancy at birth deterministically. The life expectancy at birth, ℓ_c,t+1 for country, c, in the next five-year period, t + 1, is projected to be the life expectancy in the current time period, ℓ_c,t, plus the expected gain in life expectancy, g(ℓ_c,t). Observed five-year gains in life expectancy for 158 countries from 1950 to 2005 are plotted in Figure 1. This figure highlights the non-constant rate of change in life expectancy. To capture this, the UN has developed models that represent the gains in life expectancy by a double-logistic function of current life expectancy. The five deterministic UN models are shown in Figure 1(b), where models vary by pace of gains in life expectancy.

Figure 1.

Observed five-year gains in life expectancy, plotted against the life expectancy at the beginning of the five-year period. UN estimates for 158 countries from 1950 to 2005 are included in this figure (n = 1, 738). Each point represents an observed five-year gain in life expectancy within a country. The black line is a locally-weighted polynomial (lowess) regression of the observations, which highlights the non-constant rate of gains in life expectancy. Included in the left plot are the fitted posterior median double-logistic functions for Japan and Madagascar from our model. The UN deterministic models are included in the right plot. (Note: 31 observations (1.8%) are outside the range of the plot and not shown, but were included in the local regression.)

The double-logistic function (Meyer, 1994) has six parameters, as illustrated in Figure 2. Four of them identify intervals of life expectancy when the rate of life expectancy gains is changing, one describes the approximate maximum gain in life expectancy, and the last parameter gives the asymptotic rate of gains as life expectancy increases. For each country, a UN analyst chooses one of five prescribed choices of the six parameters¹ by assessing the recently observed pace of mortality decline (United Nations, 2009). The model implies that past a certain point, life expectancy increases at an effectively constant rate. This is consistent with research indicating that there is no evidence of an upper limit to life expectancy (Oeppen & Vaupel, 2002; Cohen & Oppenheim, 2011).

Figure 2.

Illustration of the double-logistic function, based on a curve from the posterior distribution for Japan. The left plot illustrates the double-logistic function of 5-year gains in life expectancy. The right plot is a time-series of life expectancy with gains modeled according to the double-logistic function.

The logistic function has been used for over a century to model population growth. Marchetti et al. (1996) have shown that a sum of logistic functions can be used to model not only the adoption and substitution of competing technological innovations (Fisher & Pry, 1971; Meyer, 1994; Meyer et al., 1999), but also the social diffusion, learning and adoption of new ideas, norms, attitudes and behaviors associated with the fertility and mortality transitions (Marchetti et al., 1996; Marchetti, 1997; Potter et al., 2010) or nuptiality (Hernes, 1972; Goldstein & Kenney, 2001; Li & Wu, 2008).

The mortality transition from high to low mortality can be decomposed into two processes, each of which can be approximated by a logistic function. The first process consists of initial slow growth and diffusion of progress against mortality (small mortality improvements at low levels of life expectancy associated with diffusion of hygiene, improved nutrition, etc.), followed by a period of accelerated improvements especially for infant and children (larger gains associated with greater social and economic development, mass immunization, etc.). The second process kicks in once the easiest gains have been achieved against infectious diseases, and produces continuing gains against non-communicable diseases. These improvements occur at a slower pace due to ever greater challenges to the prevention of premature deaths at older ages due to cardiovascular diseases or neoplasms and to delaying the onset of aging (Riley, 2001; Fogel, 2004).

To summarize, the UN projects life expectancy in the next time period deterministically using the equation

$${\ell }_{c,t+1}={\ell }_{c,t}+g({\ell }_{c,t}).$$ (1)

The expected five-year gain in life expectancy is a double-logistic function of the current level of life expectancy, namely

$$g({\ell }_{c,t}|{\mathit{\theta }}^c)=\frac{k^c}{1+\text{exp}(-\frac{A_1}{{\mathrm{\Delta }}_2^c}({\ell }_{ct}-{\mathrm{\Delta }}_1^c-A_2{\mathrm{\Delta }}_2^c))}+\frac{z^c-k^c}{1+\text{exp}(-\frac{A_1}{{\mathrm{\Delta }}_4^c}({\ell }_{ct}-{\sum }_{i=1}^3{\mathrm{\Delta }}_i^c-A_2{\mathrm{\Delta }}_4^c))}.$$ (2)

In (2), ${\mathit{\theta }}^c=({\mathrm{\Delta }}_1^c,{\mathrm{\Delta }}_2^c,{\mathrm{\Delta }}_3^c,{\mathrm{\Delta }}_4^c,k^c,z^c)$ are the six parameters of the double-logistic function for country c, whose meaning is illustrated in Figure 2. The vector θ^c of the parameters for country c are chosen by a UN analyst from the five possibilities (θ^{Very Slow}, θ^Slow, θ^Medium, θ^Fast, θ^{Very Fast}). The constants A₁ = 4.4, A₂ = 0.5 are chosen so that the parameters $\{{\mathrm{\Delta }}_i^c:i=1,2,3,4\}$ are on an interpretable scale. but they are arbitrary in that they could be changed without altering the results, provided that their product, A₁A₂, remains unchanged.

Stochastic Model

The UN projection method is deterministic and does not account for uncertainty. We now extend it to a stochastic model to allow for uncertainty. This involves two extensions. The first allows for stochastic changes within a country by replacing the deterministic model (1) by a stochastic one by adding a random perturbation to (1). It then becomes a random walk with drift, where the drift term is given by the double-logistic function.

The second extension is to allow the parameters of the double-logistic function to vary between countries over a continuous range rather than among the current five UN possibilities. The resulting hierarchical model is

$${\ell }_{c,t+1}={\ell }_{c,t}+g({\ell }_{c,t}|{\mathit{\theta }}^{(c)})+{\epsilon }_{c,t+1},$$ (3)

where

$$g({\ell }_{c,t}|{\mathit{\theta }}^c)=\text{Double}-\text{Logistic function with parameters}{\mathit{\theta }}^c,{\mathit{\theta }}^c=({\mathrm{\Delta }}_1^c,{\mathrm{\Delta }}_2^c,{\mathrm{\Delta }}_3^c,{\mathrm{\Delta }}_4^c,k^c,z^c),{\mathrm{\Delta }}_i^c|{\sigma }_{{\mathrm{\Delta }}_i}\stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{~}{\text{Normal}}_{[0,100]}({\mathrm{\Delta }}_i,{\sigma }_{{\mathrm{\Delta }}_i}^2),i=1,\dots ,4,k^c|{\sigma }_k\stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{~}{\text{Normal}}_{[0,10]}(k,{\sigma }_k^2),z^c|{\sigma }_z\stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{~}{\text{Normal}}_{[0,1.15]}(z,{\sigma }_z^2),$$

where Normal_[a,b](μ, σ²) denotes a normal distribution with mean μ and standard deviation σ, truncated to lie between a and b.

This model allows us to pool information about the rates of gains across countries by assuming that each set of country-specific double-logistic parameters is randomly sampled from a common truncated normal distribution. The normal distribution is truncated such that all the double-logistic parameters are positive.

The parameter z^c is the asymptotic average rate of increase in life expectancy per five-year period. Our prior distribution for this is informed by the results of Oeppen & Vaupel (2002), who found a strong positive linear trend in the “best practices” life expectancy (i.e. the highest life expectancy in a given year) from the mid-19th century through 2000. By assuming that z^c is non-negative, we are assuming that life expectancy will continue to increase on average. In their regression of highest male life expectancy on year, Oeppen & Vaupel (2002) estimated a slope of 1.11 years per five-year period with R² = 0.98. Because this is the rate of increase for “best practices” countries, we assume that the asymptotic rate of increase for any given country will not exceed the upper bound of a 99.9% confidence interval for this estimate, namely 1.15.

To specify the distribution of the random perturbations, ε_c,t, we first estimated the model assuming them to be normally distributed with a constant variance, using the estimation method described later. Figure 3 shows the absolute residuals from this fit with a fitted regression spline. The spread of the residuals clearly decreases with increasing life expectancy. To account for this, we modeled the ε_c,t as normally distributed with standard deviation proportional to the regression spline fitted to the absolute residuals shown in Figure 3, so that

$${\epsilon }_{ct}\stackrel{\mathrm{i}\mathrm{i}\mathrm{d}}{~}N(0,(\omega \times f{(l_{c,t-1}))}^2).$$ (4)

Figure 3.

Absolute residuals from the constant variance model plotted against across life expectancy, with fitted regression spline. (Note: 44 (2.8%) of the residuals are outside the range of the plot, but were included in the regression spline fit.)

Our stochastic model builds on that proposed by Alkema et al. (2011) for probabilistic projection of the Total Fertility Rate (TFR) for all countries. However, it differs in several respects. The double logistic model for the gains in life expectancy is more general than that for total fertility rate, as it asymptotes at a non-zero level, z^c, which is estimated from the data for each country c. Also, prior information about the range of plausible values of z^c is available from other research, and this is incorporated explicitly via the Bayesian prior distribution. In the TFR model, in contrast, the prior distributions were largely uninformative.

Parameter Estimation

We adopt a Bayesian approach to estimating our model, making it a Bayesian hierarchical model. This requires specifying prior distributions for the 13 world parameters of the model: $({\mathrm{\Delta }}_i,{\sigma }_{{\mathrm{\Delta }}_i}^2)$ for $i=1,\dots ,4;k,{\sigma }_k^2,z,{\sigma }_z^2$ and ω. We specify prior distributions that are proper but much more diffuse than the posterior distributions.

We set ${\mathrm{\Delta }}_i~N_{[0,100]}(a_i,{\delta }_i^2)$ for i = 1, …, 4, $k~N_{[0,10]}(a_5,{\delta }_5^2)$ and $z~N_{[0,1.15]}(a_6,{\delta }_6^2)$. We set (a₁, …, a₆) to the values specifying the UN medium-pace model, namely (15.77, 40.97, 0.21, 19.82, 2.93, 0.40). We set $({\delta }_1^2,\dots ,{\delta }_6^2)$ to the variances of the parameters among the different UN models.

For the world variance parameters, ${\sigma }_{{\mathrm{\Delta }}_i}^2(i=1,\dots ,4),{\sigma }_k^2$ and ${\sigma }_z^2$, we used inverse-gamma prior distributions with 4 degrees of freedom (i.e. a shape parameter equal to 2). To set the parameters of these priors, we first fit the double-logistic model by least squares to the data from each country individually, and then for each parameter we computed the empirical average squared deviations from the values for the UN medium-pace model. We then set the prior means of the reciprocals of the world variance parameters equal to the reciprocals of these values. This yields rate parameters (15.6², 23.5², 14.5², 14.7², 3.5², 0.6²) for the six inverse gamma prior distributions. The resulting prior distributions are guaranteed to be much more spread out than the posterior distribution. Finally, a diffuse Uniform[0,10] prior was used for ω.

Experiments showed that the results were insensitive to changes in these priors, which is to be expected because the resulting prior distribution is much more spread out than the posterior distribution.

The posterior distribution of the world and country-level parameters was approximated by Markov chain Monte Carlo implemented in R. The approximately 1,000 parameters were updated one at a time, using either Gibbs sampling (Gelfand & Smith, 1990), Metropolis-Hastings (Hastings, 1970; Chib & Greenberg, 1995), or slice sampling (Neal, 2003). We used three chains of length 100,000 scans each, with a burn-in of 10,000 scans. Visual inspection of the trace plots, the Raftery-Lewis diagnostic (Raftery & Lewis, 1992) and the Gelman-Rubin statistic (Gelman & Rubin, 1992) all indicated that the chains had converged and had explored the posterior distribution enough to yield good estimates of posterior quantiles of interest.

A free publicly available R software package called bayesLife is available to implement the method (Ševčíková & Raftery, 2011). An additional R package called bayesDem, also freely and publicly available, provides a graphical user interface for bayesLife (Ševčíková, 2011).

MODEL VALIDATION

To assess our probabilistic projections, we fit our model to data from 1950 to 1995, giving 1,422 country-period combinations. We then used the resulting model to forecast life expectancy for males for the two five-year periods 1995–2000 and 2000–2005, and compared the forecasts with what was actually observed. We had 316 out-of-sample predictions.

Comparing probabilistic forecasts with outcomes is challenging because it involves comparing two different kinds of objects : a probability distribution and a single value (Gneiting et al., 2007). A probabilistic forecast should be calibrated, i.e. x% prediction intervals should contain the truth x% of the time on average. It should also be sharp, i.e. the intervals should be as narrow as possible.

To assess the predictive ability of our model, we examined the mean absolute error (MAE) of our point forecasts. To assess the calibration of our probabilistic forecasts, we used two measures: the standardized absolute predictive error (SAPE) of our predictions, and the calibration of our predictive intervals. The SAPE is the absolute difference between the actual observed life expectancy (l_ct) and the median forecast (l̂_ct), standardized by the standard deviation of the predictive distribution. It is then scaled so that when the model is correctly specified, the expected SAPE value is equal to one. It is defined as $\sqrt{\frac{\pi }{2}}*\frac{|l_{ct}-{\mathrm{l̂}}_{ct}|}{{\mathrm{\sigma ̂}}_{\mathit{\text{pred}},ct}}$. If the model is well calibrated, we expect the mean SAPE to be close to one.

These metrics are given in Table 1. The MAE of our median predictions was 1.07 years. Thus our “best guess” was within just over one year of the actual observation, on average. The mean SAPE was 1.04, which is close to the theoretical mean of 1, indicating that our predictive standard deviations were accurate. Overall, our model was well calibrated with our 95% prediction intervals capturing the actual observations 92% of the time and the 80% intervals capturing the observations 82% of the time.

Table 1.

Summary measures for 10 year out-of-sample predictions for the Bayesian hierarchical model and the current UN methodology.

Measure	BHM	UN replication
Mean absolute error (MAE)	1.07	1.86
Std absolute prediction error (SAPE)	1.04	—

Prediction Intervals
Nominal	Actual	Mean half-width
95%	92.1%	2.54
90%	89.2%	2.13
80%	82.0%	1.66

We compared the predictive ability of our model with the current UN methodology. Replicating our cross-validation methodology by using WPP 2008 data through 1995, a UN analyst computed life expectancy forecasts for 1995–2005 using one of the five prescribed UN models of gains in life expectancy at birth based on levels and trends in the preceding two decades. The MAE of our method was substantially smaller than that of the current UN method (1), improving on it by over 40%.

We assessed the sharpness of our projections using the distribution of 80% prediction interval half-widths. (If a prediction interval is symmetric, it is equal to the point prediction plus or minus the half-width.) For the 1995–2000 time period, the prediction interval half-widths for different countries ranged from 0.7 to 1.9 years with an average half-width of 1.3 years. For the next five-year period, 2000–2005, the interval half-widths increased to a range of 1.0 to 3.2 years with an average half-width of 1.9 years. Average life expectancy at birth and prediction interval widths both varied by region. Among the continental regions from 1995 to 2005, Africa had the lowest life expectancy of 59.8 years with an average interval half-width of 2.2 years, even after excluding the 38 countries with generalized HIV/AIDS epidemics. With an average life expectancy of 73.5 years and an interval half-width of 1.3 years, North America had both the highest life expectancy and the narrowest prediction intervals.

Our model implies the possibility of crossovers between countries in the future, and one concern is whether this is plausible. Crossovers have happened in the past, so a probabilistic projection method should allow for the possibility in the future. The question is whether the model overestimates the probability of crossovers.

To investigate this, we computed the proportion of pairs of countries for which there was a cross-over during our 55-year data period, 1950–2005, and compared it with our projected probability of crossover during the next 55-year period, 2005–2060. During 1950–2005 there were 4,202 crossovers among the $\left(\begin{array}{c}\hfill 158\hfill \\ \hfill 2\hfill \end{array}\right)=12,403$ pairs of countries, a proportion of 0.34. For the projected 2005–2060 period, the posterior predictive probability of a crossover is 0.24, actually lower than the proportion in the historical period. The proportion of crossovers between the posterior median projections in 2005–2060 is 0.11, considerably less than the historic proportion. Thus, it seems unlikely that the projection method is substantially overestimating the probability of crossovers in the future.

CASE STUDIES

Typical country: Madagascar

In the WPP 2008, the UN estimated the current life expectancy at birth among males in Madagascar to be 58.5 years. Figure 4(a) shows projections of life expectancy starting from 2005–2010. The median forecast from our BHM is similar to the UN’s WPP 2008 forecasts through 2050. In 2045–2050, the WPP 2008 projects male life expectancy will be 69.7 years. We project life expectancy will be 71.4 years with an 80% prediction interval of (65.5, 77.8). Fifty years after that, in 2095–2100, we project life expectancy to reach 80.4 years with a wider 80% prediction interval of (72.6, 88.5). Figure 4(b) shows out-of-sample projections for Madagascar with projections beginning in 1990–1995. UN observed estimates are indicated in the plot by brown squares and are close to our median projection. The exclusion of two time periods results in more uncertainty in our projections for 2095–2100 with a median of 76.6 years and an 80% prediction interval of (64.7, 87.8).

Figure 4.

Life Expectancy Projections for Males in Madagascar. The above plots include the UN projections and our median projections, with 80% and 95% prediction intervals. The life expectancy values used to estimate our model are indicated by grey circles. (a) Projections from 2005–2010. A typical stochastic trajectory is shown in black, illustrating that the future trajectory is likely to be less smooth than the median projection. (b) Cross-validation projections from 1990–1995. Observed life expectancies from 1995–2005 are shown as squares.

Along with quantiles of the projected life expectancy distribution, in Figure 4(a) (as well as Figures 5(a) and 6(a) for Latvia and Japan), we also include a sample stochastic trajectory for Madagascar. We see that unlike the quantiles, the sample trajectory does not follow a smooth path. For this trajectory the mean absolute deviation from the median is equal to the median mean absolute deviation among the posterior sample of projected trajectories. It can be viewed as a trajectory with typical deviation from the projected median.²

Figure 5.

Life Expectancy Projections for Latvia. The above plots include the UN projections and BHM median projections, with 80% and 95% prediction intervals. The past values of life expectancy values used to estimate our model are shown by grey circles. (a) Projections to 2100 starting from 2005–2010. A typical stochastic trajectory is shown in black, illustrating the non-smoothness of individual projections. (b) Out-of-sample projections starting from 1990–1995. Observed life expectancies from 1995–2005 are shown as squares. By 1995, Latvia had not yet recovered from its mortality crisis; the BHM projection intervals reflect uncertainty about a full recovery.

Figure 6.

Life Expectancy Projections for Japan. The above plots include the UN projections and BHM median projections, with 80% and 95% prediction intervals. The life expectancy values used to estimate our model are show by grey circles. (a) Projections from 2005–2010 with a sample trajectory. National Institute of Population and Social Security Research (IPSS) medium variant projections are the same as the UN projections with uncertainty bounds indicated in the shaded region. We include a trajectory with a constant increase of 1.11 years per five-year period, as estimated by Oeppen and Vauppel (O & V) (2002) for the “best practices” country. A typical stochastic trajectory is shown in black. (b) Cross-validation projections from 1990–1995. Observed life expectancies from 1995–2005 are shown as squares.

Mortality Crisis: Latvia

Figure 5 shows estimated and projected life expectancies for Latvia. Male life expectancy in Latvia increased from 62.5 years in 1950 to 66.3 years in 1965. However, in the subsequent 15 years, male life expectancy in Latvia decreased by 2.2 years, to 64.1 years. Life expectancy increased again until a 3.8 year decline was recorded between 1985–1990 and 1990–1995. Since then, life expectancy in Latvia has been increasing again. Both the UN’s and our median projections predict a continuous increase in life expectancy, but ours increase more slowly.

As can be seen from Figure 5(b), our 80% prediction intervals capture the observed estimates of life expectancy from 1995 to 2005. For the first time period, from 1995–2000, the upper bound of our 80% prediction interval was 64.4 years. Yet the lower bound of our 80% prediction interval, 61.1 years, indicates that life expectancy may continue to decrease. In fact, our prediction intervals allow for the possibility of life expectancy not increasing for the following 50 years, reflecting the erratic progress over the previous 40 years.

Leading country: Japan

One of the difficulties with projecting mortality is accurately projecting the country with the highest life expectancy. Historically, “pessimists” believed that life expectancy could not keep rising at historic rates and assumed that there must be a “ceiling” to life expectancy for humans (Fries, 1980, Olshansky et al., 1990, Olshansky et al., 2001, Olshansky et al., 2002, Olshansky et al., 2005). “Optimists”, on the other hand, saw no evidence of a limit to increases in life expectancy (Oeppen & Vaupel, 2002, Tuljapurkar et al., 2000, Tuljapurkar, 2005, Cohen & Oppenheim, 2011). Past estimates of the “maximum life expectancy” have continually been surpassed (Oeppen & Vaupel, 2002), and old-age mortality rates continue to decline (Vaupel et al., 1998). Oeppen & Vaupel (2002) presented evidence that the world’s highest, or “best practices”, life expectancy at birth has increased linearly across time and shows no signs of leveling off. They estimated that the “best practices” life expectancy for males has increased at a rate of 1.11 per five-year period. See Bongaarts (2006) for a review of the historical debate on life expectancy limits.

Although Japan does not currently have the highest male life expectancy (that title has belonged to Iceland since 2000), it is the country with the highest overall life expectancy and has been since 1980. Figure 6(a) is a plot of male life expectancy in Japan. Also included in the plot is what the trajectory would be if male life expectancy in Japan increased at the “best practices” rate of 1.11 per five-year period. Vallin & Meslé (2009) updated and expanded the data time period (from 1840–2000 to 1750–2005) for “best practices” life expectancy. They found a segmented line fit the extended time frame better, with the most recent segment (1960–2005) still with a strong positive slope (1.13 years per five-year period for women), and concluded that the Oeppen-Vaupel line may be too optimistic for the long-term future. Our median projection increases more slowly than the Oeppen-Vaupel “best practices” linear projection, reflecting the fact that the “best-practices” lines is for the best country at each time, and thus likely to increase faster than for an individual country. However, the “best practices” trajectory is just within the upper bound of our 80% prediction interval.

Bongaarts (2006) also found the Oeppen-Vaupel “best-practices” rate to be overly optimistic. By decomposing mortality into juvenile, background, and senescent mortality, he observed that historically large gains in life expectancy were due to declines in juvenile mortality. Then as juvenile mortality reached low levels, the rate of gains in life expectancy diminished. The rationale for this decomposition is similar to that for the double-logistic function, where there are periods of high gains in life expectancy (i.e., when juvenile mortality is declining) followed by a leveling off of gains (i.e., when the gains in life expectancy are due to incremental declines in senescent mortality). Bongaarts (2006) found that senescent life expectancy in countries with low mortality, on average, increased at a rate of 0.75 years per five-year period. The average asymptotic rate of gains estimated in our model was 0.84 years per five-year period, which is closer to the Bongaarts (2006) projected gains in senescent life expectancy than the “best practices” rate of increase of about 1.11.

Recently, the Japanese official projections made by the National Institute of Population and Social Security Research (IPSSR) extended the Lee-Carter method to provide more refined estimates of mortality at higher ages. The original Lee-Carter method estimated age-specific mortality rates for five-year age groups, with the last age group aggregating those 85 and older. More recently, the IPSSR has used the shifting logistic model (Bongaarts, 2005) to account for continued increases in life expectancy in Japan. IPSSR projections (low/medium/high rates of mortality decline variants, with the medium variant being equivalent to the UN projections) (Kaneko et al., 2008) are included in Figure 6(a). The IPSSR projections are more conservative and project an earlier leveling off of life expectancy than our projections, but are still within our prediction intervals.

When looking at out-of-sample projections in Figure 6(b), which begin in 1990–1995, we see that the UN out-of-sample projections suggested an immediate leveling off of life expectancy, unlike our projections. In fact, the observed life expectancy in 1995–2005 did not level off and instead continued to increase.

Other countries

Updated projections of both male and female life expectancy for all the 158 countries analyzed in this article are shown in the Online Resource.

AGGREGATION TO REGIONAL PROJECTIONS: SOUTH ASIA

Researchers at the International Institute for Applied Systems Analysis (IIASA) (Lutz et al., 2004) produced regional probabilistic prediction intervals for life expectancy using Delphi-type methods. A group of experts were asked to give 90% prediction intervals for future life expectancy in each of 13 specified regions. Linear paths were then drawn from a normal distribution to produce probabilistic predictive distributions. This method uses demographic knowledge as its main input whereas time-series methods (including ours) rely on past trends.

Country-specific projections allow regional projections to be made regardless of how the region is defined. To compare our projections with those of IIASA for South Asia, we aggregated the UN estimates and projections, as well as our projections, weighting country values proportionately to the male populations from 2005 to 2010.³ The countries included in the IIASA-defined region of South Asia (with population percentage) are: India (75.1%), Pakistan (10.5%), Bangladesh (9.9%), Nepal (1.7%), Afghanistan (1.5%), Sri Lanka (1.3%), Bhutan (0.04%), Maldives (0.02%).

Our model takes no account of correlations between the random perturbations in life expectancy gains in different countries. Previous work (Alho, 2008) has suggested that cross-country correlations are non-zero and should be modeled as such. Within South Asia, in the past 60 years, life expectancies were indeed highly correlated, as follows:

$$\left(\begin{array}{cccccccc}\hfill \text{Afghanistan}\hfill & \hfill 0.91\hfill & \hfill 0.91\hfill & \hfill 0.99\hfill & \hfill 0.98\hfill & \hfill 0.94\hfill & \hfill 0.98\hfill & \hfill 0.99\hfill \\ \hfill 0.91\hfill & \hfill \text{Bangladesh}\hfill & \hfill 0.99\hfill & \hfill 0.92\hfill & \hfill 0.96\hfill & \hfill 0.99\hfill & \hfill 0.97\hfill & \hfill 0.89\hfill \\ \hfill 0.91\hfill & \hfill 0.99\hfill & \hfill \text{Bhutan}\hfill & \hfill 0.92\hfill & \hfill 0.96\hfill & \hfill 0.99\hfill & \hfill 0.96\hfill & \hfill 0.89\hfill \\ \hfill 0.99\hfill & \hfill 0.92\hfill & \hfill 0.92\hfill & \hfill \text{India}\hfill & \hfill 0.99\hfill & \hfill 0.95\hfill & \hfill 0.99\hfill & \hfill 0.99\hfill \\ \hfill 0.98\hfill & \hfill 0.96\hfill & \hfill 0.96\hfill & \hfill 0.99\hfill & \hfill \text{Maldives}\hfill & \hfill 0.98\hfill & \hfill 0.99\hfill & \hfill 0.98\hfill \\ \hfill 0.94\hfill & \hfill 0.99\hfill & \hfill 0.99\hfill & \hfill 0.95\hfill & \hfill 0.98\hfill & \hfill \text{Nepal}\hfill & \hfill 0.98\hfill & \hfill 0.92\hfill \\ \hfill 0.98\hfill & \hfill 0.97\hfill & \hfill 0.96\hfill & \hfill 0.99\hfill & \hfill 0.99\hfill & \hfill 0.98\hfill & \hfill \text{Pakistan}\hfill & \hfill 0.97\hfill \\ \hfill 0.99\hfill & \hfill 0.89\hfill & \hfill 0.89\hfill & \hfill 0.99\hfill & \hfill 0.98\hfill & \hfill 0.92\hfill & \hfill 0.97\hfill & \hfill \text{Sri Lanka}\hfill \end{array}\right).$$

However, for probabilistic projections, what matters are the cross-country correlations between the random perturbations (effectively the forecast errors) rather than between the life expectancies themselves. The hierarchical modeling of the change in life expectancy allows for between-country correlation in life expectancy. We are interested in the residual correlations in life expectancies gains, ρ_ci,cj, between countries c_i and c_j, namely:

$$({\mathrm{\rho ̂}}_{c_i,c_j})=\left(\begin{array}{cccccccc}\hfill \text{Afghanistan}\hfill & \hfill -0.28\hfill & \hfill -0.23\hfill & \hfill 0.09\hfill & \hfill 0\hfill & \hfill -0.07\hfill & \hfill -0.08\hfill & \hfill 0.19\hfill \\ \hfill -0.28\hfill & \hfill \text{Bangladesh}\hfill & \hfill 0.05\hfill & \hfill -0.03\hfill & \hfill 0.03\hfill & \hfill 0\hfill & \hfill 0.05\hfill & \hfill -0.19\hfill \\ \hfill -0.23\hfill & \hfill 0.05\hfill & \hfill \text{Bhutan}\hfill & \hfill -0.03\hfill & \hfill 0.05\hfill & \hfill 0.09\hfill & \hfill 0.07\hfill & \hfill 0.2\hfill \\ \hfill 0.09\hfill & \hfill -0.03\hfill & \hfill -0.03\hfill & \hfill \text{India}\hfill & \hfill 0.16\hfill & \hfill 0.03\hfill & \hfill 0.03\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0.03\hfill & \hfill 0.05\hfill & \hfill 0.16\hfill & \hfill \text{Maldives}\hfill & \hfill 0.24\hfill & \hfill 0.25\hfill & \hfill 0.34\hfill \\ \hfill -0.07\hfill & \hfill 0\hfill & \hfill 0.09\hfill & \hfill 0.03\hfill & \hfill 0.24\hfill & \hfill \text{Nepal}\hfill & \hfill 0.08\hfill & \hfill 0.05\hfill \\ \hfill -0.08\hfill & \hfill 0.05\hfill & \hfill 0.07\hfill & \hfill 0.03\hfill & \hfill 0.25\hfill & \hfill 0.08\hfill & \hfill \text{Pakistan}\hfill & \hfill -0.01\hfill \\ \hfill 0.19\hfill & \hfill -0.19\hfill & \hfill 0.2\hfill & \hfill 0\hfill & \hfill 0.34\hfill & \hfill 0.05\hfill & \hfill -0.01\hfill & \hfill \text{Sri Lanka}\hfill \end{array}\right).$$

These correlations are much smaller than those between the life expectancies themselves, and in most cases are consistent with the absence of any correlation. Thus our projection model has accounted for all or most of the between-country correlation in South Asia.

Country-specific projections were then made by sampling the vector of random perturbations, (δ_ci,t), from a multivariate normal distribution whose covariance matrix incorporates the residual correlation matrix, (ρ̂_ci,cj). The aggregated projections for the South Asian region are shown in Figure 7. The 2007 IIASA projections available on their website (Lutz et al., 2008) are also depicted. We found that our median projections were fairly similar to IIASA’s median projections, but our intervals were much sharper, ranging from 36–72% narrower than those of IIASA.

Figure 7.

Life expectancy projections for South Asia (IIASA-defined) for our BHM model, IIASA and the UN. The median projections for BHM and IIASA are similar, but the IIASA 80% intervals are much wider than the BHM 80% interval.

Our approach yields stochastic trajectories that can fluctuate around the projected life expectancy. By contrast, the IIASA method samples random linear trajectories, which change at a constant rate throughout the projection period, with no fluctuations.

DISCUSSION

We have proposed a Bayesian way to produce probabilistic projections of life expectancy for all the countries of the world. One possible use of these projections is as inputs to the United Nations estimation and projections. We have shown by out-of-sample cross-validation that the method gives well-calibrated and sharp prediction intervals, and also gave better forecasts than the current UN method.

We have restricted our analysis to countries without generalized HIV/AIDS epidemics because of the singular nature of their demographic impact, mainly on sexually active adults. To explore the possible applicability of our method to the excluded countries, we loosened the exclusion rule and fitted our model to all countries with a 2000–2005 HIV/AIDS prevalence rate of less than 4% (n =179), an additional 21 countries. In the 10-year cross-validation of these countries, the 80% prediction intervals included the observed life expectancy 84% of the time. Including the additional countries did not greatly degrade sharpness: the mean absolute error was 1.2 years and the average 80% prediction interval half-width was 1.9 years, compared to 1.0 and 1.7 years respectively without the HIV/AIDS countries. Further research is needed to generalize our model to countries with a generalized HIV/AIDS epidemic while properly accounting for the uncertainty in AIDS mortality, but these results suggest that the current model could provide reasonable approximate results.

Our method projects male life expectancy only. Further research is needed to apply this model to life expectancy at birth among males and females simultaneously while ensuring that trajectories by sex do not diverge or cross. Lee & Carter (1992) suggested doing this by modeling the two sexes independently and introducing a new parameter to ensure that the stochastic trajectories do not cross or diverge. Such a model could also be made more complex by allowing the double-logistic parameters for the two sexes to be correlated. An alternative approach would be to combine two stochastic models: one model for one sex, as here, or for average life expectancy, and a second model for the gap between the sexes.

Our model assumes that the rate of increase of life expectancy will decline in the future, but that gains will continue, asymptotically, at a linear rate on average. It would be possible to modify the model to allow the rate of increase to decline to zero over time, but experience to date does not provide much support for this (Oeppen & Vaupel, 2002).

It could also be dangerous. Olshansky et al. (2009) argued that current official forecasts of life expectancy by the U.S. Social Security Administration and the U.S. Census Bureau may be underestimated, by about three years for males and eight years for females. They estimated that this discrepancy could cost as much as $3 to $8 trillion more than currently projected for Medicare and Social Security. The current official governement forecasts of U.S. male life expectancy in 2050 are 80–81 years, whereas Olshansky et al. (2009) projected 83–86 years under their two main scenarios, with a range of 78–113 years for their more extreme scenarios. Our BHM projections are close to theirs, with a median of 84.5 years and an 80% prediction interval of 82.2–86.4.

Our model estimates a different asymptotic average rate of increase for each country, and so it is possible theoretically that the gaps between countries could grow over time, which might be viewed as unrealistic. However, over our 90-year projection period this does not happen, and in fact the opposite occurs. The between-country standard deviation of the country-specific posterior predictive medians declines from 6.4 in 2005–2010 to 6.1 for 2095–2100. (For comparison, the between-country standard deviation of observed life expectancies in 2000–2005 was 6.5.)

This is because the Bayesian hierarchical model assumes that each country’s average rate of improvement converges to an asymptote as life expectancy increases, and finds the asymptotes for different countries to be similar (the standard deviation of the posterior medians of the asymptotes is only 0.05 years per five-year period.) This convergence effect dominates the small growth in the gap due to differences in the estimated asymptotes over the projection period.

Much other research has been done on the forecasting of mortality (Booth, 2006; Dowd et al., 2010). However, efforts have focused on developed countries where reliable age-specific data are available. The best known time-series method for forecasting age-specific mortality rates is the Lee-Carter method and its various parallels (e.g., Renshaw & Haberman, 2006), generalizations (e.g., de Jong & Tickle, 2006; Hyndman & Ullah, 2007; Pedroza, 2006; Koissi et al., 2006; Brouhns et al., 2002) and extensions (e.g., Li & Lee, 2005; Li et al., 2004; Ishii, 2008). As discussed earlier, the empirical evidence about the Lee-Carter method’s performance for forecasting life expectancy is mixed.

Other time-series methods to estimate and project age-specific mortality rates have been proposed. The Brass relational method fits a two-parameter model where the age-specific mortality rates are assumed to be given by a linear function of a user-chosen model life table on a logit scale (Brass, 1971). The Heligman-Pollard model is an eight-parameter model with three parts describing mortality in childhood, young adulthood, and later life (Heligman & Pollard, 1980). Although both models have fitted mortality data well (Keyfitz, 1991; Hartmann, 1987), difficulties may arise in projecting the parameters (Keyfitz, 1981). Like the Heligman-Pollard model, the Bongaarts shifting logistic model (Bongaarts, 2005) differentiates mortality at different ages by fitting a three-parameter logistic model and fixing the slope parameter across time while allowing the other two parameters to vary with time. However, the shifting logistic model focuses on senescent mortality and may be most relevant when infant/child and adult mortality are already negligible. Other models have focused on senescent mortality in terms of biological and evolutionary phenomena that incorporate the idea of heterogeneity in the population and frailty (Steinsaltz & Wachter, 2006, Yashin et al., 2000). These approaches require age-specific data.

Torri & Vaupel (2011) proposed a model for forecasting life expectancy in which the life expectancy for the leading country is modeled by an ARIMA model with constant drift, and the gap between the life expectancy for any given country and the leading country is modeled by a geometric Brownian motion model, estimated separately for each country. They applied their method to Italy and the U.S.

Instead of modeling age-specific mortality rates, Gage (1993) used a competing hazards model originally developed by Siler (1979) for animal mortality. The five-parameter model describes the hazard function as made up of three components, immature (or childhood), residual (or background), and sensecent mortalities. Bongaarts (2006) also suggested de-composing mortality into these three components with future projections focusing on the senescent mortality component. Both approaches require cause of death data, and Gage (1993) acknowledged that this is a limitation due to issues of data quality and cause of death classification.

In addition to time-series approaches, there are two other main approaches to developing probabilistic projections (Lee, 1998). As previously discussed, expert-based probabilistic projections have been produced by Lutz and colleagues at the IIASA (Lutz et al., 1998, 2004, 2008). However, this method does not explicitly rely on the use of available data, instead relying on a collection of experts and their ability to specify probabilistic bounds, which may or may not be accurate (Alho, 2005). The other main alternative to time-series methods is ex-post analysis of previous projections (Keyfitz, 1981; Stoto, 1983; Smith & Sincich, 1990). In this method, previous forecast errors are used to create probabilistic errors on future projections.

Several other Bayesian approaches have been proposed. As already discussed, Girosi & King (2008) proposed a Bayesian method that can incorporate covariates. They showed that their method outperformed the Lee-Carter method (without covariates) on average for 48 countries with better mortality data. This result was obtained when covariates were used, but this requires additional data that may not be reliable or even available in many countries. They did not give probabilistic forecasts and their software does not produce them, although their method may in principle be able to provide them.

Czado et al. (2005) presented a Bayesian method for estimating the Poisson log-bilinear formulation of the Lee-Carter model of Brouhns et al. (2002). Pedroza (2006) proposed a Bayesian approach to the Lee-Carter model by accounting for the uncertainty in the age parameters as well as the mortality index usually forecasted. While the latter two approaches account for uncertainty in the Lee-Carter model, their generalization to all countries is hindered by the non-availability of age-specific mortality rates.