A Semiparametric Risk Score for Physical Activity

Abstract

We develop a generalized partially additive model to build a single semiparametric risk scoring system for physical activity across multiple populations. A score comprised of distinct and objective physical activity measures is a new concept that offers challenges due to the nonlinear relationship between physical behaviors and various health outcomes. We overcome these challenges by modelling each score component as a smooth term, an extension of generalized partially linear single-index models. We use penalized splines and propose two inferential methods, one using profile likelihood and a nonparametric bootstrap, the other using a full Bayesian model, to solve additional computational problems. Both methods exhibit similar and accurate performance in simulations. These models are applied to the National Health and Nutrition Examination Survey (NHANES) and quantify nonlinear and interpretable shapes of score components for all-cause mortality.

1 |. INTRODUCTION

Scoring systems have been recognized as key tools to simplify intricate behavioural patterns and assess individual health status, with wide applications in nutrition and other fields. Given a set of health behaviors, an individual is assigned a score, usually ranging from 0 to 100, where 0 indicates poor adherence and 100 indicates perfect compliance. One example is the 2005 Healthy Eating Index (HEI), ^1,2 which comprises 12 distinct but interrelated dietary components. The total score is defined as the sum of all component scores. Some other examples include the Mediterranean Diet Score ³ and the 2010 Healthy Eating Index. ⁴ Despite extremely successful applications in nutrition, scoring systems remain a new concept in the physical activity research. One particular challenge is the reliability of physical activity measures, since the physical activity information collected from traditional self-reports is often crude, inaccurate, and biased. ⁵ For a scoring system based on these self-reported measures, see Keadle et al. ⁶ In contrast, modern wearable devices enable researchers to extract physical activity information from an accelerometer, a tool that measures proper acceleration. This provides an objective alternative to assess physical activity with less bias. ^7,8 A growing literature has shown strong associations between the objectively measured physical activity and different health outcomes in many large cohort studies. ^9,10,11 However, building a single scoring system using these objective measures remains unexplored and challenging due to their complex nonlinear relationships with various health assessments among different populations.

In this study, we focus on the National Health and Nutrition Examination Survey (NHANES), a nationally representative survey conducted by the US Centers for Disease Control and Prevention (CDC) to assess the health and nutritional status of the US population. The hip-worn devices were deployed in the 2003–2004 and 2005–2006 cohorts, among whom the accelerometry data are available in an analysis-ready format using the R rnhanesdata package. ¹² The accelerometry data were processed by NHANES and released as minute-level activity counts (AC), a proprietary measure of physical activity. For this analysis, we focus on a subset of summary measures of physical activity described in Smirnova et al, ⁹ which capture physical activity features at different levels and exhibit superb predictive performance of all-cause mortality, as shown in the same paper. We also exclude study participants who were younger than 50 or older than 85 years of age when the accelerometry data were collected or had missing variables of interest. The detailed data descriptions, including the selected physical activity measures and exclusion criteria, are fully discussed in Section 4. The final data set contains n = 3073 study participants, including 1559 men and 1514 women with 399 and 258 deceased events, respectively, within 9 years of data collection.

We propose a semiparametric risk score for physical activity for the NHANES data set, but our methodology is applicable to multiple populations and multiple diseases within each population. We use the word “disease” as a generic notation that can represent numerous health outcomes. Crucially, the score is shared across populations and diseases, enabling general recommendations about adequate level of physical activity, rather than myriad confusing recommendations varying across populations and diseases.

This work extends the generalized partially linear single-index model, ¹³ modelling each score component as a non-linear smooth function. A nonlinear model is needed since physical activities have thresholds above which disease risk decreases more slowly, if at all. Our model differs from recent methodology developments ^14,15 where the overall score was a transformation of the linear combination of its components. The smoothness of each score component function is ensured using penalized splines, ¹⁶ where we propose estimation and inference methods based on profile likelihood and Bayesian analysis using MCMC. Both methods are computationally fast and exhibit similar and interpretable results in our application. We provide the R code for model implementation in the supplementary material.

The remainder of the paper is organized as follows. We introduce our model in Section 2. Details of two estimation and inference methods are outlined in Section 3. Section 4 shows the results of our model applied to the NHANES data. Section 5 contains a simulation study. We conclude with discussion in Section 6.

2 |. MODEL

Consider the setting of one common disease among K > 1 populations. For simplicity we assume the disease outcome is binary, but our method easily extends to general outcomes. Within the kth population, there are n_k study participants. For the ith study participant in the kth population, Y_ik denotes the binary disease outcome and X_ik = (X_ik1,…,X_ikJ)^T the covariates used in the score. For a physical activity risk score, X_ik are J physical activity features of interest. In addition we observe Z_ik = (Z_ik1,…,Z_ikq)^T, the covariates that are not score components. Let H(·) be the logistic function. The risk score is defined as $S\left(X_{ik}\right)={\sum }_{j=1}^J{s}_j\left(X_{ikj}\right)$, where s₁(·), ···, s_J(·) are unspecified smooth score component functions. The unknown smooth functions s_j are assumed to have mean equal to zero for identifiability. Crucially, S(·) does not depend on k, which Ma et al ¹⁴ argue makes public health sense and which is common to all currently existing scoring systems.

For the ith study participant in the kth population, we propose the model

$$\mathrm{pr}\left(Y_{ik}=1\mid {\mathit{X}}_{ik},{\mathit{Z}}_{ik}\right)=\mathit{H}\left\{{\beta }_{0k}+{\beta }_{1k}S\left({\mathit{X}}_{ik}\right)+{\mathit{Z}}_{ik}^{\text{T}}{\mathit{\theta }}_k\right\}.$$ (1)

Here ${\beta }_{0k}\in ℝ$ and ${\beta }_{1k}\in ℝ$ are the population-specific intercept and slope, and ${\mathit{\theta }}_k\in {ℝ}^q$ are population-specific coefficients of non-score-component covariates.

The novelty in (1) is to build the score as a nonlinear combination of individual components, making it different from the partially linear single-index model ¹⁴ where $S\left({\mathit{X}}_{ik}\right)=m\left({\mathit{X}}_{ik}^{\text{T}}\mathbf{\gamma }\right)$ for unspecified function m(·). The nonlinear component assumption is crucial in our application, where each physical activity measure exhibits a non-monotonic effect on the health outcome. To ensure the smoothness of each s_j(·) and, implicitly, of S(·), we use penalized splines, ¹⁶ as follows.

For the ith study participant in the kth population, the jth score component is $s_j\left(X_{ikj}\right)={\mathit{B}}_j^{\text{T}}\left(X_{ikj}\right){\mathit{\alpha }}_j$, where B_j(·) is a P × 1 vector of spline basis functions and ${\mathit{\alpha }}_j\in {ℝ}^P$ are spline coefficients. Neither B_j(·) nor α_j depend on k. The model becomes

$$\mathrm{pr}\left(Y_{ik}=1\mid {\mathit{X}}_{ik},{\mathit{Z}}_{ik}\right)=H\left\{{\beta }_{0k}+{\beta }_{1k}{\sum }_{j=1}^J{\mathit{B}}_j^{\text{T}}\left(X_{ikj}\right){\mathit{\alpha }}_j+{\mathit{Z}}_{ik}^{\text{T}}{\mathit{\theta }}_k\right\}.$$ (2)

However, for any non-zero C, ${\beta }_{1k}{\sum }_{j=1}^J{\mathit{B}}_j^{\text{T}}\left(X_{ikj}\right){\mathit{\alpha }}_j=\left({\beta }_{1k}/C\right)\cdot {\sum }_{j=1}^J{\mathit{B}}_j^{\text{T}}\left(X_{ikj}\right)\left(C\cdot {\mathit{\alpha }}_j\right)$, leading to an identifiability problem. We solve that problem by adding the constraint β₁₁ = −1, assuming that a higher score means less disease risk in the first population. We find this assumption reasonable in our physical activity application. This is a common solution for partially linear logistic single-index models that is also applicable to (2), due to their very similar forms. However, the smoothness assumption of s_j(·) makes estimation of our model fundamentally different from that of other models in the literature, e.g., Ma et al ¹⁴ and Kravitz and Carroll. ¹⁵ We propose two novel methods in Section 3. The first method maximizes the profile likelihood iteratively for point estimates and uses a nonparametric bootstrap to draw inference, while the second method fits the model jointly using a Bayesian graphical model. After model fitting, the estimated score $\widehat{S}\left({\mathit{X}}_{ik}\right)$ is rescaled from 0 to 100 across study participants of the different populations.

3 |. ESTIMATION AND INFERENCE

3.1 |. Profile Likelihood

The multiplicative term β_1kS(X_ik) in (1) is nonstandard and requires new methodology. We propose an iterative estimation method that updates the slopes β₁₂, …, β_1K and the other parameters separately. The smoothness of each score component is ensured by assuming that all spline coefficients are random effects in a generalized mixed model. Their covariance matrix is a known matrix times a scalar variance component that is estimated by penalized quasilikelihood using the gamm() function in the mgcv package. The amount of smoothing is determined by the variance component and so is data-driven. ¹⁶ We initialize β_1k with pre-specified starting values ${\beta }_{1k}^{(0)}$ for k = 2, …, K and impose the identifiability constraint β₁₁ = −1. The tth iteration t = 1, 2, …, is implemented as follows:

Step 1.

Let β₀₁, …, β_0K, θ₁, …, θ_K be fixed effects parameters. For α₁, …, α_J where ${\mathit{\alpha }}_j={\left({\alpha }_{j1},\dots ,{\alpha }_{jP}\right)}^{\text{T}},j=1,\dots ,J$, assume all of their elements are random variables with the same variance so that $\mathrm{var}\left({\alpha }_{jp}\right)={\sigma }_{\alpha }^2$, p = 1, …, P, j = 1, …, J. Fit a joint logistic linear mixed model among K populations

$$\mathrm{pr}\left(Y_{ik}=1\mid {\mathit{X}}_{ik},{\mathit{Z}}_{ik}\right)=\mathit{H}\left\{{\beta }_{0k}+{\beta }_{1k}^{(t-1)}{\sum }_{j=1}^J{\mathit{B}}_j^{\text{T}}\left(X_{ikj}\right){\mathit{\alpha }}_j+{\mathit{Z}}_{ik}^{\text{T}}{\mathit{\theta }}_k\right\}.$$ (3)

Denote the estimated parameters as ${\beta }_{01}^{(t)},\dots ,{\beta }_{0K}^{(t)}$,${\theta }_1^{(t)},\dots ,{\theta }_K^{(t)}$. For spline coefficients, use their best linear unbiased predictions (BLUP)${\mathit{\alpha }}_1^{(t)},\dots ,{\mathit{\alpha }}_J^{(t)}$.

Step 2.

For the kth population, k = 2, …, K, fix β_0k, θ_k, α₁, …, α_J at their estimates, ${\beta }_{0k}^{(t)}$, ${\mathit{\theta }}_k^{(t)}$, ${\mathit{\alpha }}_1^{(t)},\dots ,{\mathit{\alpha }}_J^{(t)}$, from Step 1 and fit a simple logistic regression model within the kth population

$$\mathrm{pr}\left(Y_{ik}=1\mid {\mathit{X}}_{ik},{\mathit{Z}}_{ik}\right)=\mathit{H}\left\{{\beta }_{0k}^{(t)}+{\beta }_{1k}{\sum }_{j=1}^J{\mathit{B}}_j^{\text{T}}\left(X_{ikj}\right){\mathit{\alpha }}_j^{(t)}+{\mathit{Z}}_{ik}^{\text{T}}{\mathit{\theta }}_k^{(t)}\right\}$$ (4)

with β_1k the sole unknown parameter. Obtain all slope parameters ${\beta }_{12}^{(t)},\dots ,{\beta }_{1K}^{(t)}$ by fitting the model K − 1 times, once per population.

Step 3.

Calculate the loglikelihood

$$\ell \left({\mathit{\alpha }}^{(t)},{\mathit{\beta }}^{(t)},{\mathit{\theta }}^{(t)}\right)={\sum }_{k=1}^K{\sum }_{i=1}^{n_k}\left\{Y_{ik}\log (p_{ik}^{(t)})+\left(1-Y_{ik}\right)\log (1-p_{ik}^{(t)})\right\},$$ (5)

using the estimates from Step 1 and Step 2, where

$$p_{ik}^{(t)}=H\left\{{\beta }_{0k}^{(t)}+{\beta }_{1k}^{(t)}{\sum }_{j=1}^J{\mathit{B}}_j^{\text{T}}\left(X_{ikj}\right){\mathit{\alpha }}_j^{(t)}+Z_{ik}^{\text{T}}{\mathit{\theta }}_k^{(t)}\right\}.$$

Iterate until the change in loglikelihood between two consecutive iterations is below a pre-specified threshold. In our experience, the model converges within a few iterations.

In Step 1, the rescaled spline basis ${\beta }_{1k}^{(t-1)}{\mathit{B}}_j^{\text{T}}\left(X_{ikj}\right)$ is updated in each iteration. Since the structure does not change fundamentally, the mixed model notation of penalized splines easily extends to our model. This method also ensures the same smoothing parameter for all J components by assuming a common variance. This assumption: (1) is reasonable in our application where all physical activity variables are scaled during estimation; and (2) contributes to fast convergence. We show the R implementation of this method in the supplementary material.

Drawing inference for our model is a more challenging task as β_1k appears in a multiplicative form, distinguishing it from generalized additive models. In addition, for non-Gaussian responses the smoothing parameter no longer has the analytical form $\lambda ={\sigma }_{ϵ}^2/{\sigma }_{\alpha }^2$ like a linear mixed model. We found it impractical to simply ignore the smoothing parameter and use a sandwich estimator as in Ma et al, ¹⁴ since doing this gives excessively wiggly confidence bands along each curve.

For inference, we use a nonparametric subject-level bootstrap, which is conceptually simple, easy to implement, and takes into account the variability of all parameters. These bootstrap inferential approaches have shown promising coverage for modelling complex high-dimensional data; see Crainiceanu et al, ¹⁷ Park et al, ¹⁸ and Cui et al. ¹⁹ Using the same notation (Y_ik, X_ik, Z_ik) and assuming the total number of bootstrap samples is B, the bootstrap approach is described in Algorithm l.

The nonparametric bootstrap on our physical activity data with B = 300 samples takes ~ 10 hours to run on a regular laptop (2.7GHz Dual-Core Intel Core i5 processor). However, this bootstrap approach can be fully parallelized on a cluster, reducing the running time to essentially the same as running a single sample which takes ~ 2 minutes in the NHANES data set. We show the application results in Section 4.

3.2 |. Bayesian Method

An alternative approach to fit model (2) is through a full Bayesian model, with the advantage of data-based inference given proper priors. Denote the parameter set A = {β₀₁, …, β_0K, β₁₂, …, β_1K, θ₁, …, θ_K, α₁, …, α_J} with dimension d_A = (2 + q)K + JP − 1. The parameters fall into three categories: (1) β₀₁, …, β_0K, θ₁, …, θ_K, the fixed effects parameters in a frequentist model; (2) spline coefficients α₁, …, α_J; (3) population-specific slopes β₁₂, …, β_1K. For the first category, we use independent identically distributed normal priors as suggested in Crainiceanu et al. ²⁰ For spline coefficients, we follow the literature on the Bayesian view of penalized smoothers ^21,22,23 and use multivariate Gaus-Sian priors with precision matrix ${\sum }_{\ell }{\lambda }_{\ell }{\mathit{S}}_{\ell }$, where the λ_ℓs are drawn from vague gamma priors and S_ℓs are known matrices. A null space penalty ²⁴ is included due to the usual rank deficiency of ${\sum }_{\ell }{\lambda }_{\ell }{\mathit{S}}_{\ell }$. All penalty matrices are accessible from the R mgcv package ²⁵; see supplementary material. The population-specific slopes β₁₂, …, β_1K quantify the score effects on health outcomes among different populations. Contrary to the iterative frequentist method, these slopes fit well into a Bayesian graphical model using independent normal priors. We found their posterior distributions insensitive to the choice of priors.

We use JAGS²⁶ to implement this Bayesian method, which is fast, clear, and convenient through the R rjags package. The steps are as follows:

1. Specify the described model and priors in a JAGS file;

2. Reorganize the data in R to the required input format of JAGS;

3. Build a Bayesian model using the jags.model ( ) function in R rjags package;

4. Obtain posterior samples from the model built in Step 3 and draw inference.

For Step 2, the data are reorganized by combining the responses and predictors with the spline bases and penalty matrices obtained from the R mgcv package. The JAGS code for Step 1 and R code for the remaining steps are provided in the supplementary material.

This Bayesian method is simple to implement and computationally fast, taking less than 1 minute to obtain 1000 samples in our application on a regular laptop (2.7GHz Dual-Core Intel Core i5 processor). We want to emphasize the extreme simplicity and efficiency of this Bayesian method for our additive model. This was possible due to both the careful design of the JAGS language and the powerful mgcv package. Indeed, it is only through these convenient and flexible software packages that Bayesian methods can become popular and computationally competitive to deal with complex and practical statistical problems such as ours.

4 |. APPLICATION

4.1 |. NHANES

Our model is motivated by and applied to the National Health and Nutrition Examination Survey (NHANES), a nationally representative study conducted by the US Centers for Disease Control and Prevention (CDC) in two-year waves to assess the health and nutritional status of the US population. Each study participant was selected based on a multilevel survey design ²⁷ and assigned a weight indicating the size of represented population. We use this weight information to compare the predictive performance of the developed risk score. The NHANES data sets consist of (1) questionnaire data, such as sociodemographic and health-related information, and (2) examination and laboratory data, such as medical results and physiological measurements; see the study website (https://www.cdc.gov/nchs/nhanes/index.htm) for a complete list. In addition, the subject-level data are linked to the National Death Index (NDI) with mortality follow-up updated through December 31, 2015. The data sets are publicly available in analysis-ready formats through the R rnhanesdata package. ¹²

For this project we focus on the 2003–2004 and 2005–2006 cohorts, where physical activity data were collected by hip-worn accelerometers (ActiGraph AM-7164) for up to 7 consecutive days. The data were processed by NHANES and released as minute-level activity counts, a proprietary measure of physical activity intensity. Many summary measures have been proposed to extract features of different levels from these high-dimensional data; see Smirnova et al ⁹ for names and definitions of the measures in this specific data set. In this analysis we choose two representative measures: moderate-to-vigorous physical activity (MVPA) and active-to-sedentary transition probability (ASTP). MVPA is defined as the total minutes of a day (min/day) with activity counts exceeding 2020 in NHANES. ²⁸ ASTP, though less well-known, measures the transition probability from an active to sedentary state and has a strong association with many health outcomes. ²⁹ We follow the literature and estimate ASTP using the inverse of average duration of active bouts, where the active state is defined as AC over 100. Both measures are Winsorized at their 5th and 95th percentiles and standardized. The number of basis functions for each score component is 10. For the profile likelihood method we use O’Sullvian splines, ³⁰ while for the Bayesian method we use thin plate regression splines. ³¹ We find the results insensitive to the choice of splines. In addition, increasing the number of spline basis functions does not change the results substantially, which is expected from the penalized spline literature. ³² We impose the same smoothing parameter for both component functions. For this application, the disease outcome Y_ik is 9-year all-cause mortality, since all study participants in these two waves have a follow up time of at least 9 years. We include age (years), smoking status (former, current, and never), race (white, Black, Mexican Hispanic, other Hispanic, other), education level (less than school, high school, more than high school), body mass index (BMI) category (underweight, normal, overweight, obese), and self-reported overall health (excellent, very good, good, fair, poor) as the non-score-component covariates Z_ik, similar to the covariates included in a recent health score study. ⁶ The categorical predictors are restructured as indicators for being a nonsmoker, white, obese, for having education beyond high school, and for having self-reported overall health above or below good (three categories: above good, good, below good). We divide age by 100 in the estimation process for numerical stability, but final results are reported in the original unit of measure (years).

The NHANES 2003–2004 and 2005–2006 waves include a total of 14,631 participants with available accelerometry data. For this analysis, we exclude participants who: (1) were younger than 50 years old or older than 85 at the time they wore the accelerometer (10,859 participants); (2) had missing covariates of interest, including age, race, BMI, education, smoking, overall health status (219 participants); (3) had fewer than 3 days of data with at least 10 hours of estimated wear time (459 participants); and (4) had missing mortality information (21 participants). The final data set contains n = 3073 participants, including 1,559 men and 1,514 women, which represent the K = 2 populations in the model. For men we observe 399 mortality events, and for women we observe 258 mortality events within the 9-year follow-up time period.

4.2 |. Results

4.2.1 |. Estimation

We apply our model to the NHANES data set using both estimation methods described in Section 3. There are K = 2 independent populations, men and women, and one disease, 9-year all-cause mortality. The final estimates of the population-specific slope β_1k and non-score-component coefficients θ_k, k = l, 2 are shown in Table 1. The standard errors are estimated based on 300 bootstrap samples in the profile likelihood approach and 1000 posterior samples in the Bayesian method. Both methods give very similar point estimates and standard errors, even though one is frequentist while the other is Bayesian. Within each method, the estimated coefficients of non-score-component covariates are comparable between men and women, suggesting that the hazard increases with age and is higher among smokers, regardless of the population. These results are consistent with previously published literature. In addition, men with “above good” self-reported overall health are associated with significantly lower hazard, though we do not observe a significant difference on the hazards between men with “below good” and “good” overall health conditions. Also, in women, “above good” reported health does not have a significant effect on the hazard. This difference between men and women in the effect of “above good” health may correspond to the gender bias prevalent in self-reported health studies that indicate women report poorer health than men. ³³

TABLE 1.

Estimated population-specific slope (β_1k) and non-score-component coefficients (θ_k) in the NHANES application using both estimation procedures described in Section 3. The outcome is 9-year all-cause mortality. The estimates and standard errors (SE) are given as well as the p-value for the test that the estimate differs from 0. The slope for men is fixed as β₁₁ = −1 for identifiability.

Method	Coefficient	Men			Women
		Estimate	SE	p-value	Estimate	SE	p-value
Profile Likelihood	Slope (β1k)	−1.00			−0.75	0.16	0.00
	Age	0.08	0.01	0.00	0.10	0.01	0.00
	Smoker	0.60	0.18	0.00	0.65	0.22	0.00
	White	0.21	0.15	0.17	0.29	0.18	0.11
	Edu. Beyond High School	−0.21	0.17	0.22	−0.14	0.16	0.39
	Obese BMI	−0.09	0.16	0.57	−0.07	0.16	0.69
	Overall Health - above good	−0.44	0.17	0.01	−0.42	0.21	0.04
	Overall Health - below good	0.18	0.18	0.34	0.67	0.17	0.00
Bayesian Method	Slope (β1k)	−1.00			−0.75	0.14	0.00
	Age	0.09	0.01	0.00	0.10	0.01	0.00
	Smoker	0.60	0.18	0.00	0.69	0.23	0.00
	White	0.23	0.15	0.12	0.30	0.18	0.10
	Edu. Beyond High School	−0.20	0.16	0.20	−0.14	0.17	0.42
	Obese BMI	−0.09	0.15	0.56	−0.06	0.17	0.73
	Overall Health - above good	−0.44	0.17	0.01	−0.44	0.22	0.04
	Overall Health - below good	0.19	0.17	0.25	0.66	0.18	0.00

For score component functions s_j(·), j = 1, 2, we show their smooth estimates and 95% pointwise confidence bands from both methods in Figure 1. The confidence bands are created using 300 bootstraps in profile likelihood and 1000 posteriors in Bayesian method, respectively. For each component, we rescale the log-odds of mortality to have the total score range from 0 to 100 and plot the estimates against the original scale. Again we observe very similar estimates from both approaches, even for such complex nonlinear functions. The score component function of MVPA increases along small MVPA values until it levels off around 15 min/day. As the time spent in MVPA continues to increase, the function remains flat in the middle range of MVPA and but then increases again when MVPA is large, although not as rapidly as before. Additionally, the confidence bands are wider in this region, as less observations are reported for high amounts of MVPA. For ASTP, we observe a slight increase of smaller ASTP values and a monotonic decrease after a turning point around a transition probability of 0.25.

FIGURE 1.

Estimated nonlinear score component functions of the NHANES application using both estimation procedures described in Section 3. Top panels are the results from profile likelihood approach while the bottom panels are from Bayesian approach. The first column shows the curve of MVPA, and the second column shows the curve of ASTP. Within each panel, solid curve denotes the nonlinear estimates and dash curves denote the 95% pointwise confidence bands. The values on the y-axis represent the effect of each variable on the log-odds of mortality and are rescaled for the score to range from 0 to 100.

To understand the shapes of these nonlinear score component functions better, we computed their first derivatives using finite difference approximations, as well as corresponding 95% pointwise confidence intervals using the variance estimate along the curve at each observation. Figure 2 shows the first derivative estimates of both score components. We use the pointwise confidence intervals to identify where the derivatives are significantly above or below zero (equivalently, where the curves are increasing or decreasing). The plots of the two estimation methods give essentially the same interpretations. The estimated first derivative of the MVPA function is statistically significantly above zero until around 12.5 min/day and then stays near zero with no statistical significance as the amount of time spent in MVPA increases. For the ASTP function, the first derivative does not significantly differ from zero until a transition probability of 0.3, and then it is statistically significantly below zero as the value of ASTP increases. Therefore, the initial increase seen in Figure 1 may be due solely to random variation.

FIGURE 2.

Finite difference-based estimate of the first derivative of smooth functions of MVPA and ASTP predicting 9-year all-cause mortality with their 95% pointwise confidence bands. The top row of plots uses 300 bootstrap estimates from the profile likelihood method in Section 3.1, and the bottom row of plots uses 1,000 posterior estimates from the Bayesian method in Section 3.2. The blue line is the estimated curve for the first derivative, and the dashed blue lines represent the pointwise interval.

These derivative results provide reasonable explanations for the shape of the estimates of the score component functions, as they suggest the increasing pattern of MVPA function is statistically significant only for small MVPA values and the decreasing pattern of ASTP function is statistically significant only when ASTP is large. These patterns are consistent with the literature as low MVPA and high ASTP, the conditions that lead to low estimated risk scores in this population, have been linked with many severe health outcomes. ²⁹ However, this analysis also reveals some additional interesting findings: (1) the effect of MVPA and ASTP on health outcomes are nonlinear, with statistical significance shown only within certain regions of the entire domain; (2) given a fixed MVPA intensity, larger ASTP is associated with higher hazard of all-cause mortality. To the best of our knowledge, these findings are new in the physical activity literature and can motivate future research in different directions, including the intensity-stratified analysis of single measures or the joint modelling of multiple physical activity measures.

The similarity between the frequentist and Bayesian approaches suggests that both are reassuring. Their similarity is not surprising, since they are based on the same likelihood function, and the Bayesian analysis uses a noninformative prior so that the estimate are mostly data-driven, not driven by the prior. We also want emphasize the importance of modern software development that ensures fast and convenient implementations of both methods, especially the rjags package.

4.2.2 |. Predictive Performance

We assess the predictive performance of our derived risk score using 10-fold survey-weighted cross-validated AUC. The survey weights are obtained by re-weighting individuals with complete data, following the methodology in Leroux et al. ¹² As shown in Smirnova et al, ⁹ age is one of the best single predictors for all-cause mortality in the NHANES population. Given the strong performance of age shown in many published literature, we focus instead on the further improvements of predictive performance using the developed physical activity score.

For our NHANES population, the cross-validated AUC of age in a single predictor model is 0.768. When adding the risk score as a new predictor in addition to age, the cross-validated AUC improves substantially to 0.824 (p-value < 0.001 using a t-test for 100 bootstrap samples of the change in AUC). We found only marginal difference of the scores obtained from two estimation methods. As a comparison seen in Table 2, the cross-validated AUC does not exceed 0.810 when simply adding each single physical activity measure, or even both, to age. These results suggest the efficacy of our model on integrating variables among multiple populations and producing a single interpretable physical activity measure that achieves excellent predictive performance in addition to age.

TABLE 2.

10-fold survey-weighted cross-validated AUC of the risk scores (PL and Bayesian methods) and selected physical activity variables with age on 9 year all-cause mortality prediction. The standard error (SE) of the AUC based on 10 folds is given in the last column. Variables included for comparison are age with MVPA and ASTP individually and jointly.

Variables	Cross-Validated AUC	SE
Risk score + age	0.824	0.012
MVPA + ASTP + age	0.810	0.012
ASTP + age	0.806	0.012
MVPA + age	0.792	0.012
Age	0.768	0.012

4.2.3 |. Score Function Population Independence

An assumption of ours is that the risk score function S(·) does not explicitly depend on the population k. We test this hypothesis by comparing the AIC and BIC of the model when the score is dependent or independent of populations using our application seting. The model with the score component independent of population is Equation (2). For the case where the score is population-specific, we fit a similar model by incorporating all covariates, but now we derive separate smooth functions for each population: men and women. In particular, we are fitting

$$\mathrm{pr}\left(Y_{ik}=1\mid X_{ik},Z_{ik}\right)=H\left\{{\beta }_{0k}+\sum _{j=1}^J{\mathit{B}}_j^{\text{T}}\left(X_{ikj}\right){\mathit{\alpha }}_{kj}+{\mathit{Z}}_{ik}^{\text{T}}{\mathbf{\theta }}_k\right\},$$

which lacks the term β_1k since the dependent score no longer needs adjustment for population-level variability. Because of this, the profile likelihood estimation procedure is no longer iterative for the dependent score model, and we can just use Step 1 from Section 3.1 to obtain the parameter estimates with ${\sum }_{j=1}^J{\mathit{B}}_j^{\text{T}}\left(X_{ikj}\right)$ as the random component. For the Bayesian method, we follow the algorithm in Section 3.2 and create a new JAGS file that excludes the β_1k term. Details on implementation of both methods are provided in the supplementary material. Table 3 shows the results for both estimation procedures from Section 3. In both methods, AIC and BIC select our proposed model where the score component is independent of k and thus support our claim.

TABLE 3.

The AIC and BIC values for models where the score does or does not depend on the population. The model with the score component independent of population (“Independent”) take the form of Equation (2). The estimates for the model with the score component dependent of population (“Dependent”) are derived by fitting two separate generalized additive models for men and women. Both techniques from Section 3 are used to fit the model and calculate the AIC and BIC.

Method	Model	AIC	BIC
Profile Likelihood	Independent	2486.6	2721.8
	Dependent	2527.4	2889.2
Bayesian Method	Independent	2483.3	2706.4
	Dependent	2518.5	2856.2

4.3 |. Goodness of Fit

To determine how well the NHANES data fits the model, we perform a goodness of fit measure with the Hosmer-Lemeshow test. ³⁴ To begin, we categorize the data into G = 10 groups. The Hosmer-Lemeshow test statistic $\widehat{C}$, which follows a χ²(G − 2) distribution, is then calculated as

$$\widehat{C}=\sum _{g=1}^G\left(\frac{{\left(O_{1g}-E_{1g}\right)}^2}{E_{1g}}+\frac{{\left(O_{0g}-E_{0g}\right)}^2}{E_{0g}}\right),$$

where O_1g and E_1g are the observed and expected all-cause mortality events and O_0g and E_0g are the observed and expected survival events for the g^th risk decile group. Using our model and the NHANES data, we get $\widehat{C}=3.51$ with a p-value of 0.90 for the profile likelihood method and $\widehat{C}=4.35$ with a p-value of 0.82 for the Bayesian method. From both estimation procedures, we fail to reject that the observed and expected proportions are the same across each group. Thus, the model provides a good fit for the data.

5 |. SIMULATION

5.1 |. Simulation Framework

To assess both estimation methods and, in particular, the results in Section 4.2, we conducted simulations with focus on estimation accuracy. The data are simulated from Model (1) so that we consider K = 2 independent populations, men and women, and one disease, similar to our application example. For design balance, we assume n₁ = n₂ and define n = n₁ + n₂ as the total number of subjects. We consider sample size n = 2000 so that n₁ = n₂ = 1000, a moderate sample size in physical activity research. In each simulation, we randomly sample n₁ men and n₂ women without replacement from the real NHANES data. For sampled subjects, we include both MVPA and ASTP values for score components, as well as age and smoker indicator for non-score components to build the simulated data set.

We set β₁₁ = −1 as our identifiability constraint and β₁₂ = −0.81. As for the other parametric coefficients for age and smoker, the values for θ₁ = (0.08, 0.52)^T and θ₂ = (0.09, 0.64)^T. These coefficients are kept the same across all simulations. The true function for MVPA is s₁(x) = −{−0.2(x − 0.8)³ − 0.4) and for ASTP is s₂(x) = −{0.3 exp(x) − 1.25} such that S(X_1k) = s₁(X_ik1) + s₂(X_ik2). The disease outcomes are then generated using the model. We simulated 200 data sets. Both estimation methods were applied to derive the estimates for parametric coefficients as well as nonlinear score components.

5.2 |. Simulation Results

Table 4 shows the simulation results for β_k and θ_k using both estimation procedures outlined in Section 3. The “Truth” column contains the specified true values of parameters described in Section 5.1. The “Estimate” column shows the average estimates of each parameter from 200 simulations, and the “Coverage” column reports the estimated coverage probability of 95% confidence intervals across all the simulations. Both methods give very reasonable point estimates with only minimal bias in the estimation of β_1k. The coverage probabilities are close to the nominal values, suggesting valid inference on the non-score components using both frequentist and Bayesian approaches. The standard error of a coverage probability, assuming that the probability is 0.95, is $\sqrt{(0.05)(.95)/200}=0.015$. The coverage probability 0.98 for β_1k is two standard deviations from 0.95. However, this is the worse case and if we make a Bonferroni correction for multiple comparisons, the deviation is not statistically significant.

TABLE 4.

Simulation results of population-specific slope (β_1k) and non-score-component coefficients (θ_k) using profile likelihood (top) and Bayesian (bottom) estimation methods from 200 simulations. The “Truth” columns show the sampled true values of parameters (β₁₁ = −1 is the identifiability constraint). The “Estimate” columns show the average estimates of each parameter from 200 simulations, and the “Coverage” columns report the estimated coverage probability of 95% confidence intervals across all the simulations.

Method	Coefficient	Men			Women
		Truth	Estimate	Coverage	Truth	Estimate	Coverage
Profile Likelihood	Slope (β1k)	−1.00			−0.81	−0.87	0.98
	Age	0.08	0.08	0.96	0.09	0.09	0.94
	Smoker	0.52	0.55	0.96	0.64	0.62	0.96
Bayesian Method	Slope (β1k)	−1.00			−0.81	−0.86	0.96
	Age	0.08	0.08	0.96	0.09	0.09	0.96
	Smoker	0.52	0.54	0.95	0.64	0.65	0.95

For the score components, we again used 10 basis functions to estimate each nonlinear component. Specifically, we use O’Sullivan splines ³⁰ for profile likelihood and thin plate regression splines ³¹ for Bayesian method. The penalties of these splines are incorporated within the estimation process as outlined in Section 3. The final estimated curves are derived on an equally spaced grid of 100 locations over respective range of MVPA and ASTP. The curves are further smoothed along the specified grid. For additive models derived from R mgcv the nonlinear functions are centered at 0, leading to estimation bias on the intercept term. As a result, we center both true and estimated curves by subtracting off the mean value of each curve. This allows for a more direct comparison since there is no longer an intercept difference between the truth and estimates.

The simulation results for the nonlinear score component functions are shown in Figure 3. The top two panels show the estimates from the profile likelihood, while the bottom panels show the results from the Bayesian method. The results of MVPA in the first column and those of ASTP are in the second column. The red solid curves denotes the true function, while the black dashed lines represents average estimates from 200 simulations. The top 50th percentile of estimated curves from the simulations with the smallest sum of squared deviations from the true fit at each observation are shown in light gray. Both estimation methods capture the shapes of MVPA and ASTP. Indeed, we observe minimal divergence from the true MVPA function using both methods despite the truth having an inflection point, which is difficult to capture. There is virtually no bias as the true ASTP function is almost exactly approximated by the average estimates from both methods. These plots also suggest that the profile likelihood method has a tendency to undersmooth more than the Bayesian method.

FIGURE 3.

Simulation results of score component functions for MVPA and ASTP from 200 data sets. Top: estimates using profile likelihood method; bottom: estimates using Bayesian method. The first column shows estimates of the MVPA function, and the second column shows estimates of the ASTP function. Within each panel, the red solid curve denotes the true function, and the black dashed curve denotes average estimates from 200 simulations. The top 50th percentile of estimated curves from the simulations with the smallest sum of squared deviations from the true fit at each observation are shown in light gray. The average coverage at each location along the curve over the 200 simulations is given in the title for each component.

These simulation results suggest reasonable inference for the parameters of interest in our model, whether linear or nonlinear, using both the profile likelihood and Bayesian methods. In addition, pairing these results with those from Section 4.2 suggest that these approaches give reliable estimates both in application and in simulations.

6 |. DISCUSSION

We have developed a generalized partially additive model to integrate information from multiple populations and built a single semiparametric risk score for physical activity. Our innovation is to model each score component as an additive smooth term, distinguishing our work from recent methodological developments ^14,15 on extending generalized partially linear single index models. ¹³ The nonlinear assumptions are crucial to our application, as different physical activity features exhibit varying nonlinear patterns on health outcomes.

For model implementation, we introduced two novel estimation methods. One extends the profile likelihood approach in linear models and utilizes nonparametric bootstrap for inference, while the other uses a full Bayesian model and draws inference from posterior samples. Although these two methods are, in principle, fundamentally different, we observe very similar estimation and inferential results, both in our application and in simulations. This important finding suggests that the frequentist and Bayesian methods are a pair of good complements for future research. In choosing which method to use, both approaches give very similar results, so that the choice is mostly a matter of convenience and one’s statistical philosophy, that is, how strongly one feels for or against a Bayesian analysis. The advantage of the profile likelihood approach is that it is based on estimating equations. The nonparametric bootstrap used to derive confidence bands provides no assumption on the model formula. For implementation, this method runs quickly on a cluster. As for the Bayesian method, it requires a likelihood function that is assumed to be correctly specified. Additionally, this method runs fairly quickly through an R session without reliance on a cluster for computational speed. With the aide of modern statistical software developments, the computation speed of both methods is quick on a local laptop, allowing an extension of this work to even larger data sets. For practical application, the R code to implement both methods are provided in the supplementary material.

In practice, when deciding which variables to include in the nonlinear component, it is recommended to begin with all variables as linear components. Then for continuous variables, we model one or several of these at a time as smooth functions via their spline bases. To decide whether to include these variables in the nonlinear part, observe the curvature of these nonlinear terms and the model improvement compared with the baseline model. Iterate this procedure until the model reaches a good fit and the shapes of nonlinear terms remain interpretable. Because the model is designed to adjust for multiple populations and diseases, incorporating additional populations and/or score components will change the results numerically. In particular, if additional nonparametric model variables are highly correlated with ones already in the model, the shapes derived from the penalized splines may change significantly. Thus, it is preferred to abide by Occam’s Razor and keep the model simple.

Our application to the NHANES data set produces a single risk score for physical activity. The score is an additive model, that is, the sum of nonlinear functions of physical activities measures. Because of the additivity, the effect of each individual measure on mortality can be plotted, independent of the other measures. Therefore, the model is easy to understand. These effects were modeled as splines, since their behaviors are clearly nonlinear. For example, Figure 1 shows that MVPA decreases mortality risk up to 15–20 minutes/day but increasing MVPA beyond this range shows no further decrease of mortality risk. Moreover, the score applies to all populations, so health professionals can recommend adequate levels of physical activities without regard to group membership, e.g., to both men and women.

The PA components in the NHANES study were measured using an ActiGraph AM-7164. From this study, only the processed activity counts are publicly available, and the conversion algorithm from raw triaxial acceleration to AC was not disclosed by the CDC. Although we cannot access the raw triaxial acceleration data, the thresholds we used to calculate physical activity summary measures in the paper were provided in Matthews et al ²⁸ and are viewed as standard thresholds for this specific NHANES sub-cohort. Thus for the PA components, the cutoff values for classifications of sedentary and moderate-to-vigorous physical activity were predetermined. Changing these definitions or using different devices or algorithms will change the results, as different algorithms of converting raw accelerometry data to activity metrics and using other definitions for these physical activity components contribute measurement errors of different magnitude in the score components.

With the aim of developing a single index independently from different populations and outcomes, there are several possibilities for future work. There could be interest in making the score components depend on some of the other covariates which are included as the linear main effects or modeling interactions. For example, the nonparametric component could take the form of a varying coefficient model or bivariate function between age and physical activity. A factor by curve model is also achievable by binning age to different levels, such as individuals with age < 60, 60–70, and so on. Additionally, many physical activity studies are designed as interventions with longitudinal assessments. Since lifestyle modifications are difficult to maintain, the functional shapes may change over time due to fatigue, which provides a new perspective on what this type of model could accomplish. Though beyond the scope of this paper, it is possible to extend our method to longitudinal data via working independence modeling. For the profile likelihood method, the extension to a longitudinal study is natural since the bootstrap takes into account the variability of all parameters. For the Bayesian method, this would require modeling the regression errors at different time points, which could be of interest for future research in scoring systems.

DATA AVAILABILITY STATEMENT

The data are organized using the R rnhanesdata package ¹² on Github at https://github.com/andrew-leroux/rnhanesdata. These data were derived from the following resource available in the public domain: https://www.cdc.gov/nchs/nhanes/index.htm. The R and JAGS programs used in the data analysis of Section 4 and simulations of Section 5 are available on GitHub at https://github.com/christithomp/semiparametric-risk-score.