Model Selection of the Effect of Binary Exposures over the Life Course

Supplemental Digital Content is available in the text.

Abstract

Epidemiologists are often interested in examining the effect on a later-life outcome of an exposure measured repeatedly over the life course. When different hypotheses for this effect are proposed by competing theories, it is important to identify those most supported by observed data as a first step toward estimating causal associations. One method is to compare goodness-of-fit of hypothesized models with a saturated model, but it is unclear how to judge the “best” out of two hypothesized models that both pass criteria for a good fit. We developed a new method using the least absolute shrinkage and selection operator to identify which of a small set of hypothesized models explains most of the observed outcome variation. We analyzed a cohort study with repeated measures of socioeconomic position (exposure) through childhood, early- and mid-adulthood, and body mass index (outcome) measured in mid-adulthood. We confirmed previous findings regarding support or lack of support for the following hypotheses: accumulation (number of times exposed), three critical periods (only exposure in childhood, early- or mid-adulthood), and social mobility (transition from low to high socioeconomic position). Simulations showed that our least absolute shrinkage and selection operator approach identified the most suitable hypothesized model with high probability in moderately sized samples, but with lower probability for hypotheses involving change in exposure or highly correlated exposures. Identifying a single, simple hypothesis that represents the specified knowledge of the life course association allows more precise definition of the causal effect of interest.

Medical research over the past two decades has examined fetal and early life antecedents of disease, and their interaction with other exposures throughout the life course to influence later-life conditions.¹ Several hypothetical relations between repeated covariate measures (e.g., repeated measures of socioeconomic position in childhood) and a subsequent outcome (e.g., adult blood pressure) can be proposed, based on theoretical models or mechanisms of action.^2,3 For example, a hypothesized “critical period” in early childhood specifies that socioeconomic position during this period has lasting effects on blood pressure. An alternative hypothesis is of an “accumulation” of risk across the life course (i.e., that adverse social circumstances at any time increases subsequent risk of high blood pressure). The hypotheses examined should inform the analytic methods used.^4–6 The first step in estimating a causal association is to specify knowledge about the system—in this case the life course—being studied.⁷ This knowledge will be incomplete unless the hypothesized relation between exposures and outcome has been investigated. The initial investigation may be thought of as exploratory: determining the most likely relation, while later analyses may be thought of as confirmatory: verifying the hypothesized relation and checking that no other relations are present.

There has been growing interest in a structured approach to life course hypotheses, in which a closed set of hypotheses is proposed and tests conducted to identify best-fitting hypotheses.⁸ One such approach uses an F test to compare a saturated model with hypothesized models concerning the association between binary exposure variables, measured over the life course, and an outcome.⁹ This method has been used in several studies with continuous outcomes^10–16 and adapted for binary outcomes.^17–19 A hypothesis may be thought of as supported by observed data if the F test yields a P value above a certain threshold, although a large P value cannot be considered to “prove” the hypothesis. If more than one hypothesis passes the threshold, the hypothesis that renders the largest P value, or smallest Akaike information criterion,^20,21 may be selected. The performance of these methods in the life course setting has not been formally assessed.

We describe an alternative model selection strategy that identifies which hypothesis, selected from an a priori-compiled set of hypotheses, explains the most variation in the outcome. We illustrate how the proposed method can be used in both exploratory and confirmatory studies. The performance is contrasted with the structured F test approach in data from a previously published example,⁹ and also through simulation.

METHODS

The proposed strategy involves selecting from an a priori-compiled set of potential hypotheses describing the association between exposure over the life course and outcome. Each hypothesis is encoded into one or more variables, which are then all included in a regression model, and the subset of variables that explains the greatest proportion of the outcome variation is selected. The number of hypotheses is not limited; any hypothesis may be examined provided there are enough exposure measurements to identify it. Several variations of similar hypothesis may be considered (e.g., a set of critical period hypotheses covering a range of possibly overlapping periods). The choice of hypotheses to examine may be informed by knowledge of causal mechanisms, perhaps using a directed acyclic graph. Examples of typical hypotheses for binary exposures are described below.

An accumulation hypothesis states that there is a linear association between the outcome and the cumulative sum of the exposure over the life course.

A critical period hypothesis states that only exposure during one period is associated with the outcome. Under a sensitive period hypothesis, the outcome is associated with the amount of exposure, as in the accumulation hypothesis, but the association is stronger in a particular period.²¹

A mobility hypothesis states that the outcome is associated with changes in the exposure over time. The simplest mobility hypotheses relate the outcome only to unidirectional changes. A more complex mobility hypothesis may relate the outcome to bidirectional changes^9,22 (e.g., a positive association with increased exposure and a negative association with decreased exposure). This would in general enhance the plausibility of a causal association. The related interaction hypothesis states that the outcome is associated with the exposure in a particular period, but that this association is altered by the exposure in a different period.

Encoding of Variables

Each hypothesis is encoded as a variable that is proportional to the hypothesized outcome as the exposure varies. Simpler hypotheses may be encoded by a single variable; more complex hypotheses need multiple variables. Below, we give details of the single variables that encode the simple hypotheses discussed above. We assume a set of m repeated binary measures of exposure X₁,…,X_m.

The accumulation hypothesis is encoded by the variable A = X₁ +…+ X_m.

If there is only one measurement occasion during a hypothesized critical period then only that measurement will be associated with the outcome. A hypothesis of a critical period at the jth measurement occasion is encoded by the variable C_j = X_j.

If there are several measurement occasions during the critical period (e.g., X_j,…,X_k), then the critical period hypothesis may be encoded by C_jk = X_j +…+ X_k, i.e., as accumulation within the critical period.

Under the simplest mobility hypothesis, the outcome varies with a unidirectional change in the exposure. A mobility hypothesis between the jth and kth measurement occasions may be encoded by = (1 − X_j)X_k if it is hypothesized that a positive change from j to k is associated with the outcome, or by = X_j(1 − X_k) if it is hypothesized that a negative change from j to k is associated with the outcome.

Some hypotheses require more than one variable to encode, and can therefore be thought of as compound hypotheses. All of these can be encoded by combinations of variables encoding simple hypotheses; some quite complex hypotheses can be encoded with two variables. The combinations of variables that encode compound hypotheses are described below, with further details provided in eAppendix A (http://links.lww.com/EDE/A940).

A sensitive period hypothesis can be encoded by the combination of the accumulation variable and the relevant critical period variable.

Two simple mobility variables can, together, encode a more complex mobility hypothesis. For instance, mobility hypotheses may combine a variable encoding positive change with a variable encoding negative change, or variables encoding change at different pairs of measurement occasions over the life course.⁹ An interaction hypothesis can be encoded by combining critical period and mobility variables. For example, combining the critical period variable C_j with mobility variable encodes a hypothesis that the outcome is associated with the exposure measurement at occasion j, but this association is modified by the exposure measurement at occasion k.

Choosing Hypotheses Most Strongly Supported by Observed Data

After encoding potential hypotheses, our approach is to examine the association between all encoded variables and the outcome, and select only those encoded variable combinations that have the strongest association with the outcome. Since there are potentially more variables than available degrees of freedom it is inappropriate to put all encoded hypotheses into a linear model and choose the variable(s) with the largest parameter estimates. Instead, we propose placing an absolute value penalty on parameter estimates, whereby unimportant variables have their estimated association shrunk to zero. Hence, the resulting fit will provide the fullest explanation of the observed data from the fewest parameters. The Least Absolute Shrinkage and Selection Operator (lasso),²³ which minimizes the residual sum of squares plus an absolute value penalty, provides a suitable method, but requires selection of a smoothing parameter. This can be simplified by implementing the Least Angle Regression (LARS) approach to the lasso,²⁴ which provides lasso estimates for all smoothing parameter values and indicates the best lasso fit for each number of selected variables, reducing the problem to that of choosing the number of variables. The lasso is a constrained version of linear regression and may be used whenever the assumptions of linear regression are satisfied. The LARS algorithm first selects the variable with the strongest association with the outcome,²⁵ hence this approach will always select first the hypothesis, or component of a compound hypothesis, that offers the strongest explanation for the observed data. Using an absolute value penalty causes subsequent variables to be added in order of strength of association with outcome variation.²⁶ The overall hypothesized view of the data is thus built from the most relevant simple hypotheses or components of compound hypotheses.

When little is known regarding the association between the exposures and outcome over the life course, the structured approach can be used to suggest the most likely of an a priori-defined set of hypothesized associations. This set may extend to several hypotheses if we have little a priori information about which are likely. In this exploratory setting, the choice of hypothesis is somewhat subjective and we require a method for choosing how many variables to include in our selected hypothesis. We use an elbow plot—a plot of the proportion of outcome variation explained by the lasso fit (the R² value) against number of variables selected at each stage in the LARS procedure. The “elbow”—a sharp concave bend at which adding more variables does not substantially increase the R² value—is used to choose the number of variables. Provided enough variables are included in the procedure, the elbow plot will show how the lasso selections approach a saturated model. It is useful to see the R² value for a saturated model to check whether there is any association between the outcome and all exposure measurements over the life course. An alternative method for selecting the number of variables is provided by the lasso covariance hypothesis test.²⁷ At each stage of the LARS procedure, this tests the null hypothesis that adding the next variable does not improve the R² value. The covariance hypothesis test accounts for the fact that the next variable will have the greatest association out of the variables not already selected. An alternative option, a nested F test to discriminate between simple and complex hypotheses,²² may be biased by the selection of the simpler model due to its greater association. In the exploratory setting, it may be necessary to reject more compound hypotheses in favor of simpler ones to maintain plausibility.

We may have a firmer idea of the nature of the life course association between exposures and outcome, and perhaps some causal information. In this setting, we might specify quite a small number of possible hypotheses a priori, and rather than choose the number of variables in our selected hypothesis we might simply choose the first variable or hypothesis selected by the LARS algorithm, to confirm our causal assumptions.

EXAMPLE

The proposed approach, in the exploratory setting, is illustrated using data on socioeconomic position and body mass index (BMI) from a cohort of 2,192 men and women, with binary measurements of the exposure, socioeconomic position, at ages 4, 26, and 43 years, and a continuous measurement of the outcome, BMI, at age 53 years. These data were previously used to illustrate the structured approach, and full details are given in the original study.⁹ Issues such as confounding and measurement error were not the focus of the original study and are therefore ignored here for sake of simplicity and comparison with the alternative structured method. Figure 1 shows a possible directed acyclic graph for this example. While there are many potential confounders of the life course association between exposure and outcome, which in this example are unmeasured (as they were in the original study), the focus in this exploratory setting is to identify likely life course associations between exposure and outcome, whether or not they are confounded. We considered the set of six hypotheses that have been previously proposed for this data: three critical period hypotheses corresponding to the three exposure measurement occasions, two mobility hypotheses concerning change between adjacent measurement occasions, and an accumulation hypothesis.⁹

Encoding of Variables

Each hypothesis was encoded based on the binary exposure measurements X₁, X₂, and X₃ at the three time points, where zero represents a manual socioeconomic position and one a nonmanual socioeconomic position. The simple hypotheses, requiring one variable each, were the three critical period hypotheses, encoded by C₁, C₂, and C₃, and the accumulation hypothesis, encoded by A = X₁ + X₂ + X₃. The two mobility hypotheses required two variables each, with and encoding mobility between ages 4 and 26 years, and and encoding mobility between ages 26 and 43 years.

Use of Elbow Plot

Figure 2 shows the elbow plot for men. The variance explained by the model with the greatest number of variables was 1.7%, which approached the 2% explained by the saturated model; hence the maximum R² value on the plot is close to the maximum R² value achievable. There is a clear elbow where one variable is selected; adding additional variables did not considerably improve the R² value. In addition, the P value for adding a second variable was 0.90, indicating no evidence that one variable was insufficient. The first variable selected encoded the hypothesis of an age 4 critical period; choosing this elbow point identified this hypothesis as offering the best explanation for the observed data in men.

Figure 3 shows the elbow plot for women. The maximum R² value on the plot is 3.8%; that of the saturated model was 3.9%. Therefore, socioeconomic position explained a greater proportion of the BMI variation in women than men. The position of an elbow point was less clear for women. The first variable selected encoded the accumulation hypothesis; the P value for adding a second variable was 0.52. It could thus be concluded that the accumulation hypothesis is the primary explanation for the observed data in women. The plot might also be considered to have an elbow at three variables, selecting variables that encoded the accumulation, childhood critical period, and adult nonmanual to manual mobility variables. Our interpretation of this is a hypothesis of two sensitive periods: sensitivity (to manual socioeconomic position) in childhood and sensitivity to change (from nonmanual to manual socioeconomic position) in adulthood.

In the exploratory setting, further study would be required to clarify these hypotheses and examine causal mechanisms.

SIMULATION STUDY

To investigate how frequently the LARS algorithm selects the correct hypothesis in a confirmatory setting, we simulated data with two and three repeated binary exposures. The performance indicator is the selection probability: the proportion of simulations where the correct hypothesis is identified.

Two Exposure Measurements

Two exposure measurements were simulated as binary random variables, being zero or one with equal probability (see eAppendix B for details; http://links.lww.com/EDE/A940). We considered the situation in which the outcome is known to be associated with change in the exposure, but it is yet to be confirmed whether a mobility or interaction hypothesis defines the true association. We simulated mobility and interaction models, varying the correlation, ρ, between exposure variables, and the residual variance, σ². In each simulation, we selected from six proposed compound hypotheses (four interaction hypotheses, full mobility, and an additive model).

We compared three approaches for identifying the hypotheses offering the best explanation for the simulated data. The first was the LARS algorithm for the lasso, which was considered to have identified the correct hypothesis if the first two selected variables encoded that hypothesis. The second approach chose the hypothesized model with the largest F test P value when compared with the saturated model,⁹ and the third approach selected the hypothesized model yielding the smallest Akaike information criterion of those models with an F test P value not less than 0.05.²¹

We varied the sample size from 400, which might represent a subset of a study, to 2,500, which might represent a moderate-sized study. We ran 500 simulations for each combination of residual variance, correlation, model, and sample size. With 500 simulations, the 95% confidence interval for the selection probability will have a radius of less than 2% for a selection probability of 95%, and less than 5% for a selection probability of 50%.

Table 1 shows the selection probabilities, in simulation, of the three methods. The selection probability of the LARS algorithm is very good in situations with low residual variance or large sample size: it was at least 83.6% when the R² value was at least 100/n. This approach always outperformed the alternative methods, except for a difference of 0.4% in one situation with the largest residual variance and smallest sample size.

TABLE 1.

Percentage of 500 Simulations in Which the Correct Model Was Identified, in Simulations with Two Binary Exposure Measurements, by Three Different Structured Approaches

Three Exposure Measurements

This hypothetical example considered that prior knowledge provided evidence for either a critical or a sensitive period; the aim being to confirm the correct association using new data. Three measurements were simulated as binary variables as before, with adjacent measurements having correlations of ρ, and the first and last measurements having correlation ρ². The models used to generate the outcome were: an early critical period, an accumulation model, and an early sensitive period model. The simple hypotheses that the LARS algorithm was allowed to choose from were three critical period hypotheses and an accumulation hypothesis. The LARS algorithm was considered to have chosen a simple hypothesis if it first selected the variable encoding that hypothesis, and the covariance test P value for including another variable was not less than 0.05. The LARS algorithm was considered to have identified a compound hypothesis if the first two selected variables encoded that hypothesis and the P value for including the second variable was less than 0.05. While we do not necessarily advocate using P value thresholds to select hypotheses, this allowed comparisons with other approaches. The other two approaches used the F test and Akaike information criterion as before, choosing between an accumulation model, three critical period models and three sensitive period models.

There was some evidence that selection probabilities decreased as the exposure correlation increased (Table 2). The selection probability of the LARS algorithm was very good with low residual variance or large sample size: it was at least 90.2%, and higher than that of other methods, when the R² value was at least 100/n. The only exception to this was in sensitive period simulations with strong exposure correlation, where there was less distinction between accumulation and critical period hypotheses. The alternative methods had better selection probabilities in compound models than simple models; it appears that Akaike information criterion or F test selection is more likely to select compound hypotheses over simple ones, regardless of the true underlying model.

TABLE 2.

Percentage of 500 Simulations in Which the Correct Model Was Identified, in Simulations with Three Binary Exposure Measurements, by Three Different Structured Approaches

Confidence Intervals

We repeated the simulation experiment with three exposure measurements, testing a null model in place of a sensitive period model. In each simulation, we calculated the usual 95% confidence interval for the regression parameter in the hypothesized model with the largest F test P value (coincidentally the model with smallest Akaike information criterion) when compared with the saturated model. We also calculated an adjusted confidence interval based on the covariance test for the lasso (see eAppendix B for details; http://links.lww.com/EDE/A940). Table 3 shows the coverage of these confidence intervals. In the null model, the coverage of the usual confidence intervals was always less than 95%, showing that the F test or Akaike information criterion approach generates bias due to the fact that they consider the largest observed association to be selected at random. However, the adjusted confidence intervals have coverage between 92.2% and 96.2% in the null model, confirming that the covariance test corrects for selection of the variable with greatest association.

TABLE 3.

Percentage of 500 Simulations in Which a 95% Confidence Interval Contained the True Parameter Value, in Simulations with Three Binary Exposure Measurements, Calculated by Two Different Structured Approaches

DISCUSSION

A causal life course association between exposure and outcome cannot be estimated without identifying knowledge of the system being studied.⁷ This can be achieved by assessing prespecified competing hypotheses regarding that system and the life course association. We have described a strategy for this, which involves encoding a set of hypotheses as covariates, and then using the LARS procedure for the lasso to identify the most appropriate covariate subset that accounts for the outcome variation. Variable selection is aided visually with an elbow plot, or guided by a hypothesis test. We showed that, for one example dataset, the LARS procedure identified the same hypotheses as earlier research using a structured approach.⁹ Furthermore, simulation showed, for reasonably large sample sizes, the LARS algorithm, even when combined with a naive P value threshold, effectively identified the correct hypotheses. Alternative methods, based on F tests and Akaike information criterion, did not identify the correct hypotheses as often and were more likely to favor compound hypotheses over simple ones.

Our proposed approach is part of the process toward estimation of causal effects. The set of hypotheses proposed a priori can be chosen using previous knowledge and theory regarding the plausibility of various hypotheses. We have demonstrated techniques for choosing the best-fitting of those hypotheses, which can then be further investigated both for replication and to estimate causal effects. Advantages of the structured approach are that it requires the hypotheses to be carefully specified a priori, and can accommodate complex compound hypotheses that involve interactions. For example, Forsdahl²⁸ argued that a poor standard of living in early years followed by later life prosperity would increase the risk of arteriosclerotic disease. In this example, the highest risk would be seen in those who change exposure between earlier and later time periods. Other hypotheses, such as nonlinear accumulation, can also be investigated. The flexibility of our approach allows many hypotheses even if they are epidemiologically implausible: it is important to triangulate the statistical findings with knowledge about biological and social plausibility. If suggested hypotheses are thought to be “too complex,” our procedure allows for retreat to simpler, more interpretable, hypotheses if necessary. In further investigation, only the identified hypothesis need be considered, allowing precise definition of the causal effect(s) of interest and reducing their number, leading to improved estimation by marginal structural or structural nested models.^7,29

Our approach has the advantage that the selected hypothesis will always be easy to interpret, provided that interpretable hypotheses are proposed a priori. This is in contrast to methods that provide a plot and invite interpretation based on curves on the graph.^3,4 The LARS algorithm has the advantage of choosing the most important hypothesis first: that corresponding to the greatest proportion of the variability in the outcome, while unimportant effects are deselected. Unlike the structured F test approach,⁹ our approach can be employed without using P values. Using the covariance test for the lasso, we calculated confidence intervals with coverage unaffected by the fact that the selected hypothesis offers the best fit to the observed data. The covariance test for the lasso should be used in preference to an F test between simple and compound hypothesized models.

If the association between exposure and outcome is weak, with little variation in the outcome explained by the exposure, then the reliability of structured approaches in identifying the true model is diminished. It is therefore important first to examine the overall amount of variation explained by the exposure, perhaps using the R² value in a saturated model or extreme end of the elbow plot, and second to examine the correlation structure among the exposure measurements. If exposure measurements are highly correlated, it may be impossible to distinguish hypotheses without a large sample size or many repeated measures. Additional study is required to extend this approach to continuous exposures and categorical outcomes, consider measurement error in the exposure, and accommodate possible confounding.

Within the same overall structure, other variable selection methods might be used instead of the lasso. One possibility is the elastic net,³⁰ although this could select more variables than parameters in the saturated model, which would not be an advantage in understanding the hypothesized life course association, as less parsimonious models are less interpretable. The grouped lasso would allow all variables encoding a compound hypothesis to be selected at the same time,³¹ and prevent only one component variable of a compound hypothesis being identified on its own. However, if compound hypotheses are misspecified, for example not all of their components are associated with the outcome, then not grouping variables still allows the important components to be extracted and the hypothesis to be identified.

Our conclusion is that the LARS procedure, implementing the lasso, can select hypotheses from a prespecified set, identifying the optimal hypothesis that offers the greatest consistency with the data. Compound hypotheses can be built from simpler hypotheses in a straightforward way, using a small number of variables. The selected hypothesis and its potential causality may then be more precisely defined and further studied.

ACKNOWLEDGMENTS

The authors are grateful to the MRC Unit for Lifelong Heath and Ageing at University College London for providing the data used as an example.

Supplementary Material

ede-26-719-s001.doc ^{(68.5KB, doc)}