Reducing the complexity of high-dimensional environmental data: an analytical framework using LASSO with considerations of confounding for statistical inference

Abstract

Purpose:

Frameworks for selecting exposures in high-dimensional environmental datasets, while considering confounding, are lacking. We present a two-step approach for exposure selection with subsequent confounder adjustment for statistical inference.

Methods:

We measured cognitive ability in 338 children using the Woodcock-Muñoz General Intellectual Ability (GIA) score, and potential predictive features across several environmental domains. Initially, 111 variables theoretically associated with GIA score were introduced into a Least Absolute Shrinkage and Selection Operator (LASSO) in a 50% feature selection subsample. Effect estimates for selected features were subsequently modeled in linear regressions in a 50% inference (hold out) subsample, initially adjusting for sex and age and subsequently adjusting for covariates selected via directed acyclic graphs (DAGs). All models were adjusted for clustering by school.

Results:

Of the 15 LASSO selected variables, eleven were not associated with GIA score following our inference modeling approach. Four variables were associated with GIA scores, including: serum ferritin adjusted for inflammation (negatively), mother’s IQ (positively), father’s education (positively), and hours per day the child works on homework (positively). Serum ferritin was not in the expected direction.

Conclusions:

Our two-step approach moves high-dimensional feature selection a step further by incorporating DAG-based confounder adjustment for statistical inference.

BACKGROUND

Several conceptual models, like the “total” environment(1, 2) and the exposome(3), integrate different measures of the environment (built, social, natural) to recognize the multiple layers of influence on health. To measure the ”total” environment, scientists collect a large array of variables resulting in high-dimensionality, wherein the number of variables exceeds the number of subjects(4). The wealth of data often collected in environmental epidemiology results in analytical challenges. Environmental epidemiologists need standardized statistical frameworks for analyzing high-dimensional data(5).

Traditional approaches test separate hypotheses for each environmental factor, resulting in inflated type I errors(6, 7). Another ill-advised approach is to add all variables in a single model. This results in wide confidence intervals, unreliable estimates of association(8), spurious inferences(9). This is often referred to as the table 2 fallacy(10). Alternatively, studies of numerous, simultaneous, chemical exposures typically use either a mixture approach or dimension reduction(11–13). However, these methods do not address how to analyze across questionnaire-, geographic- or personal monitor-based exposure measures. Furthermore, the reduction of all variables into subsets of indices or clusters may not be possible, or the created index may be difficult to interpret(14). These considerations merit investigation of alternative or complementary methods.

We propose the use of a machine learning (ML) method, Least Absolute Shrinkage and Selection Operator (LASSO), to select a set of predictive features (variables) thereby reducing the data to a manageable feature set. Once a feature set has been selected, however, confounding may persist between the selected features and outcome of interest. Prosperi and colleagues (2020) provide a thorough discussion of the limitations of ML techniques; especially when not considering confounding after ML feature selection(15). In epidemiology, DAG-based confounder-adjustment is considered the gold-standard(16). DAGs help to minimize over-adjustment, prevent adjusting for causal mediators (variables along the causal pathway), and conceptualize potential biases (17). In short, a DAG-based confounder selection approach helps provide the least confounded effect estimate, an invaluable tool for statistical inference.

Therefore, we present a two-step process for statistical inference in high-dimensional environmental datasets: (i) LASSO feature selection on a 50% feature selection subsample, (ii) post-selection inference testing in a separate 50% subsample using DAG-based confounder adjustment. We demonstrate this technique by using an interpretable, normally distributed IQ-like measure of ~7-year-old children and their families from Montevideo, Uruguay and 111 features representing chemical, dietary, and psychosocial environments.

METHODS

Sample Recruitment

The original sample consisted of 357 children (~7 years of age) and their caregivers (typically mothers) recruited from eleven primary schools in Montevideo, Uruguay, in neighborhoods with suspected toxic metal exposure. Participants without the outcome of interest (n=18) were excluded, alongside one participant with a high number missing values, resulting in a final sample of 338 participants. Parents provided consent and children assented to study procedures. The study protocol was approved by the Pennsylvania State University, the University at Buffalo, the University of the Republic of Uruguay, and the Catholic University of Uruguay. Additional detail is provided elsewhere(18).

Variables Collected

Primary Outcome

Our outcome was the Woodcock-Muñoz general intellectual ability (GIA) score, standardized by child sex and age, and validated for Spanish-speaking populations(19). Trained psychologists administered the test battery at the child’s school.

Caregiver Questionnaire, Home Visit, and 24-Hour Dietary Recall

Caregivers completed questionnaires at the child’s school. Example topics included: child demographics, health history, parental characteristics (i.e., demographics and smoking), and pregnancy information of the study child. Based on household characteristics (i.e. removing shoes before entering, dusting and hours windows open) an environmental exposure index was created(20). Possessions of wealth (i.e. TV, car, computer) were reduced into a single factor, after principal component analysis(21). Caregivers reported the child’s diet aided by trained nutritionists using two 24-hour dietary recalls administered 2 weeks apart. Dietary nutrients were extracted(22–25), and individual foods reduced into processed and nutrient dense food patterns(21, 26). A social worker visited the home and administered the Home Observation for the Measurement of the Environment Inventory (HOME)(27). During the visit, household water samples were analyzed for metal content (iron, lead, manganese, cadmium, and arsenic)(28).

Clinical Measures

During a separate school visit, height and weight were taken by a trained nurse or nutritionist and anthropometric z-scores were calculated (height, weight, and BMI)(29). Visual functioning was measured by placing the child 3 meters from an eye chart, covering one eye at a time. The eye chart contained 12 progressively smaller lines of letters from top to bottom. For each eye, the child read across and down the chart, reading each line until unable to correctly identify the letters. Each eye was scored independently from 0 to 10; 10 indicating that the child correctly read up to line 10. Blood and hair samples were collected at the child’s school by a trained nurse. Blood hemoglobin was measured at the school(22). Serum samples were analyzed for C-reactive protein, zinc, folate, vitamin B12, and ferritin following published procedures(22–25). Blood lead was analyzed at the University of the Republic of Uruguay(24), and hair metals at Pennsylvania State University(30). First morning urine samples were collected at home in previously supplied cups rinsed with 10% HNO₃ and deionized water. Urinary metals (including urinary arsenic and cadmium) were analyzed at Karolinska Institutet, Stockholm(22, 24). Analyses of urinary pesticide metabolites (including urinary creatinine) were completed at the University at Buffalo Analytical Toxicological Laboratory (Director, Dr. James Olson). Details of the laboratory methods are provided in the Online Supplemental Material (https://osf.io/spfte/).

Neighborhood Variables

Census segment characteristics from the Municipality of Montevideo Geographic Services (MMGS) were assigned based on home location, and a neighborhood disadvantage factor was developed(31). The MMGS maintains locations of informal settlements and parks. Distances to the nearest informal settlement and greenspace were calculated using the gDistance function in R(32). A total of 163 Montevideo “ferias” (outdoor markets and food outlets) were geocoded. Using the GISTools R package we calculated the number of ferias within a 1 kilometer radius from the child’s home(33). Normalized difference vegetation index (NDVI) was calculated using a cloud free 3-meter resolution 4-bands satellite image from the Planet Imagery archive(34) (October 15^th, 2018). NDVI was assigned using bilinear interpolation in ArcGIS, based on the location of the child’s home.

Indices and Factors

Administered questions were grouped into child sleep problems, caregivers’ thoughts, and attitudes toward child rearing at home and school, and child’s preparation for school. Online Supplemental Material (https://osf.io/spfte/) explains the creation of these 4 factor variables.

Variable Descriptions

Our final variable set contained 111 variables theoretically associated with child IQ. Supplemental Table 1 contains the source (i.e., questionnaire or laboratory report), number of variables used to create each index, and citations (where appropriate/available) of study variables (https://osf.io/spfte/).

Missing Data

We imputed up to the total analytical sample (n=338), where 329 children had at least one missing data point. We used an unsupervised random forest model to impute missing data with the missForest package in R. missForest outperforms many other imputation techniques(35, 36). Specifications are in Online Supplemental Material (https://osf.io/spfte/).

Least Absolute Shrinkage and Selection Operator (LASSO)

LASSO is appropriate for high-dimensional data and is useful for selecting predictive features. We performed 5-fold cross validated (CV) LASSO in STATA to obtain the optimal value of the tuning parameter λ on a 50% subsample of our data. We further adjusted for our recruitment methods by clustering by school using STATA’s vce(cluster) statement. The LASSO was repeated 100 times with 100 different random seeds for CV fold selection, resulting in 100 final models. This method is often used to increase model robustness(37). For each model, we used the selected value of λ corresponding to the model with the minimum CV error. We recorded the number of times each variable was selected by each LASSO model. Variables consistently selected by LASSO (>50 times) were advanced to the confounder-adjustment stage.

DAG-Based Confounder Adjustment for LASSO selected features

To prevent biased estimates during post-selection inference (38) we used the other 50% inference (hold out) subsample for confounding adjustment. First, we estimated partially adjusted associations between the LASSO selected features and GIA score using separate generalized linear models (GLMs) for each LASSO selected feature adjusted for age, sex, and survey clustering by school. DAGs were subsequently created only for features that maintained an association with GIA score in the partially adjusted models. DAGs were created using expert knowledge to determine a minimal confounding adjustment set in dagitty(39). DAG-based models were then estimated for each LASSO selected feature using the DAG identified confounder adjustment set including age, sex, and clustering by school. All continuous ML-selected variables were standardized so the effect estimate represented the change in GIA score per one standard deviation (SD) change in LASSO selected feature.

RESULTS

Study children were ~7 years of age (mean months=81, SD=6.5, n=338) with slightly more males (55%, n=338). The average number of possessions of wealth was 3 (SD=1.2, n=300), average years of education for the mother was 9 (SD=2.7, n=324) and average age of the mother was 33 years (SD=6.4, n=302).

Fifteen variables were selected (Figure 1): Parent’s familial situation (married vs other), whether the child has ever been hospitalized (Y/N), serum ferritin level (ng/mL), child preparation for school factor, caregiver attitude towards discipline, mother’s IQ, parent reported child’s weight at birth (g), frequency reading to the child, the nutrient dense dietary pattern factor, father’s education – secondary (Y/N), hours per day the child works on homework, urinary λCA, urinary cobalt (g/L), urinary uranium (g/L), and HOME inventory score. Examples of non-selected variables included: blood lead concentration (μg/dL), number of books owned by the child, and parental smoking status.

Figure 1.

List of 20 most frequently selected features for general cognitive abilities in children. Bars represent the number of times a feature was selected across 100 iterations of random 5-fold partitioning. Solid vertical line indicates ≥50 times feature was selected.

LASSO selected features were added to separate GLMs each accounting for sex, age, and clustering by school. Eight variables were not associated with GIA score in these partially adjusted models: Parent’s familial situation (married vs other), whether the child has ever been hospitalized (Y/N), child preparation for school factor, caregiver attitude towards discipline, parent reported child’s weight at birth (g), the nutrient dense dietary pattern factor, urinary λCA, and urinary cobalt (g/L).

In the DAG-based confounder-selection, Table 1 illustrates which confounders were considered and selected for each LASSO selected feature. After DAG-based confounder adjustment, two additional variables were no longer associated with GIA score: urinary uranium (g/L), frequency reading to the child, and HOME inventory score. Four variables were associated with GIA score after DAG-based confounder adjustment: serum ferritin adjusted for inflammation (ng/mL) (β= −3.10 CI=( (−5.16, −1.04)), mother’s IQ (β=4.43 CI=(1.27, 7.59)), father’s education – secondary (β=4.10 CI=(1.29, 6.91)), and hours per day the child works on homework (β=−1.08 CI=(−1.81, −0.36)). Figure 2 and Online Supplemental Table 2 (https://osf.io/spfte/) present estimates of association of the LASSO selected features with GIA scores, in partially (Panel A) and DAG-based adjusted models (Panel B).

Table 1.

Variables considered and selected as confounders using DAGs (rows) across LASSO selected features (columns)

		Exposure Variables
		Serum Ferritin (ng/mL)	Mother’s IQ	Father’s Education (Secondary/ Other)	Urinary Uranium (g/L)	Frequency Reading to Child	HOME Inventory Score	Hours per day Child Works on Homework
Confounders Considered	Maternal Education in Years		✓	✓	✓	✓	✓	✓
	Mother’s IQ		---	✓
	HOME Score	✓			✓		---	✓
	BMI Z-Score
	Serum Zinc (μg/dL)
	Hemoglobin (g/dL)
	Lead (Pb) (μg/dL)	✓			✓
	Fruits and Vegetables Factor	✓
	Dietary Protein (mg/day)
	Dietary Iron (mg/day)				✓
	Dietary B12 (mg/day)
	Dietary Folate (mg/day)
	Urinary Arsenic Metabolites (μg/L)				✓
	Household Crowding (Persons/Rooms)	✓					✓
	Caregiver Support of Child Autonomy			✓		✓	✓	✓
	Neighborhood Disadvantage Factor
	Total Population in Census Segment
	Children <5 in Household (Yes/No)					✓
	Child Sleep Problems Factor							✓
	Times Child Spanked (Past Week)
	Child Preparation for School
	Any Developmental Disability (Yes/No)		✓			✓	✓	✓
	Caregiver Occ. Exposure to Metals		✓
	Covariates
	Age	✓	✓	✓	✓	✓	✓	✓
	Sex	✓	✓	✓	✓	✓	✓	✓

Grey Indicates variable considered as a confounder in DAG for selected feature.

--- Indicates same variable as selected feature.

✓ Indicates a selected confounder based on minimal adjustment set.

Figure 2.

Estimates of association of LASSO selected features and general cognitive ability in children using inference subsample

Note. All continuous LASSO selected features standardized. All models adjusted for age, sex, and clustering by school.

DISCUSSION

The “total” environment requires collection of data across multiple contexts, often resulting in high dimensionality. We presented an analytical framework for analyzing high-dimensional data that integrates both ML and causal epidemiology, recognizing that some variables may be predictive but not causally associated with the outcome or vice versa. LASSO selected 15 out of 111 potential features. Eight LASSO selected features were not associated with GIA score in the 50% inference subsample after partial adjustment. After DAG-based adjustment for confounders three additional variables were not associated with GIA score. The final four variables associated with GIA score were serum ferritin adjusted for inflammation (ng/mL), mother’s IQ, father’s education – secondary, and hours per day the child works on homework. Our methodology can be applied to reduce high-dimensional data, account for post-selection bias, and adjust for confounding. DAG creation and interpretation of findings from this approach requires expert knowledge. To that end, we first discuss our methodology and subsequently use exert knowledge to consider plausibility and potential mechanisms between LASSO selected features and cognitive ability.

Caveats of Our Statistical Framework

Because LASSO feature selection does not adjust for confounding, we suggest a two-step process of LASSO feature selection followed by DAG-based confounding adjustment. We note, however, that our framework might not be appropriate for all scenarios. The specific outcome of interest, size of dataset, and research goals may warrant other approaches. For example, expert knowledge could further exclude variables before LASSO feature selection. Biomarkers that do not accurately reflect the internal dose of an environmental factor might also be excluded. For example, as urinary cadmium is the gold-standard for long-term cadmium exposure(24, 40, 41), and reported relationships between hair cadmium and cognitive performance in children are mixed(42), we might have removed hair cadmium. However, when previous scientific knowledge does not exist, ML feature selection might be useful.

Hold-Out Subsample and DAGs for confounder selection

Eight features were not associated with GIA score in the 50% inference (hold out) subsample after partial adjustment. When data-driven approaches such as LASSO are used to select features, statistical inferences from that same dataset may be incorrect due to overfitting (43). As bootstrapping approaches cannot correct for this issue (44), data splitting is recommended for post-selection inference. Using the same dataset for feature selection and inference in environmental epidemiology may incorrectly result in greater type 1 error.

Similarly, three additional features were no longer associated with GIA score after DAG-based confounder adjustment. While these features may have predictive power and were frequently selected by the LASSO model, confounding was not considered during feature selection. DAGs provide a way to consider relationships between potential confounders, exposure, and outcome; selecting a minimal adjustment set for statistical inference. To neglect adjustment for potential confounders after LASSO feature selection, may have resulted in incorrect statistical inferences. Our two-step method is crucial for statistical inference in high-dimensional data.

Examining Features Associated with GIA score after DAG-based Confounder Adjustment

Four features were associated with child GIA score after DAG-based confounder adjustment: serum ferritin, mother’s IQ, father’s education, and hours per day the child works on homework. First, serum ferritin adjusted for inflammation was inversely associated with GIA score. This is contrary to many published studies on iron deficiency (measured by serum ferritin) and cognitive achievement in children (45). Additional examination of this study sample may be warranted to better understand this finding. Second, maternal IQ was positively associated with child IQ. This finding is highly consistent with prior literature, as maternal IQ is a well-established, positive predictor of child cognitive ability, with IQ heritability between 0.4–0.8 (1.0 indicates all variation is genetically explained) (46). Third, the father’s education was positively associated with child cognitive ability. In the case of feature selection, where if there is a group of variables among which the pairwise correlations are high, then the LASSO tends to arbitrarily select only one variable from the group. This may be an instance of arbitrary selection of father’s education over mother’s education by the LASSO. In our sample, children of fathers with at least secondary education had mothers with approximately two more years of education (M= 9.63, SD= 2.70), compared to fathers without secondary education (M= 7.86, SD= 2.03) on average (t_df=336 = −5.84, p<0.001). Lastly, time spent working on homework was inversely associated with child cognitive ability. This could be due to reverse causality between time spent on homework and GIA score, reflected by difficulty in school. In the literature, the homework problems checklist (HPC) (47) which includes length of time to complete assignments is negatively correlated with GPA among students with ADHD (48). While LASSO selected this feature and DAG confounder adjustment supported the statistical association, expert knowledge helps assess the nature of the relationship.

Variables Not Selected by LASSO or Not Associated with Cognitive Ability

Many variables were not selected despite prior evidence of associations with children’s cognitive performance. Noting the purpose of LASSO feature selection – maximizing prediction – can help interpret these findings. Non-selected variables may be causally associated with the outcome of interest, but not strongly predictive. Measurement error or selection forces could affect selection. For example, child’s birthweight (reported by the mother) may be subject to recall error. Dietary variables may only be predictive if the sample includes children with dietary deficiencies. Specification of a variable in the model may further affect selection. Previously, we demonstrated a non-linear relationship between blood lead and cognitive performance profiles(49). As we did not test non-linearity, this may have affected selection. Lastly, we chose the largest λ within one standard error of the minimum CV error to decrease the number of selected variables. The largest λ at the minimum CV error might have selected additional variables.

Strengths and Limitations of Our Methodology

Our methodology is not “one size fits all”. Specific situations may require additional data cleaning, index creation or changes in ML algorithm specification. Targeted learning, for example, may be a promising approach for causal inference in high-dimensional data(50). The approach outlined in this paper, however, may be a useful guide to identify important predictors and adjust for confounding using a traditional DAG-based framework. The integration of feature selection with causal inference thinking is an important topic, and our methodology provides one framework for such integration.

Our approach has some limitations. While LASSO can handle high multicollinearity, a variable may be left out in favor of another collinear variable. In our example, the selection of father’s education over mother’s education may have been such an instance. Our approach is also time-consuming. The computer processing power needed to run 100 CV models is not insignificant. Furthermore, time and expert knowledge is required to construct each DAG. Also, because our data was cross-sectional, we cannot establish temporality as demonstrated by the variable “time spent on homework”. Lastly, ML is agnostic to measurement error, bias, and interpretation of biomarkers. Further consideration of the relevant exposure window, dose, duration, and frequency might aid in understanding why certain variables are left un-selected by ML.

Conclusion

Our two-step process using LASSO feature selection and consideration of confounding allowed us to estimate non-confounded relationships between numerous variables representing different environmental domains with a well-characterized developmental outcome. Our methodology can be adopted to other high-dimensional environmental datasets.