Using gastrointestinal distress reports to predict youth anxiety risk: Implications for mental health literacy and community care

Abstract

This study investigates the generalizability and predictive validity of associations between gastrointestinal (GI) symptoms and youth anxiety to establish their utility in community mental health decision-making. We analyzed data from youth ages 3 to 21 years in volunteer cohorts collected in Los Angeles (N = 327) and New York City (N = 102), as well as the Healthy Brain Network cohort (N = 1957). Youth GI distress was measured through items taken from the parent-reported Child Behavior Checklist (CBCL). We examined generalizability of GI–anxiety associations across cohorts and anxiety reporters, then evaluated the performance of these models in predicting youth anxiety in holdout data. Consistent with previous work, higher levels of gastrointestinal distress were associated with more parent-reported youth anxiety behaviors in all three cohorts. Models trained on data from the Healthy Brain Network cohort predicted parent-reported and child-reported anxiety behaviors, as well as clinician-evaluated anxiety diagnoses, at above chance levels in holdout data. Models which included GI symptoms often, but not always, outperformed models based on age and sex alone in predicting youth anxiety. Based on the generalizability and predictive validity of GI–anxiety associations investigated here, GI symptoms may be an effective tool for child-facing professionals for identifying children at risk for anxiety (Preprint: https://psyarxiv.com/zgavu/).

1 |. INTRODUCTION

Psychiatric distress is increasingly common, especially among youth (Sakolsky & Birmaher, 2008; Twenge et al., 2019; Yatham et al., 2018). For example, the median age of onset for anxiety disorders is 11 years (Kessler et al., 2005), and early onset mental health problems are associated with more chronic courses of illness (Beesdo-Baum & Knappe, 2012; Godoy et al., 2019). As such, early intervention for mental health issues is a public health concern, prompting several professional bodies to prioritize recognition, referral, and treatment competencies in the domain of youth mental health amongst their members (Bell & McKay, 2004).

Mental health care for youth is dispersed across multiple systems, including family members, primary care providers (PCPs), schools (Luthar, Ebbert, et al., 2020; Luthar, Kumar, et al., 2020; Moon et al., 2017), juvenile justice (Duong et al., 2020; Loyd et al., 2019; Yoder et al., 2017), and child welfare (Gudiño et al., 2012; Shin, 2005). As PCPs are often the initial access point for mental health services, particularly within economically disadvantaged communities, primary care is typically the first line of defense for managing young people’s mental health (Heneghan et al., 2008; O’Brien et al., 2016). Over 25% of presentations to pediatric primary care are for psychosocial distress (Cooper et al., 2006), and more presentations occur for unexplained physical and somatic complaints that might be psychogenic (Rushton et al., 2002). Not surprisingly, universal mental health screening has been recommended for PCPs, as well as teachers, school nurses, and other child-facing professionals (Dowdy et al., 2015; Dvorsky et al., 2014). Nonetheless, uptake of such screening is varied across contexts and is far from universally adopted, even within pediatric primary care. Common concerns from teachers and primary care pediatricians include a lack of training and confidence in providing treatments or making psychiatric referrals based on observed or reported psychological states (Green et al., 2019; Reinke et al., 2011). Moreover, as many children may lack the knowledge/insight to effectively report their mental health symptoms (Muris et al., 2010), screening which relies only on self-reported psychological states may miss youth who present with primarily physical manifestations of distress. Given such constraints, tools that go beyond direct observation and understanding of psychological states are needed to enhance the detection of youth psychosocial distress.

One strategy to address limitations in professional expertise and child self-reports of mental states is to use comorbid or prodromal physical health symptoms in treatment and referral decisions. Mental health disorders are highly comorbid with somatic symptoms (Bodas et al., 2008; Cohen et al., 1998; Koloski et al., 2012; Waters et al., 2013), and gastrointestinal (GI) complaints (e.g., unexplained abdominal pain) in particular are associated with concurrent and future anxiety in youth (Callaghan et al., 2020; Campo et al., 2004; Kim et al., 2019; Saps et al., 2009) and are common (8%–10%) in primary health settings (Starfield et al., 1980). GI distress symptoms may thus be helpful in psychiatric referral processes for youth. Nonetheless, in primary care settings, psychiatric referrals occur in less than 20% of GI distress cases even when a psychogenic root is suspected (Rushton et al., 2002), suggesting there is room to improve the use of physical symptoms for psychiatric referral. Considering that GI symptoms are common and might be more easily reported by youth, as well as observable to parents/PCPs/teachers/other child-facing professionals, routine screening of GI symptoms within primary care and community settings may assist individuals in making accurate treatment or referral decisions for youth for mental health services, bolstering early intervention and prevention efforts.

Previously (Callaghan et al., 2020), we demonstrated an association between parent-reported child GI distress and child anxiety using five questions assessing GI symptoms taken from the Child Behavior Checklist (CBCL; Achenbach & Edelbrock, 1983). In the current study, rather than investigating causal mechanisms behind this relationship, we seek to generalize GI–anxiety associations across samples and anxiety outcomes and evaluate their potential for clinical use. If responses on those items could be used in a predictive framework to identify youth at risk for anxiety, this would highlight an opportunity for primary care and community settings to incorporate reports of GI symptoms into mental health screening and referral decisions. Beyond the knowledge of correlations between mental illness and GI distress (which may already be used in many settings), predictive validity of these associations may enable more efficient and accurate use of this knowledge for decision-making in individual cases. In the present study, we work to establish the generalizability and predictive validity of such associations prerequisite for their use in serving youth mental health needs.

2 |. METHODS

2.1 |. Participants

We analyzed data from three cohorts in this study (see Tables 1–3). Two of these cohorts were collected by our lab in two cities: Los Angeles (LA, N = 327) and New York City (NYC, N = 102). The third was a publicly available dataset based on New York City: Healthy Brain Network (HBN, N = 1957; Alexander et al., 2017; Das et al., 2012). Each cohort included children and adolescents between ages 3 and 21 years. In the LA and NYC cohorts, some youth had experienced caregiving adversity (institutional or foster care). In the HBN cohort, recruitment strategies (community-referred recruitment to increase participation of families who were concerned about the psychiatric health of their children) resulted in a higher proportion of participants meeting clinical diagnoses for mental illness than expected in the general population.

TABLE 1.

Sample information on sex, age, GI symptoms, anxiety symptoms, and median annual household income for all cohorts

Row	HBN	LA	NYC
N	1957	327	102
Sex (M/F)	1238/719	139/188	44/58
Mean age in years (SD, [range])	10.5 (3.7, [5, 21.9])	8.4 (4.1, [2.8, 18.6])	10.8 (3.5, [4.4, 17.6])
Mean GI sum score (SD, [range])	0.866 (1.31, [0, 8])	0.361 (0.77, [0, 4])	0.363 (0.77, [0, 4])
Nausea	0.83/0.15/0.03	0.94/0.06/0.01	0.93/0.07/0
Vomiting	0.93/0.06/0.01	0.99/0.01/0	0.99/0.01/0
Stomachache/cramps	0.71/0.25/0.04	0.85/0.13/0.02	0.85/0.15/0
Constipated	0.8/0.15/0.05	0.9/0.09/0.01	0.89/0.08/0.03
Mean SCARED-P (SD, [range])	14.73 (12.2, [0, 72])	13.71 (11.08, [0, 67])	12.93 (10.98, [0, 56])
Proportion meeting SCARED-P cutoff	0.19	0.15	0.13
Mean SCARED-C (SD, [range])	23.21 (16.13, [0, 80])
Proportion meeting SCARED-C cutoff	0.4
Proportion w/KSADS anxiety diagnosis	0.32
Median annual household income	$90,000–$99,999	$100,001–$150,000	$150,001–$200,000

Abbreviations: GI, gastrointestinal; HBN, Healthy Brain Network; KSADS, Kiddie Schedule for Affective Disorders and Schizophrenia; LA, Los Angeles; NYC, New York City; SCARED, Screen for Child Anxiety Related Disorders.

TABLE 3.

Child ethnicity in each cohort for all participants with available ethnicity data. Questionnaire items assessing child ethnicity differed somewhat across cohorts

Cohort	Ethnicity	N	Pct
HBN	Not Hispanic or Latino	1149	66.19
	Hispanic or Latino	443	25.52
	Decline to specify	116	6.68
	Unknown	28	1.61
LA	Not Hispanic or Latino	281	87.27
	Hispanic or Latino	41	12.73
NYC	Not Hispanic or Latino	37	84.09
	Hispanic or Latino	7	15.91

Abbreviations: HBN, Healthy Brain Network; NYC, New York City; LA, Los Angeles.

Associations between GI symptoms and anxiety were previously published in the LA cohort (Callaghan et al., 2020). Here, we analyzed data from the LA and NYC cohorts before preregistering replication and predictive modeling analyses for the HBN cohort (https://osf.io/687ky). We completed most preregistered analyses as planned and acknowledge several differences in the supplement. Within the HBN dataset, we analyzed both complete-case data (reported in analyses in the main manuscript) and all data using imputation strategies (supplemental) in parallel. Before conducting analyses, we split the HBN dataset into a training set (75% of the participants) and a holdout set (75% of the participants). Henceforth, we refer to “HBN training data” to mean the training set, “HBN holdout data” to mean the holdout set, and “HBN full dataset” to mean all of the available HBN data (both training and holdout).

2.2 |. Child GI symptoms

In all cohorts, we assessed child GI symptomatology using parent reports from the CBCL for ages 1.5–5 and 4–18 years (Achenbach, 1991). We included four items present in both versions so that symptoms could be compared across ages. Using a three-point scale (0 = not true, 1 = somewhat true, 2 = very true) to indicate the degree to which their children were experiencing “physical problems without known medical cause,” parents reported incidence of nausea, stomachaches/cramps, vomiting, and constipation (Supporting Information Methods). The internal consistency of these four items, as measured by Cronbach’s alpha, was 0.63 (95% CI [0.61, 0.66]) in the full HBN cohort, 0.45 (95% CI [0.37, 0.54]) in the LA cohort, and 0.47 (95% CI [0.33, 0.62]) in the NYC cohort. We used sum scores of these four items to calculate each participant’s GI symptom score. While previous work (Callaghan et al., 2020) within the LA cohort included one item from the Revised Child Anxiety and Depression Scale (RCADS; Chorpita et al., 2000) within total GI scores, RCADS data were not available for the HBN and NYC cohorts, so we only used CBCL items for the present study.

2.3 |. Child anxiety symptoms

2.3.1 |. Parent report

We measured anxiety-related behaviors in all cohorts using the Screen for Child Anxiety Related Disorders–Parent version (SCARED-P; Birmaher et al., 1997). Parents reported frequencies of each of 41 behaviors on this measure using a three-point scale (0 = never, 1 = sometimes,2 = often). We removed item #11 (“my child gets stomachaches at school”) to avoid including explicit GI symptoms in our anxiety measure. Scores on this measure were summed (see Table 1). Cronbach’s alpha for the SCARED-P was 0.93 (95% CI [0.92, 0.93]) for the full HBN cohort, 0.92 (95% CI [0.91, 0.94]) for the LA cohort, and 0.92 (95% CI [0.90, 0.94]) for the NYC cohort.

2.3.2 |. Child report

In the HBN cohort only, youth reported on their own anxiety-related behaviors using a child version of the SCARED scale (SCARED-C; Birmaher et al., 1997). We summed anxiety symptom scores on SCARED-C using the same process as with the parent-reported version. Cronbach’s alpha for the SCARED-C was 0.94 (95% CI [0.94, 0.95)).

2.3.3. Consensus clinical diagnoses

In the HBN cohort, youth completed a computerized web-based version of the Kiddie Schedule for Affective Disorders and Schizophrenia–Children’s version (KSADS; Kaufman et al., 1997), and both youth and parents completed semi-structured DSM-5-based psychiatric interviews with a licensed clinician (KSADS-COMP). A clinical team reviewed these materials, as well as relevant materials collected during study participation to arrive at a consensus for diagnoses of DSM-5 disorders. We created a binary variable indicating if each participant was given any one of 13 anxiety-related diagnoses (Supporting Information Methods).

2.4 |. Replication and robustness

To replicate associations between GI symptoms and anxiety previously reported (Callaghan et al., 2020), in each cohort (LA, NYC, and HBN training data) we regressed continuous SCARED-P (parent-reported) scores on GI symptoms, sex, age, and interactions of GI symptoms and sex using the rstanarm package in R (Goodrich et al., 2018). To test the robustness of such associations to different reporters of anxiety symptoms in the HBN training data, we fit a similar linear regression model to continuous SCARED-C (child reported) scores, and a logistic regression to model whether participants were diagnosed through clinician consensus using the KSADS.

2.5 |. Predictive utility

Step 1: Model selection using the HBN training set:

For each anxiety outcome, we selected the best-performing models across several regression model formulations for validation in our holdout sets (see Figure 1 and Step 2). The four model formulations included (1) a summed total GI score plus age and sex regressors, (2) each GI item as a separate regressor, plus age and sex regressors, (3) no GI terms with only age and sex regressors, and (4) an intercept-only model (i.e., predicting the group average level of anxiety for each person, included as a baseline comparison). For each formulation including a GI term (1–2), models contained terms for age and sex, and we also examined whether including interaction terms with age and sex (i.e., GI*age, GI*sex, GI*age*sex) would improve model performance (see Supporting Information Methods). We also used models from formulation 2 to examine which GI items were most predictive of anxiety (Figures S18–S22). From each of the four model formulations, we selected the best-performing model from cross-validation in the training set, then fit the best-performing model variants to the entire training set (Table S5) before validation in the holdout sets (both the HBN holdout set and the combined LA + NYC data).

FIGURE 1.

Flowchart summarizing our procedures and results for measuring the predictive validity of models predicting anxiety from gastrointestinal (GI) symptoms. Step 1: Model selection includes the models that were built on the HBN training dataset. There were four different model formulations built on the outcome measures: (1) GI summed (including age and sex), (2) GI individual items (including age and sex), (3) no GI (just age and sex), (4) intercept only. Within those model formulations, several model variants were tested which included models with no interactions between predictors, as well as models that interacted with GI symptoms, age, sex, or included three-way interactions with GI symptoms age and sex. One model variant was selected for each model formulation and carried through to Step 2. Step 2: Model validation occurred on the holdout data (Healthy Brain Network (HBN) test dataset, as well as the combined Los Angeles (LA) + New York City (NYC) cohorts). The LA and NYC cohorts were combined into a single holdout dataset due to highly similar demographics and to increase the sample size. Some of the models (i.e., those built on the clinical cutoff scores from SCARED-C and SCARED-P within the HBN training data—see boxes surrounded by broken lines) were tested within the HBN holdout data but were not discussed in the main manuscript and are not represented in the colored boxes to the bottom right of the graph. Colored boxes in the bottom right of the figure represent the relative level of evidence for the predictive validity of each model for each of the validation tests (>No-GI: better than a model with no-GI symptoms included; >Baseline: better than baseline threshold formulation, e.g., AUC = 0.5; >Null model: better than chance performance based on a permuted null distribution). The level of evidence represented is relative, rather than absolute, meaning that relative to the other models, those highlighted in green provide the strongest evidence for predictive performance of GI models

Step 2: Model validation with holdout data:

For each selected linear and logistic regression model from Step 1, we examined out-of-sample predictive performance on the HBN holdout set as, well as the combined LA and NYC cohorts (for SCARED-P). Performance was assessed in Step 2 through bootstrapping and permutation testing with q² (proportion of variance predicted, see Supporting Information Methods), log loss, and receiver-operating characteristic curves (ROC, and corresponding area under the curve [AUC]) metrics. Because we made six comparisons with each set of models (all four selected model formulations against the permuted null, plus both GI model formulations against the no GI model), we used a Bonferroni adjusted alpha level of 0.0083 (.05/6) for each permutation test.

2.6 |. Exploratory analyses: GI symptoms as a predictor for broader psychopathology

We also asked whether GI symptoms could function as a predictor of general psychopathology risk within the HBN full dataset. We constructed logistic regression models using summed GI symptoms as well as age and sex covariates to explore associations between GI symptoms and the likelihood of any diagnosis, as well as each respective KSADS DSM-5 diagnosis that occurred in more than 5% of participants. In a separate set of models with only GI sum term (and without age and sex predictors), we examined predictive performance through repeated cross-validation (Figure S30).

3 |. RESULTS

3.1 |. Replicability and robustness

Equivalent regression models fit to all three cohorts were consistent such that on average, higher total GI symptoms were associated with more parent-reported child anxiety symptoms (${\hat{\beta }}_{\mathit{\text{HBN}}}=3.88$, 95% CI [3.37, 4.38], ${\hat{\beta }}_{\mathit{\text{LA}}}=5.02$, 95% CI [3.46, 6.52], ${\hat{\beta }}_{\mathit{\text{NYC}}}=5.49$, 95% CI [2.85, 8.10]) as per the SCARED-P (Figure 2, Table S3). In addition, we found that associations between GI symptoms and anxiety were robust to responder (Figure 3), such that within the HBN training dataset higher GI symptoms were on average associated with higher anxiety behaviors on SCARED-C ($\hat{\beta }=1.79$1.79, 95% CI [1.07, 2.51]) and higher probability of diagnosis on the KSADS ($\hat{\beta }=0.28$, 95% CI [0.19, 0.37]).

FIGURE 2.

Replication of linear associations between gastrointestinal (GI) symptoms and anxiety across the New York City (NYC) (green) and Los Angeles (LA) (blue) cohorts, and Healthy Brain Network (HBN) training data (red). (a) Fitted model predictions (solid lines) plus 95% uncertainty intervals (semi-transparent overlays) for linear regressions fit to each of the three cohorts. Dotted line represents a score of 25, which is the clinical threshold for this measure. Predictions for the NYC and LA cohorts are cut off above a GI sum score of 4 because no participants in these datasets scored over 4 for GI symptoms. Model predictions are displayed for an average participant of age 10.5 years. (b) Posterior distributions for the GI sum score predictor in the linear regression models fit to each cohort (colored distributions). Underneath the colored distributions are the 80% (thick line) and 95% (thin line) uncertainty intervals. The posterior estimates for the GI predictor represent the predicted increase in Screen for Child Anxiety Related Disorders–Parent (SCARED-P) scores associated with a 1 unit increase in GI symptoms for an average participant of age 10.5 years. The dotted line at 0 indicates no estimated average change in SCARED-P scores as a function of GI symptoms

FIGURE 3.

Robustness of associations between parent reports of child gastrointestinal (GI) symptoms and both child-reported and clinician-evaluated anxiety in the Healthy Brain Network (HBN) training set. All panels show model fitted predictions (solid lines) for an average participant of age 10.5 years with a 95% uncertainty interval (semi-transparent overlay). (a) Linear regression with parent-reported anxiety behaviors on Screen for Child Anxiety Related Disorders–Parent (SCARED-P) (orange) and child-reported anxiety behaviors on SCARED-C (green) as the outcomes. Points show individual participant scores on each measure. (b) Logistic regression with clinician-consensus anxiety diagnosis based on the Kiddie Schedule for Affective Disorders and Schizophrenia (KSADS) as the outcome. Points indicate proportions of participants at each possible GI score with diagnoses, and point size indicates the number of participants in a given bin

3.2. Predictive validity

Step 1: Model selection:

Within each of the two model formulations that included GI terms, inclusion of age/sex interaction terms had little impact on model performance. Through cross-validation with the HBN training data, we found that both the best-performing GI summed and GI individual items models predicted anxiety scores on the SCARED-P better than the best-performing No-GI model or intercept-only model (Figure 4). There were not consistent performance differences between the best GI summed model and best GI individual items model. While GI models also tended to perform better than No-GI models during cross-validation with the SCARED-C and KSADS outcomes, they did not do so as consistently (Figures S15–S17).

FIGURE 4.

Cross-validation of linear regression models predicting Screen for Child Anxiety Related Disorders–Parent (SCARED-P) from gastrointestinal (GI) symptoms within the Healthy Brain Network (HBN) training data. (a) For each model formulation (GI summed, GI individual items, No-GI Term, and Intercept Only, represented by the different boxes) the median q², 80% uncertainty interval (thick line), and 95% uncertainty interval (thin line) from all cross-validated folds are plotted for each model variant. (b–d) q², calculated for the model variants selected in Step 1 across all folds (each data point represents a fold). Data points that fall above the line indicate a higher q², for the model represented on the y-axis, whereas those that fall below the solid line indicate higher q², from the model represented on the x-axis. Both the GI summed model (b) and the GI individual items model (c) outperformed the No-GI model, and there were no consistent differences in performance between the two types of GI models (individual items and summed; d)

Step 2: Model validation with holdout data:

For parent-reported anxiety (as per the SCARED-P) in the HBN holdout data, GI summed (q²_median = 0.139, 95% CI [0.036, 0.23]) and GI individual items (q²_median = 0.144, 95% CI [0.034, 0.242]) models both predicted variance over baseline and outperformed the No-GI model (q²_median = 0.005, 95% CI [−0.011, 0.018]) in predicting variance in anxiety (p_summed = 0.0045, p_indiv.items = 0.0049; Figure 5A). For child-reported anxiety, GI summed (q²_median = 0.073, 95% CI [0.005, 0.135]) and GI individual items (q²_median = 0.071, 95% CI [0.004, 0.138]), but not No-GI model formulations (q²_median = 0.045, 95% CI [−0.005, 0.091]) reliably predicted variance in anxiety symptoms over a baseline q² of 0, though neither GI model formulation consistently outperformed the No-GI model (p_summed = 0.0904, p_indiv.items = 0.1467).

FIGURE 5.

Bootstrapped model predictive performance and null permuted distributions on Healthy Brain Network (HBN) holdout data for parent-report Screen for Child Anxiety Related Disorders–Parent (SCARED-P, panel a), child-report (SCARED-C, panel a), and clinician consensus Kiddie Schedule for Affective Disorders and Schizophrenia (KSADS) anxiety diagnoses (panels b–d). Bootstrapped distributions of model performance are shown in color (purple = GI summed (model formulation 1), blue = GI individual items (model formulation 2), red = No-GI (model formulation 3), green = Intercept only (model formulation 4)), and bootstrapped medians (colored points) are compared to the distribution of the best-performing null model (gray). (a) Bootstrapped distributions of q², metrics from resampling of the HBN holdout data for both parent-reported (SCARED-P) and child-reported (SCARED-C) anxiety symptoms. For the parent-reported anxiety, only GI models (GI summed and GI individual items) performed better than the null. For the child-reports, all models except for the intercept-only model tended to perform better than the null, with generally positive q², scores. (b) Receiver-operating characteristic curves for all model formulations for the full HBN holdout dataset. The x-axis represents specificity (proportion of those true diagnoses correctly predicted), and the y-axis represents sensitivity (proportion of true diagnoses correctly predicted). The diagonal black line represents chance. (c and d) Bootstrapped distributions of area under the curve (AUC) (panel C) and log loss metrics (panel d) from resampling of the HBN holdout data for clinician-consensus diagnoses on the KSADS. GI summed, GI individual items, and No-GI models

To assess whether the performance of any models was above chance, anxiety outcomes in the HBN holdout dataset were shuffled 10,000 times to calculate empirical null distributions and compared to the actual performance of each model. For parent-reported anxiety, both the GI sum model and the GI individual item model performed better than the null (p_summed = 0.0001, p_indiv.items = 0.0001), whereas the no-GI model did not (p_no-GI = 0.0351, Figure 5A), suggesting that only models including a GI term could predict parent-reported anxiety at above chance levels. For child-reported anxiety, the GI summed, GI individual items, and No-GI model performed better than the null (p_summed = 0.0001, p_{indiv. items} = 0.0001, p_no-GI = 0.0001), suggesting that models based on age and sex alone, as well as those which also included GI symptoms, predicted variance in child-reported anxiety symptoms at above chance levels.

In terms of classification of anxiety diagnoses within the HBN holdout dataset (KSADS), the GI summed (AUC_median = 0.634, 95% CI [0.577, 0.69]), GI individual items (AUC_median = 0.637, 95% CI [0.579, 0.693]), and no GI (AUC_median = 0.602, 95% CI [0.543, 0.659]) models consistently performed above a baseline AUC of 0.5, displaying some sensitivity and specificity (Figure 5B). In addition, all three models outperformed the null in predicting clinician-consensus diagnoses of anxiety under both AUC (p_summed = 0.0001, p_indiv.items = 0.0001, p_no-GI = 0.0004) and log loss metrics (p_summed = 0.0001, p_indiv.items = 0.0001, p_no-GI = 0.0002, Figure 5C and D). While classification models with a GI term performed numerically better than the No-GI model on average, they did not consistently outperform the No-GI model in either AUC (p_summed = 0.0433, p_indiv.items = 0.0425) or log loss metrics (p_summed = 0.011, p_indiv.items = 0.0298).

Model validation results for the combined LA/NYC holdout data were largely similar to those with the HBN holdout data, such that GI model formulations predicted variance in the SCARED-P continuous outcome over both the null and the no GI model (Figure S24), while performance was variable for classification based on clinical cutoff scores (Figure S25).

3.3 |. Exploratory analyses: GI symptoms as a predictor for broader psychopathology

While covarying for age and sex, GI symptoms were positively associated in the full HBN dataset with several mental health diagnoses, notably in the categories of anxiety, behavioral, elimination, and mood disorders (Figure 6A). Further, GI symptoms were negatively associated with the likelihood of participants having no mental health diagnoses. Additional cross-validation of logistic regression models with summed GI symptoms as the sole predictor (no age or sex predictors) revealed evidence that several diagnoses could be classified above a baseline AUC = 0.5 (Figure 6B).

FIGURE 6.

Exploring gastrointestinal (GI) model performance for predicting a range of mental illness diagnoses (left y-axis). Numbers in parenthesis next to the name of each diagnosis (left y-axis) indicate the proportion of participants in the full Healthy Brain Network (HBN) cohort with that diagnosis. Each outcome is grouped into a diagnostic category (box), with the grey title indicating the diagnostic category (e.g., “behavioral disorders”). The beta estimate is indicated with a colored point and the lines around the point indicate the 95% uncertainty interval. (a) Estimated logistic regression betas for a summed GI symptom term (including age and sex covariate) in models fit to the entire HBN cohort, associated with likelihood of each respective outcome. Estimates are in log-odds. (b) Cross-validated area under the curve (AUC) estimates for each respective disorder from iterative rounds of cross-validation leaving out 25% of the HBN dataset. While the models in (a) included covariates for age and sex, cross-validated models only had a GI predictor, but no terms for age or sex to avoid inflated model performance based on age-related differences in specific psychopathologies. Points represent median AUC, and lines represent 95% uncertainty intervals across rounds of cross-validation

4 |. DISCUSSION

Here we establish generalizability and predictive performance of GI–anxiety associations. We first attempted to replicate associations between GI symptoms and anxiety in several cohorts using different informants (parent, child, clinician) to demonstrate the generalizability of these associations. We then assessed predictive accuracy of GI symptom models in holdout datasets. Across three cohorts and with different informants, we found that GI distress was positively associated with youth anxiety, demonstrating a replicable and robust relationship. These findings are consistent with prior work from other groups within both adult and youth cohorts without enrichment for early life adversity (Haug et al., 2002; Jones et al., 2017; Koloski et al., 2012; Waters et al., 2013). Models trained on one cohort (HBN training set) were also successful in predicting anxiety outcomes above chance levels in two holdout datasets (HBN holdout set and NYC + LA holdout set), although predictive performance was quite variable. Specifically, in the HBN holdout data, models predicted anxiety at above chance levels for all anxiety informants. Further, GI models outperformed equivalent models without GI terms for parent-reported child anxiety. However, models were less successful in predicting child-reported anxiety or clinician-consensus diagnoses beyond what could already be predicted based on age and sex. Similar findings emerged when the combined LA/NYC cohorts were used to test model performance. Together, these findings establish both generalizability and predictive validity of associations between GI symptoms and anxiety.

While most of the present work focused on anxiety, exploratory analyses indicated that GI symptoms were also associated with broader psychopathology within the full HBN cohort. Models with a sole GI term predicted a range of clinician-evaluated psychopathologies above a baseline AUC = 0.5, specifically generalized anxiety disorder, separation anxiety, specific phobias, major depressive disorder, dysthymia, oppositional defiant disorder, and encopresis. While these findings are preliminary and need to be confirmed in separate cohorts, links between GI symptoms and broader psychopathology, mood disorders in particular, are consistent with prior work (Bagayogo et al., 2013; Campo et al., 2004; Haug et al., 2002; Kennis et al., 2019; Okulate et al., 2004). While such findings indicate that these models may lack specificity in their ability to predict anxiety symptoms, they also suggest that somatic symptoms may be useful tools for transdiagnostic referrals or treatment for children’s mental health (McGorry et al., 2018; Weersing et al., 2012), which may in fact be preferred given the high comorbidity in psychiatric presentations (Kessler et al., 1994). Indeed, comorbidity within the HBN cohort was high.

Given the associations between GI distress and anxiety, routine screening for anxiety symptoms within all GI distress presentations by pediatricians, teachers, school nurses, and other child-facing professionals could likely help to increase appropriate referrals (Cunningham et al., 2018). However, because these individuals at initial access points for youth mental healthcare do not always have training, resources, or time to administer such mental health evaluations, using GI symptoms to assess psychopathologcal risks may be useful as a preliminary step. Particularly, because somatic symptoms may be readily observed by teachers and school nurses and are often assessed at primary care visits, increased attention to such symptoms as potential indicators of mental health risk may help improve community monitoring of youth mental health. Moreover, because anxiety symptom screening may miss youth who are exhibiting prodromal or subthreshold anxiety, as well as youth who lack insight to report anxiety symptoms, GI symptoms (either alone or in addition to anxiety screening) may help identify youth who would benefit from psychiatric referral but may be missed by mental distress screening.

Even with increasing awareness towards issues of mental health over time, many key gatekeepers to community youth mental healthcare still express a lack of confidence in their preparedness to identify youth psychopathology and their abilities to provide adequate care (Green et al., 2019; Horwitz et al., 2015; Reinke et al., 2011). Based on the generalizability and predictive validity of GI–anxiety associations investigated here, more easily observable GI complaints might then be one tool for addressing difficulties in identifying children at risk for psychopathology, requiring little extra time and training. Thus, while many calls have been issued for increasing mental health competencies in both pediatrics and in schools, increasing widespread intuitions about relationships between physical and mental health symptoms serves as a relatively simple and low-cost measure. Even small increases in knowledge of the interconnected nature of mental and physical health may serve as an incremental addition to the skills and confidence of child-facing professionals in identifying those at risk for psychopathology and improve community mental healthcare for youth (Green et al., 2019).

A key consideration for these data is the possibility that referring youth with GI distress to psychiatric services might impede an appropriate referral to gastroenterology. Importantly, in this study, GI complaints are presumed nonorganic (questions asked about symptoms with “no known medical cause”). Nevertheless, as psychological comorbidities in GI presentations are common in both organic and nonorganic abdominal pain (Raymer et al., 1984), we suggest that psychiatric referrals may remain appropriate for even organic abdominal pain in youth and encourage concurrent screening for organic causes for GI pain.

4.1 |. Limitations

While the current work demonstrates the prerequisite generalizability of GI–anxiety associations for future development of screening instruments, we regard such models in their current form as valuable predictors of risk, but not stand-alone diagnostic screening tools. Though our models often performed above chance levels in predicting out-of-sample anxiety symptoms, such performance does not yet meet a level sufficient for clinical utility as a diagnostic screener. Our GI model AUC values in the range of 0.6–0.7 for classifying diagnoses and 10%–15% predicted variance for continuous symptoms at best, while reliably above chance, fall below standards for diagnostic screening algorithms (Dobrow et al., 2018). In fact, despite that GI–anxiety associations have been previously reported using similar measures within the LA cohort (Callaghan et al., 2020), GI model prediction of whether youth met clinical threshold on the SCARED-P was not reliably higher than chance in a holdout set of the combined LA + NYC cohorts (Figure S24). Nevertheless, that GI models demonstrate predictive ability above chance provides evidence for potential future uses of such approaches, particularly if more targeted GI symptom questions are used, or in combination with other potentially informative variables (such as other somatic distresses, sleep, early-life history).

A second limitation is that while GI models for parent-reported anxiety reliably outperformed models based on age and sex alone (No-GI models), this was not consistently true for child-reported anxiety or clinician-consensus diagnoses. One explanation for this is that because GI items were also parent-reported, better predictive performance for parent-reported anxiety outcomes may have been driven by parental reporting effects, rather than youth anxiety symptoms themselves. On the other hand, stronger relative performance for GI models for the parent-reported anxiety outcome could be attributable to the fact that parent-reported anxiety scores (SCARED-P) were not strongly associated with age, while child-reported anxiety tended to decrease with age, and clinician-consensus diagnoses tended to increase with age (see Tables S3 and S4). Accordingly, models based on age and sex alone performed somewhat better than the null for child-reported and clinician-diagnosed anxiety (see Figure 5). Consistent with prior work (Carter, 2015; Carter et al., 2011; Crystal et al., 1994), females also had higher average anxiety symptoms in all three cohorts and likelihood of anxiety diagnoses in the HBN training set (see Figure S4). Thus, despite the fact that including interaction terms between GI symptoms and age or sex did not meaningfully improve model performance during cross-validation within the HBN training data (see Figure 4, Figures S15 and S16), age and sex helped in predicting both child-reported and clinician-diagnosed anxiety outcomes.

Relatedly, parent-reported GI items were taken from a screening measure for psychosocial development (CBCL), rather than from an instrument designed to assess GI distress (e.g., ROME IV). Because for each GI symptom, response options were limited to “not true,” “somewhat true,” or “very true,” this measure lacks the precision to measure nuanced differences in GI distress. Even when we summed all four of these GI items to create a scale from 0 to 8, scores above 3 were rare in all cohorts, and no participants scored above 4 in the LA or NYC cohorts (see truncated model fits in Figure 2). In addition, this four-item composite demonstrated poor internal consistency, which may have hampered its use in predictive models. Better performance might be expected in future models where predictors are designed to capture more specific GI distress. Future work could also leverage child-reported GI symptoms and/or biological measures associated with GI distress (e.g., the gut microbiota) to mitigate potential biases in parent reporting of child GI distress and improve prediction of youth anxiety.

Additionally, in the current study, we evaluate the ability of models to predict concurrent anxiety as an outcome, rather than future anxiety symptoms. While we believe there is value in this approach for informing community mental healthcare, work is also needed to evaluate model predictive performance for future anxiety in longitudinal datasets.

Another limitation is that models performed best in predictions of continuous anxiety symptoms in both holdout sets, rather than classification of binary anxiety diagnoses. While such performance does fit with spectrum-based conceptualizations of mental illness (i.e., Research Domain Criteria), future iterations of models that employ more specific assessments of GI distress should continue to monitor predictive performance on continuous and discrete outcomes.

A final concern regards the generalizability of the cohorts assessed here. The LA and NYC cohorts included youth who had experienced institutional or foster care abroad, both of which have been associated with GI distress and elevated anxiety (Bos et al., 2011; Bradford et al., 2012). The HBN cohort employed sampling strategies that lead to elevated rates of psychopathology relative to the population. In all cohorts, distributions of annual family income were well above local (NYC or LA) medians (see Table 1 and Figure S2), and there were more White participants than any other racial group (see Table 2).

TABLE 2.

Child race in each cohort for all participants with available race data. Questionnaire items assessing child race differed somewhat across cohorts. While US federal government agencies define Hispanic origin as an ethnicity (US Census Bureau, 2020), not a race, Hispanic was one of the racial categories for which parents could indicate their children belonged in each cohort

Cohort	Race	N	Pct
HBN	White	863	50.03
	Two or more races	284	16.46
	Black	276	16.00
	Hispanic	190	11.01
	Asian	51	2.96
	Other	31	1.80
	Unknown	14	0.81
	Indian	11	0.64
	Native American	3	0.17
	Alaskan Native	1	0.06
	Native Hawaiian	1	0.06
LA	White	125	43.10
	Asian	82	28.28
	Two or more races	40	13.79
	Black	26	8.97
	Hispanic	15	5.17
	Native American	2	0.69
NYC	White	24	26.67
	Asian	22	24.44
	Black	17	18.89
	Two or more races	12	13.33
	Hispanic	11	12.22
	Other	4	4.44

Abbreviations: HBN, Healthy Brain Network; LA, Los Angeles; NYC, New York City.

An important consideration in relation to the racial and ethnic compositions of the samples included here is that prior work has shown that somatic symptom expression in youth can vary by ethnicity. For example, Latino youth of some ethnocultural backgrounds express more somatic anxiety symptoms relative to European-American youth (Pina & Silverman, 2004; Varela & Hensley-Maloney, 2009; Varela et al., 2008). In our study, we did not find consistent evidence of differences in associations between GI symptoms and anxiety (see Figures S13 and S14) or model performance (see Figures S26–S29) as a function of ethnicity or race. However, the absence of such differences may have been a consequence of the relatively low numbers of non-White, and Hispanic or Latino youth (see Tables 1 and 2), or because the racial and ethnic categorizations analyzed may not have been sufficiently nuanced to detect differences in youth anxiety expression across sociocultural communities (Alegria et al., 2019; Quinones-Camacho, 2018). As with other clinical measures, development of tools assessing youth anxiety risk must consider differences in symptom expression across cultures, as well as how to prevent interactions with existing systems of systemic discrimination that perpetuate further inequity in referral and treatment (Chen et al., 2019; Merikangas et al., 2011; Obermeyer et al., 2019; Yu & Kohane, 2019). To this end, future studies will need to test model performance in cohorts with representation of mental illness, caregiving experience, race, and ethnici background that more closely parallels the populations seen in community settings.

CODE AND DATA AVAILABILITY STATEMENT

Code used for all data preprocessing and analyses is available at https://github.com/pab2163/giAnxiety. Further information on Data Usage Agreements for Healthy Brain Network data can be found at http://fcon_1000.projects.nitrc.org/indi/cmi_healthy_brain_network/Pheno_Access.html.