Predicting Future Risk of Depressive Episode in Adolescents: The Chicago Adolescent Depression Risk Assessment (CADRA)

Abstract

PURPOSE A risk prediction index, similar to those used for other disorders, such as cardiovascular disease, would facilitate depression prevention by identifying those who would benefit most from preventative measures in primary care settings.

METHODS The National Longitudinal Study of Adolescent Health enrolled a representative sample of US adolescents and included a baseline survey in 1995 and a 1-year follow-up survey in 1996 (n = 4,791). We used baseline risk factors (social and cognitive vulnerability and mood) to predict onset of a depressive episode at 1-year follow-up (eg, future risk of episode) and used boosted classification and regression trees to develop a prediction index, The Chicago Adolescent Depression Risk Assessment, suitable for a personal computer or hand-held device. True and false positives and negatives were determined based on concordance and discordance, respectively, between the prediction-category–based index and actual classification-category–based 1-year follow-up outcome. We evaluated the performance of the index for the entire sample and with several depressive episode outcomes using the standard Center for Epidemiologic Studies Depression (CES-D) scale cutoffs.

RESULTS The optimal prediction model (including depressed mood and social vulnerability) was a 20-item model with an area under the receiver operating characteristics curve of 0.80 (95% CI, 0.714–0.870), a sensitivity of 75%, and a specificity of 76.5%. For depressive episode, the positive predictive values in the highest risk group (level 4) was from 13.75% for a depressive episode to 63.57% for CES-D score of greater than 16 (mild to moderate depressed mood or above) at follow-up. Conversely, the negative predictive value of being in the lowest 2 levels (0 or 1) was 99.38% for a depressive episode and 89.19% for a CES-D score of greater than 16.

CONCLUSIONS Our model predicts a depressive episode and other depressive outcomes at 1-year follow-up. Positive and negative predictive values could enable primary care physicians and families to intervene on adolescents at highest risk.

INTRODUCTION

As many as 24% of adolescents will experience a depressive episode by the age of 24 years, with a median of onset age of 15 years.¹ The onset of depression during this critical period may be particularly detrimental to the individual and society as a whole.² Depressive illness, a recurrent life course disorder, is the second most costly disorder in developed countries with more than $100 billion in annual losses and expenditure in the United States alone.³ With full remission rates remaining less than 60% to 70%,⁴ reports in the United States and Europe and by the World Health Organization have called for the development of preventive interventions to reduce the burden of this disorder.^5,6 Depressive disorders in adolescence are the largest single contributor to suicide risk during this period and the third leading cause of death in this age-group.⁷ Even with treatment, adolescents who have experienced one major depressive episode have higher risk of suicide and continued functional impairment.^8,9

Preventive approaches in primary care hold the promise of potentially reducing both the risk of depressive disorder and risk of suicide in this age-group (Figure 1 ▶) by intervening early in the sequence of depression pathogenesis. Risk for depression is derived substantially from demographic, behavioral, interpersonal, and cognitive vulnerability factors from within the adolescent’s social context. Both the American Psychological Association^10,11 and the American Academy of Pediatrics have recommended the physician play a more active role in modifying social risk.^12,13 Pediatricians and family physicians interact closely with adolescents and families within the community settings from which risk is derived and are increasingly be called upon to intervene with screening, referral, and treatment.^10,14–16 A risk prediction index could enable primary care physicians to triage adolescents by risk level to an appropriate level of intervention (eg, face-to-face counseling for those at high risk,¹⁷ and perhaps Internet-based approaches (currently being evaluated by the author) for those at intermediate risk.)^18–20

Figure 1.

Prediction and early intervention model.

The purpose of this study was to develop a valid and practical risk prediction index (eg, future risk of episode), with a sensitivity and specificity of greater than 75%, using baseline vulnerability measures to predict new-onset depressive episodes at 1-year follow-up in US adolescents. We used the National Longitudinal Study of Adolescent Health data set because it is from the most recent longitudinal study of a national probability sample of US adolescents. The other major longitudinal studies either are either based on regional samples^21,22 or do not include younger adolescents,¹ and none is more recent.

METHODS

Sample and Survey Design

Mandated by the US Congress, The National Longitudinal Study of Adolescent Health (Ad Health) was a probability sample of US adolescents to provide information on the on the determinants and contexts of youth health and risk behaviors.²³ Wave 1 included an in-home, face-to-face interview and an in-school survey in 1995 (76.6% enrollment rate; n = 6,504; public-use data set), and wave 2, a follow-up survey, was completed in 1996 (73.5% follow-up; n = 4,791). The surveys were organized by schools, and adolescents were selected by a stratified sampling method within schools. A particular focus in this study was obtaining an accurate understanding of the health of ethnic minorities. Consequently, ethnic minority youth were oversampled.²⁴ Items used in the Ad Health surveys were developed based on multiple sources, and no complete scales were used.²⁵ The present data analysis was approved by the University of Chicago Institutional Review Board and subsequently classified as exempt.

Outcome Variable

We sought to create an outcome that included all those individuals who had new cases of depressive disorder at 1-year follow-up. This outcome variable includes all youth who had 4 or more symptoms of depressive disorder at 1-year follow-up who were not already experiencing this level of symptoms at the baseline interview. The primary outcome variable of depressive episode was constructed using depression-specific items from the Center for Epidemiologic Studies Depression (CES-D) scale (20 items, score 0–60) at wave 2. Youth were considered to have a depressive episode if they met the following criteria: (1) reported experiencing at least 1 core symptom (depressed mood or anhedonia) “most all of the time for (5–7 days for the last week)” and (2) endorsed a similar level of severity for 4 or more other Diagnostic and Statistical Manual of Mental Disorders (Third Edition) criteria symptoms. We included within the depressive disorder outcome all those who may have major depressive disorder (5 or more symptoms) and those with severe minor depression (4 symptoms).^26,27 We chose this cutoff because those with 4 or more symptoms are thought to warrant treatment intervention.²⁸ To ensure that we were predicting only new-onset depressive episodes, we excluded those with a depressive episode at baseline (n = 100 who were present in both the wave-1 and wave-2 assessments).

Independent Variables

All 5,800 variables available at wave 1 were considered for inclusion in the model in a 2-step process. In the first step, 2 investigators (J.G. and B.W.V.) reviewed all available variables for analysis to determine which were either derived from standard instruments relating to depressive episode pathogenesis or were items that appeared to have face validity as potential predictors of future depressive episodes. These factors included sociodemographic characteristics (ethnicity, physical health, age), and interpersonal relations (social skills/problem solving deficits, lack of social support), interpersonal/family relations, and cognitive (pessimistic expectation, avoidant problem solving) and delinquency items.^10,29,30 Current theoretical models of pathogenesis of depressive disorders formed the basis of this review (Figure 1 ▶).^10,30,31 We identified 119 variables by this method. Next, to select the variables from this list with fewest number of missing observations, we screened all variables to identify any with more than 10% of missing observations. We retained 52 variables, which enabled us to include 93.4% of the wave-2 sample in the analysis (Table 1 ▶).

Table 1.

List of Variables Included in Analysis

Vulnerability Item	Response Type
Demographic factors
Age	Years
Sex	Male or female
Mother’s years of education	Years
Father’s years of education	Years
Ethnicity	Self-report
General health
Height	Height in meters
Weight	Weight in kilograms
Body mass index	kg/m2
“Do you usually get enough sleep?”	Yes/no
Interpersonal relations (peers)
“You felt lonely.”	Frequencya
“I feel socially accepted.”	Frequency
Interpersonal relations (family)
“How close do you feel to (Mom’s name)”	Feelingb
“How close do you feel to (Dad’s name)”	Feeling
Closeness to either parent was constructed from higher value of close to mom and close to dad.	Feeling
“How much do you feel that adults care about you?”	Likertc
“Please tell me whether you are a member of any of the following: parent teacher association.”	Yes/no
“How much do you feel that your family pays attention to you?”	Likert
“How much do you feel that people in your family understand you?”	Likert
“How much do you feel that you want to leave home?”	Feeling
Cognitive
“Compared with other people your age, how intelligent are you?”	Comparative scaled
“I like myself just the way I am.”	Likert
“You felt that people disliked you.”	Frequency
“People were unfriendly to you.”	Likert
“You thought your life had been a failure.”	Frequency
“You felt hopeful about the future.”	Frequency
“What is your religion? (yes if respondent answered with a religion)”	Yes/no
“You usually go out of your way to avoid having to deal with problems in your life.”	Likert
Behavioral activation
“During the past week, how many times did you play an active sport, such as baseball, softball, basketball, soccer, swimming, or football?”	Number of times in past week
“How much do you feel that you and your family have fun together?”	Likert
Delinquency
Constructed from answers about various violent acts.	Yes/no
Constructed from: In the past 12 months: How often did you run away from home?	Frequency
Affect regulation
“In the past year how often: have you had trouble relaxing.”	Frequency
“Please tell me how often you have had each of the following conditions in the past 12 months: moodiness”	Frequency
Anxiety
“Over the past week: You felt fearful.”	Frequency
Depressed mood
“You were bothered by things that usually don’t bother you.”	Frequency
“You didn’t feel like eating, your appetite was poor.”	Frequency
“You felt that you could not shake off the blues, even with help from your family and your friends.”	Frequency
“You had trouble keeping your mind on what you were doing.”	Frequency
“You felt depressed.”	Frequency
“ You felt that you were too tired to do things.”	Frequency
“Over the past week: You were happy”	Frequency
“Over the past week: You talked less than usual.”	Frequency
“Over the past week: You enjoyed life.”	Frequency
“Over the past week: You felt sad.”	Frequency
“Over the past week: It was hard to get started doing things.”	Frequency

^a Frequency: 0 = never or rarely, to 3 = almost all the time.

^b Feeling: 1 = not at all, to 5 = very much.

^c Likert: 0=strongly disagree, to 5=strongly agree.

^d Comparative scale: 0 = moderately below average to 6 = extremely above average (1–6).

Analysis

Boosted Classification and Regression Trees Method

We constructed a model using advanced predictive techniques (boosted regression and split-sample validation, training sample [60%], and a test sample [40%].) This method has shown satisfactory results predicting relatively rare outcomes involving complex interaction between independent variables.^32,33 To evaluate the validity of the model, we report the sensitivity and specificity of the prediction model based on an optimal cutoff based on the receiver operator characteristics (ROC) curve. To develop the ROC curve, a risk score was calculated for each individual in the test sample based on the model derived from the training sample. The optimal point of the ROC curve achieves the greatest sensitivity and specificity (defined below). We report the ROC curve for the final model and the sensitivity and specificity of this prediction model at the optimal cutoff point in the test sample.

True and False Positives

True and false positives and negatives were determined based on concordance and discordance, respectively, between the index prediction category and the actual outcome classification category (based on follow-up outcome data). True positives and negatives were determined based on their prediction to be in the same category (eg, depressive episode at 1 year follow-up) and whether they actually were found to be in that category at follow-up. False positives and negatives were defined as cases predicted to be in the incorrect category at follow-up (eg, predicted to be unaffected at follow-up, but actually the individual had a depressive disorder at follow-up, false negative). Sensitivity, the ability to identify cases of depressive disorder, was defined by the following equation: Sensitivity = number of true positives/(true positives + false negatives). Specificity, the ability to classify “noncases” correctly, was defined by the equation: specificity = (true negatives/true negatives + false positives). We then converted these results to a 5-level prediction index for clinical use based on weighed probabilities derived from each individual predictor. We reported these probabilities for both the depressive episode outcome and those derived from standard cutoffs from CES-D scores representing clinically meaningful levels of severity.²⁷ A more complete description of the methods can be found in Supplemental Appendix 1 and 2, available online at http://www.annfammed.org/cgi/content/full/6/6/503/DC1.

Positive and Negative Predictive Value

We selected the most parsimonious model with the maximum predictive power. Because many adolescents may be impaired at varying levels of depressed mood, we also report similar results (positive predictive values) for standard CES-D score cutoffs at greater than 16 (mild depressed mood or greater), greater than 23 (moderate depressed mood or greater), and greater than 28 and (severe depressed mood consistent with major depression).²⁷ To help clinicians evaluate the clinical utility of this assessment tool, we calculated the positive predictive value for each level of risk predicted from baseline data for the test sample for depressive episode outcome and for standard CES-D score cutoffs. Similarly, we report the negative predictive value for the low-risk levels (0 and 1 combined). The positive predictive value represents the prevalence of cases at that level (eg, depressive disorder, CES-D score of greater than 16). The negative predictive value indicates the percentage of noncases of depressive disorder in those at the predefined level of risk. A more complete description of this method can be found in Supplemental Appendix 1 and 2. We used a Stata 9.0 plug-in to implement the boosted regression (StataCorp LP, College Station, Texas).

RESULTS

Study Sample and Comparison of Dropouts

The sample was a diverse, representative sample of US youth, 48% male (mean age 15.7 years), 57% white, 23% African American, 11% Hispanic, 1% American Indian, 3% Asian, and 5% multiracial. The mean family annual income was $47,700 (SD $3,097). With regard to parental education, 14.9% reported less than a high school degree, 53.4% a high school degree, and 31.6% graduated college or more. The study had an overall response rate of 76.8%, with a total sample of n = 6,504 (public-use data set, Sociometrics, Inc, Los Altos, California).²⁴ Nearly three quarters (73.5%) of the wave-1 participants completed the wave-2 survey (n = 4,791).

Results from Boosting Regressions

When all 52 covariates were used, we found that a 6th-order interaction (6-way interactions among the independent variables) was the most optimal in terms of out-of-sample predictions. This model generated a pseudo-R² of 0.159 in the test sample. Based on the predictions of this model, we ranked the covariates in terms of their degree of influence. Table 2 ▶ provides a list of the top 20 covariates and their means and standard deviations for the training and test data sets. When only 20 covariates were used in the model, we found that a 3rd-order interaction was the most optimal in terms of out-of-sample predictions; it generated a pseudo-R² of 0.122 in the test sample. The ROC curve generated using out-of-sample predictions from our 20-variable model with 3rd-order interactions is displayed in Figure 2 ▶. The curve shows discrete changes in probability that result in a stairstep appearance. It produces an area under the curve of 0.80 (95% CI, 0.714–0.870), a sensitivity of 75%, and a specificity of 76.5% at a cutoff point of approximately 0.0177. Each of the top baseline 5 items strongly predicts the CES-D score in a linear regression model (Table 3 ▶).

Table 2.

Top 20 Variables in Final Model in Order of Importance in Predicting a Depressive Episode at 1-Year Follow-up

Rank	Wave-I Covariates (Range)	In Training Sample (n=2,623) Mean (SD)	In Test Sample (n=1,750) Mean (SD)
1	“Over the past week: You were bothered by things that usually don’t bother you.”	0.44 (0.64)	0.44 (0.63)
2	Age (11–20 y)	15.6 (1.55)	15.6 (1.56)
3	“Over the past week: You felt sad.”	0.52 (0.63)	0.51 (0.63)
4	“Over the past week: You felt that you could not shake off the blues, even with help from your family and your friends.”	0.33 (0.64)	0.32 (0.62)
5	“Over the past week: You felt depressed.”	0.44 (0.64)	0.46 (0.68)
6	“Over the past week: You felt fearful.”	0.29 (0.53)	0.30 (0.54)
7	“Over the past week: You felt that you were too tired to do things.”	0.70 (0.71)	0.69 (0.71)
8	Higher value of: “How close do you feel to your [residential father]/[residential mother]?”	4.66 (0.66)	4.63 (0.70)
9	“Over the past week: You felt hopeful about the future.”	1.14 (0.98)	1.16 (0.98)
10	“Over the past week: You didn’t feel like eating, your appetite was poor.”	0.42 (0.66)	0.44 (0.66)
11	“Over the past week: It was hard to get started doing things.”	0.57 (0.65)	0.59 (0.64)
12	Body mass index (13.2–50.7) Constructed from height and weight	22.2 (4.25)	22.4 (4.50)
13	“Over the past week: You had trouble keeping your mind on what you were doing.”	0.78 (0.78)	0.77 (0.78)
14	“Please tell me how often you have had each of the following conditions in the past 12 months: moodiness”	1.29 (0.98)	1.30 (1.00)
15	“Over the past week: People were unfriendly to you.”	0.38 (0.61)	0.39 (0.61)
16	“Over the past week: You enjoyed life.”	2.28 (0.83)	2.29 (0.84)
17	“Over the past week: You talked less than usual.”	0.53 (0.72)	0.54 (0.71)
18	“Over the past week: You thought your life had been a failure.”	0.15 (0.44)	0.18 (0.48)
19	“Compared with other people your age, how intelligent are you?”	3.88 (1.08)	3.91 (1.09)
20	“I feel socially accepted.”	1.89 (0.75)	1.89 (0.73)

Figure 2.

Receiver operator characteristics curve using wave-1 independent variables to predict depressive episode at wave 2 in test data set.

CART=classification and regression trees.

Area under receiver operating characteristics curve = 0.80 (95% confidence interval, 0.71–0.87).

In test sample (N = 1,750; 40%).

Table 3.

Linear Regression Coefficients for the Top 5 Predictors of Depressive Episode at 1-Year Follow-up

Baseline Item	Coefficient	95% CI	95% CI	PValue
“Over the past week: You were bothered by things that usually don’t bother you.”	2.84	2.58	3.10	>.001
Age (11–20 years)	0.50	0.37	0.63	>.001
“Over the past week: You felt sad.”	3.83	3.56	4.09	>.001
“Over the past week: You felt that you could not shake off the blues, even with help from your family and your friends.”	3.18	2.93	3.44	>.001
“Over the past week: You felt depressed.”	3.45	3.22	3.69	>.001

CI=confidence interval.

Results From the Prediction Index

Finally, we developed the 5-level risk prediction scale (0 = very low risk, 4 = very high risk) based on the weighted average predictions of the boosted regressions (details given in the supplemental appendixes). These predictions generated an area under the curve of 0.78 (95% CI, 0.71–0.86) in the out-of-sample test data. Because of the skewed distribution of these weighted average predictions, we developed cutoffs based on the 10th (0.01932), 50th (0.02096), 90th (0.0272), and 95th (0.0309) percentiles of the predictions in the training data set to develop the 0 to 4 prediction scale. We evaluated other cutoffs and obtained similar results. We selected these cutoffs because they resulted in the optimal of the sample predictions. When we used the same cutoffs in our test data set, this scale performed remarkably well in discriminating among patients based on their observed likelihood of developing a depressive episode at a 1-year follow-up (Figure 3 ▶). Using this index, the 1-year risk of developing a depressive episode is level 0 (very low risk, 0%), level 1 (low risk, 1%), level 2 (intermediate risk, 2%), level 3 (high risk, 5%), and level 4 (very high risk, 15%).

Figure 3.

Predictive power of the 5-level prediction index in training and test sample.

Note: The y axis represents the predicted prevalence of depressive episodes at 1-year follow-up among subjects with a given level of score today.

We named this index the Chicago Adolescent Depression Risk Assessment (CADRA). Using the CADRA questionnaire and the Excel spreadsheet application suitable for a hand-held device or a personal computer (Microsoft Corp, Redmond, Washington), we calculated the baseline risk for each participant in the baseline survey and placed them in 5 levels of risk; the prediction index performed well for each depressive outcome examined (Table 4 ▶). For depressive episodes, the positive predictive value for level 4 ranged from 15.0% for a depressive episode to 60.9% for a CES-D score of greater than 16 (mild to moderate depressed mood or above) at follow-up. Conversely, the negative predictive value of being in the lowest 2 levels (0 or 1) was 99.3% for a depressive episode and 89.4% for a CES-D score of greater than 16.

Table 4.

Positive and Negative Predictive Values for Depressive Episode and Standard CES-D Cutoff Scores

Risk Level	Depressive Episode (≥4 Symptoms)	CES-D Score >28a	CES-D Score >23b	CES-D Score >16c
Positive predictive value
4	15.5	23.6	30.9	60.9
3	6.3	7.3	9.1	53.6
2	2.0	3.6	6.4	33.1
≤1	0.7	0.5	1.0	10.6
Negative predictive value <1	99.3	99.5	99.1	89.4
Total sample prevalence	2.79	3.6	5.5	25.1

CES-D=Center for Epidemiologic Studies–Depression scale.

^a Severe depressed mood.

^b Moderate depressed mood or greater.

^c Mild depressed mood or greater.

Supplementary Analyses to Evaluate Effects of Missing Data

Individuals missing from the analyses (primarily because they did not participate in the wave-2 survey) differed in some respects from those who were available for analysis. Nonresponders to the wave-2 survey were more likely to be African American, have lower income and lower parental education, and be slightly more depressed (CES-D score 11.2 vs 10.7, P >.011). The baseline mean values of the top 5 predictors of depressive episode at wave 2 differed by being slightly higher (more depressed) in those missing at wave 2 than those who were present and who were incorporated in these analyses.

DISCUSSION

This analysis shows the feasibility of developing risk prediction indexes for depressive episodes in adolescents. With a 20-item index (CADRA), we were able to achieve a greater than 75% sensitivity and specificity in predicting depressive episodes. CADRA also had excellent positive and negative predictive values for depressive episodes and varying levels of depressed mood (based on standard CES-D score cutoffs) at follow-up. Items found to be predictive included those related to mood, adverse events, and social vulnerability. These findings are consistent with current models that describe the pathogenesis of depression, suggesting that interactions between depressed mood, adverse events, and social vulnerability precipitate the onset of the disorder.^10,30 Similarly, the high levels of interaction in the model are consistent with current theoretical models of causation.^10,30

We are not aware of any current prediction indexes for depressive episodes intended for use in community or primary care settings for either adults or children. The sensitivity and specificity (92% and 62%) attained in this analysis are consistent with those of other prediction indices developed for adult-onset depression in much more homogenous samples with specific risk profiles (traumatic brain injury) and higher prevalence rates.³⁴ In a study that evaluated the predicted value of multiple standard psychological distress questionnaires in predicting future depression in breast cancer patients, the best performance had a sensitivity and specificity of 76.9% and 78.8%, respectively, without a validation sample.³⁵ Results of our study are also comparable with many other prediction rules and screening procedures currently used in primary care. For example the Framingham Risk Score has a sensitivity and specificity of 74.3% and an area under the ROC curve of 74.4,³⁶ and the conventional Papanicolaou smear has a sensitivity and specificity of 76% and 68%, respectively.³⁷ The Downtown fall risk index has a sensitivity ranging from 81% to 95% and specificity of 35% to 40%.³⁸

The representative nature of the sample, the longitudinal design, the availability of multiple variables relating to established theoretical models of depression vulnerability and protection, and the boosted regression and split sample modeling are the key strengths of this study. There was a missing-data bias toward lower income and a greater likelihood of being ethnic minority and being slightly more depressed at baseline. The oversampling of ethnic minority groups, however, resulted in proportional overrepresentation in the sample. Although this data set is a decade old, and individuals missing from wave 2 tended to be more depressed than those who were analyzed, we believe fundamental relations between established vulnerability factors and depressive episodes are unlikely to change with time or differ meaningfully between those who were missing and those who were included in the analyses.^29,39 Poor attachment to parents, low peer social acceptance, and baseline depressed mood or poor affect regulation have been identified as predictors of depressive disorders in multiple samples across historical periods and birth cohorts^40,41 and in non-US samples^42–44 and are the basis for current working models of depressive disorder onset.^11,30,31 Although there has been a decline in substance abuse and other delinquent behaviors and a rise in the prevalence of adolescents living in nontraditional family structures, we do not believe these issues affected the risk factor of disorder relationships.⁴⁵ This instrument will require validation and adaptation to hand-held device application (currently in progress in primary care population), and particular attention should be paid in this process to the experiences of ethnic minority youth.

As pediatricians and family physicians increasingly focus on optimizing development of prediction indices across all domains, including socioemotional health, such indices could play a critical role in preventing depressive episodes. Screening for mental disorders in youth is gaining increasing social acceptance, and the feasibility of screening for depressive episodes has been shown in primary care settings.^14,15 Screening for depressive episodes will identify many mildly to moderately distressed youth for whom the family and physician would like to evaluate future risk of depression to appraise the need for early intervention (as is currently done with cardiovascular disease and breast cancer with paper indexes or hand-held devices). To illustrate, individuals at intermediate to highest risk for a depressive episode (level 4) in the upcoming year may be considered for face-to-face counseling for the adolescent and parents. Those at intermediate risk, levels 2 and 3, might be engaged in Internet-based learning programs based on cognitive behavioral interventions that have been evaluated in randomized trials in adults⁴⁶ now being used in the United Kingdom⁴⁷ and currently being tested in primary care here in United States in adolescents by the authors.¹⁸ Practical and reliable risk prediction for depression will increase awareness for the risk of depression and facilitate decisions regarding care for at-risk adolescents and their families.

Physicians and parents can access and use the risk index program at at http://catchit-public.bsd.uchicago.edu.”