Abstract
Context
Substantial resources are being devoted to identify candidate genes for complex mental and behavioral disorders through inclusion of environmental exposures following the report of an interaction between the serotonin transporter linked polymorphic region (5-HTTLPR) and stressful life events on an increased risk of major depression.
Objective
To conduct a meta-analysis of the interaction between the serotonin transporter gene and stressful life events on depression using both published data and individual-level original data.
Data Sources
Search of PubMed, EMBASE, and PsycINFO databases through March 2009 yielded 26 studies of which 14 met criteria for the meta-analysis.
Study Selection
Criteria for studies for the meta-analyses included published data on the association between 5-HTTLPR genotype (SS, SL, or LL), number of stressful life events (0, 1, 2, ≥3) or equivalent, and a categorical measure of depression defined by the Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) or the International Statistical Classification of Diseases, 10th Revision (ICD-10) or use of a cut point to define depression from standardized rating scales. To maximize our ability to use a common framework for variable definition, we also requested original data from all studies published prior to 2008 that met inclusion criteria. Of the 14 studies included in the meta-analysis, 10 were also included in a second sex-specific meta-analysis of original individual-level data.
Data Extraction
Logistic regression was used to estimate the effects of the number of short alleles at 5-HTTLPR, the number of stressful life events, and their interaction on depression. Odds ratios (ORs) and 95% confidence intervals (CIs) were calculated separately for each study and then weighted averages of the individual estimates were obtained using random-effects meta-analysis. Both sex-combined and sex-specific meta-analyses were conducted. Of a total of 14 250 participants, 1769 were classified as having depression; 12 481 as not having depression.
Results
In the meta-analysis of published data, the number of stressful life events was significantly associated with depression (OR, 1.41; 95% CI,1.25-1.57). No association was found between 5-HTTLPR genotype and depression in any of the individual studies nor in the weighted average (OR, 1.05; 95% CI, 0.98-1.13) and no interaction effect between genotype and stressful life events on depression was observed (OR, 1.01; 95% CI, 0.94-1.10). Comparable results were found in the sex-specific meta-analysis of individual-level data.
Conclusion
This meta-analysis yielded no evidence that the serotonin transporter genotype alone or in interaction with stressful life events is associated with an elevated risk of depression in men alone, women alone, or in both sexes combined.
The successful statistical identification and independent replication of numerous genetic markers in association studies have confirmed the utility of the genome-wide approach for the detection of genetic markers for complex disorders.1,2 However, recent genome-wide association studies have also indicated that most common genetic risks, at least when studied individually, are modest in magnitude, with relative risks in the range of 1.3 or less.2 This suggests that complex disorders result from the combination of numerous individual genetic and environmental contributors, with the potential for interactions among them. However, there is a lack of consensus regarding whether gene × gene or gene × environment interactions should be examined at the stage of gene detection or only after a gene effect has been clearly identified and replicated.3
Despite progress in risk gene identification for several complex diseases, few disorders have proven as resistant to robust gene finding as psychiatric illnesses. The slow rate of progress in psychiatry and behavioral sciences partly reflects a still-evolving classification system, absence of valid pathognomonic diagnostic markers, and lack of well-defined etiologic pathways.4 Although these disorders have long been assumed to result from some combination of genetic vulnerability and environmental exposure, direct evidence from a specific example has not been forthcoming. Few if any of the genes identified in candidate gene association studies of psychiatric disorders have withstood the test of replication5-7 and to date, genome-wide association studies of psychiatric disorders have also had limited success. In terms of environmental factors, however, stressful life events have been well-established as a risk factor for a range of mental disorders, most commonly major depression.8,9 It is therefore appropriate to consider gene identification studies in the context of this environmental risk factor and in particular the simultaneous consideration of both genetic and environmental agents.
One prominent example has been the study by Caspi and colleagues10 who concluded that, in interaction with stressful life events, genetic variation in the promoter region of the serotonin transporter gene (5-HTTLPR; [OMIM 182138]) plays a role in predisposition to major depression. In the context of the introduction of serotonin-specific reuptake inhibitors in treating depression11 and the well-established link between stressful life events and depression,8 this finding offered a plausible biological link. This result was also striking and potentially paradigm shifting because numerous prior association studies of the same polymorphism (but without examining environmental risk factors or life events) had not consistently shown either a strong or replicated association with depression.12,13
Because of the increasing interest in incorporating environmental risk factors in gene discovery studies, the National Institute of Mental Health convened a workshop in 200614 to review the status of research on gene-environment interaction in mental and behavioral disorders and to provide input on future directions of this work. Because there was a lack of consensus on the status of replication of gene-environment interaction studies, a comprehensive review and meta-analysis of this research was proposed. A subset of the workshop participants and a team of other experts in statistical genetics, epidemiology, and behavior genetics followed up by conducting a comprehensive review of the literature and conceptualizing and implementing the meta-analysis.
METHODS
Study Selection
A search of the PubMed, EMBASE, and PsycINFO databases was conducted through March 2009 using the search terms life stress, life event, depression, depress, serotonin transporter, 5-HTTLPR, interaction, and moderation. Bibliographies of studies were also reviewed. A total of 26 articles published or set for publication were identified through this search strategy (Figure 1). Inclusion criteria for studies for the meta-analyses were published articles that included data on the association between the (1) 5-HTTLPR genotype (S, SL, or LL), (2) number of stressful life events (0, 1, 2, or ≥3) or equivalent, and (3) categorical measure of depression defined by the Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) or International Statistical Classification of Diseases, 10th Revision (ICD-10) or ability to use a cut point to define depression from standardized rating scales. To maximize our ability to use a common framework for variable definition, we also requested original data from all studies that appeared to meet inclusion criteria published prior to 2008.10,15-28 We received original data in a usable format for the meta-analysis from 10 of the 14 study investigators. A total of 14 studies were included in the analysis, 15-20,23,25-27,29-32 10 of which15-20,23,25-27,29,30 were also included in a second sex-specific meta-analysis of original data. Nine other studies did not meet inclusion criteria.33-41
Data Coding
To provide a uniform approach to analysis consistent with the variable definitions of the Caspi et al study,10 the variables of each of the studies were coded as follows: major depression as a dichotomous outcome when based on diagnostic interviews, or standard cut points or above the 85th percentile of the scales’ score distribution when based on dimensional measures; stressful life events (or their equivalent) were coded on an ordinal scale (0, 1, 2, ≥3); and the 5-HTTLPR genotype was coded as 0, 1, or 2 for the number of S alleles. Sex, age, and twin status were also coded. Our coding differs from that of some of the original studies that used a different coding strategy such as dichotomized high and low stress (eg, Taylor et al26) or multiple types of environmental stressors (eg, Eley et al18) rather than the ordinal scale of life events that was included in most studies.42,43
We report 2 meta-analyses. The first includes a total of 14 250 participants from studies published through March 2009 that included data on stressful life events, 5-HTTLPR genotype, and depression in a consistent manner. The second meta-analysis includes sex-specific data on a total of 10 943 participants based on the individual-level data from the subset of the studies from which it was available. Several studies were not included in these meta-analyses due to lack of sufficient data for analysis or differential operationalization of the key study variables, most commonly, the use of a continuous outcome measure for depression, which did not allow for logistic regression as was performed for the included studies.
Analytic Methods
We first examined the association between the number of stressful life events compared with none as the explanatory variable with depression as the response variable. For each number of stressful life events, we calculated an odds ratio (OR) and its 95% confidence interval (CI). Second, we investigated the associations between stressful life events alone (entered as an ordinal variable) and the number of S alleles alone on depression as the response variable. Third, we examined the joint effect of number of stressful life events and number of S alleles on the risk of depression to test whether there is an interaction in their influence on the risk of depression, as indicated by a significant OR for their joint effect on the response variable of depression. Fourth, to determine whether gene-environment correlation exists,we evaluated the association between the proportion of S alleles (dependent variable) and number of stressful life events (independent variable) among those without depression by logistic regression. As another test of gene-environment interaction, we performed the same regression as above in those with depression instead of those without depression and then calculated the difference (δ) in β coefficients and the standard error of δ.
The analyses were conducted first for sexes combined from the published data of 14 studies and then for men and women separately for each of the 10 studies with individual-level data. A random-effects meta-analysis was used to obtain weighted averages across studies. There was minimal change in the coefficients or their standard errors when ethnicity and age were included in the models, and we conservatively included only 1 random member of each twin pair from twin study data. The statistical models for the formal meta-analysis included the actual number of S alleles to approximate the dosage model applied by Caspi et al,10 although the findings were comparable for a recessive model (SS vs SL plus LL) and a dominant model (SS plus SL vs LL).
Logistic regression models were used to test the associations between the explanatory (number of S alleles; number of life events) and response variables (depression). Inclusion of the interaction term tests for deviation from a log-additive model of risk, and may be significant even when terms for main effects are not. To test for heterogeneity in all analyses, we performed χ2 tests of heterogeneity, based on the derived regression β coefficients and their standard errors, assuming them to be normally distributed. Because of potential heterogeneity of the effects of explanatory variables across the individual studies, we used random-effects models to estimate the average effect across studies. Inclusion of an ordinal variable in logistic regression provides a statistical test for trends in the effects of this variable on a dichotomous outcome. We had 80% power to reject the null hypothesis of no interaction between stressful life events and 5-HTTLPR genotype in favor of a positive interaction if the true OR was 1.10 or greater. All statistical analyses were conducted using SAS version 9 (SAS Institute Inc, Cary, North Carolina), Microsoft Excel 2007 (Microsoft, Redmond, Washington), and Stata version SE 9 (StataCorp, College Station, Texas).
RESULTS
Table 1 presents the characteristics of the 14 studies that were included in the meta-analyses. There were a total of 14 250 participants of whom 1769 were classified as having depression, and 12 481 did not have depression. Published data were used from 4 studies and original data were used from 10 studies. There was wide variation in the demographic characteristics (eg, proportion of women, age distribution) and sample sizes, which ranged from 118 to 4175 participants. Most of the participants were white except for a multiethnic sample in one study26 and an Asian sample in another study.30 Genotype distributions were similar among white participants; however, Asian individuals had higher S allele frequency than white individuals. Nine of the studies used a structured interview to assess either DSM-IV or ICD-10 major depression,10,15,17,19,25,27,30-32 whereas the other 5 assessed depressive symptoms via self-rated symptoms scales.16,18,20,23,26 Stressful life events measures were assessed consistently across most of the studies using the Brugha List of Threatening Experiences,42,43 whereas a few studies included measures of somatic illness, unemployment, and social stressors. The final column of Table 1 describes whether the findings replicated those of Caspi et al.10
Table 1.
Source | Total No. of Participants (No. Women) | Age, Mean (Range), y | Genotype of Participants, %a |
Study Design | Siblings/Twins | Life Events Measure | Depression Measure | Findings | ||
---|---|---|---|---|---|---|---|---|---|---|
LL | LS | SS | ||||||||
Caspi et al,10 2003 | 847 (407) | 26 | 31 | 51 | 17 | Prospective cohort | No | Study-specific | Major depression (DSM-IV) | Interaction between the number of stressful life events and the number of 5-HTTLPR S alleles on the risk of depression |
Original data provided by investigators | ||||||||||
Eley et al,18 2004 | 377 (220) | 16 (12-19) | 17 | 54 | 29 | Cross-sectional | 22% Sibling pairs | List of Threat-ening Eventsa | Mood Rating Scale | Replication in female sex only |
Gillespie et al,19 2005 | 1091 (666) | 39 (19-78) | 30 | 48 | 22 | Cross-sectional | Twins | List of Threat-ening Eventsb | Major depression (DSM-IV) | Failed to replicate |
Grabe et al,20 2005 | 976 (673) | 52 (20-79) | 37 | 47 | 16 | Cross-sectional | No | Study specific | Somatic and psycho-logical Distress scale | Replication in female sex only |
Surtees et al,25 2006 | 4175 (1950) | 60 (41-80) | 33 | 49 | 18 | Cross-sectional | No | List of Threat-ening Eventsb | Major depression (DSM-IV) | Failed to replicate |
Wilhelm et al,27 2006 | 127 (85) | 48 (44-52) | 30 | 49 | 21 | Prospective cohort | No | List of Threat-ening Eventsb | Major depression (DSM-IV) | Replication |
Taylor et al,26 2006 | 118 (67) | 21 (18-29) | 25 | 48 | 27 | Cross-sectional | No | Study-specific | Beck Depression Inventory | Partial replication (SS genotype only) |
Chorbov et al, (2007)17 | 247 (247) | 22 (13-23) | 36 | 48 | 16 | Prospective cohort | Twins | Study-specific | MajorDepression (DSM-IV) | Failed to replicate(opposite direction) |
Middeldorp et al,29 2008 | 1154 (785) | 39 (26-65) | 33 | 49 | 19 | Prospective cohort | Twins | Study-specific | Anxiety-Depression Rating Scale | Failed to replicate |
Chipman et al,16 2007 | 2095 (1089) | 23 (20-24) | 33 | 46 | 21 | Cross sectional | No | List of Threat-ening Eventsb | Depression rating scale | Failed to replicate |
Cervilla et al,15 2007 | 737 (529) | 49 (13-75) | 26 | 50 | 24 | Cross-sectional | No | List of Threat-ening Eventsb | Major depression (ICD-10) | Replication |
Data from published articles | ||||||||||
Kim et al,30 2007 | 732 (425) | ≥65 | 13 | 34 | 53 | Prospective cohort | No | List of Threat-ening Eventsb | Clinical depression | Replication |
Power et al,31 2008 | 1421 (831) | ≥65 | 24 | 47 | 29 | Prospective cohort | No | Study-specific | Major depression (MINI) | Failed to replicate |
Laucht et al,32 2009 | 309 (167) | 19 | 34 | 51 | 15 | Cross-sectional | No | Study-specific | Major depression (DSM-IV) | Failed to replicate (opposite direction) |
Abbreviations: DSM-IV, Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition); ICD-10, International Statistical Classification of Diseases, 10th Revision (ICD-10); SCID, structured clinical interview for DSM-IV disorders.
Percentages may not sum to 100 due to rounding.
Table 2 presents the results of the meta-analysis of the relationship between depression and number of stressful life events for each study for both sexes combined (n=12 520). Although the estimates for the individual studies were highly variable, the results of the meta-analysis across all studies reveals a direct association between the number of life events and depression (1 vs 0 stressful life event: OR, 1.31; 95% CI, 0.98-1.75; 2 vs 0 stressful life events: OR, 1.95; 95% CI, 1.33-2.87; and ≥3 vs 0, stressful life events: OR, 3.21; 95% CI, 2.07-4.99). Sex-specific estimates show a similar pattern (eTable 1 available at http://www.jama.com).
Table 2.
Study | Total No. of Participants | No. of Stressful Life Events (vs 0) | ||
---|---|---|---|---|
1 | 2 | ≥3 | ||
Caspi et al,10 2003 | 845 | 1.34 (0.75-2.37) | 1.60 (0.88-2.89) | 3.28 (1.97-5.45) |
Eley et al,18 2004 | 328 | 1.67 (0.98-2.85) | 1.20 (0.62-2.31) | 1.37 (0.68-2.76) |
Gillespie et al,19 2005 | 2253 | 1.50 (1.05-2.13) | 1.35 (0.75-2.43) | 2.78 (1.57-4.94) |
Grabe et al,20 2005 | 999 | 3.85 (2.06-7.20) | 7.44 (3.87-14.30) | 11.27 (6.14-20.68) |
Surtees et al,25 2006 | 4060 | 1.87 (1.28-2.75) | 4.30 (2.97-6.22) | 7.87 (5.55-11.14) |
Wilhelm et al,27 2006 | 127 | 0.45 (0.16-1.24) | 1.32 (0.52-3.39) | 2.04 (0.71-5.84) |
Taylor et al,26 2006 | 110 | 0.51 (0.15-1.78) | 0.29 (0.05-1.53) | 0.63 (0.18-2.21) |
Chipman et al,16 2007 | 1844 | 1.25 (0.91-1.72) | 1.37 (0.96-1.95) | 2.53 (1.84-3.49) |
Cervilla et al,15 2007 | 735 | 0.71 (0.41-1.24) | 1.91 (1.12-3.26) | 2.97 (1.70-5.18) |
Middeldorp et al,29 2007 | 367 | 0.60 (0.29-1.21) | 1.32 (0.61-2.85) | 0.80 (0.22-2.89) |
Chorbov et al,17 2007 | 120 | 1.34 (0.51-3.52) | 3.27 (1.05-10.19) | 11.43 (2.14-61.01) |
Kim et al,30 2007 | 732 | 2.18 (1.13-4.23) | 2.78 (1.40-5.52) | 6.72 (3.08-14.66) |
Alla | 12 520 | 1.31 (0.98-1.75) | 1.95 (1.33-2.87) | 3.21 (2.07-4.99) |
Sex-specific associations are presented in eTable 2. Data are odds ratios (95% confidence intervals).
Figure 2 presents the results of the first logistic regression analyses for sexes combined for 14 studies for the effect of 5-HTTLPR genotype, the effect of the number of stressful life events, and the interaction between genotype and number of life events on risk of depression.
The analysis shows no significant allele frequency difference between those with and without depression in any of the single studies or in the meta-analysis of all studies combined (OR, 1.05; 95% CI, 0.98-1.13; Figure 2A). Thus, the genotype alone did not predict depression.
Similar to the results of Table 2 for the specific number of life events, the meta-analysis shows that the number of stressful life events was significantly associated with depression (OR, 1.40; 95% CI, 1.25-1.57; Figure 2B). Finally, in the analysis of interaction (Figure 2C), the ORs were significant for the studies of Caspi et al10 and Wilhelm et al27 but were not significant for any of the other replication studies. The aggregate estimate did not yield a significant interaction (OR, 1.01; CI, 0.94- 1.10) indicating that 5-HTTLPR genotype did not interact with stressful life events on the risk of depression.
In the analysis by sex, no significant allele frequency difference existed for either sex between those with and without depression in any of the single studies or in the meta-analysis of all studies within either sex or for sexes combined (female sex: OR, 1.07; 95% CI, 0.96-1.18; male sex: OR, 1.00; 95% CI, 0.87-1.14, all participants: OR, 1.04; 95% CI, 0.96-1.13; Figure 3A). Thus, the genotype alone did not predict depression.
However, the number of stressful life events was significantly associated with depression in both sexes in 4 studies,15,16,20,25 and in 2 studies of females (Figure 3B).17,19 The meta-analysis showed a significant association for females alone (OR, 1.39; 95% CI, 1.23-1.58), males alone (OR, 1.43; 95% CI, 1.16-1.75), and for all participants (OR, 1.41; 95% CI, 1.27-1.57). Assuming a normal distribution for the β coefficient (log of the OR), this estimate is statistically significant, with a Z score of 6.35 and a P value of <.001.
Finally, in the analysis of the interaction between genotype and number of life events on risk of depression, the regression coefficient was not significant for females in any of the studies and was nominally significant for males in 2 of the 10 studies.16,27 The meta-analysis showed no interaction effect for either females (OR, 0.95; 95% CI, 0.86-1.04), males (OR, 1.02; 95% CI, 0.86-1.20), or both sexes combined (OR, 0.98; 95% CI, 0.90-1.07). Thus, there was no evidence that the serotonin transporter genotype alone or in interaction with stressful life events is associated with an elevated risk of depression in males alone, females alone, or both sexes combined. The only significant finding across studies was the potent association of stressful life events with the risk of depression.
When examining the studies for the possibility of heterogeneity of effect—consistent with Figure 2, eTable 2, and eTable 3 available at http://www.jama.com)—we found no evidence of heterogeneity for the genotype alone (P=.68); however, we did find a significant degree of heterogeneity for life events alone (P<.001). Heterogeneity of the interaction was also not significant (P>.05), and none of the individual interaction coefficients was significantly deviant from the overall average.
eTable 2 and eTable 3 show the results of another series of meta-analyses that test gene-environment correlation, gene-environment interaction, and whether stratification of those with and without depression by the environmental risk factor (stressful life events) would enhance the power to detect an association between the 5HTTLPR genotype and depression. We divided the samples of those with and without depression into strata defined by the number of stressful life events and examined the difference in the frequency of the S alleles between those with and without depression within each stressful life events stratum. eTable 3 presents these frequencies further stratified by sex.
We first examined the association between allele frequency and number of stressful life events within those without depression, our proxy for testing for gene-environment correlation in the population. No significant relationship was found between genotype and number of life events among those without depression for both sexes combined (eTable 2) or for males or females alone (eTable 3) in most of the individual studies, or in the metaanalysis (β, −0.003; SE, 0.014). Likewise, there was no association between stressful life events and allele frequency within those with depression either in the individual studies, except for the studies of Caspi et al10 (positive regression) and Cervilla et al15 (negative regression), or in the meta-analysis (β, −0.004 ; SE, 0.034). However, the meta-analysis yielded no significant differences in the β coefficients between those with and without depression across studies (β, −0.001; SE, 0.047) and sexes, indicating a lack of gene environment interaction.
In the test of whether stratification by exposure to life events would enhance power to detect an association between depression status and genotype, we found no relationship between number of stressful life events (the exposure level) and allele frequency difference between those with and without depression as indicated by the δ coefficient. Similar findings occurred for females alone and males alone (eTable 3).
In fact, the largest allele frequency difference occurred at 1 life event (δ, 0.045; SE, 0.020), which was of nominal significance, but none of the differences in any of the strata was statistically significant after adjusting for multiple testing. Furthermore, the findings of an increase in the S frequency at 1 life event is clearly at odds with the original findings of Caspi et al10 (eTable 2), for which the allele frequency difference is in the opposite direction (δ, −0.091; SE, 0.069). Therefore, in this case, stratifying the allele frequency test based on the environmental exposure did not provide additional support for a genetic association.
All of the above analyses involving a dosage model or allele frequencies were repeated for genotypes, for which either the SS genotype was compared with the SL plus LL genotypes (recessive model) or the SS plus SL genotypes were compared with the LL genotype (dominant model). None of these analyses provided evidence different from that presented above based on the original models.
COMMENT
The results of this meta-analysis clearly demonstrate that stressful life events have a potent relationship with the risk of depression, an association that has been one of the most widely studied environmental factors for a range of mental disorders.8,9,44 Addition of the serotonin transporter genotype did not improve the prediction of risk of depression beyond that associated with exposure to negative life events. Prior studies45-48 have also raised concerns about validity of the claims of replication of the significant interactive influence of stressful life events on the association between 5-HTTLPR genotype and depression,49 and a recent meta-analysis using a different approach to that of the present study concluded that this association was compatible with chance findings.50
This work highlights several pertinent issues for interpreting reports of replication and for conducting future meta-analyses of genetic association studies. First, it was not possible to conduct a traditional meta-analysis using the standards of randomized clinical trials because few of the studies included sufficient descriptive data to conduct a standard meta-analysis. This explains why a previous report included only 5 studies that dichotomized the exposure.50 To supplement our analysis of published data, we also requested original data from many of the authors in order to classify the data in the same way as those of the original study.10 Second, as highlighted in earlier reviews of this topic45,50 and a recent critical review of life stressors across these studies,51 the samples, study designs, measures, and analyses were highly divergent across studies, thereby limiting the comparability of the studies and their evidence regarding replication. For example, several studies only found significant effects for 1 genotype (SS vs SL plus LL or SS plus SL vs LL) rather than the dose effect reported by Caspi et al21,22,24,26,35,37 or found an interaction effect in the opposite direction from that in the original report (the L allele was associated with stressful life events and depression15,20,24,32,35,41). Third, several studies that did not replicate the overall association between genotype, stressful life events, and depression reported replication based on post hoc subgroup analyses of specific age or sex subgroups. 18,20,24,35,41 Although such subgroup analyses may test the generalizability of the original findings, they do not constitute replications unless their findings occur using the same stratification and directionality as those of the original study.
Strengths of the analysis reported herein include: (1) use of the same variable definitions (ie, the number of S alleles and an ordinal measure of life events) and analytic approach that was used in the original report in order to provide a valid test of replication; (2) inclusion of primary data from numerous studies to supplement the analyses of published data to maximize our ability to use a common framework across studies; and (3) our presentation of extensive analyses of expectations of gene-environment interaction in the absence of main effects, potential gene-environment correlation, and evaluation of whether stratification by an environmental risk factor will enhance power to detect a genotype-depression association.
Limitations of this meta analysis should also be considered in interpreting our findings. First, recoding of the data from some of the studies to use the same analytic framework used by Caspi et al10 may have led to findings that differed from those originally reported.18,26 Second, we could only use individual-level data from 10 of the 14 studies that met inclusion criteria for this meta analysis. However, omission of the other 4 studies would not have biased our conclusions because the 1418 participants in these studies represented only about 10% of the 14 250 total participants included in the meta analysis. Moreover, evaluation of the studies that were published subsequent to 2008 (the cutoff for our analysis of original individual level data) revealed only 1 replication,33 4 nonreplications,31,34,36,39 4 studies with findings in the opposite direction from the original report (eg, L rather than S allele was related to susceptibility),32,37,41,52 and 1 that reported an interaction only after post hoc exclusion of 13% of the participants who were taking psychotropic medications.38 Therefore, review of the aggregate evidence from these studies does not support replication and suggests that the findings of the meta-analysis presented herein were not biased by their noninclusion.
The results of this meta-analysis should not deter investigators from including environmental risk factor information in their studies, once robust marginal gene associations have been identified. Characterization of gene-environment interaction has been most successful for diseases or traits that allow the study of a single gene with a major effect in the context of a relevant environmental exposure of varying magnitude, and also when the environmental exposure has a strong effect.53 For example, the identification of the protective influence of the Δ32 mutation in the CCR5 chemokine receptor against human immunodeficiency virus infection was based on evaluation of unaffected controls with high exposure to the virus.54
On the other hand, few examples of gene-environment interaction exist for modest gene effects or small environmental impacts, most likely due to lack of power to characterize such an interaction.55 Unless the statistical relationship between genotype and environmental exposure on disease risk strongly deviates from a multiplicative (or log-additive) model, the power to detect an interaction will be low, and thus weak positive results should be interpreted carefully.
In the present study, we also considered likely scenarios for an interactive influence between 5-HTTLPR genotype with stressful life events on depression in the absence of evidence for a main effect between this genotype and depression in prior studies.13,56 The most likely explanation for an interaction without a main effect would require a reversal in the direction of the life events-by-genotype association, with the risk of depression increasing with the number of S alleles in the presence of stressful life events and decreasing with the number of S alleles in the absence of life events. Our analysis showed there was no such decrease. Therefore, attempts to rescue an unsuccessful candidate gene disease association, no matter how strong the candidate, by invoking an interaction with a common environmental exposure (such as life events) may be fraught with similar rates of false-positive associations as the original marginal gene association studies themselves.57-59
Despite the lack of valid confirmation of the Caspi et al10 results, the approach to implicate candidate genes that had failed previous direct association studies through inclusion of an environmental exposure has been rapidly embraced, and substantial resources have been devoted to subsequent research.5 The widespread acceptance of these findings is likely to have been in part attributable to the acclaim the original article received,60,61 as well as a field that was eager for a new approach due to the frustrating lack of progress in gene identification for mental disorders despite intensive efforts for more than a decade.62,63 The disciplines of behavioral and social sciences have rapidly adopted this approach as seen by an increasing number of reports that attempt to link candidate genes with a wide range of human behaviors that may not even be under strong genetic influence such as number of sexual partners64 and delinquency.65 A more serious concern, however, is that the findings of this and other nonreplicated genetic associations are now being translated to a range of clinical, legal, research, and social settings such as forensics,66 diagnostic testing,67,68 study participants,67,69 and the general public.70 It is critical that health practitioners and scientists in other disciplines recognize the importance of replication of such findings before they can serve as valid indicators of disease risk or have utility for translation into clinical and public health practice.
Supplementary Material
Acknowledgments
Funding/Support: This work was supported in part by the National Institutes of Health, National Institute of Mental Health, Intramural Research Program, Division of Developmental Translational Research, and Division of Neuroscience and Basic Behavioral Science.
Footnotes
Author Contributions: Dr Lehner had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Risch, Herrell, Lehner, Liang, Eaves, Hoh, Kovacs, Ott, Merikangas.
Acquisition of data: Lehner, Kovacs.
Analysis and interpretation of data: Risch, Herrell, Liang, Eaves, Ott, Merikangas.
Drafting of the manuscript: Risch, Herrell, Lehner, Griem, Merikangas.
Critical revision of the manuscript for important intellectual content: Risch, Herrell, Lehner, Liang, Eaves, Hoh, Griem, Kovacs, Ott, Merikangas.
Statistical analysis: Risch, Herrell, Liang.
Administrative, technical, or material support: Risch, Herrell, Griem, Lehner.
Study supervision: Lehner, Liang, Risch.
Additional Contributions: The contribution of data from the investigators of the original studies included in this meta-analysis is gratefully acknowledged.
Financial Disclosures: None reported.
Disclaimer: The views and opinions expressed in this report are those of the authors and should not be construed to represent the views of any of the sponsoring organizations or agencies or the US government.
Additional Information: Online tables are available at http://www.jama.com.
References
- 1.Risch N, Merikangas K. The future of genetic studies of complex human diseases. Science. 1996;273(5281):1516–1517. doi: 10.1126/science.273.5281.1516. [DOI] [PubMed] [Google Scholar]
- 2.Donnelly P. Progress and challenges in genomewide association studies in humans. Nature Genet. 2008;456(7223):728–731. doi: 10.1038/nature07631. [DOI] [PubMed] [Google Scholar]
- 3.Hunter DJ. Gene-environment interactions in human diseases. Nat Rev Genet. 2005;6(4):287–298. doi: 10.1038/nrg1578. [DOI] [PubMed] [Google Scholar]
- 4.Merikangas KR, Risch N. Will the genomics revolution revolutionize psychiatry? Am J Psychiatry. 2003;160(4):625–635. doi: 10.1176/appi.ajp.160.4.625. [DOI] [PubMed] [Google Scholar]
- 5.Chanock SJ, Manolio T, Boehnke M, et al. NCI-NHGRI Working Group on Replication in Association Studies. Replicating genotype-phenotype associations. Nature. 2007;447(7145):655–660. doi: 10.1038/447655a. [DOI] [PubMed] [Google Scholar]
- 6.Lohmueller KE, Pearce CL, Pike M, Lander ES, Hirschhorn JN. Meta-analysis of genetic association studies supports a contribution of common variants to susceptibility to common disease. Nat Genet. 2003;33(2):177–182. doi: 10.1038/ng1071. [DOI] [PubMed] [Google Scholar]
- 7.Allen NC, Bagade S, McQueen MB, et al. Systematic meta-analyses and field synopsis of genetic association studies in schizophrenia: the SzGene database. Nat Genet. 2008;40(7):827–834. doi: 10.1038/ng.171. [DOI] [PubMed] [Google Scholar]
- 8.Brown GW, Harris TO, Peto J. Life events and psychiatric disorders, II: nature of causal link. Psychol Med. 1973;3(2):159–176. doi: 10.1017/s0033291700048492. [DOI] [PubMed] [Google Scholar]
- 9.Dohrenwend BP, Dohrenwend BS. Social and cultural influences on psychopathology. Annu Rev Psychol. 1974;25:417–452. doi: 10.1146/annurev.ps.25.020174.002221. [DOI] [PubMed] [Google Scholar]
- 10.Caspi A, Sugden K, Moffitt TE, et al. Influence of life stress on depression: moderation by a polymorphism in the 5-HTT gene. Science. 2003;301(5631):386–389. doi: 10.1126/science.1083968. [DOI] [PubMed] [Google Scholar]
- 11.Olfson M, Klerman GL. Trends in the prescription of antidepressants by office-based psychiatrists. Am J Psychiatry. 1993;150(4):571–577. doi: 10.1176/ajp.150.4.571. [DOI] [PubMed] [Google Scholar]
- 12.Anguelova M, Benkelfat C, Turecki G. A systematic review of association studies investigating genes coding for serotonin receptors and the serotonin transporter, II: suicidal behavior. Mol Psychiatry. 2003;8(7):646–653. doi: 10.1038/sj.mp.4001336. [DOI] [PubMed] [Google Scholar]
- 13.Levinson DF. The genetics of depression: a review. Biol Psychiatry. 2006;60(2):84–92. doi: 10.1016/j.biopsych.2005.08.024. [DOI] [PubMed] [Google Scholar]
- 14.National Institute of Mental Health Workshop Research Challenges and Opportunities. Washington, DC: Jun, 2006. Gene X environment interactions and developmental psychopathology; pp. 27–28. [Google Scholar]
- 15.Cervilla JA, Molina E, Rivera M, et al. PREDICT Study Core Group. The risk for depression conferred by stressful life events is modified by variation at the serotonin transporter 5HTTLPR genotype: evidence from the Spanish PREDICT-Gene cohort. Mol Psychiatry. 2007;12(8):748–755. doi: 10.1038/sj.mp.4001981. [DOI] [PubMed] [Google Scholar]
- 16.Chipman P, Jorm AF, Prior M, et al. No interaction between the serotonin transporter polymorphism (5-HTTLPR) and childhood adversity or recent stressful life events on symptoms of depression: results from two community surveys. Am J Med Genet B Neuropsychiatr Genet. 2007;144B(4):561–565. doi: 10.1002/ajmg.b.30480. [DOI] [PubMed] [Google Scholar]
- 17.Chorbov VM, Lobos EA, Todorov AA, Heath AC, Botteron KN, Todd RD. Relationship of 5-HTTLPR genotypes and depression risk in the presence of trauma in a female twin sample. Am J Med Genet B Neuropsychiatr Genet. 2007;144B(6):830–833. doi: 10.1002/ajmg.b.30534. [DOI] [PubMed] [Google Scholar]
- 18.Eley TC, Sugden K, Corsico A, et al. Gene-environment interaction analysis of serotonin system markers with adolescent depression. Mol Psychiatry. 2004;9(10):908–915. doi: 10.1038/sj.mp.4001546. [DOI] [PubMed] [Google Scholar]
- 19.Gillespie NA, Whitfield JB, Williams B, Heath AC, Martin NG. The relationship between stressful life events, the serotonin transporter (5-HTTLPR) genotype and major depression. Psychol Med. 2005;35(1):101–111. doi: 10.1017/s0033291704002727. [DOI] [PubMed] [Google Scholar]
- 20.Grabe HJ, Lange M, Wolff B, et al. Mental and physical distress is modulated by a polymorphism in the 5-HT transporter gene interacting with social stressors and chronic disease burden. Mol Psychiatry. 2005;10(2):220–224. doi: 10.1038/sj.mp.4001555. [DOI] [PubMed] [Google Scholar]
- 21.Kaufman J, Yang B-Z, Douglas-Palumberi H, et al. Social supports and serotonin transporter gene moderate depression in maltreated children. Proc Natl Acad Sci U S A. 2004;101(49):17316–17321. doi: 10.1073/pnas.0404376101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Kendler KS, Kuhn JW, Vittum J, Prescott CA, Riley B. The interaction of stressful life events and a serotonin transporter polymorphism in the prediction of episodes of major depression: a replication. Arch Gen Psychiatry. 2005;62(5):529–535. doi: 10.1001/archpsyc.62.5.529. [DOI] [PubMed] [Google Scholar]
- 23.Middeldorp CM, Cath DC, Beem AL, Willemsen G, Boomsma DI. Life events, anxious depression and personality: a prospective and genetic study. Psychol Med. 2008;38(11):1557–1565. doi: 10.1017/S0033291708002985. [DOI] [PubMed] [Google Scholar]
- 24.Sjöberg R, Nilsson K, Nordquist N, et al. Development of depression: sex and the interaction between environment and a promoter polymorphism of the serotonin transporter gene. Int J Neuropsychopharmacol. 2006;9(4):443–449. doi: 10.1017/S1461145705005936. [DOI] [PubMed] [Google Scholar]
- 25.Surtees PG, Wainwright NW, Willis-Owen SA, Luben R, Day NE, Flint J. Social adversity, the serotonin transporter (5-HTTLPR) polymorphism and major depressive disorder. Biol Psychiatry. 2006;59(3):224–229. doi: 10.1016/j.biopsych.2005.07.014. [DOI] [PubMed] [Google Scholar]
- 26.Taylor SE, Way BM, Welch WT, Hilmert CJ, Lehman BJ, Eisenberger NI. Early family environment, current adversity, the serotonin transporter promoter polymorphism, and depressive symptomatology. Biol Psychiatry. 2006;60(7):671–676. doi: 10.1016/j.biopsych.2006.04.019. [DOI] [PubMed] [Google Scholar]
- 27.Wilhelm K, Mitchell PB, Niven H, et al. Life events, first depression onset and the serotonin transporter gene. Br J Psychiatry. 2006;188:210–215. doi: 10.1192/bjp.bp.105.009522. [DOI] [PubMed] [Google Scholar]
- 28.Zalsman G, Huang YY, Oquendo MA, et al. Association of a triallelic serotonin transporter gene promoter region (5-HTTLPR) polymorphism with stressful life events and severity of depression. Am J Psychiatry. 2006;163(9):1588–1593. doi: 10.1176/ajp.2006.163.9.1588. [DOI] [PubMed] [Google Scholar]
- 29.Middeldorp CM, de Geus EJ, Beem AL, et al. Family based association analyses between the serotonin transporter gene polymorphism (5-HTTLPR) and neuroticism, anxiety and depression. Behav Genet. 2007;37(2):294–301. doi: 10.1007/s10519-006-9139-7. [DOI] [PubMed] [Google Scholar]
- 30.Kim JM, Stewart R, Kim SW, et al. Interactions between life stressors and susceptibility genes (5-HTTLPR and BDNF) on depression in Korean elders. Biol Psychiatry. 2007;62(5):423–428. doi: 10.1016/j.biopsych.2006.11.020. [DOI] [PubMed] [Google Scholar]
- 31.Power T, Stewart R, Ancelin ML, Jaussent I, Malafosse A, Ritchie K. 5-HTTLPR genotype, stressful life events and late-life depression: no evidence of interaction in a French population. Neurobiol Aging. 2008 doi: 10.1016/j.neurobiolaging.2008.06.006. [published online ahead of print July 18, 2008] [DOI] [PubMed] [Google Scholar]
- 32.Laucht M, Treutlein J, Blomeyer D, et al. Interaction between the 5-HTTLPR serotonin transporter polymorphism and environmental adversity for mood and anxiety psychopathology: evidence from a high-risk community sample of young adults. Int J Neuropsychopharmacol. 2009:1–11. doi: 10.1017 /S1461145708009875. [published online ahead of print January 20, 2009] [DOI] [PubMed] [Google Scholar]
- 33.Aguilera M, Arias B, Wichers M, et al. Early adversity and 5-HTT/BDNF genes: new evidence of gene-environment interactions on depressive symptoms in a general population. Psychol Med. 2009:1–8. doi: 10.1017/S0033291709005248. [published online ahead of print February 12, 2009] [DOI] [PubMed] [Google Scholar]
- 34.Araya R, Hu K, Heron J, et al. Effects of stressful life events, maternal depression and 5-HTTLPR genotype on emotional symptoms in pre-adolescent children. Am J Med Genet B Neuropsychiatr Genet. doi: 10.1002/ajmg.b.30888. [published online ahead of print November 14, 2008] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Brummett BH, Boyle SH, Siegler IC, et al. Effects of environmental stress and gender on associations among symptoms of depression and the serotonin transporter gene linked polymorphic region (5-HTTLPR) Behav Genet. 2008;38(1):34–43. doi: 10.1007/s10519-007-9172-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Katsuyama H, Tomita M, Hidaka K, et al. Association between serotonin transporter gene polymorphisms and depressed mood caused by job stress in Japanese workers. Int J Mol Med. 2008;21(4):499–505. [PubMed] [Google Scholar]
- 37.Lazary J, Lazary A, Gonda X, et al. New evidence for the association of the serotonin transporter gene (SLC6A4) haplotypes, threatening life events, and depressive phenotype. Biol Psychiatry. 2008;64(6):498–504. doi: 10.1016/j.biopsych.2008.03.030. [DOI] [PubMed] [Google Scholar]
- 38.Scheid JM, Holzman CB, Jones N, et al. Depressive symptoms in mid-pregnancy, lifetime stressors and the 5-HTTLPR genotype. Genes Brain Behav. 2007;6(5):453–464. doi: 10.1111/j.1601-183X.2006.00272.x. [DOI] [PubMed] [Google Scholar]
- 39.Veletza S, Samakouri M, Emmanouil G, Trypsianis G, Kourmouli N, Livaditis M. Psychological vulner-ability differences in students—carriers or not of the serotonin transporter promoter allele S: effect of adverse experiences. Synapse. 2009;63(3):193–200. doi: 10.1002/syn.20598. [DOI] [PubMed] [Google Scholar]
- 40.Wichers M, Kenis G, Jacobs N, et al. The BDNF Val(66)Met x 5-HTTLPR x child adversity interaction and depressive symptoms: an attempt at replication. Am J Med Genet B Neuropsychiatr Genet. 2008;147B(1):120–123. doi: 10.1002/ajmg.b.30576. [DOI] [PubMed] [Google Scholar]
- 41.Zhang K, Xu Q, Xu Y, et al. The combined effects of the 5-HTTLPR and 5-HTR1A genes modulates the relationship between negative life events and major depressive disorder in a Chinese population. J Affect Disord. 2009;114(1-3):224–231. doi: 10.1016/j.jad.2008.07.012. [DOI] [PubMed] [Google Scholar]
- 42.Brugha T, Bebbington P, Tennant C, Hurry J. The List of Threatening Experiences: a subset of 12 life event categories with considerable long-term contextual threat. Psychol Med. 1985;15(1):189–194. doi: 10.1017/s003329170002105x. [DOI] [PubMed] [Google Scholar]
- 43.Brugha TS, Cragg D. The List of Threatening Experiences: the reliability and validity of a brief life events questionnaire. Acta Psychiatr Scand. 1990;82(1):77–81. doi: 10.1111/j.1600-0447.1990.tb01360.x. [DOI] [PubMed] [Google Scholar]
- 44.Eaton WW. Life events, social supports, and psychiatric symptoms: a re-analysis of the New Haven data. J Health Soc Behav. 1978;19(2):230–234. [PubMed] [Google Scholar]
- 45.Zammit S, Owen MJ. Stressful life events, 5-HTT genotype and risk of depression. Br J Psychiatry. 2006;188:199–201. doi: 10.1192/bjp.bp.105.020644. [DOI] [PubMed] [Google Scholar]
- 46.Eaves LJ. Genotype x environment interaction in psychopathology: fact or artifact? Twin Res Hum Genet. 2006;9(1):1–8. doi: 10.1375/183242706776403073. [DOI] [PubMed] [Google Scholar]
- 47.Joober R, Sengupta S, Schmitz N. Serotonin transporter, stressful life events, and depression severity. Am J Psychiatry. 2007;164(5):829–830. doi: 10.1176/ajp.2007.164.5.829b. author reply 830-831. [DOI] [PubMed] [Google Scholar]
- 48.Wray NR, Coventry WL, James MR, Montgomery GW, Eaves LJ, Martin NG. Use of monozygotic twins to investigate the relationship between 5HTTLPR genotype, depression and stressful life events: an application of Item Response Theory. Novartis Found Symp. 2008;293:48–59. doi: 10.1002/9780470696781.ch4. discussion 59-70. [DOI] [PubMed] [Google Scholar]
- 49.Uher R, McGuffin P. The moderation by the serotonin transporter gene of environmental adversity in the aetiology of mental illness: review and methodological analysis. Mol Psychiatry. 2008;13(2):131–146. doi: 10.1038/sj.mp.4002067. [DOI] [PubMed] [Google Scholar]
- 50.Munafò MR, Durrant C, Lewis G, Flint J. Gene X environment interactions at the serotonin transporter locus. Biol Psychiatry. 2009;65(3):211–219. doi: 10.1016/j.biopsych.2008.06.009. [DOI] [PubMed] [Google Scholar]
- 51.Monroe SM, Reid MW. Gene-environment interactions in depression research: genetic polymorphisms and life-stress polyprocedures. Psychol Sci. 2008;19(10):947–956. doi: 10.1111/j.1467-9280.2008.02181.x. [DOI] [PubMed] [Google Scholar]
- 52.Brummett BH, Muller CL, Collins AL, et al. 5-HTTLPR and gender moderate changes in negative affect responses to tryptophan infusion. Behav Genet. 2008;38(5):476–483. doi: 10.1007/s10519-008-9219-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Ottman R. Gene-environment interaction: definitions and study designs. Prev Med. 1996;25(6):764–770. doi: 10.1006/pmed.1996.0117. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Dean M, Carrington M, Winkler C, et al. Genetic restriction of HIV-1 infection and progression to AIDS by a deletion allele of the CKR5 structural gene. Hemophilia Growth and Development Study, Multicenter AIDS Cohort Study, Multicenter Hemophilia Cohort Study, San Francisco City Cohort, ALIVE Study. Science. 1996;273(5283):1856–1862. doi: 10.1126/science.273.5283.1856. [published correction in Science. 273(5209):1069] [DOI] [PubMed] [Google Scholar]
- 55.Mather K, Jinks J. Biometrical Genetics The Study of Continuous Variation. London, England: Chapman & Hall; 1982. [Google Scholar]
- 56.Lotrich FE, Pollock BG. Meta-analysis of serotonin transporter polymorphisms and affective disorders. Psychiatr Genet. 2004;14(3):121–129. doi: 10.1097/00041444-200409000-00001. [DOI] [PubMed] [Google Scholar]
- 57.Caspi A, McClay J, Moffitt TE, et al. Role of genotype in the cycle of violence in maltreated children. Science. 2002;297(5582):851–854. doi: 10.1126/science.1072290. [DOI] [PubMed] [Google Scholar]
- 58.Caspi A, Moffitt TE, Cannon M, et al. Moderation of the effect of adolescent-onset cannabis use on adult psychosis by a functional polymorphism in the catechol-O-methyltransferase gene: longitudinal evidence of a gene X environment interaction. Biol Psychiatry. 2005;57(10):1117–1127. doi: 10.1016/j.biopsych.2005.01.026. [DOI] [PubMed] [Google Scholar]
- 59.Caspi A, Williams B, Kim-Cohen J, et al. Moderation of breastfeeding effects on the IQ by genetic variation in fatty acid metabolism. Proc Natl Acad Sci U S A. 2007;104(47):18860–18865. doi: 10.1073/pnas.0704292104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Holden C. Behavioral genetics: getting the short end of the allele. Science. 2003;301(5631):291–293. doi: 10.1126/science.301.5631.291a. [DOI] [PubMed] [Google Scholar]
- 61.Breakthrough of the year: the runners-up. Science. 2003;302(5653):2039–2045. [PubMed] [Google Scholar]
- 62.Bearden CE, Reus VI, Freimer NB. Why genetic investigation of psychiatric disorders is so difficult. Curr Opin Genet Dev. 2004;14(3):280–286. doi: 10.1016/j.gde.2004.04.005. [DOI] [PubMed] [Google Scholar]
- 63.Risch N, Botstein D. A manic depressive history. Nat Genet. 1996;12(4):351–353. doi: 10.1038/ng0496-351. [DOI] [PubMed] [Google Scholar]
- 64.Guo G, Tong Y, Xie CW, Lange LA. Dopamine transporter, gender, and number of sexual partners among young adults. Eur J Hum Genet. 2007;15(3):279–287. doi: 10.1038/sj.ejhg.5201763. [DOI] [PubMed] [Google Scholar]
- 65.Guo G, Roettger ME, Shih JC. Contributions of the DAT1 and DRD2 genes to serious and violent delinquency among adolescents and young adults. Hum Genet. 2007;121(1):125–136. doi: 10.1007/s00439-006-0244-8. [DOI] [PubMed] [Google Scholar]
- 66.Bernet W, Vnencak-Jones CL, Farahany N, Montgomery SA. Bad nature, bad nurture, and testimony regarding MAOA and SLC6A4 genotyping at murder trials. J Forensic Sci. 2007;52(6):1362–1371. doi: 10.1111/j.1556-4029.2007.00562.x. [DOI] [PubMed] [Google Scholar]
- 67.Bazeloni E. A question of resilience. [May 14, 2007];New York Times Magazine. 2006 April 30; http://www.nytimes.com/2006/04/30/magazine/30abuse.html.
- 68.Quest Diagnostics. Methods and compositions for predicting compliance with an anti-depressant treatment regimen. 11/010,244. [May 29, 2009];US patent application. http://www.pathwaydx.com/proprietary/novel_biomarker.php?id=5HTTLPR.
- 69.Wilhelm K, Meiser B, Mitchell P, et al. Issues concerning feedback about genetic testing and risk of depression. Br J Psychiatry. 2009;194(5):404–410. doi: 10.1192/bjp.bp.107.047514. [DOI] [PubMed] [Google Scholar]
- 70.Begley S. But I did everything right! [August 15, 2008];Newsweek. 2008 April 25;:40–41. http://www.newsweek.com/id/151758. [PubMed]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.