Abstract
Background
Intravenous ketamine has shown rapid antidepressant effects in early trials, making it a potentially attractive candidate for depressed patients at imminent risk of suicide. The Implicit Association Test (IAT), a performance-based measure of association between two concepts, may have utility in suicide assessment.
Methods
Twenty-six patients with treatment-resistant depression were assessed for suicidality 2 hours prior to, and 24 hours following, a single subanesthetic dose of intravenous ketamine using the suicidality item of the Montgomery-Asberg Depression Rating Scale (MADRS-SI). Ten patients also completed IATs assessing implicit suicidal associations at comparable time points. In a second study, 9 patients received thrice-weekly ketamine infusions over a 12-day period.
Results
24-hours after a single infusion, MADRS-SI scores were reduced by an average of 2.08 points on a 0–6 scale (p<.001; d=1.37), and 81% of patients received a rating of 0 or 1 post-infusion. Implicit associations between self- and escape-related words were also reduced following ketamine (p=.003; d=1.36), with reductions correlated across implicit and explicit measures. MADRS-SI reductions were sustained for 12 days by repeated-dose ketamine (2.9-point mean reduction; p<.001; d=2.42).
Conclusions
These preliminary findings support the premise that ketamine has rapid beneficial effects on suicidal cognition and warrants further study.
Keywords: ketamine, suicide, implicit association test
INTRODUCTION
Current treatment options for severe mood disorders are limited by the slow time course of change in suicidal thoughts. For instance, in major depressive disorder (MDD) patients receiving thrice-weekly electroconvulsive therapy, suicidal thoughts persisted in 62% of patients after 1 week of treatment and 39% after 2 weeks (1). Conventional antidepressant treatment produced slower and less robust response in elderly MDD patients with moderate-to-high suicide risk than in non-suicidal patients (2).
Treatment of acute suicidality is further constrained by inaccuracies in patients’ explicit reports of suicidal thoughts (3, 4). The Implicit Association Test (IAT)(5) may be useful as a behavioral measure of suicidal cognition, as the task is reliable (6) and resistant to attempts to intentionally control its outcome (7). Furthermore, when socially “taboo” cognitions are assessed (e.g., prejudicial attitudes), the IAT is a superior predictor of future behavior relative to explicit measures (8). IAT variants assessing suicide- and self-injury-related cognition have shown promise in discriminating between self-injurious and non-injurious adolescents (9), suicidal and non-suicidal adolescents (3), and adult suicide attempters and non-attempters presenting to a psychiatric emergency department (10).
Early evidence suggests that a single subanesthetic dose of intravenous (IV) ketamine, a glutamate-modulating agent, acutely reduces depressive symptoms in approximately 70% of MDD patients 24 hours after infusion (11–13). We tested ketamine’s impact on suicidal cognition in a sample of adults with treatment-resistant depression (TRD). We hypothesized that ketamine would yield rapid, correlated reductions in explicit and implicit suicidal indices. Furthermore, we expected that rapid initial reductions in explicit suicidality would be sustained through repeated ketamine infusions.
MATERIALS AND METHODS
Twenty-six TRD patients were recruited via media advertisement or clinician referral. Treatment-resistance was defined as two or more failed, adequate antidepressant trials in the current episode, as determined by the Antidepressant Treatment History Form (14). DSM-IV-TR diagnoses of MDD were established by SCID-I/P interview. Eligible participants had moderate-to-severe depression (Inventory of Depressive Symptomatology score ≥32)(15); were psychotropic medication-free for ≥2 weeks prior to infusion (4 weeks for fluoxetine); free of substance abuse/dependence for ≥6 months; denied lifetime use of ketamine and PCP; had no lifetime history of psychotic disorder, mania or hypomania; and had no clinically unstable medical or neurological conditions. Patients whom research team psychiatrists deemed unsafe for study participation due to highly active suicidality were excluded.
Patients were admitted to a private hospital room for a 28-hour period for racemic ketamine hydrochloride infusion (0.5 mg/kg diluted in saline, administered over 40 minutes by IV pump(12)) and cardiorespiratory monitoring. Patients were assessed for depressive symptoms 150 minutes prior to, and 24-hours following, infusion using the Montgomery-Asberg Depression Rating Scale (MADRS), a 10-item clinician-administered measure that includes a single suicidality item (Table 1)(16). MADRS raters held graduate degrees and achieved high inter-rater reliability both during training and when co-rating a random sample of videotaped study interviews (ICC’s ≥ 0.96). The timeframe for post-infusion MADRS was modified to reflect the period since last assessment. Ketamine’s antidepressant effects in this sample (not including suicidality analysis) have been reported previously (13).
Table 1.
Montgomery-Asberg Depression Rating Scale scoring guidelines for item 10: “Suicidal Thoughts”
| Score | Description |
|---|---|
| 0 | “Enjoys life or takes it as it comes.” |
| 1 | [none provided] |
| 2 | “Weary of life. Only fleeting suicidal thoughts.” |
| 3 | [none provided] |
| 4 | “Probably better off dead. Suicidal thoughts are common, and suicide is considered as a possible solution, but without specific plans or intention.” |
| 5 | [none provided] |
| 6 | “Explicit plans for suicide when there is an opportunity. Active preparation for suicide.” |
A subset of patients (n=12)1 completed the IAT and the 21-item self-report Beck Scale for Suicidal Ideation (BSI)(17) at baseline. Ten patients repeated these measures 24-hours post-infusion2 (Supplement 1).
A distinct subset of single-dose ketamine responders (n=10)3 enrolled in a subsequent study of repeated-dose IV ketamine. Detailed methods, tolerability, and antidepressant effects will be reported separately (18). Following a Day 1 infusion identical to that described above, patients were assessed for 24-hour antidepressant response (MADRS ≤ 50% of baseline score). Responders (9/10 participants) then received up to 5 additional infusions (Days 3,5,8,10,12) identical to the first, except that patients were assessed and discharged 4-hours post-infusion.
Table 2 presents clinical and demographic characteristics of the three samples. All participants signed informed consent. The Mount Sinai School of Medicine Institutional Review Board approved procedures.
Table 2.
Baseline descriptive and clinical characteristics of full sample treated with single infusion intravenous ketamine; subsample who subsequently completed the repeated infusions study; and IAT subsample who completed additional implicit and explicit measures of suicidality at baseline
| Single Infusion Sample (n=26)† | IAT Subsample (n=12) | Repeated Infusions Subsample (n=10) | |
|---|---|---|---|
| Age, mean (SD), y | 48.2 (11.8) | 50.1 (10.3) | 51.4 (14.6) |
| Female, No. (%) | 10 (39%) | 5 (42%) | 5 (50%) |
| Non-Hispanic Caucasian, No. (%) | 18 (69%) | 8 (67%) | 7 (70%) |
| IQ, mean (SD) | 115.0 (10.3) | 118.7 (10.2) | 115.7 (11.8) |
| Median household annual income | $15,000–24,999 | $10,000–14,999 | $15,000–24,999 |
| Time Since Illness Onset, mean (SD), y | 29.3 (13.3) | 29.7 (10.5) | 29.6 (13.0) |
| Age-of-Onset, mean (SD), y | 18.5 (12.2) | 20.4 (11.2) | 20.9 (15.4) |
| Number of Episodes, mean (SD) | 1.9 (1.7) | 2.3 (2.1) | 2.1 (1.7) |
| Duration of Current Episode >= 2y, No. (%) | 26 (100%) | 12 (100%) | 10 (100%) |
| Number of Failed Antidepressant Trials in Current Episode, mean (SD) | 6.0 (4.1) | 6.8 (3.9) | 7.1 (4.2) |
| History of Suicide Attempts, No. (%) | 5 (19%) | 3 (25%) | 1 (10%) |
| Clinically Significant Suicidal Ideation (MADRS-SI score >=4), No. (%) | 13 (50%) | 6 (50%) | 6 (60%) |
| MADRS-SI=4 | 10 (38.5%) | 5 (41.7%) | 5 (50%) |
| MADRS-SI=5 | 3 (11.5%) | 1 (8.3%) | 1 (10%) |
| MADRS-SI=6 | 0 | 0 | 0 |
Implicit Association Test (IAT)
Two recently developed variants of the IAT (IAT-Death, assessing the strength of association between words related to “Death” and “Me”; IAT-Escape, assessing associations between “Escape” and “Me”) were selected based on the hypothesis that individuals contemplating suicide would be characterized by greater self-identification with death (relative to life) and escape (relative to stay). Preliminary evidence suggests these associations are stronger in suicide attempters than non-attempters (10). IATs were administered and scored in accordance with recommended procedures (19) and followed a design described previously (9)(Supplement 2). “Escape=Me” and “Death=Me” D-scores were calculated for each participant, where D=[(mean RT during Escape=Me [or Death=Me] block) − (mean RT during Stay=Me [or Life=Me] block) ÷ (SD of RT across all trials)].
Statistical Analysis
A composite suicidality index (SIcomposite) was calculated by summing z-scores on the BSI and MADRS suicidality item (MADRS-SI). Change scores for all measures were calculated as 24-hour value – baseline value. Baseline and post-ketamine scores were compared by paired t-tests, with effect sizes calculated as Cohen’s d (20), and by repeated-measures analysis of covariance (ANCOVA). Due to violation of Small’s test for multivariate normality in several baseline measures, baseline correlations were calculated nonparametrically with Spearman’s rho. Correlations for change scores were calculated using Pearson’s r. Two-tailed alpha level was set at .05, unadjusted.
RESULTS
A single infusion of ketamine reduced scores on the MADRS-SI by an average of 2.08 points on a 0–6 scale (t25=6.42, p<.001; d=1.37), with 81% of patients achieving a rating of 0 or 1 24-hours post-infusion (Table 3; Figure 1). Of the 13 patients with clinically significant suicidal ideation at baseline (MADRS-SI scores ≥4), 8 (62%) received a rating of 0 or 1 24-hours post-infusion, 3 (23%) endorsed fleeting suicidal thoughts (ratings of 2 or 3), while 2 (15%) remained at or above a rating of 4. With change scores in non-suicide-related MADRS items (MADRS-totalnonSI) entered as a covariate, repeated-measures ANCOVA of baseline and 24-hour MADRS-SI scores was not significant (F1,24=.38, p=.54), suggesting ketamine’s anti-suicidal effects are mediated by depression reduction.
Table 3.
Ketamine effects on explicit and implicit suicidality
| Measure | Single Infusion Study (n=26) | IAT Subsample (n=10) | Repeated Infusions Study (n=10) | |||
|---|---|---|---|---|---|---|
| Mean (SD) | Effect Size: Cohen’s d [95% CI] | Mean (SD) | Effect Size: Cohen’s d [95% CI] | Mean (SD) | Effect Size: Cohen’s d [95% CI] | |
| MADRS total | ||||||
| Baseline | 36.92 (5.41) | 37.80 (5.8) | 32.70 (6.43) | |||
| Day 2: 24-hours after infusion 1 | 14.85 (13.14)*** | d=2.11 [1.25–2.97] | 9.40 (8.9)*** | d=3.79 [1.21–6.37] | 8.10 (4.63)*** | d=4.43 [0.89–7.97] |
| Day 12: 4-hours after infusion 6 | -- | -- | -- | -- | 5.11 (3.66)*** † | d=5.98 [1.52–10.44] |
| MADRS-SI | ||||||
| Baseline | 2.85 (1.6) | 2.90 (1.7) | 3.00 (1.63) | |||
| Day 2: 24-hours after infusion 1 | 0.77 (1.4)*** | d=1.37 [0.79–1.95] | 0.40 (.97)*** | d=1.67 [0.70–2.64] | 0.20 (0.63)*** | d=2.17 [0.75–3.59] |
| Day 12: 4-hours after infusion 6 | -- | -- | -- | -- | 0.00 (0.00)*** † | d=2.42 [1.20–3.63] |
| BSI | ||||||
| Baseline | -- | -- | 8.00 (9.3) | -- | -- | |
| Day 2: 24-hours after infusion 1 | -- | -- | 3.3 (7.8)* | d=0.53 [0.18–0.87] | -- | -- |
| IAT: Escape=Me | ||||||
| Baseline | -- | -- | −.04 (.31) | -- | -- | |
| Day 2: 24-hours after infusion 1 | -- | -- | −.53 (.40)** | d=1.36 [0.38–2.34] | -- | -- |
| IAT: Death=Me | ||||||
| Baseline | -- | -- | −.48 (.63) | -- | -- | |
| Day 2: 24-hours after infusion 1 | -- | -- | −.28 (.43) (ns) | d=−0.38 [−1.56–0.79] | -- | -- |
Note: TRD = treatment-resistant depression; MADRS = Montgomery-Asberg Depression Rating Scale (16); MADRS-SI = MADRS Suicidality Item; BSI = Beck Scale for Suicidal Ideation (17); IAT = Implicit Association Test (5)
p <.001;
p < .01;
p < .05; ns = not significant; baseline and post-infusion scores compared by paired t-tests; baseline-to-post infusion effect size (d) calculated from means, SDs, and pre-post correlation coefficients (20)
n=9 for Day 12 assessments
FIGURE 1.
Individual patient scores on the Montgomery-Asberg Rating Scale—Suicide item at baseline (Day 1; 150 minutes prior to infusion), 24-hours following a single subanesthetic infusion of ketamine (Day 2), and 4-hours following the final repeated infusion (Day 12 of study; Panel 2 only).
In the TRD subsample completing baseline IATs (n=12), stronger “Escape=Me” implicit associations were associated with greater MADRS-SI scores (rho=.60; p=.04), and marginally with SIcomposite scores (rho=.57; p=.052), but not with non-suicide-related depression severity (MADRS-totalnonSI: rho=.24; p=.46). Baseline “Death=Me” associations were unrelated to other measures (ps >.34). In patients who repeated the measures 24-hours post-infusion (n=10), there was a reduction in “Escape=Me” associations (t9=3.76; p=.006; d=1.37)(Figure 2) and in BSI (t9=3.15; p=.012) and MADRS-SI (t9=5.24; p<.001). “Death=Me” associations were not significantly changed (t9=.658; p=.52). “Escape=Me” reductions were correlated with reductions in BSI (r=.65; p=.042), SIcomposite (r=.64; p=.048), and MADRS-SI at the trend level (r=.57; p=.09), but not with MADRS-totalnonSI changes (r=−.03; p=.94). “Death=Me” changes showed a trend-level association with BSI changes only (r=.60; p=.06). Most zero-order correlations were maintained or increased after controlling for change in MADRS-totalnonSI and baseline SIcomposite (Table 4).
FIGURE 2.
Mean IAT-Escape D-scores (± SEM) representing the strength of association between words related to “Me” and words related to “Escape” at baseline and 24-hours following a single subanesthetic infusion of ketamine. D-scores calculated as D = [(mean response latency during Escape/Me block − mean response latency during Escape/Not Me block) ÷ SD of response latency across all trials]. More positive D-score indicates stronger implicit self-identification with words related to “Escape,” in comparison to words related to “Stay.”
Table 4.
Partial correlations between change scores (24-hours – baseline) in implicit and explicit suicidality measures, controlling for change in other depressive symptoms (MADRS, excluding suicidality item)
| MADRS-SI | BSI | SIcomposite | |
|---|---|---|---|
| IAT: Escape=Me | .74 (p=.022) | .72 (p=.028) | .78 (p=.014) |
| IAT: Escape=Me, adding baseline SIcomposite as an additional covariate | .64 (p=.086) | .60 (p=.118) | .71 (p=.048) |
| IAT: Death=Me | .34 (p=.366) | .69 (p=.038) | .55 (p=.122) |
| IAT: Death=Me, adding baseline SIcomposite as an additional covariate | .31 (p=.460) | .80 (p=.016) | .65 (.078) |
In patients who subsequently enrolled in the repeated-dose ketamine study (n=10), the first infusion again significantly reduced MADRS-SI scores (2.8-point mean decrease; t9=5.47, p<.001; d=2.17), with 90% of patients receiving a 24-hour rating of 0 (Table 3; Figure 1). Acute reductions were maintained throughout the 12-day treatment period by the 9 patients receiving repeated infusions (baseline-to-day-12 mean decrease=2.89; t8=5.12, p=.001; d=2.42), with no patient scoring >2 at any post-baseline assessment (before and after each infusion).
DISCUSSION
These preliminary findings support the premise that a single subanesthestic dose of IV ketamine has rapid effects on suicidal cognition in TRD, and that acute improvements in suicidality can be sustained through repeated ketamine infusions. Confidence intervals for MADRS-SI suggested moderate to very large effects, despite small sample sizes.
An IAT assessing the association between ‘Me’ and ‘Escape’ was related to explicit suicidal ideation at baseline and showed sensitivity to therapeutic change, an important criterion if the IAT is to have utility as a clinical assessment tool. Associations between ‘Me’ and ‘Death’ did not change as predicted, suggesting that the implicit representation of suicide in TRD might more closely relate to the concept of escape than to death itself. “Death=Me” associations were low at baseline, possibly limiting room for improvement. Prospective studies in high-risk samples should test whether the IAT can improve prediction of suicide risk in clinical settings, where motivation to conceal suicidal thoughts may exist (4). The IAT might also be useful in revealing psychological mechanisms of change. For instance, mediational analysis in larger samples could test the hypothesis that ketamine reduces depressed mood in suicidal patients, thereby decreasing implicit desire to escape from an unbearable emotional state, leading to downstream reductions in explicit suicidal thoughts.
The rapid onset and maintenance of improvement we observed suggests that IV ketamine, administered in the hospital setting with appropriate safety monitoring, may offer an attractive therapy for acutely suicidal depressed patients. Whether high-risk patients with markedly active suicidality will respond similarly remains an open question. Controlled studies are needed to establish whether ketamine is efficacious in such samples and whether decreased suicidality, once achieved, can be maintained through alternative pharmacological or psychosocial interventions. Given that all three datasets analyzed here were obtained from a single group of patients, these findings require independent replication.
Supplementary Material
Acknowledgments
This work was supported by National Institutes of Health grants F31 MH081468 (RBP), K23-MH-069656 and MO1-RR-00071 (SJM) and NARSAD (SJM, DSC). We thank Marije aan het Rot, Kate Collins, Kimberly Hunter, Michele Gonen, James Murrough, M.D., Andrew Perez, M.D., David Reich, M.D., all additional members of the Mood & Anxiety Disorders Program, and the staff of the Mount Sinai General Clinical Center for their assistance with the study.
Footnotes
[Registered at ClinicalTrials.gov as trial numbers NCT00419003, “Research Study for Major Depressive Disorder: Investigation of Glutamate Medications” and NCT00548964, “Continuation Intravenous Ketamine in Major Depressive Disorder”]
IAT and BSI data were collected from patients enrolled during the latter half of the enrollment period only
Time constraints prevented post-infusion data collection in 2 patients
3 participants in the repeated-dose study were also participants in the IAT subsample
FINANCIAL DISCLOSURES
Ms. Price and Dr. Nock report no biomedical financial interests or potential conflicts of interest. Dr. Mathew has received lecture or consulting fees from AstraZeneca and Jazz Pharmaceuticals, and has received research support from Alexza Pharmaceuticals, GlaxoSmithKline Pharmaceuticals, and Novartis Pharmaceuticals. Dr. Charney discloses consultant activities with Unilever UK Central Resources Limited in the past two years. Drs. Charney and Mathew have been named as an inventor on a use-patent of ketamine for the treatment of depression. If ketamine were shown to be effective in the treatment of depression and received approval from the Food and Drug Administration for this indication, Dr. Charney and the Mount Sinai School of Medicine could benefit financially. Dr. Mathew has relinquished his claim to any royalties and will not benefit financially if ketamine is approved for this use.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Kellner CH, Fink M, Knapp R, Petrides G, Husain M, Rummans T, et al. Relief of expressed suicidal intent by ECT: a consortium for research in ECT study. Am J Psychiatry. 2005;162:977–982. doi: 10.1176/appi.ajp.162.5.977. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Szanto K, Mulsant BH, Houck P, Dew MA, Reynolds CF., 3rd Occurrence and course of suicidality during short-term treatment of late-life depression. Arch Gen Psychiatry. 2003;60:610–617. doi: 10.1001/archpsyc.60.6.610. [DOI] [PubMed] [Google Scholar]
- 3.Nock MK, Banaji MR. Prediction of suicide ideation and attempts among adolescents using a brief performance-based test. J Consult Clin Psychol. 2007;75:707–715. doi: 10.1037/0022-006X.75.5.707. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Busch KA, Fawcett J, Jacobs DG. Clinical correlates of inpatient suicide. Journal of Clinical Psychiatry. 2003;64:14–19. doi: 10.4088/jcp.v64n0105. [DOI] [PubMed] [Google Scholar]
- 5.Greenwald AG, McGhee DE, Schwartz JL. Measuring individual differences in implicit cognition: the implicit association test. J Pers Soc Psychol. 1998;74:1464–1480. doi: 10.1037//0022-3514.74.6.1464. [DOI] [PubMed] [Google Scholar]
- 6.Cunningham WA, Preacher KJ, Banaji MR. Implicit attitude measures: consistency, stability, and convergent validity. Psychol Sci. 2001;12:163–170. doi: 10.1111/1467-9280.00328. [DOI] [PubMed] [Google Scholar]
- 7.Banse R, Seise J, Zerbes N. Implicit attitudes towards homosexuality: reliability, validity, and controllability of the IAT. J Exp Psychol. 2001;48:145–160. doi: 10.1026//0949-3946.48.2.145. [DOI] [PubMed] [Google Scholar]
- 8.Greenwald AG, Poehlman TA, Uhlmann EL, Banaji MR. Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity. J Pers Soc Psychol. doi: 10.1037/a0015575. in press. [DOI] [PubMed] [Google Scholar]
- 9.Nock MK, Banaji MR. Assessment of self-injurious thoughts using a behavioral test. Am J Psychiatry. 2007;164:820–823. doi: 10.1176/ajp.2007.164.5.820. [DOI] [PubMed] [Google Scholar]
- 10.Nock MK, Deliberto TL, Dour HJ, Finn CT, Park JL, Banaji MR. Identification of a behavioral marker that predicts suicide attempts submitted. [Google Scholar]
- 11.Zarate CA, Jr, Singh JB, Carlson PJ, Brutsche NE, Ameli R, Luckenbaugh DA, et al. A randomized trial of an N-methyl-D-aspartate antagonist in treatment-resistant major depression. Arch Gen Psychiatry. 2006;63:856–864. doi: 10.1001/archpsyc.63.8.856. [DOI] [PubMed] [Google Scholar]
- 12.Berman RM, Cappiello A, Anand A, Oren DA, Heninger GR, Charney DS, et al. Antidepressant effects of ketamine in depressed patients. Biol Psychiatry. 2000;47:351–354. doi: 10.1016/s0006-3223(99)00230-9. [DOI] [PubMed] [Google Scholar]
- 13.Mathew SJ, Murrough JW, Aan Het Rot M, Collins KA, Reich DL, Charney DS. Riluzole for relapse prevention following intravenous ketamine in treatment-resistant depression: a pilot randomized, placebo-controlled continuation trial. Int J Neuropsychopharmacol. 2009;17:1–12. doi: 10.1017/S1461145709000169. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Sackeim HA. The definition and meaning of treatment-resistant depression. J Clin Psychiatry. 2001;62(Suppl 16):10–17. [PubMed] [Google Scholar]
- 15.Rush AJ, Gullion CM, Basco MR, Jarrett RB, Trivedi MH. The Inventory of Depressive Symptomatology (IDS): psychometric properties. Psychol Med. 1996;26:477–486. doi: 10.1017/s0033291700035558. [DOI] [PubMed] [Google Scholar]
- 16.Montgomery SA, Asberg M. A new depression scale designed to be sensitive to change. Br J Psychiatry. 1979;134:382–389. doi: 10.1192/bjp.134.4.382. [DOI] [PubMed] [Google Scholar]
- 17.Beck AT, Steer RA. Manual for the Beck Scale for Suicide Ideation. San Antonio, TX: Psychological Corporation; 1991. [Google Scholar]
- 18.aan het Rot M, Collins KA, Murrough JW, Perez AM, Reich DL, Charney DS, et al. Safety and efficacy of repeated-dose intravenous ketamine for treatment-resistant depression. doi: 10.1016/j.biopsych.2009.08.038. submitted. [DOI] [PubMed] [Google Scholar]
- 19.Nosek BA, Greenwald AG, Banaji MR. Understanding and using the Implicit Association Test: II. Method variables and construct validity. Pers Soc Psychol Bull. 2005;31:166–180. doi: 10.1177/0146167204271418. [DOI] [PubMed] [Google Scholar]
- 20.Cohen J. Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.