Skip to main content
NCBI home page
Neurobiology of Stress logoLink to Neurobiology of Stress
. 2022 Nov 29;22:100505. doi: 10.1016/j.ynstr.2022.100505

A randomized pilot study of the prophylactic effect of ketamine on laboratory-induced stress in healthy adults

Sara Costi a,b,c,∗∗, Audrey Evers a, Manish K Jha a, Matthew Klein a, Jessica R Overbey d, Ki A Goosens e, JoColl Burgess e, Kelvin Alvarez a, Adriana Feder a, Dennis S Charney a,f,g, James W Murrough a,f,∗
PMCID: PMC9813787  PMID: 36620306

Abstract

Background

Stress exposure is a key risk factor for the development of major depressive disorder and posttraumatic stress disorder. Enhancing stress resilience in at-risk populations could potentially protect against stress-induced disorders. The administration of ketamine one week prior to an acute stressor prevents the development of stress-induced depressive-like behavior in rodents. This study aimed to test if the prophylactic effect of ketamine against stress also applies to humans.

Methods

We conducted a double-blind, placebo-controlled study wherein 24 healthy subjects (n = 11 males) were randomized to receive either ketamine (0.5 mg/kg) or midazolam (0.045 mg/kg) intravenously one week prior to an acute stress [Trier Social Stress Test (TSST)]. The primary endpoint was the anxious-composed subscale of the Profile of Mood States Bipolar Scale (POMS-Bi) administered immediately after the TSST. Salivary and plasma cortisol and salivary alpha amylase were also measured at 15-min intervals for 60 min following the stressor, as proxies of hypothalamic pituitary adrenal (HPA) and sympathetic-adrenal-medullary (SAM) axis activity, respectively.

Results

Compared to the midazolam group (n = 12), the ketamine group (n = 12) showed a moderate to large (Cohen's d = 0.7) reduction in levels of anxiety immediately following stress, although this was not significant (p = 0.06). There was no effect of group on change in salivary cortisol or salivary alpha amylase following stress. We conducted a secondary analysis excluding one participant who did not show an expected correlation between plasma and salivary cortisol (n = 23, ketamine n = 11). In this subgroup, we observed a significant reduction in the level of salivary alpha amylase in the ketamine group compared to midazolam (Cohen's d = 0.7, p = 0.03). No formal adjustment for multiple testing was made as this is a pilot study and all secondary analyses are considered hypothesis-generating.

Conclusions

Ketamine was associated with a numeric reduction in TSST-induced anxiety, equivalent to a medium-to-large effect size. However, this did not reach statistical significance . In a subset of subjects, ketamine appeared to blunt SAM reactivity following an acute stressor. Future studies with larger sample size are required to further investigate the pro-resilient effect of ketamine.

1. Background

Stress exposure is one of the greatest risk factors for psychiatric illnesses, including major depressive disorder (MDD) and posttraumatic stress disorder (PTSD), both chronic conditions that affect more than 300 million people and are leading causes of disability worldwide (Hasin et al., 2018). Stress resilience includes the ability to experience stress without developing clinically significant psychopathology and is also associated with the concept of adaptation, the ability to quickly recover after a stressor, or the capacity to maintain functioning following adversity (American Psychological Association, 2018). Enhancing stress resilience in at-risk populations could potentially protect against the development of stress-induced psychiatric disorders. Despite this, no resilience-enhancing pharmaceuticals have been identified yet. Currently available treatments to protect against the development of psychiatric disorders are still in their infancy and limited mainly to psychotherapy and physical exercise (Marques et al., 2020).

Preclinical studies show that the administration of the glutamate N-methyl-D-aspartate (NMDA) receptor antagonist ketamine one week prior to an acute stress prevents the developing of depressive-like or PTSD-like behavior in animals (Brachman et al., 2016; McGowan et al., 2018). In preclinical work, mice treated with ketamine one week prior to a chronic Social Defeat (SD) stress paradigm, wherein the experimental mice are repeatedly exposed to aggressor mice, showed a significant reduction in the pro-depressive effects of SD and enhancement of stress resilience during the Forced Swim Test (FST). These results have since been replicated (Mastrodonato et al., 2018) and similar outcomes have been observed when ketamine was administered at different times prior to the stressor (Amat et al., 2016). These findings have then been further generalized to a variety of rodent stress paradigms (Brachman et al., 2016; Krzystyniak et al., 2019; Camargo et al., 2020). Shifting to PTSD-relevant models, McGowan et al. (2017) reported a significant reduction in fear behavior in mice treated with ketamine one week prior to contextual fear conditioning. Of note, animal studies (Mastrodonato et al., 2018) also suggest that ketamine prophylactic effects are mediated by neural changes at the level of the ventral hippocampus, and not by prefrontal cortex neurons as observed for ketamine antidepressants effect (Duman et al.,; Moda-Sava et al. 2019). These basic data suggest the possibility that different neural pathways are involved in ketamine pro-resilient effects. In this set of experiments, mice were treated with saline or ketamine one week before SD. Ketamine-treated mice showed reduced learned fear and altered memory traces representing the stressful experience, specifically at the hippocampal level, possibly explaining the longer time required to observe a pro-resilient effect of ketamine (one week) in striking contrast with ketamine rapid antidepressant effect (24 h).

Here we report the results of a randomized. placebo-controlled, proof-of-concept trial testing if the administration of ketamine, compared to the midazolam control condition, can attenuate the behavioral and physiological effects of a laboratory-induced acute stress on healthy volunteers when administered one week prior. Subjects received a single received intravenous (IV) infusion of either ketamine or midazolam one week prior to stress induction. Midazolam is a benzodiazepine anesthetic which mimics some of the acute subjective effects of ketamine (hence aiming at preserving the study blind for both participants and researchers) but is expected to have no pro-resilience effect. The Trier Social Stress Test (TSST), a well-validated laboratory stressor, was implemented to induce an acute stress response. The TSST is known to provoke moderate stress in most participants and a reliable response in the hypothalamic pituitary adrenal (HPA) axis (Kirschbaum et al., 1993). This paradigm is designed to probe the domain of Negative Valence System, primarily responsible for responses to aversive situations or context, such as fear, anxiety, and loss, within the Research Domain Criteria (RDoC) of Acute Threat (“Fear”) by NIH/NIMH (Insel, 2014). Acute elevations of SAM axis activity and self-reports of negative affect using the Profile of Mood States (POMS) and the Positive and Negative Affect Scale (PANAS) have also been reported (Allen et al., 2014). Responses to the TSST appear to be altered in cases of stress-related disorders like major depression or anxiety disorders, and both pharmacological and psychotherapeutic interventions can moderate the TSST response (Allen et al., 2014).

This proof-of-concept study of the stress prophylactic properties of ketamine represents the first effort to translate the findings of recent preclinical work to humans. A change on the composed-anxious subscale score of the Profile of Mood States – Bipolar (POMS-Bi; Lorr and McNair, 1984; 1988) from pre-TSST to post-TSST in the ketamine group compared to the midazolam group one week after the administration of the study drug represented the primary behavioral outcome.

2. Methods

2.1. Study participants and design

The study procedures were conducted at the Depression and Anxiety Center for Discovery and Treatment at the Icahn School of Medicine at Mount Sinai (ISMMS) in New York City between August 2019 and March 2021. The ISMMS review board approved the study, and written informed consent was obtained from all participants prior to any study procedure. Participants were compensated for their time and effort. The study was registered at clinicaltrials.gov (NCT04173962). Participants were between the ages of 18 and 45 years and did not meet diagnostic criteria for any lifetime psychiatric disorder as determined by a trained rater using the Structured Clinical Interview for DSM-5 Disorders (Research Version; SCID-5-RV) (First et al., 2015). As age is a well-established moderating factor of the response to the TSST and the age of participants may alter their perception of the stressor (Allen et al., 2014), we opted to limit inclusion of subjects up to 45 years of age to minimize the confounding effect of age. All participants underwent a physical examination, biochemistry and hematological laboratory testing, urine toxicology and pregnancy (if applicable) testing, and an electrocardiogram (ECG). Subjects with concomitant unstable medical illnesses, hypertension, or previous use of phenylcyclohexyl piperidine (PCP) or ketamine were excluded.

Following screening (Day −28 – Day −1), subjects who met all eligibility criteria returned to the research clinic to undergo randomization (Day 0, Infusion Day). Participants completed self-report assessments including the POMS-Bi (Lorr and McNair, 1984, 1988), PANAS (Watson et al., 1988), BAI (Beck et al., 1988), and VAS–Stressed, and clinician-administered assessments like the Clinician-Administered Dissociative States Scale (CADSS; Bremner et al., 1998) and the Brief Psychiatric Rating Scale (BPRS; Overall and Gorham, 1962) prior to study drug administration. The same self-report and clinician administered scales were repeated within 1 h after study drug administration. After confirming that participants were nil per os (NPO), tested negative on point-of-care urine pregnancy (child-bearing potential participants) and toxicology tests, and met the cutoff for baseline blood pressure, the subjects were randomized by the ISMMS research pharmacy. The randomization schedule was generated by the ISMMS Investigational Drug Services, employing randomly permuted blocks to assign participants to receive one infusion of either ketamine hydrochloride (0.5 mg/kg) or midazolam (0.045 mg/kg). The identity of test and control treatments were not known to investigators, research staff or patients. To ensure double-blind administration of study treatment, access to the randomization code was strictly controlled by the Investigational Drug Services, packaging and labeling of test (ketamine) and control (midazolam) treatments were identical to maintain the blind.

The study drugs were administered according to well-established standard protocols. An indwelling catheter was placed in the antecubital vein and the drug was administered over 40 min at a constant rate via an infusion pump. Pulse, blood pressure, and oxygen saturation were monitored continuously during infusion and every 15 min after the infusion for at least 1 h. Twenty-four hours after the infusion, participants were contacted by a member of the study team for adverse events (AEs) documentation, self-report questionnaires, and concomitant medication documentation.

Subjects returned to the research clinic 7 days later for the Assessment visit (Study Visit 2) wherein the TSST was administered according to Allen et al. (2014). Upon arrival and after testing negative on a urine toxicology test, an indwelling catheter was placed for repeated blood sample collection. The baseline rest and stress procedures were carried out in separate rooms (referred to here as Room A and Room B). The baseline rest phase took place in Room A. At least 75 min of rest were required to allow HPA and SAM axes activity to normalize, as the process of attending the laboratory session itself and the placement of the indwelling catheter could cause an increase in stress levels at the behavioral and biological level. Baseline self-report assessments (POMS-Bi, PANAS, BAI, and VAS-Stressed) and blood samples were collected 15 min prior to the start of the TSST. Saliva samples were collected 30 and 15 min prior to the start of the TSST and the average value was used as the baseline saliva measure. Following task instructions, participants then completed the stress procedure in Room B. The procedure ran as follows: participants were introduced to a role-playing scenario wherein they had to prepare a speech to convince a panel of two judges that they are the perfect candidate for their dream job, followed by a mental arithmetic task (serial subtraction). The TSST lasted 10 min. Participants were told they were being video and audio recorded. The panel of assessors was instructed to refrain from giving any verbal or non-verbal feedback during the tasks other than the scripted instructions. Following these tasks, participants returned to Room A for the resting period and post-stressor measure collection. A recovery period of 60 min was used. Repeated samples were collected at five 15 min intervals (t +0 through t +60). A number of physiological parameters were measured throughout the procedure via saliva (cortisol, α-amylase) and physiological tests (heart rate, blood pressure), and blood samples were collected for plasma analysis (cortisol). The Assessment Visit was scheduled during late morning/early afternoon hours to ensure an appropriate stress-induced increase in level of cortisol in consideration of the cortisol circadian rhythm (Bellingrath et al., 2010; Pace et al., 2006; Young et al., 2000). Similarly, to limit the confounding effect of food, caffeine, and physical exercise on cortisol levels, participants were asked to avoid food for 3 h prior to the visit, avoid exercise and caffeine for 2 h prior to the visit, and avoid low pH drinks (e.g., orange juice) and water for 30 min prior to the visit. Salivary samples were collected at the time points described above using Sarstedt Cortisol Salivette synthetic swab system and then centrifuged at 1000×g for 2 min at ambient temperature and frozen at −20 °C until the end of the study. Salivary cortisol and alpha amylase were assayed by Salimetrics, LLC with commercially available immunoassay kits. Blood samples were collected using EDTA Tubes (4 mL) and processed within 2 h of the blood draw. Samples were collected from 21 subjects (n = 11 randomized to ketamine). Reasons for failure in blood collection were participant distress and indwelling catheter clot. Samples were centrifuged at 3000 RCF for 20 min at ambient temperature (23 °C) and isolated plasma was frozen at −20 °C until completion of the study. Circulating levels of cortisol were assessed using a commercially available ELISA kit by ENZO Life Sciences. After the 60 min of monitoring was complete and all samples (surveys, blood, and saliva) had been collected, the participant was debriefed. It was explained that they were not evaluated or scored during the task and that the study team was interested in investigating how they responded to the TSST. Participants all met with a study investigator prior to discharge.

To capture the response to the TSST, a change on the composed-anxious subscale score of the POMS-Bi (Lorr and McNair, 1984, 1988) from pre-TSST to post-TSST in the ketamine group compared to the midazolam group one week after the administration of the study drug represented the primary behavioral outcome. A higher score on the POMS-Bi indicates higher functioning (i.e., more composed/less anxious) within that domain. Whilst most assessment tools have been developed to measure negative affect in clinical populations, the POMS-Bi assesses both positive and negative emotional states using a continuum scale within six different domains (composed–anxious, clear-minded–confused, elated–depressed, energized–tired, agreeable–hostile, and confident–unsure), appearing consequentially more suitable to investigate resilience to stress in a non-clinical population (Lorr et al., 2003; Kass et al., 1991). The behavioral secondary endpoints were changes from pre-TSST to post-TSST on the other subscale scores of the POMS-Bipolar and other self-reports (PANAS, BAI, VAS – Stressed) in the ketamine group compared to midazolam. The biological primary endpoint was a change in HPA activity, as measured by salivary concentration of the stress hormone cortisol in response to the TSST in the ketamine group compared to midazolam. The biological secondary endpoint was a change in SAM activity, as measured by salivary α-amylase in response to the TSST in subjects who received ketamine compared to midazolam one week prior.

2.2. Statistical analysis

Demographic and clinical characteristics, blood, salivary biomarkers, and behavioral measures were summarized by randomization group using descriptive statistics. Safety and tolerability data are also summarized descriptively by randomization group. AEs were categorized using the Medical Dictionary for Regulatory Activities (MedDRA) system organ class and preferred terms. This was the first study to test the prophylactic effect of ketamine on a laboratory-induced acute stress in healthy volunteers and no preliminary data were available for a sample size calculation. Since this was a pilot study, no formal sample size calculation was performed. We reasoned that a sample size of 24 would provide an initial estimate of effect size that would be used to power future studies (Julious, 2005).

The primary behavioral outcome was represented by the change in the composed-anxious subscale score of the POMS-Bipolar in response to an acute stress (the +0 time point) from pre-TSST (the −25-min time point). The primary endpoint was compared between randomization groups using a linear regression model of the change score and randomization group adjusted for baseline score. The primary behavioral null hypothesis was that the regression coefficient for randomization group was equal to zero. The null hypothesis was tested using a two-sided, 0.05 level Wald test. Secondary analyses of the POMS-Bi scale compared the trajectory of scores across each study time point by randomization group using linear mixed-effects models. These models included a random intercept for participant and fixed effects for randomization group, time, and a randomization group-by-time interaction. Secondary behavioral outcomes including other POMS-Bi subscale scores, PANAS, VAS, and BAI were analyzed in the same manner described above for the behavioral primary outcome.

The trajectory of salivary cortisol over each study time point was compared between randomization groups using a linear mixed effects model. The model included a random intercept for participant and fixed effects for randomization group, time, and a randomization group-by-time interaction. The primary biological null hypothesis was that there was no interaction between time and randomization group. The null hypothesis was tested using a type III test of fixed effects with a two-sided, 0.05 level F-test. Secondary biological outcomes including alpha amylase were analyzed in the same manner described above for the biological primary outcome. Plasma levels of cortisol were also collected and correlated with salivary cortisol values. Salivary and plasma cortisol were log transformed prior to modelling. Estimation of correlation with repeated measures data was based on the mixed model approach proposed by Hamlett et al. (2004) to calculate the correlation coefficient from the within subject variance matrix. An approximate 95% confidence interval was obtained using the normal approximation method using Fisher's z transformation and applying the delta method. Subsequent analysis on salivary cortisol and alpha amylase were conducted on participants who showed an expected correlation between plasma and salivary cortisol (Hellhammer et al., 2009).

All analyses were conducted at the 0.05 significance level. No formal adjustment for multiple testing was made as this is a pilot study and all secondary analyses are considered hypothesis-generating. Effect sizes for differences in mean change from pre-TSST (t-25) to post-TSST (t+0) between treatment arms were computed as Cohen's d (Cohen’s 1988). Exploratory analyses were performed on blood samples, to test the plasma cortisol levels before and after the TSST.

3. Results

Thirty-three subjects consented to participate in the trial. Of these, 24 (72.7%) were randomized, 6 (18.2%) were screen failures, and 3 (9.1%) were exited prior to randomization due to the outbreak of the COVID-19 pandemic in New York. See Table 1 for a summary of the socio-demographic characteristics of the randomized subjects. No subject dropped out from the study between the infusion and the Assessment Visit. More information is reported in Fig. 1. Consort Chart representing subject flow by study arm (ketamine versus midazolam). No formal adjustment for multiple testing was made as this is a pilot study and all secondary analyses are considered hypothesis-generating.

Table 1.

Demographic and clinical characteristics of study sample.

All (N = 24) Ketamine (N = 12) Midazolam (N = 12)
Age at enrollment (years), M (SD) 28.3 ± 7.6 28.4 ± 7.7 28.2 ± 7.9
BMI, M (SD) 25.5 ± 5.4 25.6 ± 6.2 25.5 ± 4.8
Gender, Male, n (%) 11 (45.8%) 8 (66.7) 3 (25.0)
Relationship Status, n (%)
 Single, Never married 19 (79.2) 9 (75.0) 10 (83.3)
Education, n (%)
 At least some college 23 (96) 11 (91.6) 12 (100)
Employment, n (%)
 At least part time 18 (75) 9 (75) 9 (75)
Race/Ethnicity, n (%)
 Black or African-American 5 (20.8) 2 (16.7) 3 (25.0)
 White or Caucasian 12 (50.0) 7 (58.3) 5 (41.7)
 Hispanic or Latino 5 (20.8) 3 (25.0) 2 (16.7)
Open in a new tab

M, means; SD, Standard deviation. Race and ethnicity were reported by the study participants.

Fig. 1.

Fig. 1

Open in a new tab

CONSORT diagram.

Three participants experienced AEs during the study. Two participants experienced one event and one participant experienced three events. All three participants were randomized to the ketamine group. No subjects discontinued the treatment protocol because of AEs, and no serious adverse events occurred during the study.

3.1. Behavioral endpoints

Ketamine mean change from the pre- TSST time point to the post-TSST time point on the composed-anxious subscale of the POMS-Bi was −6.8 (±5.8), while midazolam mean change from the pre- TSST time point to the post-TSST time point was −11.9 (±8.6). Individuals treated with ketamine showed more composed/less anxious scores in the anxiety-composed subscale of the POMS-Bi post-stress compared to midazolam, as documented by an average change score that was 5.66 points higher compared to midazolam adjusting for baseline score, albeit not statistically significant (95% CI -0.34, 11.60; p = 0.06; Table 2 and Fig. 2). The effect size for the difference in change in means was Cohen's d = 0.7, equivalent to a medium to large effect size.

Table 2.

Observed Distributions of Self-Reported Measures of Stress Following the Trier Social Stress Task in Healthy Volunteers who received Ketamine versus Midazolam one week prior.

Ketamine (Mean, SD)
 Midazolam (Mean, SD)
Pre-TSST Post-TSST Difference Pre-TSST Post-TSST Difference
POMS-Bipolar
 Composed-Anxious 30.6 5.3 23.8 5.1 −6.8 5.8 29.3 7.4 17.4 9.9 −11.9 8.6
 Elated - Depressed 26.3 6.5 24.9 7.6 −1.3 4.7 23.7 6.9 22.8 8.3 −0.8 6.8
 Energetic - Tired 24.9 7.6 26.8 8.9 1.8 6.1 21.9 6.7 25.3 7.9 3.4 7.3
 Agreeable - Hostile 31.3 5.0 26.7 9.3 −4.7 8.7 28.8 7.6 22.4 11.1 −6.3 7.5
 Confident – Unsure 26.8 7.0 24.2 7.6 −2.4 6.1 24.8 5.6 22.7 9.0 −2.1 5.4
 Clear – Confused 29.3 4.7 24.9 7.3 −4.4 6.4 28.1 7.0 26.8 6.7 −1.3 5.6
PANAS
 Negative Affect 11 1.3 13.2 4.4 2.2 3.7 13.3 5.2 13.2 4.4 2.2 3.7
 Positive Affect 32.7 11.4 30.5 14.3 −2.2 10.1 31.3 8.3 32.5 10.3 1.2 6.5
BAI 0.4 0.9 1.2 2.0 0.8 1.4 0.8 1.1 3.4 4.5 2.6 4.1
VAS - Stressed 11.7 9.4 24.4 15.8 12.8 11.6 23.3 22.3 35.4 23.5 12.1 22.3
Open in a new tab

Table 2 reports Self-Reported Measures of Stress Following the Trier Social Stress Task. Descriptive statistics are provided for observed values for each measure. A higher score in one of the POMS-Bi domains indicates higher functioning within that domain (e.g., more composed/less anxious for the Composed-Anxious domain). Abbreviations: BAI, Beck Anxiety Inventory; PANAS, Positive and Negative Affect Scale; POMS, Profile of Mood State; VAS, Visual Analog Scale. For descriptive purposes we report the means at pre-TSST (t-25) and immediately after the TSST (t+0).

Fig. 2.

Fig. 2

Open in a new tab

Effect of Ketamine Compared to Midazolam Administered One Week Prior On Salivary Cortisol and Salivary Alpha Amylase Levels during in Healthy Volunteers. A. Change in POMS-Bi Composed-Anxious Sub-Score during the Trier Social Stress Task (TSST) in healthy volunteers randomized to ketamine 0.5 mg/kg or midazolam 0.045 mg/kg one week prior. B. Individual POMS-Bi Composed-Anxious Sub-Score data points during the TSST in healthy volunteers randomized to ketamine 0.5 mg/kg or midazolam 0.045 mg/kg one week prior. Higher scores on the Composed-Anxious subscale indicate higher functioning (i.e., more composed/less anxious). The timepoints reported in the X axis are expressed in minutes from the stressor. min, minutes; POMS, Profile of Mood State.

Secondary behavioral endpoints include the POMS elated-depression score, the POMS energetic-tired score, the POMS agreeable-hostile score, the POMS confident-unsure score, the PANAS negative affect score, the PANAS positive affect score, the BAI score, and the VAS –Stressed score. The observed distributions of each measure at time −25, time +0, and the change score are shown in Table 2. Model estimates are shown in Table 3.

Table 3.

Model Estimates of Change in Self-Reported Measures of Stress Following the Trier Social Stress Task in Healthy Volunteers who received Ketamine versus Midazolam one week prior.

Ketamine
 Midazolam
 Beta Coefficient (95% CI) p-value
LS Mean (SE) LS Mean (SE)
POMS-Bipolar
 Composed-Anxious −6.50 (2.04) −12.16 (2.04) 5.66 (−0.34, 11.60) 0.06
 Elated - Depressed −1.10 (1.71) −1.07 (1.71) −0.03 (−5.13, 5.00) 0.99
 Energetic - Tired 2.21 (1.93) 3.04 (1.93) −0.84 (−6.58, 5.00) 0.77
 Agreeable - Hostile −4.62 (2.42) −6.38 (2.42) 1.76 (−5.44, 9.00) 0.62
 Confident - Unsure −2.38 (1.70) −2.12 (1.70) −0.26 (−5.29, 4.80) 0.92
 Clear - Confused −4.22 (1.69) −1.45 (1.69) −2.77 (−7.74, 2.20) 0.26
PANAS
 Positive −2.12 (2.59) 1.11 (2.48) −3.23 (−10.71, 4.20) 0.38
 Negative 2.25 (1.39) 4.52 (1.33) −2.27 (−6.38, 1.80) 0.26
BAI 0.95 (0.86) 2.39 (0.86) −1.44 (−3.99, 1.20) 0.25
VAS-Stressed 10.68 (5.09) 14.16 (5.09) −3.48 (−18.89, 12.00) 0.64
Open in a new tab

Table 3 reports the model estimates of changes in self-report measures of stress pre- and post-TSST, as described in the Methods. Abbreviations: BAI, Beck Anxiety Inventory; LS, Least Squares; POMS, Profile of Mood State; PANAS, Positive and Negative Affect Scale; SE, Standard Error; VAS, Visual Analog Scale.

3.2. Biological endpoints

There was no significant difference in the trajectories of salivary cortisol over time between the randomization groups (p = 0.09; Fig. 3 and Table 4). Cortisol levels were also tested on plasma samples available from a subset of participants (N = 21, 11 ketamine). Circulating and salivary cortisol were overall positively correlated (estimate = 0.67, 95% CI [0.5, 0.8]), except for one subject and one time point from another participant for whom salivary and circulating values of cortisol did not correlate. The effect size for the difference in change in means was Cohen's d = 0.47, equivalent to a small-medium effect size.

Fig. 3.

Fig. 3

Open in a new tab

Effect of Ketamine Compared to Midazolam Administered One Week Prior On Salivary Cortisol and Salivary Alpha Amylase Levels during in Healthy Volunteers A. Change in salivary cortisol levels during the Trier Social Stress Task (TSST) in healthy volunteers randomized to ketamine 0.5 mg/kg or midazolam 0.045 mg/kg one week prior. B. Individual salivary cortisol levels during the TSST in healthy volunteers randomized to ketamine 0.5 mg/kg or midazolam 0.045 mg/kg one week prior. C. Change in salivary alpha amylase levels during the Trier Social Stress Task (TSST) in healthy volunteers randomized to ketamine 0.5 mg/kg or midazolam 0.045 mg/kg one week prior. D. Individual salivary alpha amylase levels during the TSST in healthy volunteers randomized to ketamine 0.5 mg/kg or midazolam 0.045 mg/kg one week prior. E. Change in salivary alpha amylase levels during the Trier Social Stress Task (TSST) in healthy volunteers randomized to ketamine 0.5 mg/kg or midazolam 0.045 mg/kg one week prior excluding a participant lacking the expected correlation between plasma and salivary cortisol. F. Individual salivary alpha amylase levels during the Trier Social Stress Task (TSST) in healthy volunteers randomized to ketamine 0.5 mg/kg or midazolam 0.045 mg/kg one week prior excluding a participant lacking the expected correlation between plasma and salivary cortisol. Values reflect means with associated standard error of the mean (SEM).

Table 4.

Observed Distributions of Salivary Cortisol and Alpha Amylase Following the Trier Social Stress Task in Healthy Volunteers who received Ketamine versus Midazolam one week prior.

Salivary Cortisol (μg/dL) *
t-25 t + 0 t + 15 t + 30 t + 45 t + 60
Ketamine, M (SD) 0.17 (0.1) 0.23 (0.00) 0.26 (0.07) 0.19 (0.06) 0.18 (0.08) 0.15 (0.06)
Midazolam, M (SD) 0.22 (0.34) 0.22 (0.2) 0.27 (0.18) 0.22 (0.17) 0.2 (0.14) 0.18 (0.12)
Salivary alpha amylase (U/mL)
Ketamine, M (SD) 104 (81.3) 213 (209.5) 111.5 (69.2) 105.6 (64.6) 103.3 (57.9) 93.5 (48.2)
Midazolam, M (SD) 129.1 (43.5) 270.8 (126.8) 174.5 (81.6) 153.7 (68.4) 147.7 (64.4) 135.6 (51)
Open in a new tab

Table 4 reports salivary cortisol and alpha amylase following the Trier Social Stress Task. * Data reported on N = 23 subjects. One subject (randomized to ketamine) has been excluded from the analysis since circulating and salivary cortisol were not correlated (more details available in the Statistical section of the manuscript). Abbreviations: M, Mean; SD, Standard Deviation.

There was no significant difference in the trajectories of salivary alpha amylase over time between the groups (p = 0.93; Table 4). When salivary samples were re-analyzed excluding data from the participant whose circulating and salivary cortisol were not correlated, the ketamine group showed a significant reduction in level of salivary alpha amylase compared to midazolam (p = 0.03; Fig. 3), suggesting an effect of ketamine in blunting SAM response to an acute stress in this subset of participants (N = 23, 11 randomized to ketamine). The effect size for the difference in change in means was Cohen's d = 0.85, equivalent to a large effect size.

4. Discussion

This is the first study to test the prophylactic effect of ketamine on a laboratory-induced acute stress in healthy volunteers. The primary hypothesis of the study was to test the effect of ketamine compared to midazolam on the POMS composed-anxious sub-score following the TSST. Although the a priori statistical threshold was not met on this measure, the numerical benefit of the ketamine group on the primary outcomes was consistent with a moderate to large (Cohen's d: 0.7) effect. No difference was observed on the biological readouts following the TSST between groups across the whole sample. In an exploratory analysis conducted on a subset of subjects excluding data from the participant whose circulating and salivary cortisol were not correlated, ketamine was associated with a significant reduction of salivary alpha amylase levels, but not cortisol, following the TSST.

Compared to midazolam, ketamine failed to blunt the biological response to an acute stressor, as measured with salivary cortisol, plasma cortisol, and salivary alpha amylase. This finding appears consistent with previously published data on the resilience-enhancing effect of ketamine (Brachman et al., 2016), wherein mice treated with ketamine did not differ from placebo-treated mice in corticosterone (the rodent homolog of cortisol) levels following SD. Overall, these data suggest that an effect of ketamine on stress responsiveness in humans when administered one week prior may not be mediated by modulation of the HPA axis. Interestingly, cortisol levels can be acutely modulated by the administration of ketamine. In a placebo-controlled crossover trial, healthy volunteers randomized to a sub-anesthetic dose of ketamine showed a dose-dependent increase in cortisol production compared to saline in the 4 h following drug administration (Khalili-Mahani et al., 2015). However, the mechanism by which ketamine induces a transient increase of cortisol following ketamine administration remains unknown. Of relevance to our findings, it is not known whether this increase is induced by increased stress levels due to the development of psychotomimetic and/or dissociative symptoms during the infusion or if it is required for ketamine's mechanism of action. As we report here, when our analysis was limited only to those participants with high correlations between plasma and salivary cortisol, ketamine appeared to reduce salivary alpha amylase levels compared to midazolam but not did differ from midazolam on plasma or salivary cortisol level post-TSST, suggesting an effect of ketamine on the SAM axis but not on the HPA axis.

Current available treatments to protect against the development of psychiatric disorders are limited and difficult to access. If supported by future work, ketamine could represent a relatively safe treatment alternative that can possibly reach a large number of individuals at risk with an acceptable cost-benefit ratio. Further, these data can also provide preliminary information on the neurobiological mechanism underpinning stress resilience. Recent lines of research suggest psychological resilience as an active state rather than simply the absence of clinically significant symptoms. The adoption of investigational medicine trials in healthy and at-risk populations may allow for a greater understanding of resilience-related neural mechanisms and ultimately for the identification of novel targets for the prevention of stress-related disorders. Although current clinical practice is focused on the treatment of symptoms, the identification of novel pro-resilient targets would represent a major breakthrough in treatment for depression and other stress-related disorders, potentially leading to the development of a vaccine-like strategy in at-risk populations where high-stress conditions can be predicted.

While this is the first study aiming to test if the pro-resilient effect of ketamine also applies to humans, there have been studies examining the effect of ketamine for the prevention of postpartum depression (PPD; Ma et al., 2019; Alipoor et al., 2021). Participants were randomized to receive ketamine or placebo immediately following cesarean section (c-section), a potentially stressful procedure, and subsequently assessed with the Edinburgh Postnatal Depression Scale (EPDS; Cox et al., 1987) at various time points postpartum. In one study (Ma et al., 2019), the primary outcome at 6–8 weeks postpartum showed significantly lower prevalence of PPD in the ketamine group (12.8%) than the control group (19.6%). In another study (Alipoor et al., 2021), the prevalence of PPD at 4 weeks postpartum was significantly lower in the ketamine group. The findings from these trials suggest that ketamine may reduce the development of PPD when administered immediately following cesarian section.

This study has several limitations. The small sample size may have may have led to Type II errors that could explain the lack of statistically significant effect of ketamine on psychological and biological readouts following an acute stress. In addition, plasma cortisol data were missing for three subjects, 1 subject in the ketamine group and 2 subjects in the midazolam group. Hence, these data may guide sample size calculation for future larger studies. Further, the current study used the benzodiazepine anesthetic midazolam as active control condition to mimic some of the acute subjective effects of ketamine and ensure blinding of participants and researchers. However, it is unknown if midazolam can exert a sustained pro-resilience effect. Further, although the levels of salivary alpha amylase can be considered a proxy of SAM axis activity, the lack of samples to test the level of catecholamines in the circulating plasma limits our ability to conclusively demonstrate the effect of ketamine on SAM activity. Finally, we tested here only a single dose and single time point for ketamine administration based on animal studies; further research is needed to determine the optimal doses and time point for administration relative to stressor exposure. It remains a possibility that the failure in capturing a pro-resilience effect of ketamine drives from the dosage utilized (0.5 mg/kg) and the one-week time elapsed between infusion and assessment visit.

In conclusion, this is the first randomized, placebo-controlled study investigating the potential resilience-enhancing effect of ketamine in healthy adults. Ketamine administered one week prior to an acute stressor was associated with a medium-to-large magnitude reduction in TSST-induced anxiety and blunted SAM reactivity in a subset of subjects with valid plasma cortisol data, although this finding did not reach statistical significance in a small sample. Based on these findings, larger randomized controlled trials investigating the pro-resilience effect of ketamine, including mechanistic studies to explore ketamine stress-prophylactic properties, are warranted.

Funding

Research reported in this publication was supported by the Ehrenkranz Laboratory for Human Resilience, a component of the Depression and Anxiety Center for Discovery and Treatment at the Icahn School of Medicine at Mount Sinai, and by a generous gift from the Gottesman Foundation. SC is currently funded by a Wellcome Trust Clinical Doctoral Research Fellowship.

CRediT authorship contribution statement

Sara Costi: Conceptualization, Methodology, Investigation, Data curation, Writing – original draft, Visualization, Writing – review & editing. Audrey Evers: Investigation, Data curation, Project administration. Manish K. Jha: Investigation, Writing – review & editing. Matthew Klein: Investigation, Writing – review & editing. Jessica R. Overbey: Formal analysis, Writing – review & editing. Ki A. Goosens: Investigation, Resources, Writing – review & editing. JoColl Burgess: Investigation, Resources, Writing – review & editing. Kelvin Alvarez: Investigation, Writing – review & editing. Adriana Feder: Conceptualization, Methodology, Writing – review & editing. Dennis S. Charney: Conceptualization, Methodology, Supervision, Writing – review & editing. James W. Murrough: Conceptualization, Methodology, Investigation, Data curation, Supervision, Writing – review & editing.

Declaration of competing interest

Dr. Dennis Charney (Dean of Icahn School of Medicine at Mount Sinai) is a named co-inventor on several issued U.S. patents, and several pending U.S. patent applications filed by the Icahn School of Medicine at Mount Sinai (ISMMS) related to pharmacologic therapy for treatment-resistant depression, suicidal ideation and other disorders. ISMMS has entered into a licensing agreement with Janssen Pharmaceuticals, Inc. and it has and will receive payments from Janssen under the license agreement related to these patents. As a co-inventor, Dr. Charney is entitled to a portion of the payments received by the ISMMS. Since SPRAVATO (esketamine) has received regulatory approval for treatment-resistant depression, ISMMS and Dr. Charney as its employee and a co-inventor, will be entitled to additional payments, under the license agreement. Drs. Charney and Feder are named co-inventors on an issued patent in the United States and several issued patents outside of the United States, filed by the Icahn School of Medicine at Mount Sinai for the use of ketamine as therapy for posttraumatic stress disorder; this intellectual property has not been licensed. Dr. Costi has provided consultation services for Guidepoint and TCG Crossover. Dr. Jha has received contract research grants from Acadia Pharmaceuticals, Neurocrine Bioscience, Navitor/Supernus and Janssen Research & Development, educational grant to serve as Section Editor of the Psychiatry & Behavioral Health Learning Network, consultant fees from Eleusis Therapeutics US, Inc, Janssen Global Services, Janssen Scientific Affairs, Worldwide Clinical Trials/Eliem, and Guidepoint Global, and honoraria from North American Center for Continuing Medical Education, Medscape/WebMD, Clinical Care Options, and Global Medical Education. Dr. Matthew Klein is currently a full-time employee of Janssen Research & Development. Dr. Klein conducted all work related to this manuscript during his previous position as a full-time employee of the ISMMS. In the past 5 years, Dr. Murrough has provided consultation services and/or served on advisory boards for Boehreinger Ingelheim, Clexio Biosciences, FSV7, Global Medical Education (GME), Otsuka, and Sage Therapeutics. Dr. Murrough is named on a patent pending for neuropeptide Y as a treatment for mood and anxiety disorders and on a patent pending for the use of ezogabine and other KCNQ channel openers to treat depression and related conditions. All the other authors report no conflict of interest.

Acknowledgments

We thank Prof Richard Foltin, Prof Charles Gillespie and Prof Thaddeus Pace for their assistance in the implementation and setup of the Trier Social Stress Task. We also thank the nursing personnel from the Clinical Research Facility and the Department of Psychiatry at the Icahn School of Medicine at Mount Sinai for their extensive work on this project.

Contributor Information

Sara Costi, Email: sara.costi@mssm.edu.

James W. Murrough, Email: james.murrough@mssm.edu.

Data availability

De-identified participant data collected during the trial will be available for data meta-analysis upon review of the project proposal by the corresponding authors, Drs. James Murrough and Sara Costi

References

  1. Alipoor M., Loripoor M., Kazemi M., Farahbakhsh F., Sarkoohi A. The effect of ketamine on preventing postpartum depression. J. Med. Life. 2021;14(1):87–92. doi: 10.25122/jml-2020-0116. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Allen A.P., Kennedy P.J., Cryan J.F., Dinan T.G., Clarke G. Biological and psychological markers of stress in humans: focus on the Trier Social Stress Test. Neurosci. Biobehav. Rev. 2014;38:94–124. doi: 10.1016/j.neubiorev.2013.11.005. [DOI] [PubMed] [Google Scholar]
  3. Amat J., Dolzani S.D., Tilden S., Christianson J.P., Kubala K.H., Bartholomay K., Sperr K., Ciancio N., Watkins L.R., Maier S.F. Previous ketamine produces an enduring blockade of neurochemical and behavioral effects of uncontrollable stress. J. Neurosci.: the official journal of the Society for Neuroscience. 2016;36(1):153–161. doi: 10.1523/JNEUROSCI.3114-15.2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. American Psychological Association The road to resilience. 2018. https://www.apa.org/helpcenter/road-resilience Available at:
  5. Beck A.T., Epstein N., Brown G., Steer R.A. An inventory for measuring clinical anxiety: psychometric properties. J. Consult. Clin. Psychol. 1988;56(6):893–897. doi: 10.1037//0022-006x.56.6.893. [DOI] [PubMed] [Google Scholar]
  6. Bellingrath S., Rohleder N., Kudielka B.M. Healthy working school teachers with high effort-reward-imbalance and overcommitment show increased pro-inflammatory immune activity and a dampened innate immune defense. Brain Behav. Immun. 2010;24(8):1332–1339. doi: 10.1016/j.bbi.2010.06.011. [DOI] [PubMed] [Google Scholar]
  7. Brachman R.A., McGowan J.C., Perusini J.N., Lim S.C., Pham T.H., Faye C., Gardier A.M., Mendez-David I., David D.J., Hen R., Denny C.A. Ketamine as a prophylactic against stress-induced depressive-like behavior. Biol. Psychiatr. 2016;79(9):776–786. doi: 10.1016/j.biopsych.2015.04.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bremner J.D., Krystal J.H., Putnam F.W., Southwick S.M., Marmar C., Charney D.S., et al. Measurement of dissociative states with the clinician-administered dissociative states scale (CADSS) J. Trauma Stress. 1998;11(1):125–136. doi: 10.1023/A:1024465317902. [DOI] [PubMed] [Google Scholar]
  9. Camargo A., Dalmagro A.P., de Souza M.M., Zeni A., Rodrigues A. Ketamine, but not guanosine, as a prophylactic agent against corticosterone-induced depressive-like behavior: possible role of long-lasting pro-synaptogenic signaling pathway. Exp. Neurol. 2020;334 doi: 10.1016/j.expneurol.2020.113459. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cohen J. Statistical power analysis for the behavioral sciences. 2nd. Lawrence Erlbaum Associates; New Jersey, NJ: 1988. [Google Scholar]
  11. Cox J.L., Holden J.M., Sagovsky R. Detection of postnatal depression. Development of the 10-item Edinburgh postnatal depression scale. Br. J. Psychiatr.: J. Ment. Sci. 1987;150:782–786. doi: 10.1192/bjp.150.6.782. [DOI] [PubMed] [Google Scholar]
  12. Duman R.S., Aghajanian G.K., Sanacora G., Krystal J.H. Synaptic plasticity and depression: new insights from stress and rapid-acting antidepressants. Nat. Med., 22(3), 238-249. doi: 10.1038/nm.4050. [DOI] [PMC free article] [PubMed]
  13. First M.B., Williams J.B.W., Karg R.S., Spitzer R.L. American Psychiatric Association; Arlington, VA: 2015. Structured Clinical Interview for DSM-5—Research Version (SCID-5 for DSM-5, Research Version; SCID-5-RV) [Google Scholar]
  14. Hamlett Anthony, Ryan Louise, Wolfinger Russ. Proceedings of the Statistics and Data Analysis Section. SAS Users Group International; 2004. On the use of PROC MIXED to estimate correlation in the presence of repeated measures; pp. 1–7. [Google Scholar]
  15. Hasin D.S., Sarvet A.L., Meyers J.L., Saha T.D., Ruan W.J., Stohl M., Grant B.F. Epidemiology of adult DSM-5 major depressive disorder and its specifiers in the United States. JAMA Psychiatr. 2018;75(4):336–346. doi: 10.1001/jamapsychiatry.2017.4602. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Hellhammer D.H., Wüst S., Kudielka B.M. Salivary cortisol as a biomarker in stress research. Psychoneuroendocrinology. 2009;34(2):163–171. doi: 10.1016/j.psyneuen.2008.10.026. [DOI] [PubMed] [Google Scholar]
  17. Insel T.R. The NIMH research domain criteria (RDoC) project: precision medicine for psychiatry. Am. J. Psychiatr. 2014;171:395–397. doi: 10.1176/appi.ajp.2014.14020138. [DOI] [PubMed] [Google Scholar]
  18. Julious S.A. Sample size of 12 per group rule of thumb for a pilot study. Pharmaceut. Stat. 2005;4:287–291. [Google Scholar]
  19. Kass J.D., Friedman R., Leserman J., Caudill M., Zuttermeister P.C., Benson H. An Inventory of Positive Psychological Attitudes with potential relevance to health outcomes: validation and preliminary testing. Behav. Med. 1991;17(3):121–129. doi: 10.1080/08964289.1991.9937555. [DOI] [PubMed] [Google Scholar]
  20. Khalili-Mahani N., Martini C.H., Olofsen E., Dahan A., Niesters M. Effect of subanaesthetic ketamine on plasma and saliva cortisol secretion. Br. J. Anaesth. 2015;115(1):68–75. doi: 10.1093/bja/aev135. [DOI] [PubMed] [Google Scholar]
  21. Kirschbaum C., Pirke K.M., Hellhammer D.H. The 'Trier Social Stress Test'--a tool for investigating psychobiological stress responses in a laboratory setting. Neuropsychobiology. 1993;28(1–2):76–81. doi: 10.1159/000119004. [DOI] [PubMed] [Google Scholar]
  22. Krzystyniak A., Baczynska E., Magnowska M., Antoniuk S., Roszkowska M., Zareba-Koziol M., Das N., Basu S., Pikula M., Wlodarczyk J. Prophylactic ketamine treatment promotes resilience to chronic stress and accelerates recovery: correlation with changes in synaptic plasticity in the CA3 subregion of the Hippocampus. Int. J. Mol. Sci. 2019;20(7):1726. doi: 10.3390/ijms20071726. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Lorr M., McNair D. Educational and Industrial Testing Service; San Diego, CA: 1984. Profile of Mood States-Bipolar Form (POMS-BI) 1988. [Google Scholar]
  24. Lorr M., McNair D.M., Heuchert J.W.P. Multi-Health Systems (MHS); North Tonawanda, NY: 2003. Profile of Mood States: Bi-polar Form. [Google Scholar]
  25. Ma J.H., Wang S.Y., Yu H.Y., Li D.Y., Luo S.C., Zheng S.S., Wan L.F., Duan K.M. Prophylactic use of ketamine reduces postpartum depression in Chinese women undergoing cesarean section. Psychiatr. Res. 2019;279:252–258. doi: 10.1016/j.psychres.2019.03.026. [DOI] [PubMed] [Google Scholar]
  26. Marques A., Bordado J., Peralta M., Gouveia E.R., Tesler R., Demetriou Y., Gomez Baya D. Cross-sectional and prospective relationship between physical activity and depression symptoms. Sci. Rep. 2020;10(1) doi: 10.1038/s41598-020-72987-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Mastrodonato A., Martinez R., Pavlova I.P., LaGamma C.T., Brachman R.A., Robison A.J., Denny C.A. Ventral CA3 activation mediates prophylactic ketamine efficacy against stress-induced depressive-like behavior. Biol. Psychiatr. 2018;84(11):846–856. doi: 10.1016/j.biopsych.2018.02.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. McGowan J.C., LaGamma C.T., Lim S.C., Tsitsiklis M., Neria Y., Brachman R.A., Denny C.A. Prophylactic ketamine attenuates learned fear. Neuropsychopharmacology: official publication of the American College of Neuropsychopharmacology. 2017;42(8):1577–1589. doi: 10.1038/npp.2017.19. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. McGowan J.C., Hill C., Mastrodonato A., LaGamma C.T., Kitayev A., Brachman R.A., Narain N.R., Kiebish M.A., Denny C.A. Prophylactic ketamine alters nucleotide and neurotransmitter metabolism in brain and plasma following stress. Neuropsychopharmacology: official publication of the American College of Neuropsychopharmacology. 2018;43(9):1813–1821. doi: 10.1038/s41386-018-0043-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Moda-Sava R.N., Murdock M.H., Parekh P.K., Fetcho R.N., Huang B.S., Huynh T.N., Witztum J., Shaver D.C., Rosenthal D.L., Alway E.J., Lopez K., Meng Y., Nellissen L., Grosenick L., Milner T.A., Deisseroth K., Bito H., Kasai H., Liston C. Sustained rescue of prefrontal circuit dysfunction by antidepressant-induced spine formation. Science, 364(6436), eaat8078. doi: 10.1126/science.aat8078. [DOI] [PMC free article] [PubMed]
  31. Overall J., Gorham D. The brief psychiatric rating scale. Psychol. Rep. 1962;10:799–812. [Google Scholar]
  32. Pace T.W., Mletzko T.C., Alagbe O., Musselman D.L., Nemeroff C.B., Miller A.H., Heim C.M. Increased stress-induced inflammatory responses in male patients with major depression and increased early life stress. Am. J. Psychiatr. 2006;163(9):1630–1633. doi: 10.1176/ajp.2006.163.9.1630. [DOI] [PubMed] [Google Scholar]
  33. Watson D., Clark L.A., Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J. Pers. Soc. Psychol. 1988;(54):1063–1070. doi: 10.1037//0022-3514.54.6.1063. [DOI] [PubMed] [Google Scholar]
  34. Young E.A., Lopez J.F., Murphy-Weinberg V., Watson S.J., Akil H. Hormonal evidence for altered responsiveness to social stress in major depression. Neuropsychopharmacology: official publication of the American College of Neuropsychopharmacology. 2000;23(4):411–418. doi: 10.1016/S0893-133X(00)00129-9. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

De-identified participant data collected during the trial will be available for data meta-analysis upon review of the project proposal by the corresponding authors, Drs. James Murrough and Sara Costi

Articles from Neurobiology of Stress are provided here courtesy of Elsevier

RESOURCES