Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Sep 1.
Published in final edited form as: Neurol Psychiatry Brain Res. 2020 Jun 11;37:33–40. doi: 10.1016/j.npbr.2020.05.002

Analyzing Non-verbal Behavior Throughout Recovery in a Sample of Depressed Patients Receiving Deep Brain Stimulation

Micaela V McCall 1, Patricio Riva-Posse 1, Steven J Garlow 2, Helen S Mayberg 3, Andrea L Crowell 1,*
PMCID: PMC7375407  NIHMSID: NIHMS1603557  PMID: 32699489

Abstract

Background

Traditional rating scales for depression rely heavily on patient self-report, and lack detailed measurement of non-verbal behavior. However, there is evidence that depression is associated with distinct non-verbal behaviors, assessment of which may provide useful information about recovery. This study examines non-verbal behavior in a sample of patients receiving Deep Brain Stimulation (DBS) treatment of depression, with the purpose to investigate the relationship between non-verbal behaviors and reported symptom severity.

Methods

Videotaped clinical interviews of twelve patients participating in a study of DBS for treatment-resistant depression were analyzed at three time points (before treatment and after 3 months and 6 months of treatment), using an ethogram to assess the frequencies of 42 non-verbal behaviors. The Beck Depression Inventory (BDI) and Hamilton Depression Rating Scale (HDRS-17) were also collected at all time points.

Results

Factor analysis grouped non-verbal behaviors into three factors: react, engage/fidget, and disengage. Two-way repeated measures ANOVA showed that scores on the three factors change differently from each other over time. Mixed effects modelling assessed the relationship between BDI score and frequency of non-verbal behaviors, and provided evidence that the frequency of behaviors related to reactivity and engagement increase as BDI score decreases.

Limitations

This study assesses a narrow sample of patients with a distinct clinical profile at limited time points.

Conclusions

Non-verbal behavior provides information about clinical states and may be reliably quantified using ethograms. Non-verbal behavior may provide distinct information compared to self-report.

1. Introduction

Major depressive disorder (MDD) is a debilitating psychiatric illness that can severely affect quality of life. MDD has a lifetime prevalence of up to 20% (Kessler et al., 2003), and a considerable number of patients do not achieve clinical remission even after multiple treatment interventions (Holtzheimer and Mayberg, 2011; Rush et al., 2006). These patients thus are considered to have treatment resistant depression (TRD). Patients with TRD have more medication trials, more hospitalizations, greater medical costs, and higher rates of disability and suicide (Amital et al., 2008; Johnston et al., 2019).

Deep brain stimulation (DBS) is being investigated as an intervention for patients with TRD. Mayberg et al. (2005) described the first open-label study to assess the effectiveness of DBS for TRD, with four of six patients responding to chronic stimulation of the white matter tracts adjacent to the subgenual cingulate gyrus. While many open-label results have described promising results in this and other brain targets, randomized double-blind DBS trials have not met their primary clinical endpoints (Crowell et al., 2019; Holtzheimer et al., 2017; Kisely et al., 2018). An emerging issue is the need for more objective and quantitative markers to capture antidepressant effects, an important goal across antidepressant research studies (Beijers et al., 2019; Fonseka et al., 2018; Mora et al., 2018; Voegeli et al., 2017).

Antidepressant treatment trials measure treatment response using mood rating scales. Commonly, scales such as the Hamilton Depression Rating Scale (HDRS-17) (Hamilton, 1960) and the Beck Depression Inventory (BDI) (Beck et al., 1996) have been used to measure the antidepressant efficacy of DBS. Both are highly reliable, valid measures for tracking response and recovery, in terms of internal reliability and retest reliability (Bagby et al., 2004; Wang and Gorenstein, 2013). The HDRS-17 is a clinician-administered scale, although it relies heavily on patient report, whereas the BDI is entirely dependent on patient self-report. Thus, evaluation based on rating scales is contingent upon how patients understand and communicate their experiences (Fiquer et al., 2013). This reliance on patients’ own perceptions of their symptom severity and understanding of the meaning of clinically relevant symptoms descriptors is a limitation of depression rating scales because individuals may not be fully aware of or able to accurately describe their experience of depression, external observable signs of depression, or the extent of their impairment due to the illness.

Non-verbal behavior accounts for more than 60% of human communication, and much of this behavior (e.g. facial expressions, posture) is unconscious (Geerts and Brune, 2009). Individuals with MDD may display distinct non-verbal behaviors, given the psychomotor symptoms of the illness (Sobin and Sackeim, 1997). Such behaviors are not fully captured by depression rating scales. The HDRS-17 includes a measure of psychomotor retardation and agitation; however, single omnibus scores like this lack specificity and can be interpreted differently by interviewers. BDI questions about agitation and energy loss are, as mentioned, contingent on patient awareness. Therefore, these scales’ lack of measures of non-verbal behaviors and their reliance on self-report may be obscuring important information about the experience and behavior of patients with MDD.

A better understanding of the psychomotor behavior associated with MDD has the potential to assist clinicians in selecting appropriate individuals for particular treatments and in assessing treatment response in specific patient populations. For example, clinicians have noticed that depressed patients who respond to DBS are characterized by psychomotor slowing, slow speech, and limited reactivity (Crowell et al., 2015). While these symptoms are associated with MDD, clinical application of this association is still limited and lacks methodological consistency. One useful methodological approach is ethology. An ethogram is a catalogue of discrete behaviors that can be used to systematically and quantitatively analyze non-verbal behavior (Geerts and Brune, 2009). Geerts and Brune (2009) have argued that psychiatrists intuitively use ethology often in assessment and diagnosis. However, intentional use of ethological methods could bring more objectivity to the judgements made by clinicians and researchers.

Bouhuys and van den Hoofdakker (1991) developed an ethogram for use with human subjects during clinical interviews of depressed patients in order to investigate the interrelatedness between the behavior of the patient and the psychiatrist. They used information about the duration and frequency of behaviors during speaking and listening to investigate the mutual influences of the patients’ and clinicians’ behavior. Several other researchers have used similar ethograms to analyze the behavior of depressed patients, naturalistically in inpatient settings and in formal psychiatric interviews (Bouhuys and van den Hoofdakker, 1991; Fiquer et al., 2013; Fiquer et al., 2017; Fossi et al., 1984; Gaebel and Wolwer, 2004; Pedersen et al., 1988). Notably, Fiquer et al. (2013) used an ethogram to analyze clinical interviews over the course of a non-invasive neuromodulation treatment. They found that several non-verbal behaviors track with scores on depression rating scales before and after treatment. The existing research utilizing ethograms has much heterogeneity in patient populations and observational contexts. Determining the applicability of such methods requires evaluating them for use in a specific context with a specific patient population.

In the current study, we used an ethogram to assess the change in the non-verbal behavior over time in patients participating in a study of DBS for TRD. We hypothesized that an ethogram would be a suitable method to observe multiple distinct dimensions of non-verbal behavior and record changes in non-verbal behavior throughout DBS treatment. Additionally, we assessed the relationship between patients’ non-verbal behavior and their perception of their depressive symptoms. We selected BDI score as the measure of patient perception of depression because it is exclusively self-report and does not include the potential confounding factor of clinician interpretation. We hypothesized that different dimensions of non-verbal behaviors would change differently over time with respect to a patient’s perception of their depression severity, such that patients would display more behaviors associated with social engagement and fewer behaviors associated with sadness or apathy as their BDI decreased (depression improved). In studying this association, we sought to define the relationship between externally observable behaviors associated with depression and the internal experience of depression (as captured by the BDI). This analysis has implications about the importance of assessing observable non-verbal behavior to complement and enhance self-report assessments of depression.

2. Methods

2.1. Patient selection and clinical assessments

Twelve participants in the investigational protocol “Deep Brain Stimulation (DBS) for Treatment Resistant Depression” (Clinical trials.gov NCT00367003, NCT01984710) were analyzed. All participants gave written, informed consent for participation in this study, which was approved by the Emory Institutional Review Board. Patients with chronic and severe treatment-resistant unipolar depression were offered inclusion in the protocol consisting of surgical implantation of deep brain stimulation leads in the subcallosal cingulate region. Inclusion and exclusion criteria for participation have been described previously (Riva-Posse et al., 2018). Briefly, participants met DSM IV criteria for Major Depressive Disorder and had a severity score of ≥20 on the 17 item Hamilton Depression Rating Scale (HDRS-17). Participants were in a current major depressive episode of at least 12 months duration and had failed to respond to at least four different antidepressant treatments. Patients were allowed to continue medication during the study, which could be changed only if side effects developed. The primary endpoint of this open label protocol is improvement in depressive symptoms, as measured by the HDRS-17, after 6 months of stimulation, which is started one month after surgery. Treatment response in the clinical DBS protocol is defined as a 50% decrease in HDRS-17 score. Participants included in this study of non-verbal behavior were implanted between 2012 and 2016. Clinical results from a portion of this cohort (6 of 12 participants) have been previously published (Riva-Posse et al., 2018). Electrophysiological characteristics from four other participants in this cohort have also been described (Veerakumar et al., 2019).

Patients were interviewed on a weekly basis, starting one month prior to surgery and for 6 months after the DBS device was turned on. Stimulation was initiated one month after implantation surgery and delivered continuously throughout this study period. Weekly clinical assessments included an interview with the study psychiatrists (ALC, SJG, PR-P) and administration of mood rating scales, including the HDRS-17 and BDI.

2.2. Audiovisual recordings

Video of the patient’s face and upper torso was recorded during each clinical interview. Videos were recorded using a Canon Vixia HF R600 digital video camera; 30 frames/second. For this study, three videos were analyzed for each participant: 1 week before surgery (Baseline), 3 months after initiation of chronic stimulation (3Mo), and 6 months after initiation of chronic stimulation (6Mo).

2.3. Ethogram

Behaviors included in this study’s ethogram were selected from a thorough literature search. Those behaviors that had the most significant and consistent relationship with a traditional outcome score, or were included in several of the reviewed studies, were prioritized for inclusion (Bouhuys and van den Hoofdakker, 1991; Fiquer et al., 2013; Fossi et al., 1984; Troisi, 1999). The ethogram was based partially on the Ethological Coding System for Interviews, described in Troisi (1999). Inclusion focused on behaviors related to eye, face, head, shoulder, and arm movement, posture, and speech. (See Table 2.)

Table 2.

Ethogram

Behavior Description
Eyes
Looking in the direction of the face of the other person
Look away from the interviewer
Look down at feet, lap, or floor
Flash eyebrows quick raising and lowering of eyebrows
Raised eyebrows eyebrows stay up for at least 2 seconds
Furrowed brow wrinkles appear between eyebrows, not an emotional frown
Head Movement
Yes nodding
No shaking
Head to side head is tilted to the side
Head down head is tilted down
Head up head is tilted upward
Head aversion horizontal change in the head position 30 degrees or more to either the left or the right, from the line from interviewee to interviewer
Head bob sharp upward movement, like an inverted nod
Mouth
Symmetric smile smiling in which the muscle that orbits the eye is active in addition to the muscle that pulls the lip corners up; mouth corner retraction occurs in a similar and synchronous way
Asymmetric smile smiling in which the muscle orbiting the eye is not active, mouth corner retraction occurs in an uneven way, showing a “wry” smile
Lip corners down lips are angled down at the corners, usually stretched horizonally
Tight lips lips are pressed together
Lip corners back corners back but not drawn up in a smile
Lip corners in corners of the mouth drawn inward
Twist mouth lips are closed, pushed forward, and twisted to one side
Lick lips tongue passes over lips
Bite lips one lip drawn into the mouth and held there between the teeth
Expression
Frown eyebrows are drawn together, lowered at the center, lip corners down
Cry sad face (downturned mouth and lowered brows), including crying
Laugh mouth corners drawn up and out, lips part to reveal teeth
Arm Gestures
Illustrative gestures hand and arm movement used to support speech
Object using hand movement purposeful manual activity such as eating/drinking
Shrug Body touching makes contact with the body
Groom hands picking or scratching at hair, face, or body. Includes combing hair
Touch hair hands touch or rub hair on head
Touch mouth hands make contact with mouth
Touch face hands touch or rub head or neck
Posture
Lean forward from the hips towards the interviewer
Posture shift (settle) shift not forward or backwards; sliding across chair at least 1 inch; slouching movement; curving of the spine; abrupt straightening of the spine
Speech
Elaborative speaking answering the question in more than one sentence
Slow speaking speaking with a <1–1 second pause between each word
Silence 3 seconds or more before start of speech or without subsequent speech; not related to listening
Pause 2–3 second silence in the middle of two segments of speech
Verbal backchannel affirmative vocalizations produced while listening
Open in a new tab

The ethogram used to analyze the non-verbal behavior in the videos of patient interviews. Contains behaviors shown in the literature to be related to depression, as well as descriptions of their constituent elements. Bolded items were used in the final data analysis; non-bolded items occurred very infrequently and were excluded from further analysis.

2.4. Collection of non-verbal behavior data

Videos were analyzed using our ethogram by a single observer (MM) who was blinded to the time-point at which each video was recorded. The first 8 minutes of each video, starting when the psychiatrist asked the first question, were analyzed. The “one-zero” sampling method, described in Troisi (1999), was used to record behaviors described in the ethogram. The 8-minute observations were split into 32 fifteen-second samples, and it was recorded whether each behavior occurred in each sample. For each video, each behavior received a score that was the proportion of the total samples in which the behavior occurred.

2.5. Data Analysis

Data were analyzed using RStudio. Plots were made using the ggplot2 package (Wickham, 2009).

Factor analysis was performed to reduce the number of dimensions and create new predictors for performing regression analyses. Exploratory factor analysis (principal axis factoring method, oblimin rotation) was used to explore covariances in behavior scores and revealed that the behaviors in the ethogram could be reduced down to three latent dimensions or “factors”. Factors consisted of non-verbal behaviors with a factor loading of > 0.3 or < −0.3 (the loading is a standardized parameter estimate of the correlation between the observed behavior and the behavioral factor). A confirmatory factor analysis (robust maximum likelihood (MLM) estimator) model was then constructed to generate scores for each patient on each latent factor at each time-point. All subsequent analyses utilized these factor scores. The R packages used were: psych (Revelle, 2017), GPArotation (Bernaards and Jennrich, 2005), nFactors (Raiche, 2010), and lavaan (Rosseel, 2012).

Repeated measures two-way analysis of variance (ANOVA) was performed to assess the differences between scores on the three non-verbal factors at the three time-points. The R package used was ez (Lawrence, 2016). Post-hoc paired t-tests were performed to assess the change in score on each non-verbal factor separately over time. These t-tests compared the scores between Baseline and 6Mo and the change in score in each of the two time intervals (baseline to 3Mo, and 3Mo to 6Mo) for each factor. A Bonferroni corrected type 1 error threshold of 0.0083 for six comparisons was used to determine significance.

Linear mixed effects modelling (LME) was used to examine the relationship between non-verbal behavior and reported depression severity. The R package used was lme4 (Bates et al., 2015). We constructed an LME model for BDI with each group of non-verbal behaviors (determined through factor analysis), time-point, and 2-way interactions between behavior and time-point included as fixed effects. Patient ID was included as a random intercept to account for variation between individuals. The significance of each fixed effect was determined using likelihood ratio tests of the full model with the effect in question against the model without the effect in question. Estimates and t-values for the coefficient of each fixed effect were obtained from the LME model parameters.

3. Results

3.1. Patient Demographics and Treatment Response

The twelve participants (9F, 3M) were largely Caucasian (1 African American) and their mean age was 54.8 years (range 35–70). At baseline, the average BDI score was 34.5 (±9.7) and the average HDRS-17 score was 22.4 (±3.2). After three months of stimulation, 8/12 participants were considered treatment responders. The average BDI was 16.8 (±12.9) and average HDRS-17 was 8.8 (±4.37). At the six-month endpoint, 10/12 participants were treatment responders and 8/12 participants were in remission from depression (HDRS-17 score ≤ 7). The average BDI score was 15.7 (±10.6) and average HDRS-17 was 8.5 (±3.5) (Table 1).

Table 1.

Treatment Response

BDI HDRS-17
Pre-op 34.5 (9.7) 22.42 (3.23)
3 months 16.83 (12.9) 8.83 (4.37)
6 months 15.67 (10.57) 8.5 (3.52)
Open in a new tab

Average BDI and HDRS-17 scores at each time point (standard deviation).

3.2. Ethogram

Forty behaviors gathered from previous studies were included and deliberately defined in this study’s ethogram. Behaviors that occurred in three or fewer observation points were discarded and not included in the analysis (see Table 2). The following low-frequency behaviors were added together into a single variable called touch head: groom, touch hair, touch mouth, touch face. In addition, asymmetric and symmetric smile measurements were combined to create a single smile variable due to difficulty reliably discriminating the two. Thirty behaviors remained after discarding and combining behaviors.

3.3. Factor analysis

Factor analysis revealed three latent factors. Twenty-two behaviors had a factor loading of > 0.3 or < −0.3 on one of the three factors (see Figure 1). Each factor consists of behaviors spanning across the groupings listed in the ethogram (Table 2) as well as different body parts (eyes, mouth, etc.). As described, these behavioral factors are based solely on covariances between behaviors over time, and not on conceptual groups. Keeping this in mind, the first factor was named the react factor because it contained many behaviors that typically occurred in response to a statement made by the interviewer, such as head to the side or smile. The second factor was called engage/fidget because it contained some behaviors that are indicative of social engagement, such as illustrative gestures and elaborative speaking, as well as several body-focused and iterative behaviors like biting lips. The third contained behaviors that are typically associated with emotional detachment from interpersonal interaction (looking down), and therefore was named disengage.

Figure 1.

Figure 1.

Open in a new tab

Diagram of factor analysis

Diagram of the loading (unidirectional arrows) of each non-verbal behavior (rectangles) on its latent dimension/factor (circles) according to the confirmatory factor analysis model. Each unidirectional arrow represents the standardized parameter estimate of the correlation between the observed behavior and the behavioral factor. Bidirectional arrows between factors indicate co-variances. Black arrows indicate positive loading, grey indicate negative loading, and width indicates the size of the loading.

3.4. Non-verbal behavior factors over time

Over the 6 months of observation, the average score on the react and engage/fidget factors increased (baseline average of 0.12 ± 0.09 and 0.09 ± 0.08, 6 month average of 0.17 ± 0.14 and 0.12 ± 0.08, respectively), while the average score on the disengage factor decreased (baseline average of 0.08 ± 0.14, 6 month average of 0.04 ± 0.09). Repeated measures two-way ANOVA revealed that there are significant differences between the three non-verbal factor scores in the way they changed over time (F4,44=2.92, p=0.032) (see Figure 2). On all three factors, the change occurred early, before the 3Mo time point (see Table 3). However, per our post hoc paired t-tests, only for the engage/fidget score were changes seen between baseline and 3Mo significantly greater than those seen between 3Mo and 6Mo (t = 3.21, p=0.0084).

Figure 2.

Figure 2.

Open in a new tab

Non-verbal factor scores over time

Non-verbal factor scores of each individual patient at each time point. Boxes show means and standard errors (middle and upper/lower bars, respectively). Repeated measures two-way ANOVA revealed significant differences between the three non-verbal factor scores (F2,22= 8.10, p<0.01), as well as a significant difference between the non-verbal factors in the way their scores change over time (F4,44= 2.92, p=0.032).

Table 3.

Non-verbal factor scores at each time point

React Engage/Fidget Disengage
Pre-op 0.12 (0.09) 0.09 (0.08) 0.08 (0.14)
3 months 0.20 (0.10) 0.15 (0.07) 0.006 (0.06)
6 months 0.17 (0.14) 0.12 (0.08) 0.03 (0.09)
Open in a new tab

Average score on each non-verbal factor at each time point (standard deviation).

3.5. Association between BDI and non-verbal factor scores

In the LME model, fixed effects were time-point, engage/fidget score, react score, and the interaction between time-point and react score. Patient ID was included as a random intercept. This LME model showed that increasing engage/fidget score is associated with decreasing BDI score, i.e. a decrease in severity of depression (X2(2)= 8.05, p=0.005, b= −7.89 ±2.48 (SE), t = −3.18). LME also showed that react score is associated with a decrease in BDI score (X2(1)= 4.69, p=0.030, b=19.01 ±9.36 (SE), t = −3.10). Disengage score was not significantly associated with BDI score (X2(1)= 0.082, p=0.37), and so was not included in the model. Figure 3 shows the relationship between BDI and factor scores at different time-points.

Figure 3.

Figure 3.

Open in a new tab

BDI by non-verbal factor score over time

Non-verbal factor score (y-axis) and BDI score (x-axis) of each patient. Linear mixed effects modelling (LME) for BDI showed that increased react and engage/fidget scores are both significantly associated with clinical improvement as measured by BDI score (® = −7.89 ± 2.48 (SE), t = −3.18; ® =19.01 ± 9.36 (SE), t = −3.10, respectively). Disengage score is not significantly associated with BDI score.

Additionally, the relationship between react score and BDI is different at different time-points, i.e. there is an interaction between react score and time (X2(2)= 6.41, p=0.041). The negative association between BDI and react score significantly decreased in magnitude between baseline and the 3 month time-point (??=19.22 ±9.85 (SE), t=1.95), as well as between baseline and the 6 month time-point (??=23.48 ±8.98 (SE), t=2.61; the positive coefficients indicate the positive change in the slope of the regression line). See Figure 3.

4. Discussion

4.1. Summary

This study evaluated the non-verbal behavior of patients with treatment-resistant depression before and during the first 6 months of treatment with deep brain stimulation and investigated the relationship between non-verbal behavior and self-report depression ratings. In support of our first hypothesis, this study provides evidence that non-verbal behavior did change markedly across the three sampled time-points (Baseline, 3Mo, and 6Mo). Additionally, our data also partially supported our second hypothesis, providing evidence that a higher occurrence of behaviors associated with reactivity, social engagement, and fidgeting were associated with participants’ perception of lower depression severity as measured by BDI score. However, behaviors related to disengagement were not significantly related to BDI score.

4.2. Ethograms capture multiple dimensions of non-verbal behavior

An ethogram was used to collect quantitative data about the occurrence of forty non-verbal behaviors. Factor analysis revealed three groups of covarying behaviors associated with reactivity, social engagement - including iterative behaviors/fidgets, and social disengagement.

These three factors had different patterns of change over time. Factor analysis and repeated measures ANOVA show that non-verbal behaviors separate into different groups that change in different ways in relation to treatment response. Additionally, LME showed that behaviors associated with reactivity and social engagement increase as BDI decreases. Such evidence showing that non-verbal behaviors correlate with BDI validates the relevance of this scale; however, these results also show that it is important to consider non-verbal behavior in its own right in clinical assessment. These results agree with literature proposing that non-verbal behavior is multidimensional and that various groups of behaviors have distinct patterns of variation (Fiquer et al., 2013; Sobin and Sackeim, 1997; Ulrich and Harms, 1985). The lack of multiple, nuanced measures of non-verbal behavior in traditional depression scales such as the BDI is a significant shortcoming of such scales. While body language may be intuitively interpreted by clinicians, the current study supports the use of ethograms as a possible avenue for systematizing and quantifying such interpretations.

Advances in automated video analysis of facial expression, including the use of supervised and unsupervised machine learning approaches, provide the means for faster and less labor-intensive analysis of non-verbal behavior (Cohn et al., 2009). However, such approaches may sacrifice interpretability in the process of automation (Harati et al., 2019; Kacem et al., 2018). The high face validity and interpretability of ethograms provide an important counterpoint to machine learning approaches.

Interestingly, the current study did not reveal an association between behaviors related to social disengagement and reported symptom severity. It is possible that the small number of discrete behaviors included in the disengage factor resulted in a low predictive value for this factor. Alternatively, this result could indicate that behaviors in this factor are not the most relevant in the context of clinical improvement, are necessary but insufficient for self-reported insight, or represent a feature not captured in the BDI. Crucially, in the cases where rating scales do include information about non-verbal behavior, it often is reported as retardation or references behaviors that are incidentally similar to those in the disenage factor. This fact emphasizes the importance of including various non-verbal measures; the non-verbal behaviors that are currently most often assessed may not be the most relevant ones.

4.3. Reported symptom severity and observed non-verbal behaviors

Interestingly, our analyses seem to support the idea that shifts in non-verbal behaviors across time are not always accompanied by shifts in reported depression severity. LME showed that overall, higher occurrence of behaviors associated with reactivity predicts lower depression severity. However, this negative association decreased in magnitude over the first 6 months of stimulation, and the majority of that shift occurred before the 3-month time point. This indicates that, while the average BDI score decreased substantially over the first 3 months of stimulation, there were a number of individuals who showed an increase in behaviors associated with reactivity that was not accompanied by a proportionate decrease in BDI. Additionally, post-hoc paired t-tests performed in conjunction with the repeated measures two-way ANOVA showed that the increase in behaviors associated with social engagement was significantly greater between baseline and 3Mo than between 3Mo and 6Mo. Together these results indicate that individuals demonstrated a significant change in non-verbal behavior before the 3 month time-point that may outstrip their reported decrease in symptom severity. This interpretation is consistent with anecdotal evidence that patients often appear or seem improved to others before they are aware of or would endorse the change themselves.

These results are also in line with findings from an unsupervised machine learning analysis of facial dynamics performed on videos from the same DBS subject pool (Harati et al., 2019). The (machine algorithm) identified a clear distinction between videos classified a priori as “depressed” and “improved” (classification based on HDRS scores), however a third hypothesized state “transitional” - occurring near the 3-month time point - tended to overlap with either the “depressed” or “improved” states and varied by individual subject. The authors raise the possibility that the process of recovery from severe depression includes a period of oscillation between depressed-appearing and well-appearing facial dynamics, even though depression ratings reflect a more linear, graduated response (severe, moderate, mild). While intriguing, such computational approaches often struggle to identify clinically relevant and interpretable features. Ethogram results from this population support the results of the computational approach with clearly defined and easy-to-interpret behavioral features.

Other authors have emphasized the distinction and potential discrepancy between reported symptom severity and non-verbal behavior. For instance, Troisi (1999) noted that in several examples, quantitative recording of patients’ behavior sometimes produce different results from rating scales. Also, Geerts and Brune (2009) state that behavior patterns can suggest clinical improvement (or deterioration) before the patient is aware of such improvement. Thus, one possible explanation for this observation in our sample is that recovery from depression is reflected in non-verbal behavior sooner than in conscious, and thus reported, experience. Again, this affirms the utility of attending to non-verbal behavior. In particular, clinicians may benefit from attending to those behaviors in the reactivity factor (head to the side, head bob (upwards), lip corners stretched back, tight lips, no-shaking, yes-nodding, laughing, smiling), given that these behaviors provide distinct information, as discussed above. While a systematic analysis of these behaviors offers a more accurate understanding of recovery overall, a simple awareness of the importance of these behaviors may inform clinical judgement during exams.

4.4. Limitations

This study assesses only a narrow sample of depressed patients. Therefore, it is relevant for a specific subtype of patients who are receiving DBS and provides insight into their treatment response time course. Due to the fact that psychomotor symptoms may be one of the strongest indicators of the melancholic subtype of depression (Sobin and Sackeim, 1997), this may be an important limitation when considering the relevance of these results for other samples of depressed patients.

However, the specificity of this study also increases its relevance for this clinical population of severely depressed patients. In fact, one of our goals was to understand the psychomotor symptoms related to this population in particular. For example, in this population, there was a lack of observed self-touching and self-grooming compared to depressed patients in other studies (Bouhuys and van den Hoofdakker, 1991; Fiquer et al., 2013; Ranelli and Miller, 1981; Troisi, 1999), such that few occurred often enough to be included in analysis. This study thus quantitatively reflects a feature of DBS patients described in Crowell et al. (2015): that they may be characterized by a distinct lack of reactivity. Information like this may also be relevant for researchers attempting to understand depression subtypes.

Another issue is the possibility that non-verbal behavior may change over time due to increasing familiarity with the interviewers. Given our small sample size and high rate of treatment response, it is not possible to disentangle the effect of time alone versus that of effective antidepressant treatment in this cohort. As this is likely to be a confounding factor in any such longitudinal studies, examining the non-verbal interactions between patient and clinician (such as in Bouhuys and van den Hoofdakker, 1991) would complement and inform studies that examine non-verbal behavior and clinical improvement.

4.5. Future directions

A crucial future direction for the analysis of non-verbal behavior in DBS patients is to determine when such a shift in behavior occurs, independently of scores on traditional depression rating scales. Determining the timing of the shift in non-verbal behavior may inform prognosis: could an early shift in non-verbal behavior indicate future response or remission? Further, non-verbal may be a useful biomarker for determining stimulation adjustments. For example, observed spikes in depression rating scores might motivate clinicians to adjust stimulation parameters, but if these scores do not accurately reflect the trajectory of recovery, such adjustments may not be helpful to the patient, and even impede recovery. In addition, ethograms could be used to investigate whether frequencies of non-verbal behaviors at baseline can predict response to treatment, which would help in DBS patient selection. Finally, the current study does not control for age and sex in its analyses. As there is evidence that both age and sex affect the expression of psychomotor symptoms (Sobin and Sackeim, 1997), future studies should address this limitation.

4.6. Conclusion

Non-verbal behavior is a rich and complex source of information. Analyzing it can give us a deeper understanding of recovery compared to the sole use of traditional rating scales. Results of this study suggest that assessing non-verbal behavior may help clinicians to better understand the trajectory of recovery from severe depression and may provide more information about depressive states than can be provided by traditional depression rating scales. An expanded evaluation of ethological methods in a larger sample of depressed patients would provide more conclusive evidence for their utility in diagnostic settings. Results here indicate that this is a promising venture.

Highlights:

  • Non-verbal behavior changes markedly throughout recovery from MDD

  • Ethograms capture multiple dimensions of non-verbal behavior

  • Non-verbal behavior can deepen traditional understandings of the recovery process

Acknowledgements

The authors would like to acknowledge TJ Murphy for providing statistical assistance, Michael Crutcher and Robert McCauley for guidance in the preparation of the manuscript, and Sinéad Quinn and Lydia Denison for the preparation and organization of research materials and data.

Role of the Funding Source

This study was supported by grants from the Hope for Depression Research Foundation, the Dana Foundation, and the National Institutes of Health (UH3NS103550) to Dr. Mayberg. DBS devices were donated by St. Jude Medical, Inc. (now Abbott Neuromodulation) and Medtronic, Inc. Funding sources had no involvement in the design, data collection, analysis, interpretation, writing, or publication of this study.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Disclosures

H. Mayberg receives consulting and Intellectual Property licensing fees from Abbott Labs. S. Garlow is a consultant and has been an investigator to Janssen Pharmaceutical company. M. McCall, P. Riva-Posse, and A. Crowell have no competing interests to declare.

References

  1. Amital D, Fostick L, Silberman A, Beckman M, & Spivak B (2008). Serious life events among resistant and non-resistant MDD patients. J Affect Disord, 110(3), 260–264. doi: 10.1016/j.jad.2008.01.006 [DOI] [PubMed] [Google Scholar]
  2. Bagby RM, Ryder AG, Schuller DR, & Marshall MB (2004). The Hamilton Depression Rating Scale: has the gold standard become a lead weight? Am J Psychiatry, 161(12), 2163–2177. doi: 10.1176/appi.ajp.161.12.2163 [DOI] [PubMed] [Google Scholar]
  3. Bates D, Maechler M, Bolker B, & Walker S (2015). Fitting linear mixed-effects models using lme4. J. Stat. Softw, 67(1), 1–48. [Google Scholar]
  4. Beck AT, Steer RA, & Brown GK (1996). Manual for the beck depression inventory-II. San Antonio, TX: Psychological Corporation, 1, 82. [Google Scholar]
  5. Beijers L, Wardenaar KJ, van Loo HM, & Schoevers RA (2019). Data-driven biological subtypes of depression: systematic review of biological approaches to depression subtyping. Mol Psychiatry, 24(6), 888–900. doi: 10.1038/s41380-019-0385-5 [DOI] [PubMed] [Google Scholar]
  6. Bernaards CA, & Jennrich RI (2005). Gradient projection algorithms and software for arbitrary rotation criteria in factor analysis. Educational and Psychological Measurement, 65(5), 676–696. [Google Scholar]
  7. Bouhuys AL, & van den Hoofdakker RH (1991). The interrelatedness of observed behavior of depressed patients and of a psychiatrist: an ethological study on mutual influence. J Affect Disord, 23(2), 63–74. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/1753038 [DOI] [PubMed] [Google Scholar]
  8. Cohn JF, Kruez TS, Matthews I, Yang Y, Nguyen MH, Padilla MT, … De la Torre F (2009). Detecting depression from facial actions and vocal prosody. Paper presented at the 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Crowell AL, Garlow SJ, Riva-Posse P, & Mayberg HS (2015). Characterizing the therapeutic response to deep brain stimulation for treatment-resistant depression: a single center long-term perspective. Front Integr Neurosci, 9, 41. doi: 10.3389/fnint.2015.00041 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Crowell AL, Riva-Posse P, Holtzheimer PE, Garlow SJ, Kelley ME, Gross RE, … Mayberg HS (2019). Long-Term Outcomes of Subcallosal Cingulate Deep Brain Stimulation for Treatment-Resistant Depression. Am J Psychiatry, 176(11), 949–956. doi: 10.1176/appi.ajp.2019.18121427 [DOI] [PubMed] [Google Scholar]
  11. Fiquer JT, Boggio PS, & Gorenstein C (2013). Talking bodies: nonverbal behavior in the assessment of depression severity. J Affect Disord, 150(3), 1114–1119. doi: 10.1016/j.jad.2013.05.002 [DOI] [PubMed] [Google Scholar]
  12. Fiquer JT, Moreno RA, Canales JZ, Cavalcanti A, & Gorenstein C (2017). Is nonverbal behavior in patients and interviewers relevant to the assessment of depression and its recovery? A study with Dutch and Brazilian patients. Psychiatry Res, 250, 59–64. doi: 10.1016/j.psychres.2017.01.053 [DOI] [PubMed] [Google Scholar]
  13. Fonseka TM, MacQueen GM, & Kennedy SH (2018). Neuroimaging biomarkers as predictors of treatment outcome in Major Depressive Disorder. J Affect Disord, 233, 21–35. doi: 10.1016/j.jad.2017.10.049 [DOI] [PubMed] [Google Scholar]
  14. Fossi L, Faravelli C, & Paoli M (1984). The ethological approach to the assessment of depressive disorders. J Nerv Ment Dis, 172(6), 332–341. doi: 10.1097/00005053-198406000-00004 [DOI] [PubMed] [Google Scholar]
  15. Gaebel W, & Wolwer W (2004). Facial expressivity in the course of schizophrenia and depression. Eur Arch Psychiatry Clin Neurosci, 254(5), 335–342. doi: 10.1007/s00406-004-0510-5 [DOI] [PubMed] [Google Scholar]
  16. Geerts E, & Brune M (2009). Ethological approaches to psychiatric disorders: focus on depression and schizophrenia. Aust N Z J Psychiatry, 43(11), 1007–1015. doi: 10.3109/00048670903270498 [DOI] [PubMed] [Google Scholar]
  17. Hamilton M (1960). A rating scale for depression. J Neurol Neurosurg Psychiatry, 23, 56–62. doi: 10.1136/jnnp.23.1.56 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Harati S, Crowell A, Huang Y, Mayberg H, & Nemati S (2019). Classifying Depression Severity in Recovery from Major Depressive Disorder via Dynamic Facial Features. IEEE J Biomed Health Inform. doi: 10.1109/JBHI.2019.2930604 [DOI] [PubMed] [Google Scholar]
  19. Holtzheimer PE, Husain MM, Lisanby SH, Taylor SF, Whitworth LA, McClintock S, … Mayberg HS (2017). Subcallosal cingulate deep brain stimulation for treatment-resistant depression: a multisite, randomised, sham-controlled trial. Lancet Psychiatry, 4(11), 839–849. doi: 10.1016/S2215-0366(17)30371-1 [DOI] [PubMed] [Google Scholar]
  20. Holtzheimer PE, & Mayberg HS (2011). Stuck in a rut: rethinking depression and its treatment. Trends Neurosci, 34(1), 1–9. doi: 10.1016/j.tins.2010.10.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Johnston KM, Powell LC, Anderson IM, Szabo S, & Cline S (2019). The burden of treatment-resistant depression: A systematic review of the economic and quality of life literature. J Affect Disord, 242, 195–210. doi: 10.1016/j.jad.2018.06.045 [DOI] [PubMed] [Google Scholar]
  22. Kacem A, Hammal Z, Daoudi M, & Cohn J (2018). Detecting Depression Severity by Interpretable Representations of Motion Dynamics. Proc Int Conf Autom Face Gesture Recognit, 2018, 739–745. doi: 10.1109/FG.2018.00116 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Kessler RC, Berglund P, Demler O, Jin R, Koretz D, Merikangas KR, … National Comorbidity Survey R (2003). The epidemiology of major depressive disorder: results from the National Comorbidity Survey Replication (NCS-R). JAMA, 289(23), 3095–3105. doi: 10.1001/jama.289.23.3095 [DOI] [PubMed] [Google Scholar]
  24. Kisely S, Li A, Warren N, & Siskind D (2018). A systematic review and meta-analysis of deep brain stimulation for depression. Depress Anxiety, 35(5), 468–480. doi: 10.1002/da.22746 [DOI] [PubMed] [Google Scholar]
  25. Lawrence M (2016). ez: Easy analysis and visualization of factorial experiments (R Package Version 4.4–0)[Computer software]. In. [Google Scholar]
  26. Mayberg HS, Lozano AM, Voon V, McNeely HE, Seminowicz D, Hamani C, … Kennedy SH (2005). Deep brain stimulation for treatment-resistant depression. Neuron, 45(5), 651–660. doi: 10.1016/j.neuron.2005.02.014 [DOI] [PubMed] [Google Scholar]
  27. Mora C, Zonca V, Riva MA, & Cattaneo A (2018). Blood biomarkers and treatment response in major depression. Expert Rev Mol Diagn, 18(6), 513–529. doi: 10.1080/14737159.2018.1470927 [DOI] [PubMed] [Google Scholar]
  28. Pedersen J, Schelde JT, Hannibal E, Behnke K, Nielsen BM, & Hertz M (1988). An ethological description of depression. Acta Psychiatr Scand, 78(3), 320–330. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/3195354 [DOI] [PubMed] [Google Scholar]
  29. Raiche G (2010). nFactors: An R package for parallel analysis and non graphical solutions to the Cattell scree test. R package version, 2(3). [Google Scholar]
  30. Ranelli CJ, & Miller RE (1981). Behavioral predictors of amitriptyline response in depression. Am J Psychiatry, 138(1), 30–34. doi: 10.1176/ajp.138.1.30 [DOI] [PubMed] [Google Scholar]
  31. Revelle WR (2017). psych: Procedures for personality and psychological research. [Google Scholar]
  32. Riva-Posse P, Choi KS, Holtzheimer PE, Crowell AL, Garlow SJ, Rajendra JK, … Mayberg HS (2018). A connectomic approach for subcallosal cingulate deep brain stimulation surgery: prospective targeting in treatment-resistant depression. Mol Psychiatry, 23(4), 843–849. doi: 10.1038/mp.2017.59 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Rosseel Y (2012). Lavaan: An R package for structural equation modeling and more. Version 0.5–12 (BETA). Journal of statistical software, 48(2), 1–36. [Google Scholar]
  34. Rush AJ, Trivedi MH, Wisniewski SR, Nierenberg AA, Stewart JW, Warden D, … Fava M (2006). Acute and longer-term outcomes in depressed outpatients requiring one or several treatment steps: a STAR*D report. Am J Psychiatry, 163(11), 1905–1917. doi: 10.1176/ajp.2006.163.11.1905 [DOI] [PubMed] [Google Scholar]
  35. Sobin C, & Sackeim HA (1997). Psychomotor symptoms of depression. Am J Psychiatry, 154(1), 4–17. doi: 10.1176/ajp.154.1.4 [DOI] [PubMed] [Google Scholar]
  36. Troisi A (1999). Ethological research in clinical psychiatry: the study of nonverbal behavior during interviews. Neurosci Biobehav Rev, 23(7), 905–913. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/10580305 [DOI] [PubMed] [Google Scholar]
  37. Ulrich G, & Harms K (1985). A video analysis of the non-verbal behaviour of depressed patients before and after treatment. J Affect Disord, 9(1), 63–67. doi: 10.1016/0165-0327(85)90011-4 [DOI] [PubMed] [Google Scholar]
  38. Veerakumar A, Tiruvadi V, Howell B, Waters AC, Crowell AL, Voytek B, … Mayberg HS (2019). Field potential 1/f activity in the subcallosal cingulate region as a candidate signal for monitoring deep brain stimulation for treatment resistant depression. J Neurophysiol. doi: 10.1152/jn.00875.2018 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Voegeli G, Clery-Melin ML, Ramoz N, & Gorwood P (2017). Progress in Elucidating Biomarkers of Antidepressant Pharmacological Treatment Response: A Systematic Review and Meta-analysis of the Last 15 Years. Drugs, 77(18), 1967–1986. doi: 10.1007/s40265-017-0819-9 [DOI] [PubMed] [Google Scholar]
  40. Wang YP, & Gorenstein C (2013). Psychometric properties of the Beck Depression Inventory-II: a comprehensive review. Braz J Psychiatry, 35(4), 416–431. doi: 10.1590/1516-4446-2012-1048 [DOI] [PubMed] [Google Scholar]
  41. Wickham H (2009). Elegant graphics for data analysis (ggplot2). In: New York, NY: Springer-Verlag. [Google Scholar]

RESOURCES