Frequency and predictors of virtual hope box use in individuals experiencing suicidal ideation: An ecological momentary assessment investigation

Abstract

Introduction:

The Virtual Hope Box (VHB) is a smartphone application designed to support emotion regulation when one is distressed, in a crisis, or experiencing suicidal ideation (SI). Initial proof of concept studies indicate that individuals are more likely to use the VHB than traditional hope boxes, and find it both easy to setup and helpful. To our knowledge, no studies have harnessed ambulatory assessment methodology to assess VHB use as it relates to incidence of suicidal thinking.

Methods:

As such, we recruited N = 50 undergraduates who endorsed SI either the past year or past 2 weeks to complete a 10-day investigation. At baseline, participants were oriented to the VHB and instructed on how to use the application. Over the next 10 days, participants responded to prompts five times per day on their personal smartphones regarding their current experiences of SI and stress as well as VHB usage.

Results:

Results found that most participants used the VHB at least once, rated its usefulness as high, and rated their perceived likelihood of future use as high. In addition, increases in state SI severity were related to subsequent VHB use.

Conclusion:

The VHB may be a useful tool for managing crises in undergraduates experiencing suicidal thoughts.

INTRODUCTION

A “hope box” is a tool often used in cognitive behavioral therapy and dialectical behavior therapy as a strategy to increase emotion regulation abilities when one is distressed or experiencing suicidal ideation (SI). Traditionally, a physical hope box contains reminders of important social connections (e.g., photographs), uplifting memories and life experiences (e.g., movie tickets), distraction tools (e.g., puzzles), and reminders of reasons for living (Wenzel et al., 2009). Berk et al. (2004) conducted a series of case studies on cognitive therapy for patients who had previously attempted suicide. One patient noted the creation of a hope box containing “… pictures of her family members, scriptures, and gifts from people she cherished, [and] an empty pill bottle, symbolizing her choice to live,” which she reported using during times of high distress and severe SI (Berk et al., 2004, p. 274). Another patient listed the hope box she had created as one of the most useful strategies gleaned from therapy (Berk et al., 2004).

The Virtual Hope Box (VHB; Bush et al., 2015, 2017) is a mobile adaptation of the traditional hope box. Similar to the physical hope box, the VHB contains six primary sections aimed at increasing emotion regulation and distress tolerance: visual reminders (e.g., photographs, videos) of important people and memories, distraction tools (e.g., picture puzzles, word search, Sudoku), relaxation exercises (e.g., guided meditation, guided progressive muscle relaxation), preloaded inspirational quotes (users can also enter their own quotes or statements found to be comforting), coping cards (adaptive behaviors to be used in times of crisis), and supportive contacts (e.g., National Suicide Prevention Lifeline, user-entered contacts). This mobile adaptation is consistent with the growing demand for smartphone-based mental health care (Figueroa & Aguilera, 2020), and unlike the physical hope box, it does not require the user to carry around numerous physical items. Further, past research has demonstrated that the time between experiencing SI and engaging in a suicide-related behavior for many individuals is small (i.e., less than 10 min; Deisenhammer et al., 2008), suggesting that just-in-time coping tools such as the VHB could play a critical role in managing patient safety.

Although only a handful of studies have analyzed the VHB, early research suggests it is a useful tool for promoting coping behaviors in individuals considering suicide. Bush et al. (2015) tested the VHB in a sample of 18 US Veterans deemed “high risk” for suicide (i.e., Veterans being treated in a DBT clinic with diagnoses such as borderline personality disorder and bipolar disorder). Veterans were randomized to either receive the traditional hope box intervention or the VHB. Bush et al. (2015) found that all but one participant used the VHB at least once daily during the study period, with the most frequently used components being the “distract me” and “remind me” components (Bush et al., 2015). Further, they found that the majority of participants either preferred the VHB to the traditional hope box or preferred to use both, found the VHB to be useful for coping with emotional distress, and were likely to refer the VHB to a friend (Bush et al., 2015). In qualitative feedback, participants indicated that the VHB “coping cards reminded me that it is okay to not be perfect,” “helped to soothe when thought about cutting,” and that “[inspirational] quotes helped turn a negative self-image into a positive self-talk” (Bush et al., 2015, p. 7). Of note, this study did not assess the VHB's impact on SI.

Bush et al. (2017) followed up on this study in a randomized controlled trial of VHB versus informational handouts for coping with SI in outpatient Veterans deemed at risk for suicide (e.g., referred from dialectical behavior therapy treatment; recently hospitalized for suicidal thoughts and/or behaviors). The results indicated that, while SI severity throughout the study time period did not significantly differ between treatment groups, the VHB condition was associated with significantly greater coping self-efficacy at three and 12 weeks follow-up, supporting the tool as a SI management strategy. Relatedly, in a sample of Veterans experiencing SI, Denneson et al. (2019) found that VHB use was related to decreased SI severity through its impact on coping self-efficacy. Interestingly, this protective effect was strongest in the first 6 weeks of the study period but did not account for changes in SI severity at week 12.

Early research on the VHB thus suggests it may be a useful tool for coping with distress and SI; however, there are notable gaps in the research literature. Most evident is that all previous studies have been conducted on Veteran samples, limiting our understanding of how other populations (e.g., college students, psychiatric inpatients and outpatients, the general adult population) interact with the app. Second, little is known regarding whether individuals tend to use the app as providers intend it to be used (e.g., when experiencing SI or increased intensity of SI, during increased times of stress, etc.). The creators of the VHB indicate it was “designed … to help a user restore emotional equilibrium during instances of distress,” and determining if indeed participants use the app in response to distress (e.g., stress, SI) is warranted (Bush et al., 2017, p. 330). Ambulatory assessment may be useful for determining such temporal relations, and no studies to our knowledge have conducted such analyses.

The purpose of the current study was to investigate VHB use in a sample of undergraduate college students with recent experiences with SI. Specifically, we sought to use ambulatory assessment to (1) observe the extent to which participants interact with the VHB (e.g., percentage of participants who used the app over the study period; average number of times the app was used per participant); (2) determine participant experiences with the app (i.e., if they perceived it to be useful; if they predict they will use it in the future); and (3) determine if changes in state perceived stress, state wish to die, and state SI severity were related to subsequent VHB use at the following time point (i.e., T + 1). Based on previous research (e.g., Bush et al., 2017), we hypothesized that most users would engage with the app at least once, perceive it to be useful, and report a high likelihood of future use. Because no previous studies have analyzed VHB use behavior in response to stress or SI, no specific hypotheses were made related to the aim of this study.

METHODS

Participants

Participants (N = 50) were undergraduates selectively sampled through a university research participation system for the endorsement of past year or past 2-week SI. The majority of participants were White (n = 33, 66%) and identified as cisgender women (n = 35, 70%). See Table 1 for a complete description of participant characteristics. Participants who completed the study were awarded up to five psychology course research credits and up to $35. Compensation was scaled based on the percent of ambulatory assessments completed; full compensation was awarded for completion rates at or above 80%. Simulated data in R (R Core Team, 2016) using the paramtest package (Hughes, 2017) determined that for a multilevel model with N = 50 and k = 1200, for a moderate effect size (i.e., b ≥ 0.2), adequate power is expected; as such, this sample size was deemed to have sufficient power. All study procedures were approved by the University Intuitional Review Board.

TABLE 1.

Descriptive and Demographic data for college student sample (N = 50).

Demographics	n (%)	ICC
Age	M = 19.20 (SD = 1.29)
Race/Ethnicity
% White	33 (66)
% Black/African American	10 (20)
% Asian/Asian-American	2 (4)
% Latino(a)(Latinx)	2 (4)
% Biracial	3 (6)
Gender
% Woman	35 (70)
% Man	8 (16)
% Gender non-conforming	5 (10)
% Not listed	2 (4)
Sexual Orientation
% Straight	22 (44)
% Gay or Lesbian	6 (12)
% Bisexual	16 (32)
% Not sure/Not listed	6 (12)
Baseline Assessment
% SI Past 2 weeks	40 (80)
% Suicide planning Past 2 weeks	5 (10)
% Lifetime suicide attempt behavior	12 (24)
Ambulatory Assessment	M (SD)
Wish to die	0.26 (0.26)	0.42
SI	0.16 (0.22)	0.38
Stress	0.54 (0.31)	0.23

Abbreviations: ICC, intraclass correlation; SI, suicidal ideation.

Procedures

Recruitment phase

Undergraduate students can receive course credit for completing research studies through the university SONA system. Interested students complete a pre-screener at the beginning of each academic year. Participants who endorsed SI in the past year (i.e., a non-zero response on a single item from the Suicidal Behaviors Questionnaire—Revised; Osman et al., 2001) or past 2 weeks (i.e., a non-zero response on a single item from the Depressive Symptom Index—Suicidality Subscale; Metalsky & Joiner, 1997) were invited to participate via email. Interested students could then follow a link in the invite email, which rerouted them to a SONA signup page where they could schedule a meeting with a graduate student research assistant.

Baseline phase

Participants met with a research assistant via a video conferencing platform (Zoom) to be informed of study procedures and provided informed consent. Regardless of continued participation in the study, all participants were provided with local and national mental health resources. Participants provided informed consent and completed baseline measures via an online survey, a link to which was provided to the participant in the platform's chat feature. A copy of the informed consent and the mental health resources discussed during the baseline session were emailed to each participant immediately following baseline procedures. Following the completion of baseline survey measures (e.g., demographics), participants downloaded the PIEL ecological momentary assessment (EMA) application (Jessup et al., 2012) to their smartphones and were instructed how to use the application to answer daily survey prompts using an example survey. Finally, participants were asked to download the VHB application to their smartphones and were shown how to use the application. Of note, the data from this study is part of a parent project on state SI risk and protective factors. As such, only VHB-related data is being reported in the current work.

Virtual Hope Box (Bush et al., 2015, 2017)

Graduate-level research assistants were trained in VHB administration. A script on how to introduce and describe the VHB was created, and research assistants were “checked out” on these procedures by the study principal investigator to assure adherence to protocol. Each research assistant had prepared a personalized VHB to show to the participants. The research assistant showed each domain of the VHB and how to personalize it using their personal VHB as an example. First, they demonstrated that the VHB could be used to show pleasant or encouraging pictures. Then, they described the features in the “Distract Me” and the “Inspire Me” sections and how these could be personalized. The research assistant then showed the meditations available in the “Relax Me” section and how the VHB could be linked to the phone's contact list, and special contacts could be included for ease of access. Lastly, the research assistant discussed the purpose of a coping card, gave an example of a coping card they had made in their VHB, and helped the participant create a coping card of their own if desired. Participants were encouraged to use the VHB over the course of the study period; however, participants were also informed that their compensation for study completion was not contingent upon VHB use.

Ambulatory phase

For 10 days, participants were prompted to respond to brief surveys five times per day via PIEL. Signal-contingent pseudorandom surveys were deployed five times a day in 3-h time blocks (e.g., 9 a.m. to 12 p.m.) between 9 a.m. and 11:59 p.m. Previous suicide-related EMA research has used a comparable number of daily surveys (e.g., four daily pseudorandom surveys in Kleiman et al., 2017). The decision to use five daily pseudorandom surveys was made based on research that demonstrates extreme vacillation in SI over a relatively short temporal period (Kleiman et al., 2017). Previous research analyzing SI using signal contingent EMA surveys has varied from 2-h delays (e.g., Stenzel et al., 2020) to 4–8-h delays (e.g., Kleiman et al., 2017). Once starting a survey, participants had 30 min to complete it. If a survey was not started within 60 min of the initial notification, it expired. Surveys included questions regarding SI, SI risk and protective factors, and VHB use. Participants received reminder emails immediately after the initial meeting and then every other day for the duration of their enrollment in the study. They were encouraged to respond to the reminder email if they had any technical difficulties.

Virtual hope box use

Self-reported use of the VHB was measured based on branching logic, with the first VHB question being “Did you use your Virtual Hope Box since the last survey?” (yes or no). If a participant endorsed using the VHB, the following items were “Using the Virtual Hope Box helped me feel better,” and “How likely are you to use the Virtual Hope Box in the future if you are feeling miserable?” each presented with a visual analog slider with anchors of zero (not at all) and one (extremely). Participants choose their value using this slider scale, up to two decimal places. If a participant denied using the VHB since the last survey, they only saw the item “How likely are you to use the Virtual Hope Box in the future if you are feeling miserable?” which was responded to using the aforementioned visual analog slider.

Suicide desire questions

Because past ambulatory SI research indicates differing endorsements of which assess passive and active SI (e.g., Kleiman et al., 2017), items measuring wish to die and active SI were included. As such, state suicide desire was measured with the items “right now, I want to die” and “right now, I have an intense desire to kill myself.” We will refer to these as wish to die and SI, respectively. Both items were presented with a visual analog slider with anchors of zero (not at all) and one (completely). Participants choose their value using this slider scale, up to two decimal places. These items were adapted from past SI EMA studies (i.e., Hallensleben et al., 2019; Kleiman et al., 2017). Of note, while single-item assessments of SI and suicide-related behaviors can be useful for predicting important outcomes (e.g., suicide attempts; Green et al., 2015), research has also suggested that minor wording changes can lead to significant differences in item endorsement. This is further discussed below in the limitations section.

Stress

As the VHB is often provided as a tool to cope with experiences of stress in addition to thoughts of suicide (Bush et al., 2015), the following measure of state stress was included: “My current situation is stressful.” Similar items have been used in past EMA research (e.g., Kiekens et al., 2020). This item was presented with a visual analog slider with anchors of zero (not at all) and one (completely). Participants choose their value using this slider scale up to two decimal places.

Debriefing

After the 10-day ambulatory assessment period, participants met with a research assistant briefly over Zoom to transfer their data and arrange compensation. At this meeting, they were encouraged to ask any questions or raise any concerns they had about the study for the research assistant to respond to. All participants who began the study met with a research assistant over Zoom for the debrief session.

Analytical plan

First, descriptive statistics of participant demographics and baseline SI-related variables were conducted. Intraclass correlation coefficients (ICC) were conducted to determine temporal stability and reliability estimates of state predictor variables (i.e., wish to die, stress, SI). Next, descriptive statistics (e.g., means, frequencies) of state VHB items were conducted to determine use frequency across the sample as well as perceptions of usefulness and perceived likelihood of future use. Finally, multilevel modeling (MLM) was conducted to determine if changes in state predictor variables are related to VHB use. First, an intercept-only and random effects model were conducted. Next, z scores were calculated for each individual response to state stress, state wish to die, and state SI items (i.e., group mean centered). These z scores represent the within-person variability of state experiences of stress, wish to die, and SI. Three, two-level binary logistic MLMs were conducted to determine if z scores of state stress, state wish to die, and SI were independently related to subsequent VHB use (i.e., lagged analysis). Participants were set as a random factor. Extreme scores (>3.5 SD) were replaced with 3.5 SD for all state predictor and outcome variables. Odds ratios were conducted to determine the magnitude of any significant effects. Analyses were conducted using the R Lme4 package (Bates et al., 2014).

RESULTS

During the ambulatory phase, participants completed an average of 43.1 surveys (out of 50 possible) over the 10-day collection period (range = 20–50 surveys), resulting in k = 2112. Temporal stability and reliability estimates using the intraclass correlation coefficient (ICC) for predictor variables can be viewed in Table 1. ICCs indicated that 42.4% of the variance in wish to die, 38.4% of the variance in SI severity, and 23% of the variance in stress is between-person, or explained by individual differences.

The majority of participants (n = 30, 60%) used the VHB at least once over the study period. Participants reported the VHB to be, on average, helpful for making them feel better (M = 0.66, SD = 0.21, range from zero [not at all] to one [extremely]) (Figure 1). Regardless of whether participants ever used the VHB, they reported, on average, a high perceived likelihood of future use (M = 0.54, SD = 34, range from zero [not at all] to one [extremely]). Those who used the VHB hope box at least once over the study period reported, on average, a high perceived likelihood of future use (M = 0.64, SD = 0.29, range from zero [not at all] to one [extremely]) (Figure 2), while those who never used the VHB reported a descriptively lower perceived likelihood of future use (M = 0.54, SD = 0.29, range from zero [not at all] to one [extremely]) (Figure 3).

FIGURE 1.

Histogram of Virtual Hope Box usefulness ratings. Participants chose their response up to two decimal places using a visual analog slider scale [range: 0–1].

FIGURE 2.

Histogram of perceived likelihood of Virtual Hope Box (VHB) use in VHB users. Participants chose their response up to two decimal places using a visual analog slider scale [range: 0–1].

FIGURE 3.

Histogram of perceived likelihood of Virtual Hope Box (VHB) use in VHB non-users. Participants chose their response up to two decimal places using a visual analog slider scale [range: 0–1].

MLM results are shown in Table 2. The intercept-only model was significant (p < 0.01) and remained significant with the addition of a random intercept (p < 0.01). State SI severity z scores were significantly related to VHB use at T + 1, such that positive deviations from an individual's mean responses predicted subsequent VHB use. Odds ratios of VHB use for SI severity were 3.31, indicating that there is a 331% increase (or 3.31 times higher) in VHB use as a result of increased state SI. Neither wish to die z scores nor stress z scores predicted subsequent VHB use.

TABLE 2.

MLMs for the prediction of VHB use (yes or no) (N = 50).

	β	SE	z value	Odds ratio
Model 1
Wish to die	0.80	0.51	1.56	2.23
Model 2
SI	1.20	0.57	2.10*	3.31*
Model 3
Stress	0.38	0.39	0.98	1.46

Abbreviations: MLM, multilevel model; SI, suicidal ideation; WTD, wish to die. *p < 0.05.

DISCUSSION

The current study employed ambulatory assessment methodology to explore VHB use in college students who have recently experienced SI. Although previous research has used a longitudinal design to analyze VHB use in at-risk individuals, this research has only been conducted in Veterans, limiting its generalizability to other populations (e.g., Bush et al., 2015, 2017). Further, no research has used ambulatory assessment, precluding analysis of temporally bound relations.

Results indicate that the majority of participants used the VHB at least once during the 10-day study period. This finding is consistent with previous research on the VHB, in which approximately half of at-risk treatment-seeking Veterans used the VHB more than once a day, almost daily, or a few times a week throughout the study period (Bush et al., 2017). Participants who used the VHB rated its usefulness for making them feel better on average (0.66) on a scale from 0 (not at all) to 1 (extremely). Perceived usefulness is consistent with previous research, which found that the majority (84%) of at-risk, treatment-seeking Veterans reported the VHB to be somewhat to very helpful (Bush et al., 2017). Of participants who used the VHB at least once during the study, they rated their likelihood of future use on average (0.64) on a scale from 0 (not at all) to 1 (extremely). Again, the findings in this sample are consistent with previous research, which found that the majority (87%) of at-risk, treatment-seeking Veterans rated their likelihood of future VHB use from somewhat likely to very likely. These data collectively indicate relatively high user acceptability of the application, and the current investigation is the first to demonstrate such acceptability in a civilian population.

Findings indicate that within-person variability in the severity of state SI was significantly related to subsequent VHB use at the subsequent survey. Significant odds ratios indicate that the likelihood of VHB use was 3.31 times higher following increases in SI severity z scores. Interestingly, neither wish to die z scores nor stress z scores were significantly related to VHB use at T + 1. Although the exact reason for this is unknown, it could be that the study was underpowered to detect a very small effect size. Another hypothesis is that participants find active SI (i.e., “Right now, I have an intense desire to kill myself”) to be more distressing than general stress, as well as passive thoughts of wanting to be dead (i.e., “Right now, I want to die”), thus guiding subsequent app usage. To our knowledge, no previous study has analyzed if instances of distress or SI are related to VHB engagement. This is an important frontier of mobile health solution research, as even the most efficacious mobile intervention will not be helpful if a user chooses not to initiate it. Within the current study, the finding that SI z scores were significantly related to subsequent VHB use is encouraging and supports the effectiveness of the VHB for use in individuals experiencing thoughts of suicide.

Although these findings are promising, it remains an important caveat that participants reported not using the VHB for the majority of daily surveys over the study period, regardless of increases in state SI. This is not very surprising, as participants were instructed that VHB use would be unrelated to compensation for student competition and because this study was not completed in the context of treatment. However, changes in SI accounted for less than 45% of the variance in VHB usage. Future research should further aim to determine what prompts individuals to use the VHB, as the current study does not indicate whether it is naturalistically used every day to support coping or is only used in times of SI. Further, it is unclear whether increased use of the VHB is unilaterally positive. It is possible that certain styles of use (e.g., only using it to cope in times of crisis) are superior to others (e.g., routine use even when not in crisis) or that there are individual differences in the effectiveness of each style of use. One avenue of future work should be to determine how and when the VHB should be used.

Limitations & future directions

First, participants were a small undergraduate sample of mostly young adult cisgender women. Although the sample was relatively diverse regarding race/ethnicity and sexual orientation, this work should be replicated in other samples to determine broader generalizability. In the same vein, it will be important to consider that results might differ by sample type and setting (e.g., undergraduates versus psychiatric inpatients, treatment-seeking versus non-treatment-seeking adults) as well as other demographic factors. For instance, although the current sample was relatively diverse, no analyses were conducted regarding VHB in certain subpopulations (e.g., White versus person of color, straight versus lesbian/gay or bisexual, cisgender versus transgender, or gender diverse). This nuanced work is warranted in light of research that indicates that suicidal thoughts and behaviors (Stephenson et al., 2006) as well as mobile app utilization (Krebs & Duncan, 2015) differ across demographic groups.

Additionally, although the use of ambulatory assessment allowed for an examination of the temporal relations between state predictor variables and subsequent SI usage, a more nuanced investigation may still be quite useful. To demonstrate, as previous ambulatory research suggests that SI severity can vary considerably over different temporal scales (e.g., Kleiman et al., 2017), it could be that the time delay between surveys in the current study (approximately 3 h) missed important fluctuations related to VHB use. The use of event contingent sampling in addition to signal contingent sampling (used in the current study) may be useful for overcoming this limitation in future work. All VHB utilization data were assessed via self-report, meaning that objective information on VHB usage (e.g., amount of time spent on the app) is not available. In addition, we did not test if VHB use reduced one's risk of suicide attempts, as no participants in our study attempted suicide during the study period. The visual analog slider scale used in the current study used bipolar anchors, meaning that any response provided between these anchors is somewhat ill-defined and can only be interpreted in terms of the general relation to the poles. Future research may benefit from using Likert-style responses. Finally, the use of single-item assessments of SI poses certain concerns. Importantly, recent research has demonstrated that seemingly inconsequential changes in the phrasing of suicide-related questions (e.g., inclusion of the word seriously in “have you ever seriously considered suicide”) lead to differential endorsement (Ammerman et al., 2021). As such, the phrasing of the single-item assessments used in the current study may have inadvertently affected participants endorsement of said items (e.g., use of the word intense in “right now, I have an intense desire to kill myself”). Future research would benefit from a more nuanced assessment of state-level SI-related questions as it relates to VHB usage.

A continued investigation of VHB use in individuals experiencing SI is certainly warranted. Research would benefit from ameliorating limitations identified in the current study and by extending this research through analysis in varying populations (e.g., older adults, psychiatric inpatients, and outpatients), analyzing the components of the VHB most used (e.g., distraction versus words of inspiration), and identifying if VHB usage is related to improvements in subsequent state mental health variables. Regarding the latter, it is of course of primary importance to determine what outcomes are most important and theoretically desired. Although some might argue that reductions in the frequency or severity of SI should be the primary outcome variable subsequent to VHB use, other factors such as coping self-efficacy (e.g., Bush et al., 2017), improvements in emotion regulation (e.g., Brausch & Woods, 2019), and decreases in suicide-related experiences such as hopelessness (e.g., Ribeiro et al., 2018), negative thinking styles (e.g., Law & Tucker, 2018), and cognitive problems/biases (e.g., Le et al., 2021) may be relevant. It will also be of importance to research VHB use in such a way that allows for the examination of the highly temporal and dynamic nature of SI (see Kleiman & Nock, 2018). Although the current study employed an ambulatory strategy, resulting in a maximum of 50 surveys per participant over a 10-day period, the time between assessments may miss important information regarding an individual's experience.

Conclusion

Mobile mental health solutions such as the VHB may prove useful for clinicians aiming to manage the suicide risk of their patients. The current study sought to extend previous research on the VHB by analyzing use behaviors and perceptions of the VHB in a sample of undergraduates who have recently/currently experienced SI. Results found that most participants used the VHB at least once, perceived it to be useful, and rated a high likelihood of future VHB use. In addition, positive deviations in state SI from an individual's mean were related to subsequent VHB use. Results indicate that the VHB is a useful and simple tool for improving emotion regulation and coping with SI in at-risk individuals.

DATA AVAILABILITY STATEMENT

The data used in this study are not available for sharing due to privacy concerns related to the relatively small sample size.