Efficacy of an Internet-based cognitive-behavioural grief therapy for people bereaved by suicide: a randomized controlled trial

ABSTRACT

Background: The loss of a loved one due to suicide can be a traumatic event associated with prolonged grief and psychological distress.

Objective: This study examined the efficacy of an Internet-based cognitive-behavioural grief therapy (ICBGT) specifically for people bereaved by suicide.

Methods: In a randomized controlled trial, 58 participants with prolonged grief disorder (PGD) symptoms who had lost a close person to suicide were randomly allocated either to the intervention group (IG) or waitlist-control group (WCG). The 5-week intervention comprised ten writing assignments in three phases: self-confrontation, cognitive restructuring, and social sharing. Symptoms of PGD, common grief reactions after suicide, depression, and general psychopathology were assessed at pre-, post-test and follow-up.

Results: Between-group effect sizes were large for the improvement of PGD symptoms in treatment completers (d_ppc2 = 1.03) and the intent-to-treat analysis (d_ppc2 = 0.97). Common grief reactions after suicide and depressive symptoms also decreased in the IG compared to the WCG (moderate to large effects). The results are stable over time. Only for general psychopathology, there was no significant time by group interaction effect found.

Conclusions: The ICBGT represents an effective treatment approach for people suffering from PGD symptoms after bereavement by suicide. Considering the effect sizes, the small treatment dose, duration, and the stability of the results, the ICBGT constitutes an appropriate alternative to face-to-face grief interventions.

HIGHLIGHTS

• Internet-based cognitive-behavioural grief therapy specifically for people bereaved by suicide with Prolonged Grief Disorder (PGD) effectively reduces symptoms of PGD, common grief reactions after suicide and depressive symptoms.

• Between-group effect sizes were large for the improvement of PGD symptoms (ITT d_ppc2=0.97).

1. Background

Close to 800.000 people worldwide die due to suicide every year, which means one death every 40 seconds (WHO, 2020). For each suicide, recent research estimated that 135 people are exposed to that death (Cerel et al., 2019). Of those exposed might be an average of 60 people affected. Study results indicate that approximately five immediate family members, 15 extended family members, 20 friends, and 20 classmates or co-workers could be directly and intimately affected by each suicide death (Berman, 2011).

Grief after such a death is a natural response and research findings indicate that there are more similarities than differences between people bereaved by suicide and other bereaved groups regarding the overall level of grief (Pitman, Osborn, King, & Erlangsen, 2014; Sveen & Walby, 2008). Specific aspects of grief, however, are more pronounced and distinctive (but not unique) in people bereaved by suicide, as they report higher levels of shame, rejection, guilt, responsibility, stigma, need for concealing the cause of death, and blaming than all other survivor groups, which could complicate the adaption to the loss. (Andriessen, Krysinska, & Tekavčič-Grad, 2017; Sveen & Walby, 2008).

Most bereaved individuals are able to adapt to a loss over time. However, for a significant minority, the grief reaction becomes abnormally persistent and causes significant impairment in functioning (Lundorff, Holmgren, Zachariae, Farver-Vestergaard, & O’Connor, 2017). The 11^th revision of the International Classification of Diseases (ICD-11) included this condition as Prolonged Grief Disorder (PGD) (WHO, 2019). The core symptoms are pervasive longing or yearning for the deceased or persistent preoccupation with the deceased coupled with functional impairment beyond six months. Additionally, the American Psychiatric Association (APA) has now approved to include PGD in the forthcoming DSM-5-TR (Prigerson, Boelen, Xu, Smith, & Maciejewski, 2021). A meta-analysis revealed a pooled prevalence of PGD symptoms¹ of 9.8% (95% CI 6.8–14.0) (Lundorff et al., 2017). This meta-analysis excluded studies investigating deaths by suicide, murder, terrorist attacks, or natural disasters due to additional dimensions associated with such losses. A much higher prevalence of 49% (95% CI 33.6–65.4) for PGD symptoms was found in a recent meta-analysis that focused on bereaved individuals following unnatural losses. The results indicate that ‘bereaved individuals following violent killings such as suicide, accidents, homicide, and war-related deaths are most vulnerable for developing PGD.’ (Djelantik, Smid, Mroz, Kleber, & Boelen, 2020, p. 155). Further studies that were not included in this meta-analysis support the evidence that people bereaved by suicide are at higher risk of developing PGD symptoms than people bereaved by natural deaths (de Groot, De Keijser, & Neeleman, 2006; Mitchell, Kim, Prigerson, & Mortimer, 2005; Nam, 2016).

PGD symptoms are, in turn, associated with several adverse health outcomes such as cancer, cardiac problems, hypertension, sleep disturbance, reduced quality of life, and psychiatric comorbidity (e.g. Major Depression and Posttraumatic Stress Disorder) (Boelen & Prigerson, 2007; Germain, Caroff, Buysse, & Shear, 2005; Newson, Boelen, Hek, Hofman, & Tiemeier, 2011; Prigerson et al., 1997; Shear et al., 2011; Silverman et al., 2000; Simon et al., 2007).

In addition, people suffering from PGD symptoms are at higher risk for suicidal ideation and behaviour, resulting in higher mortality rates in this population (Latham & Prigerson, 2004; Mitchell et al., 2005; Szanto, Prigerson, Houck, Ehrenpreis, & Reynolds, 1997; Szanto et al., 2006). Moreover, suicide bereavement itself represents a risk factor for suicide, particularly among partners/spouses and parents of people who have died by suicide (Agerbo, 2005; Pitman et al., 2014; Qin & Mortensen, 2003). These results underline the need for interventions aimed at reducing the grief of people bereaved by suicide. As specific aspects of grief (e.g. shame, rejection, guilt, responsibility, stigmatization) seem to be elevated in this group, interventions targeting these aspects might be appropriate.

Two recent systematic reviews investigated the effectiveness of interventions for people bereaved by suicide (Andriessen et al., 2019; Linde, Treml, Steinig, Nagl, & Kersting, 2017). Only eight grief intervention studies for this specific group of bereaved were found, with mostly low methodological quality. The majority of studies evaluated group interventions, mostly secondary interventions for those at risk of developing PGD symptoms. The results indicate some inconsistent positive results for the secondary interventions focusing on grief in general (not on PGD symptoms). Three interventions were based on cognitive-behavioural therapy (CBT). Of these studies, only one investigated the efficacy of a tertiary intervention for PGD symptoms in people bereaved by suicide compared to people bereaved by accident/homicide or natural causes (Zisook et al., 2018). The results indicated comparable changes for grief symptoms, suicidal ideation, grief-related impairment, avoidance, and maladaptive beliefs in all bereavement groups (suicide, accident/homicide, natural causes), providing evidence that CBT can be effective. However, the clinician-rated improvement was lower for those bereaved by suicide.

Both reviews conclude that there is only scarce evidence of effective grief interventions for people bereaved by suicide and that future research should adapt and evaluate effective PGD interventions for people bereaved by suicide in need of support (Andriessen et al., 2019; Linde et al., 2017).

There is also a discrepancy between the need for psychosocial support and the extent to which this need is met for this particular bereavement group. Wilson and Marshall (2010) revealed that 94% of the people bereaved by suicide indicated needing professional support in managing their grief. At the same time, only 44% of them reported having actually received help from crisis teams, mental health services, self-help or guided support groups, psychiatrists, psychologists, nurses, or other counsellors. In addition, 60% of those who received professional support were dissatisfied with it. Andriessen (2009) reported an even smaller number of approximately 25% who find their way into support groups or therapy where available.

Identified barriers for support seeking are, for instance, fear of being judged or stigmatized, distrust of professionals, reluctance to ask for help, concerns about what others might think, poor mental health literacy, distance/unavailability, as well as a lack of information, time or financial resources (Andriessen et al., 2019; Hanschmidt, Lehnig, Riedel-Heller, & Kersting, 2016; Mcmenamy, Jordan, & Mitchell, 2008; Provini, Everett, & Pfeffer, 2000; Wilson & Marshall, 2010).

Easily available and accessible support can be provided via Internet-based programmes (Lange et al., 2003; Lange, Van De Ven, Schrieken, & Emmelkamp, 2001). Compared to face-to-face therapy, Internet-based interventions offer more geographic and time flexibility and anonymity as well as faster attainability (Aboujaoude, Salame, & Naim, 2015; Musiat & Tarrier, 2014) and have demonstrated comparable positive effects, e.g. for depression, social phobia, or panic disorder (Hedman, Ljótsson, & Lindefors, 2012). A systematic review, including 108 studies on Internet-based cognitive-behavioural therapy (ICBT), concluded that ICBT can be as effective as conventional CBT for respective clinical disorders (Hedman et al., 2012).

A few studies have investigated Internet-based interventions for PGD symptoms and revealed some promising results. Wagner, Knaevelsrud, and Maercker (2006) examined the efficacy of an ICBT for PGD symptoms in terms of intrusion, avoidance, and failure to adapt and found significant improvement for participants in the treatment group. The programme consisted of self-confrontation, cognitive restructuring, and restoration and integration and was later adapted and evaluated by Kersting, Kroker, Schlicht, Baust, and Wagner (2011) for mothers after pregnancy loss. Their results also revealed a significant decrease in grief symptoms. Eisma et al. (2015) examined the effectiveness of Internet-delivered exposure and behavioural activation for PGD symptoms and provided more evidence for the effectiveness and applicability of exposure therapy via the Internet.

To our knowledge, there is no ICBT for people bereaved by suicide suffering from PGD symptoms available, nor has an ICBT been tested specifically within this bereavement group, even though ICBTs seem particularly appropriate for this vulnerable group. Given the advantages and positive effects of Internet-based interventions and the limited availability of interventions for people bereaved by suicide, we adapted the ICBT from Wagner et al. (2006) to this vulnerable population and tested whether people bereaved by suicide would also benefit. The program consists of CBT elements such as confrontation and cognitive restructuring, which were demonstrated to be efficacious in reducing PGD symptoms (e.g. Boelen, De Keijser, Van Den Hout, & Van Den Bout, 2007; Shear, Frank, Houck, & Reynolds, 2005; Wagner et al., 2006), and could be especially helpful for people bereaved by suicide. We anticipated that targeting the specific aspects of grief that are common themes after suicide bereavement (e.g. shame, rejection, guilt, responsibility, stigma) would be particularly effective. The anonymity of the Internet could further reduce fears of being judged or stigmatized and facilitate disclosure of painful thoughts and feelings, such as shame or guilt.

To evaluate its efficacy, we assessed symptoms of PGD, as well as common grief reactions after suicide (e.g. shame, rejection, guilt), depression, and general psychopathology. We hypothesized that all assessed symptoms would show a significantly greater decrease from pre-test to post-test in the intervention group (IG) than in the waitlist-control group (WCG).

2. Method

2.1. Participants

The study was conducted according to the Declaration of Helsinki and was approved by the Ethics Committee of the Medical Faculty of the University of Leipzig (reference number: 319–14-06102014). Participants were recruited between July 2015 and March 2017 via the Internet, social media, press information, as well as links and flyers sent to psychology websites, insurance companies, churches, support groups, clinics and medical practices, online communities, and blogs in Germany.

Inclusion and exclusion criteria, as well as demographic data, were assessed in a screening process in an online diagnostic survey and a telephone interview. Participants were included in the study if they were 18 years or older, bereaved due to suicide of a loved one, spoke German, had Internet access, and met the diagnostic criteria for PGD according to the criteria by Prigerson et al. (2009). The PGD diagnoses according to the 2009 criteria was confirmed in a telephone interview using the German version (Pfoh & Rosner, 2014) of the PG-13 (Prigerson et al., 2009). The PG-13 does not assess all of the newest PGD criteria included in the upcoming ICD-11. At the time, there was no tool available assessing the PGD_ICD-11 criteria (Treml, Kaiser, Plexnies, & Kersting, 2020). Therefore, we use the term PGD symptoms instead of PGD throughout the manuscript because the current PGD_ICD-11 criteria could not be assessed then. The criterion time since loss was set to at least 14 months to avoid anniversary effects.

Exclusion criteria were the following: 1) ongoing psychotherapy, 2) unstable psychopharmacological treatment with changes within the last six weeks, 3) history of psychotic or dissociative symptoms (assessed using the Brief Symptom Inventory by Derogatis, 1993), 4) any severe substance abuse or dependence disorder (assessed using questions about consuming behaviour on alcohol and other drugs) and 5) acute suicidal ideation (assessed using the Yale Evaluation of Suicidality Scale, YES by Latham & Prigerson, 2004). If participants met one or more exclusion criteria in the online diagnostic survey, we conducted telephone interviews to clarify these and to inform about alternative support if needed.

Overall, 129 participants completed the screening, of which 68 had to be excluded. Most met at least one exclusion criterion (n = 50), 13 declined after screening without indicating specific reasons, and five were non-responsive after the screening. Sixty-one eligible participants gave their written informed consent and were included in the study (see Figure 1). Two participants were not randomized, as they participated at the end of the recruitment period and a 5-week waiting period, and then starting the delayed treatment would not have been possible. Therefore, they were directly assigned to the IG and excluded from all analyses, leading to a final dataset of n = 58 participants (intention-to-treat). Of these, two participants dropped out during the 5-week intervention, one after the intervention at post-test and one during the waiting period. Another participant was excluded from completer analyses due to missing data at post-test (T1). In this case, the online survey was not working. So the participant was asked to fill out a paper/pen questionnaire which was incomplete. Completer analyses are therefore based on the data of 53 participants.

Figure 1.

Study enrolment flow

2.2. Design

Participants were randomly assigned to either the intervention group (IG) or waitlist-control group (WCG). The computer-assisted randomization procedure was conducted using the software RITA-Randomization In Treatment Arms (Pahlke, König, & Ziegler, 2004). The sample was stratified according to the patient’s relationship to the deceased, i.e. a child or other kinship form. The research team was not blind to the group allocation. However, this is not expected to lead to biased results, as all assessments after randomization were anonymously and automated via online questionnaires.

Immediately after randomization, all participants completed a pre-test (T0). Participants in the IG were then given access to the encrypted Internet-based communication system. The treatment and the waiting period lasted five weeks, after which both groups completed a post-test (T1). For ethical reasons, the WCG started the delayed treatment immediately after the end of the IG treatment period instead of waiting until the follow-up assessments had been completed. Further assessment points were a 3-month (T2), 6-month (T3), and 12-month (T4) follow-up.

2.3. Measures

2.3.1. Primary outcome

The primary outcome was the severity of grief symptoms measured with the Inventory of Complicated Grief (ICG by Prigerson et al., 1995) and common grief reaction after the loss due to suicide measured with the Grief Experience Questionnaire (GEQ by Barrett & Scott, 1989). The ICG is a 19-item self-report questionnaire, which has been shown to be valid and reliable (Prigerson et al., 1995). The items are rated on a 5-point Likert-scale, and a cut-off sum score of 25 is considered to be indicative of PGD symptoms. Cronbach’s alpha for the ICG in the present study indicated good internal consistency (α = .83).

The GEQ was used to get a more detailed picture of various aspects of grief. It is a self-report measure containing 55 items, designed to measure various grief reactions, that means (1) somatic reactions and reactions that have been associated with (but are not unique to) grief after suicide, such as (2) search for explanation, (3) stigmatization, (4) guilt, (5) responsibility, (6) shame/embarrassment, (7) rejection/abandonment, and (8) self‐destructive behaviour. The GEQ consists of these eight subscales, which can be summed to a total grief score (GEQ_total). The frequencies of the grief reactions are rated on a 5-point Likert-scale. Reliability and Validity of the GEQ were demonstrated (Bailley, Dunham, & Kral, 2000; Barrett & Scott, 1989). In the present study, the internal consistency of the GEQ_total was excellent (α = .92) and for the subscales acceptable to good (α = .73-.88), except for self-destructive behaviour with a lower internal consistency of α = .59.

2.3.2. Secondary outcomes

Secondary outcome measures were depressive symptoms and general psychopathology. Depressive symptoms were measured with the well validated revised version of the Beck Depression Inventory (BDI-II, Hautzinger, Keller, & Kühner, 2009). The BDI-II consists of 21 items assessing the severity of the typical symptoms of depression, such as low mood, self-accusation, insomnia, and fatigue. The BDI-II demonstrated good internal consistency in the present study (α = 0.89).

General psychopathology was measured using the Brief Symptom Inventory (BSI, Derogatis, 1993). The 53 items record the experience of physical and psychological distress and include the following nine symptom domains: somatization, obsessive–compulsive, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation, and psychoticism. Furthermore, the self-report measure also provides several indices, including a Global Severity Index (GSI) of overall mental health, a Positive Symptom Distress Index (PSDI), and a Positive Symptom Total (PST). The PSDI measures the intensity for all answers, and the PST gives information about the number of symptoms for which there is distress (Derogatis, 1993). The internal consistency of the GSI in our sample was excellent α = 0.96.

2.4. Intervention

The participants took part in an Internet-based cognitive-behavioural grief therapy (ICBGT). The rationale was initially developed by Lange et al. (2001) for people suffering from posttraumatic stress and later adapted for PGD symptoms by Wagner and colleagues (Wagner, Knaevelsrud, & Maercker, 2005; Wagner et al., 2006). The program comprises ten writing assignments in three phases (1) self-confrontation with four assignments, (2) cognitive restructuring with four assignments, and (3) social sharing with two assignments, which we adapted for people bereaved by suicide. The program started with psychoeducation on suicide and suicide bereavement, including information on grief and PGD as well as the meaning of common symptoms and reactions after losing a loved one to suicide (e.g. feelings of guilt, responsibility, shame). Each phase included further psychoeducation on the meaning and background of the treatment technique. The first phase of self-confrontation addresses loss-oriented coping. In four writing tasks, the participants were asked to report on their most painful experience related to the suicide of their loved one. They were instructed to write in as much detail as possible, focusing on their thoughts and emotional and sensory perceptions, using the present tense and the first person. This phase aims to mitigate feelings such as anxiety and guilt by reprocessing and reduce avoidance behaviour.

The second phase of cognitive restructuring focuses on the restoration and integration of the loss experience. In these writing tasks, a change of perspective was instructed to help participants develop realistic and helpful coping strategies. The participants were asked to write a supporting letter to a (possibly hypothetical) friend who has suffered the same kind of loss. The letter should reflect and acknowledge burdensome feelings such as guilt, shame, or anger, but also correct unrealistic assumptions and dysfunctional thoughts (e.g. search for an explanation or responsibility).

Participants were furthermore instructed to encourage their friend to activate resources, and to find rituals to express their grief. The aim of these writing assignments is to help participants regain a sense of control over their lives and define a new role for themselves.

The third and final phase of social sharing also focuses on the restoration and integration of the loss experience. In the last two writing tasks, the participants were instructed to write a letter to a person affected by the loss, for instance themselves, the deceased or a close person. The final letter served as an opportunity to summarize and share what they have learned during the therapeutic process and what they want to implement to better cope with their loss.

The participants were instructed to write two assignments a week, each lasting 45 minutes. Twice in each phase, the therapist provided individual written feedback along with instructions for the next writing assignment within one working day. All therapists were trained psychologists. The instructions for all writing tasks were standardized. The therapist’s individual feedbacks were structured, and they were encouraged to address common themes for people bereaved by suicide such as stigmatization, search for an explanation, guilt, shame, or responsibility, especially within the second phase of cognitive restructuring.

2.5. Data analyses

Statistical analyses were performed using SPSS, version 25 (SPSS Inc). Significance level was set to α = 0.05. The equivalence of the both groups (IG and WCG) with regard to demographic and clinical characteristics at baseline was examined using two-tailed independent sample t-tests for continuous variables and Pearson χ²-tests or Fisher’s exact test for categorical variables. Dropout analyses were performed likewise.

Linear mixed models for repeated measures were used to evaluate the efficacy of the ICBGT. These models allow for an intention-to-treat analysis (ITT) in which all available data from randomized participants are included to estimate unbiased variable estimates under the missing at random assumption applying a maximum-likelihood-algorithm. Treatment condition (IG vs WCG), time, and their interaction were treated as fixed effects, and the intercept was specified as a random effect to account for the repeated observations within participants. In addition to ITT analyses, completer analyses were conducted, including only observed data of participants who had completed the pre-test (T0) und post-test assessment (T1). The Benjamini–Hochberg procedure was used to adjust for the false discovery rate (FDR) for multiple testing (Benjamini & Hochberg, 1995). The FDR is the likelihood of an incorrect rejection of a null hypothesis. Controlling for FDR instead of the Family Wise Error Rate (as in the Bonferroni correction) increases the method’s power and is less stringent. Thus, more discoveries may be made and more hypotheses may be rejected (Haynes, 2013).

For all outcome measurements, effect sizes for within and between-subjects were calculated (see Table 2). Within subject effect sizes were established by calculating the difference in means, divided by the pooled within-group standard deviation (Cohen, 1988). Between-subject effect sizes were calculated based on d_ppc2 by Morris (2008)(mean pre-post change in the IG minus the mean pre-post change in the WCG, divided by the pooled pre-test standard deviation). Effect sizes from 0.2 to 0.4 were considered as small, between 0.5 and 0.7 as moderate, and ≥0.8 as large.

Table 2.

Intention-to-treat analyses (ITT), N = 58 (IG = 30, WCG = 28)

Primary Outcome	Group	Pre	Post	Interaction effect		Effect sizes
		M (SD)	M (SD)	Time*group	p-Value	dpppc2	dwithin
ICG	IG	35.43 (11.19)	24.79 (11.47)	F(1, 53.69) = 24.89	<0.001*	0.97	0.94
	WCG	36.39 (11.19)	36.72 (11.29)				−0.03
GEQ total	IG	151.97 (34.10)	129.77 (34.80)	F(1, 54.04) = 13.90	0.002*	0.65	0.64
	WCG	150.07 (34.10)	150.15(34.34)				0.00
GEQ	IG	32.73 (8.42)	27.54 (8.56)	F(1, 55.14) = 14.23	0.002*	0.72	0.61
Abandonment/Rejection	WCG	31.39 (8.42)	32.34 (8.49)				−0.11
GEQ	IG	30.97 (9.56)	26.68 (9.68)	F(1, 54.04) = 7.61	0.019*	0.36	0.45
Stigmatization	WCG	30.36 (9.56)	29.53 (9.60)				0.09
GEQ	IG	25.00 (6.52)	19.36 (6.63)	F(1, 55.93) = 26.02	<0.001*	0.87	0.86
Search for Explanation	WCG	25.82 (6.52)	25.92 (6.58)				−0.11
GEQ	IG	20.30 (6.74)	16.22 (6.83)	F(1, 55.22) = 10.41	.005*	0.56	0.60
Guilt	WCG	17.82 (6.74)	17.58 (6.79)				0.04
GEQ	IG	7.80 (3.46)	6.87 (3.54)	F(1, 55.54) = 0.95	.335	0.23	0.26
Somatic Reaction	WCG	8.82 (3.46)	8.69 (3.50)				0.04
GEQ	IG	11.07 (5.36)	10.03 (5.45)	F(1, 55.61) = 0.11	.739	0.12	0.14
Responsibility	WCG	11.64 (5.36)	11.25 (5.41)				0.07
GEQ	IG	7.70 (2.78)	6.90 (2.83)	F(1, 54.72) = 2.43	.125	0.26	0.28
Self-Destructive Orientation	WCG	8.03 (2.78)	7.95 (2.80)				0.03
GEQ	IG	16.37 (6.04)	15.83 (6.13)	F(1, 55.59) = 0.17	.681	0.07	0.09
Shame/Embarrassment	WCG	15.86 (6.04)	15.77 (6.09)				0.02
Secondary outcomes
BDI-II	IG	21.10 (11.49)	12.98 (11.66)	F(1, 55.80) = 6.81	.024*	0.49	0.70
	WCG	23.18 (11.49)	20.75 (11.58)				0.21
BSI Global Severity Index	IG	0.88 (0.60)	0.55 (0.60)	F(1, 54.12) = 2.38	.138	0.26	0.58
(GSI)	WCG	1.12 (0.61)	0.95 (0.60)				0.30
BSI Positive Symptom Total	IG	28.53 (10.22)	18.66 (10.52)	F(1, 54.48) = 5.91	.032*	0.52	0.95
(PST)	WCG	33.18 (10.22)	28.68 (10.32)				0.45
BSI Positive Symptom	IG	1.54 (0.55)	1.29 (0.56)	F(1, 54.00) = 2.39	.128	0.25	0.47
Distress Index (PSDI)	WCG	1.70(0.55)	1.59 (0.56)				0.20

*p-values adjusted based on the Benjamini-Hochberg procedure.

Clinical significance of the improvement was evaluated for all completers by calculating a Reliable Change Index (RCI) following Jacobson and Truax (1991). To calculate the RCI, the post-test score is subtracted from the pre-test score and this result is then divided by the standard error of the differences. If the product is larger than the z-score level of significance, in this case 1.96, then change can be considered to be beyond that of chance variation. The formula uses the standard error of the mean, which is calculated using standard deviations and a reliability coefficient. In this case Cronbach’s Alpha from the pre-test was used. Furthermore, the ICG cut-off score of 25 was used to examine whether participants still indicate symptoms of PGD at post-test. The number of participants indicating PGD symptoms at post-test was compared between IG and WCG using a Pearson χ²-test. Recovery from PGD symptoms was operationalized by an ICG Score < 25 and RCI above 1.96.

To test the stability of the treatment effects, linear mixed models for repeated measures were used including time as fixed effect (time: post-test, 3-month, 6-month, 12-month follow-up). The analysis included both groups IG and WCG after treatment. In case of significant time effects, post-hoc tests were examined for post-test to each of the three follow-ups.

3. Results

Following randomization, there were no significant differences between the two groups (IG and WCG) in any of the demographic data or clinical characteristics (see Table 1). Most of the 58 participants were female (86.2%), married or partnered (55.2%) and highly educated (≥12 years, 60.3%). The majority of participants lost their child (n = 19; 32.76%), eleven participants lost a parent due to suicide, another eleven a sibling (18.97% respectively). Ten participants indicated having lost a partner (17.24%), three a close friend (5.17%) and four another close person (e.g. neighbour, grandparent, 6.90%). The most common method of suicide was hanging/suffocation (27.59%), followed by poisoning (15.52%), jumping from high places, jumping in front of moving objects or carbon monoxide poisoning (13.79%, respectively), and use of firearm (8.62%).

Table 1.

Baseline demographic and clinical characteristics of the study sample

	Total sample (N = 58)	IG (N = 30)	WCG (N = 28)	p-value
Age. M (SD. range)	44.47 (14.25. 22–79)	43.93 (15.31. 23–79)	45.04 (13.26. 22–71)	.77 a
Females, n (%)	50 (86.2)	26 (86.7)	24 (85.7)	1.00b
Married/partnered, n (%)	32 (55.2)	18 (60.0)	14 (50.0)	.44b
Children, n (%)	34 (58.6)	17 (56.7)	17 (60.7)	.75b
School education, n (%)				.81b
Low	4 (6.9)	2 (6.7)	1 (3.6)
Medium	19 (32.8)	9 (30.0)	10 (35.7)
High	35 (60.3)	19 (63.3)	17 (60.7)
Grief (ICG), M (SD)	35.90 (10.34)	35.43 (10.57)	36.40 (10.25)	.73a
Grief (GEQ), M (SD)	151.05 (32.39)	151.97 (33.40)	150.07 (31.85)	.83 a
Depression (BDI), M (SD)	22.10 (10.56)	21.1 (10.24)	23.18 (10.99)	.46 a
General Psychopathology (BSI), M (SD)	0.99 (0.58)	0.88 (0.53)	1.12 (0.61)	.11 a

^atwo-tailored t-test,

^bχ² or Fisher’s exact test.

Overall, five participants were considered study dropouts (8.62%), two of those dropped out during the intervention, one after the intervention at post-test and one during the waiting period. One participant completed the intervention and the post-test-assessment but had too much missing data at T1. No statistically significant differences between these dropouts and completers were found in any of the demographic data and the clinical characteristics at baseline (all p-values >.05).

3.1. Primary outcomes

For the ITT analysis, linear mixed models yielded a significant time by group interaction effect for PGD symptoms (see Table 2). PGD symptoms measured with the ICG decreased significantly within the IG compared to the WCG (F(1, 53.69) = 24.89, p < .001) with a large effect size of d_ppc2 = 0.97. The same applies for completer analyses (n = 53; d_ppc2 = 1.03). Grief symptoms measured with the GEQ_total were also significantly reduced within the IG compared to the WCG (F(1, 54.04) = 13.9, p = .002) with a moderate effect size of d_ppc2 = 0.65. For the subscales of the GEQ there were significant time by group interaction effects for Abandonment/Rejection (d_ppc2 = 0.72, moderate effect), Stigmatization (d_ppc2 = 0.36, small effect), Search for Explanation (d_ppc2 = 0.87, large effect), and Guilt (d_ppc2 = 0.56, moderate effect). The other subscales revealed no significant reduction (see Table 2). These results are also valid for completer analyses: GEQ_total (d_ppc2 = 0.65); Abandonment/Rejection (d_ppc2 = 0.73), Stigmatization (d_ppc2 = 0.35), Search for Explanation (d_ppc2 = 1.04), and Guilt (d_ppc2 = 0.62). The time by group interaction effects are visualized in Figure 2 and Figure 3.

Figure 2.

Time by group interaction effect for PGD symptoms measured with the ICG (means and 95% CI, ICG cut-off =25, possible range: 0-76)

Figure 3.

Time by group interaction effect for grief symptoms (GEQ_total) (means and 95% CI, possible range: 55-275)

Table 3 presents the time effect from post-test (T1) to 12-month follow-up (T4) for both groups after the intervention. For the ICG there was a significant time effect (F(3, 117.01) = 3.68, p = .014), indicating a significant change over time. Post-hoc tests revealed that none of the follow-ups differed significantly from post-test (post to 3-month follow-up: t = −0.614, p = .541; post to 6-month follow-up: t = 1.097, p = .275; post to 12-month follow-up: t = −1.009, p = .315). For the GEQ_total and all but two subscales there were no significant time effects. Only for Guilt and Shame there were significant time effects (F(3, 117.09) = 2.80, p = .043; F(3,118.15) = 3.61, p = .015), indicating a significant change over time. Post-hoc tests revealed again that none of the follow-ups differed significantly from post-test (Guilt: post to 3-month follow-up: t = −0.544, p = .588; post to 6-month follow-up: t = 1.46, p = .147; post to 12-month follow-up: t = −0.098, p = .922; Shame: post to 3-month follow-up: t = −0.239, p = .812; post to 6-month follow-up: t = 0.80, p = .426; post to 12-month follow-up: t = −1.509, p = .133). All results appear to be stable one year after the treatment.

Table 3.

Stability of treatment effects N = 49 (ITT)

Primary Outcome	Post	3-month FU	6-month FU	12-month FU	Time effect*
	M (SD)	M (SD)	M (SD)	M (SD)		p-Value
ICG	25.33 (12.15)	24.67 (12.38)	26.91 (12.58)	23.59 (12.97)	F(3, 117.01) = 3.68	.014
GEQ total	129.76 (37.60)	127.83 (38.07)	129.44 (38.51)	122.08 (39.40)	F(3, 118.99) = 2.36	.075
GEQ	27.65 (9.35)	26.93 (9.48)	27.35 (9.61)	26.86 (9.85)	F(3, 119.43) = 0.57	.639
Abandonment/Rejection
GEQ	27.41 (10.43)	27.21 (10.61)	26.50 (10.78)	24.86 (11.09)	F(3, 118.78) = 1.35	.260
Stigmatization
GEQ	20.31 (7.56)	20.54 (7.67)	20.71 (7.77)	19.57 (7.96)	F(3, 118.81) = 1.17	.325
Search for Explanation
GEQ	15.20 (6.66)	14.87 (6.80)	16.42 (6.92)	15.11 (7.15)	F(3, 117.09) = 2.73	.043
Guilt
GEQ	6.78 (3.01)	6.63 (3.14)	6.24 (3.22)	6.10 (3.35)	F(3, 114.38) = 0.55	.649
Somatic Reaction
GEQ	9.82 (4.77)	9.59 (4.89)	9.80 (4.98)	9.44 (5.16)	F(3, 117.55) = 0.26	.855
Responsibility
GEQ	7.53 (2.20)	7.15 (2.27)	6.91 (2.32)	6.73 (2.41)	F(3, 116.01) = 1.58	.197
Self-Destructive Orientation
GEQ	14.63 (5.97)	14.50 (6.10)	15.23 (6.21)	13.30 (6.41)	F(3, 118.15) = 3.61	.015
Shame/Embarrassment
Secondary outcomes
BDI-II	11.88 (10.53)	12.14 (10.75)	11.76 (10.93)	10.72 (11.28)	F(3, 118.75) = 0.41	.744
BSI Global Severity Index	0.59 (0.53)	0.61 (0.54)	0.56 (0.55)	0.48 (0.57)	F(3, 118.87) = 0.98	.404
(GSI)
BSI Positive Symptom Total	20.47 (11.96)	21.72 (12.41)	20.53 (12.77)	18.15 (13.23)	F(3, 114.61) = 1.16	.327
(PST)
BSI Positive Symptom	1.30 (0.45)	1.31 (0.46)	1.25 (0.47)	1.25 (0.49)	F(3, 119.95) = 0.48	.699
Distress Index (PSDI)

*reference = post-test.

3.2. Secondary outcomes

The linear mixed models also yielded a significant time by group interaction effect for depressive symptoms (see Table 2). The BDI-II score decreased significantly within the IG compared to the WCG (F(1, 55.80) = 6.81, p = .024) with a moderate effect size of d_ppc2 = 0.49 (ITT analyses). Completer analyses revealed similar results (n = 53; d_ppc2 = 0.56).

For general psychopathology measured with the BSI there was no significant time by group interaction effect for overall mental health (GSI) in both ITT and completer analyses. There was a significant time effect (F(1, 55.24) = 25.40, p < .001), indicating that both groups improved over time. Similar results were found for the intensity for all answers (PSDI, no significant time by group interaction effect). The PSDI decreased significantly in both groups over time (F(1, 55.18) = 15.46, p < .001). Regarding the number of symptoms for which there was distress (PST), a significant time by group interaction effect was found. The PST significantly decreased within the IG compared to the WCG (F(1, 55.17) = 5.82, p = .032), with a moderate effect size of d_ppc2 = 0.52 (ITT, completer: d_ppc2 = 0.57) (see Table 2). These results appear to be stable over time as no significant time effects from post-test (T1) to 12-month follow-up (T4) were found (see Table 3).

3.3. Clinical significance

According to the RCI, 37% (n = 10) of the IG improved clinically significant (i.e. the ICG score decreased by at least 12 points), versus only 7.4% in the WCG (n = 2), χ²(1) = 6.86, p = .009.

Within the IG 55.6% (n = 15) scored below the cut-off of the ICG after treatment at the post-test compared to 18.5% (n = 5) at pre-test, while the participants of the WCG did not change (14.8%; n = 4 scored below the cut-off at pre- and post-test) (see Figure 4).

Figure 4.

Percentage of participants above the ICG cut-off score

Note that reaching ICG cut-off was not an inclusion criteria (only PG-13), ICG scores of both groups did not differ at pretest (see Table 1).

*** χ²(1)=9.83, p=0.002.

Taken these two criteria together, 25.9% (n = 7) of the IG were considered to be recovered (i.e. they had a score below 25 at the ICG at post-test and a showed a clinically reliable improvement), versus 0% in the WCG, χ²(1) = 8.04, p = .005. Three participants showed a reliable improvement but did not meet the cut-off criterion. Likewise, there were eight other participants in the IG who scored below the cut-off at post-test, but started with a comparably low score, so that they could not achieve reliable improvement (of at least 12 points). None of the participants deteriorated reliably in either group.

4. Discussion

This is the first study to evaluate whether an ICBGT for people bereaved by suicide was efficacious in reducing grief symptoms, depressive symptoms, and general psychopathology. A total of 61 participants were included in the study, which to our knowledge, is the largest sample of people bereaved by suicide with PGD symptoms participating in an RCT. In accordance with our hypotheses, the grief intervention led to substantial improvements in symptoms of PGD, common grief reactions after suicide, and depression. We could not find a significant time by group interaction effect for general psychopathology, as both groups (IG and WCG) seemed to have improved over time.

These findings suggest that the ICBGT is effective in terms of reducing grief severity. The effect sizes for completers and those intended to treat are large (d_ppc2 = 1.03, d_ppc2 = 0.97). This result is comparable to ICBTs for other mental health conditions (e.g. depression, anxiety) (Hedman et al., 2012). Furthermore, the effect size exceeds the average pooled effect sizes from two recent meta-analyses. Wagner, Rosenberg, Hofmann, and Maass (2020) summarized the evidence for web-based bereavement care and found a pooled effect size of g = 0.54 for PGD. Johannsen et al. (2019) examined conventional and Internet-based interventions for PGD and found a pooled effect size of g = 0.45.

Besides the ICG, the GEQ was also used to assess reactions that have been commonly associated with grief after suicide. With the ICBGT, the dimension search for an explanation was reduced (large effect) as well as feelings of abandonment/rejection, stigmatization, and guilt (moderate to small effects). The scores on all other subscales were not significantly reduced, as they were low from the beginning. Depressive symptoms were also diminished in the IG compared to the WCG (moderate effect). That effect size was also slightly larger than the pooled effect size found in the recent meta-analysis for web-based interventions for bereaved people (Wagner et al., 2020). These results further emphasize that the intervention is an effective treatment approach.

Our results also indicate that the effects remain stable over time, as no changes were evident at 12-month follow-up. This is in line with previous studies that revealed that ICBTs have enduring effects (Hedman et al., 2013, 2011; Knaevelsrud & Maercker, 2010).

Considering the low dropout rate in this study, the ICBGT seems to be also highly accepted. Especially for the vulnerable group of people bereaved by suicide, there are hardly any specific grief interventions available. The ICBGT can fill this gap and lower some barriers to seeking support. The anonymity of the Internet might reduce the fear of being judged or stigmatized and facilitate the disclosure of painful thoughts and feelings, such as shame or guilt. Another advantage lowering a barrier is the temporal and geographical independence of the treatment. Therefore, ICBGT can be considered a viable option in bereavement care after suicide.

However, a problem with ICBTs, in general, is the potential for misunderstanding during treatment interaction. For example, the therapist’s feedback could be misinterpreted, or the therapist could draw the wrong conclusions. It is also more difficult to intervene in a crisis. In case of a crisis, we had both the patient’s phone number and the personal physician’s number available. We contacted one patient by telephone during treatment to resolve any problems. We did not have to involve physicians. Even though crisis interventions might be more difficult, they are still feasible.

Despite the promising results, several limitations of the study should be noted. One limitation is related to the findings’ generalizability, as 86% of the participants were female, even though the intervention was aimed at both male and female participants. To some extent, this may reflect that women may be more likely to be open to therapist contact or seek support via the Internet (Mackenzie, Gekoski, & Knox, 2006). Also a sample size of N = 116 was intended (expecting a dropout rate of 14%), to ensure enough power to detect a moderate effect. Although we did not meet this criterion, the achieved sample size of N = 58 with a small dropout rate was sufficient to detect the large effect that the intervention had on PGD symptoms.

Additionally, our sample was relatively homogeneous concerning socio-economic variables. The majority of our sample was highly educated, living with a partner, and of middle to high income. Our sample may, therefore, not be representative of all people bereaved by suicide. A challenging task for future studies should be the inclusion and treatment of more male participants as well as less-educated participants. A further limitation is that treatment outcomes were assessed exclusively via self-report instruments. Structured clinical interviews would have increased the validity of the results. Also, the self-report instruments did not assess the newest ICD-11 criteria for PGD, which should be addressed in future research.

In addition, the exclusion of bereaved with a history of psychotic or dissociative symptoms, severe substance abuse or dependence disorder, and acute suicidal ideation might not represent the naturalistic setting in treating patients with PDG. Hence, effectiveness studies should be conducted to obtain better knowledge about the generalization of results from previous efficacy studies.

4.1. Conclusion and clinical implications

Despite these limitations, we can conclude that ICBGT is an appropriate treatment approach for people suffering from PGD symptoms after bereavement by suicide. PGD is a painful and debilitating condition, and treatment is vital to prevent adverse mental and physical health outcomes. Effective treatment of PGD among the vulnerable group of people bereaved by suicide should be a priority for prevention and intervention efforts. Given the general lack of specific psychotherapeutic support for people bereaved by suicide, this Internet-based intervention – which has proven to be effective and accepted in the present sample – appears to be a promising approach. The program led to long-lasting significant reductions in symptoms of PGD, common grief reactions after suicide, and depressive symptoms. Considering the current effect sizes, the small treatment dose, duration, and the stability of the results, the ICBGT constitutes an appropriate and effective alternative to face-to-face grief interventions.

Funding Statement

This work was supported by research grants from the Roland Ernst Stiftung, which had no role in the design of this study, its execution, analysis and interpretation of data, or publication of results.

Disclosure statement

The authors have no competing interests to declare in relation to this article.

Data availability statement

The data used in the present study cannot be shared due to the EU general Data Protection Regulation.