Skip to main content
Taylor & Francis Open Select logoLink to Taylor & Francis Open Select
. 2012 May 24;41(4):321–334. doi: 10.1080/16506073.2012.689323

Internet-Based Treatment of Pathological Gambling with a Three-Year Follow-Up

Per Carlbring 1,, Nicklas Degerman 2, Jakob Jonsson 1, Gerhard Andersson 2,3
PMCID: PMC3516818  PMID: 22620990

Abstract

Effective therapies for pathological gambling exist, but their use is limited to about 10% of the target population. In an attempt to lower the barriers for help, Internet-based cognitive behavioural therapy (ICBT) has been shown to be effective when delivered to anon-depressed sample with pathological gambling. This study sought to extend this finding to a larger, more representative population, and also test a model to predict responder status. Following advertisement, a total of 284 participants started an 8-week ICBT programme with minimal therapist contact via e-mail and weekly telephone calls of less than 15 min. The average time spent on each participant, including telephone conversations, e-mail, and administration, was 4 h. In addition to a mixed effects model to evaluate the effectiveness of the treatment, two logistic regression analyses were performed with the following eight pre-defined response predictor variables: work-life satisfaction, primary gambling activity, debts due to gambling, social support, personal yearly salary, alcohol consumption, stage of change, and dissociative gambling. ICBT resulted in statistically significant reductions in the scores of pathological gambling, anxiety, and depression as well as an increase in quality of life compared to pre-treatment levels. Follow-ups carried out in the treatment group at 6, l8, and 36 months indicated that treatment effects were sustained. Using the eight predictor variable model rendered an acceptable predictive ability to identify responders both at post-test (AUC = .72, p < .01) and at 36-month follow-up (AUC = .70, p < .01). We conclude that ICBT for pathological gamblers, even if depressed, can be effective and that outcome can partly be predicted by pre-treatment characteristics.

Keywords: gambling, Internet, treatment, prediction

Introduction

Since pathological gambling was first introduced in the Diagnostic and statistical manual of mental disorders (American Psychiatric Association, 1980), there has been a fast development in the availability of games, primarily via the Internet (Hodgins, Stea, & Grant, 2011). One example is Sweden which, in 2006, introduced a state-owned Internet poker site which is essentially open all the time. Nowadays you can, in principle, sit in your underwear at home in the middle of the night with a whisky in your hand and gamble away all your savings. Social responsibility is said to be prioritised by the gaming operator, yet no peer-reviewed research has been published evaluating the effects of the preventive steps taken (e.g. warning messages) to ensure responsible gaming. The risks of developing pathological gambling habits are probably increased by the availability of online games, but this increased access has not been mirrored by increased access to psychological help in case gambling progresses into pathological gambling. This is unfortunate, since there are effective psychosocial treatments(Gooding & Tarrier, 2009; Pallesen, Mitsem, Kvale, John-sen, & Molde, 2005). However, only about 10% of persons with pathological gambling seek treatment (Ladouceur, 2005). Besides a lack of therapists, long waitlists, and costs, it is likely that shame and stigma influence the decision to not seek help (Evans & Delfabbro, 2005). Therefore, there is a need to lower the barriers for seeking help. One way of achieving this objective is to offer help online.

However, the progress of online treatments for gambling problems has been slow. For instance, in a recent systematic review of Internet-based therapy studies on addictions, Gainsbury and Blaszczynski (2011) concluded that only one study has been carried out with acceptable methodology on pathological gambling. In that particular study, carried out by our research group, Carlbring and Smit (2008) randomised 66 participants to a treatment group or a waitlist control. The results were generally encouraging, with treatment gains maintained up to 36 months after treatment completion. However, the trial excluded all participants with a depression score exceeding 21 points on the self-rated version of the Montgomery-Asberg Depression Rating Scale (Svanborg & Åsberg, 1994). Although the idea was to reduce the risk of including participants with suicidal ideation, since no face-to-face contact was offered, this procedure limits the ability to generalise, especially since more people were excluded (n = 158) than included (n = 66). It is uncertain if the results from that particular study can be generalised to the larger population of pathological gamblers. Consequently, the present study was designed to keep the exclusion frequency to a minimum by, for example, including participants regardless of their depression score.

Not everyone entering treatment for pathological gambling completes it and, of those who do, not everyone reaches high end-state functioning (Carlbring, Jonsson, Josephson, & Forsberg, 2010). Attempts have been made to try to predict who will drop out (Dowling, 2009) and who will benefit from treatment (Dowling, 2009; Dowling & Cosic, 2010; Dunn, Delfabbro, & Harvey, 2011; Raylu & Oei, 2007). In Internet-based programmes this is an appealing idea. If we knew who would not likely benefit from a low-intensity intervention, we could override the stepped care approach in which a person receives the lowest service tier in the first instance, only stepping up to more intensive services as clinically required (Bower & Gilbody, 2005). Unfortunately, in order for patients to progress to the next level, a treatment failure needs to be recognised. If we had away of predicting who would benefit from treatment, then those who would not benefit could skip the Internet-based treatment and go directly to a more intensive intervention.

The results of the pathological gambling treatment prediction studies are mixed. Some studies have suggested that high pre-treatment alcohol use is a negative predictor, while others have failed to replicate that finding (Dowling, 2009). Other studies have found that there is a lower likelihood of treatment response if the participant is young, living alone, has a low income and a low readiness to change (Dowling, 2009). In a study by Dowling on females presenting with pathological gambling, no differences could be found between dropouts and completers (Dowling, 2009). One reason for the inconclusiveness could be that researchers use different definitions of what a dropout is (cf. Eysenbach, 2005) and different predictors. Another problem is that groups are generally small, making statistical methods like logistic regression less suitable. Finally, there are reports that there might be different predictors in face-to-face therapy and Internet-based treatment (Andersson, Carlbring, & Grimund, 2008; Nordgreen et al., 2012).

In this study, we investigated predictors of who will have a successful outcome following ICBT by the use of eight variables collected in the screening phase. Previous studies have reasoned that since pathological gambling and alcohol and substance abuse share many commonalities, it is reasonable to assume that predictors of outcome in those studies also could have some relation to pathological gambling (Dowling, 2009). Even if results are inconclusive in the gambling field as to what variables can be used to predict the outcome, we decided to use eight different predictors in different domains that both theoretically and clinically would be plausible predictors of outcome. First, we entered work-life satisfaction as suggested by Sander and Peters (2009).

Second, primary gambling activity was used as a predictor variable since severity of gambling and psychosocial problems have been associated with preferred form of gambling activity (Champine & Petry, 2010). Third, debts due to gambling were entered, because financial difficulties have been reported to be an important motivator (Suurvali, Hodgins, & Cunningham, 2010). The fourth variable was social support, as lower levels have been shown to correlate with therapy dropout (Dunn et al., 2012; Melville, Casey, & Kavanagh, 2007; Oakes et al., 2011). Fifth, personal yearly salary was used as a proxy for socioeconomic status which is a known risk factor for pathological gambling (Welte, Barnes, Wieczorek, Tidwell, & Parker, 2004). Sixth, alcohol consumption was included as suggested by Rash, Weinstock, and Petry (2011), since it has been shown that those who drink while gambling tended to gamble in more risky ways and experience more negative consequences of gambling (Cronce & Corbin, 2010). Hazardous alcohol use has previously been linked to continued difficulties with gambling over a 7-year period (Abbott, Williams, & Volberg, 2004). Seventh, in accordance with the transtheoretical model of behaviour change (Prochaska, DiClemente, & Norcross, 1992), the individual's readiness to act on anew healthier behaviour (stage of change) was included as suggested by Wohl and Sztainert (2011). Finally, dissociative gambling was included as suggested by Jacobs (1988). Although pathological gamblers might not experience dissociative symptoms at a higher rate than normal controls (Ledgerwood & Petry, 2006), it still can be a possible predictor of treatment outcome as it can differentiate different subtypes (Milosevic & Ledgerwood, 2010).

Methods

Design

Since the superiority of the treatment over a waitlist control had previously been demonstrated (Carlbring & Smit, 2008), this trial was designed as a non-comparative, single group study with measurements at baseline and at 3, 6, 18, and 36 months. The effect of the intervention was assessed using four criteria: Gambling (NORC DSM-IV Screen for gambling problems [NODS]; Gerstein et al., 1999), anxiety, depression (Hospital Anxiety and Depression Scale [HADS]; Zigmond & Snaith, 1983), and overall life satisfaction(quality of life inventory [QOLI]; Frisch, Cornell, Villanueva, & Retzlaff, 1992). The study protocol was approved by the regional medical ethics committee (Vetting the ethics of research involving humans in Sweden) and written informed consent was obtained from the participants. The trial is registered in Clinical Trials with the identifier NCT 0 1381250.

Participants and recruitment

Advertisements in newspapers with the heading “Do you have trouble controlling your gambling?” and “Do you want to stop gambling?” were used to recruit participants (cf. Doiron & Nicki, 2007). Selection took place with a computerised screening interview consisting of the NODS (Gerstein et al., 1999), the HADS (Zigmond & Snaith, 1983), the QOLI (Frisch et al., 1992), and a number of additional questions related to gambling activities and demographics.

To be included in the study, participants had to meet the following criteria: (1) fulfil the DSM-IV-TR (American Psychiatric Association, 2000) criteria for pathological gambling according to the 1-month version of NODS, (2) live in Sweden, and (3) have gambled at least once in the past 30 days.

Of the 464 individuals who applied between May 2005 and May 2007, 316 fulfilled the criteria and 284 subsequently started treatments. The reasons for exclusion are specified in the CONSORT flowchart (see Figure 1). The mean age of the included 284 participants (81% male) was 32.2 (SD = 8.8; range: 18–62) and the mean age of onset of regular gambling was 23.7 (SD = 9.1). The mean duration of gambling problems was 5.7 years (SD = 5.5). The most prevalent type of game played was poker and video lottery terminals (VLTs). For a more detailed description, see Tables 1 and 2.

Figure 1.

Figure 1.

Participant flow, reasons for exclusion and number of participants providing data at different assessment points throughout the trial.

Table 1.

Treatment outcome at pre-, post-, 6-, 18-, and 36-month follow-up for the collapsed group (n = 284) as well as the five different subtypes of primaryatic game using mixed effects model

p Difference (overall) Mean estimates (SE), type of game
p Differences p Interaction
Time Mean estimates (SE) across all types of games Significant pair wise comparisons between time points Time Poker Time (n = 102) VLT (n = 76) Casino (n = 34) Betting (n = 43) Bingo (n = 29) Type of game Significant differences between each type of game at each time point Time × type of game
Gambling (NORC DSM-IV Screen for gambling problems)
Pre 8.1 (0.1) Pre > all times <0.01 8.2 (0.1) 8.5 (0.2) 8.0 (0.2) 8.0 (0.2) 7.7 (0.3) 0.042 VLT > Betting, Bingo 0.748
Post 1.8 (0.2) 2.0 (0.2) 2.3 (0.2 1.8 (0.3) 1.8 (0.3) 1.4 (0.3) VLT > Betting, Bingo
6 mo 2.0 (0.2) 2.1 (0.2) 2.4 (0.2) 1.9 (0.3) 1.9 (0.3) 1.5 (0.3) VLT > Betting, Bingo
18 mo 2.1 (0.2) 2.2 (0.2) 2.5 (0.3) 2.0 (0.3) 2.0 (0.3) 1.7 (0.3) VLT > Betting, Bingo
36 mo 2.0 (0.2) 2.1 (0.2) 2.4 (0.3) 1.9 (0.3) 1.9 (0.3) 1.5 (0.3) VLT > Betting, Bingo
Anxiety (Hospital anxiety and depression scale)
Pre 11.0 (0.3) Pre > all times <0.01 10.7 (0.4) 11.7 (0.4) 11.7 (0.6) 10.5 (0.6) 10.5 (0.7) 0.169 No difference 0.425
Post 7.2 (0.3) Post > 6, 18 6.8 (0.4) 7.9 (0.5) 7.8 (0.6) 6.6 (0.6) 6.7 (0.7) No difference
6 mo 6.4 (0.3) 6.1 (0.4) 7.1 (0.5) 7.1 (0.6) 5.9 (0.6) 5.9 (0.7) No difference
18 mo 6.3 (0.3) 5.9 (0.4) 7.0 (0.5) 7.0 (0.7) 5.8 (0.6) 5.8 (0.7) No difference
36 mo 6.5 (0.4) 6.1 (0.4) 7.2 (0.5) 7.1 (0.7) 5.9 (0.6) 6.0 (0.7) No difference
Depression (Hospital anxiety and depression scale)
Pre 9.3 (0.3) Pre > all times <0.01 8.8 (0.4) 9.3 (0.4) 10.6 (0.6) 9.3 (0.5) 8.6 (0.6) 0.061 Casino > Poker, VLT, Bingo 0.483
Post 5.5 (0.3) Post > 6 5.0 (0.3) 5.5 (0.4) 6.8 (0.6) 5.6 (0.5) 4.8 (0.6) Casino > Poker, VLT, Bingo
6 mo 4.8 (0.3) 4.3 (0.4) 4.8 (0.4) 6.1 (0.6) 4.8 (0.5) 4.1 (0.6) Casino > Poker, VLT, Bingo
18 mo 5.0 (0.3) 4.5 (0.4) 5.0 (0.4) 6.3 (0.6) 5.0 (0.5) 4.3 (0.6) Casino > Poker, VLT, Bingo
36 mo 5.0 (0.3) 4.6 (0.4) 5.1 (0.4) 6.4 (0.6) 5.2 (0.5) 4.4 (0.6) Casino > Poker, VLT, Bingo
Overall life satisfaction (quality of life inventory)
Pre 0.3 (0.1) Pre < all times <0.01 0.4 (0.2) 0.2 (0.2) −0.2 (0.3) 0.5 (0.2) 0.4 (0.3) 0.286 No difference 0.960
Post 1.3 (0.1) Post < 6, 18 1.5 (0.2) 1.3 (0.2) 0.9 (0.3) 1.5 (0.2) 1.5 (0.3) No difference
6 mo 1.6 (0.1) 1.7 (0.2) 1.5 (0.2) 1.2 (0.3) 1.8 (0.2) 1.7 (0.3) No difference
18 mo 1.6 (0.1) 1.7 (0.2) 1.5 (0.2) 1.2 (0.3) 1.8 (0.2) 1.7 (0.2) No difference
36 mo 1.4 (0.1) 1.5 (0.2) 1.3 (0.2) 1.0 (0.3) 1.6 (0.2) 1.5 (0.3) No difference

Note. VLT, video lottery terminal.

Table 2.

Factors associated with responder status at post-treatment (n = 218)

ORs, 95 CI (p value)
Number of individuals Responder (%) Unadjusted Adjusted
Work-life satisfaction
   Very poor 46 30.4 Ref. (p = 0.09) (p = 0.42)
   Poor 51 35.3 1.2 0.5–2.9 1.0 0.4–2.7
   Moderate 67 37.3 1.4 0.6–3.0 1.3 0.5–3.2
   Good 54 53.7 2.7 1.2–6.0 2.0 0.7–5.6
Primary gambling
   Poker 79 38.0 Ref. (p = 0.38) (p = 0.57)
   VLT 56 32.1 0.8 0.4–1.6 0.5 0.2–1.2
   Casino 25 36.0 0.9 0.4–2.3 1.1 0.3–3.2
   Betting 33 51.5 1.7 0.8–3.9 0.8 0.3–2.3
   Bingo 25 48.0 1.5 0.6–3.7 1.1 0.3–3.2
Debts due to gambling
   No 58 37.9 Ref. (p = 0.80) (p = 0.75)
   Yes 158 39.9 1.1 0.6–2.0 0.9 0.4–1.9
Social support
   No 65 32.3 Ref. (p = 0.24) (p = 0.29)
   Moderate 100 40.0 1.4 0.7–2.7 1.8 0.8–4.0
   Yes 50 48.0 1.9 0.9–4.1 1.7 0.7–4.2
Personal yearly salary (Swedish krona)
   0–150,000 48 37.5 Ref. (p = 0.85) (p = 0.54)
   151,000–210,000 52 44.2 1.3 0.6–2.9 1.5 0.5–4.0
   211,000–300,000 57 36.8 1.0 0.4–2.2 0.9 0.3–2.3
   301,000 or greater 53 41.5 1.2 0.5–2.6 0.7 0.3–2.1
Number of standard drinks during atypical drinking day
   0–2 drinks 51 54.9 2.7 1.4–5.4 2.7 1.2–6.3
   3–4 drinks 48 35.4 1.2 0.6–2.5 1.3 0.6–2.9
   5drinks or more 103 31.1 Ref. (p = 0.02) (p = 0.06)
Stage of change
   Contemplation 173 37.0 Ref. (p = 0.14) (p = 0.31)
   Action 38 50.0 1.7 0.8–3.5 1.5 0.7–3.5
Dissociative gambling
   Never 18 66.7 7.0 1.6–30.8 9.8 1.4–65.8
   Sometimes 83 43.4 2.7 0.8–8.8 3.0 0.7–12.7
   Often 93 33.3 1.7 0.5–5.8 1.9 0.5–8.2
   Always 18 22.2 Ref. (p = 0.03) (p = 0.08)

Notes. The total number of participants does not always add up to 218 for all factors owing to partly missing data. The goodness-of-fit test was positive (Hosmer and Lemenshow: p = .74).

Treatment

The intervention was based on established CBT methods, as described in self-help books (Hodgins, 2002; Ladouceur & Lachance, 2006) and is described in more detail in Carlbring and Smit (2008). Briefly, the treatment can be described as an eight-chapter book on the Internet, with each of the chapters covering a specific topic. The modules include information and exercises and end with three to eight essay-style questions. Participants were asked to answer the questions and provide worksheets and report on the outcomes of different exercises. For each module they were required to post at least one message in an online discussion group about a predetermined topic. Feedback on homework assignments was usually given via e-mail within 24 h after participants had sent their answers. Once a week telephone calls were made by the therapists to each participant. The purpose was to provide positive feedback and encouragement as well as to answer any questions the subjects had regarding the modules. Each conversation lasted approximately 15 min and a mean total of 6.7 (SD = 3.9) calls were made during the eight weeks.

The therapists were four social workers with an additional 2-year basic training in CBT and motivational interviewing (MI; Miller & Rollnick, 2002). The mean total time per week spent on each participant in this study was approximately 20 min, including telephone calls, administration, and responding to e-mails.

Predictor variables

A total of eight predictor variables were selected in order to predict responder status: (1) Work-life satisfaction was defined by the score in the corresponding question in the QOLI (Frisch et al., 1992) and subsequently categorised using Swedish norm data (Öst, Breitholtz, & Thulin, 1997). (2)Primary gambling activity was categorised by asking the participant (at screening) what game was most accountable for their problems. If two or more gambling types were reported, the one responsible for the majority of the problem was chosen. (3) Debts due to gambling were categorised as yes or no, regardless of the amount. (4) Social support was defined by the participant answering the question “Do you feel that you have someone available who can give you the right type of support to help you cope with life's stresses and problems?” (5) Personal yearly salary was divided into four categories, in an attempt to simulate no income, low income, moderate income, and high income. (6) The number of standard drinks during atypical drinking day was based on the corresponding question in the Alcohol Use Disorders Identification Test (Wennberg, 1996) (“How many drinks containing alcohol do you have on atypical day when you are drinking?”) (7) Stage of change was coded by analysing the response to the Swedish version of the Readiness to Change questionnaire. (8) Dissociative gambling was coded by analysing the response to a modified version of Jacobs’ question now reading “Have you ever completely lost track of time when gambling?” (Jacobs, 1988). Treatment response was defined as having a total of 0 points on the 1-month version of NODS and no incident of gambling over the past 30 days.

Statistical analysis

To test for treatment outcome we used mixed-effects models as suggested by Gueorguieva and Krystal (2004). To investigate possible predictive factors associated with the outcomes, we used logistic regression analysis. Continuous variables were categorised to gain clarity of effect and clinical interpretation. First, we studied crude (unadjusted) association of each factor with the odds of response. Second, to study the adjusted associations for possible confounders, we used multivariable logistic regression models with all factors included. The associations are presented as odds ratios (ORs) with 95% confidence intervals (CIs). There was no transformation or centreing with the pre intervention measurements.

This analysis was done because in the analysis of longitudinal data, repeated obser-vations for the same individual are correlated. If the repeated measures structure correlation is ignored, this may lead to imprecise variation estimates from the regression models, leading to incorrect statistical conclusions. Such correlation violates the assumption of independence necessary for more traditional, repeated measures analysis and might lead to bias in regression parameters which could lead to wrong conclusions (Brown & Prescott, 1999; Gueorguieva & Krystal, 2004). Furthermore, mixed effect models are more effective in accommodating missing data and the integration of time-varying factors, which are issues in this study.

When comparing the outcome for different subtypes of problematic games we used a covariance pattern model (Brown & Prescott, 1999), a special case of mixed-effects models. A separate model was estimated for each of the four outcome measures (HADS Anxiety, HADS Depression, NODS, and QOLI). The variance-covariance matrix was assumed to be block diagonal but unstructured within a block defined by subjects. To study if groups differed across the time points, with respect to the outcome measures, we tested the interaction between time and group. We used the restricted maximum likelihood (REML) as our model estimation method and present estimated means and differences between treatments and their respective standard error (SE) means.

To evaluate the ability of the predictor model to correctly discriminate between responders vs. non-responders, we calculated the area under the receiver operating curve (ROC; trapezoid rule). An area under the curve (AUC) equal to 0.5 suggests no discrimination, 0.7 to < 0.8 is acceptable, 0.8–0.9 is excellent, and > 0.9 is outstanding discrimination (Hosmer & Lemenshow, 1989). We used the nonparametric method to calculate the SE for the AUC used in the CI and in the comparison of the ROC curves. This gives a conservative estimate of the SE, which implies that the risk of a false difference (type I error) in the evaluation and comparison of the ROC curves is low (Hanley & McNeil, 1982). Hosmer and Lemeshow's goodness-of-fit test was used to examine whether the adjusted models adequately fitted the data, and a p value > .05 indicates acceptable fit.

Analyses were done in SPSS 18.0 (SPSS, Inc., Chicago, IL, USA).

Results

Treatment outcome

The effects of the treatment are presented in three ways. First, we examine whether the treatment had any effect if type of gambling problem was not taken into account and the participants were analysed as a whole. Then the participants are divided in to five subgroups based on the principal type of gambling problem and the analysis is repeated to see whether the results persist. Finally, we analysed whether there was any interaction between the type of gambling problem and the treatment outcome.

As shown in the first four columns of Table 1, there were statistically significant main effects of time on all outcome measurements when analysing the whole group, with statistically significant improvements in all four outcomes from pre- to post-intervention. This suggests that the treatment might have a significant positive impact on the participants and that improvements were maintained after 6, 18, and 36 months.

One of the objectives of the present study was to see how results would differ from a previous study (Carlbring & Smit, 2008) by including participants with more severe depression (MADRS-S score of 21 or higher). For the participants with more severe depression, the changes in outcome scores (pre-compared to post-intervention) for the gambling-specific scale NODS (n(241) = .93; p = .352) and for the anxiety measure (HADS-A; n(241) = 1.58; p = .115) were similar to those for participants with less severe depression. However, and fully consistent with the regression to the mean effect, the more severely depressed group had a significantly larger change in depression score (HADS-D; n(241) = 3.04; p = .003) compared to the less severely depressed. The more severely depressed group also benefited more in terms of quality of life (QOLI; n(241) = 2.43; p = .016).

Since there was no significant interaction between time and gambling problem, the conclusion drawn is that there is no evidence that the type of gambling activity affects the treatment outcome (see Table 1). However, there is a significant main effect with regard to time on NODS. This is explained by the fact that there was a significant difference between those who gambled at VLTs, those who bet money on sports and those who played bingo. This initial difference in the NODS score was then maintained throughout over all measuring points. For those who were primarily casino gamblers, there was asimilar trend in depression scores. What was common throughout was that none of these significant differences between the types of gambling are clinically relevant, since all effect sizes were insignificant (Cohen's d < .20).

Prediction

As evident from Table 2, responder status at post-treatment was significantly associated with the background variables of alcohol consumption and dissociative gambling in the unadjusted analysis. Specifically, consumption of three or more standard drinks during a typical drinking day reduced the ratio of responder status to half compared with those who typically consume two drinks or less. Therefore, low alcohol consumption was associated with a higher response rate. In the adjusted analysis only trends were identified (p = .056 and p = .075, respectively).

As evident from Table 3, three variables have a significant individual predictive ability: work-life satisfaction, number of standard drinks during a typical drinking day, and dissociative gambling. The type of primary problematic game showed only a trend (p = .054). How-ever, the AUC for each of these individual variables was relatively low (.58-.61).

Table 3.

Evaluation of the ability to predict responder status at post-treatment for different predictor variables and models

AUC
Asymptotic 95% CI
Test result variable (s) Area SEa Asymptotic sig.b Lower bound Upper bound
1. Work-life satisfaction 0.596 0.042 0.023 0.515 0.678
2. Primary gambling 0.582 0.042 0.054 0.500 0.663
3. Debts due to gambling 0.510 0.042 0.812 0.427 0.593
4. Social support 0.548 0.042 0.257 0.466 0.630
5. Personal yearly salary 0.526 0.042 0.547 0.442 0.609
6. Standard drinks 0.597 0.042 0.023 0.514 0.679
7. Stage of change 0.548 0.043 0.262 0.464 0.631
8. Dissociative gambling 0.609 0.041 0.010 0.528 0.691
Model with all eight predictors 0.718 0.038 0.000 0.643 0.793

Note. Results are expressed by the area under the ROC.

a

Under the nonparametric assumption.

b

Null hypothesis: true area = .5.

As evident from Table 3, the model with all eight variables resulted in an acceptable predictive ability to identify responders at post-treatment (AUC = .72, p < .01). How-ever, similar results were obtained with a model with four variables (work-life satisfaction, primary gambling, dissociative gambling, and number of standard drinks during atypical drinking day; AUC = .69), and a model with only two variables (dissociative gambling and number of standard drinks during atypical drinking day; AUC = .67). Furthermore, there were no significant differences between the three models since the 95% CI for AUC all overlap. In conclusion, the number of standard drinks during atypical drinking day and dissociative gambling have the strongest association with responder status at post-treatment and have a marginal predictive ability in this sample (AUC = .67).

Table 4 shows how responder status was associated with the prediction variables at 36-month follow-up. Only one variable, debts due to gambling, was significantly associated with a positive response. Both the unadjusted and the adjusted analyses showed that the response ratio was about twice as high for individuals with debt compared with the reference category of not having a debt. As shown in Table 5, the proportion of responders was relatively evenly distributed. No single variable had any significant predictive ability to discriminate between responder vs. non-responder at 36 months—not even debts due to gambling (AUC = .57, p = .129). However, a model with all eight variables together rendered an acceptable predictive ability to identify responders (AUC = .70, p < .01). Table 5 shows that an eight-variable model has a greater chance of identifying a responder.

Table 4.

Factors associated with responder status at 36-month follow-up (n = 196)

ORs, 95 CI (p value)
Number of individuals Responder (%) Unadjusted Adjusted
Work-life satisfaction
   Very poor 43 37.2 Ref. (p = 0.39) (p = 0.29)
   Poor 45 37.8 1.0 0.4–2.4 0.8 0.3–2.1
   Moderate 60 38.3 1.0 0.5–2.4 0.9 0.4–2.4
   Good 48 52.1 1.8 0.8–4.2 1.9 0.7–5.1
Primary gambling
   Poker 75 38.7 Ref. (p = 0.78) (p = 0.40)
   VLT 51 45.1 1.3 0.6–2.7 1.8 0.8–4.3
   Casino 27 48.1 1.5 0.6–3.6 1.2 0.4–3.5
   Betting 27 33.3 0.8 0.3–2.0 0.6 0.2–1.9
   Bingo 16 43.8 1.2 0.4–3.7 1.5 0.4–5.8
Debts due to gambling
   No 51 29.4 Ref. (p < 0.05) (p < 0.05)
   Yes 143 45.5 2.0 1.0–3.9 2.2 1.0–4.9
Social support
   No 59 35.6 Ref. (p = 0.09) (p = 0.20)
   Moderate 89 38.2 1.1 0.6–2.2 1.1 0.5–2.3
   Yes 45 55.6 2.3 1.0–5.0 2.2 0.9–5.7
Personal yearly salary (Swedish krona)
   0–150,000 47 40.4 Ref. (p = 0.88) (p = 0.90)
   151,000–210,000 46 41.3 1.0 0.5–2.4 0.7 0.3–2.1
   211,000–300,000 51 47.1 1.3 0.6–2.9 1.1 0.4–2.8
   301,000 or greater 43 39.5 1.0 0.4–2.2 0.9 0.3–2.7
Number of standard drinks during atypical drinking day
   0–2 drinks 44 36.4 Ref. (p = 0.49) (p = 0.34)
   3–4 drinks 39 35.9 1.0 0.4–2.4 1.0 0.3–2.7
   5drinks or more 98 44.9 1.4 0.7–3.0 1.7 0.7–3.8
Stage of change
   Contemplation 160 39.4 Ref. (p = 0.16) (p = 0.94)
   Action 30 53.3 1.8 0.8–3.9 1.5 0.6–3.7
Dissociative gambling
   Never 14 42.9 1.2 0.3–5.4 1.7 0.2–12.3
   Sometimes 76 38.2 1.0 0.3–3.1 1.2 0.3–4.9
   Often 85 44.7 1.3 0.4–4.0 1.1 0.3–4.5
   Always 16 37.5 Ref. (p = 0.84) (p = 0.95)

Notes. The total number of participants does not always add up to 196 for all factors owing to partly missing data. Hosmer and Lemenshow: p = .65.

Table 5.

Evaluation of the ability to predict responder status at post-treatment for different predictor variables and models

AUC
Asymptotic 95% CI
Test result variable (s) Area SEa Asymptotic sig.b Lower bound Upper bound
1. Work-life satisfaction 0.558 0.045 0.189 0.471 0.646
2. Primary gambling 0.566 0.044 0.135 0.481 0.652
3. Debts due to gambling 0.567 0.044 0.129 0.482 0.653
4. Social support 0.577 0.044 0.084 0.490 0.664
5. Personal yearly salary 0.529 0.045 0.516 0.441 0.616
6. Standard drinks 0.552 0.044 0.242 0.466 0.638
7. Stage of change 0.532 0.045 0.478 0.444 0.619
8. Dissociative gambling 0.526 0.044 0.552 0.440 0.613
Model with all eight predictors 0.702 0.041 0.000 0.622 0.781

Note. Results are expressed by the area under the ROC.

a

Under the nonparametric assumption.

b

Null hypothesis: true area = .5.

Discussion

The purpose of this study was twofold—to investigate whether the treatment was effective when participants with symptoms of depression were included and whether treatment response could be predicted by a set of pre-treatment characteristics. First, the intervention showed promise both immediately following the treatment termination and up to the 36-month follow-up. Second, it was possible to predict who will benefit from Internet-based treatment both at post-test and at 36 months by using a model of eight background variables.

While the results point to the value of guided ICBT for pathological gambling, we cannot be certain since no control group was included. This is a weakness in the design as it opens up the possibility that the observed effects are better explained by regression to the mean or spontaneous remission. Indeed, there are reports of significant proportions of participants who naturally recover (Petry, 2005; Slutske, 2006). On the other hand, when the same treatment was compared to a waitlist control in a similar, controlled study (Carlbring & Smit, 2008), there were statistically significant changes as a result of the intervention. The changes observed in the present study are similar in magnitude to those seen previously for all four outcomes (gambling, anxiety, depression, and quality of life), indicating that the effects are probably not due to regression to the mean.

There was an important difference between the population in the previous study, which did not include subjects with a score higher than 21 on the MADRS-S, and the population in this study which did include such subjects. When these populations were analysed separately in the present study, the intervention had a similar effect in more severely depressed participants as for those with less severe depression for gambling and anxiety, but a greater effect for depression and quality of life. These results should be considered in light of the fact that they are not as statistically robust since they were shown in a subgroup of the main study population. Furthermore, the difference in depression score is likely to be an effect of regression to the mean, since these subjects had higher depression scores pre-intervention. Hence, future studies could investigate whether participants meeting DSM criteria for comorbid major depression really benefit more from the intervention.

Interestingly, there were significant differences in depression levels and NODS points between the subgroups of gamblers depending on what game they played, but that did not appear to affect outcome. Since type of game did not interact with treatment outcome, we conclude that no subgroup benefited more from the Internet-based treatment. This is also reflected by the overlapping ORs in the prediction analysis. It seems that other variables are more important for the outcome rather than the type of game played.

The results from the logistic regression suggested that it is possible to predict who will respond to ICBT both at post-intervention and at 36 months using a model with eight variables. The single most important background variables vary over time from alcohol consumption and dissociative gambling at post-intervention to whether one had a gambling debt or not at the 3-year follow-up. While the results are interesting, it is still unclear whether the model is valid for other treatment modalities (e.g. group treatment and individual psychotherapy). The fact that having a gambling debt increased the ratio of later being a treatment responder is not consistent with what has been found previously. However, in a study by Ingle, Marotta, McMillan, and Wisdom (2008), having a small gambling debt was better than not having one. It is possible that the dichotomy in this study can account for the difference, but it is more likely that the two studies used different time frames. Consistent with the Ingle et al's study was that having social support seemed to have a positive influence.

A major issue in prediction analysis is how the outcome variable is defined. In this study, it was defined as having 0 points on the 1-month version of NODS and not having gambled at all over the past 30 days. It is possible that the results would have been different had the response been defined, for example, using the Jacobson and Truax formula for clinical significant improvement (Jacobson & Truax, 1991). Also, the results might be specific to different subgroups. This trial grouped together participants regardless whether they, for example, had any comorbid psychiatric disorder. Furthermore, it could be argued that other predictor variables could be important, such as sex, age, whether or not a person is gambling primarily for sensations seeking or to escape from anxiety, and the reason for seeking help (Dowling, 2009; Grant, Kim, Hollander, & Potenza, 2008; Jamieson, Mazmanian, Penney, Black, & Nguyen, 2011).

In summary, the results from this study generally provide evidence for the continued use and development of guided Internet-delivered self-help with telephone support for pathological gamblers. In addition, this study was probably the first to suggest a model for predicting treatment outcomes for pathological gambling by using guided Internet-based treatment. However, the results need to be replicated to ensure that this is not only sample-specific associations. In addition, mechanisms of change needs to be addressed (cf. Andersson, Carlbring, Berger, Almlöv, & Cuijpers, 2009).

Acknowledgements

This study was funded by a grant from the Swedish National Institute of Public Health. We would like to thank Ann-Katrin Edlund, Birgitta Enmark, Anna Marklund, and Emma Edlund for serving as therapists and Maria Svensk, Maria Bergström, and Lise Bergman-Nordgren for their help with data collection.

References

  1. Abbott M. W. Williams M. M. Volberg R. A. A prospective study of problem and regular nonproblem gamblers living in the community. Substance Use & Misuse. 2004;39(6):855–884. doi: 10.1081/ja-120030891. [DOI] [PubMed] [Google Scholar]
  2. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 3rd ed. Washington, DC: American Psychiatric Press; 1980. [Google Scholar]
  3. American Psychiatric Association. Diagnostic and statistical manual of mental disorders— Text revision. 4th ed. Washington, DC: American Psychiatric Press; 2000. [Google Scholar]
  4. Andersson G. Carlbring P. Berger T. Almlöv J. Cuijpers P. What makes Internet therapy work? Cognitive Behaviour Therapy. 2009;38(Suppl. 1):55–60. doi: 10.1080/16506070902916400. [DOI] [PubMed] [Google Scholar]
  5. Andersson G. Carlbring P. Grimund A. Predicting treatment outcome in Internet versus face to face treatment of panic disorder. Computers in Human Behavior. 2008;24(5):1790–1801. doi: 10.1016/j.chb.2008.02.003. [Google Scholar]
  6. Bower P. Gilbody S. Stepped care in psychological therapies: Access, effectiveness and efficiency: Narrative literature review. The British Journal of Psychiatry. 2005;186(1):11–17. doi: 10.1192/bjp.186.1.11. doi: 10.1192/bjp.186.1.11. [DOI] [PubMed] [Google Scholar]
  7. Brown H. Prescott R. Applied mixed models in medicine. New York: Oxford University Press, Inc.; 1999. [Google Scholar]
  8. Carlbring P. Jonsson J. Josence H. Forsberg L. Motivational interviewing versus cognitive behavioral group therapy in the treatment of problem and pathological gambling: A randomized controlled trial. Cognitive Behaviour Therapy. 2010;39(2):92–103. doi: 10.1080/16506070903190245. doi: 10.1080/16506070903190245, 917437376 [pii] [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Carlbring P. Smit F. Randomized trial of Internet-delivered self-help with telephone support for pathological gamblers. Journal of Consulting and Clinical Psychology. 2008;76(6):1090–1094. doi: 10.1037/a0013603. doi:10.1037/a 0013603, 2008-16943-010 [pii] [DOI] [PubMed] [Google Scholar]
  10. Champine R.B. Petry N.M. Pathological gamblers respond equally well to cognitive-behavioral therapy regardless of other mental health treatment status. American Journal of Addictions. 2010;19(6):550–556. doi: 10.1111/j.1521-0391.2010.00085.x. doi: 10.1111/j.1521-0391.2010.00085.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Cronce J. M. Corbin W.R. Effects of alcohol and initial gambling outcomes on within-session gambling behavior. Experimental and Clinical Psychopharmacology. 2010;18(2):145–157. doi: 10.1037/a0019114. doi: 10.1037/a0019114, 2010-06751-005 [pii] [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Doiron J. P. Nicki R. M. Prevention of pathological gambling: A randomized controlled trial. Cognitive Behaviour Therapy. 2007;36(2):74–84. doi: 10.1080/16506070601092966. doi: 10.1080/16506070601092966. [DOI] [PubMed] [Google Scholar]
  13. Dowling N. Client characteristics associated with treatment attrition and outcome in female pathological gambling. Addiction Research & Theory. 2009;17(2):205–219. doi: 10.1080/16066350802346193. [Google Scholar]
  14. Dowling N. Cosic S. Client engagement characteristics associated with problem gambling treatment outcomes. International Journal of Mental Health and Addiction. 2010;9(6):656–671. doi: 10.1007/s11469-010-9298-x. [Google Scholar]
  15. Dunn K. Delfabbro P. Harvey P. A preliminary, qualitative exploration of the influences associated with drop-out from cognitive-behavioural therapy for problem gambling: An Australian perspective. Journal of Gambling Studies. 2012;28(2):253–272. doi: 10.1007/s10899-011-9257-x. doi: 10.1007/s10899-011-9257-x. [DOI] [PubMed] [Google Scholar]
  16. Evans L. Delfabbro P. H. Motivators for change and barriers to help-seeking in Australian problem gamblers. Journal of Gambling Studies. 2005;21(2):133–155. doi: 10.1007/s10899-005-3029-4. [DOI] [PubMed] [Google Scholar]
  17. Eysenbach G. The law of attrition. Journal of Medical Internet Research. 2005;7(1):e11. doi: 10.2196/jmir.7.1.e11. doi: 10.2196/jmir.7.1.e11, v7e11 [pii] [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Frisch M. B. Cornell J. Villanueva M. Retzlaff P. J. Clinical validation of the Quality of Life Inventory. A measure of life satisfaction for use in treatment planning and outcome assessment. Psychological Assessment. 1992;4(1):92–101. [Google Scholar]
  19. Gainsbury S. Blaszczynski A. A systematic review of Internet-based therapy for the treatment of addictions [Review] Clinical Psychology Review. 2011;31(3):490–498. doi: 10.1016/j.cpr.2010.11.007. doi: 10.1016/j.cpr.2010.11.007. [DOI] [PubMed] [Google Scholar]
  20. Gerstein D. Murphy S. Toce J. Hoffmann A. Palmer R. Johnson C. Larison L. Chuchro A. Bard L. Englemann M. Hill T. Buie R. Volberg H. Harwood A. Tucker E. Christiansen W. Cummings Sinclair S. Gambling impact and behaviour. Chicago, IL: National Gambling Impact Study Commission; 1999. [Google Scholar]
  21. Gooding P. Tarrier N. A systematic review and meta-analysis of cognitive-behavioural interventions to reduce problem gambling: Hedging our bets? Behaviour Research and Therapy. 2009;47(7):592–607. doi: 10.1016/j.brat.2009.04.002. doi: 10.1016/j. brat.2009.04.002, S0005-7967(09)00087-4 [pii] [DOI] [PubMed] [Google Scholar]
  22. Grant J. Kim S. Hollander E. Potenza M. Predicting response to opiate antagonists and placebo in the treatment of pathological gambling. Psychopharmacology. 2008;200(4):521–527. doi: 10.1007/s00213-008-1235-3. doi: 10.1007/s00213-008-1235-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Gueorguieva R. Krystal J. H. Move over ANOVA: Progress in analyzing repeated-measures data and its reflection in papers published in the archives of general psychiatry. Archives of General Psychiatry. 2004;61(3):310–317. doi: 10.1001/archpsyc.61.3.310. [DOI] [PubMed] [Google Scholar]
  24. Hanley J. A. McNeil B. J. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36. doi: 10.1148/radiology.143.1.7063747. [DOI] [PubMed] [Google Scholar]
  25. Hodgins D. C. Becoming a winner: Defeating problem gambling: A self-help manual for problem gamblers. Calgary, AB: Addiction Centre, Calgary Regional Health Authority; 2002. [Google Scholar]
  26. Hodgins D. C. Stea J. N. Grant J. E. Gambling disorders. Lancet. 2011;378(9806):1874–1884. doi: 10.1016/S0140-6736(10)62185-X. doi: 10.1016/S0140-6736(10)62185-X. [DOI] [PubMed] [Google Scholar]
  27. Hosmer D. W. Lemenshow S. Applied logistic regression. New York: Wiley; 1989. [Google Scholar]
  28. Ingle P. Marotta J. McMillan G. Wisdom J. Significant others and gambling treatment outcomes. Journal of Gambling Studies. 2008;24(3):381–392. doi: 10.1007/s10899-008-9092-x. doi: 10.1007/s10899-008-9092-x. [DOI] [PubMed] [Google Scholar]
  29. Jacobs D. F. Evidence for a common dissociative-like reaction among addicts. Journal of Gambling Studies. 1988;4(1):27–37. doi: 10.1007/bf01043526. [Google Scholar]
  30. Jacobson N. S. Truax P. Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology. 1991;59(1):12–19. doi: 10.1037//0022-006x.59.1.12. [DOI] [PubMed] [Google Scholar]
  31. Jamieson J. Mazmanian D. Penney A. Black N. Nguyen A. When problem gambling is the primary reason for seeking addiction treatment. International Journal of Mental Health and Addiction. 2011;9(2):180–192. doi: 10.1007/s11469-009-9268-3. [Google Scholar]
  32. Ladouceur R. Controlled gambling for pathological gamblers. Journal of Gambling Studies. 2005;21(1):49–57. doi: 10.1007/s10899-004-1923-9. [DOI] [PubMed] [Google Scholar]
  33. Ladouceur R. Lachance S. Overcoming your pathological gambling: Workbook. Oxford: Oxford University Press; 2006. [Google Scholar]
  34. Ledgerwood D.M. Petry N. M. Psychological experience of gambling and subtypes of pathological gamblers. Psychiatry Research. 2006;144(1):17–27. doi: 10.1016/j.psychres.2005.08.017. doi: 10.1016/j.psy-chres.2005.08.017, S0165-1781(05)00281-7 [pii] [DOI] [PubMed] [Google Scholar]
  35. Melville K. M. Casey L. M. Kavanagh D. J. Psychological treatment dropout among pathological gamblers. Clinical Psychology Review. 2007;27(8):944–958. doi: 10.1016/j.cpr.2007.02.004. doi:10.1016/j.cpr. 2007.02.004, S0272-7358(07)00057-8 [pii] [DOI] [PubMed] [Google Scholar]
  36. Miller W. R. Rollnick S. Motivational interviewing: preparing people for change. New York: Guilford Press; 2002. [Google Scholar]
  37. Milosevic A. Ledgerwood D. M. The subtyping of pathological gambling: A compre-hensive review. Clinical Psychology Review. 2010;30(8):988–998. doi: 10.1016/j.cpr.2010.06.013. doi: 10.1016/j.cpr.2010.06.013, S0272-7358(10)00107-8 [pii] [DOI] [PubMed] [Google Scholar]
  38. Nordgreen T. Havik O. E. Ost L. G. Furmark T. Carlbring P. Andersson G. Outcome predictors in guided and unguided self-help for social anxiety disorder [randomized controlled trial] Behaviour Research and Therapy. 2012;50(1):13–21. doi: 10.1016/j.brat.2011.10.009. doi: 10.1016/j.brat. 2011.10.009. [DOI] [PubMed] [Google Scholar]
  39. Oakes J. Pols R. Battersby M. Lawn S. Pulvirenti M. Smith D. A focus group study of predictors of relapse in electronic gaming machine problem gambling. Part 2: Factors that ‘pull’ the gambler away from relapse. Journal of Gambling Studies. 2011. doi: 10.1007/s10899-011-9267-8. [DOI] [PubMed]
  40. Öst L.-G. Breitholtz E. Thulin U. Quality of life in anxiety disorder patients. 1997. Unpublished manuscript. Stockholm.
  41. Pallesen S. Mitsem M. Kvale G. Johnsen B.-H. Molde H. Outcome of psychological treatments of pathological gambling: A review and meta-analysis. Addiction. 2005;100(10):1412–1422. doi: 10.1111/j.1360-0443.2005.01204.x. [DOI] [PubMed] [Google Scholar]
  42. Petry N. M. Stages of change in treatment-seeking pathological gamblers. Journal of Consulting and Clinical Psychology. 2005;73(2):312–322. doi: 10.1037/0022-006X.73.2.312. [DOI] [PubMed] [Google Scholar]
  43. Prochaska J. O. DiClemente C. C. Norcross J. C. In search of how people change: Applications to addictive behaviors. American Psychologist. 1992;47(9):1102–1114. doi: 10.1037//0003-066x.47.9.1102. [DOI] [PubMed] [Google Scholar]
  44. Rash C. J. Weinstock J. Petry N. M. Drinking patterns of pathological gamblers before, during, and after gambling treatment. Psychology of Addictive Behaviors. 2011;25(4):664–674. doi: 10.1037/a0025565. doi:10. 1037/a0025565, 2011-21202-001 [pii] [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Raylu N. Oei T. Factors that predict treatment outcomes in a community treatment agency for problem gamblers. International Journal of Mental Health and Addiction. 2007;5(2):165–176. doi: 10.1007/s11469-007-9056-x. [Google Scholar]
  46. Sander W. Peters A. Pathological gambling: influence of quality of life and psychological distress on abstinence after cognitive-behavioral inpatient treatment. Journal of Gambling Studies. 2009;25(2):253–262. doi: 10.1007/s10899-009-9128-x. doi: 10.1007/s10899-009-9128-x. [DOI] [PubMed] [Google Scholar]
  47. Slutske W. S. Natural recovery and treatment-seeking in pathological gambling: Results of two U.S. National Surveys. American Journal of Psychiatry. 2006;163(2):297–302. doi: 10.1176/appi.ajp.163.2.297. [DOI] [PubMed] [Google Scholar]
  48. Suurvali H. Hodgins D. C. Cunningham J. A. Motivators for resolving or seeking help for gambling problems: A review of the empirical literature. Journal of Gambling Studies. 2010;26(1):1–33. doi: 10.1007/s10899-009-9151-y. doi: 10.1007/s10899-009-9151-y. [DOI] [PubMed] [Google Scholar]
  49. Svanborg P. Åsberg M. A new self-rating scale for depression and anxiety states based on the comprehensive psychopathological rating scale. ACTA Psychiatrica Scandinavica. 1994;89(1):21–28. doi: 10.1111/j.1600-0447.1994.tb01480.x. [DOI] [PubMed] [Google Scholar]
  50. Welte J. W. Barnes G. M. Wieczorek W. F. Tidwell M. C. Parker J. C. Risk factors for pathological gambling. Addictive Behaviors. 2004;29(2):323–335. doi: 10.1016/j.addbeh.2003.08.007. S0306460303001205 [pii] [DOI] [PubMed] [Google Scholar]
  51. Wennberg P. The alcohol use disorders identification test (AUDIT): A psychometric evaluation. 1996. Reports from the Department of Psychology, Stockholm University.
  52. Wohl M. J. Sztainert T. Where did all the pathological gamblers go? Gambling symptomatology and stage of change predict attrition in longitudinal research. Journal of Gambling Studies. 2011;27(1):155–169. doi: 10.1007/s10899-010-9186-0. doi: 10.1007/s10899-010-9186-0. [DOI] [PubMed] [Google Scholar]
  53. Zigmond A. S. Snaith R. P. The Hospital Anxiety and Depression Scale. Acta Psychiatrica Scandinavica. 1983;67(6):361–370. doi: 10.1111/j.1600-0447.1983.tb09716.x. [DOI] [PubMed] [Google Scholar]

Articles from Cognitive Behaviour Therapy are provided here courtesy of Taylor & Francis

RESOURCES