Abstract
Objective To evaluate the efficacy of two internet interventions for community-dwelling individuals with symptoms of depression—a psychoeducation website offering information about depression and an interactive website offering cognitive behaviour therapy.
Design Randomised controlled trial.
Setting Internet users in the community, in Canberra, Australia.
Participants 525 individuals with increased depressive symptoms recruited by survey and randomly allocated to a website offering information about depression (n = 166) or a cognitive behaviour therapy website (n = 182), or a control intervention using an attention placebo (n = 178).
Main outcome measures Change in depression, dysfunctional thoughts; knowledge of medical, psychological, and lifestyle treatments; and knowledge of cognitive behaviour therapy.
Results Intention to treat analyses indicated that information about depression and interventions that used cognitive behaviour therapy and were delivered via the internet were more effective than a credible control intervention in reducing symptoms of depression in a community sample. For the intervention that delivered cognitive behaviour therapy the reduction in score on the depression scale of the Center for Epidemiologic Studies was 3.2 (95% confidence interval 0.9 to 5.4). For the “depression literacy” site (BluePages), the reduction was 3.0 (95% confidence interval 0.6 to 5.2). Cognitive behaviour therapy (MoodGYM) reduced dysfunctional thinking and increased knowledge of cognitive behaviour therapy. Depression literacy (BluePages) significantly improved participants' understanding of effective evidence based treatments for depression (P < 0.05).
Conclusions Both cognitive behaviour therapy and psychoeducation delivered via the internet are effective in reducing symptoms of depression.
Introduction
The development of effective large scale intervention programmes for depression is a priority for the community, given that depression is a major cause of disability and many individuals with depression do not receive adequate treatment.1-3 The internet has the potential to deliver self help interventions globally and to people who do not seek or receive help for depression4,5 and harness the technology of the computer, already shown to provide effective treatment for depression in clinical settings.6,7
Internet based programmes to prevent anxiety and panic in populations of students and patients have been effective.8,9 However, to date only one randomised controlled trial of depression treatment using the internet has been published. Clarke et al randomised patients to either a website (offering cognitive behaviour therapy) or a control intervention (the home page of a health maintenance organisation).5 The study did not find differences in depression, but the modal number of visits to the site was low. Post hoc analyses showed a modest effect in people with lower initial scores, which implies that the internet may be a useful tool, at least for some groups.
We used participants recruited directly from the community to investigate this possibility by comparing the effects of a website for psychoeducation and a website offering cognitive behaviour therapy with a control intervention,. One site (BluePages, http://bluepages.anu.edu.au) provided depression literacy, offering evidence based information (at 8th grade reading level) on depression and its treatment.10 The other site (MoodGYM, http://moodgym.anu.edu.au) offered cognitive behaviour therapy for the prevention of depression.11 These interventions were compared with a control intervention using an “attention placebo,” which provided weekly contact with a lay interviewer to discuss lifestyle factors such as exercise, education, and health habits.
We expected both sites to be more effective than the control intervention in reducing depression symptoms and in improving “depression literacy.” BluePages was predicted to improve depression literacy, whereas MoodGYM was predicted to improve symptoms of depression and dysfunctional thoughts more than BluePages.
Methods
Participants and design
Recruitment was via a questionnaire posted to 27 000 people aged 18-52 years in Canberra. We randomly selected participants from the electoral roll by using the “select cases” random selection option in SPSS. Altogether 6122 people (22.7%) returned questionnaires. Of these, 752 indicated a willingness to participate, had access to the internet, scored 22 or above on the Kessler psychological distress scale,12 and were not receiving clinical care from either a psychologist or psychiatrist. Consent forms and pre-intervention questionnaires were posted to 656 of the 752 individuals. Of these, 525 (150 men, 375 women), aged 36.43 (SD = 9.4) years, completed the forms and were randomised to groups. The 525 represented about 2% of the original survey group, 9% of those who returned surveys, and 69% of those eligible to complete the trial (figure). Participants randomised to the trial had higher scores on the Kessler scale than the remainder of the 6122 who returned questionnaires (17.8 (5.2) compared with 7.34 (6.7); P = 0.0001).
Figure 1.
Flow of participants through the trial
Once individuals had returned consent forms a statistician used an SPSS function to assign them randomly to one of three treatment groups. The intervention was staggered into intake phases from August 2002, with post-intervention questionnaires collected by May 2003. The mean time from random allocation to the start of the programme was 1.7 (1) weeks.
Procedures for BluePages and MoodGYM
Lay interviewers contacted participants weekly by phone to direct their use of the websites. Interviewers received instruction booklets providing timetables for tracking phone calls and verbatim instructions. Participants were sent detailed guides outlining navigation and weekly assignments for MoodGYM or BluePages. Both websites are available freely on the world wide web. For BluePages, participants were directed each week to one of five sections of the website (for example, the “Symptoms” section in week 1), with an overview. at six weeks. Participants in both interventions were given a login identification number and MoodGYM participants undertook online assessments. MoodGYM consisted of five interactive modules, which were made available sequentially weekly. The participants revised all aspects of the programme in the sixth week. Post-intervention questionnaires were posted at six weeks.
Procedures for the control intervention
Participants were phoned weekly by interviewers to discuss lifestyle and environmental factors that may have an influence on depression. Topics covered included physical and artistic activities (week 1); education and hobbies (week 2); social, financial, and family roles (week 3); work habits and stress (week 4); physical health, medications, and pain (week 5); and nutrition and alcohol (week 6).
Measures
Descriptive variables—Sex, age, marital status, years of education, and previous history of depression.
Screening—Scores on the Kessler psychological distress scale range from 0 to 40, with higher scores indicating greater distress.12
Preference for treatments—Participants' preferences for treatments were assessed by an item asking which of three activities would interest them the most: to visit a website that provides a set of skills for preventing depression; to visit a website that provides information about depression; or to do an assessment with a trained interviewer about lifestyle and environmental factors that could increase their risk of depression.
Measures of symptom change—The self report 20 item depression scale from the Center for Epidemiologic Studies was the primary outcome measure.13 Scores range from 0 to 60, with scores 16 or higher reflecting clinical depression. Dysfunctional thoughts were measured by the automatic thoughts questionnaire.14 Scores range from 30 to 150, with higher scores indicating greater severity.
Depression literacy—We assessed attitudes to treatments by asking the respondent to rate whether each of the interventions listed on the website were likely to be helpful, harmful, or neither for someone with depression. These were scored by summing the number of “helpful” ratings given to interventions that had two or three “smiley faces.” Medical literacy was scored 0-2, with higher scores indicating correct responses for the two medical treatments supported by the evidence base (antidepressant medication and electroconvulsive therapy). The maximum score for psychological literacy was 3 (scores for cognitive behaviour therapy, interpersonal psychotherapy, and bibliotherapy) and the maximum score for lifestyle literacy was 3, based on rating exercise, St John's wort, and light therapy, as supported by the evidence. We used a 16 item “true or false” test that assessed the content and key principles of MoodGYM to assess knowledge of cognitive behaviour therapy principles.
Statistical analysis
We estimated effect sizes of 0.2 for the control intervention, 0.4 for BluePages, and 0.6 for MoodGYM before the trial. Our sample of 525 provided a power of 88% to detect a difference in effect size of 0.3. We carried out analyses with and without adjustment for education on an intention to treat basis; we assigned their baseline scores to participants who did not respond after the intervention so that no change was assumed to have occurred. We also undertook additional analyses with and without adjustment for education for individuals who completed the treatments without dropout (completers). We tested all effects at the P < 0.05 level. To determine differences in rates of change among the interventions, we subtracted the post-intervention scores from the pre-intervention scores, and we used SPSS-11 to perform analysis of variance with planned contrasts and analysis of covariance of these difference scores. Positive difference scores indicated a reduction from pre-intervention measurements. We determined both unadjusted and Bonferroni adjusted significance levels to account for multiple group comparisons. No differences in the pattern or magnitude of the effects were evident after adjustment for education. We therefore report unadjusted scores.
Results
Participation rates
Eighty three per cent (435) returned post-intervention questionnaires, and 79% (414) completed the intervention. The return rates were significantly different across the sites (χ2 = 14.177, df = 2, P = 0.001) with more dropout from MoodGYM compared with BluePages (χ2 = 5.449, df = 1, P = 0.02). At pre-assessment participants who did not return post-intervention questionnaires scored higher on the Kessler psychological distress scale (P = 0.006) and knowledge of psychological treatments (P = 0.002) but did not differ on the depression scale, automatic thoughts questionnaire, medical and lifestyle literacy, or use of the internet.
Table 1 shows demographic characteristics at baseline.
Table 1.
Baseline characteristics. Values are numbers (percentages) of participants unless otherwise indicated
|
Condition
|
|||
|---|---|---|---|
| Variable | BluePages (n=165) | MoodGYM (n=182) | Control (n=178) |
| Sex: | |||
| Female | 115 (69) | 136 (75) | 124 (70) |
| Years (SD) spent in education | 15.0 (2.4) | 14.6 (2.4) | 14.4 (2.3) |
| Marital status:* | |||
| Married/cohabiting | 100 (61) | 98 (54) | 100 (56) |
| Divorced/separated | 24 (15) | 26 (14) | 24 (14) |
| Never married | 53 (30) | 57 (31) | 53 (36) |
| Mean (SD) age in years | 37.25 (9.4) | 35.85 (9.5) | 36.29 (9.3) |
| Intervention preference†: | |||
| BluePages | 27 (17) | 21 (12) | 20 (11) |
| MoodGYM | 68 (42) | 68 (38) | 67 (38) |
| Control | 65 (40) | 85 (48) | 88 (50) |
| Mean (SD) score on Kessler psychological distress scale | 17.5 (4.9) | 17.9 (5.0) | 18.0 (5.7) |
| Mean (SD) score on Center for Epidemiologic Studies depression scale | 21.1 (10.4) | 21.8 (10.5) | 21.6 (11.1) |
| Caseness (score on Center for Epidemiologic Studies depression scale >16) | 115 (70) | 131 (79) | 123 (69) |
Not all participants completed all questions.
Marital status was based on 523, with one in each of MoodGYM and BluePages widowed; education was based on 510.
Missing data in intervention preference are due to individuals ticking more than one category. All differences of site or condition were non-significant, with the exception of years of education, F (2509)=3.33, P=0.04).
Symptoms of depression and mental health literacy
In the intention to treat analysis, both BluePages and MoodGYM were effective in reducing symptoms of depression (table 2). MoodGYM, but not BluePages, significantly improved dysfunctional thinking compared with the control. Knowledge of medical, psychological, and lifestyle treatments for depression significantly improved for BluePages compared with both MoodGYM and control. Knowledge of effective psychological treatments was improved in MoodGYM relative to control. Literacy in cognitive behaviour therapy improved most with exposure to MoodGYM. These effects were present for completers and for individuals with Center for Epidemiologic Studies depression scores of 16 and over.
Table 2.
Improvement in symptoms and literacy after six weeks
|
Mean (SD) score
|
Difference (95% confidence interval)
|
|||||
|---|---|---|---|---|---|---|
| Outcome measure | BluePages | MoodGYM | Control | BluePages v MoodGYM | MoodGYM v control | BluePages v control |
| Intention to treat (n=525) | ||||||
| Center for Epidemiologic Studies depression scale | 3.9 (9.1) | 4.2 (9.1) | 1.0 (8.4) | −0.3 (−2.6 to 2.0) | 3.2* (0.9 to 5.4) | 2.9* (0.6 to 5.2) |
| Automatic thoughts questionnaire | 6.4 (18.1) | 9.3 (16.9) | 3.1 (15.8) | −2.8 (−7.2 to 1.5) | 6.1* (1.9 to 10.4) | 3.3 (−1.1 to 7.7) |
| Medical literacy | −0.6 (0.7) | −0.1 (0.5) | −0.1 (0.5) | −0.5* (−0.7 to −0.4) | 0.0 (−0.1 to 0.2) | −0.5* (−0.6 to −0.3) |
| Psychological literacy | −0.7 (1.1) | −0.5 (1.0) | −0.0 (0.9) | −0.3* (−0.5 to −0.0) | −0.4* (−0.7 to −0.2) | −0.7* (−1.0 to −0.4) |
| Lifestyle literacy | −0.6 (0.9) | −0.0 (0.5) | 0.1 (0.8) | −0.5* (−0.7 to −0.4) | −0.1 (−0.3 to 0.0) | −0.7* (−0.9 to −0.5) |
| Cognitive behaviour therapy literacy | −1.1 (2.0) | −2.0 (2.4) | 0.1 (1.6) | 0.9* (0.4 to 1.4) | −2.1* (−2.6 to −1.6) | −1.2* (−1.7 to −0.7) |
| Completers (n=414) | ||||||
| Center for Epidemiologic Studies depression scale | 4.9 (9.8) | 5.8 (9.9) | 1.2 (8.9) | −0.9 (−3.7 to 2.0) | 4.5* (1.8 to 7.3) | 3.6* (1.0 to 6.3) |
| Automatic thoughts questionnaire | 7.9 (19.5) | 12.0 (17.4) | 3.6 (16.7) | −4.1* (−9.5 to 1.3) | 8.4* (3.2 to 13.6) | 4.3 (−0.8 to 9.3) |
| Medical literacy | −0.8 (0.7) | −0.1 (0.5) | −0.2 (0.5) | −0.6* (−0.8 to −0.5) | 0.1 (−0.1 to 0.3) | −0.6* (−0.7 to −0.4) |
| Psychological literacy | −0.9 (1.1) | −0.7(1.1) | −0.0 (1.0) | −0.2 (−0.6 to 0.1) | −0.6* (−0.9 to −0.3) | −0.9* (−1.2 to −0.6) |
| Lifestyle literacy | −0.7 (0.9) | −0.0 (0.6) | 0.1 (0.7) | −0.7* (−0.9 to −0.5) | −0.1 (−0.4 to 0.1) | −0.8* (−1.0 to −0.6) |
| Cognitive behaviour therapy literacy | −1.4 (2.2) | −2.8 (2.4) | 0.1 (1.7) | 1.4* (0.8 to 2.0) | −2.8* (−3.4 to −2.2) | −1.4* (−2.0 to −0.9) |
| Completers 16 or more on Center for Epidemiologic Studies depression scale (n=369) | ||||||
| Center for Epidemiologic Studies depression scale† | 5.9 (9.3) | 5.7 (9.8) | 2.1 (8.2) | 0.2 (−2.6 to 3.0) | 3.6* (0.8 to 6.3) | 3.8* (0.9 to 6.6) |
| Automatic thoughts questionnaire | 10.4 (20.0) | 16.2 (20.0) | 4.5 (18.3) | −5.9 (−12.4 to 0.9) | 11.7* (5.1 to 18.3) | 5.9 (−0.6 to 12.4) |
| Medical literacy | −0.7 (0.7) | −0.1 (0.5) | −0.1 (0.5) | −0.6* (−0.8 to −0.4) | 0.1 (−0.1 to 0.2) | −0.5* (−0.7 to −0.3) |
| Psychological literacy | −0.8 (1.0) | −0.4 (1.0) | −0.1 (1.0) | −0.4 (−7 to −1.0) | −0.3 (−0.6 to 0.0) | −0.7* (−1.0 to −0.4) |
| Lifestyle literacy | −0.6 (0.8) | −0.1 (0.5) | 0.1 (0.6) | −0.5* (−0.7 to −0.3) | −0.2 (−0.4 to 0.0) | −0.7* (−0.9 to −0.5) |
| Cognitive behaviour therapy literacy | −1.1 (1.9) | −1.8 (2.1) | −0.0 (1.6) | 0.8* (0.2 to 1.4) | −1.8* (−2.4 to −1.3) | −1.0* (−1.6 to −0.4) |
All results remained significant with adjustment using Bonferroni correction.
The mean difference is significant at the 0.05 level.
In the intention to treat condition, the percentage of clinical cases (Center for Epidemiologic Studies depression score > 16) was 50% (BluePages), 54% (MoodGYM) and 61% (control) at post-intervention, representing a drop of 20%, 25%, and 8%, respectively, from caseness levels before intervention.
Effect sizes
The pre-post effect sizes of the Center for Epidemiologic Studies depression score (calculated as the difference between scores before and after the intervention divided by the pooled standard deviation) were 0.4, 0.4, and 0.1 for the MoodGYM, BluePages and the control intervention in the intention to treat analysis; 0.6, 0.5, and 0.1 for completers; and 0.9, 0.75, and 0.25 for completers with Center for Epidemiologic Studies depression scores 16 and over.
Clinical cases
Assignment to a particular intervention predicted clinical status (Center for Epidemiologic Studies depression score ≥ 16) after the intervention for both the intention to treat and completer analyses when we used a logistic regression procedure which took account of baseline clinical status. For analysis of completers, odds ratios for MoodGYM and BluePages were 0.2 (95% confidence interval 0.2 to 0.5) and 0.6 (0.4 to 0.9).
Website activity
Web logs from BluePages showed that the site was visited an average of 4.49 (SD = 1.35, n = 113) occasions, with an average of 67.2 (23.9) hits. MoodGYM participants completed half of 29 exercises(14.8, 9.7).
Relation between preference and outcomes
Preference for a specific intervention or incompatibility of preference and assigned intervention was not a predictor of change on the Center for Epidemiologic Studies depression scale.
Discussion
Findings and their implications
The internet is a feasible and powerful tool in delivering community based health interventions. The study's hypotheses were confirmed, with the exception that depression literacy was found to be as effective as cognitive behaviour therapy in reducing symptoms of depression. The mechanisms for this effect are unclear, although research from general practice15 indicates higher recovery rates in patients who receive educational material. The size of the treatment effect in patients with a clinical level of symptoms (pre-post effect of 0.9 for cognitive behaviour therapy) was comparable, although smaller than brief cognitive therapy assisted by a therapist, self directed manualised computer therapy, and bibliotherapy, where pre-post effect sizes have ranged from about 0.70 to 1.20 standard deviation units for mixed or depressed samples.16,17 Computer assisted cognitive behaviour therapy in general practice has produced pre-post effect sizes of approximately 1.20.7 The lower effect size in our study might be attributable to sample differences, the community setting, reduced individual contact, or to a less effective therapy package. The effect size of 0.2 for the control group is comparable to conditions where active intervention is postponed but weekly phone contact is maintained, as reported by other studies.18,19 Both the MoodGYM site, and (especially) the psychoeducational site were acceptable to patients, with dropout rates of 25% and 15%. Although trial participants were highly selected, these rates are remarkably low in comparison with other interventions. For example, 37% refused or discontinued a problem solving intervention, and 54% refused or discontinued the “coping with depression” course in a community setting.20 The design and interactivity of our websites may explain in some part their greater acceptability. Other reasons might be the enthusiasm of the lay interviewers or sample characteristics of the sample.
Our participants were highly educated and not recruited through general practice. However, more than 90% (489) reported having been markedly depressed previously, with 64% (329) reporting that they sought help from a doctor or counsellor at the time. This implies that our findings may be relevant to a subset of patients from general practice.
Limitations
Follow up of the results at 12 months will provide information about the sustainability of internet interventions. A higher dropout rate occurred for the MoodGYM intervention, possibly because of lower acceptability. Nevertheless, the pattern of findings was identical for those who completed the study and for the intention to treat analysis. This shows that the higher attrition rate did not substantially alter the effectiveness of the treatment but leaves open the possibility that participants who completed the MoodGYM intervention were differentially biased towards lower scores relative to the other interventions.
What is already known on this topic
Internet interventions may offer help for many individuals without access to effective treatments
Internet based cognitive behaviour therapy may be effective but has not been tested at the community level
No randomised controlled trials have examined the effect of internet psychoeducation on symptoms of depression
What this study adds
Psychoeducation delivered on the web reduces symptoms and increases the public's knowledge of depression and its effective treatments
Cognitive behaviour therapy delivered via the internet with weekly contact reduces dysfunctional thinking and depressive symptoms
Internet sites offer feasible and powerful public health interventions
We thank the BlueMood Trial survey managers, Kimberley Evans and Chloe Groves; Keith Dear (statistician); and David Hawking and his colleagues at the Commonwealth Scientific and Industrial Research Organisation (CSIRO Australia) for maintaining the BluePages site and facilitating data collection; David Berriman and colleagues at Corporate Information Systems, Australian National University, for maintaining the MoodGYM site; the Centre for Mental Health Research Mental Health Literacy Team including Claire Kelly and Anthony Bennett; the lay interviewers from the Centre for Mental Health Research; members of the BluePages advisory board; and individuals involved in the development of MoodGYM, including Jo Medway. We thank the individuals in the trial for their participation.
Contributors: All authors contributed to the conception and design of the study. HC and KMG led the teams responsible for the development and evaluation of the websites. HC wrote the paper, and KMG and AFJ contributed to the writing. HC undertook the data analysis. Kimberley Evans prepared the participation flow figures. Keith Dear undertook the randomisation procedures and power analysis. HC and KMG will act as guarantors for the paper.
Funding: National Health and Medical Research Council Australia programme grant to the Centre for Mental Health Research, Australian National University, Canberra, ACT (chief investigators: A F Jorm, H Christensen, B Rodgers, K Dear, and K Anstey).
Competing interests: None declared.
Ethical approval: Human Ethics Committee of the Australian National University.
References
- 1.Andrews GA, Henderson AS. Unmet need in psychiatry: problems, resources, responses. Cambridge: Cambridge University Press, 1999.
- 2.Kessler RC, Berglund PA, Bruce ML, Koch JR, Laska EM, Leaf PJ, et al. The prevalence and correlates of untreated serious mental illness. Health Serv Res 2002;36: 987-1007. [PMC free article] [PubMed] [Google Scholar]
- 3.Bebbington PE, Brugha TS, Meltzer H, Jenkins R, Ceresa C, Farrell M, et al. Neurotic disorders and the receipt of psychiatric treatment. Psychol Med 2000;30: 1369-76. [DOI] [PubMed] [Google Scholar]
- 4.Christensen H, Griffiths KM. The prevention of depression using the internet. Med J Aust 2002;177(suppl): S122-5. [DOI] [PubMed] [Google Scholar]
- 5.Clarke G, Reid E, Eubanks D, O'Connor E, DeBar LL, Kelleher C, et al. Overcoming depression on the internet (ODIN): a randomised controlled trial of an internet depression skills intervention program. J Med Internet Res 2002;4: E14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Kaltenthaler E, Shackley P, Stevens K, Beverley C, Parry G, Chilcott J. A systematic review of computerized cognitive behaviour therapy for depression and anxiety. Health Technol Assess 2002;6: 1-100. [DOI] [PubMed] [Google Scholar]
- 7.Proudfoot J, Goldberg D, Mann A, Everitt B, Marks I, Gray JA. Computerized, interactive, multimedia cognitive-behavioural program for anxiety and depression in general practice. Psychol Med 2003;33: 217-27. [DOI] [PubMed] [Google Scholar]
- 8.Richards JC, Alvarenga ME. Extension and replication of an internet-based treatment program for panic disorder. Cognit Behav Ther 2002;31: 41-7. [Google Scholar]
- 9.Kenardy J, McCafferty K, Rosa V. Internet-delivered indicated prevention for anxiety disorders: a randomized controlled trial. Behav Cognit Psychother 2003;31: 279-89. [Google Scholar]
- 10.Jorm AF, Christensen H, Griffiths KM, Rodgers, B. Effectiveness of complementary and self-help treatments for depression. Med J Aust 2002;176: S84-96. [DOI] [PubMed] [Google Scholar]
- 11.Christensen H, Griffiths KM, Korten A. Web-based cognitive behaviour therapy: analysis of site usage and changes in depression and anxiety scores. J Med Internet Res 2002;4(1): e3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Andrews G, Slade, T. Interpreting scores on the Kessler psychological distress scale (K10). Aust N Z J Public Health 2001;25: 494-97. [DOI] [PubMed] [Google Scholar]
- 13.Randolf, LS. The CES-D scale: a self-report depression scale for research in the general population. Appl Psychological Measurement 1977;1: 385-401. [Google Scholar]
- 14.Hollon SD, Kendall PC. Cognitive self statements in depression: development of an automatic thoughts questionnaire. Cognit Ther Res 1980;4: 383-95. [Google Scholar]
- 15.Jacob KS, Bhugra D, Mann AH. A randomised controlled trial of an educational intervention for depression among Asian women in primary care in the United Kingdom. Int J Soc Psychiatry 2002;48: 139-48. [DOI] [PubMed] [Google Scholar]
- 16.Jacobs MK, Christensen A, Snibbe JR, Dolezal-Wood S, Huber A, Polterok A. A comparison of computer-based versus traditional individual psychotherapy. Prof Psychol Res Pr 2001;32: 92-96. [Google Scholar]
- 17.Cuijpers, P. Bibliotherapy in unipolar depression: a meta-analysis. J Behav Ther Exp Psychiatry 1998;28: 139-47. [DOI] [PubMed] [Google Scholar]
- 18.Jamison C, Scogin F. The outcome of cognitive bibliotherapy with depressed adults. J Consult Clin Psychol 1995;63: 644-650. [DOI] [PubMed] [Google Scholar]
- 19.Ackerson, J, Scogin F, McKendree-Smith N, Lyman RD. Cognitive bibliotherapy for mild and moderate adolescent depressive symptomatology. J Consult Clin Psychol 1998;66: 685-690. [DOI] [PubMed] [Google Scholar]
- 20.Dorwick C, Dunn G, Ayuso-Mateos JL, Dalgard OS, Page H, Lehtinen V, et al. Problem solving treatment and group psychoeducation for depression: multicentre randomised controlled trial. BMJ 2000;321: 1-6. [DOI] [PMC free article] [PubMed] [Google Scholar]