Internet cognitive–behavioural treatment for panic disorder: randomised controlled trial and evidence of effectiveness in primary care

Abstract

Background

Internet cognitive–behavioural therapy (iCBT) for panic disorder of up to 10 lessons is well established. The utility of briefer programmes is unknown.

Aims

To determine the efficacy and effectiveness of a five-lesson iCBT programme for panic disorder.

Method

Study 1 (efficacy): Randomised controlled trial comparing active iCBT (n=27) and waiting list control participants (n=36) on measures of panic severity and comorbid symptoms. Study 2 (effectiveness): 330 primary care patients completed the iCBT programme under the supervision of primary care practitioners.

Results

iCBT was significantly more effective than waiting list control in reducing panic (g=0.97, 95% CI 0.34 to 1.61), distress (g=0.92, 95% CI 0.28 to 1.55), disability (g=0.81, 95% CI 0.19 to 1.44) and depression (g=0.79, 95% CI 0.17 to 1.41), and gains were maintained at 3 months post-treatment (iCBT group). iCBT remained effective in primary care, but lower completion rates were found (56.1% in study 2 v. 63% in study 1). Adherence appeared to be related to therapist contact.

Conclusions

The five-lesson Panic Program has utility for treating panic disorder, which translates to primary care. Adherence may be enhanced with therapist contact.

Declaration of interest

None.

Copyright and usage

© The Royal College of Psychiatrists 2016. This is an open access article distributed under the terms of the Creative Commons Non-Commercial, No Derivatives (CC BY-NC-ND) licence.

Panic disorder with or without agoraphobia has a 12-month prevalence of approximately 3%.¹ Given the disabling impact of the disorder when it goes untreated, recent years have seen the development and evaluation of internet-delivered cognitive–behavioural therapy (iCBT) programmes. Such programmes typically deliver traditional CBT components of psychoeducation, de-arousal, exposure and cognitive restructuring in a self-help format, completed over several weeks with therapist support and guidance provided remotely. iCBT is efficacious for treating panic disorder,^2,3 effective when supported by medical practitioners⁴ and in psychiatric settings,⁵ and enables improvement equivalent to that achieved with face-to-face individual or group delivery, but is more cost-effective.^6,7 Indeed, a recent review indicated that iCBT for panic disorder is well established⁸ (though iCBT may be somewhat less efficacious for milder panic symptoms).⁹ We previously reported the efficacy of a six-lesson iCBT programme for panic disorder.¹⁰ We subsequently reduced this programme to five lessons to increase adherence and facilitate completion and explored whether similar gains could be found with minimal therapist guidance. The current paper presents efficacy and effectiveness data from two studies on the revised five-lesson iCBT programme for panic (the Panic Program). Study 1 was a randomised controlled trial (RCT) of the Panic Program compared with waiting list control. Study 2 investigated the effectiveness of the same programme, when prescribed by primary care practitioners to their patients via THISWAYUP Clinic, an online treatment clinic for depression and anxiety disorders (www.thiswayupclinic.org). We hypothesised that the iCBT programme would be effective in reducing panic symptomatology, distress and comorbid depression symptoms, and would be more effective than waiting list control. We also predicted that the iCBT programme would remain effective when completed under the supervision of primary care practitioners, but we would find lower completion rates.

Method: study 1

Design

A CONSORT 2010 compliant RCT design was used to compare an immediate treatment group to a deferred-treatment group (waiting list control). The immediate treatment group was followed up until 3 months post-treatment. The waiting list control group was enrolled in the iCBT course after the treatment group had completed the programme. During the waitlist period, the waiting list control group completed outcome questionnaires at time-points matched to the immediate treatment group, but received no intervention. A minimum of 25 participants were required in each group to detect a between-groups effect size (ES) of 0.80 with power of 80%.

Participants applied online to www.virtualclinic.org.au after reading details about the study, including the eligibility criteria for inclusion as follows: (i) aged over 18, (ii) self-identified as suffering from panic attacks or panic disorder, and with scores on the Panic Disorder Severity Scale Self-Report version (PDSS-SR) above clinical threshold (8 or higher), (iii) prepared to provide name, phone number and address, and the name and address of their local general practitioner and (iv) had access to a phone, computer and printer. Participants applied in two waves of data collection, first in 2011, and second in 2013 to complete participant recruitment.

Details of participant flow are shown in Fig. 1. Seventy-five applicants were excluded after completing initial online screening questions. One hundred and three applicants met the online selection criteria, provided informed consent and then participated in a brief phone interview. Trained interviewers administered the Mini International Neuropsychiatric Interview Version 5.0.0 (MINI)¹¹ to confirm whether the applicant met DSM-IV criteria for panic disorder with or without agoraphobia. Thirty-six individuals were excluded after telephone interview, leaving 67 applicants who met inclusion criteria and were randomised. Random numbers were generated using a computer random number generator (www.random.org) by a team member who was not involved in the study; this team member placed the group allocation numbers in a sealed opaque envelope. Group allocation was therefore concealed from the interviewer until the offer of participation was made, and the interviewer opened the sealed envelope to inform the participant of their allocated group. The study was approved by the Human Research Ethics Committee (HREC) of St Vincent’s Hospital (Sydney, Australia) (HREC 08/SVH/36), and the trial was registered as ACTRN12611001120965.

Fig. 1. Participant flow diagram.

Diagnostic assessment

The MINI panic disorder and agoraphobia modules were administered to assess DSM-IV diagnoses of panic disorder and agoraphobia. The MINI possesses excellent inter-rater reliability (k=0.88–1.00) and good concurrent validity with the Composite International Diagnostic Interview (CIDI, World Health Organization, 1990).¹²

Primary and secondary outcome measures

The PDSS-SR¹³ is a 7-item measure of panic disorder symptoms. Items (e.g. ‘How many panic and limited symptoms attacks did you have during the week?’) are assessed on a 5-point scale, ranging from 0 to 4. Cut-off scores ≥8 suggest clinical levels of panic disorder.¹³ The scale has good internal consistency (0.92) and test–retest reliability (0.81), and is sensitive to change with treatment.¹⁴ In the current sample, internal consistency was good (baseline α=0.89; post-treatment α=0.91).

The Kessler 10-item Psychological Distress scale (K-10)¹⁵ is a 10-item measure of psychological distress. Items (e.g. ‘About how often did you feel nervous?’) are assessed on a 5-point scale over the past fortnight. The K-10 has excellent psychometric properties,¹⁶ and higher scores indicate higher distress, with a score above 20 indicating clinical distress levels. In the current sample, internal consistency was excellent (baseline α=0.90; post-treatment α=0.95).

The Patient Health Questionnaire-9 (PHQ-9)¹⁷ is a 9-item self-report scale that assesses DSM-IV criteria for major depressive disorder (MDD). Participants rate the frequency of symptoms (e.g. ‘Feeling down, depressed, or hopeless’) over the past fortnight on a scale ranging from 0 (not at all) to 3 (nearly every day), where 1=several days, 2=more than half of the days. Scores range from 0 to 27, and a score ≥10 is used as a clinical cut-off for probable MDD.¹⁸ The PHQ-9 has been shown to have good sensitivity and specificity¹⁹ and excellent reliability and validity (0.86; 18). In the current sample, internal consistency was good to excellent (baseline α = 0.89; post-treatment α = 0.92).

The 12-item World Health Organization Disability Assessment Schedule (WHODAS-II)²⁰ measured functional impairment and activity limitation over the past 30 days on a range of items (e.g. ‘How much difficulty did you have in standing for long periods such as 30 minutes’) on a 5-point scale (0 = none to 5 = extreme/cannot do). The WHODAS-II possesses good psychometric properties.^21,22 In the current sample, internal consistency was good to excellent (baseline α=0.89; post-treatment α=0.92).

The NEO-Five Factor Inventory – Neuroticism Subscale (NEO-FFI-N)²³ measured the personality dimension of Neuroticism and has good psychometric properties.²⁴

Description of treatment

The Panic Course is a five-lesson online CBT programme. Lesson content is presented in the form of an illustrated comic-style story about a character who experiences panic disorder and gains mastery over their symptoms with the help of a clinician and through the use of CBT techniques (e.g. thought challenging, controlled breathing and graded exposure^*; see Table 1 for programme content). Participants have access to frequently asked questions for each lesson, ‘Patient Recovery Stories’ from former patients of www.virtualclinic.org.au, and extra resources on key information including: good sleep, assertive communication, healthy boundaries, shifting attention and structured problem solving.

Table 1. Skills covered over the Panic Program by lesson.

Lesson	Skills
1	Psychoeducation on panic disorder, anxiety, diagnosis and treatment
2	Psychoeducation:  Fight-or-flight response  Controlled breathing
3	Link between thoughts and feelings Thought monitoring Thought challenging
4	Psychoeducation on the role of avoidance Interoceptive exposure In vivo exposure
5	Continued exposure Key skills review Relapse prevention

The patient follows the character’s journey to recovery across the five lessons. At the end of each illustrated lesson, patients download and print a lesson summary (homework document), which includes practical homework exercises, such as graded exposure, to complete before next lesson. A lesson is deemed complete once the patient has downloaded the homework document. This document only becomes available once the entire illustrated story has been displayed. Automatic emails are sent congratulating the patient when they complete a lesson. Patients have 8 weeks to complete the entire programme.

Outcome measurement

The MINI was administered to all participants at pre-treatment and at 3-month follow-up for the treatment group (assessors were not masked to treatment condition at 3-month follow-up). Outcome measures were administered at pre-treatment (prior to lesson 1), before lesson 4 (PDSS-SR and K-10 only), at post-treatment (1 week after the treatment group finished the programme) and 3-month follow-up (treatment group only) or at matched time points for waiting list control. The treatment group completed the K-10 before they commenced each lesson, as a measure to alert the clinician if participants’ scores rose by more than 0.5 s.d. between lessons, indicating a significant increase in distress.

Clinical contact with clinician and therapist

The treatment group participants received email and/or phone contact with the Clinical Trials Manager (A.M. or J.S.) after the first and second lessons, after which contact was made in response to patient request or if the clinician initiated contact because of a deterioration in the K-10 score. If clinically indicated, or if patients’ K-10 scores deteriorated, the therapist (M.B., a registered psychiatrist) would make telephone contact with the participant.

Statistical analyses

Independent samples t-tests and χ² (where the variables consisted of nominal data) were conducted to compare the groups at baseline on demographic and pre-treatment measurements on all outcome measures. Intent-to-treat (ITT) linear mixed-model analyses were used to account for missing data due to participant dropouts. This approach is appropriate for RCTs with multiple time points²⁵ and does not assume that the last measurement was stable (an assumption of the the last observation carried forward approach).²⁶ Effects for the primary measures were modelled using the restricted maximum likelihood (REML) model estimation method with an identity covariance structure. Significant effects were followed up with pairwise contrasts comparing pre-treatment with post-treatment scores. Analyses were performed in SPSS version 22.

Results: study 1

Linear mixed-model ANOVAs with time as a within-subjects variable and group as the between-subjects variable were conducted separately for each of the outcome measures (see Table 2 for results). The main effects for the PDSS-SR, K10, WHODAS and NEO scores were qualified by significant group × time interactions (PDSS-SR and K10: Fs (dfs 2, 95.92–95.96)>9.03, Ps≤0.001; WHODAS and NEO: Fs (dfs 1, 47.48–48.21)>4.04, Ps < 0.05). The time × group interaction for PHQ-9 scores (F(1,50.52)=3.91, P=0.053), and NEO scores (F(1,48.21)=4.04, P=0.05) approached significance with a medium between-groups ES difference at post-treatment for PHQ-9 (Hedges g=0.79, 95% CI 0.17 to 1.41), and small ESs for the NEO (Hedges g=0.25, 95% CI –0.15 to 0.86).

Table 2. Estimated marginal means (standard deviations) for primary and secondary outcome measures and within- and between-group effect sizes for internet cognitive–behavioural therapy (iCBT) and waiting list control groups.

		Pre-treatment (iCBT, n=27, control, n=36)		Mid-treatment		Post-treatment (iCBT, n=16, control, n=31)		t(df) Pre-treatment comparisons	F(df) Within-group comparison	F(df) Between-group comparison	r (for within-group ES)	Effect size Hedges g (95% CI) Within-group	Effect size Hedges g (95% CI) Between-group (post)
Measure	n	M	s.d.	M	s.d.	M	s.d.
PDSS-SR iCBT	27	13.00	5.73	7.90	5.45	6.36	5.20	t(57)=−0.54, P=0.59	F(2,96.61)=23.51, P<0.001	F(2,95.92)=14.14, P<0.001	0.48	1.24 (0.55−1.92)	0.97 (0.34−1.61)
Control	36	12.14	5.74	12.39	5.62	11.75	5.57		F(2,94.63)=0.33, P=0.72		0.73	0.08 (−0.41 to 0.56)
K10 iCBT	27	24.04	7.15	18.24	6.76	17.43	6.44	t(55)=0.73, P=0.47	F(2,96.96)=16.64, P<0.001	F(2,95.96)=9.03, P<0.001	0.20	1.29 (0.60 to 1.98)	0.92 (0.28 to 1.55)
Control	36	25.58	7.34	26.14	7.07	23.80	7.02		F(2,94.19)=3.12, P=0.049		0.83	0.18 (−0.30 to 0.67)
PHQ-9 iCBT	27	8.33	4.90	–	–	4.64	4.64	t(55)=0.79, P=0.43	F(1,52.02)=10.84, P<0.01	F(1,50.40)=3.91, P=0.053	0.01	1.16 (0.48 to 1.84)	0.79 (0.17 to 1.41)
Control	36	9.40	4.91	–	–	8.45	4.79		F(1,47.55)=1.28, P=0.26		0.70	0.16 (−0.33 to 0.64)
WHODAS-II iCBT	27	25.42	7.99	–	–	20.26	7.16	t(55)=0.51, P=0.61	F(1,48.47)=15.11, P<0.001	F(1,47.48)=10.09, P<0.01	0.47	1.00 (0.33 to 1.67)	0.81 (0.19 to 1.44)
Control	36	26.51	8.22	–	–	26.58	7.85		F(1,45.69)=0.00, P=0.95		0.85	0.00 (−0.48 to 0.48)
NEO iCBT	27	30.92	7.94	–	–	26.63	6.96	t(56)=−0.54, P=0.59	F(1,48.91)=12.36, P<0.001	F(1,48.21)=4.04, P=0.05	0.40	0.96 (0.30 to 1.63)	0.25 (−0.35 to 0.86)
Control	36	29.79	8.05	–	–	28.57	7.74		F(1,47.01)=1.81, P=0.19		0.92	0.10 (−0.39 to 0.58)

NEO, NEO Five Factor Personality Inventory, Neuroticism Scale; PDSS-SR, Panic Disorder Severity Scale Self-Report Version; PHQ-9, The Patient Health Questionnaire-9; K-10, Kessler 10-Item Psychological Distress Scale; WHODAS-II, The 12-Item World Health Organization Disability Assessment Schedule; iCBT, internet-based cognitive–behavioural therapy group.

Between-group comparisons showed that PDSS-SR, K10 and WHODAS scores were significantly lower at post-treatment in the iCBT group relative to the waiting list control group, with large between-group ESs found for the PDSS-SR (g=0.97, 95% CI 0.34 to 1.61), K10 (g=0.92, 95% CI 0.28 to 1.55) and WHODAS (g=0.81, 95% CI 0.19 to 1.44). Within-group comparisons for the iCBT group showed large ESs for the reductions in all measures (ranging from NEO: g=0.96, 95% CI 0.30 to 1.63, to PDSS-SR: g=1.24, 95% CI 0.55 to 1.92). The reductions in the waiting list control group between baseline and post-treatment time-points were not significant (see Table 2 for ESs and 95% CIs).

Primary outcome measures and effect sizes at 3-month follow-up

Linear mixed-model ANOVAs, with time as the within-subjects variable, were conducted separately to compare mean reductions in scores from post-treatment to 3-month follow-up for the iCBT group. Baseline scores were entered as a covariate for their respective analysis. For the PDSS-SR scores (r=0.83), the main effect of time was significant, F(1, 13.78)=5.37, P=0.04, estimated marginal means: post-treatment: M=4.87, s.d.=3.56; 3-month follow-up: M=4.27, s.d.=3.25). The reduction corresponded to a small ES (0.41). For the K10 scores (r=0.82), the main effect of time was not significant F(1,14.67)=0.49, P=0.49, estimated marginal means: post-treatment: M=16.95, s.d.=6.88; 3-month follow-up: M=17.93, s.d.=6.33), and the ES was small (0.13).

Clinical significance

Of 27 participants in the iCBT group, 17 completed all five lessons, yielding 63% adherence. Eleven participants in the iCBT group completed questionnaire measures at 3-month follow-up. Of these, only five participants in the iCBT group could be reached for 3-month follow-up diagnostic interview. Of these, three no longer met diagnostic criteria for panic disorder or agoraphobia, one met diagnosis for panic disorder without agoraphobia and one met criteria for both diagnoses at 3-month follow-up. According to the self-report measures, 75 and 82% of participants in the iCBT group fell within the non-clinical range of the PDSS-SR (i.e. total score of 7 or below)²⁷ at post-treatment and follow-up (respectively), whereas 29% of participants in the waiting list control group fell within this range at post-treatment.

Treatment satisfaction

iCBT participants were asked to rate how satisfied they were with the online course on a 1 to 5 scale where 1=very dissatisfied and 5=very satisfied. Of 15 participants who completed post-treatment satisfaction questionnaires, the mean score on this question was 4.53 (s.d.=0.83) with 93% of respondents reporting that they were very or mostly satisfied. When these iCBT participants were asked how confident they would be in recommending the iCBT course to a friend with similar problems (where 1=not at all confident and 10=very confident), 87% reported that they were mostly or very confident (M=9.20, s.d.=1.15).

Time spent per participant

The clinician and therapist combined spent on average 6.04 min (s.d.=10.66, range 0–53 min) emailing and calling participants in the iCBT group during the treatment course. Excluding the individual requiring 53 min (>2 s.d.s from the mean), the average time spent per participant was 4.23 min (s.d.=5.16, range 0–25 min).

Discussion: study 1

As hypothesised, the Panic Program produced significantly greater reduction in panic symptoms in the iCBT group at post-treatment compared with the waiting list control group, with small continued reduction in panic symptoms between post-treatment and 3-month follow-up for the treatment group. A similar pattern was observed on reduction in general psychological distress, though there was not continued reduction in symptoms between post-treatment and 3-month follow-up. Depressive symptoms were significantly reduced within the iCBT group at post-treatment. There was also significantly reduced disability in the iCBT group compared with the waiting list control group at post-treatment, along with reduced neuroticism. The waiting list control group did not improve on any symptom measure from pre- to post-treatment. These results suggest that the Panic Program is efficacious for producing sustained reduction in panic symptoms and associated distress, with improvements in comorbid depressive symptoms and neuroticism. The results also indicate that iCBT for panic disorder reduces the functional impairment experienced by patients with this disabling disorder.

The efficacy of the Panic Program is consistent with our prior research showing the utility of a six-lesson iCBT programme for panic.¹⁰ It is also consistent with prior research from other groups, which has established the efficacy of iCBT for panic disorder and agoraphobia.^2,3,8,28 The large within-group and between-group ESs observed in the current study on panic symptoms, general distress symptoms and disability compare favourably with our previous study and are consistent with findings from other research groups.^8,10 This is despite the briefer five-lesson format than the six-lesson format we used previously. As such, the current results demonstrate the utility of a brief guided self-help internet-delivered CBT programme for panic disorder with minimal therapist input. Although our results diverge from those of van Ballegooijen et al⁹ who found no significant impact of iCBT on reducing panic symptoms in their RCT, this may reflect milder pre-treatment panic symptom severity in their treatment group (mean PDSS-SR of 8.8) compared with the present study (mean PDSS-SR of 13.0). The time spent by clinicians during the treatment period was minimal (average 4–6 min per patient, with the majority of time spent emailing participants), and lower than the time spent in other previous RCTs per participant (e.g. mean of 75 min per participant).¹⁰

The adherence rate of 63% was less than observed (79%) in the previous RCT that we conducted on the six-lesson version of the Panic Program.¹⁰ A potential explanation may be reduced duration of therapist time per participant during this programme (4–6 min on average) compared with our previous RCT (75 min on average). Therapist contact has been noted previously to enhance treatment compliance⁷; therefore, additional therapist contact may have enhanced adherence in this programme.

We note the following limitations. The sample size was small; therefore, the results should be interpreted with caution. There was no active control condition, so we cannot rule out whether the superior effects of the Panic Program in this study were due to non-specific therapeutic factors of participating in some form of active psychological treatment (the ‘trial effect’).²⁹ Third, we had difficulty contacting participants at follow-up. This meant that we were not able to determine the diagnostic status of the majority of participants at follow-up. Also, assessors of 3-month follow-up diagnostic status were not masked to group allocation, which may have introduced biased assessments. Finally, this study was completed in the context of a strictly controlled rigorous clinical trial setting, and it is unclear how generalisable these results are to patients who complete internet CBT under the supervision of clinicians in ‘real world’ community settings, and in primary care. To address this limitation, we conducted an effectiveness pre–post study of the Panic Program in study 2.

Method: study 2

Setting

We aimed to evaluate the effectiveness of the Panic Program in primary care by making it available in www.crufadclinic.org which is now called THISWAYUP Clinic (www.thiswayupclinic.org). THISWAYUP Clinic is an online treatment clinic for depression and anxiety. There are currently six validated online six-lesson CBT programmes available on the website: depression, GAD, social phobia, panic disorder, obsessive compulsive disorder and mixed anxiety and depression. At the end of the programme evaluation period in this study, there were 3991 clinicians who were registered to use the website, and the majority are in primary care (89.8%, with medical specialists comprising the remaining 10.2%). Of these clinicians, 1118 are active users of the service. Registered clinicians provide their patient with a written prescription for an iCBT programme that tells the patient how to enrol and provides a secure passcode linking the patient to the supervising clinician. Patients complete a K-10, a diagnosis-independent measure of psychological distress,¹⁵ prior to the commencement of each lesson. Automated emails are sent to the patient’s supervising clinician once (a) the patient has completed a lesson (the email includes a lesson-by-lesson summary of the K-10 scores) (b) if the patient’s score on the K-10 rises 0.5 s.d. between lessons, (c) if their K-10 score rises above 30 (severe range) or (d) if patients miss their nominated lesson date.

As providers of the programmes, we have responsibility to ensure that the course runs as intended and the current study was conducted as part of the routine quality assurance activities of the Clinical Research Unit for Anxiety and Depression (CRUfAD) at St Vincent’s Hospital, Sydney. All measures are routine and required for the safe conduct of this programme. Prior to enrolment in any of the THISWAYUP programmes, all individuals provided electronic informed consent that their pooled data could be collated and used for quality assurance research purposes, as per the following. ‘Data are collected on your progress for quality assurance purposes, namely alerting your clinician as to your progress and, when pooled with scores from other patients, informing us of the effectiveness of the course. We may use pooled data for quality assurance reports that may be published in scientific journals. In any publication, information will be published in such a way that you cannot be identified. Please email us if you do not want your progress to contribute to these published quality assurance reports’.

Description of Panic Program

The Panic Program was the same as in study 1, with the following exceptions to the procedures. Patients had to complete the first two lessons within 30 days and were allowed 90 days to complete the entire programme. Once patients completed each lesson and downloaded the homework, they were required to book in a date on which they would commence the following lesson. Reminder emails were sent if patients missed the date.

Outcome measurement

Patients completed the K-10, PDSS-SR and WHODAS-II prior to commencing lesson 1, and prior to commencing lesson 5. Approximately 1 year into the evaluation period, the PHQ-9 (measure of depression severity) was also included in the battery of outcome measures at pre- and post-treatment, leaving a smaller sub-set of participants with PHQ-9 scores at baseline (n=251) and post-treatment (n=156).

Statistical analyses

All analyses were performed in SPSS v. 22. Univariate analyses were first conducted to investigate the baseline demographic and clinical variables associated with adherence. Next, to investigate reductions in the outcome measures from pre- to post-treatment, a linear mixed model for each of the outcome measures was implemented using the MIXED procedure with a random intercept for subject. Mixed models estimate parameters in repeated measures studies with unbalanced data using maximum likelihood estimation. This makes use of the incomplete data in a way that does not bias the parameter estimates.³⁰ For each outcome, measurement occasion (pre–post) was treated as a categorical variable, and an identity covariance structure was specified to model the covariance structure of the random intercept. Initial model building focused on the selection of the most appropriate covariance structure for the residual correlation matrix. Model fit indices and inspection of the variance–covariance matrix supported the selection of the identity covariance structure for each of the outcome measures.

The fixed effects of age, gender, rurality, clinician’s profession, and their interactions with time, were then added to each of the models. For each outcome measure, the fixed effects of gender and rurality, and their interactions with measurement occasion, were not statistically significant and were removed from the model. Chi-square difference testing of the −2 log likelihoods indicated that the removal of these fixed effects did not decrease model fit for any of the outcome variables, and they were excluded from further analyses.

Reliable change

Following Jacobson and Truax,³¹ reliable change index (RCI) values for the PDSS-SR scores were calculated for the completer sample. RCI values were calculated using test–retest reliability values of 0.71 from Shear et al¹³ for PDSS-SR scores. To calculate standard error of measurement values, standard deviations were derived from the current sample (PDSS-SR pre-treatment: s.d.=5.45).

Results: study 2

Participants

There were 330 patients prescribed the Panic Program between August 2011 and June 2014. Patients’ mean age was 38.93 (s.d.=13.93, range=18–79), 63.9% were female (n=211), and 24.8% (n=82) were living in rural Australia. Prescribing clinicians were: general practitioners (n=171, 51.8%), psychologists (n=79, 23.9%), medical specialists (n=55, 16.7%) and nurses and other allied health practitioners (n=25, 7.6%). Participants’ PDSS-SR scores were on average in the clinical range (M=12.56, s.d.=5.57). According to the proportion of participants who fell above cut-off on the PDSS-SR, 80.6% (n=266) met probable criteria for panic disorder (≥8). Of the sub-sample who were also administered the PHQ-9 (n=251), 129 (51.4%) met criteria for panic disorder comorbid with probable MDD. On average, scores on the PHQ-9 for this sub-sample were on average in the moderate range and above clinical cut-off criteria (≥10) (M=11.01, s.d.=6.83).

Adherence to the Panic Program

Patients completed on average, 3.94 lessons (s.d.=1.42, range=1–5) out of the five possible lessons. Out of 330 participants who began the programme, 185 completed all five lessons (56.1% adherence). Of the non-completers, 10.0% (n=33) completed one lesson only, 10.9% (n=36) completed two lessons, 10.6% (n=35) completed three lessons and 12.4% (n=41) completed four lessons. Older patients were more likely to complete the entire course than their younger counterparts (t (327)=−3.89, P<0.001). There was no difference between course completers and non-completers in baseline K-10, PDSS-SR, PHQ-9 or WHODAS scores, and no differences in terms of gender, rurality, nor the type of supervising clinician. Whether or not a clinician contacted the patient after lesson 1 and/or 2 was not associated with programme completion (χ² (1, N=330)=3.42, P=0.064). However, if a clinician did not contact the patient at all during the entire course, their patients were less likely to have completed the programme (χ² (1, N=330)=12.14, P<0.001).

To further investigate the predictors of the number of lessons completed, we conducted a linear regression with the number of lessons completed as the dependent variable and age, gender, rurality, baseline K-10 scores, baseline PDSS-SR scores and the number of lessons the clinician contacted the patient as the independent variables. This regression equation accounted for 18.4% of the variance in lessons completed (F(6,322)=12.14, P<0.001). Patient’s age, the number of lessons during which the patient had clinician contact and baseline psychological distress were unique predictors of the number of lessons completed (age: β=0.18, s.e.=0.01, t=3.52, P<0.001; K-10 scores: β=−0.17, s.e.= 0.01, t=−2.51, P<0.05, clinician contacts: β=0.36, s.e.=0.05, t=6.95, P<0.001). The results showed that patients with higher levels of K-10 scores at baseline completed fewer lessons, whereas those who were older, and those who had a greater number of contacts from the clinician, completed more lessons on average. In contrast, baseline panic severity, rurality and gender were not significant predictors of the number of lessons completed.

Effectiveness of the Panic Program

Table 3 includes the estimated marginal means and the linear mixed model results for each of the outcome measures at pre- and post-treatment. For all outcome measures, the reduction in symptoms from pre- to post-treatment was statistically significant at the P<0.001 level. Table 3 also reports the ESs of the pre- to post-treatment changes on outcome measures. ESs were calculated from both (a) the model with measurement occasion as the only predictor, and (b) the final covariate adjusted model. In the covariate adjusted models, we found moderate ESs for reductions in panic disorder symptoms (g=0.55, 95% CI 0.36 to 0.73), to large reductions in psychological distress (K-10: g=0.93, 95% CI 0.74 to 1.12), and small reductions in PHQ-9 (g=0.32, 95% CI 0.12 to 0.51) and WHODAS scores (g=0.29, 95% CI 0.11 to 0.47).

Table 3. Estimated marginal means and linear mixed models for all outcome measures (study 2).

	Pre-treatment Mean (s.d.) n=330	Post-treatment Mean (s.d.) n=185	Mean difference	(df)	F	r	Effect sizea Hedges g (95% CI)	Effect sizeb Hedges g (95% CI)
PDSS-SR	9.83 (5.81)	6.29 (5.85)	3.54	(1, 251.34)	61.51*	0.55	1.00 (0.81−1.19)	0.55 (0.36−0.73)
K-10	24.59 (9.26)	17.62 (7.89)	6.97	(1, 258.36)	258.36*	0.69	0.92 (0.73−1.11)	0.93 (0.74−1.12)
WHODAS-II	23.98 (10.54)	20.93 (9.39)	3.05	(1, 207.40)	28.25*	0.72	0.45 (0.27−0.63)	0.29 (0.11−0.47)
PHQ-9c	8.91 (7.92)	5.19 (7.06)	3.72	(1, 169.74)	29.23*	0.73	0.53 (0.33−0.73)	0.32 (0.12−0.51)

PHQ9, Patient Health Questionnaire – 9 item; PDSS-SR, The Panic Disorder Severity Scale Self-Report Version; K-10, Kessler Distress Scale – 10 Item; WHODAS, World Health Organization Disability Assessment Schedule-II; r, Pearson correlation between lesson 1 and lesson 6 scores for calculation of within-group effect sizes.

^aEffect size calculated from model with measurement occasion as the only predictor.

^bEffect size calculated according to covariate adjusted models.

^cn=251 had PHQ-9 scores at baseline, and n=159 had PHQ-9 scores at post-treatment.

*P<0.001.

Covariate adjusted models

For the PDSS-SR, there were statistically significant interactions between clinician’s profession and measurement occasion, with greater reductions in PDSS-SR scores amongst patients prescribed the course by their GPs, when compared with patients prescribed by psychologists and other allied health professionals (PDSS-SR: β=2.15, t(248.00)=2.58, P<0.05). For the PDSS-SR and K-10, there were also statistically significant interactions between diagnostic status and measurement occasion, with greater reductions in PDSS-SR and K10 scores amongst those with probable panic disorder compared with those without a diagnosis (PDSS-SR: β=−4.96, t(508.23)=−6.12, P<0.001; K-10: β=−9.54, t(383.99)=−8.48, P<0.001). Finally, for the PHQ-9, there were statistically significant interactions between patient’s age and measurement occasion (F(1,214.62)=3.72, P<0.05), with younger adults reporting greater reductions in depression symptoms than older adults.

Reliable clinical change and normalisation of symptoms at post-treatment

Of the 185 completers, 41 (21.9%) showed reliable improvements on the PDSS-SR. None of the completer sample showed reliable deterioration. We also calculated the proportion of participants who achieved normalisation (achieved scores below validated clinical cut-off scores) on the PDSS-SR at the end of iCBT. Out of 185 completers, 122 (65.9%) had achieved normalisation of symptoms. For the sub-sample who were above clinical cut-off for panic disorder at baseline and completed the programme (n=152), 88 (57.9%) had symptoms below cut-off for panic disorder at post-treatment.

Discussion: study 2

As hypothesised, the Panic Program appears effective for reducing panic symptom severity in a primary care setting, under the supervision of a range of allied health and primary care practitioners (e.g. general practitioners). Secondary reduction in general psychological distress, depressive symptoms and disability was also observed. The ES of symptom change was moderate for panic symptoms and large for general psychological distress symptoms. Taken together, these findings suggest that online CBT for panic can be effectively delivered in a primary care setting. The majority of treatment completers reported panic symptoms below the clinical threshold following the programme, even when above this threshold at pre-treatment. These effectiveness findings converge with previous research that has shown the utility of such a treatment approach in a psychiatric setting^5,32 and in primary care.⁴

Overall adherence (i.e. completing all lessons in the Panic Program) was 56%. It is interesting that increasing number of clinician contacts during the course predicted better adherence in the programme, whereas high initial psychological distress predicted poorer adherence. It may be a good practice then for the primary care clinicians to make brief regular contact with programme participants to facilitate their completion, particularly with patients showing higher pre-treatment distress.

It is unclear why there was a larger reduction in panic symptoms for participants when supervised by general practitioners compared with psychologists and other allied health providers. It may be that more complex cases are referred to psychologists and allied health clinicians who then show more modest improvement in the Panic Program. However, this is speculative and requires further research.

A limitation of the study was the inability to measure frequency (though we could determine the number of lessons after which patients received at least one clinician contact), duration and type of clinician contact with patients, meaning the impact of these factors on treatment adherence and symptomatic change could not be determined. Moreover, the lack of a control group means we cannot rule out symptomatic improvement for non-treatment-related reasons.

Discussion

These studies demonstrate the efficacy of internet-based CBT for treating panic disorder compared with waiting list controls and its effectiveness in a real-world primary care setting. There was a significant, meaningful and enduring reduction in panic symptoms as well as observed reductions in psychological distress, depressive symptoms and global disability. These findings converge with previous RCTs, effectiveness studies and systematic reviews demonstrating the utility of internet-based treatment of panic disorder.^{2–5,8,28,32}

Adherence findings from both studies also appear to converge to support the utility of therapist contact for facilitating treatment completion. Both the lower adherence in the primary care setting (where we often anecdotally observe less frequent clinician contact than in our research trials) compared with the RCT setting and the positive association between primary care clinician contact and adherence are consistent with this. It is also consistent with the relatively lower adherence in the present RCT compared with one that we conducted previously in which there was longer average clinician contact time per participant. As such, we continue to recommend brief regular participant contact to facilitate treatment completion.

In addition to study limitations addressed in previous sections, we note the following limitations. Self-selection bias may have been present in both studies. This is inherent in the voluntary nature of the RCT, but may also be present in the effectiveness study, which may have over-represented those patients more oriented to internet-based treatment. Also, we did not measure change in agoraphobia symptoms in the programme so cannot speak to the utility of the current programme in reducing agoraphobic symptom severity.

In summary, our findings from both an RCT and effectiveness study in primary care suggest that our iCBT programme for panic disorder (the ‘Panic Program’) is both acceptable and effective in reducing symptoms of panic disorder, as well as comorbid depression, distress and functional impairment. However, adherence differs depending on clinician guidance and treatment setting (clinical trial v. primary care). In future, greater emphasis needs to be placed on educating primary care practitioners to support patients through online CBT, to improve adherence and outcomes of iCBT in routine clinical care.

Funding

J.M.N. is supported by an Australian National Health and Medical Research Council (NHMRC) Fellowship (1037787). The NHMRC had no involvement in any aspect of the study, nor the preparation of this manuscript.