The Use of Diary Methods to Evaluate Daily Experiences in First-Episode Psychosis

Abstract

Integrated Coping Awareness Therapy (I-CAT) is an intervention that targets stress reactivity in first-episode psychosis (FEP). This study extends prior outcome research on I-CAT by examining predictors of online daily diary completion among 38 young adults with FEP and treatment group differences in diary ratings. We found no significant predictors of daily diary completion rate and no effect of treatment condition on diary ratings. These results are consistent with Halverson et al. (2021) and suggest that diaries are a valuable method of data collection in FEP.

1. Introduction

Relative to the general population, young adults with first episode psychosis (FEP) have an increased vulnerability to stressful life experiences, high sensitivity to stress, and dysregulated stress responses that may herald poor long-term functioning (Berger et al., 2018; Docherty et al., 2009). However, it is only recently that clinical interventions which explicitly target stress reactivity and promote well-being have been developed. Integrated Coping Awareness Therapy (I-CAT) is a manualized, individual intervention that draws upon positive psychology and mindfulness-based approaches to build adaptive responses to stress through combined in-session and at-home skills practice (Meyer-Kalos et al., 2018). An early pilot study found that I-CAT was feasible and may reduce stress and improve quality of life in FEP (Meyer-Kalos et al., 2018). A randomized controlled trial (RCT) of I-CAT augmented coordinated specialty care (CSC) versus CSC alone (treatment-as-usual [TAU]) showed that I-CAT recipients experienced greater symptom reduction, increased mindfulness, and better maintenance of education and/or employment throughout the study period (clinicaltrials.gov NCT03067311). However, there were no between-group differences in the primary outcomes of positive emotions and stress reactivity (Halverson et al., 2021). Diary ratings of emotions and stress collected as part of daily homework assignments across nine months of the I-CAT RCT may offer additional insight on treatment effects not captured by analyzing aggregate change scores at discrete assessment timepoints by Halverson and colleagues (2021).

Although several studies have established that diary-style experience sampling is a feasible and acceptable methodology in schizophrenia (Edwards et al., 2016; Granholm et al., 2008), prior meta-analytic research indicates relatively lower compliance with these methods among individuals with psychosis when compared to other clinical groups or the general population (Vachon et al., 2019). Yet, previous research has found no relationship between important indicators of clinical functioning and completion rate in psychotic disorders (Hartley et al., 2014), resulting in an incomplete understanding of the predictors of diary compliance especially at early stages of the illness. Additionally, while several studies have used experience sampling to establish temporal patterns in FEP, such as the relationship between stress and psychotic symptoms, (Gerritsen et al., 2019, Klippel et al., 2017, Reininghaus et al., 2016), to date, no studies have used daily diary ratings to evaluate the impact of psychosocial interventions on positive emotions and stress. Thus, we extended Halverson et al. (2021) by examining 1) baseline differences in clinical and demographic features associated with diary completion; and 2) whether participants in I-CAT showed reduced daily stress and increased happiness compared to TAU at mid-treatment and post-treatment.

2. Methods

This study is a secondary data analysis of 38 participants with FEP randomized to either the manualized ICAT intervention (n = 19) or TAU (n = 19) in an RCT conducted between November 2016 and March 2020. The sample of participants was 53% male (n = 20) and 47% female (n = 18), with 66% of participants identifying as White, 18% as Black, 11% as Asian, and 5% as American Indian/Alaskan Native. The average duration of illness was 1.74 years (SD = 1.73), and 68% of participants were employed or students. For study details, see Halverson et al. (2021).

All participants completed an online daily diary beginning with their first therapy session. The diary was located on a secure website which was accessible to any device with internet access and required a username and password to log in. Participants were asked to rate their happiness, sadness, stress, relaxation, and adaptation on a 7-point Likert scale, with higher scores indicating greater intensity of experience (see Supplemental Materials). Reminders were sent via text or e-mail to participants who missed more than two days of diary entries in a row. Additionally, participants received $1 per diary entry, with the possibility of receiving up to $196. Participants also completed four assessments (baseline, mid-treatment [4.5 months], post-treatment [9 months], 3-month follow-up).

All analyses were conducted in IBM SPSS Software (Version 26). Using effect sizes from each test and an alpha level of .05, we conducted post-hoc power analyses with G-Power 3.1 (Faul et al., 2009) to determine the adequacy of the observed sample size. For our first aim, we used bivariate linear regressions to assess predictors of compliance. Additionally, we used exploratory t-tests to examine differences between participants with a diary completion rate greater than 70% and those below a 70% completion rate. This cut-off was derived by inspecting the distribution of participant completion rates (see Supplemental Materials). Six participants (four from TAU; two from I-CAT) were excluded because they did not attend a post-treatment visit and no completion rate could be calculated. Finally, independent samples t-tests were used in our second aim to determine the effect of treatment condition on daily diary ratings at mid- and post-treatment (Note: Diary ratings for each mood domain were averaged across two weeks prior to each assessment point at mid-treatment and post-treatment. In order to be included, each participant must have attended the clinical assessment and completed at least one diary entry in the two-week period leading up to the assessment). Following a significant result, multiple regressions were used to control for baseline diary ratings.

3. Results

3.1—. Predictors of Diary Completion

The participants (n = 32) had a mean completion rate of 52.4% over an average eight-month period in the study. We found no significant relationships between PANSS positive symptoms (B = −0.99, p = .85), PANSS negative symptoms (B = −5.71, p = .29), PANSS depressive symptoms (B = 1.01, p = .85), negative emotions (B = 0.26, p = .65), social functioning (B = 17.18, p = .18), or treatment condition (B = −12.40, p = .25) and diary completion rate. Additionally, there were no significant differences between participants with a greater than 70% completion rate (n = 14) and those with a less than 70% (n = 18) completion rate in terms of age (t(30) = 0.03, p = .97), education (t(30) = 0.68, p = .51), gender (χ² = 0.10, p = .76), or race (χ² = 2.64, p = .45). We found that achieved power varied between 5.0% and 37.7% for each test in this aim.

3.2 –. Diary Data as an Outcome Measure

There were no significant differences between I-CAT and TAU participants on any daily diary rating at mid-treatment or post-treatment, except for stress which was significantly higher in the I-CAT group at post-treatment (t(17) = 2.33, p = .03) (Table 1). However, this finding was no longer significant after controlling for baseline stress (B = 1.18, t(16) = 2.10, p = .05). Achieved power for each comparison varied between 5.9% and 59.5% across all timepoints.

Table 1:

Results of Independent Samples T-Tests Comparing I-CAT and TAU Diary Scores (Aim 2)

		I-CAT		TAU
		Mean	SD	Mean	SD	t	p value	Cohen’s d	Achieved power (%)
		n = 19		n = 19
Diary Baseline n = 38	Happiness	4.39	0.89	4.67	1.07	−0.88	0.39	−0.29	13.7
	Sadness	2.56	0.92	2.81	1.18	−0.74	0.47	−0.24	11.1
	Stress	3.31	1.06	3.10	1.12	0.58	0.56	0.19	8.8
	Relaxation	3.86	0.91	4.17	1.13	−0.92	0.36	−0.30	14.6
	Adaptation	4.29	0.94	4.77	1.41	−1.25	0.22	−0.40	22.8
		n = 15		n = 11
Mid-treatment n = 26	Happiness	4.49	1.28	4.87	1.28	−0.75	0.46	−0.30	11.2
	Sadness	2.64	1.24	2.44	1.39	0.38	0.70	0.15	6.5
	Stress	2.99	0.97	2.66	1.36	0.71	0.48	0.28	10.2
	Relaxation	4.38	1.04	4.60	1.19	−0.49	0.63	−0.19	7.5
	Adaptation	5.01	1.24	4.83	1.50	0.34	0.74	0.14	6.2
		n = 10		n = 9
Post-Treatment n = 19	Happiness	4.94	1.07	5.11	1.39	−0.30	0.77	−0.14	5.9
	Sadness	2.49	0.9	2.26	1.18	0.47	0.65	0.22	7.2
	Stress	3.80	1.33	2.43	1.22	2.33	0.03*	1.07	59.5
	Relaxation	4.64	0.98	5.16	1.46	−0.93	0.37	−0.43	13.9
	Adaptation	4.82	1.25	5.84	1.29	−1.75	0.10	−0.81	37.9

Note: Participants averaged a similar number of entries at baseline (M = 8.55, SD = 4.79), mid-treatment (M = 9.04, SD = 4.53) and post-treatment (M = 8.89, SD = 4.14).

4. Discussion

This study extended the findings of Halverson and colleagues (2021) by evaluating predictors of diary completion rate and by determining the effects of the I-CAT intervention on diary scores. The bivariate regression analyses revealed no demographic or clinical predictors of daily diary completion rate. These findings are consistent with past ESM research that found no demographic or clinical predictors of compliance in experience sampling studies (Hartley et al., 2014). Given that inclusion criteria for the I-CAT study included no recent hospitalizations and that 89.5% of participants were on medication, it is possible that there was not sufficient variability in symptoms to observe an impact on daily diary compliance. This is supported by the fact that participants were only moderately symptomatic at baseline (Halverson et al., 2021). Future research may benefit from analyzing the predictive value of additional variables such as motivation or the presence of daily routines in a daily diary context.

In our second aim, we found no differences between I-CAT and TAU on diary ratings after controlling for baseline scores. The quality of supportive/non-specific therapy given to TAU participants may have made it difficult to determine differences between I-CAT and TAU. Most participants were recruited from clinics specializing in early intervention services, which have been shown to be highly effective forms of therapy (Correll et al., 2018). Therefore, the constructs measured by the diary could have been comparably addressed in both I-CAT and TAU. This finding is consistent with the results from the I-CAT RCT, which found that I-CAT and TAU participants demonstrated similar increases in positive emotions and reductions in stress (Halverson et al., 2021).

There are several limitations to consider. The first limitation was a small sample size, which was especially pronounced at post-treatment. This may have been the result of a long study period. Additionally, our study was under-powered in both Aims 1 and 2, which may have limited our ability to detect significant predictors or group differences. Regarding our first aim: there is extensive variation in how compliance is defined in diary studies (Dale & Hagen, 2007). We relied upon the distribution of completion rates across participants to inform our 70% completion cut-off to conduct t-tests; however, we recognize that this is not a universal benchmark of compliance.

In conclusion, we identified no predictors of daily diary completion in individuals with FEP and found no differences in diary ratings based on treatment group. Given the potential importance of methods like the daily diary and ESM in capturing individuals’ mood and functioning, future work should continue to investigate clinical and demographic predictors of daily diary completion in individuals with FEP.