Patient Responsiveness to a Sleep and Circadian Intervention in a Sample of Adults with Serious Mental Illness

Abstract

Understanding patient responsiveness, a component of fidelity, is essential as it impacts treatment outcome and ongoing use of treatment elements. This study evaluated patient responsiveness—operationalized as receptivity to treatment modules and ratings of the usefulness and the utilization of treatment elements—to the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) in a sample of adults with serious mental illness (SMI) and sleep/circadian dysfunction. Adults with SMI and sleep/circadian dysfunction (N=104) received TranS-C in a community mental health setting. Independent raters rated TranS-C sessions to assess receptivity. At post-treatment and 6-month follow-up, participants completed the Usefulness Scale, Utilization Scale, PROMIS Sleep Disturbance (PROMIS-SD) and Sleep-Related Impairment (PROMIS-SRI) scales, DSM-5 Cross Cutting Measure (DSM-5-CC), and Sheehan Disability Scale (SDS). Receptivity was rated as somewhat to fully understood, and predicted a reduction on the DSM-5-CC. On average, participants rated TranS-C as moderately useful and utilized treatment elements occasionally. Ratings of usefulness were associated with the PROMIS-SD, PROMIS-SRI, and DSM-5-CC at post-treatment, but not with the SDS. Ratings of utilization were not associated with outcome. The findings add to the literature on patient responsiveness, an implementation outcome, and provide data on the utility of TranS-C within a community mental health setting.

Introduction

Patient responsiveness to a treatment program is an element of fidelity, which is “the degree to which an intervention is implemented as intended” (Carroll et al., 2007; Dane & Schneider, 1998; Dusenbury et al., 2003). Patient responsiveness, as operationalized within Carroll et al.’s (2007) theory of fidelity, includes three components: patients’ receptivity to an intervention, patients’ perception of the usefulness of an intervention, and patients’ utilization of the tasks required by an intervention (Carroll et al., 2007). These three components of patient responsiveness will be the focus of the present study. Further understanding patient responsiveness is important as it can directly impact ongoing engagement in treatment and continued use of treatment elements (Allen et al., 2012), which can impact the maintenance of gains made in treatment (Kwasnicka et al., 2016). Patient responsiveness can be measured throughout the intervention development process, as this data offers information on how patient populations view a treatment in the setting in which it was delivered (Carroll et al., 2007).

In prior research, patient responsiveness—operationalized as receptivity, usefulness, and utilization—has been associated with treatment outcomes across several treatment modalities and populations. Starting with receptivity, several studies have measured this construct as patient understanding or embracing of the intervention as rated by providers (Mihalic et al., 2008; Schaap et al., 2018; Vroom et al., 2020) or by patients (Dowling & Barry, 2020; Doyle et al., 2019; Kutash et al., 2012; Schaap et al., 2018). Receptivity has been evaluated across a range of treatments including socioemotional, substance use, mindfulness, obesity prevention, and peer-to-peer parental support interventions implemented with youth and adults in outpatient and school-based settings (Dowling & Barry, 2020; Kutash et al., 2012; Mihalic et al., 2008; Schaap et al., 2018). Additionally, usefulness has been measured via patient ratings of how useful they found an intervention as a whole (Doyle et al., 2019) or how useful they found the elements of an intervention (Cox et al., 1994; Kutash et al., 2012; Lawton et al., 2015; Vincent & Lionberg, 2001). For example, one study examined patient ratings of the usefulness of seven components of cognitive behavior therapy for insomnia. Ratings of the usefulness of sleep hygiene, sleep restriction, and stimulus control were correlated with improvements in sleep (Vincent & Lionberg, 2001). Finally, responsiveness has also been operationalized as patient ratings of how often they utilize treatment elements. Studies focused on adults with depression, anxiety, and sleep problems and with caregivers of individuals with dementia found that greater use of treatment elements was associated with improved treatment outcomes, such as lower depression symptoms and stress symptoms (Gallagher-Thompson et al., 2008; Hawley et al., 2017; Hundt et al., 2016; Jacob et al., 2011). Additionally, a study evaluating cognitive behavior therapy for insomnia (CBT-I) found that the majority of participants reported utilizing seven of the 10 total treatment elements, but the utilization of only two of these treatment elements (stimulus control/sleep restriction and cognitive restructuring) was associated with improvement, or a reduction in sleep latency and wake after sleep onset, following treatment (L. Harvey et al., 2002). Taken together, existing research has demonstrated a relationship between the three elements of participant responsiveness and treatment outcomes across a range of treatments, populations, and settings.

Using data from a randomized controlled trial (A. G. Harvey, Dong, et al., 2021), the present study focuses on the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) delivered to adults with a serious mental illness (SMI) within a community mental health setting. TranS-C is a modular intervention that addresses cognitive, behavioral, and psychosocial contributors to sleep and circadian dysfunction (A. G. Harvey & Buysse, 2017) and is grounded in the Sleep Health Framework (Buysse, 2014). It draws from the literature indicating the efficacy of CBT-I for SMI (Edinger et al., 2009; Manber et al., 2008; Myers et al., 2011). It also incorporates principles from Interpersonal and Social Rhythms Therapy (Frank et al., 2005), chronotherapy (Wirz-Justice et al., 2009), and motivational enhancement (Miller & Rollnick, 2002). TranS-C addresses a range of sleep and circadian problems—such as insomnia, hypersomnia, delayed phase, and advanced phase—which predict and predate SMI symptoms including depression, mania, anxiety, substance use, and psychosis (Baglioni et al., 2016; Hertenstein et al., 2019; Kaplan et al., 2015; Kivelä et al., 2018). These sleep and circadian problems persist regularly after treatment is provided for SMI (López et al., 2017). Moreover, common cognitive, behavioral, and biological processes maintain sleep and circadian problems as well as SMI (A. G. Harvey et al., 2004). Data from this clinical trial indicate that TranS-C improves sleep as well as SMI symptoms and functional impairment in adults with SMI in this community mental health setting (A. G. Harvey, Dong, et al., 2021).

Despite advances in understanding the role of patient responsiveness in treatment outcome, gaps in knowledge remain. First, few studies have examined the three elements of fidelity to any treatments delivered for SMI (Schoenwald & Garland, 2013). To address this gap, the present study examines patient responsiveness to TranS-C (A. G. Harvey & Buysse, 2017) via ratings of receptivity, usefulness, and utilization for adults with SMI treated in a community mental health setting (A. G. Harvey et al., 2016). Second, while a review of the literature identified four studies that have examined patient responsiveness to sleep or circadian interventions (Gumport et al., 2019; L. Harvey et al., 2002; Manber et al., 2011; Vincent & Lionberg, 2001), none of these studies examined receptivity to the treatment. Understanding patient receptivity, especially when rated by providers, offers further insight on how patients view a treatment beyond ratings of usefulness and utilization. However, one prior study examined the ratings of usefulness and utilization of TranS-C elements six months and 12 months following treatment completion. In this study, the participants were 64 adolescents aged 10 to 18 years (Gumport et al., 2019). All treatment elements were rated as at least somewhat useful, although they were rarely utilized. Furthermore, select ratings of the usefulness and utilization of treatment elements—including maintaining regular bedtimes and regular wake times, avoiding naps, keeping the bed for sleep only, and using techniques to reduce sleep-related worry—were associated with improved treatment outcome. Third, to the best of our knowledge, no prior studies have evaluated responsiveness to a treatment delivered in a community mental health setting. Understanding this element of fidelity in community mental health settings is important as these are the primary, publicly-funded providers of treatment for SMI in the United States. These services provide care for marginalized individuals with high rates of comorbidity, complexity, and symptom severity. Existing research has demonstrated that maintaining fidelity is challenging when interventions are implemented in most real-world settings (Schoenwald, 2011), including community mental health settings (e.g., Lau et al., 2017; Stirman et al., 2013). This discouraging reality might be explained by the impracticality of certain portions of evidence-based treatment protocols for certain populations or settings (Allen et al., 2012). Hence, identifying patient responsiveness to TranS-C treatment elements among adults with SMI in a community mental health setting may allow for the identification of the elements of TranS-C with the best fit within community mental health.

The overall goal of this study was to evaluate patient responsiveness—operationalized as understanding and embracing modules, ratings of the usefulness of treatment elements, and ratings of the utilization of treatment elements—to TranS-C in a sample of adults with SMI and sleep and circadian dysfunction in a community mental health setting. The first aim was to determine patient responsiveness to TranS-C. Based on prior research, the hypothesis tested was that TranS-C modules will be understood and embraced, or that scores will be close to 1 on the Independent-Rater TranS-C Checklist (rated 1–3 with 1 being fully understanding/embracing and 3 being not all) (Gumport et al., 2020), that usefulness of treatment elements will be rated as at least somewhat useful (Gumport et al., 2019), and that utilization of treatment elements will be rated as utilized at least rarely (Gumport et al., 2019). The second aim was to examine if patient responsiveness to TranS-C was associated with treatment outcome. Based on prior research, the hypothesis tested was that understanding/embracing TranS-C modules will be associated with treatment outcome (Dowling & Barry, 2020; Schaap et al., 2018), that ratings of the usefulness of TranS-C as a whole will be associated with treatment outcome (Gumport et al., 2019; Lawton et al., 2015), that utilization of TranS-C as a whole will be associated with treatment outcome (Gumport et al., 2019), and that specific TranS-C elements will be associated with treatment outcome, including regular bedtimes and regular wake times, avoiding naps, keeping the bed for sleep only, and using techniques to reduce sleep-related worry (Gumport et al., 2019; L. Harvey et al., 2002).

Methods

Participants

The 104 participants included in this study were drawn from a National Institute of Mental Health-funded randomized controlled trial with adults who met criteria for SMI and sleep and circadian disturbance and who were recruited from multiple sites within Alameda County Behavioral Health Care Services (ACBHCS; Alameda County, CA, USA) (A. G. Harvey et al., 2016). SMI was operationalized according to prior research as the presence, for 12 months, of at least one DSM-5 mental disorder that leads to substantial interference with one or more major life activities (Kessler et al., 2003). The primary trial from which the data were drawn included 121 participants. However, 17 participants were excluded from the present study as they did not complete both the post-treatment and 6-month follow-up assessments following the receipt of the treatment. Participant characteristics are displayed in Table 1.

Table 1.

Participant Characteristics

Characteristic	Independent-Rater TranS-C Checklist (n = 33)		Usefulness and Utilization Scales (N = 104)
	Mean or N	SD or %	Mean or N	SD or %
Age (years)	48.54	11.31	46.95	11.86
Female	17	51.52	54	51.92
Race
African-American or Black	17	51.52	44	42.31
American Indian/Alaskan Native	2	6.06	2	1.92
Asian	1	3.03	7	6.73
Caucasian	9	27.27	38	36.54
Native Hawaiian or Other Pacific Islander	1	3.03	2	1.92
Mixed Race	1	3.03	7	6.83
Not specified	2	6.06	4	3.85
Ethnicity
Hispanic or Latino	3	9.09	15	14.42
Not Hispanic or Latino	29	87.88	88	84.62
Not specified	1	3.03	1	0.96
Employment
Full-time	2	6.06	2	1.92
Part-time	4	12.12	13	12.50
Unemployed	27	81.82	83	79.81
Other	0	0.00	5	4.81
Missing	0	0.00	1	0.96
Education (years)	14.36	3.17	13.51	4.27
Highest level of education completed
High school or below	9	36.36	32	30.77
Vocational school	3	9.09	10	9.62
Some college or completed college	21	63.64	49	47.12
Graduate school	0	0.00	5	4.81
Annual personal income ($)	13,777	9,780	10,967.99	7,603.63
Annual household income ($)	23,956	22,258	23,365.33	22,933.65
Receiving government assistance	31	93.94	100	96.15
DSM diagnoses at pre treatment1
Schizophrenia spectrum disorder	13	39.39	50	48.08
Bipolar disorder2	6	18.18	27	25.96
Major depressive disorder3	12	36.36	23	22.12
Any anxiety disorder4	9	27.27	50	48.08
Obsessive compulsive disorder4	6	18.18	21	20.19
Post-traumatic stress disorder	8	24.24	15	14.42
Substance use disorder	11	33.33	32	30.77
Psychotic symptoms/features5	18	54.55	72	69.23
Sleep and circadian diagnoses at pre treatment1
Insomnia	28	84.85	86	82.69
Hypersomnolence (provisional)6	5	15.15	26	25.00
Delayed sleep phase	2	6.06	7	6.73
Advanced sleep phase	1	3.03	2	1.92
Irregular sleep-wake disorder	0	0.00	1	0.96
Restless leg syndrome	1	3.03	5	4.81
Periodic limb movements (provisional)7	2	6.06	5	4.81

Note.

¹Participants could meet diagnostic criteria for multiple problems.

²Bipolar disorder with psychotic features is listed in this category, not in the schizophrenia spectrum or psychotic disorders category.

³Depression with psychotic features is listed in this category, not in the schizophrenia spectrum or psychotic disorder category.

⁴No participants were solely diagnosed with an anxiety disorder or obsessive compulsive disorder – all also received a comorbid schizophrenia spectrum, bipolar disorder, major depressive disorder, post-traumatic stress disorder, and/or substance use disorder diagnosis.

⁵Psychotic symptoms/features includes depression with psychotic features, bipolar disorder with psychotic features, a schizophrenia spectrum or psychotic disorder diagnosis.

⁶A hypersomnolence diagnosis requires a multiple sleep latency test (American Academy of Sleep Medicine, 2014).

⁷A periodic limb movement diagnosis requires a polysomnography assessment (American Academy of Sleep Medicine, 2014).

Individuals were eligible if they met the following inclusion criteria: (a) 18 years of age or older; (b) English language fluency; (c) presence of at least one DSM-5 mental disorder for 12 months; (d) having a guaranteed bed to sleep in for the next three months; (e) receiving care for SMI at ACBHCS and consenting to regular communication between the research team and their ACBHCS psychiatrist and/or case manager; and (f) presence of one or more of the following problems, on three or more nights per week, for three months assessed via the Sleep and Circadian Problems Interview, which is an adapted version of the Insomnia Interview Schedule (Morin, 1993): taking 30 minutes or longer to fall asleep, waking in the middle of the night for 30 minutes or longer, obtaining less than six hours of sleep per night, obtaining nine or more hours of sleep per 24 hour period (i.e., nighttime sleep plus daytime napping), maintaining a bedtime later than 2:00am, or having more than 2.78 hours of variability in sleep-wake schedule across one week.

Individuals were excluded if they met any of the following criteria: (a) presence of an active and progressive physical illness or neurological degenerative disease and/or substance use that would make participation in the study unfeasible; (b) current serious suicide risk or homicide risk (both assessed by study staff and a case manager or psychiatrist); (c) night shift work two or more nights per week in the past three months; (d) pregnancy or breastfeeding; or (e) unable or unwilling to participate in and/or complete the pretreatment assessments.

Treatment

Treatment was delivered by nine therapists hired by the University of California, Berkeley system. Therapists traveled between ACBHCS clinic sites to deliver treatment. Clinicians attended a one-day workshop, used a treatment manual, and received weekly supervision.

TranS-C (Harvey & Buysse, 2017), which was administered in eight weekly 50-minute sessions, is grounded in basic sleep and circadian science and the sleep health framework (Buysse, 2014). TranS-C includes four cross-cutting modules featured in every session (functional analysis, education, behavior change and motivation, and goal-setting), four core modules that apply to the vast majority of participants (establishing regular sleep-wake times including learning a wind-down and wake-up routine, improving daytime functioning, correcting unhelpful sleep-related beliefs, and maintaining behavior change), and seven optional modules used less commonly, depending on the needs of each participant (improving sleep efficiency, reducing time in bed, dealing with delayed or advanced phase, reducing sleep-related worry/vigilance, promoting compliance with CPAP/exposure therapy for claustrophobic reactions to CPAP, negotiating sleep in a complicated environment, and reducing nightmares). Core and optional modules can be delivered in any sequence and are customized to each participant based on their presentation and goals for treatment.

Responsiveness Measures

Independent-Rater TranS-C Checklist.

The Independent-Rater TranS-C Checklist (Gumport et al., 2020) assesses which modules were delivered in a TranS-C session. Two of the items on the checklist are not TranS-C modules: progress monitoring and encouraging the patient to follow up on referrals for a sleep apnea assessment, but were included on the checklist as they were often discussed. The TranS-C core module of establishing regular sleep-wake times including learning a wind-down and wake-up routine was split into three items for clarity: establishing regular sleep-wake times, learning a wind-down routine, and learning a wake-up routine. Therefore, the Independent-Rater TranS-C Checklist contains 19 items, including all 15 of the TranS-C modules.

Trained independent raters listened to audio recordings of treatment sessions and made four ratings for each of the 19 items. These four domains follow the conceptualization of fidelity offered by Allen et al. (2012) and Schoenwald et al. (2011). Inter-rater reliability was 87.91% (Gumport et al., 2020). First, independent raters selected the treatment element they observed to be present. Second, they recorded the duration or the time (rounded to the nearest minute) that the therapist spent on each element. Third, they rated the quality of delivery of each element on a three-point scale (1= excellent, 2 = adequate, 3 = inadequate). Fourth, they rated the participant’s response to the module on a four-point scale (1 = participant fully understood and embraced the module, 2 = participant somewhat understood and embraced the module, 3 = participant did not understand nor embrace the module, 4=unknown, although no ratings of 4 were given). A scoring rule was developed which allowed raters to select multiple optional or cross-cutting modules, but required raters select only one core module. This arose because therapists rarely selected multiple core modules at one time. For the present study, we only used the fourth of the four ratings that observers made – scores indicating how a participant understood or embraced a given TranS-C treatment module. The ratings on the other items on this measure have been reported elsewhere (Gumport et al., 2020). An average response score was calculated by taking the mean score of ratings of a participant’s response to all modules they received.

A total of 257 session checklists from 33 participants were included in this sample. Demographic characteristics of this subsample are displayed in Table 1.

Usefulness Scale.

A Usefulness Scale evaluates how helpful patients find treatment elements from TranS-C. It lists 17 treatment elements from TranS-C and was administered at post-treatment and 6-month follow-up. Each treatment element is rated on a 5-point Likert scale (0 = not at all useful; 1 = somewhat useful; 2 = moderately useful; 3 = very useful; 4 = extremely useful). An Average Usefulness Treatment Score was created by calculating the mean of all 17 items on the scale at post-treatment and at 6-month follow-up individually. Cronbach’s alpha is 0.92, which is considered excellent. Items on this measure were developed based on a version of this scale used in a study examining the youth version of TranS-C conducted with adolescents (Gumport et al., 2019).

Utilization Scale.

A Utilization Scale evaluates how often patients make use of TranS-C treatment elements. It lists 17 treatment elements from TranS-C and was administered at post-treatment and 6-month follow-up. Each treatment element is rated on a 5-point Likert scale (0 = I never use it; 1 = I rarely use it; 2 = I occasionally use it; 3 = I often use it; 4 = I always use it). An Average Utilization Treatment Score was created by calculating the mean of all 17 items on the scale at post-treatment and at 6-month follow-up individually. Cronbach’s alpha is 0.91, which is considered excellent. Items on this measure were developed based on a version of this scale used in a study examining the youth version of TranS-C conducted with adolescents (Gumport et al., 2019).

Outcome Measures

Patient-Reported Outcomes Measurement Information System – Sleep Disturbance (PROMIS-SD).

The PROMIS-SD was developed as a part of the NIH Roadmap initiative and designed to improve patient-reported outcomes using state-of-the-art psychometric methods. It assesses sleep disturbance. The 8-item measure is scored 1 (not at all; never; very poor) to 5 (very much, always, very good). The scale has established reliability and validity (Buysse et al., 2010; Yu et al., 2011).

Patient-Reported Outcomes Measurement Information System – Sleep-Related Impairment (PROMIS-SRI).

The PROMIS-SRI was developed as a part of the NIH Roadmap initiative and designed to improve patient-reported outcomes using state-of-the-art psychometric methods. It assesses impairment related to sleep. The 16-item measure is scored 1 (not at all; never) to 5 (very much; always). The scale has established reliability and validity (Buysse et al., 2010; Yu et al., 2011).

DSM-5 Self-Rated Level 1 Cross-Cutting Symptom Measure – Adult (DSM-5 Cross-Cutting Measure).

The DSM-5 Cross-Cutting Measure (Narrow et al., 2013) is used as a measure of disorder-focused symptoms. It contains 23 questions that assess symptoms in the most recent two weeks across 13 psychiatric domains: depression, anxiety, mania, psychosis, substance use, anger, somatic symptoms, suicidal ideation, sleep problems, memory, repetitive thoughts and behaviors, dissociation, and personality functioning. Items are rated on a 5-point Likert scale (0=none, 1=slight, 2=mild, 3=moderate, 4=severe).

Sheehan Disability Scale (SDS).

The SDS assessed functional impairment. The SDS evaluates the extent to which work/school, social life, and home/family responsibilities are impaired on a 0–10 (not at all to extremely) scale. Its psychometric properties are well established (Sheehan et al., 1996). The three items were averaged to assess global functional impairment (0 [not impaired] to 10 [highly impaired]).

Procedure

All procedures were approved by the University of California, Berkeley, Committee for the Protection of Human Subjects. All participants provided informed consent.

Participants were randomly assigned to receive TranS-C immediately, or to usual care plus delayed-treatment. The latter group was on a waitlist for eight months and then received TranS-C. At post-treatment and at 6-month follow-up, participants completed the Usefulness Scale, the Utilization Scale, and the outcome measures. All data included in the present study was from the treatment phase of both groups.

Data Analysis

All analyses were conducted in Stata15 (StataCorp, 2017). A significance level of 0.05 was used throughout. For the first aim examining responsiveness to TranS-C, means and standard deviations were calculated for each of the three metrics of responsiveness (Independent-Rater TranS-C, Usefulness, and Utilization Scales). For the second aim examining the association between responsiveness and treatment outcomes, hierarchical linear modeling with restricted maximum likelihood estimation was used. The random part of all models included a random intercept for participant, assumed to have a bivariate normal distribution with a mean of zero and an unstructured covariance matrix. The fixed part of all models included the baseline score of each outcome measure (i.e. PROMIS-SD, PROMIS-SRI, DSM-5 Cross Cutting Measure, SDS). All variables included in each model were standardized. Standardized coefficients were calculated, as these are interpretable as effect sizes (Lorah, 2018). We elected to only examine the relationship between the treatment outcome measures that were significantly associated with the Average Usefulness Treatment Score with the Usefulness Scores of specific treatment elements and to only examine the relationship between treatment outcome measures that were significantly associated with the Average Utilization Treatment Score with the Utilization Scores of specific treatment elements.

Results

Responsiveness to TranS-C (Aim 1)

Independent-Rater TranS-C Checklist.

Mean ratings of participants’ understanding and embracing of TranS-C modules are presented in Table 2. The average rating was 1.20, which is in the “fully” to “somewhat” understood and embraced range. The three modules of Too Much Time in Bed (mean = 1), Promoting Compliance with CPAP (mean = 1), and Functional Analysis (mean = 1.10) were rated as having been the most understood/embraced.

Table 2.

Mean Ratings of Participant Responsiveness (Understanding/Embracing) to TranS-C Modules on the Independent-Rater TranS-C Checklist

Module	Mean	Standard Deviation
Cross-Cutting Modules
Functional analysis	1.10	0.26
Education	1.15	0.33
Motivation enhancement	1.17	0.34
Goal setting	1.15	0.42
Progress monitoring (not a formal module)	1.20	0.21
Core Modules
Irregular sleep-wake times	1.28	0.48
Difficulty winding down	1.15	0.33
Difficulty waking up	1.14	0.47
Daytime impairment	1.24	0.59
Unhelpful beliefs about sleep	1.32	0.65
Maintenance of behavior change	1.14	0.44
Optional Modules
Poor sleep efficiency	1.28	0.39
Too much time in bed	1.00	0.00
Delayed or advanced phase	1.17	0.35
Sleep-related worry	1.25	0.47
Promoting compliance with CPAP/other treatments for comorbid sleep problems	1.00	0.00
Negotiating sleep in a complicated environment	2.00	1.00
Reducing Nightmares	1.75	1.06
Follow up on sleep apnea assessment referrals (not a formal module)	1.27	0.39
Overall Responsiveness	1.20	0.34

Note. N = 33. For each session, participant understanding/embracing to a given module is rated as: 1 = The participant fully understood and embraced the module; 2 = The participant somewhat understood and embraced the module; 3 = The participant did not understand nor embrace the module.

Usefulness Scale.

See Table 3. The Average Usefulness Treatment Score was 2.49 at post-treatment and 2.34 at 6-month follow-up, which is in the “moderately” to “very” useful range. Usefulness Scores of each treatment element at post-treatment and 6-month follow-up are presented in Table 3. Each treatment element was rated above a 1 (“somewhat useful”). The treatment elements that were rated as the most useful (i.e. had the highest usefulness scores) at post-treatment were “I try to wake up about the same time each morning” (mean = 2.87), “I try to go to sleep at about the same time each night” (mean = 2.86), and “I use a wake up routine to help me get up and get going in the morning” (mean = 2.82). The treatment elements that were rated as the most useful at 6-month follow-up were “I try to go to sleep at about the same time each night” (mean = 2.78), “I try to wake up about the same time each morning” (mean = 2.69), and “I avoid napping in the evening” (mean = 2.64).

Table 3.

Mean Ratings of Participant Responsiveness (Usefulness and Utilization Scales) at Post-Treatment and 6-Month Follow-Up

Usefulness and Utilization Scale Items	Post-Treatment		6-Month Follow-Up
	Usefulness	Utilization	Usefulness	Utilization
1. I try to go to sleep about the same time each night	2.86(1.09)	3.04(1.08)	2.78(1.08)	2.77(1.17)
2. I try to wake up about the same time each morning	2.87(1.13)	3.00(1.14)	2.69(1.07)	2.79(1.14)
3. I try to get about 7–8.5 hours of sleep per night	2.64(1.28)	2.60(1.40)	2.57(1.13)	2.62(1.29)
4. If my bedtime gets too late, I bring it forward by about 20–30 minutes each week	2.01(1.31)	1.91(1.44)	1.83(1.41)	1.71(1.40)
5. If my bedtime is too early, I bring it later by about 20–30 minutes each week	2.00(1.34)	1.79(1.47)	1.87(1.30)	1.73(1.39)
6. I engage in a wind-down before bedtime	2.59(1.33)	2.51(1.43)	2.58(1.29)	2.61(1.24)
7. I avoid napping in the evening	2.74(1.29)	3.00(1.28)	2.64(1.40)	2.78(1.43)
8. I have an electronic curfew to reduce light exposure via cell phones, computers television etc before my bedtime	2.65(1.33)	2.74(1.36)	2.37(1.45)	2.15(1.51)
9. I use a wake up routine to help me get up and get going in the morning (eg., try to Refrain from snoozing, Increase activity, Shower or wash face and hands, Expose yourself to sunlight, Upbeat music, Phone a friend)	2.82(1.22)	2.98(1.20)	2.43(1.30)	2.31(1.41)
10. When I feel sleepy in the day I purposively generate energy	2.14(1.26)	2.21(1.34)	2.02(1.43)	2.01(1.31)
11. I use techniques to reduce worry interfering with my sleep via savoring or ‘worry time’ earlier in the day or gratitude etc.	2.29(1.32)	2.42(1.45)	2.17(1.33)	2.18(1.37)
12. I get out of bed if I am not able to sleep within about 20 minutes	2.40(1.31)	2.29(1.48)	1.91(1.46)	2.02(1.48)
13. I keep my bed for sleeping only (I do not work in bed, watch TV in bed etc)	2.51(1.42)	2.73(1.42)	2.27(1.51)	2.35(1.53)
14. I avoid caffeine, energy drinks, and alcohol to help me sleep better	2.48(1.39)	2.78(1.37)	2.53(1.43)	2.50(1.54)
15. I use dim lights to help me feel sleepy at bedtime	2.59(1.39)	2.77(1.51)	2.63(1.36)	2.49(1.53)
16. I turn on bright lights in the morning to help me wake up	2.30(1.53)	2.39(1.61)	2.27(1.53)	2.14(1.61)
17. I avoid going to sleep while listening to the radio, watching TV, etc	2.42(1.48)	2.57(1.45)	2.47(1.40)	2.51(1.45)
Average Usefulness Treatment Score	2.49(0.85)	-	2.34(0.92)	-
Average Utilization Treatment Score	-	2.35(0.85)	-	2.54(0.92)

Note. Mean (standard deviation) presented. Usefulness scale rated: 0 = not useful at all; 1 = somewhat useful; 2 = moderately useful; 3 = very useful; 4 = extremely useful. Utilization Scale rated: 0 = never use it; 1 = rarely use it; 2 = occasionally use it; 3 = often use it; 4 = always use it.

Utilization Scale.

See Table 3. The Average Utilization Treatment Score was 2.35 at post-treatment and 2.54 at 6-month follow-up, which is in the use “occasionally” to “often” range. Utilization scores of each treatment element at post-treatment and 6-month follow-up are also presented in Table 3. The treatment elements that were utilized the most (i.e. those with the highest utilization scores) at both post-treatment and 6-month follow-up were “I try to go to sleep at about the same time each night” (post-treatment mean = 3.04, 6-month follow-up mean = 2.77), “I try to wake up about the same time each morning” (post-treatment mean = 3.00, 6-month follow-up mean = 2.79), and “I avoid napping in the evening” (post-treatment mean = 3.00, 6-month follow-up mean = 2.78).

Responsiveness and Treatment Outcome (Aim 2)

Independent-Rater TranS-C Checklist.

As evident in Table 4, higher ratings of understanding and embracing TranS-C modules were associated with a significant reduction in self-reported mental health symptoms (DSM-5 Cross Cutting Measure) between post-treatment and 6-month follow-up. Ratings of understanding and embracing TranS-C modules were not associated with any of the outcome measures at post-treatment or a change between post-treatment and 6-month follow-up.

Table 4.

Multilevel Models Examining if Participant Responsiveness to TranS-C is Associated with Treatment Outcome

Outcome Measure	Participant responsiveness measure effect on outcome measure at post-treatment				Participant responsiveness measure effect on change in outcome measure between post-treatment and 6-month follow-up
	Beta	SE	p	95% CI	Beta	SE	p	95% CI
Independent-Rater TranS-C Checklist (understanding and embracing)
PROMIS-SD	0.03	0.11	0.79	−0.19, 0.25	0.02	0.14	0.91	−0.25, 0.28
PROMIS-SRI	−0.03	0.12	0.83	−0.26, 0.21	0.02	0.15	0.92	−0.29, 0.32
DSM-5 Cross Cutting	0.05	0.12	0.64	−0.18, 0.28	−0.29	0.14	0.04*	−0.57, −0.02
SDS	−0.05	0.11	0.63	−0.28, 0.17	0.08	0.16	0.61	−0.23, 0.38
Usefulness Scale (Average Usefulness Treatment Score)
PROMIS-SD	−0.33	0.08	0.00***	−0.50, −0.17	0.05	0.10	0.61	−0.15, 0.25
PROMIS-SRI	−0.20	0.08	0.02*	−0.36, −0.04	0.00	0.10	0.99	−0.19, 0.19
DSM-5 Cross Cutting	−0.25	0.08	0.00**	−0.40, −0.09	0.14	0.10	0.15	−0.05, 0.33
SDS	−0.15	0.08	0.05	−0.31, 0.00	0.01	0.10	0.89	−0.18, 0.21
Utilization Scale (Average Utilization Treatment Score)
PROMIS-SD	0.10	0.09	0.24	−0.07, 0.28	−0.18	0.12	0.12	−0.42, 0.05
PROMIS-SRI	0.08	0.08	0.34	−0.08, 0.25	−0.16	0.11	0.16	−0.38, 0.06
DSM-5 Cross Cutting	0.03	0.08	0.73	−0.13, 0.19	0.05	0.11	0.67	−0.17, 0.27
SDS	0.13	0.08	0.12	−0.03, 0.29	−0.18	0.11	0.10	−0.40, 0.04

Note.

^*p < 0.05.

^**p < 0.01.

^***p < 0.001.

PROMIS-SD = Patient-Reported Outcomes Measurement Information System – Sleep Disturbance. PROMIS-SRI = Patient-Reported Outcomes Measurement Information System – Sleep-Related Impairment. DSM-5 Cross-Cutting = DSM-5 Self-Rated Level 1 Cross-Cutting Symptom Measure – Adult. SDS = Sheehan Disability Scale.

As displayed in Table 5, the following TranS-C modules were significantly associated with improvement on the DSM-5 Cross Cutting Measure scores between post-treatment and 6-month follow-up: Goal Setting, Irregular Sleep-Wake Times, Daytime Impairment, and Negotiating Sleep in a Complicated Environment. None of the other TranS-C modules were significantly associated with a change in DSM-5 Cross Cutting scores between post-treatment and 6-month follow-up. None of the TranS-C Modules were significantly associated with the DSM-5 Cross Cutting Measure at post-treatment.

Table 5.

Multilevel Models Examining if Independent-Rater TranS-C Checklist Responsiveness Items (Embracing/Understanding) are associated with Treatment the DSM-5 Cross Cutting Measure

Independent-Rater Checklist Module	Effect of Independent-Rater TranS-C Checklist understanding/embracing item on DSM-5 Cross Cutting at post-treatment				Effect of Independent-Rater TranS-C Checklist understanding/embracing item on change in DSM-5 Cross Cutting scores between post-treatment and 6-month follow-up
	Beta	SE	p	95% CI	Beta	SE	p	95% CI
Cross-Cutting Modules
Functional analysis	0.02	0.12	0.84	−0.21, 0.25	−0.25	0.15	0.09	−0.53, 0.04
Education	0.03	0.12	0.83	−0.21, 0.26	−0.18	0.15	0.22	−0.48, 0.12
Motivation enhancement	0.02	0.12	0.90	−0.22, 0.25	−0.09	0.15	0.56	−0.39, 0.21
Goal setting	0.12	0.13	0.38	−0.14, 0.37	−0.41	0.15	0.01**a	−0.71, −0.12
Progress monitoring (not a formal module)	0.13	0.12	0.27	−0.10, 0.36	0.06	0.15	0.70	−0.23, 0.35
Core Modules
Irregular sleep-wake times	0.01	0.12	0.94	−0.23, 0.25	−0.30	0.15	0.05*b	−0.59, −0.01
Difficulty winding down	0.02	0.12	0.85	−0.20, 0.25	−0.11	0.14	0.43	−0.40, 0.19
Difficulty waking up	0.01	0.13	0.93	−0.25, 0.27	−0.15	0.16	0.34	−0.47, 0.16
Daytime impairment	0.07	0.14	0.60	−0.20, 0.34	−0.44	0.16	0.01**c	−0.76, −0.12
Unhelpful beliefs about sleep	−0.01	0.14	0.97	−0.27, 0.26	0.05	0.18	0.79	−0.30, 0.39
Maintenance of behavior change	−0.02	0.12	0.86	−0.26, 0.21	−0.07	0.15	0.62	−0.36, 0.22
Optional Modules
Poor sleep efficiency	−0.04	0.13	0.77	−0.30, 0.22	−0.07	0.16	0.66	−0.39, 0.25
Too much time in bedd	-	-	-	-	-	-	-	-
Delayed or advanced phase	−0.01	0.24	0.72	−0.56, 0.39	0.25	0.28	0.36	−0.29, 0.79
Sleep-related worry	0.05	0.13	0.71	−0.20, 0.29	−0.16	0.16	0.30	−0.47, 0.15
Promoting compliance with CPAP/other treatments for comorbid sleep problemsd	-	-	-	-	-	-	-	-
Negotiating sleep in a complicated environment	0.07	0.64	0.91	−1.19, 1.33	−1.27	0.52	0.02*	−2.29, −0.25
Reducing Nightmaresd	-	-	-	-	-	-	-	-
Follow up on sleep apnea assessment referrals (not a formal module)	0.09	0.27	0.74	−0.44, 0.63	−0.71	0.35	0.05	−1.40, −0.02

Note.

^*p < 0.05.

^**p < 0.01.

^***p < 0.001.

^aThis p value = 0.006.

^bThis p value = 0.046.

^cThis p value is 0.007.

^dThe models for these three modules can’t run because only 2–3 participants received them and there is not enough variability. DSM-5 Cross-Cutting = DSM-5 Self-Rated Level 1 Cross-Cutting Symptom Measure – Adult.

Usefulness Scale.

Also evident in Table 4, the Average Usefulness Treatment Score was associated with a significant reduction in the PROMIS-SD, PROMIS-SRI, and DSM-5 Cross Cutting Measure at post-treatment, such that a greater Average Usefulness Treatment Score was associated with a reduction in scores on these measures, or fewer sleep disturbance symptoms, less sleep-related impairment, and fewer mental health symptoms.

The relationship between Usefulness Scores for individual treatment elements and the PROMIS-SD, PROMIS-SRI, and DSM-5 Cross Cutting Measure are displayed in Table 6. We conducted these analyses on the PROMIS-SD, PROMIS-SRI, and DSM-5 Cross Cutting Measure, but not on the SDS, as these three outcome measures were significantly associated with the Average Usefulness Treatment Score. The following treatment elements were significantly associated with the PROMIS-SD, the measure of sleep disturbance, at post-treatment: “I try to go to sleep at about the same time each night,” “I try to wake up about the same time each morning,” “I try to get about 7–8.5 hours of sleep per night,” “I engage in a wind-down before bedtime,” “I avoid napping in the evening,” “I use a wake up routine to help me get up and get going in the morning,” “When I feel sleepy in the day I purposively generate energy,” “I get out of bed if I am not able to sleep within about 20 minutes,” and “I keep my bed for sleeping only.” None of the Usefulness Scores of other treatment elements were significantly associated with the PROMIS-SD at post-treatment or with a change in PROMIS-SD scores from post-treatment to 6-month follow-up.

Table 6.

Multilevel Models Examining if Usefulness Scores are associated with Treatment Outcome

Treatment Elements	Effect of usefulness on each outcome measure at post-treatment				Effect of usefulness on change in each outcome measure between post-treatment and 6-month follow-up
	Beta	SE	p	95% CI	Beta	SE	p	95% CI
PROMIS-SD
1. I try to go to sleep about the same time each night	−0.31	0.08	0.00***	−0.46, −0.16	0.04	0.10	0.68	−0.16, 0.24
2. I try to wake up about the same time each morning	−0.31	−0.08	0.00***	−0.46, −0.16	0.04	0.11	0.74	−0.17. 0.24
3. I try to get about 7–8.5 hours of sleep per night	−0.22	0.08	0.00**	−0.38. −0.07	−0.06	0.10	0.54	−0.27, 0.14
4. If my bedtime gets too late, I bring it forward by about 20–30 minutes each week	−0.11	0.09	0.19	−0.28, 0.06	−0.11	0.11	0.30	−0.32, 0.10
5. If my bedtime is too early, I bring it later by about 20–30 minutes each week	−0.10	0.08	0.22	−0.26, 0.06	−0.10	0.11	0.36	−0.32, 0.11
6. I engage in a wind-down before bedtime	−0.21	0.08	0.01*	−0.37. −0.05	0.09	0.11	0.38	−0.11, 0.30
7. I avoid napping in the evening	−0.25	0.08	0.00**	−0.41, −0.08	0.04	0.10	0.68	−0.16, 0.25
8. I have an electronic curfew to reduce light exposure via cell phones, computers television etc before my bedtime	−0.10	0.09	0.23	−0.27. 0.06	−0.05	0.11	0.65	−0.27, 0.16
9. I use a wake up routine to help me get up and get going in the morning (eg., try to Refrain from snoozing, Increase activity, Shower or wash face and hands, Expose yourself to sunlight, Upbeat music, Phone a friend)	−0.33	0.08	0.00***	−0.49, 0.17	0.01	0.10	0.89	−0.18, 0.21
10. When I feel sleepy in the day I purposively generate energy	−0.25	0.09	0.00**	−0.42, −0.08	0.00	0.11	0.97	−0.20, 0.22
11. I use techniques to reduce worry interfering with my sleep via savoring or ‘worry time’ earlier in the day or gratitude etc.	−0.16	0.08	0.06	−0.32, 0.01	−0.01	0.11	0.90	−0.23, 0.20
12. I get out of bed if I am not able to sleep within about 20 minutes	−0.40	0.09	0.00***	−0.57, −0.23	0.18	0.10	0.07	−0.02, 0.39
13. I keep my bed for sleeping only (I do not work in bed, watch TV in bed etc)	−0.28	0.08	0.00**	−0.44, −0.11	−0.13	0.11	0.22	−0.08, 0.34
14. I avoid caffeine, energy drinks, and alcohol to help me sleep better	−0.11	0.09	0.20	−0.28, 0.06	−0.05	0.11	0.65	−0.26, 0.16
15. I use dim lights to help me feel sleepy at bedtime	−0.07	0.08	0.39	−0.23, 0.09	−0.02	0.11	0.83	−0.24, 0.18
16. I turn on bright lights in the morning to help me wake up	−0.08	0.08	0.32	−0.25, 0.08	0.14	0.11	0.20	−0.07, 0.34
17. I avoid going to sleep while listening to the radio, watching TV, etc
PROMIS-SRI
1. I try to go to sleep about the same time each night	−0.24	0,07	0.00**	−0.38, −0.09	0.04	0.10	0.71	−0.15, 0.22
2. I try to wake up about the same time each morning	−0.10	0.08	0.17	−0.25, 0.05	−0.04	0.10	0.67	−0.25, 0.16
3. I try to get about 7–8.5 hours of sleep per night	−0.08	0.07	0.31	−0.22, 0.07	−0.09	0.10	0.36	−0.28, 0.10
4. If my bedtime gets too late, I bring it forward by about 20–30 minutes each week	−0.15	0.08	0.07	−0.31, 0.01	−0.05	0.10	0.62	−0.24, 0.15
5. If my bedtime is too early, I bring it later by about 20–30 minutes each week	−0.08	0.08	0.29	−0.23, 0.07	−0.14	0.10	0.18	−0.34, 0.06
6. I engage in a wind-down before bedtime	−0.16	0.08	0.04*	−0.30, −0.00	0.08	0.10	0.44	−0.12, 0.27
7. I avoid napping in the evening	0.03	0.08	0.71	−0.13, 0.19	−0.11	0.10	0.28	−0.20, 0.09
8. I have an electronic curfew to reduce light exposure via cell phones, computers television etc before my bedtime	−0.12	0.08	0.13	−0.28, 0.03	−0.04	0.10	0.71	−0.24, 0.16
9. I use a wake up routine to help me get up and get going in the morning (eg., try to Refrain from snoozing, Increase activity, Shower or wash face and hands, Expose yourself to sunlight, Upbeat music, Phone a friend)	0.00	0.08	0.97	−0.15, 0.16	−0.20	0.10	0.04*	−0.39, −0.01
10. When I feel sleepy in the day I purposively generate energy	−0.19	0.08	0.02*	−0.36, −0.03	0.03	0.10	0.76	−0.17, 0.23
11. I use techniques to reduce worry interfering with my sleep via savoring or ‘worry time’ earlier in the day or gratitude etc.	−0.08	0.08	0.31	−0.23, 0.07	−0.06	0.10	0.56	−0.25, 0.14
12. I get out of bed if I am not able to sleep within about 20 minutes	−0.21	0.08	0.01*	−0.38, −0.04	0.13	0.10	0.20	−0.07, 0.23
13. I keep my bed for sleeping only (I do not work in bed, watch TV in bed etc)	−0.21	0.08	0.01*	−0.36, −0.05	0.06	0.10	0.53	−0.13, 0.26
14. I avoid caffeine, energy drinks, and alcohol to help me sleep better	−0.09	0.08	0.27	−0.25, 0.07	−0.08	0.10	0.41	−0.28, 0.11
15. I use dim lights to help me feel sleepy at bedtime	−0.14	0.08	0.06	−0.30, 0.01	0.10	0.10	0.31	−0.10, 0.30
16. I turn on bright lights in the morning to help me wake up	−0.08	0.08	0.30	−0.24, 0.07	0.11	0.10	0.26	−0.08, 0.31
17. I avoid going to sleep while listening to the radio, watching TV, etc	−0.02	0.08	0.76	−0.17, 0.12	−0.03	0.10	0.81	−0.22, 0.17
DSM-5 Cross Cutting
1. I try to go to sleep about the same time each night	−0.21	0.08	0.00**	−0.36. −0.06	0.14	0.10	0.16	−0.05, 0.33
2. I try to wake up about the same time each morning	−0.20	0.07	0.00**	−0.34, −0.06	0.06	0.10	0.54	−0.14, 0.26
3. I try to get about 7–8.5 hours of sleep per night	−0.12	0.07	0.11	−0.26, 0.03	−0.00	0.10	0.99	−0.19, 0.19
4. If my bedtime gets too late, I bring it forward by about 20–30 minutes each week	−0.11	0.08	0.18	−0.27. 0.05	0.03	0.10	0.76	−0.17, 0.23
5. If my bedtime is too early, I bring it later by about 20–30 minutes each week	−0.07	0.08	0.39	−0.22, 0.08	0.02	0.10	0.88	−0.19, 0.22
6. I engage in a wind-down before bedtime	−0.18	0.08	0.02*	−0.33, 0.03	0.15	0.10	0.11	−0.04, 0.34
7. I avoid napping in the evening	−0.08	0.08	0.33	−0.24, 0.08	−0.01	0.10	0.93	−0.21, 0.19
8. I have an electronic curfew to reduce light exposure via cell phones, computers television etc before my bedtime	−0.13	0.08	0.11	−0.29, 0.03	0.01	0.10	0.91	−0.19, 0.21
9. I use a wake up routine to help me get up and get going in the morning (eg., try to Refrain from snoozing, Increase activity, Shower or wash face and hands, Expose yourself to sunlight, Upbeat music, Phone a friend)	−0.18	0.08	0.02*	−0.34, −0.02	0.07	0.10	0.45	−0.12, 0.27
10. When I feel sleepy in the day I purposively generate energy	−0.23	0.08	0.00**	−0.39. −0.08	0.10	0.10	0.31	−0.09, 0.30
11. I use techniques to reduce worry interfering with my sleep via savoring or ‘worry time’ earlier in the day or gratitude etc.	−0.12	0.08	0.11	−0.27, 0.03	0.05	0.10	0.65	−0.15, 0.25
12. I get out of bed if I am not able to sleep within about 20 minutes	−0.22	0.08	0.00**	−0.39, −0.06	0.18	0.10	0.07	−0.02, 0.38
13. I keep my bed for sleeping only (I do not work in bed, watch TV in bed etc)	−0.15	0.08	0.07	−0.31, 0.01	0.12	0.10	0.24	−0.08, 0.32
14. I avoid caffeine, energy drinks, and alcohol to help me sleep better	−0.09	0.08	0.25	−0.25, 0.06	0.05	0.10	0.64	−0.15, 0.25
15. I use dim lights to help me feel sleepy at bedtime	−0.14	0.08	0.06	−0.29, 0.01	0.09	0.10	0.37	−0.11, 0.29
16. I turn on bright lights in the morning to help me wake up	−0.19	0.08	0.01*	−0.34, −0.04	0.19	0.10	0.06	−0.01, 0.38
17. I avoid going to sleep while listening to the radio, watching TV, etc	−0.11	0.08	0.15	−0.26, 0.04	0.08	0.10	0.45	−0.12, 0.38

Note.

^*p < 0.05.

^**p < 0.01.

^***p < 0.001.

PROMIS-SD = Patient-Reported Outcomes Measurement Information System – Sleep Disturbance. PROMIS-SRI = Patient-Reported Outcomes Measurement Information System – Sleep-Related Impairment. DSM-5 Cross-Cutting = DSM-5 Self-Rated Level 1 Cross-Cutting Symptom Measure – Adult.

The following treatment elements were significantly associated with the PROMIS-SRI, the measure of sleep-related impairment, at post-treatment: “I try to go to sleep at about the same time each night,” “I engage in a wind-down before bedtime,” “When I feel sleepy in the day I purposively generate energy,” “I get out of bed if I am not able to sleep within about 20 minutes,” and “I keep my bed for sleeping only.” “I use a wake up routine to help me get up and get going in the morning” was significantly associated with improvement in PROMIS-SRI scores between post-treatment and 6-month follow-up. None of the Usefulness Scores of other treatment elements were significantly associated with the PROMIS-SRI at post-treatment or with a change in PROMIS-SRI scores from post-treatment to 6-month follow-up.

The following treatment elements were significantly associated with the DSM-5 Cross Cutting Measure, the measure of self-report mental health symptoms, at post-treatment: “I try to go to sleep at about the same time each night,” “I try to wake up about the same time each morning,” “I engage in a wind-down before bedtime,” “I use a wake up routine to help me get up and get going in the morning,” “When I feel sleepy in the day I purposively generate energy,” “I get out of bed if I am not able to sleep within about 20 minutes,” and “I turn on bright lights in the morning to help me wake up.” None of the Usefulness Scores of other treatment elements were significantly associated with the DSM-5 Cross Cutting Measure at post-treatment or with a change in DSM-5 Cross Cutting Measure scores from post-treatment to 6-month follow-up.

Utilization Scale.

As evident in Table 4, the Average Utilization Treatment Score was not significantly associated with treatment outcome at post-treatment or with a change in outcome measures between post-treatment and 6-month follow-up.

Discussion

The present study examined patient responsiveness – operationalized as receptivity, usefulness, and utilization – to TranS-C provided to adults with SMI in a community mental health setting. The first aim was to describe patient responsiveness to TranS-C. In support of our hypotheses, on average, all but one TranS-C module was rated as fully understood and embraced (with a mean score close to 1, with 1 being fully understanding/embracing), the average usefulness of treatment elements was rated as “moderately” useful with all elements rated as at least “somewhat” useful, and the average utilization of treatment elements was rated as “occasionally” utilized with all elements utilized at least “rarely.” These findings are aligned with ratings of usefulness and utilization of TranS-C made by adolescents (Gumport et al., 2019). Consistent with the existing literature on the efficacy and acceptability of stimulus control (Bootzin, 1972), and a prior study examining ratings of CBT for insomnia (L. Harvey et al., 2002), the TranS-C elements rated as the most useful and utilized are all drawn from stimulus control (e.g., avoid naps, consistent bed and wake times). The module that was not rated as fully understood was the optional module of Negotiating Sleep in a Complicated Environment, which was rated on average as “somewhat” understood and embraced. As this module was provided to only seven adults with SMI in this sample, our ability to generalize the results of this finding may be limited. However, the findings suggest that providers should carefully explain how to navigate challenging environments for sleep (e.g., shared rooms, different schedules from housemates, etc.) to ensure the patient understands. In addition, it is possible that this module may be difficult to implement in certain settings such a board and care home due to factors outside of the control of the individual (e.g., overhead lights that the patient may not be able to turn off, room-sharing with others with SMI and sleep disturbance, etc. (Kyle & Dunn, 2008; T. L. Taylor et al., 2009). Taken together, these results add to the literature on patient responsiveness to TranS-C by offering data that the treatment elements were understood, perceived as useful, and were used by participants. More generally, these findings suggest that TranS-C is well received by adults with SMI receiving treatment in a community mental health setting, in addition to adolescents (Gumport et al., 2019), although future research should consider how to further increase these ratings. The current findings also advance our knowledge on patient responsiveness to TranS-C in the literature: namely by examining ratings of patient understanding/embracing of TranS-C.

The second aim was to examine if patient responsiveness to TranS-C was associated with improved treatment outcome. In partial support of our hypothesis, higher ratings of understanding and embracing TranS-C were associated with a decrease in self-reported mental health symptoms via the DSM-5 Cross Cutting Measure scores between post-treatment and 6-month follow-up. This finding is consistent with other research examining patient receptivity to interventions (Mihalic et al., 2008; Vroom et al., 2020), and to the best of our knowledge, this is the first study to examine these ratings in the context of a sleep and circadian intervention. These findings are aligned with other literature that has demonstrated that improvement in sleep and circadian functioning result in improvements in other mental health symptoms (Freeman et al., 2017; D. J. Taylor & Pruiksma, 2014). However, ratings of embracing and understanding TranS-C were not associated with the other outcome measures, including sleep measures. It is possible that the lack of association is due to the smaller sample of patients (n = 33) selected for ratings or to the somewhat restricted range, as the mean rating for most modules was close to 1, with small standard deviations.

As expected, ratings of the usefulness of TranS-C as a whole – or the Average TranS-C Usefulness Score – were associated with lower scores on measures of sleep disturbance, sleep impairment, and mental health symptoms. This is consistent with prior research that has established a relationship between ratings of usefulness with improvement in sleep symptoms focusing on both TranS-C in adolescents (Gumport et al., 2019) and CBT-I for adults (Vincent & Lionberg, 2001) as well as with physical and mental health symptoms (Cox et al., 1994; Lawton et al., 2015). Consistent with our hypothesis, when examining the ratings of the usefulness of individual TranS-C elements, establishing a wind down, avoiding naps, maintaining a consistent bed and wake time, and getting out of bed if unable to sleep were associated with improvements in treatment outcome, both sleep and SMI symptoms. Given that these elements are drawn from stimulus control (Bootzin, 1972), an established evidence-based treatment for insomnia (Edinger et al., 2021), it is encouraging that belief in the helpfulness of these elements was associated with improvement in outcomes among adults with SMI in this publicly-funded mental health setting. The usefulness findings offer additional evidence of the acceptability of delivering evidence-based sleep interventions in a community mental health setting (Gumport et al., 2020).

Inconsistent with our hypothesis, ratings of the utilization of TranS-C as a whole – or the Average TranS-C Utilization Score – were not associated with any of the measures of treatment outcome used, specifically sleep duration, sleep-related impairment, self-reported mental health symptoms, and functional impairment. However, the findings were in the expected direction for three of the outcome measures: PROMIS-SD, PROMIS-SRI, and SDS. This null finding is surprising, given that ratings of utilization of TranS-C were associated with sleep improvement in adolescents (Gumport et al., 2019) and in mid-life and older adults (Sarfan, Gasperetti, Gumport, & Harvey, under review) and ratings of utilization of CBT for insomnia elements have been associated with treatment outcome in adults with insomnia (L. Harvey et al., 2002). As SMI is associated with memory impairment (e.g., Boyer et al., 2007; Gotlib & Joormann, 2010), perhaps adults with SMI struggle to accurately report how often they truly utilize treatment elements. It is also possible that more emphasis is needed to integrate the science of habit formation into TranS-C, particularly for people diagnosed with a SMI (A. G. Harvey, Callaway, et al., 2021). Taken together, the results examining the association of the three elements of responsiveness – receptivity, usefulness, and utilization – with treatment outcome highlight which modules and elements of TranS-C may be the most important for providers to prioritize when working with patients with SMI in a community mental health setting.

Several limitations should be noted. First, most measures in this study relied on self-report, including the sleep assessment. Future studies may consider also including objective assessments, although this may be challenging in routine practice settings. Second, the sample was relatively small, particularly for examining patient understanding and embracing of TranS-C modules, and multiple comparisons were used. Future research with a larger sample is needed to replicate these findings. Based on Nakagawa and Cuthill (2007), corrections for multiple comparisons further reduce power, increase the likelihood of a Type II error, and may contribute to publication bias. Therefore, we present effect sizes rather than correct for multiple comparisons as suggested by Nakagawa and Cuthill (2007). Standardized coefficients, presented above, are interpretable as effect sizes (Lorah, 2018). Third, analyses were conducted on a treatment-completing sample, rather than intent-to-treat, so may not be generalizable to individuals who terminate treatment early. Fourth, the Usefulness and Utilization Scales have not been validated. Future research is needed to validate these scales. Finally, we examined only one treatment for sleep and circadian problems, TranS-C. Elements of cognitive behavior therapy for insomnia, the frontline recommendation for addressing insomnia (Edinger et al., 2021), are incorporated into TranS-C. Studies may want to examine responsiveness to other treatments for sleep and circadian problems, such as light boxes and melatonin (Burgess et al., 2002). However, these treatments may not be feasible within publicly-funded settings due to the cost.

Notwithstanding these limitations, the present study adds to the literature examining the role of patient responsiveness specifically for SMI and for sleep and circadian treatment (L. Harvey et al., 2002; Schoenwald & Garland, 2013). Although not all measures of patient responsiveness were associated with outcome, the findings from this study are promising because responsiveness is a part of fidelity, an important implementation outcome (Proctor et al., 2011). That individuals with SMI receiving treatment in a publicly-funded mental health care setting and a marginalized population appear to embrace and understand and find TranS-C helpful offers further support for the utility of TranS-C within real world settings.