Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Jul 1.
Published in final edited form as: Psychiatry Res. 2021 Apr 21;301:113956. doi: 10.1016/j.psychres.2021.113956

Feasibility and Acceptability of Remotely Accessed Cognitive Remediation for Schizophrenia in Public Health Settings

Alice Medalia a,*, Alice M Saperstein b, Ana Stefancic b, Shanique Meyler b, Sarah Styke b, Min Qian c, Jun Liu d, Leopoldo J Cabassa e
PMCID: PMC8206004  NIHMSID: NIHMS1699684  PMID: 33962354

Abstract

Cognitive remediation (CR) is an evidence-based therapy used to improve cognition in people with schizophrenia. However, it often requires multiple in-person clinic sessions per week, which can limit scalability. This mixed methods study considered the feasibility and acceptability of a hybrid approach, which allowed for half the sessions to be conducted remotely as homework, without the clinician present. Individuals with schizophrenia were randomized to either all in-clinic or hybrid conditions and completed questionnaires and individual interviews about their experience. CR clinicians provided feedback in complement. Because of limited access to technology, most Hybrid CR participants had to come to clinic to access computers and often sought clinician support to do their homework. Participants in the two conditions were equally satisfied per the Client Satisfaction Questionnaire, and the majority reported perceived benefit and enjoyment. Both CR participants and clinicians identified access to technology as a barrier to program feasibility, while availability of clinician support positively impacted acceptability. Suggestions to improve CR highlighted adopting a flexible approach to providing CR that accounts for participant access to technology, potential benefit from peer interaction, and need for clinician support.

Keywords: schizophrenia, cognitive remediation, telepsychiatry, mixed methods

1. Introduction

Schizophrenia is associated with persistent neurocognitive deficits that are largely unresponsive to pharmacologic treatment and add to illness burden by imposing significant limitations on the ability to benefit from psychosocial programs, adhere to pharmacotherapy, and function in the community (Bowie and Harvey, 2006). To address cognitive health needs, cognitive remediation (CR), an evidence-based behavioral training intervention that improves neurocognitive processes and real-world functioning, is increasingly being incorporated into clinic settings (Medalia and Erlich, 2017). Typically, CR involves clinician run sessions at least twice weekly (Bowie et al., 2019) so that participants gain adequate practice on the cognitive exercises, which are usually provided via internet. However, multiple in-person sessions a week can pose a transportation and/or scheduling burden on participants, limiting their ability to participate. Clinicians participating in a large system of care CR implementation project (Medalia et al., 2019) suggested more clients would attend CR if the in-person requirement were reduced to once weekly. Such a scheduling change would, however, require that clients could independently access the internet and sufficiently practice exercises to make cognitive gains.

There is a precedent for offering CR remotely in a research context (Ventura et al., 2013). In theory, CR participants with access to web-enabled tablets or computers could be given cognitive exercises as a form of homework. However, racial/ethnic and socio-economic differences can impact access to technology (Eberly et al., 2020; Torous and Keshevan, 2020). Further, lack of computer experience could lead to a frustrating learning experience, if a clinician were not present to lend support and guidance. Thus, remotely delivered CR may have the potential of increasing access to the service and lowering the cost of treatment, but its feasibility and acceptability in public sector clinics first needs to be established.

This study used quantitative and qualitative methods to test the feasibility and acceptability of delivering CR when half the sessions required independent remote access to cognitive exercises. The study was conducted at clinics already offering CR at scheduled clinician-led, all clinic-based sessions (Medalia et al., 2019). Development of the Hybrid CR approach followed both practice-based evidence and published guidelines for the use of internet interventions in psychosis (Alvarez-Jimenez et al., 2012; Gaebel et al., 2016) which recommend that: (1) remote interventions are developed in accordance with stakeholder input, (2) remote online-based interventions augment rather than replace existing models of care, and (3) technology is integrated with evidence-based therapy and professional support to promote acceptance and effectiveness. By offering Hybrid CR participants flexibility in when and where they scheduled some of cognitive exercise practice, we tested if they were able to amass the same exposure to practice as people who completed all sessions in-clinic. Methods were designed to provide preliminary data for a future effectiveness trial as well as address important practice issues. The Hybrid approach to CR piloted in this study is the first we are aware of that tested using remote CR in a public system of psychiatric care, and used a mixed methods approach to consider feasibility, stakeholder acceptability, and the facilitators and barriers to implementation.

2. Methods

2.1. Participants and procedures

This registered clinical trial (NCT03576976) was conducted under the guidance of the New York State Psychiatric Institute Institutional Review Board at 6 outpatient clinics operated by New York State Office of Mental Health (OMH) in the New York City metro area. All clinics serve adults with serious mental illness and have an active CR program within which research procedures were integrated. Adults referred for CR were provided the opportunity to participate in research and signed informed consent to participate. Research criteria for inclusion were age 18-65 years old, a DSM-5 diagnosis of schizophrenia or schizoaffective disorder and English fluency. Exclusion criteria were documented auditory or visual impairment that precluded completing assessment, intellectual disability, neurologic conditions affecting brain physiology, active substance use disorder, and participation in CR in the 12 months prior to study entry. Individuals not meeting eligibility criteria or not interested in research continued with the standard CR service. All participants continued receiving their usual mental health services, which included psychopharmacology, care coordination and group therapies.

The study used a repeated-measures, intent to treat, stratified randomized design. Eligible participants were randomized to one of two conditions, all-clinic CR (Clinic) or clinic-plus-remote (Hybrid) CR, both of which were run by clinic staff. Randomization was stratified by clinics. A research assistant, unblind to treatment condition, monitored fidelity to the protocol and conducted post-treatment qualitative interviews with Hybrid CR participants about their experience using remote technology. A second research assistant blind to treatment condition conducted structured post-treatment satisfaction surveys with all participants. CR clinicians who provided informed consent were also interviewed between study mid- and end-point about facilitators and barriers to Hybrid CR delivery and the perceived benefits of Hybrid as compared to Clinic CR. Patient participants were compensated for assessment procedures.

2.2. The CR intervention

Clinic and Hybrid CR conditions were structured to differ in time spent with clinician and peers, and group discussion. Clinic CR (Medalia et al., 2019) consists of thirty 60-minute clinician-led sessions delivered twice weekly in a small group format with rolling admission. Within each session, participants spend 45 minutes training on 3-4 computerized exercises selected by the clinician from a menu of web-based programs to improve the cognitive functions identified as impaired on a baseline assessment. Exercises came from three websites: Brain HQ, Lumosity, and Scientific Brain Training Pro. All exercises continuously adjust the difficulty level to user performance to maintain an approximately 80% rate of correct responses. Training time and performance are tracked by the web-based cognitive exercise programs. Each session includes a 15-minute manualized group-based discussion based on the concept of “Bridging,” intended to promote generalization by linking cognitive activities to daily activities and recovery goals. In Hybrid CR, participants attended one clinician-led session that followed the above format. Hybrid CR participants were provided a weekly “homework sheet” which guided 60 minutes of independent practice on cognitive exercises assigned by the clinician, to be completed between group sessions on any available internet-connected computer or tablet. As implemented, access issues were mitigated by making clinic computer workstations available for independent practice. During the scheduled group session, clinicians individually discussed homework activities, problem solved access or activity related issues, provided personalized bridging, encouragement and feedback, and reviewed the exercises to be completed before the next session.

2.3. Quantitative data collection and analysis

CR participant baseline characteristics in the two treatment arms were compared using Wilcoxon’s rank sum test for continuous variables and Fisher’s exact test for categorical variables. Feasibility was quantified by summing time on computer tasks divided by weeks trained for each completer to yield a measure of weekly training time which was compared between Clinic and Hybrid groups using Wilcoxon’s Rank Sum Test. Acceptability was operationalized and analyzed as follows. (1) Acceptability of randomization was calculated as the % of randomized participants who attended the first session of the intervention to which they were assigned and compared using Fisher’s exact test. (2) Acceptability of in-person intervention was calculated as the % of randomized participants who completed the in person sessions, defined as having attended at least 20 clinician-led sessions in the Clinic condition or 10 clinician-led sessions in the Hybrid condition and compared using Fisher’s exact test. (3) Acceptability of homework was calculated as percent of assigned weekly homework that was completed, per client’s homework sheet. (4) Treatment Satisfaction was assessed using questionnaires. The Client Satisfaction Questionnaire (CSQ-8; Attkisson and Greenfield, 2004) was administered at post-treatment. Each of eight items is rated on a 4-point Likert scale; overall scores range from 8 to 32, with higher values indicating higher satisfaction. Responses on the CSQ-8 were compared between Clinic and Hybrid participants using Wilcoxon’s Rank Sum Test. Additional survey feedback, obtained from CR participants and clinicians using a 6-point Likert scale, assessed perceptions of the Hybrid experience (see supplemental data). Responses from CR participants and CR clinicians were summarized using descriptive statistics.

2.4. Qualitative data collection and analysis

Semi-structured qualitative interviews were conducted in person with Hybrid CR completers (n=17; 3 were unavailable due to the pandemic) and CR clinicians (n=6; 2 were unavailable due to staff turnover). Two experienced qualitative researchers (AS,LJC) oversaw development of the interview guides, trained and supervised qualitative interviewers (SM,SS), and oversaw data analysis. Interviews lasted 30 minutes, were audio-recorded, and transcribed verbatim. See supplemental data for sample questions.

A content analysis approach was used to systematically label and organize transcript excerpts into broad categories and concepts, and to identify patterns within and across codes and interviewee types (Bernard, 2002). Authors SM, SS and AS read an initial set of transcripts to assess the depth and range of content and identify emerging topics and concepts in memos that were used to develop a preliminary codebook. The codebook consisted of a priori category labels rooted in interview questions (e.g., homework experience), as well as concepts emerging directly from the data (e.g., computer accessibility, clinician assistance). Authors SM and SS coded an initial five transcripts, resolved discrepancies through consensus with input from author AS, and revised the codebook after iterative reading and discussion of coded data with other authors. Once coding application by SM and SS was consistent with the revised codebook, they completed final coding independently in ATLAS.ti (Scientific Software Development GmbH, Berlin, Germany). Authors SM, SS, AS, and AM reviewed and clustered reports of codes to identify preliminary themes through iterative discussion. Strategies to enhance rigor included frequent interview and analysis debriefings, multiple researchers coding and interpreting the data, and development of memos constituting an audit trail of key analytic decisions and processes (Creswell, 2003).

2.5. Integration of methods

Collection and analysis of quantitative and qualitative data from CR participants and clinicians occurred simultaneously. Interim quantitative findings further guided the focus of qualitative analyses to focus on identifying factors that contributed to understanding facilitators and barriers to Hybrid CR implementation. All authors reviewed results emerging from each analysis and subsequently developed a thematic matrix that organized qualitative findings by the domains of feasibility and acceptability, and interviewee type in ways that elaborated and expanded on quantitative results.

3. Results

3.1. Subjects

3.1.1. CR participants

Of 67 individuals consented to participate, 62 (92.5%) met inclusion criteria and 55 (88.7%) were randomized to a CR condition (Clinic n=27, Hybrid n=28). Prior to randomization, 6 consented individuals lost interest and 1 commenced employment. Characteristics of the randomized sample are shown in Table 1.

Table 1.

Sample Characteristics

Randomized Sample (N = 55) Hybrid Condition (N = 28) All-Clinic Condition (N = 27) p value
Age
 Mean (SD) 45.71 (12.73) 48.99 (12.05) 42.32 (12.73) 0.051
Sex n (%)
 Male 39 (70.9%) 19 (67.86%) 20 (74.07%) 0.833
 Female 16 (29.1%) 9 (32.14%) 7 (25.93%)
Race n (%)
 American Indian or Alaska Native 1 (1.8%) 1 (3.57%) 0 (0%) 0.256
 Asian 2 (3.6%) 2 (7.14%) 0 (0%)
 Black or African American 32 (58.2%) 14 (50%) 18 (66.67%)
 White 18 (32.7%) 11 (39.29%) 7 (25.93%)
 More than one race 1 (1.8%) 0 (0%) 1(3.7%)
 Unknown 1 (1.8%) 0 (0%) 1 (3.7%)
Ethnicity n (%)
 Hispanic/Latinx 17 (30.9%) 8 (28.57%) 9 (33.33%) 0.928
 Not Hispanic/Latinx 38 (69.1%) 20 (71.43%) 18 (66.67%)
Psychiatric Diagnosis n (%)
 Schizophrenia 31 (56.4%) 18 (64.29%) 13 (48.15%) 0.35
 Schizoaffective Disorder 24 (43.6%) 10 (35.71%) 14 (51.85%)
Years Education
 Mean (SD) 12.2 (2.2) 12.68 (1.72) 11.7 (2.54) 0.10
Employment Status n (%)
 Competitively Employed 2 (3.64%) 1 (3.57%) 1 (3.7%) 0.682
 Volunteer Work 1 (1.82%) 0 (0%) 1 (3.7%)
 Active Job Search/Training 3 (5.45%) 1 (3.57%) 2 (7.41%)
 Unemployed 49 (89.09%) 26 (92.86%) 23 (85.19%)
Independent Living Status n (%)
 Independent Housing 16 (29.09%) 9 (32.14%) 7 (25.93%) 0.74
 Supported Housing 27 (49.09%) 14 (50%) 13 (48.15%)
 Living with Family 12 (21.82%) 5 (17.86%) 7 (25.93%)
Computer Access at Home n (%)
 No 37 (67.27%) 18 (64.29%) 19 (70.37%) 0.847
 Yes 18 (32.73%) 10 (35.71%) 8 (29.63%)

3.1.2. CR clinicians

Questionnaires and qualitative interviews were completed by 6 CR clinicians, all master’s or doctoral level licensed mental health professionals, representing 5 of the 6 clinical sites.

3.2. Feasibility of Hybrid CR

3.2.1. Quantitative results

Of the randomized subjects, 38 completed the intervention (Clinic n=18, Hybrid n=20) prior to service discontinuation due to the pandemic. Differences in weekly training time were not statistically significant between completed individuals in Clinic (42.8±15.9 min) and Hybrid (42.9±16.6 min) CR (p=0.810).

On the clinician questionnaire, five of six CR clinicians reported ease designing homework plans and monitoring homework activity. Ratings were mixed regarding feasibility of offering independent practice as an alternative to twice weekly clinician-led sessions, with 4 clinicians finding it feasible. Half the clinicians perceived Hybrid participants to benefit as much as Clinic participants.

3.2.2. Qualitative results

Four themes emerged from the semi-structured interviews.

Lack of Access to a Computer.

The biggest and most frequently mentioned challenge to implementing Hybrid CR was most clients’ lack of access to a computer outside of the clinic. As one client explained, “not having internet access made it hard for me because I wasn’t able to do the exercise at my house. So I had to travel all the way [to the clinic]. I tried going to the library, but it didn’t work…I tried getting my own laptop, but I couldn’t afford the internet.” Libraries, often discussed as potential community settings for clients to access a computer, were not a viable option due to clients’ reports of long wait times and time limits on computer usage, technical difficulties accessing the CR exercise sites, and issues related to crowding and distractions. Clinicians echoed these numerous barriers.

Lack of Computer Skill.

Most clients had limited experience with computers and reported difficulties logging onto computers, accessing and navigating the internet, logging onto the CR exercise sites, and understanding or remembering steps to do the exercises. Further compounding this challenge was some clients’ hesitation to ask for help because they did not want to “bother” staff. Both clients and staff acknowledged that many clients “had to rely on clinicians” at least to “get me started.” Indeed, clients’ most frequent response to what helped them complete homework was support from clinicians. While clinicians were generally available to help clients launch homework exercises, they sought to limit the intensity of support offered during homework sessions compared to the formal in-clinic group sessions.

Clinic Infrastructure:

While not mentioned as consistently as limited computer access and skills, several clients and clinicians identified clinic technology and physical infrastructure as impeding intervention delivery, sometimes leading to a frustrating therapeutic experience. Aging computers, poor bandwidth, and lack of quiet space compromised the experience in both CR conditions.

Variable Clinician Use of Homework Sheets.

Some clinicians rarely utilized homework sheets, some used them only to assign and track homework, while others went further and used completed sheets to engage clients in discussions of how they felt about the homework. While most clinicians felt the homework sheets were somewhat helpful, overall, they expressed that it was “kind of hard to track” homework, and whether exercises were completed correctly.

3.3. Acceptability of Hybrid CR

3.3.1. Quantitative results

Group assignment and retention:

Of the 55 randomized subjects, 53 (96.4%) attended the first session of the intervention to which they were assigned. The comparison of initial attendance rates between Clinic (96.4%) and Hybrid (96.3%) CR was not statistically significant (p=1.00). Retention rate, adjusted for the 4 participants unable to begin or complete the intervention phase following randomization because of COVID-19 service shut down, was 74.5% (n=38). Retention rates in clinician-led groups were similar between groups (Clinic 72.0%, Hybrid 76.9%) with no statistically significant difference (p=0.755). Within the Hybrid group, clients completed 89.36% of weekly homework assignments, per their homework sheets.

Satisfaction:

Thirty-five CR participants (Clinic n=18, Hybrid n=17) completed satisfaction surveys at treatment end-point. Level of treatment satisfaction on the CSQ-8 was high; there was no significant difference in mean CSQ-8 total score between Clinic (27.71±4.1; range 17-32) and Hybrid (27.12±4.57; range 19-32) groups (p=0.80), also after adjusting for age. Among Hybrid CR participants, over 70% agreed with statements reflecting acceptability (e.g., enjoyed doing the Cognitive Training exercises by myself as much as in the clinic session”), and perceived benefit (e.g., “I believe that doing the Cognitive Training exercises on my own benefited me as much as doing them during clinic sessions”). About half indicated an ability to complete weekly homework assignments but over half had difficulty finding a computer outside of clinic sessions.

3.3.2. Qualitative results

Four themes related to acceptability emerged from semi-structured interviews.

Independence.

While clients, as noted above, reported that using computers, navigating the internet, and completing homework was “challenging,” most viewed the overall experience positively and appreciated the opportunity to learn. They described positive feelings associated with trying to learn to do something on their own and a sense of accomplishment: “It was difficult, but at a time I got going, it was amazing.” Both clients and clinicians emphasized that Hybrid CR conferred a sense of independence: “It gave me some independence” (Client) and “I think it is important to have people try to do things independently…to see how they would do it” (Clinician).

Clinician Support.

With the exception of two clients who wanted more support, clients spoke positively of the support clinicians provided, noting that clinicians “were always present,” “helped a lot,” and were there to “guide me.” Clinicians provided homework reminders, encouragement, and helped brainstorm concrete strategies for homework completion. Clinicians similarly acknowledged the value of supporting clients who completed homework in clinic, indicating it served as instrumental support to clients as well as “helped kind of relax them a little bit.” While clinicians noted Hybrid CR presented a “lighter workload,” it still placed a strain on their time: “it became a little bit more challenging, because we had to find the time to not only just monitor the computer room…but also just to make sure that they understand how to manipulate the computer.”

Social Environment.

Clients differed in the degree to which they valued working alone or engaging with peers. For some, working alone offered more opportunities to think, reflect, and focus on the homework: “I liked that I had some quiet time to think about it because, myself, in class, it might have been too overwhelming because there was a lot of people. So when I was home, I concentrated better” (Client). Others preferred the group setting because it was “more fun” with people around. Clinicians similarly weighed the potential advantages of peer interaction for clients with the value of a quiet space and sense of independence for others.

Perceived Fit/Suitability.

Clinicians believed that Hybrid CR might be best suited for clients who have “access to a computer,” “basic computer skills,” fewer problems “staying focused,” and are “independently motivated.” They also noted Hybrid CR could be beneficial for participants with travel and scheduling challenges.

4. Discussion

Cognitive remediation is an evidence-based practice that is increasingly implemented in clinical settings to treat the pervasive and significant cognitive deficits that contribute to functional decline in schizophrenia. CR practice recommendations include sufficient exposure to cognitive exercises to promote cognitive change (Bowie et al., 2019), thus services typically offer at least two CR sessions a week. Yet, the challenge of attending multiple in person sessions in a week may limit scalability and access to care. It is therefore important to consider alternate CR delivery methods, such as offering participants the opportunity to access training exercises remotely.

Provision of remote CR is not novel- there is a precedent in treatment protocols developed through research in specialized settings with volunteer populations who may be paid for participation and/or provided technology (Fisher et al., 2015). However, methods developed in such settings face tremendous challenges when used in routine clinical practice conditions. For example, significant social disparities in access to technology are increasingly recognized as a barrier to the feasibility of remote access to psychiatric services like CR (Torous and Keshevan, 2020). Indeed, the clinical settings where remote CR has been successfully implemented are private and serve a higher SES clientele (Lynch et al., 2020).

This trial was conducted at NY state operated clinics, using a ‘deployment-focused’ model of intervention development and testing, to speed the transfer of information about CR between the research and community clinic settings. Qualitative and quantitative data were used to consider feasibility, stakeholder acceptability, and the facilitators and barriers to remote CR. Consideration of only the quantitative results would suggest that it is feasible to assign homework and have participants amass practice hours equal to those attending clinician-led sessions. Further, the retention rates and satisfaction ratings were similar in the two conditions, and consistent with the high level of satisfaction previously reported for clinician led CR in similar settings (Soumet-Leman et al., 2018). Only when also considering qualitative data did the full picture of feasibility and acceptability emerge.

In fact, there were several significant challenges to implementing Hybrid CR as designed. With most clients lacking access to a computer outside of the clinic, they needed to complete the homework component in the clinic. Further, limited computer skills led to frustration as clients struggled to access the websites and navigate the cognitive exercises. A weak clinic infrastructure to manage technology exacerbated the frustration; complaints of poor bandwidth and slow computers were common. Clients often relied on reminders from clinicians to do homework and sought clinician assistance to get them started. Of interest, neither clinicians nor clients referenced motivation and/or cognition as barriers to doing homework; technology access and know-how were instead considered paramount. Clinicians’ use of homework sheets was variable, and they found tracking of homework to be onerous. Clients, however, found the homework sheets acceptable, as indicated by the high completion rate.

Despite these barriers and tribulations, clients reported satisfaction with CR regardless of the condition they were in. Positive aspects of doing homework were the participants’ experiences of independence and pride in overcoming challenge. As much as the clinicians’ support and guidance were valued, so was the opportunity for independence. Clinicians appreciated this in their comments about the need to personalize delivery of CR, suggesting that those who require more assistance attend all sessions in person, while those with access to technology and capacity for more independent functioning be given homework.

Interestingly, there was no consensus about the social benefits of group-based CR. Clients differed in the degree to which they valued working alone or engaging with peers. This is congruent with the findings of prior qualitative CR research which indicates that for some, the presence of peers can feel distracting or anxiety provoking, while for many there is a perceived benefit of peer support, for reducing isolation, engaging in shared problem-solving, and promoting a positive learning environment (Bryce et al., 2018; Contreras et al., 2016). These findings further support the importance of personalization, to provide an optimal learning environment according to individuals’ needs.

This study has limitations. The focus is on the feasibility and acceptability of using independent cognitive exercise practice in metropolitan public service settings and the results may therefore not generalize to other services. The sample size is small. These results are particularly relevant to CR services for chronic schizophrenia where the clientele is from a lower SES. Provided that a flexible approach is feasible and acceptable, it will be important to next demonstrate clinical effectiveness.

Taken together, this mixed methods study has implications for current clinical practice (see Table 2). At this time, the widespread availability of functioning technology to administer CR is still a goal, and therefore remote administration of CR in public sector clinics is challenging. Most participants relied on the clinic for technology to do their cognitive exercise practice. However, even in the clinic there were technological challenges, and internet access was insufficient to support completing the target time on computer exercises. Further, our findings and those of prior CR studies indicate that clinician support is integral to a successful experience (Bryce et al., 2018; Contreras et al, 2016). Clients react positively when clinician support and session structure promote self-direction, empowerment and autonomy. Even when access to technology is limited, it is possible to offer a flexible, personalized approach to CR delivery, where appropriate participants do some of the exercise practice independently.

Table 2.

Clinical Practice Implications of the Qualitative Data

Theme Client/ Staff Suggestion Clinical Practice Implication
Limited Computer access Loan tablets/computers that have internet access Limits feasibility to conduct any form of remote CR, homework or attending virtual CR telehealth sessions
Inadequate clinic infrastructure Larger space for CR Regular technology upgrades Group size, client/staff ratio are impacted by space. Technology malfunction causes client/ staff frustration and disengagement
Limited computer skills Provide practice opportunities and support Provide training in computer skills and support navigating CR exercises
Utility of homework sheets Develop additional ways to track homework Clinicians require dedicated time to review homework.
Accomplishment and independence Provide opportunities to learn and support autonomy When level of challenge is appropriate clients react positively to experience of accomplishment and independence.
Need for clinician support Support ranged from technical assistance to coaching, to homework reminders Clinicians play an active role in CR, providing much needed and valued support to navigate the learning experience
Social Environment Participants place differing value on working alone or with peers. Use shared decision making to personalize the CR social environment to needs and values of the participant
Personalized services Remote CR is not for everyone Remote CR works best for those clients with access to technology, basic computer skills, need for flexible scheduling, ability to focus attention and to work independently.

Supplementary Material

1

Highlights.

  • Public health (PH) settings may face challenges providing care remotely on web connected devices.

  • This trial assessed the feasibility of remote cognitive remediation (CR) in a PH setting.

  • Client access to and competency with devices were significant barriers to remotely delivered CR

  • Clinician support was considered integral to a positive experience with remote and in-person CR

  • The study highlights disparities in access to the technology needed to access CR remotely.

Acknowledgement

We thank all the participants and facility administrators who made this study possible. The continued commitment of NY State OMH to address cognitive health in people with serious mental illness is unique and laudable. We thank the ALACRITY OPAL Center team for supporting this investigation.

Funding

Funding was provided by a NIMH center grant, Optimizing and Personalizing Interventions for Schizophrenia Across the Lifespan, 1 P50 MH 115843 – 01

Footnotes

Declaration of Conflicting Interests

None

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Alvarez-Jimenez M, Gleeson JF, Bendall S, Lederman R, Wadley G, Killackey E, McGorry PD, 2012. Internet-based interventions for psychosis: a sneak-peek into the future. Psychiatr. Clin. North. Am 35, 735–747. 10.1016/j.psc.2012.06.011. [DOI] [PubMed] [Google Scholar]
  2. Attkisson CC, Greenfield TK, 2004. The UCSF Client Satisfaction Scales: I. The Client Satisfaction Questionnaire-8. In Maruish M (Ed.), The Use of Psychological Testing for Treatment Planning and Outcome Assessment, Third edition. Lawrence Erlbaum Associates, New Jersey. [Google Scholar]
  3. Bernard HR, 2002. Research Methods in Anthropology: Qualitative and Quantitative Approaches, third ed. AltaMira Press, California. [Google Scholar]
  4. Bowie CR, Bell MD, Fiszdon JM, Johannesen JK, Lindenmayer JP, McGurk SR, Medalia AA, Penadés R, Saperstein AM, Twamley EW, Ueland T, Wykes T, 2019. Cognitive remediation for schizophrenia: An expert working group white paper on core techniques. Schizophr. Res 215, 49–53. 10.1016/j.schres.2019.10.047. [DOI] [PubMed] [Google Scholar]
  5. Bowie CR, Harvey PD, 2006. Cognitive deficits and functional outcome in schizophrenia. Neuropsychiatr. Dis. Treat 2, 531–536. 10.2147/nedt.2006.2.4.531. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bryce S, Warren N, Ponsford J, Rossell S, Lee S, 2018. Understanding the lived experience of cognitive remediation in schizophrenia: A qualitative comparison with an active control. Psychiatr. Rehabil. J 41, 302–311. 10.1037/prj0000309. [DOI] [PubMed] [Google Scholar]
  7. Contreras NA, Lee S, Tan EJ, Castle DJ, Rossell SL, 2016. How is cognitive remediation training perceived by people with schizophrenia? A qualitative study examining personal experiences. J. Ment. Health 5, 260–266. 10.3109/09638237.2016. [DOI] [PubMed] [Google Scholar]
  8. Creswell JW, 2003. Research Design: Qualitative, Quantitative and Mixed Methods Approaches, second ed. Sage Publications, California. [Google Scholar]
  9. Eberly LA, Kallan MJ, Julien HM, Haynes N, Khatana SAM, Nathan AS, Snider C, Chokshi NP, Eneanya ND, Takvorian SU, Anastos-Wallen R, Chaiyachati K, Ambrose M, O’Quinn R, Seigerman M, Goldberg LR, Leri D, Choi K, Gitelman Y, Kolansky DM, Cappola TP, Ferrari VA, Hanson CW, Deleener ME, Adusumalli S, 2020. Patient characteristics associated with telemedicine access for primary and specialty ambulatory care during the COVID-19 pandemic. JAMA Netw. Open 3, e2031640. 10.1001/jamanetworkopen.2020.31640. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Fisher M, Loewy R, Carter C, Lee A, Ragland JD, Niendam T, Schlosser D, Pham L, Miskovich T, Vinogradov S, 2015. Neuroplasticity-based auditory training via laptop computer improves cognition in young individuals with recent onset schizophrenia. Schizophr Bull. 41(1), 250–258. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Gaebel W, Großimlinghaus I, Kerst A, Cohen Y, Hinsche-Böckenholt A, Johnson B, Mucic D, Petrea I, Rössler W, Thornicroft G, Zielasek J, 2016. European Psychiatric Association (EPA) guidance on the quality of eMental health interventions in the treatment of psychotic disorders. Eur. Arch. Psychiatry Clin. Neurosci 266,125–137. 10.1007/S00406-016-0677-6. [DOI] [PubMed] [Google Scholar]
  12. Lynch DA, Medalia A, Saperstein A, 2020. The design, implementation, and acceptability of a telehealth comprehensive recovery service for people with complex psychosis living in NYC during the COVID-19 crisis. Front Psychiatry. 11, 581149. 10.3389/fpsyt.2020.581149. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Medalia A, Erlich M, 2017. Why cognitive health matters. Am. J. Public Health 107, 45–47. 10.2105/AJPH.2016.303544. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Medalia A, Saperstein AM, Erlich MD, Sederer LI, 2019. Cognitive remediation in large systems of psychiatric care. CNS Spectr. 24, 163–173. 10.1017/S1092852918000822. [DOI] [PubMed] [Google Scholar]
  15. Soumet-Leman C, Medalia A, Erlich M, 2018. Acceptability and perceived effectiveness of cognitive remediation in clinical practice. Psychiatr Serv, 69, 493–494. doi: 10.1176/appi.ps.201700308 [DOI] [PubMed] [Google Scholar]
  16. Torous J, Keshavan M, 2020. COVID-19, mobile health and serious mental illness. Schizophr. Res 218, 36–37. 10.1016/j.schres.2020.04.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Ventura J, Wilson SA, Wood RC, Hellemann GS, 2013. Cognitive training at home in schizophrenia is feasible. Schizophr. Res 143, 397–398. 10.1016/j.schres.2012.11.033. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1

RESOURCES