The Acceptability of Remotely Delivered Cognitive Adaptation Training

Abstract

Cognitive Adaptation Training (CAT) is a psychosocial treatment using environmental supports such as signs, checklists, technology, and the organization of belongings to bypass cognitive and motivational impairments for those with serious behavioral health problems. We conducted a survey of 204 members of managed Medicaid in Texas to examine the acceptability of, opinions about and preferences for CAT delivered in-person (CAT) or remotely (R-CAT) where supplies would be mailed and visits would occur via videoconferencing. The telephone survey presented descriptions of CAT and R-CAT in counterbalanced order eliciting general opinions about the treatments, such as (1) whether they would accept the treatments if they were offered the day of the survey at no cost, (2) which treatment was preferred, and (3) the extent to which they agreed or disagreed with a number of statements about components of the treatments. Results indicated that both R-CAT and CAT were acceptable to respondents with overall acceptance rates significantly higher for R-CAT 87% than for CAT (78%). With respect to preferences, 27% and 28% of respondents preferred CAT and R-CAT, respectively, and 41% of respondents preferred both equally. Black respondents more often preferred in-person CAT to other alternatives. Respondents agreed that they needed help, that they were comfortable with technology, and that they believed the programs would help them. The vast majority of qualitative comments about the treatments were positive. Results suggest that it will be important to assess the efficacy and effectiveness of CAT delivered remotely in randomized trials.

Introduction

Cognitive Adaptation Training (CAT) is a psychosocial treatment using environmental supports such as signs, checklists, technology, and the organization of belongings to bypass cognitive and motivational impairments for those with serious behavioral health problems.^1–3 Following a comprehensive assessment of the person’s recovery goals, behavior, cognition, and home environment, customized supports are designed to cue and sequence medication follow-through and adaptive behaviors that target an individual’s recovery goals.⁴ Environmental supports are provided, set up in partnership with the individual, and maintained on weekly visits to the person’s home or work environment. CAT interventions include voice alarms to remind people to shower, exercise, or perform other important tasks and the setting up of pill containers in a place where the medication will most likely be seen and taken at appropriate times. Figure 1 depicts images of CAT interventions targeting medication follow-through.

Fig. 1.

Remote and in-person cognitive adaptation training (CAT) interventions for medication follow-through.

Studies in the United States, Netherlands, and Canada have demonstrated the efficacy of CAT in improving adherence to medication and functional outcomes and reducing symptoms and hospitalizations for those with schizophrenia spectrum disorders. The studies found medium to large effect sizes for CAT versus standard community care.^1–3,5,6 CAT has successfully been used in value-based programs for individuals who have serious mental illnesses with positive results for Medicaid payors, providers, and patients.⁷ In fee-for-service models, CAT is typically reimbursed under codes for rehabilitation, medication management, and skill building. Web-based and in-person training (www.iceebp.com) in CAT is available for mental health providers.

CAT interventions are thought to work in part by bypassing controlled processing and capitalizing on automatic processes.⁸ Dual-process theory⁹ holds that behavior is impacted not only through conscious, “controlled,” processing, but also through nonconscious “automatic” processing.^10,11 “Controlled” processes are under one’s awareness, intention, and conscious control and are relatively inefficient and labor-intensive. Most health interventions target controlled processes. In contrast, CAT and R-CAT target “automatic” processes which occur outside of one’s awareness, control, and intention. Automatic processes are highly efficient, fast, and minimally labor-intensive. The majority of day-to-day human behavior is guided by automatic processes.¹² Use of antecedent control to increase priming (the presence of a cue that activates behavior) and fluency experiences (creating the sense that something is easy by reducing steps and organizing item placement) minimizes the effort needed for behavior change and helps individuals to develop and sustain new adaptive habits that improve adherence and promote better outcomes.¹¹ Focus on automatic rather than controlled processes is particularly important for individuals with SMI who are known to have impairments in problem-solving, remembering, directing attention, and motivation.¹³

In an effort to make CAT more accessible to a wider variety of individuals with serious behavioral health conditions and to reduce costs associated with providing the treatment, we developed a remote version of CAT. While psychotherapies delivered via telehealth are widely accepted and have been shown to be efficacious across multiple populations,¹⁴ CAT is different from psychotherapy. CAT is not primarily a talk therapy, but rather a home-delivered, hands-on, physically active program of establishing and using environmental supports customized to the individual. The extent to which individuals would accept this type of treatment being delivered remotely is unclear.

Reasons to pursue telehealth delivery include that telehealth allows the provision of specialized behavioral healthcare services in areas where there has historically been poor access.¹⁵ Underserved areas include those in remote or rural locations and in marginalized communities with critical shortages of health care professionals.^15,16 Transportation, childcare, and stigma are also everyday barriers for people seeking treatment for behavioral health conditions that may be ameliorated with the use of telehealth service delivery.¹⁶

In an initial pilot study, CAT interventions were applied in a telephone and mail-delivery program called the Helpful Habit Program.¹⁷ The Helpful Habit Program used master’s level nurses and case managers to assist individuals with schizophrenia over the phone, helping them to develop regular habits to take medication, setting reminders to take medication on time, and getting refills regularly. A standard set of environmental supports was mailed to the participant, including pill containers, a calendar with stickers to track medication ingestion and cue provider appointments, and occasional coin tokens as reinforcement for participation in the program. Results indicated that automaticity and habit-formation increased during treatment. Moreover, the Helpful Habit Program was found to improve adherence to medication based upon in-home pill counts conducted by independent researchers¹⁷

Many investigators are working to develop and test telehealth-delivered versions of successful treatments.^18,19 This push to move treatment online has only increased because of the recent pandemic. However, it is important to determine what the demand may be for telehealth versions of specific psychosocial treatments before spending substantial resources on their adaptation to a telemedicine model and on studies examining their efficacy. The acceptability of a remotely delivered version of CAT (R-CAT) has never been examined. It is unclear what percentage of those with behavioral health conditions would choose to participate in CAT using the traditional home-based model vs a remotely delivered option and what barriers or concerns may be for these approaches.

To examine whether a remote version of the CAT program would be acceptable to Medicaid members with serious behavioral health challenges, we used a mixed-methods telephone survey to collect quantitative and qualitative data from 204 members of a large managed Medicaid program in Texas. We examined the following research questions.

1. What would Medicaid recipients think about CAT and Remote CAT, what aspects of telemedicine or in-home delivered treatment would be appealing and which would present barriers?

2. Would similar numbers of managed care members accept Remote CAT and CAT if these programs were available immediately?

3. To what extent would in-person or remotely delivered CAT (both or neither) be preferred by Medicaid recipients?

4. To what extent would Medicaid recipients agree or disagree with statements describing aspects of in-person and remotely delivered CAT, including the role played by the recent pandemic in individuals’ preferences for treatment?

Methods

Survey Development

A survey was developed based on exit interviews used in previous studies of CAT^2,3 and the Helpful Habit Program.¹⁷ The survey was piloted with ten volunteers with behavioral health conditions (serious mental illness). An interview was conducted with these participants as they were completing the survey to examine how respondents understood and interpreted the material and the questions. Survey content was simplified based on their input.

Participants

A large Medicaid managed care plan in Texas identified 2569 individuals between 18 and 65 with serious behavioral health conditions including mood disorders, severe anxiety disorders, and psychotic disorders. Potential participants had at least 1 emergency department visit or hospital stay in the past 12 months for behavioral health reasons and documented medication adherence lower than 80% for at least 1 of the previous quarters during the past year. Participants were contacted by phone by UT research staff from November 24, 2020 to September 27, 2021, and asked to participate in a research survey. For context, the study started 7 months after Texas reopened all businesses that were closed during lockdown due to the Coronavirus Disease 2019 (COVID-19) pandemic. Participants who completed the interview gave verbal consent for participation, which was approved by an Institutional Review Board. The procedures were consistent with internationally recognized standards for the ethical conduct of human research.

Procedure

Medicaid recipients meeting the criteria were called by trained research assistants and invited to participate in a research survey about their interest in and evaluation of specific types of services. The survey was administered using a web-based, semi-structured script to guide the interviewer and was audio recorded with the verbal consent of the participant. The researcher gathered basic demographic information, including age, gender, living situation, and method of transportation for attending prescriber appointments. Next, the researcher described 2 treatments in counterbalanced order; in-person CAT (called In-person Wellness Support) or R-CAT (called Remote Wellness Support). In-person CAT was described as a wellness program using environmental supports delivered to the home by the provider on weekly visits and text messages between visits to assist individuals with taking medication and attending medical appointments, coping with stress, anger and sadness, and working toward recovery goals. Remote CAT was described as a wellness program with the same goals. In remote delivery, environmental supports would be mailed to the participants. We explained that in R-CAT, reasons for specific supports, where to place them, and how to use them would be discussed during weekly video visits.

For both treatments, the types of supports participants might receive and their behavioral targets were described in the same manner. The order of presentation of the treatments was randomized to allow us to examine any effects of order on acceptance of treatments. Questions were asked throughout treatment descriptions to ensure understanding on the part of the participant. Following a description of each service, the researcher asked 3 open-ended questions regarding the participant’s general impressions of the treatment, what the respondent liked and what the respondent disliked about the treatment. Following these open-ended questions, participants were asked whether they would accept and participate in the service if it were available that day at no cost. After both services were described and discussed, participants were then asked to state a preference for either 1, both, or neither treatment. Finally, participants were asked to state the extent to which they agreed or disagreed with a series of statements describing characteristics of CAT and R-CAT. Comments were scored on a series of Likert scales from completely disagree to completely agree. Participants were sent a $10 gift card to compensate them for their time participating in the survey. The survey took approximately 15–20 minutes to complete.

Data Analysis

Qualitative data were audio recorded during the survey, transcribed by the research team, and organized thematically using an open coding scheme to develop themes using an interactive process.^20–22 Data were independently coded by 4 reviewers (DV, FL, CW, and CK). This open coding framework was chosen because it allows the coding structure to be developed without imposing the coder’s beliefs about the phenomena under study into the coding process. Coders assigned meaning to snippets of qualitative data (statements from transcripts) without any prior constraints. A constant comparative method was employed while coding transcripts. To determine the appropriateness of a code, all coders compared their code for each segment to determine whether the segment reflected the same or different concepts. This constant comparison encouraged refinement in the coding structure and allowed for the synthesis of some codes and the elimination of others. Throughout the process, as the coding structure emerged, the coders became aware of patterns of coding that could be grouped to form themes within the data. Codes were tallied and stratified, with the most frequently emerging codes presented as major themes. Saturation was reached at the point at which no new codes were emerging and this was defined as the point at which a complete range of theoretical constructs was fully represented by the data. Analysis of answers to open-ended questions generated 2 major themes; positive comments and negative comments each with subcategories that are listed and defined in Tables 1 and 2 for R-CAT and CAT respectively.

Table 1.

Coding Scheme–Remote-Cognitive Adaptation Training (R-CAT)

Themes of Qualitative Data	R-CAT
	n	Statements
Positive
General	215	“It sounds like a good program, it’d be very helpful”
Support/treatment supplies	65	“I liked all of it. I would be interested in supplies, creams, ointments, things of that nature.”
Remote platform	77	“you can take advantage of timing and the technology…it’s very convenient.”
Coping	27	“They would support me in helping me with my sadness and stress. Trying to help me with ways to manage my stress.”
General assistance	27	“I think it would be perfect for me cause I can’t do anything on my own”
Medication assistance	25	“The fact could help me by reminding me to take my meds. Help me out with reminders to get things done”
Convenience/accessibility	18	“Well that is it good they send it by mail and the video, the instructions by video”
Negative
Lack of in-person contact	8	“I would think its not that good the fact that you don’t get to meet face to face”
Technology issues	8	“… it may be hard for other members due to difficulty of technology.”
Lack of availability	6	“If I receive a call, i might be busy doing something else.”

Table 2.

Coding Scheme–Cognitive Adaptation Training (CAT)

Themes of Qualitative Data	CAT
	n	Statements
Positive
General	216	“I think it’s great… It seems like a useful program.”
Support/treatment supplies	91	“As far as having an alarm for the meds and so on and so forth…I think that is great.”
Medication assistance	60	“Having an extra person would help me remember to take my medications”
Coping	50	“I like the fact they can give tools to deal with stress. Coping skills, and weekly visits”
Bypassing cognitive problems	34	“That they do reminder signs I think it is a good program helps you stay together.”
Negative
Time commitment	18	“only thing is the time commitment and having to interact with somebody”
Lack of availability	14	“The time they would want to do has to work into my schedule”
In-person contact not desired	10	“Sounds great except that I am not really comfortable having someone come to my home every week”

With respect to quantitative data, we present descriptive statistics regarding sample characteristics. Proportions were examined for those accepting treatment when it was offered first versus offered second. We used binomial regression for mixed models, (1) order of offer (first or second, type of treatment (CAT or R-CAT), and (2) acceptance of the offer (yes or no) to examine differences in proportions. We were interested in the acceptance rate of CAT vs R-CAT when only one treatment was offered as an ecologically valid way of looking at acceptance rates in a real-world situation where there is often only one option offered. We looked at overall stated acceptance rates for each treatment once both were described and whether the order in which the treatments were offered impacted acceptance. We also examined whether time (7 months through 17 months after lockdown in Texas ended) influenced treatment acceptance. This analysis used the nonparametric Mann-Kendall statistic S, calculated by forming all pair wise combinations of time points, counting the number of times the later observation is higher than the earlier observation, and subtracting the number of times the later observation is lower.²³ Finally, after both treatment options were presented, we examined the stated preference for 1, both, or neither treatment (CAT, R-CAT, both, and no treatment desired) using Logistic Regression. We examined preference by demographic and clinical variables using X² analysis for categorical variables and Multinomial regression for the continuous variable age. The percentages of respondents endorsing various statements describing aspects of each treatment (eg, usefulness of services and supplies provided, comfort with technology, home visits, texting, and the impact of COVID-19) are presented.

Results

Sample Characteristics

Figure 2 is a consort diagram indicating the number of individuals researchers attempted to contact, the number contacted and the number participating in the survey. Out of a total of 2569 attempts, the research team successfully contacted 368 (14%) medicaid recipients and obtained responses from 204 individuals (55%). Issues with contact included phones being out of service, numbers being incorrect, or individuals not answering or responding to messages left. Some questions were not answered by all participants so numbers for individual queries varied slightly. Interviewed Medicaid recipients came from 38 counties in Texas. The average age of respondents was 43.24 and ranged from 19 to 85. 39% of respondents were male (n = 80). 65% were White (n = 130), 18% were African American (n = 37), 7% were mixed race (n = 15) and 8% chose not to reveal their race (n = 16). Over half of the sample (52%, n = 106) were Hispanic. The majority of respondents reported living with family or other adults (78%, n = 158), 25% of them reported living in households with other adults and child(ren). Family or friends(43.3%) are the most common providers of transportation for participants to attend their doctor’s appointments. Median income for participant zip codes based on the 2020 US Census was $52,455. See Tables 3 and 4 for complete demographics and psychosocial attributes of the sample.

Fig. 2.

Consort Diagram.

Table 3.

Demographic Characteristics of the Sample

Demographic Characteristic	N	%
Sex
Female	121	59.6
Male	80	39.4
Other or nonbinary	2	1
Ethnicity
Hispanic	106	52.2
Non-Hispanic	89	43.8
Unknown	8	3.9
Race
American Indian/Alaska Native	3	1.5
Asian	N/A	N/A
Native Hawaiian or other Pacific Islander	N/A	N/A
Black or African American	37	18.2
White	130	64
More than one race	15	7.4
Unknown or not reported	16	7.9

Table 4.

Psychosocial Characteristics of Participants

Psychosocial Characteristic	n	%
DSM-5 diagnosis
Depressive disorders	42	25.5
Bipolar disorders	45	27.3
Schizophrenia spectrum disorders	72	43.6
Anxiety disorders	3	1.0
Other	3	1.0
Access to device(computer/smart phone/tablet)	162	79.8
Access to internet	164	80.8
Responsible for taking their own medications	102	91.9
Receiving ACT Services	14	12.6
Missed Medication at least 2x in the past 3 weeks	69	62.1

Qualitative Data: Statements From Open-Ended Questions

The vast majority of comments about both treatments were positive. Out of 449 and 493 coded segments, 95 % and 91% were positive for R-CAT and CAT respectively. For R-CAT, individuals liked the treatment supplies, the convenience of the remote platform, that the treatments helped them with coping, general assistance, medication assistance, and overall accessibility. The most common specific positive statement regarded liking the support for health and wellness provided during the treatment. Negative comments included those reflecting a preference for in-person contact and concerns related to using technology or not having the technology available.

For CAT, individuals indicated that they would appreciate the treatment supplies (environmental supports), medication assistance, coping support, and assistance in bypassing cognitive problems. Negative comments included concerns about the time commitment, and lack of availability and discomfort about in-person contact both in general and in the context of the pandemic.

Quantitative Data

Acceptability of CAT and R-CAT and Order of Presentation.

Some of the respondents did not answer all questions so n = 192 for the quantitative results below. Results of binomial regression for mixed models examining acceptance of the offered treatment (yes or no if the treatment was available to you today would you accept it) as a function of the order of offer (first or second) and type of treatment (CAT or R-CAT) revealed that both main effects were significant. Overall, R-CAT was significantly more likely to be accepted than CAT (F (1,189) = 5.36, P = .022), with 87% (167/192) accepting the telemedicine delivered R-CAT and 78% (150/192) accepting in-home CAT. These results indicate that the vast majority of respondents stated that they would accept CAT and R-CAT if they were offered to members on the day of the call. Both treatments were more likely to be accepted if they were offered first (R-CAT 92% (88/96), CAT 84% (81/96) rather than second (R-CAT-82% (79/96), CAT 72%(69/96), F (1,190) = 8.17, P = .005).”). Mann-Kendall tests did not indicate significant time trends for acceptance within R-CAT or CAT or a difference between treatments accepted with respect to time. (all P’s > .13).

Specific Aspects of Treatment.

We elicited participants’ responses regarding their need for additional services as well as their opinions about different characteristics of the treatments. We asked how strongly they agreed that additional services are needed, what they thought about receiving text messages and calls between visits, their comfort level with technology and video/telephone visits, concern about their data plan, the overall comfort level of home visits as well as comfort level of home visits during the pandemic, the desire for assistance in coping with emotional distress, and the perceived need for treatment supplies (Figure 3). The vast majority (71%) agreed that they needed the additional services or help in addition to their current standard health care program. Over 80% (183/195) stated that they would find text messages and phone calls helpful as a part of treatment. A majority (125/195, 64%) stated that they were comfortable with technology, while 30% (51/195) indicated some difficulty. Additionally, 40%(79/195) indicated that they would worry about their data plan using telemedicine. Over 93% (181/195) agreed that skills to help them cope with anger, stress, and sadness would be beneficial. In addition, over 95% (186/195) agreed that the supplies and assistive supports provided in CAT and R-CAT would be helpful.

Fig. 3.

Participant’s views on treatment aspects of cognitive adaptation training (CAT) delivered in-person and remotely.

With respect to comfort with home visits, while almost half of the participants endorsed they would be afraid for someone to come to their home due to the pandemic, nearly 44% (86/195) disagreed with that statement. Over 75% (147/195) of participants would be comfortable with home visits “if COVID-19 were over.” Twenty percent would be uncomfortable with home visits regardless of pandemic status. With regard to video or telephone visits, over 66% (130/195) indicated that they were comfortable with telehealth.

Treatment Preference.

After hearing about both treatments, respondents were asked whether they preferred CAT, R-CAT, both equally, or neither. Twenty-eight percent (n = 55/195) preferred R-CAT, 27% preferred CAT(52/195), 41% (79/195) preferred both, and less than 5% (9/195) preferred neither. These differences were statistically significant (X² = 52.2, df = 3, P = .0001). There was a significant relationship between race and treatment preference (X2(4) = 9.51 P = .05) with blacks preferring in-person CAT more than other options. There was a significant difference in preference by diagnosis (X2 = 13.03, df = 4, P = .011), with those with a bipolar diagnosis more likely to prefer both treatment options and less likely to prefer only R-CAT than other diagnoses. No other significant relationships between demographic/clinical variables and preference were found.

Discussion

Results of the study suggest that both CAT and R-CAT are acceptable to a managed Medicaid population in Texas. It appears that R-CAT treatment was preferred by more respondents than in-person CAT. This may have had to do with that the survey was conducted following an end to lockdown during the COVID-19 pandemic. Order effects suggesting that the first treatment presented was more likely to be accepted may have to do with fatigue later in the interview.

Once both treatments had been described and respondents were asked for a preference, there were no statistically significant differences between the percentage preferring R-CAT and the percentage preferring CAT in the sample as a whole. While the majority of respondents preferred these options equally, individuals of Black or African-American race preferred in-person CAT to a greater extent than other options. Understanding the racial and clinical differences in treatment preference can help to ensure that preferred treatments are available to Medicaid recipients. Overall, results suggest that more than 95% of participating Medicare recipients would prefer either one or both of these treatments

Results suggest that telehealth delivery is likely to be an important direction for CAT treatment. Remotely delivered CAT requires less travel time for providers increasing the number of individuals that can be seen by each provider. A remote version of CAT is important for rural areas where long travel distances make home visits less feasible. Rural and remote areas have fewer mental health and substance use treatment services than urban communities. According to the US Health Resources and Services Administration,²⁴ almost two-thirds of mental health professional shortage areas are found in rural settings. In Texas, 85% of the land mass is rural and 185 out of the 254 Texas counties containing over 3 million people have no psychiatrist. This represents 15% of the population of Texas. A large-scale efficacy study of R-CAT would be important to pursue for this and similar populations.

Although the enthusiasm for R-CAT may have partially been driven by the pandemic, results indicate that most participants feel comfortable with technology. Having home visits during the pandemic was of concern to 40% of individuals interviewed, suggesting modification of in-home CAT would be necessary if similar situations occur in the future. Accommodations such as having visits outside and using a CAT/R-CAT hybrid may be workable. This hybrid approach was something we did clinically at our site and in an ongoing effectiveness trial of in-person CAT during the pandemic (MH117101 CAT: Effectiveness in real-world settings and Mechanisms of Action [CAT-EM]).

It was perhaps surprising that despite as many as 30% of participants identifying challenges with technology, a vast majority of individuals would agree to televisits. However, assistance with technology would likely need to be provided for those who have difficulty. The person providing technology support would not necessarily need to be a trained treatment provider, but could be an administrative or help desk staff member that could assist the individual in getting the appropriate software or clicking on the correct link. It is also noteworthy that more than 70% of respondents in managed care Medicaid disagreed that they did not need additional services. In examining the qualitative data, aspects of the treatments that individuals liked included the modality (home or video), the support received, assistance with coping, help with medication, and specific assistance to bypass problems in memory and organization. Individuals’ concerns involved potentially not being available for or not needing proposed visits.

Limitations of the study include that only 1 Medicaid managed care organization participated, and the sample was restricted to Texas. Moreover, while 55% of individuals actually reached agreed to participate, many calls were unanswered, not working or incorrect raising questions about generalizability. However, results are consistent with other studies indicating that remote treatments are acceptable. In addition, the acceptance of and preferences for treatment were hypothetical for respondents who were not offered these treatments through managed Medicaid. Despite these limitations, this is the first large-scale survey of Medicaid recipients’ views on CAT delivered in-person or remotely.

Future research should focus on the efficacy and effectiveness of R-CAT (a current study is ongoing), cost analyses, methods for matching individuals to the most suitable modality, implementation in managed care and other types of health maintenance organizations, and investigation of efficacy across additional populations with chronic illness (eg, diabetes, heart disease, chronic obstructive pulmonary disease, etc). Results suggest that R-CAT would be an acceptable treatment for a majority of individuals.

Funding

This work was supported by grant R56 MH123797-01 from the National Institute of Mental Health.

Conflict of Interest

Dr. Velligan has speaker bureau agreement with Janssen; consulting agreements with Janssen, Alkermes, Lundbeck and Otsuka; Advisory Board for Indivior, Merck and Lundbeck. The other authors have declared that there are no conflicts of interest in relation to the subject of this study.