Mobile Assessment and Treatment for Schizophrenia (MATS): A Pilot Trial of An Interactive Text-Messaging Intervention for Medication Adherence, Socialization, and Auditory Hallucinations

Abstract

Mobile Assessment and Treatment for Schizophrenia (MATS) employs ambulatory monitoring methods and cognitive behavioral therapy interventions to assess and improve outcomes in consumers with schizophrenia through mobile phone text messaging. Three MATS interventions were developed to target medication adherence, socialization, and auditory hallucinations. Participants received up to 840 text messages over a 12-week intervention period. Fifty-five consumers with schizophrenia or schizoaffective disorder were enrolled, but 13 consumers with more severe negative symptoms, lower functioning, and lower premorbid IQ did not complete the intervention, despite repeated prompting and training. For completers, the average valid response rate for 216 outcome assessment questions over the 12-week period was 86%, and 86% of phones were returned undamaged. Medication adherence improved significantly, but only for individuals who were living independently. Number of social interactions increased significantly and a significant reduction in severity of hallucinations was found. In addition, the probability of endorsing attitudes that could interfere with improvement in these outcomes was also significantly reduced in MATS. Lab-based assessments of more general symptoms and functioning did not change significantly. This pilot study demonstrated that low-intensity text-messaging interventions like MATS are feasible and effective interventions to improve several important outcomes, especially for higher functioning consumers with schizophrenia.

Introduction

Ecological momentary assessment (EMA), also called the experience sampling method (ESM), is an ambulatory data collection technique that allows the real-time in vivo assessment of behaviors, moods, thoughts, symptoms, and other daily experiences.^1–3 Modern EMA takes advantage of mobile devices such as personal digital assistants and smart phones to signal participants several times throughout the day to respond to questionnaires about their daily lives. EMA provides a temporal accounting of daily experiences that can reveal microprocesses within individuals, such as dynamic relationships between one’s immediate state and subsequent symptoms or impairment in the context of one’s natural environment. Prior research has demonstrated the feasibility and validity of mobile-device EMA methods in consumers with schizophrenia.^4–6 Using mobile-device EMA methods in this population, studies have identified important real-time associations between greater anxiety, stress, and arousal and greater severity of psychotic symptoms,^3,6–11 negative mood states, and substance abuse¹² as well as associations between greater positive affect and better social functioning.¹³

Mobile devices have also been used to deliver interventions for a variety of health and mental health problems,^14,15 but there has been little prior application of mobile interventions in serious mental illness (SMI). Mobile technologies that incorporate EMA methods have great potential for real-time real-world interventions for schizophrenia.¹⁶ Mobile devices can be used to deliver services outside the clinic as well as strengthen clinic-based services. For example, daily EMA sampling (eg, of symptoms, moods, medication adherence) via mobile devices could be used to prompt coping responses in consumers reporting increased warning signs or to alert providers to trends in their consumers’ symptoms (eg, escalating severity of hallucinations) to prompt contact and intervention. Mobile devices could also be used to prompt health-promoting behaviors (eg, medication adherence, diet, exercise) or strengthen interventions by prompting in vivo skills practice (homework), which could reduce the intensity of interventions (eg, number or duration of face-to-face sessions). All of these potential uses of mobile-device interventions could also help reduce service costs.

We are aware of only 1 prior study that employed mobile technology to provide an intervention beyond self-monitoring for consumers with SMI.¹⁷ Spaniel and colleagues¹⁷ described a 1-year open trial with 45 consumers and 39 family members in which participants were sent a weekly 10-item questionnaire about early warning signs of schizophrenia via text messaging on mobile devices. A threshold number of warning signs triggered an alert to their psychiatrist. The study reported a 60% reduction in the number of hospitalizations relative to the year prior to ambulatory monitoring.¹⁷

Given the limited research on mobile-device interventions in SMI, we conducted a pilot trial of a novel mobile phone text message intervention called Mobile Assessment and Treatment for Schizophrenia (MATS).¹⁸ The MATS project is the product of collaboration between academic clinical researchers, programming/technology experts, and various stakeholders (ie, program officials, providers, and consumers) from Assertive Community Treatment (ACT) programs in San Diego County. MATS provides mobile interventions to improve high-priority outcomes for stakeholders. Through focus groups, stakeholders recommended 3 treatment targets: Medication adherence, socialization, and auditory hallucinations. The MATS intervention prompted health-promoting behaviors and used cognitive behavioral therapy (CBT) techniques.^19,20 Participants were given mobile phones and received 3 sets of interactive text-exchanges each of 6 days per week for 12 weeks. It was hypothesized that significant improvements would be found for each treatment target over the 12-week MATS intervention.

Methods

Participants

Community-dwelling individuals with schizophrenia or schizoaffective disorder (N = 55) over the age of 18 were enrolled (no other inclusion/exclusion criteria). Participants were recruited from outpatient residential and treatment settings (N = 14 from ACT teams) in the Veterans Affairs San Diego Healthcare System and the San Diego County Mental Health System from 2007 to 2010. Participants were not excluded for active substance use disorders, but one participant who reported using methamphetamine on the baseline assessment day was excluded for inability to complete assessments. Diagnoses of schizophrenia included 32 paranoid, 10 undifferentiated, 2 disorganized, and 11 schizoaffective disorders. At baseline, 37 participants were prescribed at least one atypical antipsychotic medication, 23 at least one typical antipsychotic, 31 both typical and atypical antipsychotics, and 1 individual was not prescribed any antipsychotic medications. Twenty-four participants were also prescribed antidepressant medications, and 29 reported using mood-stabilizers.

Procedures

All study procedures were approved by the institutional review board of the University of California, San Diego. Text messages focusing on 3 intervention domains (medication adherence, socialization, and auditory hallucinations) were sent to participants daily from Monday through Saturday for a 12-week period. Following informed consent and baseline assessments, participants were given a basic Motorola cellular phone (model V195 or W490) with full access to domestic calling, calendar, alarm, and gaming functions. Participants were trained to send and receive text messages and typically required one additional in-home visit (about 10 min) for retraining during the initial days of the trial. A PowerPoint presentation was used to describe text-messaging procedures in an initial 30-minute training session in the lab, and participants were guided through approximately 3 practice trials with each of the 3 treatment targets (medications, socializing, and voices). Information about how to use and charge the phone was also provided (eg, how to make outgoing calls, set an alarm, etc.), and participants were given a copy of the PowerPoint presentation to take home. The primary outcome measures were self-reported medication adherence, number of social interactions, and severity of auditory hallucinations obtained through daily ambulatory monitoring (responses to text message questions) over the 12-week intervention period. Secondary outcomes were also assessed using a battery of laboratory-based symptom and functioning measures administered at baseline and the end of the 12-week intervention. Participants received $35 for completing assessment visits and a $20 gift card (Subway or Starbucks) incentive every 2 weeks for completing mobile assessments. All text message responses for each participant were viewable on a secure website, so staff could contact individuals to remind them to answer the text messages or to provide technical support.

Mobile Assessment and Treatment for Schizophrenia

Three sets of 4 text messages (12 total) were sent to participants each day, Monday through Saturday, with each message set targeting 1 of the 3 intervention domains: medication adherence, socialization, or auditory hallucinations. All 3 interventions were delivered in random order each day in the morning, afternoon, and evening. The number and frequency of text messages were based, in part, on focus group feedback and the amount of time needed to send and respond to text messages. Our goal was to adequately sample daily behavior and intervene frequently enough to have a potential impact on the target outcomes, without overburdening individuals with time-consuming messages. Focus group feedback was that more frequent messaging, especially for a 3-month period might be too much. To accommodate daily routines, participants were allowed to choose the specific times they would receive messages within a 2-hour window. Each time a text message was received, the phone generated an auditory signal and/or a vibration that prompted participants to read the message.

The text-messaging interventions incorporated CBT techniques.^19,20 Thoughts about medications, socializing, and voices were elicited (eg, “Do you think your voices are powerful?”), and the next messages encouraged participants to question unhelpful beliefs (eg, “Maybe your voices can’t really do what they say”) and try a behavioral experiment (eg, “Try ignoring them and see what happens”). Evidence used to challenge unhelpful beliefs included personalized information provided by the participant. During a baseline interview, the rater who performed the research assessments asked a standard set of questions to elicit personalized information to be used in the text messages. Participants were asked to report at least one benefit of medications and socializing, and a coping strategy that reduced the frequency or distress related to voices (eg, “What is a benefit or something good about taking your medication?”; “What is something you like to do for fun with other people?”; “What do you do to help cope with voices?”). This information was used to create personalized thought-challenging messages. For medication adherence, the messages were “But you said taking meds helped you (personal reported benefit from taking medications).” For social functioning, the message was, “But you said that (personal enjoyable social activity) was fun.” For auditory hallucinations, the message was “You said that (personalized effective coping strategy) helps.” The individualized text messages were entered into a secure website, which could only be accessed by research staff. Once the content was entered, the interventions were sent automatically by a remote secure server.

The flow of messaging, including branching according to participant responses, is shown for each intervention domain in figures 1–3. The 4 types of messages sent for each intervention domain were as follows:

Fig. 1.

Sequence of text messages for the medication adherence intervention with branching according to consumer responses. Participants provided single key responses to multiple-choice questions. Responses to the first (“outcome assessment”) question are shown in figure 4. Responses to the second (“current cognitions”) question are shown here as the percentage of total valid responses across all daily assessments.

Fig. 2.

Sequence of text messages for the socialization intervention with branching according to consumer responses. Participants provided single key responses to multiple-choice questions. Responses to the first (“outcome assessment”) question are shown in figure 5. Responses to the second (“current cognitions”) question are shown here as the percentage of total valid responses across all daily assessments.

Fig. 3.

Sequence of text messages for the auditory hallucinations intervention with branching according to consumer responses. Participants provided single key responses to multiple-choice questions. Responses to the first (“outcome assessment”) question are shown in figure 6. Responses to the second (“current cognitions”) question are shown here as the percentage of total valid responses across all daily assessments.

1. A multiple-choice outcome assessment question (eg, “Have you been bothered by voices? … No Voices, A Little, Moderate, and Extreme”), which required a reply using a single key press from the phone pad corresponding to one of the choices.

2. A multiple-choice current cognitions assessment question (eg, “Do you think your voices are … Uncontrollable, All Knowing, Powerful, and Other”), which required a reply using a single key press from the phone pad corresponding to one of the choices.

3. A thought-challenging message for unhelpful beliefs, which was dependent on their response to question 2 (eg, if voices were thought to be “All Knowing,” a possible message 3 would be, “Maybe voices make mistakes too”).

4. A behavioral coping or behavioral experiment suggestion, which was also dependent on their response to question 2 (eg, for “All Knowing,” a possible message 4 would be, “Ask a friend if they know information that contradicts the voices”).

If the participant did not reply to the first or second question, the next messages in the sequence were not sent. If the participant reported positive outcomes (eg, they were taking medications or voices were absent or they socialized with 4 or more people), they still received a second question that required a response (eg, “What’s helping?”) plus the 2 additional messages suggesting coping strategies. Therefore, there was no reduced hassle or burden for reporting a positive outcome. At maximum compliance, each participant would have received 840 total messages over the 12-week intervention period (4 messages 3 times per day = 12 per day × 6 days per week (10 messages on Friday) = 70 per week × 12 weeks = 840 total messages, with 420 requiring responses). Every Friday, a message was sent to the participants asking them how helpful they found the messages to be that week (1 = Not at all; 2 = Somewhat; 3 = Moderately; and 4 = Very).

Measures

The primary outcome measure for each intervention domain was the daily ambulatory monitoring outcome assessment question for that domain (question 1 described above). Secondary outcomes were also assessed using a battery of laboratory-based symptom and functioning measures. The battery included the Positive and Negative Syndrome Scale (PANSS),²¹ Beck Depression Inventory—2nd Edition (BDI-II),²² Independent Living Skills Survey (ILSS),²³ and American National Adult Reading Test (ANART).²⁴ Interrater reliability was .88 for PANSS total.

Statistical Analyses

Hierarchical generalized linear modeling (HGLM) was used to analyze the text message outcome measures. All models used a multinomial sampling model with a multinomial logit link function. Models were estimated using time (in days) as the lone predictor, except for the model for medication adherence, which also included living situation as a predictor. Given that individuals in supported living environments are often aided by staff in taking their prescribed medications, a dichotomous variable that indicated whether the consumer lived independently or in an assisted living facility (independent living = 0.5; assisted living = −0.5) was added as a predictor of both the intercept and growth parameters for the medication adherence outcome model. The test of the effect of day on outcome in these models examines whether there is an association between an additional day of treatment and the log-odds of being in one response category relative to the reference category.

Paired samples t tests were used to test for differences between pretreatment and posttreatment on the secondary outcome lab-based measures. To examine potential differences between the sample of consumers used in the analyses and the consumers who withdrew, independent samples t tests were used to test for baseline differences on the lab-based measures. HLM v6.06 was used for the HGLMs; SPSS v11.5 was used for all other statistical analyses; and MATLAB r12.1 was used to produce the HGLM figures.

Results

Of the 55 participants enrolled, 13 were defined as noncompleters because they did not send any valid messages or stopped sending valid messages within 2 weeks, despite repeated trainings and reminders. Independent samples t tests showed several baseline differences between completers and noncompleters (table 1). Noncompleters had lower self-reported living skills (ILSS), more severe negative symptoms (PANSS negative), and lower estimated premorbid verbal IQ (ANART) than completers. No differences in positive symptoms (PANSS positive), depression (BDI-II), age, or education were found between groups. The 42 remaining active participants had a mean age of 48.7 years (SD = 9.1), mean of 12.3 years of education (SD = 1.3) and 69% were male, 74% Caucasian, 7% African American, and 10% Hispanic. Fifty-seven percent of participants resided in assisted living facilities (board and care).

Table 1.

Characteristics of Protocol Completers and Noncompleters

	Completer (n = 42)		Noncompleter (n = 13)		Statistics
Clinical Measure	M	SD	M	SD	t	df	P	d
ILSS 10 domain average	0.682	0.105	0.616	0.102	1.99	51	.052	0.65
PANSS total	63.9	18.2	69.3	19.7	0.92	53	.364	0.30
PANSS positive total	17.8	6.4	16.2	6.1	0.78	53	.437	−0.25
PANSS negative total	15.2	6.3	20.7	8.1	2.56	53	.013	0.83
BDI-II total	15.7	12.6	15.8	10.9	0.03	53	.979	−0.01
ANART IQ estimate	103.7	8.6	98.2	7.8	2.04	52	.046	0.66
Age (y)	48.7	9.1	48.9	7.9	0.07	53	.947	−0.02
Education (y)	12.4	1.3	11.8	0.7	1.57	53	.123	0.51

Note: ILSS, Independent Living Skills Survey; PANSS, Positive and Negative Syndrome Scale; BDI, Beck Depression Inventory; ANART, American National Adult Reading Test.

For the 42 completers, the valid response rate over the 12-week intervention period for question 1 for each intervention domain was M = 86%, Median = 93%, SD = 19% for medication adherence; M = 83%, Median = 88%, SD = 19% for socialization; and M = 86%, Median = 94%, SD = 19% for the auditory hallucination intervention. There were 2 possible questions for question 2, depending on how participants responded to question 1 (see figures 1–3). The valid response rates for these 2 questions for each intervention domain were M = 85% and 85%, Median = 94% and 98%, SD = 21% and 30% for medication adherence; M = 78% and 85%, Median = 88% and 90%, SD = 27% and 18% for socialization; and M = 85% and 84%, Median = 94% and 98%, SD = 21% and 30% for the auditory hallucination intervention.

The majority of phones (86%) were returned intact. One phone was never returned, as the participant moved out of state without notice, and 5 phones were damaged or malfunctioned (eg, exposure to water; cracked screen). Participant responses to the question asked each Friday, “How helpful were the text messages this week?” (1 = Not at all, 2 = Somewhat, 3 = Moderately, 4 = Very; Mean = 3.15, Median = 3.42, SD = 0.84), suggested that participants found the MATS intervention to be moderately to very helpful. In addition, relative to the “not at all helpful” response, the odds of reporting all other response categories increased significantly with an additional day (“somewhat,” OR = 1.03, t = 2.33, P = .025; “moderately,” OR = 1.04, t = 3.35, P = .002; “very helpful,” OR = 1.04, t = 2.77, P = .009), suggesting greater experience with the intervention increased the likelihood of reporting the intervention was helpful.

Results from the HGLM analysis for the first outcome assessment question from each intervention are presented in table 2. In the model for medication adherence, the effect of an additional day of treatment and the living situation by time interaction were significantly negatively associated with the log-odds of forgetting to take medication, relative to taking medication as prescribed. This suggested that an additional day of treatment aided consumers with remembering to take their medication, and consumers living independently benefited to a greater extent than those in assisted living situations. Figure 4 depicts the probability of endorsing each answer to the medication adherence outcome question each day over the 12-week treatment period. Medication adherence was reported to be high at initial assessment and remained high over time for all participants. Although consumers living independently were initially less likely to report medication adherence than those in assisted living facilities, by the end of treatment, they eventually caught up and even exceeded the adherence rates reported by consumers living in assisted living settings by the end of the 3-month MATS intervention (figure 4, top). The statistically significant intercept terms indicated that at initial assessment, participants were much more likely to report medication adherence rather than forgetting to take medication, not wanting to take medication, or only taking some of their medication. However, participants living independently were less likely to report medication adherence than those in assisted living facilities. Those living independently reported a higher probability of forgetting to take medication at baseline than those in an assisted living setting, but the probability of reporting forgetting diminished over the course of the MATS intervention (figure 4, bottom).

Table 2.

HGLM Results for the Outcome Question of Each Text Intervention

Outcome Domain	Parameter Estimate	OR	t	P
Medication adherence
“Did you take your meds today?”
For “forgot” (relative to “yes”)
Intercept	−2.539	0.079	−7.21	<.001
Living situation	1.405	4.076	1.99	.053
Day number	−0.025	0.976	−3.17	.003
Day number × living situation	−0.039	0.962	−2.48	.018
For “don’t want to” (relative to “yes”)
Intercept	−4.961	0.007	−7.58	<.001
Living situation	0.400	1.492	0.31	.761
Day number	0.003	1.003	0.21	.835
Day number × living situation	−0.012	0.988	−0.50	.621
For “only some” (relative to “yes”)
Intercept	−3.803	0.022	−8.11	<.001
Living situation	1.756	5.788	1.87	.068
Day number	−0.011	0.990	−1.16	.252
Day number × living situation	−0.035	0.966	−1.93	.060
Socialization
“How many people have you socialized with outside the home?”
For “no interactions” (relative to “4+”)
Intercept	−0.256	0.774	−0.95	.349
Day number	−0.002	0.998	−0.48	.633
For “1 person” (relative to “4+”)
Intercept	−0.477	0.621	−1.85	.071
Day number	−0.011	0.989	−2.21	.033
For “2–3 people” (relative to “4+”)
Intercept	−0.211	0.809	−0.94	.352
Day number	−0.002	0.998	−0.49	.627
Auditory hallucinations
“Have you been bothered by voices?”
For “extremely” (relative to “no voices”)
Intercept	−2.370	0.093	−4.80	<.001
Day number	−0.003	0.997	−0.39	.695
For “moderately” (relative to “no voices”)
Intercept	−1.121	0.326	−2.55	.015
Day number	−0.019	0.981	−2.43	.020
For “a little” (relative to “no voices”)
Intercept	−0.309	0.734	−0.86	.393
Day number	−0.002	0.998	−0.53	.597

Note: HGLM, hierarchical generalized linear modeling. Separate models for socialization and auditory hallucinations that included living situation (staff assisted vs independent) did not show any significant main effects or interactions involving living situation.

Fig. 4.

Hierarchical generalized linear modeling–derived trajectories of responses to “Did you take your meds today?” are shown separately for consumers living in assisted living and consumers living independently.

In the model for socialization, the effect of an additional day of treatment was significantly negatively associated with the log-odds of socializing with 1 person, relative to 4 or more. This suggested that an additional day of treatment significantly increased the odds of having 4 or more social interactions relative to having only 1 social interaction. Figure 5 depicts the probability of endorsing each answer to the socialization outcome question each day over the 12-week treatment period. The figure shows that the probability of maximal socialization (4+ interactions) outside of the home increased steadily over the course of the 3-month treatment, while the probability of having only 1 social interaction per day decreased over 10%. There was no significant change in the probability of reporting 0 or 2–3 interactions.

Fig. 5.

Hierarchical generalized linear modeling–derived trajectories of responses to “How many people have you socialized with outside of home?”.

In the model for auditory hallucinations, there was a statistically significant negative association between time and the log-odds of being moderately bothered by hallucinations, relative to having no hallucinations. This suggested that an additional day of treatment significantly increased the odds of reporting not having any voices, relative to being moderately bothered by voices. Figure 6 depicts the probability of endorsing each answer to the auditory hallucination outcome question each day over the 12-week treatment period. The figure shows that the probability of reporting being moderately bothered by hallucinations decreased at a rate similar to the rate at which the probability of reporting no hallucinations increased over the course of treatment. Statistically significant intercept terms indicated that at initial assessment, consumers were more likely to report no voices than extreme or moderate hallucinations (figure 6).

Fig. 6.

Hierarchical generalized linear modeling–derived trajectories of responses to “Have you been bothered by voices?”.

Similar HGLM analyses were also used to examine changes in responses to the second (current cognitions) question during treatment. For medication adherence question, “Do meds help you stay healthy?,” a significant negative association was found between time and the log-odds of responding “Not at all” (OR = 0.96, t = −3.33, P = .002) or “Not sure” (OR = 0.96, t = −2.66, P = .012), relative to the “Yes definitely” response category. This suggests participants changed from believing medications were not helpful to believing medications help them stay healthy. For the socialization intervention, a significant negative association was found between time and the log-odds of responding that socializing is a “waste of time” (OR = 0.98, t = −2.27, P = .029) and “dangerous” (OR = 0.98, t = −2.79, P = .009), relative to the “other” response category. This suggests a reduction in conviction in these negative beliefs about socializing. For the auditory hallucinations intervention, a significant negative association was found between time and the log-odds of reporting voices are “uncontrollable” (OR = 0.98, t = −2.28, P = .028) and marginally for “all knowing” (OR = 0.98, t = −1.95, P = .058), relative to the “other” category. This suggests a reduction in conviction in the beliefs that voices are uncontrollable or all knowing.

With regard to secondary outcomes, paired samples t tests showed no significant differences between baseline and posttreatment assessments for any of the laboratory assessments of symptoms (PANSS, BDI-II) or functioning (ILSS) (d’s = −0.14–0.05). Because these scales measured global symptom domains, we explored pre-post differences on individual PANSS items that more specifically indexed the target outcomes. However, the pre-post difference on the PANSS hallucinations item (P3) also was not significant, M_pre = 3.67, SD = 1.76, M_post = 3.60, SD = 1.74, t₄₁ = 0.29, P = .867, d = 0.04, and the pre-post difference for the PANSS social withdrawal item (N4), was marginally significant, M_pre = 3.05, SD = 1.64, M_post = 2.54, SD = 1.65, t₄₁ = 1.85, P = .072, d = 0.31.

Discussion

This pilot trial of the MATS intervention demonstrated that interactive text message assessments and interventions are feasible for many consumers with schizophrenia. Notably, the average valid response rate for the 3 intervention domains was 83%–86% for question 1 for participants who completed more than 2 weeks of the MATS intervention, and the vast majority of phones (86%) were returned at the end of the trial without incident or damage. The MATS intervention was also effective at improving medication adherence, socialization, and auditory hallucinations for many consumers.

With regard to medication adherence, the MATS intervention was most effective for individuals who were living independently. These consumers likely benefited most from MATS because, unlike those residing in supported living environments, they had fewer supports in place to assist in taking medications. The improvement in reporting medication adherence was relative to reporting “forgot,” which suggests the text messages may have served as a reminder to take daily medications, especially for consumers who did not live in a setting with staff who would remind them. This finding suggests mobile interventions and other behavioral interventions that incorporate routines and other natural prompts in daily life to remind consumers to take medications may improve medication adherence in this population. Over the course of treatment, consumers were also less likely to report that medications do not help them “stay healthy.” This may suggest that the intervention was associated with a reduction in negative beliefs about medications, which may have also contributed to improved adherence.

MATS was associated with improvement in socialization. The probability of maximal socialization (4+ interactions) outside of the home increased steadily over the course of the 3-month treatment, while the probability of having only 1 social interaction per day decreased over 10%. A significant reduction over the trial period was also found in the probability of reporting socializing is “waste of time” or “dangerous.” It is possible that these changes in negative cognitions about social engagement positively impacted day-to-day socialization behavior. Behavioral suggestions (eg, “Try asking a friend to go for a walk”) may have also prompted interactions with others. The probability of reporting zero social interactions did not change during the MATS intervention.

MATS was also associated with a reduction in severity of auditory hallucinations. The probability of reporting moderate severity of auditory hallucinations decreased, and the rate of increase in reporting no hallucinations increased. In contrast, the probability of reporting extreme hallucinations was rare and did not change significantly over the course of the MATS intervention. The probability of endorsing thoughts that hallucinations are “uncontrollable” or “all knowing” also decreased significantly during MATS. The thought-challenging and behavioral experiment interventions in MATS may have contributed to reduced conviction in these beliefs about voices, and this reduction in negative beliefs may have been associated with a reduction in hallucinations. It is also possible that the improvements found in medication adherence, rather than changes in cognitions, lead to the reduction in hallucinations, but medication adherence only improved for consumers living independently, whereas improvement in hallucinations was comparable for consumers living independently and living in assisted housing (table 2).

The version of MATS used in this pilot study may not be helpful to all consumers with schizophrenia. MATS was delivered on an older generation Motorola phone platform, which was clearly challenging to navigate for some consumers with schizophrenia. Lower functioning consumers with more severe negative symptoms and lower estimated premorbid IQ did not complete the intervention, despite multiple reminders and training sessions. These participants were less successful in mastering the use of these older generation phones, but as discussed below, mobile interventions may be more accessible to these participants with a newer smart phone touch-screen platform that is easier to navigate. More intensive individual or group CBT interventions (eg, a standard course of face-to-face sessions or a combination of MATS and some therapy sessions) may be needed for these consumers.

No significant differences in lab-based secondary outcome measures were found between baseline and end of treatment assessments, which suggests that the benefits of this brief 12-week low-intensity MATS intervention did not generalize to these broader outcomes during the limited pilot study period. The discrepancy between the findings for the lab-based measures and the ambulatory monitory outcome measures may also suggest that the daily ambulatory self-reports were not valid assessments of symptom status and functioning. However, these lab-based measures were aggregated global measures of psychopathology and functioning, not specific measures of the MATS treatment targets, like the ambulatory monitoring outcome questions. The pre-post difference on the PANSS hallucinations item (P3) and PANSS social withdrawal item (N4) also were not significant. These item analyses more closely index the target outcomes than the global scores, but they still do not measure the target outcome with the same precision as the EMA text message questions. For example, the PANSS hallucinations item includes auditory, visual, olfactory, and tactile hallucinations, not just the auditory hallucinations targeted in this study and indexed by the text message question. In addition, ESM has been extensively used and validated in consumers with schizophrenia.^3,4,6–11 The ambulatory reports provided in this study also were systematically associated with factors that would be predictably associated with the target outcomes (eg, consumers in assisted living situations reported significantly higher medication adherence than consumers living independently). It is possible that ambulatory monitoring is more sensitive to change in specific treatment targets than global retrospective summaries provided in the context of lab-based assessments, which may be less accurate representations of day-to-day symptom, mood, and functioning behaviors.^1,2 This topic of disparities between momentary self-reports generated in naturalistic settings and retrospective lab summaries of the same period of time is addressed more extensively in a separate article in the current special issue.

Lessons Learned

In addition to the study findings, the development and pilot testing of the MATS intervention provided valuable experiences to inform future development of ambulatory interventions for consumers with severe mental illness. In this pilot study, the older generation Motorola phone required more steps and manipulation for each action, which proved too difficult for some participants to master. For instance, the backlight of the phone screen remained on for a maximum of 60 seconds, so if a participant required more time to read a question, the screen went blank. Participants described pressing a random button to return to the screen, which would result in a completely different message or phone setting, leading to frustration and lack of response. Some consumers were also confused when they missed previous text messages and found multiple unopened questions in their inbox at one time. Others were confused by the phone configuration, when they were brought back to the previous screen after responding to a text message. These problems prompted some consumers to send multiple responses to the multiple questions or to believe that they had not accurately replied, so they sent additional text message answers to the same question. Multiple responses were coded as invalid.

These problems can be avoided by configuring the phone to delete messages after a response is made or when they are not opened within a specific time period. Using more advanced “smart” phones may also help improve response rates in cellular phone-based assessments and interventions. With “smart” phones, applications could be developed with longer response windows and ease of touch-screen navigation and responding, and once an application was initiated, participants would be unable to accidentally exit it until completion of the survey. A platform that is easier for consumers to navigate may improve the adherence rates for lower functioning consumers who struggled with the MATS intervention in this pilot study.

This study had several limitations. First, medication adherence findings should be interpreted cautiously because consumers tend to inflate their self-reports of medication adherence, when compared with more objective measures, such as pill counts.²⁵ Future studies should include more objective measures of medication adherence, in addition to EMA self-reports. Second, consumers reported high medication adherence, low severity of voices, and multiple social interactions at baseline. Even, greater improvements in these outcomes might have been observed had consumers been selected for nonadherence, severe voices, or social isolation at baseline. Third, we suggested that the change in specific beliefs about medications, socialization, and voices found in this pilot study contributed to change in these outcomes, but it is also possible that change in the outcomes contributed to change in the beliefs. Time-lagged analyses could be used to examine the causal direction of these relationships in future mobile interventions research using larger samples with greater frequency of specific beliefs. Fourth, consumers received gift card incentives for responding to text messages, which may not be feasible outside of a research study. Future research is needed to determine text-messaging response rates and MATS efficacy without such incentives. Finally, the study also lacked a comparison group, so the improvements found could be attributed to standard care, rather than MATS.

In conclusion, this pilot study demonstrated that relatively long-term use of mobile technologies to assist in the assessment and treatment of people with serious mental illness is feasible and holds great potential. As cellular phones incorporate more sophisticated technologies, they will likely become more intuitive, affordable, and widespread in use. These preliminary findings for the MATS intervention were encouraging and suggest that, with further development and validation, mobile technologies might facilitate more naturalistic interventions outside of the clinical setting.

Funding

National Institute of Mental Health (P30 MH80002) through the Advanced Center for Innovations in Services and Interventions Research (PI: Dilip Jeste, M.D.) at the University of California, San Diego.