What Do People With Schizophrenia Do All Day? Ecological Momentary Assessment of Real-World Functioning in Schizophrenia

Abstract

Schizophrenia is a major cause of disability worldwide. As new treatments for functioning are tested, the need grows to demonstrate real-world functioning gains. Ecological momentary assessment (EMA) may provide a more ecologically valid measure of functioning. In this study, smartphone-based EMA was used to signal participants with schizophrenia (N = 100) and controls (N = 71) 7 times a day for 7 days to respond to brief questionnaires about social interactions and functioning behaviors. Excellent adherence was found, with both groups completing an average of 85% of surveys and only 3% of participants with schizophrenia excluded for poor adherence. Four-week test–retest reliability was high (r = .83 for total productive behaviors). Relative to controls, participants with schizophrenia reported significantly less total productive activity (d = 1.2), fewer social interactions (d = 0.3), more nonproductive behaviors (d = 1.0; watching TV, resting), and more time at home (d = 0.8). Within the schizophrenia group, participants living independently showed better functioning on EMA relative to participants in supported housing (d = 0.8) and participants engaged in vocational activities showed better functioning than individuals not engaged in vocational activities (d = 0.55). Modest correlations were found between EMA and an in-lab self-report measure of functioning activities performed in the community, but not between EMA and measures of functional capacity or potential. This study demonstrated the feasibility, sensitivity reliability, and validity of EMA methods to assess functioning in schizophrenia. EMA provides a much-needed measure of what individuals with schizophrenia are actually doing in real-world contexts. These results also suggest that there may be important disjunctions between indices of abilities and actual real-world functioning.

Introduction

Schizophrenia is a leading cause of disability worldwide.^1–4 To adequately test new treatments for disability in schizophrenia, better outcome measures are needed to demonstrate improvements in real-world functioning. There is widespread agreement in the field that currently available functional assessment methods do not adequately capture the day-to-day functioning behaviors of patients with schizophrenia.^5–9 Existing functioning measures rely on self-reports that are compromised by retrospective recall problems and subjective biases, or attempt to collect reports from informants who may not be available or have limited knowledge of the patient’s functioning.¹⁰ Proxy role-play measures, although quite reliable, have complex relationships to actual performance of real-world functioning behaviors.¹¹ Otherwise stated, just because someone can perform a skill in the lab, does not mean they do perform it in the real world. The field has been lacking a low cost, efficient, and more objective functioning measure, and ecological momentary assessment (EMA) holds promise in responding to this need. What EMA could add to the study of functioning in people with serious mental illness is simple and powerful: an assessment of what people are actually doing in their daily lives on a moment to moment basis.

EMA^7,12 is an ambulatory data collection technique that allows the real-time in vivo assessment of functioning behaviors, including educational, employment, socialization, active leisure, self-care, and home-care activities. Modern EMA, also called Experience Sampling Method (ESM), uses smartphones to signal participants several times throughout the day to respond to very brief (eg, 3 minutes) questionnaires about their daily lives. Smartphones are now widely used by individuals with psychosis and the majority of individuals respond favorably to using mobile phones for support and self-management.¹³ EMA offers several advantages over lab-based assessments,¹⁴ including frequently sampling current moods, thoughts, and behaviors which can fluctuate rapidly over the course of the day, rather than relying on recall of general feelings, thoughts, and behaviors over weeks or months. Dissociations between daily experiences of emotions and lab-based emotion assessments have been reported in participants with schizophrenia.^15,16 We previously demonstrated the general feasibility and validity of digital EMA methods in schizophrenia,^17,18 and EMA has been used in this population to assess stress reactivity,^19,20 emotion regulation in social interactions,²¹ autonomic regulation and auditory hallucinations,²² self-stigma,²³ suicidal ideation,²⁴ substance use,^25,26 paranoia and positive symptoms,^27–29 and motivational negative symptoms³⁰ in schizophrenia.

Despite a growing number of EMA studies in schizophrenia, few studies have examined social activity and daily functioning. Previous EMA studies have found that participants with schizophrenia spectrum illness spend more time alone, and when with others, they report less pleasure and greater interest in being alone.^30–37 We previously found that EMA of defeatist appraisals of social interactions were associated with decreased positive emotions and less engagement in social interactions later in the day.¹⁸ Schneider et al³⁷ used EMA to specifically sample social functioning in a very large (N = 235) sample of participants with nonaffective psychosis and found participants spent more time home, alone, and doing nothing relative to controls, but the groups did not differ significantly in engagement in leisure activities. The Schneider et al EMA survey, however, did not broadly sample leisure and functioning behaviors across multiple domains (eg, self-care, home-care, work, school).

This present study examined the feasibility, reliability, and validity of EMA of functioning behaviors in schizophrenia across multiple domains. A brief survey of social interactions, self-care, home-care, leisure, work, and educational functioning behaviors performed within the past hour was delivered by smartphones 7 times a day for 7 days to participants with schizophrenia and to controls. Adherence (survey response) rates and device failure/loss were used to examine feasibility, and 4-week test–retest reliability was examined. Sensitivity to disability and validity were examined by comparing participants with schizophrenia to controls and comparing EMA responses to objective indicators of functioning (independent vs supported living and vocational engagement vs none). Finally, convergence between EMA and in-lab functional capacity, self-report, and informant functioning measures was examined. We predicted that EMA of functioning would be feasible and reliable, reveal poorer functioning in participants with schizophrenia relative to controls, as well as demonstrating within-diagnosis sensitivity to poorer functioning in participants living in supported housing or not engaged in vocational activities, and show convergent associations with in-lab functioning measures.

Methods

Participants

Inclusion criteria for participants with schizophrenia (S) were as follows: (1) met DSM-5 diagnostic criteria for schizophrenia (N = 82) or schizoaffective (N = 18) disorder (based on Structured Clinical Interview for DSM or SCID³⁸); (2) age 18–65 years; (3) taking antipsychotic medication(s); (4) no medication changes in the prior month or anticipated in near future; (5) outpatient; (6) fluent English so as to be able to complete testing validly (method developed by Artiola i Fortuny³⁹); (7) able to give valid informed consent; and (8) able to identify one informant who agrees to provide real-world functioning ratings. Outpatients were recruited from board-and-care homes/supported housing residences, mental health clinics, and clubhouses in the UC San Diego Health, San Diego County Mental Health and Veterans Affairs San Diego Healthcare Systems. The Brief Psychiatric Rating Scale (BPRS; 24-item version⁴⁰) showed moderate symptom severity (M = 52.2; SD = 12.6).

Control participants (C) were recruited using advertisements. Inclusion criteria for controls were as follows: (1) no DSM-5 diagnoses of past or current mood, anxiety, or psychotic disorders (based on the SCID-Nonpatient Version³⁸); (2) age 18–65 years; (3) fluent English; and (4) able to give valid informed consent. Exclusion criteria for both groups were as follows: (1) history of head trauma with loss of consciousness longer than 15 minutes; (2) evidence of seizure disorder; (3) evidence of cerebrovascular accident or dementia; (4) current substance dependence meeting DSM-5 criteria in the past year; (5) uncooperativeness with in-lab assessments; and (6) sensory limitations including vision uncorrectable to 20/40, color blindness, or hearing loss that interferes with assessment.

EMA Procedures

A Samsung smartphone with Android OS was provided to participants to deliver EMA surveys. The device was programmed using Samplex⁴¹ software to administer surveys 7 times per day for 7 days. The signals occurred at stratified random intervals that vary from day to day within, on average, 1.5-hour windows starting at approximately 9:00 am and ending at 9:00 pm each day (adjusted to accommodate individual sleep/wake schedules), alarms could be silenced for 30-minute intervals (eg, driving, classes). All responses were time-stamped and were only allowed within a 15-minute period following the signal. The devices were disabled between assessments, and no other voice, text, internet, or phone applications were available. A training session (typically <20 minutes) was provided on how to operate and charge the device and respond to surveys. Participants were also contacted by telephone on the first and third day of EMA to troubleshoot and encourage adherence.

A second test–retest reliability follow-up assessment was offered to all participants 4 weeks following the initial week of EMA sampling until approximately N = 75 patients and N = 50 controls were reassessed. For completing in-lab assessments, patients were paid $50 for their initial visit and $25 for test–retest follow-ups, and due to the shorter assessment battery, controls were paid $25 for initial visits and $15 for follow-ups. In addition, to encourage EMA adherence, all participants were paid an additional $1 per EMA survey ($49 maximum; with a running total displayed on the device after each survey). All payments were made when participants returned the device.

Surveys (see table 1) were predominantly check-box questions asking about time spent at home and functioning behaviors performed during the past hour, including work/school, self-care, home-care, at-home and outside-home leisure, transportation, and treatment engagement activities, as well as number of social interactions and social context (eg, family, friends, strangers, coworkers/classmates). Nonproductive activities were also queried (eg, watching TV, resting, pacing). One screen was presented with several activities listed and participants tapped the screen to check boxes if the activity was performed in the past hour, with an option to check “None of these” on each screen. The functioning activities queried were based on items from highly rated functional outcome scales.⁴² Questions about moods and experiences and attitudes during social interactions and functioning activities were also asked, but will be described in a separate report.

Table 1.

EMA Functioning Survey Questions

The total number of functioning behaviors reported during the week of sampling was summed for each functioning domain and a total productive behavior score was computed as the sum of all vocational, self-care, home-care, in-home-leisure activities, and outside-home-leisure activities. Because greater adherence (number of surveys completed) could increase the number of behaviors reported, functioning behavior scores were computed as the proportion of the total number of behaviors reported (all productive, nonproductive and treatment activities). Number of interactions reported was divided into social (family, friends or acquaintances, coworkers or classmates) and nonsocial (roommates or fellow residents, residential staff, treatment providers) interactions.

An in-lab questionnaire about the EMA experience was administered. Statements regarding burden (eg, “The device was comfortable to carry,” “The beeps interfered with my activities”), difficulty (eg, “I had difficulties understanding the questions,” “I had difficulties operating the device”), pleasantness (eg, “Overall, this experience was pleasant,” “Overall, this experience was stressful”), and other aspects of acceptability were rated on a 5-point Likert scale (reverse coded when necessary such that higher ratings indicated more positive experience).

In-Lab Functioning Measures

The Specific Levels of Functioning patient and informant report forms (SLOF-P, SLOF-I⁴³), UCSD Performance-Based Skills Assessment-Brief (UPSA-B⁴⁴), and the self-report version of the Independent Living Skills Survey (ILSS⁴⁵) were administered. For the SLOF, we used the average item scores because some informants stated that they could not answer all questions, and the SLOF total was the average of the interpersonal relationships (items 1–7), activities (items 14–24), and work skills (items 25–30) domains. SLOF informants were a relative or friend, or a high-contact clinician, such as a case manager, social worker, or residential facility staff, who had seen the patient at least weekly for at least 3 months. For the UPSA, the average of financial and communication domains was used. For the ILSS, the 51 items across 10 domains of functioning (appearance and clothing, personal hygiene, care of possessions and living space, food preparation, health maintenance, transportation, money management, leisure and recreational activities, job seeking, job maintenance) were scored 0 = not performed, 1 = performed, or “Not Able to Demonstrate,” and the average of available items was computed as a total functioning score.

Statistical Analyses

Four-week test–retest reliability (within groups and for the full sample) was estimated using Pearson’s r correlations, and paired t-tests were used to compare EMA variables at time 1 vs time 2. T-tests were used to compare participants with schizophrenia and controls and, within the schizophrenia group, to compare: (1) participants living independently with those residing in assisted living and (2) participants reporting any paid or volunteer work or school activity and those not engaged in either. Finally, Pearson’s r correlations were used to examine associations between EMA and in-lab functioning measures.

Results

Sample

Controls did not differ significantly from participants with schizophrenia with regard to age (C: M = 50.2, SD = 10.8; S: M = 51.7, SD = 9.3; t(168) = 1.00, P = .318), gender (C: female = 37% [N = 26]; S: female = 29% [N = 29]; χ ²(1) = 1.25, P = .264), or race (C: Caucasian = 46% [N = 32]; S: Caucasian = 36% [N = 36]; χ ²(1) = 1.62, P = .203). Controls (M = 14.6, SD = 1.8) had approximately 1.5 more years of education that those with schizophrenia on average (M = 13.0, SD = 1.9; t(167) = 5.47, P < .001). Scores on in-lab measures for participants with schizophrenia were as follows: SLOF-P: M = 3.9, SD = 0.7; SLOF-I: M = 3.8, SD = 0.8; ILSS: M = 0.76, SD = 0.09; UPSA-B: M = 15.5, SD = 2.3.

Adherence and Feasibility

Only 4 phones were lost (1 S; 3 C), and 2 phones (1 S; 1 C) malfunctioned resulting in EMA data loss. Lost or malfunctioning phones were replaced. Adherence was excellent with 42 (SD = 7.6) of the 49 (85%) programmed surveys being completed on average, and the 2 groups did not differ significantly in adherence (C = 41.6 [SD = 7.4]; S = 41.9 [SD = 7.8]; t(169) = 0.20, P = .842). The groups also did not differ significantly in the number of participants excluded for inadequate adherence (defined as <33% surveys completed) (C = 6.6% [5/76]; S = 2.9% [3/103]; χ ²(1) = 1.38, P = .241). After excluding the 3 S and 5 C participants for inadequate adherence, the final sample included in analyses was 100 S and 71 C.

Results of the EMA Experience Survey showed that controls found the experience more positive than the schizophrenia group (average of all items; C: M = 4.5, SD = 0.4; S: M = 4.2, SD = 0.6; t(167) = 2.63, P = .009, d = 0.3), although both groups reported that it was a positive experience (both means greater than 4, with a score of 5 being the best possible). The schizophrenia group reported more difficulty than controls in understanding questions (S: M = 1.3, SD = 0.7; C: M = 1.1, SD = 0.3; t(167) = 2.24, P = .027, d = 0.3) and responding (S: M = 1.4, SD = 0.7; C: M = 1.1, SD = 0.4; t(167) = 2.45, P = .015, d = 0.3), although all these effect sizes were small and difficulty ratings were low. The schizophrenia group also found the experience more challenging (S: M = 2.6, SD = 1.5; C: M = 1.5, SD = 0.9; t(167) = 5.47, P < .001, d = 1.0) and more stressful (S: M = 1.9, SD = 1.3; C: M = 1.3, SD = 0.8; t(166) = 2.95, P = .004, d = 0.5) than controls, but both groups reported low levels of challenge and stress in general.

Test–Retest Reliability

Test–retest reliability results are presented in table 2. Estimated reliabilities were generally very good, with most variables >.80 (range = .64–.85). Reliabilities for functioning behaviors that are typically more routinely performed across weeks (eg, vocational, home- or self-care) were generally higher than less routine behaviors (eg, leisure, treatment), as would be expected.

Table 2.

Four-Week Test–Retest Reliability

EMA Variable	Schizophrenia (N = 75)	Healthy Control (N = 47)	Total Sample (N = 122)
At-home (% surveys)	.67	.67	.71
Self-care	.68	.73	.70
Home-care	.74	.71	.73
At-home leisure	.77	.69	.76
Outside-home leisure	.64	.66	.67
Vocational	.76	.83	.83
Treatment	.67	.43	.67
Nonproductive	.78	.78	.83
Total productive	.74	.80	.83
Social interactions	.82	.73	.79
Nonsocial interactions	.85	.62	.76

Note: EMA, ecological momentary assessment.

Group Comparisons

Participants with schizophrenia reported spending significantly more time at home during the past hour than controls (C: M = 32.4 minutes, SD = 15.2; S: M = 43.1 minutes, SD = 10.6; t(169) = 5.42, P < .001, d = 0.8), and reported being at home the entire past hour on a significantly greater proportion of surveys (C: M = 46%, SD = 25%; S: M = 62%, SD = 20%; t(169) = 4.71, P < .001, d = 0.7). Participants with schizophrenia also reported significantly fewer productive functioning behaviors than controls (see figure 1) for outside-home leisure (t(169) = 3.82, P < .001, d = 0.6); at-home leisure (t(169) = 5.18, P < .001, d = 0.8); vocational (t(169) = 4.01, P < .001, d = 0.6); and total productive behaviors (t(169) = 8.01, P < .001, d = 1.2), but reported significantly more treatment activities (t(169) = 3.90, P < .001, d = 0.6) and nonproductive behaviors (t(169) = 6.39, P < .001, d = 1.0). When leaving the home, controls transported themselves independently more often (C: M = 65%, SD = 29%; S: M = 45%, SD = 32%; t(164) = 4.06, P < .001, d = 0.65) and drove themselves where they went more often than participants with schizophrenia (C: M = 45%, SD = 40%; S: M = 17%, SD = 33%; t(164) = 4.80, P < .001, d = 0.75). No significant differences between groups were found for productive activities involving self-care (t(169) = 0.72, P = .474, d = 0.1) or home-care (t(169) = 0.42, P = .674, d = 0.1). Controls and participants with schizophrenia did not differ significantly in total interpersonal interactions, but controls reported more social interactions since the last survey (C: M = 1.48, SD = 1.08; S: M = 1.12, SD = 1.04; t(169) = 2.14, P = .034, d = 0.3) and fewer nonsocial interactions since the last survey (C: M = 0.47, SD = 0.70; S: M = 0.76, SD = 0.72: t(169) = 2.58, P = .011, d = 0.4), than participants with schizophrenia.

Fig. 1.

Total functioning behaviors reported during the week of sampling for each domain as a proportion of total behaviors reported for controls and participants with schizophrenia.

EMA and Objective Indicators of Functioning

Figure 2 shows that participants in the schizophrenia group who were living independently reported significantly more home-care (t(98) = 3.16, P = .002, d = 0.7) and total productive behaviors (t(98) = 3.58, P = .001, d = 0.8), as well as fewer nonproductive behaviors (t(98) = 3.04, P = .003, d = 0.65), than those living in supported living arrangements. Those living independently also reported being at home the entire past hour on a significantly smaller proportion of surveys (M = 60%, SD = 20%) relative to those in supported living (M = 68%, SD = 19%; t(98) = 2.01, P = .047, d = 0.4). Figure 2 shows that participants in the schizophrenia group who were engaged in work or school activities endorsed significantly more EMA vocational activities (t(98) = 4.84, P < .001, d = 1.1) and total productive behaviors (t(98) = 2.34, P = .021, d = 0.55), as well as fewer nonproductive activities (t(98) = 2.04, P = .044, d = 0.5) than those not engaged in work or school. Those engaged in work or school also spent significantly less time at home in the past hour (M = 38.2 minutes, SD = 13.5) than those not engaged in work or school (M = 44.6 minutes, SD = 9.2; t(98) = 2.62, P = .010, d = 0.6), and reported being at home the entire past hour on a significantly smaller proportion of surveys (M = 55%, SD = 23%) relative to those not engaged in work or school (M = 65%, SD = 19%; t(98) = 2.02, P = .047, d = 0.5).

Fig. 2.

Total functioning behaviors reported during the week of sampling for each domain as a proportion of total behaviors reported for participants with schizophrenia living in supported vs independent housing and engaged vs not engaged in any work or school activity.

Relationships Between EMA and In-Lab Measures

Correlations between EMA and in-lab measures are shown in table 3. Significant but modest correlations were found between EMA and the ILSS, but not the UPSA-B, SLOF-P, or SLOF-I total scores. In addition, no significant correlations were found between any SLOF-I or SLOF-P subscale and any EMA variable, with the exception of the SLOF-P Interpersonal Relationships subscale, which was significantly correlated with the number of social interactions reported on EMA (r = .30, P < .01).

Table 3.

Pearson’s Correlations Between EMA and In-Lab Functioning Measures in the Schizophrenia Group

EMA Variable	UPSA-B	SLOF-P	SLOF-I	ILSS
Total productive	.01	.14	.13	.25*
Nonproductive	.07	−.14	−.11	−.25*
Social interactions	−.05	.24* +	−.03	.10
At-home (% surveys)	.09	-.04	−.07	−.19
Independent transportation	.04	.03	.06	.23*

Note: *P < .05; ⁺r = .30** for SLOF-P Interpersonal Relationships with no other significant SLOF-P subscale correlations. EMA, ecological momentary assessment; SLOF-P and SLOF-I, Specific Levels of Functioning Scale, Patient and Informant versions; UPSA-B, UCSD Performance-based Skills Assessment—Brief; ILSS, Independent Living Skills Survey.

Discussion

EMA was found to be a feasible, reliable, and valid method to assess functioning in schizophrenia. To our knowledge, this is the first EMA study to broadly sample multiple functioning domains in schizophrenia and demonstrate the reliability and validity of EMA as a measure of real-world functioning. The results support the use of EMA as an outcome measure of functioning with excellent sensitivity to disability and stability for clinical trials, although additional research is needed to examine sensitivity to change in a clinical trial.

With regard to feasibility, adherence rates were excellent, with 97% of participants with schizophrenia producing useable data and 85% of surveys completed, when using 1 week of sampling with 7 surveys per day. On the EMA experience survey, participants with schizophrenia reported that EMA was more challenging than did controls, but both groups rated the experience as positive (>4, 5 being best). The 3% exclusion and 85% adherence rates were higher than found in prior EMA studies of schizophrenia, which are typically 80% or less (Schneider et al³⁷ = 69% adherence, Granholm et al¹⁷ = 87%, Granholm et al¹⁸ = 72.1%). The excellent adherence found was likely the result of paying participants $1 for each completed survey, which was not done in prior research.

Only 4 phones were lost (1 S; 3 C), and 2 phones (1 S; 1 C) malfunctioned, but lost or malfunctioning phones were replaced without EMA data loss. It may not be necessary to provide participants with phones in future research, as the number of patients with schizophrenia who own their own smartphone is rapidly increasing, which reduces the cost of EMA. In a meta-analysis, Firth et al¹³ reported that mobile phone ownership in individuals with psychosis was 66.4% and has been growing significantly, so could be even higher now. That investigation also found that the majority of participants favored using mobile phones to enhance health, so smartphones may provide a feasible platform, not only for EMA, but for ecological momentary interventions.⁴⁶

Good to excellent 1-month test–retest reliability was found for the majority of variables. With regard to functioning domains, activities that are less consistently performed from week to week showed lower reliabilities (eg, outside leisure like cinema, sporting events; treatment like medical appointments), whereas activities performed more consistently (eg, vocational) showed higher reliabilities. Aggregate total productive and nonproductive scores that might be more useful in clinical trials showed excellent test–retest reliability of .83. The test–retest reliability of the EMA measures are very comparable to or superior to those seen with functional status rating scales.⁴⁷

With regard to validity, EMA was highly sensitive to differences between participants with schizophrenia and controls, with very large effect sizes (eg, d = 1.2 for total productive behaviors). EMA was also highly sensitive to objective indicators of functioning (independent living and vocational participation) in predictable ways (eg, more home care if living independently; more vocational activities if working or attending classes). The most sensitive indicators of poor functioning were total productive and total nonproductive activities, and the proportion of surveys at home all the time. Participants with schizophrenia reported being at home the majority of the waking day and their activities were nearly evenly split between productive and nonproductive behaviors, whereas controls spent less time at home and about 75% of their activities were productive. These findings suggest EMA is a valid measure of functioning in schizophrenia.

Schneider et al³⁷ also found that participants spent more time home, alone and doing nothing relative to controls, but the groups in that EMA study did not differ significantly in leisure activities. The present study, in contrast, found that patients and controls differed significantly in both at-home leisure (d = 0.8) and outside-home leisure (d = 0.6). These discrepant findings are likely due to the breadth of leisure activities sampled. The survey in the present study asked about multiple specific leisure activities, not categories of activities, whereas in Schneider et al,³⁷ the participant was left to decide whether specific activities were doing “nothing” or “leisure” activities, so nonproductive activities like watching TV could be reported as a “leisure” activity.

Participants with schizophrenia did not differ significantly from controls with regard to total number of interactions, but did report significantly fewer “social” interactions (family, friends, coworkers, classmates) and significantly more “nonsocial” interactions (roommates/fellow residents, staff, providers) than controls. This finding illustrates the importance of asking about the context of interactions. Participants with schizophrenia were talking with others throughout the day, but these interactions were often with care providers and fellow residents/roommates (which may not have been initiated by patients). These results are consistent with previous EMA studies that found participants with schizophrenia spectrum illness spent more time alone.^{30,32–34,36,37}

Although EMA was found to be a highly sensitive measure of disability and engagement in activities aligned with functional milestones with very large effect sizes, EMA was not strongly related to in-lab measures of functioning. This raises the question of whether in-lab scales are measuring the “right stuff.” ⁴⁸ Significant but modest correlations were found between EMA and the ILSS, which is a self-report measure of whether or not specific functioning behaviors were actually performed in the community. In contrast, EMA was not associated with the UPSA-B, which measures functional capacity, or the SLOF, which primarily measures an individual’s potential ability to perform tasks without assistance regardless of whether or not the activities were actually performed in the community. This finding is consistent with prior work indicating that the SLOF and UPSA-B do not directly index functional milestones.⁴⁹ The ILSS may have shown stronger associations with EMA because the ILSS asks about participation in activities and not capacity or potential. In-lab measures also may be compromised by retrospective recall problems and subjective biases (eg, defeatist attitudes), as well as reports from informants who may not have adequate knowledge of the patient’s functioning,¹⁰ which may have contributed to the lack of strong associations found between EMA and in-lab measures.

Measures of functional capacity or potential ability to perform tasks in the community have complex relationships to actual performance of real-world functioning behaviors.¹¹ Multiple previous studies have identified factors that mediate the relationship between indices of functional capacity and actual real-world functioning. These include previous experience, reduced motivation,⁵⁰ history of previous long-term institutionalization, self-efficacy,⁵¹ global severity of illness,⁵² and defeatist attitudes.⁵³ Thus, just because someone can perform a skill in the lab, does not mean they do perform it in the real world.⁵⁴ What people actually do may be impacted by a number of factors (eg, motivation, attitudes, resources, and supports), beyond an individual’s capacity or potential ability to perform a task. In particular, several studies^55,56 have shown that motivation is a key factor contributing to the capacity-performance discrepancy in schizophrenia. People have to be motivated to use the skills they have. If we only develop interventions that improve functional capacity or potential but not actual performance of functioning activities, then lives may not be meaningfully improved. We need to use EMA to determine whether interventions improve what people actually do in the real world and to better understand the determinants of actual participation rather than potential. In a future paper, we will report on some of the determinants (eg, neurocognition, mood, motivation, defeatist beliefs) of EMA measures of actual participation.

The present study had several limitations that should be considered in interpreting the findings. We chose to validate EMA reports relative to in-lab assessments and functioning milestones, and did not examine convergent validity relative to video, diary, or phone recall interviews. We did not use diary or phone interviews because both are also subjective self-reports and a diary is not necessarily sampled contemporaneously as is an EMA. That is, if EMA reports are inaccurate, these 2 other forms of reporting should be inaccurate as well, and for the same reasons. Validation of EMA relative to coded activities from video recordings could be used as the “ground truth” to validate EMA reports, but feasibility and privacy issues present significant challenges. It is also possible to objectively verify some reports using GPS (eg, time spent at home), although such data do not inform what the person is actually doing. We recorded GPS in a subgroup of participants in the present study and found that EMA reports of being outside the home were consistent with objective GPS location.⁵⁷ We also found that correlations between negative symptoms, especially diminished motivation, and GPS distance traveled were much larger than the correlations between GPS and functioning measures, suggesting that negative symptoms may over-ride competence in predicting activities outside the home and distance traveled.

Another limitation involved possible bias in recruiting lower functioning controls, who were primarily recruited from job ads in a local free newspaper, and 41% were unemployed. Thus, the control participants may have been more likely to be at home more of the day and engaged in fewer productive activities than would a sample recruited using other methods (eg, random calling). It is possible that sampling bias toward lower functioning controls may underestimate the observed large effects in comparing people with schizophrenia and controls, and contributed to nonsignificant differences between patients and controls on some functioning domains (eg, self-care and home-care). It is also possible that low frequency of performing some of these activities across a week contributed to lack of group differences in these domains. Despite any possible bias toward sampling lower functioning controls, differences between patients and controls were still large or very large on key total productive and nonproductive EMA measures. The patient sample in this study was also somewhat older than typical schizophrenia samples, which may have contributed to poorer functioning, and the sample was selected to be outpatients on medications, so it is unclear if the findings would generalize to acutely ill or inpatient samples, who may show poorer functioning and possibly poorer adherence. Finally, this study was funded by the Department of Veterans Affairs, so there was also a larger proportion of Veterans in both groups than is typical of schizophrenia research, which may limit generalization. Despite these limitations, the results suggest EMA is a promising measure of functioning in schizophrenia.