Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Mar 1.
Published in final edited form as: Schizophr Res. 2013 Jan 30;144(1-3):87–92. doi: 10.1016/j.schres.2012.12.028

Initial validation of a computerized version of the UCSD Performance-Based Skills Assessment (C-UPSA) for assessing functioning in schizophrenia

Raeanne C Moore a,c, Alexandrea L Harmell a,b, Jennifer Ho a,b, Thomas L Patterson a,*, Lisa T Eyler a,c,d, Dilip V Jeste a,c, Brent T Mausbach a
PMCID: PMC3572290  NIHMSID: NIHMS435923  PMID: 23375625

Abstract

Objective

This study aimed to validate the Computerized UCSD Performance-Based Skills Assessment (C-UPSA), a newly developed scale for assessing functional capacity in patients with schizophrenia.

Methods

The C-UPSA was administered to 21 middle-aged and older adults with schizophrenia and 20 healthy comparison (HC) subjects. Schizophrenia participants also completed the original UPSA and a symptom inventory (during a separate visit), and cognitive functioning was assessed in both groups using a brief neuropsychological screening battery.

Results

The C-UPSA total score was significantly correlated with UPSA total scores, and the magnitude of the correlation was comparable to the test-retest reliability of the original UPSA. The C-UPSA was also significantly correlated with UPSA-Brief scores and neuropsychological status among schizophrenia participants. Furthermore, the schizophrenia group scored significantly lower than the HCs on the C-UPSA. ROC curves were generated to determine the optimal C-UPSA value for discriminating between the two groups, with results indicating an optimal cutoff of 75, which is consistent with the derived cutoff from the original UPSA. The C-UPSA identified persons with schizophrenia with 95% accuracy.

Conclusions

The C-UPSA appears to be highly related to the original UPSA. It has several advantages over the standard version, including increased portability, decreased administration time, and minimization of examiner impact on participant performance. Future research would benefit from establishing this test as a clinical and research tool to effectively assess functional capacity.

Keywords: schizophrenia, computerized assessment, neuropsychology, functioning, rehabilitation, recovery

1. Introduction

Schizophrenia is a chronic and severe mental illness frequently resulting in diminished capacity to function independently. Many individuals with schizophrenia reside in assisted living facilities such as board and care (B&C) homes and skilled nursing facilities (Cohen et al., 2000; Palmer et al., 1999). Schizophrenia is estimated to have an economic burden of $62.7 billion annually in the U.S. alone, a result of high unemployment, reduced productivity, and the need for ongoing supervision (Wu et al., 2005). Indeed, fewer than 15% of patients engage in paid employment (Slade and Salkever, 2001). This is likely due to cognitive deficits that prevent individuals with schizophrenia from learning basic functional skills needed to achieve developmental milestones.

To determine if someone has the basic skills needed to reach independence or employment, sophisticated methods of functional assessment are needed. Within the last two decades, there have been significant advances in the assessment of everyday functioning in patients with schizophrenia. Most techniques have focused on detecting symptom severity and functional capacity via self-report measures, proxy ratings, clinician ratings, and/or direct observations of behavior (Patterson et al., 2001). Despite these strategies, however, significant challenges and limitations remain. For example, self-report measures are susceptible to a patient’s level of insight and may be influenced by symptom severity and current affective state (Atkinson et al., 1997). Sabbag et al. (2011) found correlations between patients’ self-reports of functioning and performance-based assessments and neuropsychological functioning to range from 0.00 to 0.36 across six rating scales. While proxy ratings are generally more dependable (Sabbag et al., 2011, 2012), they vary based on the reporter’s relationship to the patient (i.e., clinician versus friend or relative) and the degree to which the caretaker actually observes the patient engaging in behaviors that are key to the assessment. Clinician reports can be subjective and are based on limited information obtained during brief contact with the patient; however, Sabbag et al. (2011) found clinician ratings to be more reliable than reports by friends or relatives. Lastly, although naturalistic observation methods appear promising, they require extensive time and labor, and there is no guarantee that the behavior being assessed will actually occur.

Taken altogether, it is apparent that many of the routinely used methods of assessing a patient’s everyday functioning are suboptimal. Therefore, performance-based measures represent a viable alternative to several of the assessment techniques discussed above. One performance-based measure that has received strong empirical support is the UCSD Performance-Based Skills Assessment (UPSA; Patterson et al., 2001), which incorporates the use of role-play to assess functional capacity in five broad domains of functioning: planning recreational activities, finances, communication, transportation, and household chores.

In a review of several instruments for measuring functional recovery in schizophrenia, we found the UPSA to have the greatest advantages in regard to ease of administration, reliability, validity, and sensitivity to change (Mausbach et al., 2009). Strong positive correlations, ranging from 0.54–0.75, have been consistently demonstrated between the UPSA and cognitive tests (Bowie et al., 2008, 2010; Depp et al., 2009; Harvey et al., 2009; Keefe et al., 2006; Mausbach et al, 2007; McClure et al., 2007; Twamley et al., 2002). Additionally, the UPSA has repeatedly been found to mediate the relationship between cognitive functioning and real-world functional outcomes (Bowie et al., 2006, 2008). The UPSA has also been shown to predict residential independence in patients with schizophrenia (Mausbach et al., 2008, 2010). Furthermore, employment status has been found to relate to scores on a brief version of the UPSA (UPSA-B; Mausbach et al., 2007); receiver operating characteristic (ROC) analysis significantly predicted patients who were employed based on their UPSA-B scores (Mausbach et al., 2011). Thus, the UPSA has established itself as a reliable and valid measure of determining functional ability.

Despite the UPSA’s psychometric strengths, a few disadvantages are inherent in its design. For example, the UPSA kit contains several props (full size maps for the transportation module, grocery items for the household chores module, etc.), which can be burdensome for testers to transport and set up. In addition, the UPSA takes approximately 30 minutes to complete. A computerized version could shorten the assessment time and thus minimize patient fatigue.

Because of the limitations of non-computerized assessments and increasing advances in technology, the benefits of transitioning traditional paper-and-pencil tests into computerized assessments have garnered a significant amount of attention (Friedl et al., 2007). Computerized assessments offer simplicity, greater measurement precision via standardization and enhanced reliability, reduced costs in administration and scoring, automated data exporting (particularly useful in the research setting), and greater availability to patients who reside in areas or settings that may be difficult for testers to reach or enter (Bauer et al., 2012). For this reason, many assessment types have been computerized and given to patients with depression or bipolar disorder (Iverson et al., 2011; Sweeney et al., 2000), mild cognitive impairment (Doniger et al., 2006), dementia (Doniger et al., 2005), and several other clinical groups. There are, of course, some challenges associated with the use of computerized assessments that should be considered, including technical issues (e.g., hardware or software malfunction, having to reset a program or device after each patient, possible greater upfront cost), patient confidentiality and security concerns (especially for web-based assessments or when data are transmitted to remote servers), diagnostic issues (some assessments print out diagnostic or prognostic reports without having gathered a patient history, mental status, and behavioral observations), psychometric development issues, and cultural, experiential, and disability factors (Bauer et al., 2012).

In view of the advantages and promise of computerized assessments, and taking into consideration the potential challenges, the purpose of this study was to investigate the psychometric properties of a newly developed computerized scale for assessing functional capacity in persons with schizophrenia. Specifically, this study explored the reliability and validity of the Computerized UCSD Performance-Based Skills Assessment (C-UPSA) in a sample of patients with schizophrenia and age-matched comparison subjects to help determine whether the C-UPSA was feasible to use with middle-aged and older patients as well as with individuals without psychiatric illness. We hypothesized that schizophrenia patients would perform significantly worse than comparison subjects (HCs) on the C-UPSA and the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolph, 1998). Additionally, given that many older schizophrenia patients have had minimal exposure to computer use compared to HCs, as well as multiple deficits (e.g., in attention, visual perception, working memory, memory and learning, executive function, and processing speed) that could impact performance on computerized testing (Reichenberg, 2010), it seemed necessary to validate whether the C-UPSA could be comparable to the in-person version. We therefore tested the hypothesis that the C-UPSA would be significantly correlated with UPSA, UPSA-B, RBANS, and PANSS Negative scores, which, if true, would suggest that it could be comparable to the non-computerized version. Lastly, we hypothesized that receiver operating characteristics (ROC) curve analysis would identify an optimal C-UPSA cutoff score that would significantly differentiate the schizophrenia participants from the HCs.

2. Methods

2.1. Sample

This study was approved by the Institutional Review Board of the University of California, San Diego (UCSD), and all participants provided written, informed consent. Participants included 21 people with schizophrenia and 20 aged-matched HCs. Individuals with schizophrenia were recruited from the UCSD Skills Training and Empowerment Program (STEP; Cardenas et al., 2012; Depp et al., 2010) and were assessed at B&C and day treatment facilities throughout San Diego County. Primary inclusion criteria for this study overlapped with the parent STEP study and included having a physician-determined DSM-IV-TR diagnosis of schizophrenia or schizoaffective disorder (verified by chart review, as diagnosed by the patient’s psychiatrist), English fluency, and being 40 years of age or older. Participants were excluded if they had a diagnosis of dementia, had suicidal ideation or intent, were unable to complete the assessment battery, or were enrolled in another psychosocial intervention or pharmacotherapy study. To determine capacity to provide informed consent, we administered the UCSD-Brief Assessment of Capacity to Consent (UBACC; Jeste et al., 2007) to all participants prior to enrollment.

Once the schizophrenia group was identified, a HC group was recruited from the San Diego community. HCs were included in this study if they were ≥ 40 years of age, fluent in English, and did not have a psychiatric diagnosis. Specific exclusion criteria were a diagnosis of dementia or another DSM-IV-TR Axis I diagnosis.

2.2. Measures

2.2.1. Computerized UCSD Performance-Based Skills Assessment (C-UPSA)

We developed a computerized version of the original UPSA (Patterson et al., 2001) using a Toshiba tablet laptop. (Please see supplementary materials for a selection of screen shots from this instrument.) The C-UPSA retained four of the five original subtests: Planning recreational activities, finances, communication, and transportation. The shopping subtest was not included because it was found to have the poorest loading on the full UPSA during factor analysis (Mausbach et al., 2007). Additionally, it required the visual stimulus of an entire pantry of food items. UPSA experts opined that graphics of the individual food items would be too small to present realistically on a computer screen (Patterson and Mausbach, 2009, personal communication). Raw scores on each subtest were converted to standard scores ranging from 0–25, and the four subscale scores were added to yield overall scores between 0–100. The C-UPSA took approximately 15 minutes to complete.

2.2.2. UCSD Performance-Based Skills Assessment (UPSA)

The original UPSA (Patterson et al., 2001) was used to test the parallel forms reliability of C-UPSA. As an indicator of functional capacity in people with schizophrenia, the UPSA requires participants to role-play functional behaviors in five domains encountered in the real world: planning recreational activities (outings), handling personal finances (e.g., calculating change), communication (using a telephone in role-plays), transportation (navigating bus routes using a schedule and scenarios presented by the tester), and household chores (creating a shopping list based on a recipe and a pantry full of grocery items). Raw scores on each subtest are transformed to standard scores (0–20), with an overall score ranging from 0–100. Higher scores indicate higher functional capacity. The UPSA has an interrater reliability coefficient of .91 and two-week test-retest reliability of .93 (Harvey et al., 2007).

2.2.3. Other measures

The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolf, 1998) was administered to evaluate overall neuropsychological status. Psychopathology was evaluated with the Positive and Negative Symptom Scale (PANSS; Kay, 1987).

2.3. Procedures

For the STEP study, participants with schizophrenia completed a comprehensive assessment of symptoms, cognitive status, and everyday functioning at their B&C facility or day treatment program at baseline and at 6-, 12- and 18-month follow-ups. We administered the C-UPSA within one to two months of participants’ 18-month assessments (except for three individuals who were administered the C-UPSA within 1–2 months of their 12-month follow-up). The HCs were administered the RBANS and C-UPSA at UCSD in one visit.

Instructions and test items on the C-UPSA remained identical to the UPSA. We developed a standard procedure for administration. Prior to allowing the participant to take the C-UPSA, an examiner explained the test’s format and provided a brief tutorial for using the tablet laptop. The tutorial included instructions on how to provide clearly recordable verbal responses and how to use the stylus. The C-UPSA required participants to listen to questions asked by the computer. Participants spoke into a microphone to make their verbal responses, and the responses were recorded for the examiner to score later. Non-verbal responses were made using a stylus. Headphones were provided if necessary. A repeat button was available for most items, which allowed for the participant to have the question repeated if necessary. To maintain consistency with the original UPSA, the repeat button allows participants to repeat each item only once. Participants were informed of this option during the pre-test tutorial. Scoring of non-verbal responses was immediately completed using a standard algorithm. After a participant completed the test, the examiner listened to the audio responses and scored items according to original UPSA scoring procedures. The examiners augmented the scoring record with narrative descriptions of difficulties or unusual incidents that occurred during testing.

2.4. Statistical analyses

Data were examined for normality and three sets of analyses were conducted. First, t-tests for continuous variables and χ2 tests for categorical variables were performed to determine group differences. Next, a receiver operating characteristic (ROC) curve was plotted for the C-UPSA. The ROC curve was used to determine the optimal cutoff score by producing a Yuden’s index value (one minus specificity subtracted from sensitivity; Fluss et al., 2005; Loong, 2003). The area under curve (AUC) with 95% confidence intervals was used as an indicator of the ability of the C-UPSA to differentiate between schizophrenia patients and HCs. A critical p-value of 0.05 was used in all analyses.

3. Results

Characteristics of both groups are presented in Table 1. One participant with schizophrenia was unable to complete the C-UPSA due to severe anxiety; there were no other tolerability issues with the test. The two groups did not differ on age or race. The schizophrenia participants were predominantly male, resulting in significant gender differences. Additionally, the average education for the schizophrenia group was significantly lower than that of the HCs, which is consistent with the schizophrenia literature. Demographic characteristics of our schizophrenia sample were comparable to those of the original UPSA validation study (Patterson et al., 2001). Overall, the schizophrenia group scored significantly lower than HCs on the C-UPSA total score and two subtests (finances and transportation). Among the schizophrenia group, the C-UPSA total score was correlated with UPSA total (r = 0.73, p < 0.001) and UPSA-Brief scores (UPSA-B, r = 0.71, p < 0.001) to a degree comparable to the test-retest reliability of the original UPSA (r = 0.63–0.80; Leifker et al., 2010). The relationship between C-UPSA and RBANS scores was also significant in the schizophrenia group (r = 0.77, p < 0.001) and were comparable to the strength of the relationship between UPSA and RBANS scores (r = 0.78, p < 0.001). However, C-UPSA and RBANS were not related in the HC group (r = 0.07, p = 0.78). Positive and negative symptoms of schizophrenia were not correlated with either C-UPSA (positive symptoms: r = −0.17, p = 0.45; negative symptoms: r = −0.28, p = 0.21) or UPSA total scores (positive symptoms: r = −0.18, p = 0.43; negative symptoms: r = − 0.26, p = 0.26).

Table 1.

Descriptive statistics for demographic variables and test scores for the comparison and schizophrenia groups

Normal comparison group (n = 20) Schizophrenia patients (n = 21) σ 2 Df p
Gender
 % Male 35% (n = 7) 76% (n = 16) 7.06 1 0.01
 % Female 65% (n = 13) 24% (n = 5)
Ethnicity
 % White 80% (n = 16) 76% (n = 16) 3.98 4 0.41
 % Black 5% (n = 1) 14% (n = 3)
 % Asian 10% (n = 2) 0% (n = 0)
 % Hispanic 5% (n = 1) 5% (n = 1)
 % American Indian 0% (n = 0) 5% (n = 1)
Living situation
 % Board and care — — 90.5% (n = 19) — — —
 % Private house or apartment — — 9.5% (n = 2)
Mean SD Mean SD t Df p
Age (years) 58.75 (12.10) 54.52 (4.09) 1.51 39 0.14
Education (years) 16.37 (3.04) 12.60 (2.09) 4.53 37 < 0.001
RBANS Total Score 97.00 (8.94) 71.14 (14.39) 6.74 38 < 0.001
UPSA Total Score 1 — 72.86 (16.97) — — —
C-UPSA Total Score 1 76.07 (9.04) 57.49 (13.83) 5.06 39 < 0.001
C-UPSA planning recreational activities 2 19.81 (3.79) 19.17 (3.93) 0.53 38 0.60
C-UPSA finances 2 19.43 (3.50) 12.03 (5.84) 4.90 39 < 0.001
C-UPSA communication 2 15.64 (3.28) 13.16 (4.47) 2.02 39 0.05
C-UPSA transportation 2 21.27 (4.37) 13.33 (5.67) 4.88 37 < 0.001
Open in a new tab

Notes. RBANS = Repeatable Battery for the Assessment of Neuropsychological Status; UPSA = UCSD Performance-Based Skills Assessment; C-UPSA = Computerized UCSD Performance-Based Skills Assessment

1

Possible range for total score: 0–100.

2

Possible range of scores for each subscale: 0–25.

To determine the optimal C-UPSA value for discriminating between the schizophrenia and HC groups, we used truncated C-UPSA scores to generate areas under the ROC curves. Sensitivity, specificity, Yuden’s index value, positive predictive value (PPV), negative predictive value (NPV), and percent correctly identified by group for different cutoff scores for the C-UPSA are presented in Table 2. Sensitivity refers to C-UPSA’s ability to identify people with schizophrenia, whereas specificity refers to its ability to correctly identify HCs (Loong, 2003). PPV is the proportion of participants who scored below the cutoff value (< 75) who actually had schizophrenia, and NPV is the proportion of participants who scored above the cutoff who did not have schizophrenia (Altman and Bland, 1994). The ROC curve is shown in Figure 1. The estimated AUC for the C-UPSA was 0.88 (95% CI: 0.78–0.99), which was significantly greater (p < 0.001) than the area of no information (AUC of 0.50). A cutoff score of 75 best discriminated between the two groups, with a sensitivity of 0.70 and specificity of 0.95.

Table 2.

Psychometric Properties of C-UPSA

Cutoff:
65 70 75 a 80 85
Sensitivity 0.85 0.75 0.70 0.40 0.10
Specificity 0.76 0.86 0.95 1.00 1.00
Youden’s Index 0.61 0.61 0.65 0.40 0.10
PPV 0.77 0.83 0.93 1.00 1.00
NPV 0.84 0.78 0.77 0.64 0.54
% correctly identified 80.49 80.49 82.93 70.73 56.10
Open in a new tab

Notes. PPV = positive predictive value (proportion of controls with higher scores who were correctly classified as controls); NPV = negative predictive value (proportion of schizophrenia patients with lower scores who were correctly classified as having schizophrenia).

a

Maximum sensitivity and specificity

Figure 1.

Figure 1

Open in a new tab

ROC curve for the detection of C-UPSA scores

Note. C-UPSA = Computerized UCSD Performance-Based Skills Assessment

4. Discussion

This study describes the development and validation of a computerized version of a test of functional capacity. The preliminary data show concurrent validity for the C-UPSA as a comparable measure to the original UPSA. In a sample of community-dwelling middle-aged and older adults with schizophrenia, C-UPSA scores were largely correlated with scores on both the original UPSA and the UPSA-B, demonstrating the potential utility of the computerized measure to research investigating this population. As expected, the C-UPSA’s positive correlation with overall cognitive status provided additional support for its concurrent validity. Specificity of the C-UPSA was 0.95, indicating that 95% of the HCs scored 75 or higher on the C-UPSA. The ROC curve demonstrated the ability of the C-UPSA to discriminate between individuals with and without schizophrenia; it is not recommended for diagnostic purposes.

Relationships with cognitive status and psychopathology were explored to determine whether the C-UPSA was associated with the same factors as the UPSA. As expected, both forms of the UPSA were highly correlated with cognitive status, and the strength of the association did not differ between the two tests (C-UPSA and RBANS: r = 0.77; UPSA and RBANS: r = 0.78). Similar to relationships with the RBANS, correlations with both UPSA tests and positive and negative symptoms were practically identical. The non-significant findings with positive symptoms and UPSA tests are consistent with previously published UPSA validation studies; however, the non-significant relationship between negative symptoms and both forms of the UPSA is contrary to our hypothesis and to the literature (for e.g., Bowie, Reichenberg, Patterson, Heaton, & Harvey, 2006; Lysaker et al., 2011; Mausbach, Harvey, Goldman, Jeste, & Patterson, 2007; B. T. Mausbach et al., 2010; Twamley et al., 2002). However, the strength of the relationship between UPSA scores and negative symptoms in our study was comparable to those published in the aforementioned studies (this study: C-UPSA and negative symptoms: r = −0.28, UPSA and negative symptoms: r = −.26; range identified in published studies: range of r = −.29 to −.36), and we may have not had adequate power for our findings to be significant.

The lack of correlation between C-UPSA and cognition in our control group warrants a brief discussion. The lack of correlation between cognition and C-UPSA performance in the comparison group likely reflects the restricted range of performance on these measures in the healthy sample. Similarly low correlations with cognitive performance among healthy individuals have been observed in previous studies that used the UPSA (unpublished observations), suggesting that this null result is not unique to the computerized version.

There are several advantages to establishing the C-UPSA as a tool to assess functional capacity in both clinical and research settings. Specifically, whereas the original UPSA requires both human resources (i.e., an examiner to administer all the items and record all responses in person) and props, the C-UPSA automatically records all verbal and non-verbal responses, making it self-administered and requiring an administrator’s involvement only at the beginning (to deliver standardized oral instructions and go through the tutorial with the participant). The C-UPSA could therefore result in significant labor savings compared to the original UPSA. Another advantage of the C-UPSA is that participants do not need to travel to special testing sites; a laptop with a tablet surface is the only equipment needed. This increase in accessibility may optimize enrollment by allowing participants with limited transportation to participate in studies. Another beneficial feature of the C-UPSA is its partial use of automated scoring, which reduces random scoring errors and overall scoring time, thereby increasing efficiency and accuracy.

Strengths of this study include its contribution to the measurement of functional capacity in patients with schizophrenia using innovative advances in technology. An understudied area, the use of technology may lead to further insight into the magnitude of performance-based deficits associated with schizophrenia. Limitations include a relatively small sample size, which may reduce power and limit generalizability. Another limitation is that we did not administer the original UPSA to the HCs, so we were unable to obtain direct evidence for the relationship between C-UPSA and UPSA scores in participants with intact cognitive ability. In addition, although the use of the C-UPSA holds substantial promise, certain caveats suggested by previous reviews of computerized assessments must be considered (Hofer and Green, 1985; Kane and Kay, 1992). For example, computer-related factors have the potential to influence performance and change the nature of the task. The use of a computer-based approach may be intimidating for some individuals, and lack of familiarity with computers may negatively affect performance. Furthermore, degree of experience using computers can vary across demographic groups, including age, socioeconomic status, and education level. The possible confound of familiarity with computers was not controlled for in this study, but certain steps can be taken to address these issues in future validation studies (Kane and Kay, 1992). One proposed method to prevent limited experience with computers from affecting performance on the C-UPSA is to encourage testing staff to ensure that participants become well-acquainted with the test interface before administration.

In sum, this study supports the incorporation of computer technology into the administration and scoring of assessment of functional capacity. Evidence already exists for the usefulness of the traditional UPSA over non-performance-based measures in predicting community reintegration and residential independence (Mausbach et al., 2008). Given the relative advantages of the C-UPSA, including its shorter duration, easier administration, the elimination of examiner bias, and a more accurate scoring method, we believe it is a useful substitute for the original test. Directions for future research include testing the C-UPSA in other populations such as patients with bipolar disorder, Alzheimer’s disease, and HIV infection, and implementing the C-UPSA in longitudinal studies to evaluate its predictive validity and to determine the temporal stability of participants’ performance over time.

Supplementary Material

01

Acknowledgments

The study team gratefully acknowledges the assistance of Thanhnha Nguyen, who developed the software for the C-UPSA, and Brian Kelly, who edited the manuscript and prepared it for submission.

Role of funding source

Funding for this study was provided by the National Institute of Mental Health (NIMH) through awards R01 MH084967, R01 MH078737, T32 MH019934, and P30 MH080002. The funding agency had no further role in the study design; in the collection, analysis or interpretation of data; in the writing of the report; or in the decision to submit the paper for publication.

Footnotes

Conflict of interest

None to declare.

Contributors

Raeanne Moore helped with design of the C-UPSA, collected data, carried out the statistical analyses, and wrote the paper. Alexandrea Harmell assisted with collecting the data and writing the paper. Jennifer Ho assisted with writing the paper. Tom Patterson conceptualized the design of the C-UPSA, provided consultation on data collection, and assisted with writing the paper. Lisa Eyler and Dilip Jeste assisted with writing the paper. Brent Mausbach helped with the conceptualization and design of the C-UPSA, supervised data collection and statistical analyses, and assisted with writing the paper. All authors contributed to and have approved the final manuscript.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Contributor Information

Raeanne C. Moore, Email: r6moore@ucsd.edu.

Alexandrea L. Harmell, Email: aharmell@ucsd.edu.

Jennifer Ho, Email: jennifer.ho.1987@gmail.com.

Thomas L. Patterson, Email: tpatterson@ucsd.edu.

Lisa T. Eyler, Email: lteyler@ucsd.edu.

Dilip V. Jeste, Email: djeste@ucsd.edu.

Brent T. Mausbach, Email: bmausbach@ucsd.edu.

References

  1. Altman D, Bland J. Diagnostic tests 2: predictive values. BMJ. 1994;309:102. doi: 10.1136/bmj.309.6947.102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Atkinson M, Zibin S, Chuang H. Characterizing quality of life among patients with chronic mental illness: A critical examination of the self-report methodology. Am J Psychiatry. 1997;154:99–105. doi: 10.1176/ajp.154.1.99. [DOI] [PubMed] [Google Scholar]
  3. Bauer RM, Iverson GL, Cernich AN, Binder LM, Ruff RM, Naugel RI. Computerized neuropsychological assessment devices: joint position paper of the American Academy of Clinical Neuropsychology and the National Academy of Neuropsychology. Clin Neuropsychol. 2012;26 (2):177–196. doi: 10.1080/13854046.2012.663001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bowie CR, Reichenberg A, Patterson TL, Heaton RK, Harvey PD. Determinants of real-world functional performance in schizophrenia subjects: correlations with cognition, functional capacity, and symptoms. Am J Psychiatry. 2006;163 (3):418–425. doi: 10.1176/appi.ajp.163.3.418. [DOI] [PubMed] [Google Scholar]
  5. Bowie CR, Leung WW, Reichenberg A, McClure MM, Patterson TL, Heaton RK, Harvey PD. Predicting schizophrenia patients’ real-world behavior with specific neuropsychological and functional capacity measures. Biol Psychiatry. 2008;63 (5):505–511. doi: 10.1016/j.biopsych.2007.05.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bowie CR, Depp C, McGrath JA, Wolyniec P, Mausbach BT, Thornquist MH, et al. Prediction of real-world functional disability in chronic mental disorders: A comparison of schizophrenia and bipolar disorder. Am J Psychiatry. 2010;167 (9):1116–1124. doi: 10.1176/appi.ajp.2010.09101406. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Cardenas V, Abel S, Bowie C, Tiznado D, Depp CA, Patterson TL, et al. When functional capacity and real-world functioning converge: the role of self-efficacy. Schizophr Bull. 2012 Feb 10; doi: 10.1093/schbul/sbs004. in press. [e-pub ahead of print] [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Cohen CI, Cohen GD, Blank K, Gaitz C, Katz IR, Leuchter A, et al. Schizophrenia and older adults: an overview: directions for research and policy. Am J Geriatr Psychiatry. 2000;8 (1):19–28. doi: 10.1097/00019442-200002000-00003. [DOI] [PubMed] [Google Scholar]
  9. Depp CA, Mausbach BT, Eyler LT, Palmer BW, Cain AE, Lebowitz BD, et al. Performance-based and subjective measuers of functioning in middle-aged and older adults with bipolar disorder. J Nerv Ment Dis. 2009;197:471–475. doi: 10.1097/NMD.0b013e3181ab5c9b. [DOI] [PubMed] [Google Scholar]
  10. Depp CA, Mausbach BT, Granholm E, Cardenas V, Ben-Zeev D, Patterson TL, et al. Mobile interventions for severe mental illness: design and preliminary data from three approaches. J Nerv Ment Dis. 2010;198 (10):715–721. doi: 10.1097/NMD.0b013e3181f49ea3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Doniger GM, Zucker DM, Schweiger A, Dwolatzky T, Chertkow H, Crystal H, et al. Towards practical cognitive assessment for detection of early dementia: a 30-minute computerized battery discriminates as well as longer testing. Curr Alzheimer Res. 2005;2 (2):117–124. doi: 10.2174/1567205053585792. [DOI] [PubMed] [Google Scholar]
  12. Doniger GM, Dwolatzky T, Zucker DM, Chertkow H, Crystal H, Schweiger A, et al. Computerized cognitive testing battery identifies mild cognitive impairment and mild dementia even in the presence of depressive symptoms. Am J Alzheimers Dis Other Demen. 2006;21 (1):28–36. doi: 10.1177/153331750602100105. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Fluss R, Faraggi D, Reiser B. Estimation of the Youden Index and its associated cutoff point. Biom J. 2005;47 (4):458–472. doi: 10.1002/bimj.200410135. [DOI] [PubMed] [Google Scholar]
  14. Friedl KE, Grate SJ, Proctor SP, Ness JW, Lukey BJ, RLK Army research needs for automated neuropsychological tests: monitoring soldier health and performance status. Arch Clin Neuropsychol. 2007;22 (S):S7–14. doi: 10.1016/j.acn.2006.10.002. [DOI] [PubMed] [Google Scholar]
  15. Harvey PD, Velligan DI, Bellack AS. Performance-based measures of functional skills: Usefulness in clinical treatment studies. Schizophr Bull. 2007;33 (5):1138–1148. doi: 10.1093/schbul/sbm040. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Harvey PD, Helldin L, Bowie CR, Heaton RK, Olsson A, Hjarthag F, et al. Performance-based measurement of functional disability in schizophrenia: a cross-national study in the United States and Sweden. Am J Psychiatry. 2009;166:821–827. doi: 10.1176/appi.ajp.2009.09010106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Hofer PJ, Green BF. The challenge of competence and creativity in computerized psychological testing. J Consult Clin Psychol. 1985;53 (6):826. doi: 10.1037//0022-006x.53.6.826. [DOI] [PubMed] [Google Scholar]
  18. Iverson GL, Brooks BL, Langenecker SA, Young AH. Identifying a cognitive impairment subgroup in adults with mood disorders. J Affect Disord. 2011;132:360–367. doi: 10.1016/j.jad.2011.03.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Jeste DV, Palmer BW, Appelbaum PS, Golshan S, Glorioso D, Dunn LB, et al. A new brief instrument for assessing decisional capacity for clinical research. Arch Gen Psychiatry. 2007;64 (8):966–974. doi: 10.1001/archpsyc.64.8.966. [DOI] [PubMed] [Google Scholar]
  20. Kane RL, Kay GG. Computerized assessment in neuropsychology: a review of tests and test batteries. Neuropsychol Rev. 1992;3 (1):1–117. doi: 10.1007/BF01108787. [DOI] [PubMed] [Google Scholar]
  21. Kay S, Fiszbein A, Lewis OA. The Positive and Negative Syndrome Scale (PANSS) for schizophrenia. Schizophr Bull. 1987;13 (2):261–276. doi: 10.1093/schbul/13.2.261. [DOI] [PubMed] [Google Scholar]
  22. Keefe RSE, Poe M, Walker TM, Harvey PD. The relationship of the Brief Assessment of Cognition in Schizophrenia (BACS) to functional capacity and real-world functional outcome. J Clin Exp Neuropsychol. 2006;28 (2):260–269. doi: 10.1080/13803390500360539. [DOI] [PubMed] [Google Scholar]
  23. Leifker FR, Patterson TL, Bowie CR, Mausbach BT, Harvey PD. Psychometric properties of performance based measurements of functional capacity: test-retest reliability, practice effects, and potential sensitivity to change. Schiz Res. 2010;119 (1–3):246–252. doi: 10.1016/j.schres.2010.03.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Loong T. Understanding sensitivity and specificity with the right side of the brain. BMJ. 2003;327:716–719. doi: 10.1136/bmj.327.7417.716. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Lysaker PH, McCormick BP, Snethen G, Buck KD, Hamm JA, Grant M, Dimaggio G. Metacognition and social function in schizophrenia: associations of mastery with functional skills competence. Schizophrenia Research. 2011;131(1–3):214–218. doi: 10.1016/j.schres.2011.06.011. [DOI] [PubMed] [Google Scholar]
  26. Mausbach BT, Harvey PD, Goldman SR, Jeste DV, Patterson TL. Development of a brief scale of everyday functioning in persons with serious mental illness. Schizophr Bull. 2007;33 (6):1364–1372. doi: 10.1093/schbul/sbm014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Mausbach BT, Bowie CR, Harvey PD, Twamley EW, Goldman SR, Jeste DV, et al. Usefulness of the UCSD Performance-based Skills Assessment (UPSA) for predicting residential independence in patients with chronic schizophrenia. J Psychiatr Res. 2008;42:320–327. doi: 10.1016/j.jpsychires.2006.12.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Mausbach BT, Moore RC, Bowie C, Cardenas V, Patterson TL. A review of instruments for measuring functional recovery in those diagnosed with psychosis. Schizophr Bull. 2009;35 (2):307–318. doi: 10.1093/schbul/sbn152. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Mausbach BT, Harvey PD, Pulver AE, Depp CA, Wolyniec P, Thornquist MH, et al. Relationship of the Brief UCSD Performance-Based Skills Assessment (UPSA-B) to multiple indicators of functioning in people with schizophrenia and bipolar disorder. Bipolar Disord. 2010;12 (1):45–55. doi: 10.1111/j.1399-5618.2009.00787.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Mausbach BT, Depp CA, Bowie CR, Harvey PD, McGrath JA, Thronquist MH, et al. Sensitivity and specificity of the UCSD Performance-Based Skills Assessment (UPSA-B) for identifying functional milestones in schizophrenia. Schiz Res. 2011;132 (2–3):165–170. doi: 10.1016/j.schres.2011.07.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. McClure MM, Bowie CR, Patterson TL, Heaton RK, Weaver C, Anderson H, et al. Correlations of functional capacity and neuropsychological performance in older patients with schizophrenia: evidence for specificity of relationships? Schiz Res. 2007;89 (1):330–338. doi: 10.1016/j.schres.2006.07.024. [DOI] [PubMed] [Google Scholar]
  32. Palmer BW, Heaton SC, Jeste DV. Older patients with schizophrenia: challenges in the coming decades. Psychiatr Serv. 1999;50 (9):1178–1183. doi: 10.1176/ps.50.9.1178. [DOI] [PubMed] [Google Scholar]
  33. Patterson TL, Goldman S, McKibbin CL, Hughs T, Jeste DV. UCSD performance-based skills assessment: development of a new measure of everyday functioning for severely mentally ill adults. Schizophr Bull. 2001;27 (2):235–245. doi: 10.1093/oxfordjournals.schbul.a006870. [DOI] [PubMed] [Google Scholar]
  34. Randolf C. Repeatable Battery for the Assessment of Neuropsychological Status. San Antonio, Texas: Psychological Corporation; 1998. [Google Scholar]
  35. Reichenberg A. The assessment of neuropsychological functioning in schizophrenia. Dialogues Clin Neurosci. 2010;12 (3):383–392. doi: 10.31887/DCNS.2010.12.3/areichenberg. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Sabbag S, Twamley EW, Vella L, Heaton RK, Patterson TL, Harvey PD. Assessing everyday functioning in schizophrenia: not all informants seem equally informative. Schiz Res. 2011;131:250–255. doi: 10.1016/j.schres.2011.05.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Sabbag S, Twamley EW, Vella L, Heaton RK, Patterson TL, Harvey PD. Predictors of the accuracy of self assessment of everyday functioning in people with schizophrenia. Schizophr Res. 2012;137 (1–3):190–195. doi: 10.1016/j.schres.2012.02.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Slade E, Salkever D. Symptom effects on employment in a structural model of mental illness and treatment: Analysis of patients with schizophrenia. J Ment Health Policy Econ. 2001;4 (1):25–34. [PubMed] [Google Scholar]
  39. Sweeney JA, Kmiec JA, Kupfer DJ. Neuropsychologic impairments in bipolar and unipolar mood disorders on the CANTAB neurocognitive battery. Biol Psychiatry. 2000;48 (7):674–684. doi: 10.1016/s0006-3223(00)00910-0. [DOI] [PubMed] [Google Scholar]
  40. Twamley EW, Doshi RR, Nayak GV, Palmer BW, Golshan S, Heaton RK, et al. Generalized cognitive impairments, ability to perform everyday tasks, and level of independence in community living situations of older patients with psychosis. Am J Psychiatry. 2002;159 (12):2013–2020. doi: 10.1176/appi.ajp.159.12.2013. [DOI] [PubMed] [Google Scholar]
  41. Wu EQ, Birnbaum HG, Shi L, Ball DE, Kessler RC, Moulis M, et al. The economic burden of schizophrenia in the United States in 2002. J Clin Psychiatry. 2005;66 (9):1122–1129. doi: 10.4088/jcp.v66n0906. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

01
NIHMS435923-supplement-01.docx (1.1MB, docx)

RESOURCES