Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2012 Dec 1.
Published in final edited form as: Community Ment Health J. 2010 Jul 30;47(6):622–627. doi: 10.1007/s10597-010-9343-z

Predictors of Participation in Community Outpatient Psychosocial Rehabilitation in Schizophrenia

Matthew M Kurtz 1,✉, Jennifer Rose 2, Bruce E Wexler 3
PMCID: PMC3046324  NIHMSID: NIHMS233204  PMID: 20676766

Abstract

This study investigated demographic, clinical and neurocognitive factors predicting drop-out from an intensive, community outpatient psychosocial rehabilitation program for people with schizophrenia or schizoaffective disorder. One-hundred and twenty-seven outpatients with DSM-IV schizophrenia or schizoaffective disorder participated. Demographic variables of age, sex, education and race/ethnicity were recorded and formal symptom measures and a neurocognitive assessment consisting of measures of crystallized verbal ability, sustained visual vigilance, verbal learning, verbal fluency and problem-solving were administered at study entry. Thirty-seven percent of the sample dropped-out of the program. In a final multivariate model, younger age, and lower verbal fluency scores in clients with a history of a high number of hospitalizations predicted a greater likelihood of drop-out. The implications of these findings are discussed.

Keywords: Schizophrenia, Rehabilitation, Drop-out, Neurocognition, Treatment outcome

Introduction

The development and application of more refined psychosocial rehabilitation strategies, when administered in conjunction with appropriate pharmacotherapy, has lead to renewed optimism regarding the potential for managing psychotic symptoms and enhancing community functioning for people diagnosed with schizophrenia. For example, the Patient Outcomes Research Team (PORT) program, initiated by the US Department of Health and Human Services, investigated the empirical support for a variety of psychosocial treatments for schizophrenia (Lehman and Steinwachs 1998; Lehman et al. 2004). With respect to behavioral treatments, PORT results showed that a variety of psychosocial interventions, including cognitive-behavioral psychological interventions, family interventions lasting 9 months or more, supported employment, and skills training that includes behaviorally-based instruction, modeling and corrective feedback, all have strong levels of evidence that support their implementation in psychosocial rehabilitation (PSR) programs for individuals diagnosed with schizophrenia. Novel behavioral interventions that target traditionally treatment refractory domains of the illness, such as neurocognitive impairment, have also shown promise (e.g., McGurk et al. 2007).

Despite these findings, high-rates of early drop-out from PSR community day programs that treat clients with severe mental illness and include interventions recommended by PORT are common, with rates ranging from mid-20% (e.g., Harding et al. 2008) to nearly 50% (e.g., Cohen et al. 1995) in some studies, with differences most likely linked to sample diagnoses, definitions for attrition and other individual and program factors. These findings are important as studies that have followed drop-outs from PSR programs have typically shown poorer employment and rehospitalization outcomes (Harding et al. 2008; Beard et al. 1978; Bond 1992; Cook and Razzano 1995) for these clients. These findings emphasize the importance of identifying factors that may be associated with early drop-out, to help develop methods for enhancing participation in PSR activities. Enhancing PSR retention can both potentially improve outcome for clients who drop-out, and provide a better assessment of whether current evidence-based interventions shown to benefit clients who complete treatment, also benefit those clients who typically drop-out of studies of outpatient psychosocial interventions.

Demographic factors that have been linked to PSR attrition include age (Young et al. 2000), and education (Harding et al. 2008), while chronic psychosis (Atwood and Beck 1985), hostile, labile or bizarre affect (Cohen et al. 1995) and total positive symptoms (Harding et al. 2008) are symptom factors that have been linked to attrition (but see Primm et al. 2000 for an exception). There is growing evidence that drop-out rates from PSR programs vary between individuals with a diagnosis of schizophrenia-spectrum illness and other diagnoses (e.g., Harding et al. 2008; Young et al. 2000), and thus the factors influencing drop-out for clients with schizophrenia remain unclear.

Taken together, studies of client factors that may influence attrition of clients with severe mental illness in PSR programs are sparse, have often not used formalized and reliable, clinician-rated symptom assessments (but see Harding et al. 2008 for an exception), and have focused on samples of severely mentally ill clients that include a range of diagnoses with different patterns of attrition. No studies have included standardized measures of neurocognition as predictors of attrition despite the growing recognition of neurocognition as a key factor influencing psychosocial function in severe mental illness (e.g., Green et al. 2004). The current study was designed to contribute to this literature by evaluating a greater range of individual factors (including standardized and reliable symptom assessment, as well as measures of neurocognition) that might predict likelihood of attrition in a homogenous sample of clients with a diagnosis of schizophrenia or schizoaffective disorder treated in a community, 3-day per-week outpatient PSR program that consisted of structured group therapy and skills training, exercise, family therapy, vocational rehabilitation and computer training. Based on previous studies, we predicted that age and education, positive symptoms and hostility and lability would be linked to drop-out from the program. We also hypothesized that measures of elementary neurocognition, including attention, verbal memory, problem-solving and verbal fluency and processing speed would be linked to greater likelihood of attrition.

Methods

The study was conducted at The Institute of Living, a large outpatient facility in Hartford, CT. The study used neuropsychological, clinical and demographic assessment data derived from a randomized, controlled study (RCT) comparing the effectiveness of two types of computer-assisted training for cognitive difficulties in people with schizophrenia (Kurtz et al. 2007) administered within a comprehensive outpatient rehabilitation program consisting of a broad array of psychosocial and vocational services. Clients in the current study were referred for comprehensive rehabilitation treatment from other clinical inpatient and outpatient units at The Institute of Living, as well as community care providers, and were enrolled in the clinical PSR program first, and then subsequently recruited for the cognitive training study. With the exception of one client, every client referred for PSR treatment over the 8 year study period (2001–2008) gave written, informed consent and was enrolled in the computer-assisted cognitive training research study as part of customary procedures on the unit. No participants during this 8 year period referred for PSR treatment were entered into the PSR program but excluded from the research study on the basis of research study inclusion/exclusion criteria.

Participants

One-hundred and twenty-seven (98 males) consumers meeting DSM-IV (APA 1994) criteria for schizophrenia or schizoaffective disorder as determined by the Structured Clinical Interview for DSM-IV (First et al. 1995) participated. Enrollment in the rehabilitation program was continuous over a period of 8 years (2001–2008). Mean age of the sample was 32.1 years (SD: 10.8), with a mean education of 13.1 years (SD: 2.2) and 9.0 years (SD: 9.1) of illness.

Day Treatment Program

The 3 day per week outpatient program consisted of once-per-program day structured group therapy, life-skills training, physical exercise, and computer training, along with individualized vocational counseling. Families of clients enrolled in the program were offered once-per-month therapy in a multi-family format.

Attrition Status

Clients were categorized for their attrition status as “completers” if they completed at least a month of outpatient rehabilitation groups and completed the clinic recommended follow-up assessment before program discharge. If a client terminated from the outpatient program after completing less than a month of rehabilitation groups (as measured by less than 10 h of computer training) and/or they refused the clinic recommended follow-up assessment they were labeled “drop-out”. None of the clients in the “dropout” group were judged by the clinical staff to have completed their outpatient rehabilitation satisfactorily and in all cases, the staff believed the client would have benefitted from continued rehabilitation treatment.

Demographic and Clinical Information

Age, race and ethnicity, education, parental education, sex, duration of illness, number of hospitalizations and disease age-of-onset were collected as part of the intake interview and a review of the medical record.

Neurocognitive Measures

Measures of crystallized verbal intelligence, (Vocabulary subtest from the WAIS-III and IV; Wechsler 1997, 2008), visual vigilance (Penn CPT; Rosvold et al. 1956; Kurtz et al. 2001), verbal memory (CVLT-II; Delis et al. 2000), problem-solving (Penn Conditional Exclusion Test; Kurtz et al. 2004a, b) and verbal fluency (COWA-FAS, Spreen and Benton 1977), were selected as these specific measures have been commonly linked to progress in psychosocial rehabilitation programs (e.g., Green et al. 2004; Kern et al. 1992; McGurk et al. 2003).

Symptom Measures

The Positive and Negative Syndrome Scale (PANSS; Kay et al. 1987) was used to assess symptoms at entry to the study. Symptom raters for the study maintained interrater reliability through periodic rater training sessions, and all raters were trained to a criterion reliability of .8 intraclass correlation coefficient (ICC), across four jointly viewed, but independently rated interviews. Scores were grouped into the following categories based on the commonly reported five-factor solution (Bell et al. 1994): (1) positive, (2) negative, (3) hostility, (4) emotional discomfort, and (5) cognitive impairment. These categories were selected as independent variables. Symptom ratings and neurocognitive assessment and scoring were supervised by a doctoral-level psychologist.

Data Analysis

Data were evaluated for normality. In no case was there evidence that variables included in the study violated the assumptions underlying the use of parametric statistical procedures. We investigated relationships between demographic, clinical, and neurocognitive measures at study entry to attrition status after outpatient psychiatric rehabilitation in two steps following the procedure recommended by Hosmer and Lemenshow (1989). In the first step we conducted a series of univariate logistic regressions between demographic measures of age, race and ethnicity, education, parental education, clinical measures of number of hospitalizations, age of onset, and the five symptom factors from the PANSS, and the neurocognitive variables of crystallized verbal ability (Vocabulary subtest of the WAIS-III or IV), sustained visual vigilance (PCPT true positives), verbal learning (total scores from the CVLT-II), problem-solving (PCET errors) and language and processing speed (verbal fluency) to group status (completer vs. drop-out). The purpose of this step was to identify the variables that were potentially related to study attrition to be included in a subsequent multivariate analysis. In step 2, we then tested a hierarchical multivariate logistic regression model that first included predictors with P-values of .25 or less in the univariate analyses and/or demographic variables that were presumed to interact with the cognitive variables (age and sex), and then added interactions between the demographic, clinical and neurocognitive variables to assess whether the clinical or demographic variables moderated the relationship between neurocognitive variables and drop out. Prior to analysis, we centered the variables in order to improve interpretability and reduce potential multicollinearity when the interaction terms were included in the model (Aiken and West 1991). Results are presented for a final multivariate analysis that excluded nonsignificant main effects and interactions. Significant interactions were interpreted using methods outlined by Aiken and West (1991). Simple slope coefficients were calculated for the relation of neuropsychological functioning to drop out at low (−1 SD), medium (mean) and high (+1 SD) levels for each demographic and clinical variable that significantly interacted with neuropsychological functioning. Alpha was set at .05 and all statistical tests were two-tailed.

Results

In terms of attrition, 47 of 127 participants (37%) dropped out of this intensive outpatient rehabilitation program. Univariate logistic regression analyses showed that of the demographic measures, older participants (odds ratio (OR) = .95; P = .010) were significantly less likely to drop out. Among neuropsychological measures, participants with higher scores on Vocabulary from the WAIS (OR = .98; P = .040), and verbal fluency measures (OR = .97; P = .030) were significantly less likely to drop out. None of the clinical variables were linked to attrition status.

Variables with a P-value of .25 or less in the univariate logistic regressions (age, race, sex, number of hospitalizations, education, age of onset, the hostility factor from the PANSS, and Vocabulary from the WAIS-III or IV) were entered into a multivariate logistic regression model. Results for the individual predictors revealed that age (OR = .93; P = .029) and verbal fluency (OR = .95; P = .042) remained significantly related to drop out in the multivariate model. In addition, an increased number of hospitalizations were marginally related to an increased likelihood of drop out when other variables related to dropout were accounted for. The only significant interaction was between number of hospitalizations and verbal fluency (P = .038). Verbal fluency was unrelated to drop out when the number of hospitalizations was low (OR for simple slope = 1.00, P = .816). However, verbal fluency was most strongly related to drop out when the number of hospitalizations was high (OR for simple slope = .91, P = .011) such that the likelihood of drop out decreased with increasing verbal fluency scores. The Nagelkerke R2 for the multivariate model was .206.

Discussion

This is the first study, to our knowledge, to investigate the relative role of demographic variables, symptoms, and neurocognition for predicting attrition from an intensive, community cognitive and psychosocial rehabilitation program in a diagnostically homogenous group of clients with schizophrenia or schizoaffective disorder. The attrition rate for the study was 37% which is generally consistent with other studies of attrition from outpatient treatment utilizing mixed diagnostic groups of clients with severe mental illness (e.g. Harding et al. 2008; Cohen et al. 1995; Atwood and Beck 1985), despite wide inter-study differences in definition of attrition and the characteristics of outpatient rehabilitation treatment. For example Harding et al. (2008) reported a drop-out rate (26.3%) in a study of SMI clients in a PSR program focused on two types of vocational training, with higher drop-out rates in the subgroup of clients in their study diagnosed with a psychotic disorder. In the current study, demographic, clinical and neurocognitive factors were linked to attrition status. More specifically, age at start of treatment, number of hospitalizations, and verbal fluency were linked to the ability of clients with schizophrenia to participate in community-based intensive cognitive and social rehabilitation programs, even when other demographic, neurocognitive and symptom variables were accounted for. Race and ethnicity, sex, education, parental education, age-of-onset, symptoms and cognitive factors of sustained attention, verbal memory and problem-solving were not related to attrition status from outpatient rehabilitation in the current study.

An important finding was the moderating effects of number of hospitalizations on the relationship between verbal fluency and attrition status. Poorer verbal fluency was linked to a greater likelihood of drop-out in clients with a high number of hospitalizations, and thus most likely, a more virulent form of the illness. In contrast, in clients with a milder disease course, verbal fluency was unrelated to attrition status.

One speculative hypothesis that may be advanced for these findings is that illness insight may be mediating the obtained findings: younger clients with less experience with the illness, and thus poorer illness insight are more likely to drop-out of PSR programs. Similarly, clients with poorer insight as evidenced by a higher number of hospitalizations and thus, presumably poorer medication compliance, are more likely to drop-out of PSR programs. Furthermore, these findings may also suggest that for those clients with high numbers of hospitalizations, verbal fluency may be serving as a proxy for the ability of the client to engage in psychosocial rehabilitation groups and acquire strategies from these groups for maintaining treatment engagement and community status during times of symptom exacerbation. Clients with poorer verbal skills in the context of high illness severity may be unable to acquire these skills and thus drop-out of PSR programs at a more rapid rate.

The obtained findings failed to reveal a relationship between symptoms and attrition. These findings are generally consistent with two reports (Atwood and Beck 1985; Primm et al. 2000) and inconsistent with two other reports. Cohen et al. (1995) noted a relationship between clinical impressions of affect, specifically bizarre, hostile and labile affect and attrition status in their study of 112 clients with severe mental illness followed over 8-weeks of PSR treatment. Standardized clinical rating scales, however, were not used in this study and may explain the disparate findings. More recently, Harding et al. (2008) in a sample of 194 patients with severe mental illness, showed links between positive symptoms as measured by the PANSS, and attrition as defined by drop-out before completion of 6-months of a vocationally-oriented PSR program. Potential explanation for this discrepancy in findings include the possibility that positive symptoms may play a smaller role for predicting drop-out for clients with schizophrenia relative to other forms of severe mental illness, and that the mean level of positive symptoms in the current sample (mild range, mean = 2.9) may have been low relative to their sample. We also note that the link of number of hospitalizations to attrition status suggests that history of intensity, rather than current levels of positive symptoms, are predictive of attrition status.

Five caveats to the current findings should be noted. First, the results of this study pertain to the internal characteristics of individuals diagnosed with schizophrenia or schizoaffective disorder that affect completion of a community PSR program, not the influence of program characteristics that might affect attrition, or the interaction of individual client characteristics with program characteristics that might relate to ideal person-treatment program fit. Second, we did not measure outcome of clients who dropped-out of treatment, and thus it remains unknown whether these clients fared better or worse than program completers. Nonetheless, (1) in all cases in the current study drop-outs left the program against clinical recommendation of the PSR staff, and (2) a range of studies indicates poorer outcome in clients with severe mental illness who do not complete PSR programs (Harding et al. 2008; Beard et al. 1978; Bond 1992; Cook and Razzano 1995; Young et al. 2000). Third, we did not study participants with active substance-abuse, which has been linked to attrition from PSR in some studies (e.g., Drake et al. 1998), but not others (e.g., Harding et al. 2008). And fourth, the R2 value for the final multivariate model suggests that our model, while statistically significant, explained a modest proportion of the variance in drop-out, and other factors, not measured in this study might be playing a role in drop-out from outpatient rehabilitation. Fifth, the sample was largely Caucasian with average estimated verbal IQ and a high-school education and thus may reflect a high-functioning group of consumers that is not optimally representative of all individuals with the disorder. Related to this concern, and consistent with previous studies, there was evidence of a trend towards a higher drop-out rate in the subgroup of African-American (AA) clients enrolled in the PSR program (50 vs. 34% for Caucasian clients). The small sample size of AA clients (n = 18 out of 127 study clients), however, precluded meaningful subgroup analysis. Future, more highly powered studies will help elucidate factors that may differentially impact attrition from PSR programs among clients of different race/ethnicity.

Acknowledgments

This work was supported by grant K08 MH-69888 from the National Institute of Mental Health (NIMH), and a Young Investigator Award from the National Alliance for Research on Schizophrenia and Depression (NARSAD).

Footnotes

The authors report no conflicts of interest.

Contributor Information

Matthew M. Kurtz, Email: mkurtz@wesleyan.edu, Department of Psychology, Wesleyan University, Judd Hall, 207 High Street, Middletown, CT 06459, USA. Department of Psychiatry, Yale School of Medicine, New Haven, CT 06511, USA

Jennifer Rose, Department of Psychology, Wesleyan University, Judd Hall, 207 High Street, Middletown, CT 06459, USA.

Bruce E. Wexler, Department of Psychiatry, Yale School of Medicine, New Haven, CT 06511, USA

References

  1. Aiken LS, West SG. Multiple regression: Testing and interpreting interactions. Newbury, CA: Sage Publications; 1991. [Google Scholar]
  2. APA, DSM-IV. Diagnostic and statistical manual of mental disorders. 4. Washington, DC: APA Press; 1994. [Google Scholar]
  3. Atwood N, Beck JC. Service and patient predictors of continuation in clinic-based treatment. Hospital and Community Psychiatry. 1985;36:865–869. doi: 10.1176/ps.36.8.865. [DOI] [PubMed] [Google Scholar]
  4. Beard JH, Malamud TJ, Rossman E. Psychiatric rehabilitation and long-term rehospitalization rates: The findings of two research studies. Schizophrenia Bulletin. 1978;4:622–635. doi: 10.1093/schbul/4.4.622. [DOI] [PubMed] [Google Scholar]
  5. Bell MD, Lysaker PH, Beam-Goulet JL, Milstein RM, Lindenmayer JP. Five-component model of schizophrenia: Assessing the factorial invariance of the positive and negative syndrome scale. Psychiatry Research. 1994;52:295–303. doi: 10.1016/0165-1781(94)90075-2. [DOI] [PubMed] [Google Scholar]
  6. Bond GR. Vocational rehabilitation. In: Liberman RP, editor. Handbook of Psychiatric Rehabilitation. New York: Macmillan; 1992. pp. 244–275. [Google Scholar]
  7. Cohen K, Eldstrom K, Smith-Papke L. Identifying early dropouts from a rehabilitation program for psychiatric outpatients. Psychiatric Services. 1995;46:1076–1078. doi: 10.1176/ps.46.10.1076. [DOI] [PubMed] [Google Scholar]
  8. Cook J, Razzano L. Discriminant function analysis of competitive employment outcomes in a transitional employment program for persons with severe mental illness. Journal of Vocational Rehabilitation. 1995;5:127–139. [Google Scholar]
  9. Delis DC, Kramer JH, Kaplan E, Ober BA. CVLT-II. New York: The Psychological Corporation; 2000. [Google Scholar]
  10. Drake RE, Mercer-McFadden C, Mueser KT, McHugo GJ, Bond GR. Treatment of substance abuse in patients with severe mental illness: A review of recent research. Schizophrenia Bulletin. 1998;24:589–608. doi: 10.1093/oxfordjournals.schbul.a033351. [DOI] [PubMed] [Google Scholar]
  11. First MB, Spitzer RL, Gibbon M, Williams BW. Structured clinical interview for DSM-IV axis I disorders-patient edition (SCID-I/P, version 2.0) New York: Biometrics Research Department, New York State Psychiatric Institute; 1995. [Google Scholar]
  12. Green MF, Kern RS, Heaton RK. Longitudinal studies of cognition and functional outcome in schizophrenia: Implications for MATRICS. Schizophrenia Research. 2004;72(1):41–51. doi: 10.1016/j.schres.2004.09.009. [DOI] [PubMed] [Google Scholar]
  13. Harding B, Torres-Harding S, Bond GR, Salyers MP, Rollins AL, Harding T. Factors associated with early attrition from a psychosocial rehabilitation programs. Community Mental Health Journal. 2008;44:283–288. doi: 10.1007/s10597-008-9128-9. [DOI] [PubMed] [Google Scholar]
  14. Hosmer DW, Lemenshow S. Applied logistic regression. New York: John Wiley and Sons; 1989. [Google Scholar]
  15. Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin. 1987;13(2):261–276. doi: 10.1093/schbul/13.2.261. [DOI] [PubMed] [Google Scholar]
  16. Kern RS, Green MF, Satz P. Neuropsychological predictors of skills training for chronic psychiatric patients. Psychiatry Research. 1992;43:223–230. doi: 10.1016/0165-1781(92)90055-8. [DOI] [PubMed] [Google Scholar]
  17. Kurtz MM, Ragland JD, Bilker W, Gur RC, Gur RE. Comparison of the continuous performance test with and without working memory demands in healthy controls and patients with schizophrenia. Schizophrenia Research. 2001;48(2–3):307–316. doi: 10.1016/s0920-9964(00)00060-8. [DOI] [PubMed] [Google Scholar]
  18. Kurtz MM, Ragland JD, Moberg PJ, Gur RC. The penn conditional exclusion test: A new measure of executive-function with alternate forms of repeat administration. Archives of Clinical Neuropsychology. 2004a;19(2):191–201. doi: 10.1016/S0887-6177(03)00003-9. [DOI] [PubMed] [Google Scholar]
  19. Kurtz MM, Seltzer JC, Shagan DS, Thime WR, Wexler BE. Computer-assisted cognitive remediation in schizophrenia: What is the active ingredient? Schizophrenia Research. 2007;89(1–3):251–260. doi: 10.1016/j.schres.2006.09.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Kurtz MM, Wexler BE, Bell MD. The penn conditional exclusion test (PCET): Relationship to the Wisconsin card sorting test and work function in patients with schizophrenia. Schizophrenia Research. 2004b;68(1):95–102. doi: 10.1016/S0920-9964(03)00179-8. [DOI] [PubMed] [Google Scholar]
  21. Lehman AF, Kreyenbuhl J, Buchanan RW, et al. The schizophrenia outcomes research team (PORT): Updated treatment recommendations. Schizophrenia Bulletin. 2004;30:193–217. doi: 10.1093/oxfordjournals.schbul.a007071. [DOI] [PubMed] [Google Scholar]
  22. Lehman AF, Steinwachs DM. Translating research into practice: The schizophrenia outcomes research team (PORT) treatment recommendations. Schizophrenia Bulletin. 1998;24:1–10. doi: 10.1093/oxfordjournals.schbul.a033302. [DOI] [PubMed] [Google Scholar]
  23. McGurk SR, Mueser KT, Harvey PD, LaPuglia R, Marder J. Cognitive and symptom predictors of work outcome for clients with schizophrenia in supported employment. Psychiatric Services. 2003;54:1129–1135. doi: 10.1176/appi.ps.54.8.1129. [DOI] [PubMed] [Google Scholar]
  24. McGurk SR, Twamley EW, Sitzer DI, McHugo GJ, Mueser KT. A meta-analysis of cognitive remediation in schizophrenia. American Journal of Psychiatry. 2007;164:1791–1802. doi: 10.1176/appi.ajp.2007.07060906. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Primm AB, Gomez MB, Tzolova-Iontchev I, et al. Severely mentally ill patients with and without substance use disorders. Characteristics associated with treatment attrition. Community Mental Health Journal. 2000;36:235–246. doi: 10.1023/a:1001953030073. [DOI] [PubMed] [Google Scholar]
  26. Rosvold HE, Mirsky AF, Sarason I, Bransome ED, Jr, Beck LH. A continuous performance test of brain damage. Journal of Consulting and Clinical Psychology. 1956;20:343–350. doi: 10.1037/h0043220. [DOI] [PubMed] [Google Scholar]
  27. Spreen O, Benton AL. Neurosensory center for comprehensive examination for aphasia (NCCEA) Victoria: University of Victoria Neuropsychology Laboratory; 1977. [Google Scholar]
  28. Wechsler D. WAIS-III manual. New York: The Psychological Corporation; 1997. [Google Scholar]
  29. Wechsler D. WAIS-IV manual. New York: The Psychological Corporation; 2008. [Google Scholar]
  30. Young AS, Grusky O, Jordan D, Belin TR. Routine outcome monitoring in a public mental health system: The impact of patients who leave care. Psychiatric Services. 2000;51:85–91. doi: 10.1176/ps.51.1.85. [DOI] [PubMed] [Google Scholar]

RESOURCES