Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2008 Jan 1.
Published in final edited form as: Schizophr Res. 2006 Oct 3;89(1-3):110–118. doi: 10.1016/j.schres.2006.08.005

PREVALENCE AND CORRELATES OF ADEQUATE PERFORMANCE ON A MEASURE OF ABILITIES RELATED TO DECISIONAL CAPACITY: DIFFERENCES AMONG THREE STANDARDS FOR THE MACCAT-CR IN PATIENTS WITH SCHIZOPHRENIA

Laura B Dunn 1,2, Barton W Palmer 1, Paul S Appelbaum 3, Elyn R Saks 4, Gregory A Aarons 1,5, Dilip V Jeste 1,2
PMCID: PMC1850805  NIHMSID: NIHMS15964  PMID: 17020803

Abstract

Despite the availability of structured decision-making capacity assessment tools, insufficient guidance exists for applying their results. Investigators often use cutpoints on these instruments to identify potential subjects in need of further assessment or education. Yet, information is lacking regarding the effects of different cutpoints on the proportion and characteristics of individuals categorized as possessing adequate or impaired decisional abilities for consent to research. To demonstrate the potential impact of different standards, we informed 91 individuals, aged 50 or older with a diagnosis of schizophrenia or schizoaffective disorder, about a hypothetical clinical trial, and assessed their decisional abilities with the MacArthur Competence Assessment Tool for Clinical Research (MacCAT-CR). Three published MacCAT-CR-based standards were applied to participants’ scores to examine the rates and correlates of categorical determinations of adequate performance. The three standards ranged in stringency: the most stringent incorporated cutpoints on all three of the major MacCAT-CR subscales (Understanding, Appreciation, and Reasoning); the other two standards required threshold performance only on the Understanding subscale. The most stringent standard resulted in a 57% rate of impaired performance; the intermediate standard, 19%; and the least stringent standard, 8%. Nearly half of the participants (n=45) were classified as having performed adequately by the least stringent standard yet inadequately by the most stringent. The majority of these 45 were impaired on the Appreciation subscale (n=9), Reasoning (n=15), or both (n=18). Cognitive functioning was correlated with performance status for the more stringent standards. These findings underscore the need for refinement of capacity assessment procedures and for improvements in the use of capacity assessment tools for screening purposes and to assist in categorical capacity determinations.

Keywords: Research ethics, decision-making capacity, competency, clinical trials, schizophrenia, neuropsychology

1. Introduction

In recent years, there have been frequent suggestions that, in at least some contexts or for certain clinical populations, investigators should systematically evaluate whether potential research participants are competent to consent (National Bioethics Advisory Commission, 1998; UCSD Human Research Protections Program, 2004). The MacArthur Competence Assessment Tool for Clinical Research (MacCAT-CR) (Appelbaum and Grisso, 2001) is generally recognized as the best of currently available scales for this purpose (Dunn et al., 2006c; Sturman, 2005); it is also the most widely used in empirical research on consent capacity (Jeste et al., 2006; Vellinga et al., 2004).

Adequate performance on the MacCAT-CR was used in one large-scale clinical trial as part of the criteria for independent consent (Stroup et al., 2003). In most studies, however, capacity assessment has been limited to asking participants a few questions about their understanding of the study. Even in the aforementioned trial, the inclusion criteria focused solely on the MacCAT-CR Understanding subscale, without considering the other three generally recognized dimensions of decisional capacity (Appelbaum and Roth, 1982) also measured on the MacCAT-CR (appreciation of the significance of the information, reasoning with the information, and expressing a choice).

Because of the contextual nature of decisional capacity (Saks et al., 2006), the MacCAT-CR has no established cut-score or algorithm for categorical determinations of capacity or incapacity. This is appropriate, as studies vary in level of risk and in the risk/benefit ratio. General consensus exists that as the degree of risk increases, a higher level of capacity is desirable (Roberts and Dyer, 2004). Thus, no particular level of ability is determinative of adequate capacity in all circumstances (Appelbaum and Grisso, 2001). Moreover, scores on capacity assessment instruments, though helpful, should generally be supplemented with other important information, such as mental status and decision-making context (Appelbaum and Grisso, 2001).

Moreover, discussion of cutpoints is inseparable from considerations of capacity assessment serving a screening function, and from growing evidence regarding educational interventions for enhancing consent. Substantively inadequate responses to MacCAT-CR items indicate a need for further consideration of an individual’s decisional abilities, and perhaps for intervention prior to enrollment—rather than as justification for making a definitive categorical decision about decisional capacity. Given that healthy controls often score less than perfectly on the MacCAT-CR (Jeste et al., 2006), perfect MacCAT-CR performance would be an untenable criterion. Rather, each investigator (and/or institutional review board, or IRB) must make some a priori decisions about what level and type of performance constitute “adequate” or “inadequate” performance, raising several key questions: How much understanding is “enough”? Are certain items particularly critical? Even when understanding is adequate, what weight should be given to less-than-perfect performance on appreciation and/or reasoning?

These issues go beyond application and interpretation of the MacCAT-CR, reflecting genuine lack of consensus regarding what defines adequate consent capacity and under what conditions. Absent such consensus, or empirical or regulatory guidance, investigators and IRBs may make idiosyncratic decisions that may not represent a contextually appropriate balance among the goals of protecting potentially vulnerable persons, fostering important research, and respecting individuals’ decision-making autonomy.

The goal of the present study was to evaluate the effects of altering some of these factors (requisite level of understanding, and additional consideration of appreciation and reasoning) on the determination of “adequate” performance on one measure related to decisional abilities (the MacCAT-CR). Although no single cut-score or algorithm on the MacCAT-CR is likely to be appropriate across all protocols or populations, there have been several MacCAT-CR cut-scores published in the literature for specific contexts (Carpenter et al., 2000; Kim et al., 2001; Stroup et al., 2005). We applied these three criteria to our sample to document the extent of disparity in the proportion of subjects categorized as having inadequate performance, and the specific sources of discrepancies under the three criteria. Our goal was not to validate any of these for general use, but rather to make explicit the sources of disagreement to foster discussion and stimulate further empirical research regarding the dimensions and degree of decisional capacity appropriately viewed as essential under differing circumstances.

2. Methods

2.1. Participants

included 91 individuals, aged ≥ 50 years, with schizophrenia (n=58) or schizoaffective disorder (n=33), enrolled in a larger study on informed consent and capacity to consent to research among older persons with psychoses. Inclusion criteria were: DSM-IV diagnosis of schizophrenia or schizoaffective disorder (determined by the patient’s clinician), current age ≥ 50 years, fluency in English, and the absence of a diagnosis of dementia. Recruitment sources included board-and-care residences, county psychiatric clinics, and inpatient and outpatient psychiatric services of the University of California, San Diego Medical Center, and the Veterans Affairs San Diego Healthcare System.

2.2. Human subjects

This study was reviewed and approved by our local IRB, and each participant provided written informed consent prior to enrollment. All participants were able to express basic understanding of the purpose, procedures, risks, the voluntary nature of participation, and an awareness that the described trial was hypothetical.

2.3. Measures and Procedures

2.3.1. Stimulus protocol

Participants were provided with detailed information regarding a simulated randomized, placebo-controlled, Phase III clinical drug trial. The described risks were modeled on studies of atypical antipsychotic medications.

2.3.2. MacCAT-CR

All participants completed the MacCAT-CR using the standardized administration procedures (Appelbaum and Grisso, 2001). The MacCAT-CR consists of a semi-structured interview that provides four subscale scores corresponding to the four commonly recognized domains of decisional capacity (understanding, appreciation, reasoning, and expression of a choice) (Appelbaum and Roth, 1982). As previously reported (Dunn et al., 2006a), interrater reliability (intraclass correlation coefficients [ICCs]) established with protocols from 15 randomly selected participants was ICC= 0.98 (Understanding), ICC=0.84 (Appreciation), and ICC=0.78 (Reasoning).

2.4. Three operational definitions of adequate versus inadequate MacCAT-CR performance

The following operational definitions were compared:

2.4.1. Understanding > 15 (out of 26 possible points)

This criterion was the primary criterion for independent consent in the federally-sponsored Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) schizophrenia study (Stroup et al., 2003), and has been evaluated in three prior studies of decisional capacity (McDermott et al., 2005; Moser et al., 2002; Palmer et al., 2005). This cutpoint was set by the study investigators based on their own best judgment as to what would comprise adequate MacCAT-CR performance. While a criterion of 16 of 26 points seems a relatively low level of requisite understanding, it should be noted that the CATIE study was a low-risk naturalistic treatment study.

2.4.2. Understanding ≥ 20

This standard was evaluated in a study of decisional capacity among people with schizophrenia (Carpenter et al., 2000), and represented the median performance of healthy comparison subjects. In that study it was used as a decision point for determining which participants would receive an educational session relevant to making a decision about participation in clinical trials research, not as a means of determining which persons had de facto impaired decisional capacity.

2.4.3. Multidimensional criterion (Understanding ≥ 18 ∩ Appreciation ≥ 5 ∩ Reasoning ≥ 6)

These cutpoints were empirically derived by Kim et al. (2001) relative to the capacity judgments of three independent expert-psychiatrist raters, in reference to a hypothetical clinical drug trial for Alzheimer Disease (AD) patients. Although the procedures, risks, and benefits of a typical AD drug trial may differ from those typical in schizophrenia drug trials, this is the only published study in which a combination of cutpoints on the three major subscales was empirically derived via use of expert judgments about patient capacity. Again, our purpose in evaluating the application of this criterion was not to categorize patients definitively as “decisionally impaired,” but rather to determine how different thresholds affect the proportion of people deemed as having inadequate MacCAT-CR performance, and the specific factors underlying discrepant categorizations.

2.5. Other participant characteristics

Demographic data were collected via interview and/or record review. Severity of psychopathology was measured with the Positive and Negative Syndrome Scale (PANSS) (Kay et al., 1987), 17-item Hamilton Depression Rating Scale (HAM-D) (Hamilton, 1960), and the Birchwood Insight Scale (Birchwood et al., 1994) (for the latter, higher scores indicate better insight; for the PANSS and HAM-D, higher scores correspond to worse symptomatology).

Cognitive functioning was assessed with the total score from Mattis’ Dementia Rating Scale (DRS) (Mattis, 1973), and using the mean z-score from five neuropsychological tests [Hopkins Verbal Learning Test - Revised (HVLT-R; total recall trials 1 through 3) (Benedict et al., 1998), the Brief Visual-Spatial Memory Test - Revised (BVMT-R; total recall trials 1 through 3) (Benedict, 1997), Similarities and Matrix Reasoning (standard scores) (Wechsler, 1997), and the Wisconsin Card-Sorting Test - 64 (WCST-64; conceptual level responses) (Kongs et al., 2000)]. Cognitive scores were all scaled so that higher scores indicate better performance.

2.6. Statistical analyses

Variables not meeting assumptions for parametric analyses were transformed using log or square root functions. The relationship of MacCAT-CR performance (“adequate performance” coded as = 1; “inadequate performance” coded as = 0) to continuous variables was evaluated with Spearman’s rho, and with Pearson’s chi-square (or Fisher’s exact test, when appropriate) for categorical variables. Significance was defined as p<.05 (two-tailed).

3. Results

Demographic characteristics, MacCAT-CR subscale scores, psychopathology ratings, and cognitive test scores are shown in Table 1. The mean psychopathology rating scales and cognitive scores suggest on average participants had mild symptoms, and mild-to-moderate cognitive impairment.

Table 1.

Demographic, Clinical, Neuropsychological, and Decision-Making Characteristics (n=91, except as indicated)

Characteristic Mean (SD) n (%)
Age, years 56.0 (5.0)
Education, years 12.7 (2.6)
Gender
 Male 63 (69.2)
 Female 28 (30.8)
Diagnosis
 Schizophrenia 58 (63.7)
 Schizoaffective disorder 33 (36.3)
Outpatient status 83 (91.2)
Ethnicity
 Caucasian 73 (80.2)
 African-American 7 (7.7)
 Hispanic/Latino 8 (8.8)
 Asian/other 3 (3.3)
MacCAT-CR Subscales (possible range)
 Understanding (0-26) 22.5 (4.4)
 Appreciation (0-6) 4.7 (1.5)
 Reasoning (0-8) 5.4 (2.0)
 Choice (0-2) 1.9 (0.3)
PANSS (possible range)
 Positive subscale (7-49) 14.6 (5.4)
 Negative subscale (7-49) 14.0 (5.9)
 General subscale (16-112) 28.6 (8.2)
HAM-D (17-item version) (possible range 0-50) 10.4 (6.7)
Birchwood Insight Scale (possible range 0-12)1 8.6 (2.8)
DRS raw scores (n=90) (possible and observed ranges)
 Total (possible 0-144; observed 83-142) 128.8 (10.3)
 Attention (possible 0-37; observed 28-37) 34.5 (2.0)
 Initiation/Perseveration (possible 0-37; observed 14-37) 32.4 (4.7)
 Construction (possible 0-6; observed 2-6) 5.3 (1.1)
 Conceptualization (possible 0-39; observed 3-39) 34.8 (4.9)
 Memory (possible 0-25; observed 8-25) 21.6 (3.1)
Mean cognitive z-score2 -1.27 (0.80)
Open in a new tab

Notes: MacCAT-CR = MacArthur Competence Assessment Tool - Clinical Research PANSS = Positive and Negative Syndrome Scale; HAM-D = Hamilton Depression Rating Scale DRS = Mattis' Dementia Rating Scale

1

The Birchwood Insight Scale was modified from the original; two of the three components comprising the total score differ because one of the questions in the original scale concerned awareness of the need for hospitalization, whereas the majority of our participants were outpatients.

2

The “Composite cognitive z-score” was created by taking the mean of the total of z-scores for performance on the following tests: 1) Hopkins Verbal Learning Test - Revised (Benedict et al., 1998) (HVLT-R, Form1, 2001 Norms, Total Recall t-score); 2) Brief Visual-Spatial Memory Test - Revised (Benedict, 1997) (BVMT-R Form 1, Total Recall, t-score), 3) and 4) Similarities and Matrix Reasoning subtests from the WAIS-III (Wechsler, 1997) (age-corrected, scaled scores); and 5) the Perseverative Responses (t-score) from the Wisconsin Card-Sorting Test (WCST) (Kongs et al., 2000).

Using the standards of MacCAT-CR Understanding >15 and Understanding ≥ 20, 92.3% and 81.3% of participants had adequate performance, respectively, but under the multidimensional standard incorporating Appreciation and Reasoning cutpoints, only 42.9% of participants were categorized as having adequate performance. Forty-five of 91 participants with adequate performance by the Understanding >15 standard were categorized as having inadequate performance using the multidimensional standard; 35 of these 45 persons had Reasoning scores < 6, 20 of whom also had Appreciation scores < 5 (n=16), Understanding < 18 (n=1) or both (n=3) (Figure 1). The decrement in performance comparing the multidimensional standard with the least stringent Understanding standard is almost entirely due to the Reasoning and Appreciation criteria, not to the somewhat stricter Understanding cut-off (18 or greater) in the multidimensional standard.

Figure 1.

Figure 1

Open in a new tab

Types of impairments among 45 participants classified as displaying adequate performance on the least stringent standard but as impaired on the most stringent standard

Correlations of patient demographic, clinical, and cognitive characteristics with MacCAT-CR performance under each standard are shown in Table 2. With one modest exception for education, there were no significant correlations of MacCAT-CR performance (under any of the three standards) with age, education, or severity of positive symptoms. In contrast, the DRS total score and mean cognitive z-score were significantly correlated with MacCAT-CR performance for the Understanding ≥ 20 standard, and for the multidimensional standard. Severity of negative symptoms and insight level also showed significant, albeit modest effect size correlations with performance status under two or more of the three standards.

Table 2.

Correlations between Demographic, Clinical, and Neuropsychological Variables and Classifications of Adequate Decisional Performance on Standards of Varying Stringency (n=91 except where indicated)

Characteristic Stringency of Standard Spearman’s rho
Age Least -0.136
Intermediate -0.062
Most -0.038
Education Least 0.089
Intermediate 0.241, p=0.021*
Most 0.024
PANSS Positive Least -0.126
Intermediate -0.169
Most 0.058
PANSS Negative Least -0.249, p=0.017*
Intermediate -0.313, p=0.003**
Most -0.205
PANSS General Least -0.105
Intermediate -0.302, p=0.004**
Most -0.050
HAM-D (17-item) Least 0.028
Intermediate -0.143
Most 0.086
Birchwood Insight Scale Least 0.318, p=0.002**
Intermediate 0.252, p=0.016*
Most 0.336, p=0.001***
DRS Total score (n=90) Least 0.201
Intermediate 0.349, p=0.001***
Most 0.365, p<0.001***
Mean cognitive z-score1 (n=84) Least 0.178
Intermediate 0.362, p=0.001***
Most 0.407, p<0.001***
Open in a new tab

Notes: PANSS = Positive and Negative Syndrome Scale HAM-D = Hamilton Depression Rating Scale DRS = Mattis’ Dementia Rating Scale

1

The “Mean cognitive z-score” = mean of the total of z-scores for performance on the following tests: 1) Hopkins Verbal Learning Test - Revised (Benedict et al., 1998) (HVLT-R, Form1, 2001 Norms, Total Recall t-score); 2) Brief Visual-Spatial Memory Test - Revised (Benedict, 1997) (BVMT-R Form 1, Total Recall, t-score), 3) and 4) Similarities and Matrix Reasoning subtests from the WAIS-III (Wechsler, 1997) (age-corrected, scaled scores); and 5) the Perseverative Responses (t-score) from the Wisconsin Card-Sorting Test (WCST) (Kongs et al., 2000).

*

p<0.05

**

p<0.01

***

p•0.001

4. Discussion

Different standards for MacCAT-CR performance resulted in substantially different proportions of individuals with schizophrenia or schizoaffective disorder being categorized as having adequate or inadequate MacCAT-CR performance. Fully half of this sample of middle-aged and older people with schizophrenia and schizoaffective disorder fell into a gray zone of performance—“adequate” by the least stringent standard but “inadequate” by the more stringent and multidimensional standards. Adding cutpoints for performance on the appreciation and reasoning domains of decision-making capacity led to a much higher rate of “inadequate” performance.

These data imply that when instruments to assess abilities related to decisional capacity are used, investigators and IRBs should be alert to where cut-offs are set for further evaluation and education of potential subjects. Given the current paucity of data demonstrating the consequences of variations in applied standards, neglect of this issue has been understandable. However, as the selection of standards will have a substantial impact on which individuals are allowed to enroll in research absent further intervention, both investigators and IRBs should be expected to offer some defensible rationale for the standard selected. Such standards may be based on a priori notions of adequacy, experience with similar populations in other studies—or, increasingly, on empirical data demonstrating the reasonableness of particular criteria, judged against external benchmarks of adequate performance.

In practice, most capacity assessments in contemporary research focus on understanding alone. Even when a common understanding cut-score is applied, there appears to be wide variation in the proportion of patients with schizophrenia deemed as having inadequate performance. In their study of 25 patients with schizophrenia (including 17 inpatients), Moser et al. (2002) found 5 (20%) had MacCAT-CR Understanding scores ≤ 15. McDermott et al. (2005) found that among forensic psychiatric patients, 16% scored below 15 on the Understanding subscale. Approximately 8% of the current sample was deemed to have inadequate MacCAT-CR performance based on the least stringent criterion (Understanding ≤ 15), which is similar to the rate of 6% we observed in a previous sample (Palmer et al., 2005). In contrast, Stroup et al. (2005) reported that only 1.6% of 1,447 individuals in the CATIE schizophrenia study earned Understanding scores < 15. It is difficult to know what to make of the wide range in these proportions, but in addition to the need to define the appropriate levels of understanding, these results suggest a need for further empirical research on the effect of protocol complexity on level of MacCAT-CR performance, and whether different outcomes within individuals parallel the views of expert and relevant stakeholders regarding the conditions under which individuals should be deemed capable of providing independent consent. In short, greater consensus is needed on what constitutes “adequate understanding” and what person-centered and contextual factors affect the level of such understanding.

Another fundamental unresolved issue is whether and when to require threshold performance on the other dimensions typically viewed as part of the overall construct of competency to make a decision about research participation—particularly appreciation and reasoning. The data presented here may help guide further consideration of this issue. A substantial proportion of the participants had difficulty with the appreciation and/or reasoning domains of the MacCAT-CR. The wide discrepancy in the proportion of individuals deemed as having “adequate understanding” by the Understanding >15 criterion versus the multidimensional criterion was not simply an artifact of the slightly higher Understanding cut-score in the latter; even with the higher criterion of Understanding ≥ 20, only 19% of patients were categorized as having “inadequate performance” relative to the 57% so categorized under the multidimensional criterion. These differences raise a very important set of questions, the answers to which may profoundly affect the enrollment of patients in clinical trials research. Inclusion of questions related to appreciation and reasoning appears to result in a substantial increase in the number of persons who would be at least initially identified as requiring further scrutiny in terms of decisional capacity.

An explicit consensus is needed in bioethics and medical research regarding the importance of incorporating measures of appreciation and reasoning: When is understanding (assuming the level deemed “adequate” can be defined) alone sufficient? Does it make a difference if the study is an intervention trial, given that differences between methods of clinical research vs. standard clinical care are not always clear to research participants (Lidz and Appelbaum, 2002), with some participants needing greater explanation and clarification of these differences (Dunn et al., 2006a)? Does the level of risk, or the risk/benefit ratio, affect the importance of considering appreciation and/or reasoning abilities? There is a growing body of empirical data showing that understanding can often be improved through enhanced consent procedures (Dunn et al., 2006b), but at present, evidence for the potential for educational remediation strategies to improve scores on appreciation and reasoning is not as strong (Carpenter et al., 2000). Consensus that a basic level of appreciation and reasoning is also important would accelerate efforts to develop effective interventions, but in the short term, would also affect whether a substantial number of patients are permitted to give consent for research enrollment.

An important caveat regarding the present study is the absence of one or more non-psychiatrically ill comparison groups. Catalyzed by the 1998 National Bioethics Advisory Commission report (National Bioethics Advisory Commission., 1998), which unfortunately focused solely on those with mental disorders (Shore and Hyman, 1999), patients with schizophrenia have been disproportionately represented in empirical research on informed consent and decisional capacity. The message from the subsequent decade of research is clear: schizophrenia patients are among those at risk for impaired capacity, but such impairment is neither characteristic of the majority of such patients, nor is the risk limited to those with psychiatric conditions (Dunn et al., 2006b)(Appelbaum, 2006). The issues under consideration in the present study are by no means unique to people with schizophrenia or other neuropsychiatric conditions. Ignoring this caveat, risks further inappropriate stigmatization of those with serious mental illness, most of whom may remain capable of giving autonomous consent under most circumstances.

Also, the present data were collected as part of a study of consent for older patients with psychotic disorders, a focus motivated by recognition of the normal age-related changes in cognitive functioning (Christensen and Kumar, 2003), and the clear association between cognitive functioning and decisional capacity (Marson et al., 1996; Palmer and Jeste, 2006). Had we studied younger patients with schizophrenia, particularly more acutely ill individuals, we might have found different patterns of impairment under the three standards. There are also other important differences between younger and older adults with schizophrenia—e.g., substance use disorders are more common in the former group whereas physical comorbidity is more prevalent in older individuals. On the other hand, age might have less impact on decisional capacity than on some other domains of functioning, for the following reasons: (a) level of cognitive functioning in schizophrenia tends to remain remarkably stable over time (Heaton et al., 2001; Nayak et al., 2006; Palmer et al., 2003; Rund, 1998); (b) positive symptoms of schizophrenia have little discernible effect on decisional capacity (Palmer et al., 2004; Palmer and Jeste, 2006); and (c) in contrast to other medical or neuropsychiatric groups (Flory and Emanuel, 2004), our recent data (as replicated in the present study) have suggested that aging has little influence on decisional capacity among patients with schizophrenia (Palmer et al., 2004; Palmer et al., 2005; Palmer and Jeste, 2006). Whether our findings would apply as well to younger adults will need to be studied empirically.

Another important consideration regarding the use of appreciation and reasoning in screening for impaired decisional capacity is that instrument development for these constructs has not been as strong as that for understanding (Dunn et al., 2006). The MacCAT-CR’s Appreciation and Reasoning subscales appear less psychometrically robust than the Understanding subscale (Dunn et al., 2006), with the former two subscales having constricted variance due to fewer items. Our data point to the potential value of more extensive,, well-validated measures of these constructs, if their inclusion as part of the basis for assessing capacity, or for identifying people who could benefit from further education regarding research, is to be effectively operationalized.

Similarly, the choice of a particular instrument may affect the rate of findings of impairment. The MacCAT-CR was selected for this and other capacity studies (Kim et al., unpublished data) because it is currently the best-studied capacity assessment scale (Dunn et al., 2006b; Sturman, 2005). Other instruments use different approaches to the assessment of capacity, including some instruments that are briefer, more highly structured, or operationalize the various domains of capacity differently (Dunn et al., 2006b). These differences further underscore the need for assessment of the comparative benefits of different instruments as they are developed (Moye et al., 2005; Moye et al., 2004). In addition, the characteristics of a particular research site, the training of staff in instrument administration, and possibly even pressures to recruit participants, may affect participants’ performance and scores on measures of decisional abilities. For example, even in carefully designed and managed studies such as the CATIE trial, research site was the strongest predictor of variance in MacCAT-CR Understanding, Appreciation, and Reasoning scores (Stroup et al., 2005).

Part of deciding when more thorough capacity assessments are warranted involves considerations of the risk factors for impaired capacity. Regardless of the standard used, the correlates of MacCAT-CR status in this study generally mirrored those of prior literature on consent capacity in schizophrenia patients (Carpenter et al., 2000; Dunn et al., 2002; Jeste et al., 2006; Kovnick et al., 2003; Moser et al., 2004; Moser et al., 2002; Palmer et al., 2004; Palmer et al., 2005; Palmer and Jeste, 2006; Stroup et al., 2005; Wirshing et al., 2005; Wirshing et al., 1998). Cognitive test scores tended to be the strongest correlates of MacCAT-CR status; negative symptoms also related to MacCAT-CR status, with generally no significant effects of age, education, positive symptoms, or depressive symptoms. In contrast to other recent findings from our research group (Palmer and Jeste, 2006), level of insight was also (modestly) correlated with MacCAT-CR status. As insight as a construct seems to have conceptual overlap with the ability to appreciate the significance of information for one’s own circumstances; the moderate effect sizes between these two abilities warrants further investigation with psychometrically sound measures of both. However, there are now sufficient empirical data to suggest that investigators should be particularly attentive to potential impairments in decisional capacity when enrolling persons with notable cognitive deficits or negative symptoms.

Our findings point to several directions for further research, namely to examine: 1) whether any specific subsets of items could help identify people at highest risk for incapacity (viz. Moser et al., 2003, Palmer et al., 2005); 2) whether there are subpopulations of people represented differentially among those identified as having impaired performance, e.g., through person-centered analyses such as latent class analysis (Muthen and Muthen, 2000); 3) whether expert judgments of capacity identify the same individuals as the various cutpoints, and what accounts for differential findings (Dawes et al., 1989; Kim, 2006; Kim et al., 2001; McDermott et al., 2005); and 4) whether screening procedures identifying people at risk of impaired capacity due to appreciation and/or reasoning deficits (in addition to understanding) are useful, and whether educational interventions can improve these domains of capacity.

4.1. Conclusions

A structured assessment tool such as the MacCAT-CR can be useful for screening for decisional impairment, but the effects of setting different cutpoints or criteria for performance are substantial, and should be considered carefully for different studies in different populations. We believe that measures of appreciation and reasoning could prove useful for capacity screening, particularly for higher risk studies, in both psychiatric and non-psychiatric medical research. This suggestion is tempered by validity and reliability concerns regarding the measurement of these constructs (Dunn et al., 2006c). The need for refinement of capacity instruments remains; the effects of adding appreciation and reasoning questions to screening for capacity should be examined even as instrument development proceeds.

Acknowledgments

This work was supported by NIMH grants MH66062, MH66248, MH64722, by the National Alliance for Research on Schizophrenia and Depression, and by the VA San Diego Healthcare System. These data have been presented in part at the 2005 Annual Meeting of the American College of Neuropsychopharmacology, and at the 2006 Annual Meeting of the American Association for Geriatric Psychiatry.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Appelbaum PS. Decisional capacity of patients with schizophrenia to consent to research: taking stock. Schizophr. Bull. 2006;32:22–25. doi: 10.1093/schbul/sbi063. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Appelbaum PS, Grisso T. MacCAT-CR: MacArthur Competence Assessment Tool for Clinical Research. Professional Resource Press; Sarasota, FL: 2001. [Google Scholar]
  3. Appelbaum PS, Roth LH. Competency to consent to research: a psychiatric overview. Arch. Gen. Psychiatry. 1982;39:951–958. doi: 10.1001/archpsyc.1982.04290080061009. [DOI] [PubMed] [Google Scholar]
  4. Benedict R. Professional Manual. Psychological Assessment Resources, Inc.; Odessa, FL: 1997. Brief Visual-Spatial Memory Test-revised. [Google Scholar]
  5. Benedict R, Schretlen D, Groninger L, et al. Hopkins Verbal Learning Test-revised: Normative data and analysis of inter-form and test-retest reliability. Clin Neuropsychol. 1998;12:43–55. [Google Scholar]
  6. Birchwood M, Smith J, Drury V, et al. A self-report Insight Scale for psychosis: reliability, validity and sensitivity to change. Acta Psychiatr. Scand. 1994;89:62–67. doi: 10.1111/j.1600-0447.1994.tb01487.x. [DOI] [PubMed] [Google Scholar]
  7. Carpenter WT, Gold JM, Lahti AC, et al. Decisional capacity for informed consent in schizophrenia research. Arch. Gen. Psychiatry. 2000;57:533–538. doi: 10.1001/archpsyc.57.6.533. [DOI] [PubMed] [Google Scholar]
  8. Christensen H, Kumar R. Cognitive changes and the ageing brain. In: Sachdev PS, editor. The Ageing Brain: The Neurobiology and Neuropsychiatry of Ageing. Swets & Zeitlinger; Lisse, The Netherlands: 2003. pp. 75–95. [Google Scholar]
  9. Dawes RM, Faust D, Meehl PE. Clinical versus actuarial judgment. Science. 1989;243:1668–1674. doi: 10.1126/science.2648573. [DOI] [PubMed] [Google Scholar]
  10. Dunn L, Palmer BW, Keehan M, et al. Assessment of therapeutic misconception in older schizophrenia patients with a brief instrument. Am. J. Psychiatry. 163(2006a):500–506. doi: 10.1176/appi.ajp.163.3.500. [DOI] [PubMed] [Google Scholar]
  11. Dunn LB, Candilis PJ, Roberts LW. Emerging empirical evidence on the ethics of schizophrenia research. Schizophr. Bull. 32(2006b):47–68. doi: 10.1093/schbul/sbj012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Dunn LB, Lindamer LA, Palmer BW, et al. Improving understanding of research consent in middle-aged and elderly patients with psychotic disorders. Am. J. Geriatr. Psychiatry. 2002;10:142–150. [PubMed] [Google Scholar]
  13. Dunn LB, Nowrangi M, Palmer BW, et al. Assessing capacity to consent to treatment and research: a review of instruments. Am. J. Psychiatry. 163(2006c):1323–1334. doi: 10.1176/ajp.2006.163.8.1323. [DOI] [PubMed] [Google Scholar]
  14. Flory J, Emanuel E. Interventions to improve research participants’ understanding in informed consent for research: a systematic review. JAMA. 2004;292:1593–1601. doi: 10.1001/jama.292.13.1593. [DOI] [PubMed] [Google Scholar]
  15. Hamilton M. A rating scale for depression. J. Neurol. Neurosurg. Psychiatry. 1960;23:56–62. doi: 10.1136/jnnp.23.1.56. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Heaton RK, Gladsjo JA, Palmer BW, et al. The stability and course of neuropsychological deficits in schizophrenia. Arch. Gen. Psychiatry. 2001;58:24–32. doi: 10.1001/archpsyc.58.1.24. [DOI] [PubMed] [Google Scholar]
  17. Jeste DV, Depp CA, Palmer BW. Magnitude of impairment in decisional capacity in people with schizophrenia compared to normal subjects: an overview. Schizophr. Bull. 2006;32:121–128. doi: 10.1093/schbul/sbj001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr. Bull. 1987;13:261–276. doi: 10.1093/schbul/13.2.261. [DOI] [PubMed] [Google Scholar]
  19. Kim SY. When does decisional impairment become decisional incompetence? Ethical and methodological issues in capacity research in schizophrenia. Schizophr. Bull. 2006;32:92–97. doi: 10.1093/schbul/sbi062. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Kim SY, Caine ED, Currier GW, et al. Assessing the competence of persons with Alzheimer’s disease in providing informed consent for participation in research. Am. J. Psychiatry. 2001;158:712–717. doi: 10.1176/appi.ajp.158.5.712. [DOI] [PubMed] [Google Scholar]
  21. Kongs S, Thompson L, Iverson G, et al. Professional Manual. Psychological Assessment Resources, Inc.; Odessa, FL: 2000. Wisconsin Card Sorting Test-64 Card Version. [Google Scholar]
  22. Kovnick JA, Appelbaum PS, Hoge SK, et al. Competence to consent to research among long-stay inpatients with chronic schizophrenia. Psychiatr. Serv. 2003;54:1247–1252. doi: 10.1176/appi.ps.54.9.1247. [DOI] [PubMed] [Google Scholar]
  23. Lidz CW, Appelbaum PS. The therapeutic misconception: problems and solutions. Med. Care. 2002;40:V55–63. doi: 10.1097/01.MLR.0000023956.25813.18. [DOI] [PubMed] [Google Scholar]
  24. Marson DC, Chatterjee A, Ingram KK, et al. Toward a neurologic model of competency: Cognitive predictors of capacity to consent in Alzheimer’s disease using three different legal standards. Neurology. 1996;46:666–672. doi: 10.1212/wnl.46.3.666. [DOI] [PubMed] [Google Scholar]
  25. Mattis S. Dementia Rating Scale. Psychological Assessment Resources, Inc.; Odessa, FL: 1973. [Google Scholar]
  26. McDermott BE, Gerbasi JB, Quanbeck C, et al. Capacity of Forensic Patients to Consent to Research: The Use of the MacCAT-CR. J. Am. Acad. Psychiatry Law. 2005;33:299–307. [PubMed] [Google Scholar]
  27. Moser DJ, Arndt S, Kanz JE, et al. Coercion and informed consent in research involving prisoners. Compr. Psychiatry. 2004;45:1–9. doi: 10.1016/j.comppsych.2003.09.009. [DOI] [PubMed] [Google Scholar]
  28. Moser DJ, Schultz SK, Arndt S, et al. Capacity to provide informed consent for participation in schizophrenia and HIV research. Am. J. Psychiatry. 2002;159:1201–1207. doi: 10.1176/appi.ajp.159.7.1201. [DOI] [PubMed] [Google Scholar]
  29. Moye J, Gurrera RJ, Karel MJ, et al. Empirical advances in the assessment of the capacity to consent to medical treatment: Clinical implications and research needs. Clin. Psychol. Rev. 2005 doi: 10.1016/j.cpr.2005.04.013. [DOI] [PubMed] [Google Scholar]
  30. Moye J, Karel MJ, Azar AR, et al. Capacity to consent to treatment: empirical comparison of three instruments in older adults with and without dementia. Gerontologist. 2004;44:166–175. doi: 10.1093/geront/44.2.166. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Muthen B, Muthen LK. Integrating person-centered and variable-centered analyses: growth mixture modeling with latent trajectory classes. Alcohol. Clin. Exp. Res. 2000;24:882–891. [PubMed] [Google Scholar]
  32. National Bioethics Advisory Commission . Research involving persons with mental disorders that may affect decisionmaking capacity. National Bioethics Advisory Commission; Rockville, MD: 1998. [Google Scholar]
  33. Nayak GV, Moore DJ, Roesch SC, et al. An evaluation of longitudinal neurocognitive performance among middle-aged and older schizophrenia patients: Use of mixed-model analyses. Schizophr. Res. 2006;83:215–223. doi: 10.1016/j.schres.2005.12.851. [DOI] [PubMed] [Google Scholar]
  34. Palmer BW, Bondi MW, Twamley EW, et al. Are late-onset schizophrenia spectrum disorders neurodegenerative conditions? Annual rates of change on two dementia measures. J. Neuropsychiatry Clin. Neurosci. 2003;15:45–52. doi: 10.1176/jnp.15.1.45. [DOI] [PubMed] [Google Scholar]
  35. Palmer BW, Dunn LB, Appelbaum PS, et al. Correlates of treatment-related decision-making capacity among middle-aged and older patients with schizophrenia. Arch. Gen. Psychiatry. 2004;61:230–236. doi: 10.1001/archpsyc.61.3.230. [DOI] [PubMed] [Google Scholar]
  36. Palmer BW, Dunn LB, Appelbaum PS, et al. Assessment of capacity to consent to research among older persons with schizophrenia, Alzheimer disease, or diabetes mellitus: comparison of a 3-item questionnaire with a comprehensive standardized capacity instrument. Arch. Gen. Psychiatry. 2005;62:726–733. doi: 10.1001/archpsyc.62.7.726. [DOI] [PubMed] [Google Scholar]
  37. Palmer BW, Jeste DV. Relationship of individual cognitive abilities to specific components of decisional capacity among middle-aged and older patients with Schizophrenia. Schizophr. Bull. 2006;32:98–106. doi: 10.1093/schbul/sbj002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Roberts LW, Dyer AR. Concise Guide to Ethics in Mental Health Care. American Psychiatric Press, Inc.; Washington D.C.: 2004. [Google Scholar]
  39. Rund BR. A review of longitudinal studies of cognitive functions in schizophrenia patients. Schizophr. Bull. 1998;24:425–435. doi: 10.1093/oxfordjournals.schbul.a033337. [DOI] [PubMed] [Google Scholar]
  40. Saks ER, Dunn LB, Palmer BW. Meta-consent in research on decisional capacity: A “Catch-22”? Schizophr. Bull. 2006;32:42–46. doi: 10.1093/schbul/sbj017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Shore D, Hyman SE. An NIMH commentary on the NBAC report. Biol. Psychiatry. 1999;46:1013–1016. doi: 10.1016/s0006-3223(99)00228-0. [DOI] [PubMed] [Google Scholar]
  42. Stroup S, Appelbaum P, Swartz M, et al. Decision-making capacity for research participation among individuals in the CATIE schizophrenia trial. Schizophr. Res. 2005;80:1–8. doi: 10.1016/j.schres.2005.08.007. [DOI] [PubMed] [Google Scholar]
  43. Stroup TS, McEvoy JP, Swartz MS, et al. The National Institute of Mental Health Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) project: schizophrenia trial design and protocol development. Schizophr. Bull. 2003;29:15–31. doi: 10.1093/oxfordjournals.schbul.a006986. [DOI] [PubMed] [Google Scholar]
  44. Sturman ED. The capacity to consent to treatment and research: a review of standardized assessment tools. Clin. Psychol. Rev. 2005;25:954–974. doi: 10.1016/j.cpr.2005.04.010. [DOI] [PubMed] [Google Scholar]
  45. UCSD Human Research Protections Program Decision making capacity guidelines. 2004 Available online at: http://irb.ucsd.edu/decisional.shtml.
  46. Vellinga A, Smit JH, van Leeuwen E, et al. Instruments to assess decision-making capacity: an overview. Int. Psychogeriatr. 2004;16:397–419. doi: 10.1017/s1041610204000808. [DOI] [PubMed] [Google Scholar]
  47. Wechsler D. Wechsler Memory Scale. Third Edition (WMS-III) The Psychological Corporation; San Antonio: 1997. [Google Scholar]
  48. Wirshing DA, Sergi MJ, Mintz J. A videotape intervention to enhance the informed consent process for medical and psychiatric treatment research. Am. J. Psychiatry. 2005;162:186–188. doi: 10.1176/appi.ajp.162.1.186. [DOI] [PubMed] [Google Scholar]
  49. Wirshing DA, Wirshing WC, Marder SR, et al. Informed consent: assessment of comprehension. Am. J. Psychiatry. 1998;155:1508–1511. doi: 10.1176/ajp.155.11.1508. [DOI] [PubMed] [Google Scholar]

RESOURCES