Assessment of competence to complete advance directives: validation of a patient centred approach

Abstract

Objective

To develop a patient centred approach for the assessment of competence to complete advance directives (“living wills”) of elderly people with cognitive impairment.

Design

Semistructured interviews.

Setting

Oxfordshire.

Subjects

50 elderly volunteers living in the community, and 50 patients with dementia on first referral from primary care.

Main outcome measures

Psychometric properties of competence assessment.

Results

This patient centred approach for assessing competence to complete advance directives can discriminate between elderly persons living in the community and elderly patients with dementia. The procedure has good interrater (r=0.95) and test-retest (r=0.97) reliability. Validity was examined by relating this approach with a global assessment of competence to complete an advance directive made by two of us (both specialising in old age psychiatry). The data were also used to determine the best threshold score for discriminating between those competent and those incompetent to complete an advance directive.

Conclusion

A patient centred approach to assess competence to complete advance directives can be reliably and validly used in routine clinical practice.

Key messages

• Advance directives could potentially be useful for patients with dementia as a means of extending their autonomy when they become incompetent

• Competence to complete an advance directive involves understanding possible future clinical situations

• Vignettes presenting hypothetical medical problems were tested in 100 elderly people, and were found to validly and reliably discriminate between volunteers living in the community and patients with dementia

• We suggest that two clinical vignettes, each followed by a semistructured interview comprising 10 points, can aid in the assessment of competence to complete advance directives

Introduction

Advance directives (“living wills”) for medical care have been widely advocated as a means of extending the autonomy of patients to situations when they are incompetent. However, their impact has been surprisingly small. Despite legislation in the United States aimed at encouraging the completion of advance directives, less than 10% of healthy Americans have completed one.¹ The question remains as to how advance directives can be developed and effectively implemented in clinical practice. A pressing ethical problem in their use is that competent people may not always be well placed to make decisions concerning their future incompetent selves.² It is difficult for healthy people to imagine the whole range of situations that might befall them. It seems more worthwhile for advance directives to be completed at a time when people already have some disease or disability, enabling doctors to give realistic guidance about poshpsible future situations.

A number of commentators, including the British Medical Association, have argued that dementia is one clinical situation for which an advance directive could potentially be useful.³,⁴ An important question, therefore, is whether individuals with dementia are competent to complete an advance directive.

Silberfeld has suggested criteria to test the capacity to complete an advance directive.⁵ These criteria examine general competence to complete a directive in a way analogous to the assessment of testamentary capacity.⁶ They focus on whether the individual understands the nature and purpose of an advance directive; but they do not assess whether an individual is capable of understanding actual possible future clinical situations. Such understanding is critical to competently completing an advance directive. The Hopkins competency assessment test, which has also been used to test the competence of elderly patients to write advance directives, has similar limitations.⁷

There is, therefore, a need for a method to assess the competence of patients with dementia to complete an advance directive. Our aim was to develop a patient centred approach to enable those with cognitive impairment to complete an advance directive, and to assess the validity of the procedure. In doing this we have taken account of the importance of properly understanding the future imagined clinical situation. We report the development and psychometric properties of such a procedure.

Subjects and methods

Instrument design

Competence is specific, not global—that is, an individual is or is not competent with respect to a specific decision or setting.⁸ The law accepts that many different competencies exist, and studies have shown that these differences presume alternative abilities and consequently tools to test them. Applebaum and Grisso have suggested that the legal standards for determining competence fall into four categories, each addressing a different skill: (1) the ability to maintain and communicate stable choices; (2) the comprehension of information presented; (3) the appreciation of the likely consequences of a decision for the individual; and (4) the ability to manipulate the information rationally.⁹ In developing our procedure, we designed a tool that fulfilled the criteria suggested in the MacArthur treatment competence study for competence measures—namely, the content of the instrument having relevance to the decision being studied and meaningful to the people involved.¹⁰

Clinical vignettes have been shown to be an effective and valid basis for assessing competence to consent to medical treatment in patients with dementia.¹¹,¹² Three vignettes were developed specifically for our study after discussion with a group of geriatricians, psychogeriatricians, and a medical ethicist (TH). These vignettes were designed to describe realistic situations where advance directives could make a real difference to medical practice. They were first tested on 14 elderly individuals who attended a lunch club in Oxford. These were subsequently revised for use. The revised versions were subjected to tests of both reliability and validity.

Each vignette presented a hypothetical medical problem with symptoms together with treatment alternatives—the two we propose for routine clinical practice are included in box 1. Each vignette was written in simple English and with a moderate amount of information. Participants were given an information sheet that summarised the main features of each vignette so as to help them cope with any memory difficulties. The three vignettes together took about 20 minutes to conduct.

The study was approved by the ethics committee of Oxfordshire Psychiatric Research.

Patients and participants

One hundred elderly people were recruited to the study. These comprised 50 patients with a diagnosis of dementia at the time of referral from primary care to two community psychogeriatric teams (covering both city and county areas). We excluded those with a clinical diagnosis of mood disorder or psychotic illness. In addition, 50 elderly volunteers living in the community were recruited from pensioners’ lunch clubs in Oxford. The volunteers were also screened for dementia using the mini-mental state examination. So that the psychometric properties of the competence instrument could be tested on a range of people from those with no cognitive impairment to those with mild impairment, we did not exclude those with cognitive impairment. Informed consent was obtained from all participants or their care givers when required.

Sixteen of the referred patients refused to participate. Three patients were not interviewed: two were not fluent in English and one was severely dysphasic. No significant differences were found in the dementia group for sex, age, or mini-mental state examination between those who refused to participate and those who did participate.

Procedures

One of us (SF) conducted all the interviews. The 50 elderly volunteers living in the community were interviewed in a quiet area at the lunch club. The patients with dementia were seen at home or, if recently admitted, in hospital. Each vignette was read to each participant, after which a semistructured interview of nine questions (10 point score) was conducted (figure). To enable those with cognitive impairment to answer the questions, we repeated and clarified parts of the vignette when necessary. Box B2 explains the scoring system. The scoring system reflected previous work, which has shown that expressing a treatment choice does not distinguish between competent and incompetent persons.¹³ After we assessed competence on the three vignettes, we used Silberfeld’s competence tool (9 point score) on each participant.

Box 2.

—Guidelines for competence question and answer sheet

Question 4

Valid reasons show that the individual is able to discriminate between the intervention and non-intervention choices. Therefore answers like “to live,” “in order to keep me alive,” or “I do not want to be a burden to others” are acceptable. Responses such as “I do not like hospitals, needles, or operations” are not.

Question 6

Valid answers indicate that the individual is able to imagine significant consequences of the decision they make. Answers that the family will “go along with my decision,” “back me up,” or “accept my choice,” or that family members will be upset, anxious, or relieved do not show that they have considered how their treatment decision will affect their families’ lives. Further clarification is needed. If interviewed individuals do not have family, then the question can be rephrased to ask what the decision would mean for a close friend.

Question 7

If no answer is given initially, the question can be rephrased to ask what effect the treatment will have after 1 or 2 weeks. For those that choose medical interventions, answers must show an awareness of recovery in hospital.

Question 8

Answers must show discrimination between intervention and non-intervention choices. If no response is given initially, the question can be rephrased to ask what effect the treatment decision will have after a few months.

Reliability

—All three vignettes were included in the test-retest reliability study. Fourteen participants—four with dementia and 10 living in the community—were reinterviewed 10-14 days after their first interview. Interviews with 19 participants—11 with dementia and eight living in the community—were audiotaped and used for the interrater reliability study. Two of us (TH and RJ) who had not conducted the original interviews independently rated them.

Validation

—The problem with all competence assessments is that there is no universally agreed gold standard. The 19 audiotaped competence assessments, conducted on a wide range of individuals, were the basis of the validation. These were played to two old age psychiatrists (TH and RJ) not present at the original interview, who made a clinical global judgment as to whether the participant was competent or not to complete an advance directive.

Internal consistency

—An individual’s scores on the three vignettes were correlated with each other. The order of vignettes was varied between participants to investigate whether there was any learning involved: 25 of the participants were randomly selected. For these participants vignette 1 was presented first (as opposed to last).

Statistical procedures

—We used non-parametric measures (Mann-Whitney) of statistical significance due to the skewed distribution of the measures being studied. We used correlation coefficients (intraclass for scales and κ for discrete scores) for the test-retest and interrater reliability studies. The κ statistic was used for the validity study.

Results

Table 1 compares elderly patients living in the community and patients with dementia on competence variables. Our approach and Silberfeld’s method for assessing competence significantly discriminates between the two groups.

Table 1.

Group comparisons on competence variables. Values are mean (SD) score unless stated otherwise

Variable	Patients living in community (n=50)	Patients with dementia (n=50)	P value
Age	74.9 (6.2)	79.8 (6.7)	—
No (%) male	21 (42)	22 (44)	—
Mini-mental state examination score	27.4 (2.6)	15.5 (6.1)	—
Vignette 1	7.7 (2.2)	2.9 (3.1)	<0.0001
Vignette 2	7.8 (2.3)	2.9 (3.5)	<0.0001
Vignette 3	7.9 (2.2)	2.6 (3.2)	<0.0001
Average vignette*	7.8 (2.1)	2.8 (3.1)	<0.0001
Silberfeld score†	6.4 (1.9)	2.7 (2.8)	<0.0001

^*10 point score. 

^†9 point score. 

Test-retest, intratest, and interrater reliability

—Table 2 presents correlation coefficients for the scores for each pair of clinical vignettes. The correlation coefficient for average score for test and retest scores was 0.97. In the interrater reliability study, the ratings (average scores) made by the two additional raters correlated highly with each other (r=0.94) and with the ratings made by the interviewer (r=0.95). The ratings for the three vignettes were highly correlated with each other (r⩾0.92). In the random sample of 25 people who received the vignettes in a different order, there was no significant difference between the score for vignette 3 when presented last or first—average score 8.0 versus 7.9 respectively.

Table 2.

Correlation coefficients for reliability study

Variable	Vignette			Average score
	1	2	3
Vignette 1	1	0.92	0.92	—
Vignette 2	0.92	1	0.93	—
Vignette 3	0.92	0.93	1	—
Interrater average vignette*	—	—	—	0.95
Test-retest average vignette†	—	—	—	0.97

^*Correlation coefficient for average vignette score of interraters compared with average vignette score of initial investigator. 

^†Correlation coefficient for average vignette score of test-retest study compared with average vignette score of original study. 

Validation

—The results of the validity study showed that a cut off score of ⩾6 for the average vignette score (out of 10), as indicating competence, correlated most closely with the global assessment of competence to complete advance directives for both independent raters (κ=1 for one independent rater, κ=0.83 for the other independent rater). In the test-retest study, all individuals who scored an average of ⩾6 in the first assessment scored ⩾6 in the retests, and all individuals who scored an average of <6 in the first assessment scored <6 in the retests. The average vignette score for our procedure correlated highly (r=0.86) with the Silberfeld competence assessment score. We therefore used a score of ⩾6 as the gold standard for the sensitivity and specificity study.

Sensitivity and specificity

—Table 3 shows the sensitivity and specificity values for the three vignettes, taking an average score of ⩾6 for the three vignettes as defining competence. A combination of vignette 1 and 2 achieved high sensitivity and specificity (96% and 98% respectively). As we aim to provide as brief an assessment tool as possible, we recommend that these two vignettes be used in assessing competence in routine clinical practice.

Table 3.

Sensitivity and specificity values for vignettes in assessing competence

Variable	Sensitivity (%) (95% CI)	Specificity (%) (95% CI)
Vignette 1	94 (83 to 99)	90 (79 to 97)
Vignette 2	98 (89 to 100)	90 (79 to 97)
Vignette 3	94 (83 to 99)	94 (84 to 99)
Vignettes 1 and 2	96 (86 to 100)	98 (90 to 100)
Vignettes 1 and 3	94 (83 to 99)	94 (84 to 99)
Vignettes 2 and 3	98 (89 to 100)	94 (84 to 99)

Discussion

With the increasing advocacy of advance directives and the rising prevalence of dementia, the issue of competence to complete advance directives is an important challenge to the medical profession. In this paper, we present a patient centred semistructured interview for the assessment of competence to complete advance directives of elderly people with or without dementia. This is a patient centred approach in that we have tried to ensure that patients are not regarded as incompetent because of cognitive impairment (such as memory difficulties), which is not critical to competence but which can interfere with assessment procedures. In developing this method, our approach has been to create a tool that will enable those with mild cognitive impairment to complete advance directives but will also validly identify people whose impairments do render them incompetent to do this. Our approach is designed for use by health professionals. Unlike previous procedures that have been advocated for the assessment of competence to complete an advance directive,⁵,⁷ our procedure takes into account the importance of being able to imagine future possible situations. It is not sufficient for an individual simply to understand what an advance directive is.

Our patient centred approach discriminates between a group of elderly people living in the community and patients with dementia. The patients with dementia were on average 5 years older than those living in the community, and it is possible that ageing is an independent factor correlated with competence—a perspective in keeping with the view that competence requires cognitive abilities, such as the ability to imagine future situations, which are more sensitive to the ageing process than can be measured by standard screening tests for dementia.

Validity was examined by relating our approach with a global assessment of competence to complete an advance directive made by two of us (who are both psychiatrists specialising in old age psychiatry). The data from our study were also used to determine the best threshold score to use in discriminating between those competent and those incompetent to complete an advance directive. Our study shows that this procedure has good interrater and test-retest reliability.

Our patient centred approach has been designed to be clinically useful. We suggest that two vignettes, each followed by a short semistructured interview comprising a 10 point score, can aid in the assessment of competence to complete advance directives, although the ultimate decision is a clinical one that takes account of an individual’s particular situation. We have suggested that those who score ⩾6 (out of a possible score of 10) are competent. The assessment takes about 15 minutes.

Our patient centred approach could be used in different ways. Primary care physicians may find it useful when elderly people approach them for advice about advance directives. Specialists in geriatrics, neurology, or psychiatry may wish to use it in discussing advance directives with patients on their first presentation for dementia. Moreover, physicians involved in the care of those with a mild degree of cognitive impairment can use this procedure. We hope that it will also prove useful for research purposes.

Box 1.

—Clinical vignettes

Vignette 1

You are in hospital recovering from a sudden stroke. It has left you half paralysed from which you are unlikely to improve. You cannot speak but you can understand. You cannot swallow food safely. There is a high risk that food directly enters your windpipe and makes you choke.

 Your doctor explains that, in order to feed you adequately and safely, he needs to use a feeding tube which passes through your nose into your stomach. This is likely to make you live longer but you need the tube all the time. The other alternative is that you are kept comfortable, but without a feeding tube.

Vignette 2

You are in a nursing home. Over the past few years you have become forgetful and occasionally confused. You have Alzheimer’s dementia. You are able to recognise relatives and nursing home staff. You are in good physical health. You seem happy and contented. However, your memory problems are going to get worse. One day you pass some blood from your bowel.

 You can leave it and not have any tests. Or your doctor can organise for you to have some tests to see where the bleeding is coming from, followed by surgery if a cancer is found.

Figure.

Question and answer sheet (total score out of 10; score in parentheses)