Carer Appraisal Scale

Abstract

Measurement of patient outcomes is an integral part of mental health service evaluation, as well as guiding clinical practice to ensure best outcomes for patients. Moreover, carers have long held a need for a voice in care outcomes. Despite there existing numerous tools for quantifying patient functioning based on clinician assessments or self-reports, there is a serious paucity of tools available for the carers of patients to appraise their functioning. This tool, developed for use in a community aged care psychiatric service, involves 4 sections—a global impression of patient progress, a scorable checklist of patient functioning in multiple domains, a qualitative section for identifying the most pressing concerns from the carer’s perspective, and an open-ended feedback on treatment to date. In this pilot study, the Carer Appraisal Scale was found to have a fair correlation with the Health of Nation Outcomes Scale for over 65. This tool has potential for use in community aged care psychiatric services, as it provides a framework for communication of concerns, assists in prioritizing care, and adds value to clinician treatment plans, as well as providing another dimension to assessment of the patient while empowering carers in care participation. Practical implications of its use, limitations, and potential for modifications are also discussed.

Introduction

Patient care involves 3 fundamental social factors—patient access to medical services, the patient’s occupation and activities, and their social milieu, which principally involves the carer (1). Carers play a vitally important role in supporting family members, friends, and neighbors to live at home and remain connected to their communities (2). Despite this, there is a surprising paucity of structured means of engaging them.

One means of engaging carers is to actively and routinely document their perspectives on patient care and to incorporate these in care planning which is currently driven by clinician assessment, at best informally guided by ad hoc discussions with family members. Structured functional assessments that are only clinician-based only provide for “snapshot” assessments of patient (during the time of interview) and therefore cannot be comprehensive. Carers are better placed to identify the care needs of patients due to their intimate knowledge of the client and their history and their daily contact (1).

There is a specific need for carer assessments particularly for older psychiatric community patients (2). Carers of older patients tend to have at least a moderate level of contact and provide some form of care assistance and therefore can better evaluate their day-to-day functioning.

A number of studies have considered the value of the information that is obtained from collateral sources in dealing with a significant range of adult physical and mental health problems (3). Collateral information is particularly important for patients with cognitive impairment, impaired insight, or physical deterioration, all 3 of which are necessary concerns in the older mental health patients (4). Jorm (5) showed that collateral information was as accurate as brief cognitive testing as a screening tool for dementia and argued for its use routinely.

As opposed to assisting with diagnosis, the role of carer or other collateral information in the clinical monitoring of a person with a known clinical history has received much less attention, and there is comparatively little information in the literature about its use in this way. A literature search using the terms aged care, collateral sources, clinician, depression, carer, carer burden, diagnosis, treatment, and monitoring identified no studies that had developed a tool comparing carer and clinician assessments.

This pilot study describes a validation of an outcome measure based on carer assessment of the patient against an existing clinician-based scale for use with ambulatory community patients. It was hypothesized that the outcome measure would have good correlation with an existing scale, would be acceptable to carers, would provide useful qualitative data, and would supplement the clinician treatment plan.

Method

The study was undertaken in a multidisciplinary old-age community mental health team that provides extended case management to patients over 65 years with mental health problems in a large metropolitan region in Northern Sydney. The scale was developed to try to address the demands of extended case management and lengthy engagements with patients.

The Carer Appraisal Scale (CAS) was designed to be brief and simple to administer, and items chosen to assess key symptoms, signs, and behaviors, which have been the focus of well-recognized and validated clinician-based performance scales. The scale was piloted with 23 consecutive patients and carer dyads, with consent being obtained from both patient and carer.

The scale has 6 sections:

• Section 1: The carer is asked to quantify a broad impression of the patient’s overall progress on a 7-point Likert scale, based on a similar concept as that used in the Clinician Global Impression Scale (6). Two weeks were chosen for consistency with the Resource Utilization Group–Activities of Daily Living (ADL) scale (7), Health of Nation Outcomes Scale (HoNOS) (8), and the state anxiety Kessler (9) scales.

• Section 2: An 18-point checklist, using a Likert scale of 1 to 4, from “not at all concerned” to “very concerned,” involving many domains comparable with HoNOS, as well as further sleep and ADL components. This section is scorable, with each item scoring from 1 to 4, leading to a minimum score of 18 and a maximum score of 72.

• Sections 3 to 6: These are qualitative open-ended questions exploring changes in the patient, the carer’s primary agenda, and service evaluation.

Analysis

The validity of the 18-item checklist section of the CAS was assessed by correlating items with corresponding HoNOS assessments completed by clinical staff of the Aged Care Mental Health Team. Of the 18 checklist items in section 2 of the CAS, 16 of them correlated with HoNOS items. These 16 CAS items were assessed against HoNOS items. The data were analyzed to assess the extent of agreement between the carer and clinician scoring of the CAS. Cohen kappa (κ) was calculated using both an equal (linear) weighting of differences and a second (weighted) scale that multiplied the contribution of errors in scoring by 1.25, 1.5, 1.75, and 2.0 for discrepancies of 1, 2, 3, and 4, respectively.

Results

Of the 23 scales offered, 3 scales were declined by the patient being unwilling to allow their carer to complete the scale. There was no negative feedback regarding the form itself. Although we did not specifically assay carer feedback regarding the scale, 5 carers (nonanonymously) volunteered positive feedback, grateful for the opportunity to contribute in a more concrete fashion to their loved one’s care. Nineteen of 20 carers completed question 3 (changes noticed), 17 of 20 completed question 4 (important issue), 12 of 20 completed question 5 (team intervention), and 5 of 20 completed question 6 (any other comments). For 20 CAS scales with 16 items each, out of a possible 320 response items, there were 6 blank spaces leading to 314 response items to be assessed.

The series of 20 CAS yielded the following data matrix:

	Clinician score
Carer score		0	1	2	3	4
	1	108	27	13	8	1
	2	31	14	10	9	0
	3	26	9	12	6	2
	4	9	6	15	3	5

The calculated weighted κ(w) of 0.276 (95% confidence interval: 0.22-0.33) indicated statistically significant “fair” inter-rater agreement.

Discussion

Correlation was only “fair” with the HoNOS-65, however, there is no comparable carer-based assessment scale that this scale can be validated against. Evaluating the scale against a clinician-based scale is the closest available comparator.

By completing this scale, there were several “hidden” pieces of information derived from it that we did not initially anticipate. We found, for example, that it was useful to know whether the scale would be completed. There was no situation when a carer did not wish to fill out the questionnaire, but patients who appeared to be engaging well with us were, as a result of refusing to allow carers to complete the questionnaire, discovered to be significantly socially isolated and unable to use their carers as resources. Naturally, this should not hold true in all circumstances as patients are certainly entitled to their privacy and confidentiality; however, it was our finding within our very limited cohort.

In terms of carer acceptability and practical usage, the scale functioned very well, with a high rate of compliance and ease of administration. It was found to be implemented easily over the telephone, consistently completed within 3 minutes. It could also be practically administered multiple times during follow-up to assess patient progress and management planning.

Some practical limitations with the scale included identifying when the scale should be given, and this is likely to vary with the service that considers using this scale. An obvious issue is that many patients do not have carers but may not necessarily be socially isolated—as in having acquaintances, friends, and paid carers. Noncompletion of the scale for this reason should encourage the service to consider 2 longer term goals—a recovery model emphasizing the patient’s current level of independence, with maintaining gains and improving capacity to obtain additional supports, and targeted social interventions to encourage community connectiveness with friendship groups and social interaction opportunities. This increases the likelihood that future engagements, when the patient is more deteriorated, will have a carer to engage with when the patient needs one. If the patient both does not have a carer and is socially isolated, this is all the more indicated.

The term “team” is also problematic as carers often conceptualize all medical caregivers as being part of the same “team,” which the clinicians may not concur with. By providing this scale to the carer of the patient, it means that the feedback is not anonymous with regard to the service as a whole, which limits the usefulness of the feedback.

The scale was also tested in a very circumscribed area of Sydney with a homogenous middle-class population of predominantly English-speaking background. Clearly, consideration and adaption to the needs of those of culturally and linguistically diverse backgrounds is needed. At the time of implementation, there were few disengaged carers, although this certainly could be an issue with longer implementation. There was a possible selection bias in how clinicians approached the carers of patients, in that this was a nonrandom convenience sample of carers.

Potential for Modifications and Future Research

There may be a role to having the scale given by a nonmanaging clinician to improve the quality of service evaluation, yet there is definite value in encouraging an interchange between carer and treatment team. There certainly should be an assay of public perception of the form to guide future modifications. The word “team” could also be changed to the name of the specific service using the form. We also considered that a different version of the form to be given at initial presentation may be of use. With regard to further research options, it would be useful to see how this form would function against carers using the HoNOS directly. We are planning to trial the scale among a much larger patient/carer base, following the findings of this pilot study.

Author Biography

Neil Jeyasingam is an old age psychiatrist, psychotherapist and lecturer in philosophy of medicine.

Appendix A

Carer Appraisal Scale