Challenges in measuring outcomes for caregivers of people with mental health problems

Abstract

Patient-reported outcomes (PROs) are increasingly important in health care and mental health research. Furthermore, caregivers become partners in care for patients with mental disorders, and health workers are more attentive to the expectations and needs of caregivers. A number of outcomes for caregivers are measured and used in daily practice in order to promote actions to improve health care systems and progress in research on the impact of mental disorders on their caregivers. This paper proposes an inventory of the different outcomes and different measurement tools used to assess the impact of disorders, raising a number of methodological and conceptual issues that limit the relevance of measurement tools and complicate their use. Finally, we propose some recommendations promoting the development of relevant outcome measures for caregivers and their integration into current systems of care.

From patient-reported outcomes to outcomes for caregivers

In recent years, successive changes in mental health care systems have made them evolve from classical institutionalized care based on hospitalization to community health care with psychosocial rehabilitation. These changes have re-established patients and their caregivers at the center of health care programs.¹ Caregivers are usually unpaid nonprofessionals, often family and friends who have significant input in the care and support of people affected by severe psychiatric illnesses. In the UK, the National Service Framework for Mental Health recognized the vital role caregivers play in delivering health care.² Patients and their caregivers are more and more involved in decision making when it comes to their care, the rehabilitation process, and preventing relapses, as well as research programs.³ This cooperation between patients, caregivers, and physicians has become the foundation of a participatory health care system in which the relationships between each of the parties are considerably reinforced.^4-6

This is the context in which patient-reported outcomes (PROs) have been developed and become standard practice in recent years for evaluating the effectiveness of health care.^7,8 PROs also help the clinician understand how patients experience their illness, by evaluating aspects such as their quality of life, disability, satisfaction, or level of functioning.

In addition to taking into account the patient's viewpoint with PROs, it is necessary to pay more attention to the expectations and the constraints of the caregivers who have become key players in today's health care system. It is currently accepted that mental disorders have a much larger impact than what was assumed before the advent of community psychiatry; this impact is especially large on the caregivers to the patients. ^8-10 As the health care systems evolve, relatives have become the main caregivers for patients with severe mental disorders. ¹¹ They must provide support to their relatives with mental disorders while withstanding some of the consequences of the illness and the treatments. The impact of mental illness on related caregivers is well documented.^12-16 It causes changes in the quality of life,^17,18 restricts roles and activities, and increases psychosomatic, anxious, or depressive symptoms.^8,19-22 Moreover, caregivers' negative experiences may affect their ability to care for the patients.^23,24

Taking into account the caregivers' experience, their viewpoint, and their expectations is of particular interest. A better understanding of the effects of mental disorders on caregivers of patients will enable us to extend help and support tailored to their needs. These actions should relieve caregivers of their constraints and improve the effectiveness of their helpful actions by guiding them.²⁵ They should also help avoid health problems frequently seen in these caregiver situations.

Current research orientations are determined by the importance of measuring caregiver expectations and the necessity of reliable outcome indicators for caregivers. Research is currently focused on two main issues, which are: (i) studying the impact of the patient's illness on their caregivers; and (ii) setting up specific and validated measuring tools for caregiver outcomes.

These indicators must be designed similarly to PROs, to be user-oriented—in this case, caregiver-oriented—taking into account their specificities, their perceptions and their expectations. It has been reported that the burden of disease for caregivers is assessed differently depending on who performs the evaluation, the clinician, or the closest caregiver.²⁶ Indeed, objective burden indicators (ie, the patient's symptoms, behavior, and sociodemographic characteristics, and also the changes in household routine, family or social relations, work, leisure time, and physical health) do not coincide with subjective burden indicators (ie, the mental health and subjective distress among family members). This difference in viewpoint in evaluating the burden of disease may be explained in part by a lack of awareness of health care workers when it comes to the needs and issues of caregiving families of psychiatric patients.²⁷ This confirms the difficulty in assessing the caregivers' constraints and the necessity in raising awareness for health care personnel of the importance of taking into account the viewpoint of caregivers and their patients.

The way they experience their relatives' illness or the impact it will have on them will depend more on the functional impact of the disorders. Their perceptions seem strongly linked to dimensions such as global functioning, social integration, or improvements in daily life management.²⁸ Thus, caregivers expect much more from health care than merely reducing their relatives' symptoms. The clinician's evaluation may therefore be enriched by taking into account the caregiver's perspective, yielding therapeutic proposals, organizational changes, or political decisions that are better suited to the expectations of the patient's family and friends. These caregiver perspectives remain little used, however, in the care of individual patients and caregivers and in the general orientations of health care systems and political decisions concerning mental health care.

Which caregiver outcomes should be studied?

Globally, studies of outcomes for caregivers of mental health patients are underdeveloped. The majority of available data to date is from studies of caregivers of patients with dementia. These advances in research in the field of dementia can be explained by the fact that caregivers of patients with dementia have dependent relatives with severe disorders, so they request much more support than caregivers of patients with other mental disorders or chronic illnesses.^29-31 To quantify this research, we did the following PubMed searches: “dementia” AND “caregiver” AND “outcome” yielded 962 results, whereas “schizophrenia” AND “caregiver” AND “outcome” yielded only 176 results and “bipolar disorder” AND “caregiver” AND “outcome” yielded only 42 results.

Generally speaking, the outcomes measured in the field of dementia were mainly focused on burden, quality of life, support needs, and coping strategies.³² These outcomes have been featured in reviews which list them along with their measuring instruments.^33-35 They are currently used on a daily basis by clinicians and researchers, and have thus become study variables in measuring instruments used for evaluating care programs for caregivers.^30,36-38 These tools have been used in randomized study protocols. They are used to quantify the advantages of structured support programs for caregivers of patients with dementia and to identify some determining factors in the observed improvements.^39-41 Although caregivers have become essential partners in mental health care, and although they must be included in the evaluation process for health care and intervention programs, Harvey et al⁴² have shown that there is no established consensus on which measurements and tools should be used to evaluate the experiences of caregivers in the field of mental health care. The same can be said for outcome measures for caregivers of patients with psychiatric disorders, which focus on the same aspects of the caregiving experience as the outcome measures used for caregivers of patients with dementia, the risk being that the lack of specificity of these measures would cause them to be of little use, as we shall see.

In the field of schizophrenia, most of the measurements are focused on the burden of the illness on the caregivers, while a small number aim to improve coping strategies, perception of needs or quality of life, with the inherent limitations of the tools we mentioned previously.¹⁰ Even fewer tools are available for caregivers of patients with mood disorders, and those that do exist focus on measuring burden,^16,43-45 quality of life,⁴⁶ stigma and coping strategies,^19,20,47-48 or family functioning.^49-50 Apart from these two nosographic frameworks, little research has been done on outcomes for caregivers of patients with other pathologies. There have been a few studies on the caregivers of patients with eating disorders,^51-53 and with attention deficit-hyperactivity disorder (ADHD).⁵⁴ We have only found one study on the outcomes for caregivers of patients with personality disorders,⁵⁵ and have found none on caregivers of patients with addictions or substance abuse disorders.

This study of the literature reveals that most outcome measures for caregivers of patients with mental disorders can be placed in three main categories: (i) caregiver's well-being (Table I); (ii) their experience of caregiving (Table II) ; and (iii) caregiver's needs for professional support (Table III).⁴² Nevertheless, a caregiving relationship is complex as it involves many parameters, which complicate the conceptual definition of outcomes of interest. These parameters are just as linked to general aspects of the caregiving relationship (ie, the type of care given, the types of interactions, and the quality of the relationship) as they are to more specific aspects inherent to the pathology or mental disorder and its functional consequences on the patient's day-to-day life.⁴² Thus, caregiver quality of life, burden, and needs are determinant to the caregiving experience, but it remains unclear whether the scales that have been developed specifically to measure them evaluate fundamentally different dimensions of the caregiving experience.³⁶ This limits the scope of the measurements taken, especially when several different measuring tools are used.

Table I. Examples of useful caregivers' quality of life questionnaires with characteristics. SR, self-rated questionnaire; SI, structured interview.

Year	Name	Type	Indication	Number of items	Format	Dimensions	Viewpoint	Country	Authors
1992	Short From 36 (SF36)	SR	Generic (Caregivers or noncaregivers)	36	Variable	8: Physical functioning; social functionning; role-physical problems; role-emotional problems; mental health; vitality; bodily pain; general health. each dimension is scored within a range of 0 (low Qol level) to 100 (high Qol level).	Developed from previous existing instrument	USA	Ware and Sherbourne90
2004	World Health Organization's Quality of Life Assessment (WHO-QOL-BREF)	SR	Generic (Caregivers or noncaregivers)	26	5-point Likert	4: Physical; psychological; social; environment.	Expert review, focus groups with healthy individuals and patients	International	Skevington91
2011	Schizophrenia Caregiver Quality of Life questionnaire (S-CGQoL)	SR	Schizophrenia	25	6-point Likert	7: Psychological and physical well-being; psychological burden and daily life; relationships with spouse; relationships with psychiatric team; relationships with family; relationships with friends; material burden.	Relatives' viewpoint	France	Richieri92

Table II. Examples of useful caregivers' burden questionnaires with characteristics. SR, self-rated questionnaire; SI, structured interview.

Year	Name	Type	Indication	Number of items	Format	Dimensions	Viewpoint	Country	Authors
1992	Caregiver Reaction Assessment (CRA)	SR	Dementia	24	5-point Likert	6: Esteem; lack of family; support; finances; schedule; health.	Review of literature + professionals' and relatives' viewpoint	USA	Given84
1996	Perceived Family Burden Scale	SR	Schizophrenia	24	3-point Likert	2: Relatives' reactions to active/aggressive behaviors and to withdrawn/passive behaviors.	Review of literature + professionals' and relatives' viewpoint	Canada	Levene85
1997	Behaviour Disturbance Scale	SR	Shizophrenia	16	3-point Likert	2: Positive symptom behaviors; negative symptom behaviors.	Developed from previous existing instrument: Social Behavioral Assessment Schedule.	Canada	Provencher et Mueser86
	Subjective Burden Scale	SR	Schizophrenia	22	4-point Likert	No dimension.
	Objective Burden Questionnaire	SR	Schizophrenia	18	3-point Likert	3: Negative consequences for children; negative consequences for primary caregiver; negative consequences for other adult family member.
1998	Involvement Evaluation Questionnaire	SR	Mental illness	31	5-point Likert	4: Tension; supervision; worrying; urging.	Review of literature and previous existing instruments.	Netherlands	Schene87
2008	Care Burden Scale for Relatives	SI	Psychotic disorders	10	4-point Likert	3: Relatives' practical burden; aspects regarding own health; emotional burden.	Previous existing questionnaire.	Sweden	Hjärthag88
2009	Mood Disorder Burden Index	SR	Mood disorders	27 +/-5	4 and 5-point Likert	3: Patients' mood symptoms; caregivers' worry about the future; caregivers' interpersonal difficulties with the patient + 1 optional: caregiver burden associated with patients' pharmacotherapy or psychotherapy.	Relatives' viewpoint	USA	Matire89

Table III. Examples of useful caregivers' coping strategies questionnaires with characteristics. SR, self-rated questionnaire; SI, structured interview.

Year	Name	Type	Indication	Number of items	Format	Dimensions	Viewpoint	Country	Authors
1996	Experience of Caregiving Inventory	SR	Mental illness	66	5-point Likert	10: Difficult behaviors; negative symptoms; stigma; problems with services; effects on family; need to backup; dependency; loss; positive personal experiences; good aspects of relationship.	Relatives' viewpoint.	UK	Szumukler93
1996	The Family Coping Questionnaire	SR	Schizoprenia, mood disorders	27	4 or 5-point Likert	7: Information; positive communication; social interests; coercion; avoidance; resignation; patient's social involvement.	Review of literature and relatives' viewpoints.	Italy	Magliano94
2007	Questionnaire of Family Functioning	SR	Schizophrenia and schizoaffective disorders	24	4-point Likert	3: Problem solving; communication skills; personal goals.	Review of literature, professionals' and relatives' viewpoints.	Italy	Roncone95

Difficulties to consider in choosing or creating measurement tools

This review of the literature on caregiver outcomes shows that the numerous tools used for these evaluations are very heterogeneous, as they are designed to investigate several more-or-less linked aspects of the caregiving relationship. They can be placed into three categories.

The most frequently represented ones are the generic tools used in caregiver outcome studies, but which weren't developed specifically for caregivers in mental health. Some of these measuring tools, such as SF 36, were even initially developed for noncaregivers.⁵⁶ Other less numerous tools, though more diverse, were developed specifically for caregivers in dementia or in general mental health. Most of these are very specific to a particular aspect of the caregiving experience (eg, Zarit Caregiver Burden Interview⁵⁷), and some allow for a more global evaluation of the caregiving experience (eg, Carers' and Users' Expectations of Services—Carer Version⁵⁸). Finally, some rare tools were specifically designed for caregivers of patients with a specific illness (eg, Caregiver Schizophrenia Quality of Life Questionnaire⁵⁹).

Once the outcome for the study has been chosen, clinicians and researchers must choose which tool to use amongst the plethora of available tools that have been developed. This decision is complicated by the lack of specificity, the heterogeneity, and the sheer number of available tools. As for specificity we have mentioned that the tools used are, for the most part, generic tools which were developed based on noncaregiver populations or populations of caregivers of patients with nonmental disorders. Regrettably, classical psychometric data such as reliability, validity, responsiveness, acceptability, and feasibility, or more precise psychometric data such as appropriateness, precision, and interpretability⁶⁰ have rarely been verified with caregivers of patients with mental disorders. These characteristics have a strong influence on the pertinence, reliability, and precision of evaluations, and yet they are rarely known or mentioned.^33,42 In their review of the literature, Harvey et al⁴² identify outcome measuring tools for caregivers and specify which psychometric properties are available for these tools. They reported that many of the tools that were used were considered irrelevant by the caregivers themselves, and that many of the psychometric characteristics were either incomplete or missing. On the one hand, the vast majority of the questionnaires which were used were designed based on expert opinions and clinician's perspectives without any actual input from caregivers. On the other hand, not all of them were tested on populations of caregivers of patients with mental disorders, and the available psychometric data concerning their use on such populations are inconsistent. The validity of using these tools is in question, as they are used in everyday practice and yet have neither been specifically designed for, nor tested on caregivers.

It is generally accepted that it is preferable for evaluation questionnaires to be designed from the perspective of the perceptions of the people who they will be used to evaluate, in order to improve their relevance and their validity.^10,61 Otherwise, some of the positive aspects of caregiving will rarely be examined in the caregiver outcome evaluation scales, although it is readily apparent that the caregiving experience is not always perceived as being only a burden.⁶² Improving the measurement of these little-known aspects of the caregiving relationship in the field of mental disorders would certainly allow for an improved understanding of the experience and for offering better support.⁶³ It can be recommended that interviews or focus groups of caregivers be pivotal in defining outcomes and designing measuring tools that will be better adapted to the expectations and perceptions of caregivers.⁶⁴

The issue here seems to be similar to that of the PROs, many different ones are being used, and they are varied and not always relevant.⁶⁵ This is largely linked to the fact that a large proportion of these measures took the expert's viewpoint into account more than that of the patient.⁶⁶ It is then necessary to be cautious when it comes to caregiver outcomes to avoid these same pitfalls. Without a consensus on the caregiver outcomes which should be promoted and used, these same issues are likely to arise: ill-adapted and irrelevant tools for measuring outcomes for caregivers of patients with mental disorders. Some other issues must then be examined carefully in order to limit the risk of getting incorrect results during the measurements which would then become generalized.

The first point to examine is whether the generic tools which have been used thus far are adapted for evaluating caregivers of patients with mental disorders. The validation process for these tools has not been used on these populations, so their validity is in doubt, especially given that the caregivers of patients without mental disorders may not have the same needs and expectations as caregivers of patients with mental disorders. As we have seen, it is essential to design tools based on the viewpoint of the populations being studied, in this case, caregivers of patients with mental disorders. This will improve the acceptability and the precision of the tool, validating its content. Although it is easier to verify these factors when designing a new tool based on a predefined target population, some alternatives include testing these data on preexisting tools based on a general population or on populations of caregivers of other diseases, although this involves a risk of questioning issues that are less relevant for caregivers of patients with mental disorders.

The same issues come up when defining and selecting outcomes of interest. A relevant outcome must echo the specific expectations and needs of caregivers of patients with mental disorders. It should also eventually yield informative measurements that will guide clinicians and researchers in their activities. Perhaps some of the outcomes currently being examined are not relevant to regular practice. Evaluating caregivers' needs may be more helpful to clinicians to orient families towards specific programs that meet their personal needs, whereas a scale measuring quality of life would be more useful for research on quality of life, and for drawing conclusions concerning decision-making when organizing health care. There is, however, a lack of consensus on the definition of quality of life and exactly which experiential aspects it involves.^67-68 Additionally, interpreting quality of life data is not straightforward (eg, response shift phenomena).⁶⁹ As we have seen previously, the various outcomes being studied refer to highly interrelated aspects of the caregiving relationship. Similarly, it has yet to be seen whether the same aspects of the caregiving relationship are involved in all clinical situations. Caregivers of patients with dementia may not have the same constraints and needs as caregivers of patients with schizophrenia or bipolar disorder, or they may not experience the same impact on their quality of life. It seems necessary to clarify the concepts of outcomes of interest. This should be a prerequisite to developing measuring tools, as this should inform us as to whether these tools should be used as general measuring instruments in mental health care and/or for specific disorders or clinical caregiving situations (eg, very young caregivers, ethnic minorities, caregivers of children, and caregivers of suicidal patients). Research on such outcomes and the development of measuring tools still require large-scale investigation efforts, which will be costly both in time and money.

As we have previously seen, if they are to be informative, it is absolutely necessary that the measures be reliable, precise, and sensitive to changes, which means any sources of bias or confusion must be detected. For the purpose of precision, it also seems to be important that these measuring tools come with a user guide. Using and interpreting scores will be more effective and informative, allowing for adjustments to the support offered to caregivers or orienting the organizational methods of the health care system.

Interesting perspectives

Generalizing the participation of caregivers of patients in defining outcomes and designing questionnaires seems to be the most effective way to overcome these hurdles in creating adapted measuring tools. The medical community will then have relevant and validated measuring tools with known psychometric properties, allowing for the development of large-scale studies and sharing results for analysis. Thus, the caregiver outcome measurements will serve as a guide to health care professionals in receiving, informing, supporting, and accompanying patients and their relatives, particularly on two levels.

In regular practice, they will be used to monitor the effects of the support groups, information systems, or psychological educational programs offered to caregivers. Some interesting results are already available. For example, in schizophrenia it appears that quality of life in both patients and caregivers are linked, with a strong relationship with the negative syndrome and the general psychopathology of the disorder.⁷⁰ Thus, a better control of patients' symptoms would reduce subjective burden in caregivers.⁷¹ In the same way psychoeducation courses for caregivers of individuals with schizophrenia improves the patients outcomes.⁷² To reduce caregiver burden, it seems important this family interventions focus more on patient and caregiver characteristics,⁷³ and consider some clinical and social features. Moreover, such caregiver outcomes allow comparisons between support programs with better consideration for caregivers' perspectives. Thus, Madigan et al found no difference for bipolar patient caregivers between Multifamily Group Psychoeducation and Solution Focused Group Therapy. Both programs demonstrated reduction in burden.⁷⁴ Using adapted and targeted questionnaires will allow the support offered to caregivers to be adjusted,^75-77 which will most likely improve the relationships between patients and their caregivers. To this end, they may eventually be integrated directly in the treatment pathways via electronic data capture modes of administration.⁷⁸ These outcome measures can also be integrated in e-health interventions that could be an efficient alternative to provide education and support for caregivers. Some experiments of Internet-based supportive interventions for caregivers of patients with dementia shows promising effects.⁷⁹

These questionnaires, used in research programs with validated tools and following similar standardized rules to the ones PROs use,⁸⁰ will make it possible to analyze the determining factors in the caregiving relationship and understand the mechanisms that come into play in how the illness is experienced. Thus, we know that caregivers of schizophrenia patients appraise caregiving more negatively than those of bipolar affective disorder patients.⁸¹ In a recent study, it was shown that caregivers of individuals with affective disorders reported higher quality of life levels than caregivers of schizophrenic patients. Interestingly, the differences between caregivers of individuals with bipolar disorders and individuals with schizophrenia concerned mental and psychological dimensions, while the differences between major depressive disorder and schizophrenia concerned only physical dimensions. These findings confirm the necessity of a multidimensional approach in identifying the most-impaired domain to improve programs for caregivers.⁴⁶ This research on caregivers' outcomes may lead to the identification of new dimensions that can be stress contributors for caregivers. MollerLeimkuhler and Jandl explore different dimensions, such as expressed and perceived emotions, which may have a relative prognostic value in caregivers' stress.⁸² This outcome research can also lead to changes in definition of some clinical dimensions. Another recent study emphasizes the importance of the point of view in the experience of the illness.⁸³ Thus, Karow showed an important difference between symptomatic remission and subjective outcome criteria, with a preference on the patients' side for subjective outcome, including well-being. In the choice of treatment decisions, this is an additional argument to supplement the experts' assessment of symptomatic remission with patients' and caregivers' assessment.

These already interesting and promising results will be refined by the use of more appropriate measurement tools and will be generalized by the use of standardized measures.

Conclusion

Caregiver outcomes are important elements and these should be taken into account in our regular practice, particularly within the context of the changes in the health care system. The current situation seems to be transitional. The medical community has globally become aware of the necessity of taking patient and caregiver viewpoints into account in the process of deinstitutionalization and developing community psychiatry. Progress still needs to be made, however, in order to have tools available for measuring caregiver experiences with relevant content that has validated psychometric properties. Similarly to PROs, these caregiver outcome measures will serve as a guide to clinicians, social workers, and therapists in treating and managing patients while also helping decisionmakers, policy-makers, payers, or regulators in adjusting and measuring the impact of health interventions. They open the way for taking caregivers into account better, meeting their needs in the treatment of patients, and should help orient health care policy and organizational methods in mental health care by giving caregivers of patients with mental disorders their rightful place, and providing them with better resources in a reinforced cooperation with health care providers.