Quality of life as patient-reported outcomes: principles of assessment

Abstract

Assessing quality of life (QoL) as a patient-reported outcome in adult psychiatry poses challenges in terms of concepts, methods, and applications in research and practice. This review will outline conceptually the construct of QoL, its dimensionality, and its representation across patient groups. Methodological challenges are examined, along with principles of QoL instrument development and testing, as well as across cultures. Application of instruments in epidemiological, clinical health economics, and health services research is reviewed based on pertinent literature. Validated measures for depression, psychosis, and anxiety disorders are available in adult psychiatry, and are increasingly used in research. Still, targeted measures are lacking for many mental health conditions and only rarely are tools applied in the practice context. Progress has been made in the development of instruments that are now ready for implementation. The information to be gained is valuable for identifying patient-reported needs for and benefits of treatment.

Introduction

Over the past decades, a paradigm shift has occurred in evaluating the outcomes of medical care. The focus of outcome assessment has moved from clinical indicators of disease activity towards the patients' perception of their health condition and related treatment. The term “patient-reported outcomes” (PROs) has been coined to denote the inclusion of the patient's perspective in epidemiological, clinical, health economics, and health services research.^1,2

With its definition of health as physical, mental, and social well-being, the World Health Organization (WHO) has provided the grounds for the introduction of the health-related quality of life (QoL) concept into medicine. QoL reflects the subjective perception of health, and is one of the core concepts within the PRO field.³

The development of the health-related QoL field in medicine was spurred on, not only by an evolving recognition of the subjective factor, but also by the increasing prevalence of chronic health conditions requiring long-term treatment and life-long care. Here, classical medical indicators of treatment outcomes (such as symptoms or survival) have been challenged regarding their relevance for capturing changes of health that matter to the patients and the societies they live in.⁴ Since the assessment of QoL in medicine has advanced considerably over the past 30 years, it is now possible to evaluate the quality of these assessment tools and the benefits of including them in research and practice—ranging from individual treatment decisions to health policy regulations on national and international levels.

Since its early years, the QoL field has been confronted with the question of how to define and operationalize health-related QoL, how to construct and evaluate methods of assessment, how to implement these measures in research and clinical practice, and how to examine the usefulness of the information gained.⁵

Assessing QoL

Concepts and dimensions

Even though philosophical, sociological, and psychological theories of QoL are to be acknowledged, the conceptual basis of health-related QoL research is closely tied to an operational definition, which identifies as its core dimensions the physical, social, and mental components within an overall model of perceived health.⁶ Operational definitions of health-related QoL share these common components, but may differ in the degree of detail and in inclusion of additional domains. In contrast to a model-based deductive approach, relevant domains can also be identified through patients themselves. When developing a measure, patient focus groups are convened to discuss the major aspects of wellbeing and functioning from the patients' perspectives, which is then used to develop a conceptual model with respective dimensions and items.

Approaches to assessment

While early assessment approaches were based on interviews, questionnaires assessing relevant dimensions with multiple questions and defined response formats are now predominantly used.⁷ Following general test-theoretical construction principles, the dimensions are assessed via items grouped together to represent a dimension or scale, so that the multidimensionality construct is adequately represented. According to the operational definition of health-related QoL the construct is represented by at least three major dimensions, namely physical, mental (emotional and cognitive), and social well-being. In addition, behavioral or function-oriented dimension regarding patients' capacity to fulfill everyday life roles is included. The dimensions or subscales yield summary scores that constitute a profile of QoL.⁸

In addition to such profile forms, indices, either derived from a combination of scales or as a single approach, are available. Profiles represent the multidimensional scope of the QoL experience, while indices represent the construct unidimensionally with a defined measure point on a ladder between 0 and 100. Indices are most frequently used in health economic assessments.⁹

Respondents

Self-reported instruments can be differentiated from observer-based assessments, with the latter representing an external person's view on the QoL of the individual. Since health-related QoL is a subjective experience, obtaining self-report is mandatory except for conditions in which such assessments are not possible due to cognitive impairment. Observer-based ratings do not represent the patients' view, but rather the perspective of the observer, so that they have to be regarded as additional and independent information.

Information about patient and observer ratings is especially relevant in child and adolescent health-related QoL assessment. The forms are usually presented both to patients for self-report (from 8 years old) as well as to their parents (for observer report). Studies have not found unequivocal results regarding the correspondence between parent and patient reports.¹⁰ This divergence again stresses the importance of treating both sources of information resulting as being from different perspectives and thus as independent.

Scope of instruments

As regards the scope of assessments, generic vs condition-specific (targeted) instruments can be distinguished. Generic instruments represent the full range of health conditions and can be used to report health-related QoL independent of the actual health state of the individual.¹¹

Condition-specific measures refer to the unique challenges of a given health condition, and focus on potential domains affected. Generic instruments are not only used in epidemiological research, but also clinical research, where they enable researchers to compare patient data with non-patient reference populations, but at the potential costs of loss of specificity for the particular health condition under study.¹²

Psychometric properties

The quality of a questionnaire to assess health-related QoL depends on the fulfillment of psychometric criteria. These include, in addition to objectivity, the reliability of the instrument as well as its validity, and for evaluative purposes, its responsiveness to change.¹³ While classical psychometric theory identifies these properties according to established methods of traditional test theory, modern psychometric approaches use item response theory to identify the dimensionality of the respective domain—thus also avoiding the interference of measurement errors. New approaches in assessing health-related QoL domains pertain to the development of item banks in which items from different instruments are assembled to be examined regarding their representation of the unidimensional concepts, such as physical, functioning, or emotional well-being.¹⁴ One example is the international Patient Reported Outcomes Measurement Information System (PROMIS) project, which has, in its 10 years of existence, succeeded in proposing both defined item banks to assess dimensions of health-related QoL (eg, physical functioning), as well as providing the basis for computer adaptive testing. Here, items are not presented in a fixed paper-and-pencil form, but on a computer screen, with an inbuilt algorithm providing a sequence of items according to their likelihood of maximizing information and increasing measurement precision.¹⁵

Cross-cultural comparability

Moreover, the international comparability of measures is an important criterion. Measures developed in one language or culture need to be adapted, translated, and validated in other countries and languages according to standardized protocols to ensure comparability of results. In contrast to the sequential approach, simultaneous developments identify cross-cultural aspects already at the concept and construct level and follow a series of steps. This process of development enhances the international and cross-cultural comparability of items, scales, and instrument properties.¹⁶

To ensure cross-cultural comparability, standard generic measures, such as the Short Form 36 (SF-36) have undergone this process, ie, have been inspected and have been examined for cross-cultural performance.¹⁷ These measures are now available in many languages, providing information on normative population data that can be compared with specific clinical reference groups. Cross-cultural testing of instruments includes a forward-backward translation procedure with international harmonization as well as psychometric testing of the instrument in each language, with additional and comparative inspection of psychometric indicators across different language versions. The difference between sequential and simultaneous cross-cultural development is that item identification and writing (eg, through focus groups) and feasibility testing (eg, through cognitive debriefing exercising) is identical. Thus, different countries and cultures produce resulting pilot and field-test versions of the instrument, which can be cross-culturally tested either per country or across all countries. A “take one country out.” approach is a helpful tool for identifying cross-cultural comparability in situations in which national patient samples are small. This approach consists of iteratively taking one country out of the common data set and sequentially testing the stability of the measurement model.¹⁸

Steps of instrument development

Over the years the process of instrument development and testing has advanced considerably, and standards for developmental steps have been set.^19,20 One of the first issues when developing an instrument is the adequate representation of the respondent's subjective experience. A worthwhile alternative to relying on expert opinion, symptom lists, or diagnostic classification systems, is asking the patients concerned. Patients can be invited to a moderator-guided focus group discussion about how their condition impacts on their QoL. After recording, patient statements may be used for item development and the construction of a dimensional measurement model. Item wording and choice of answer categories are the next steps, leading to the preparation of the pilot test version of the questionnaire. In case of multinational instrument development, a forward-backward translation process follows so that language versions reflecting core contents can be tested respectively. Pilot testing involves completing the questionnaire followed by a review of the acceptability of the measure in terms of detailed feedback on the item level by means of patient interviews or written response. Results of such cognitive debriefing exercises, together with preliminary psychometric testing results, provide the opportunity to modify the instrument before including it in a field test.

Field testing of the new questionnaire requires an adequate sample size, with patient numbers depending on the psychometric approach chosen. For example, at least five respondents per item are recommended for exploratory factor analysis, and many more for structural equation modeling. Inspection of item distribution characteristics guides decisions on inclusion or exclusion of items, and factor analysis informs about the composition of scales. In classical test theory, reliability testing with eg, internal consistency determined via Cronbach's α is frequently chosen, as is factorial validity to determine construct validity. A retest phase aids in determining test-retest reliability and—in absence of a prospective design—may inform about differences between groups with reported clinical change as compared with those without. Many of the newer measures, such as comprehensive modular measurement systems, are based on probabilistic rather than classical test theory, and an item response theory approach is also leading the way to the construction of comprehensive item-banks derived from available instruments.²¹

Regulatory agencies have provided guidance on instrument development to ascertain that the development process complies with quality criteria. When registering a randomized clinical trial (RCT) to evaluate a potential QoL benefit for a given treatment, it is required to document that the instruments chosen are methodologically sound—ie, that they comply with development and testing standards as documented in an instrument dossier. Claims related to an expected QoL impact of treatment regimens, as tested in clinical trials, thus have to be based on an approved dossier.²²

QoL assessment in the mental health field

A reflection about QoL assessment in psychiatry requires an overview of instruments available, assessment purposes and study designs, as well as challenges encountered in applying these instruments.²³

Generic instruments

To date, there is a wealth of generic instruments assessing health-related QoL in adults as well as children, adolescents, and their parents, which have been developed according to psychometric standards. The most widely used generic QoL instrument for adults is the SF-36 with its eight dimensions and two summary scores for physical and mental health.²⁴ Additionally the WHOQOL-BREF questionnaire and the Nottingham Health Questionnaire are commonly used.^25,26 This is also the case for the European Organization of Research and Treatment of Cancer (EORTC) questionnaire which started out as a cancer-specific questionnaire, but is now included in research across many health conditions beyond oncology.²⁷ These instruments, as well as the utility-based health economic instrument, have been employed in a range of studies as well as in psychiatry. Several instruments, however, have been specifically developed for this patient population, to represent the challenges of the specific mental health conditions.

Condition specific instruments

The development of health-related QoL assessment in psychiatry was somewhat delayed as compared with other fields in medicine.²⁸ This delay, or even reluctance, regarding assessment appears to be due to a longstanding view that classical psychiatric instruments are already measuring well-being and therefore reflect the QoL of patients. The focus of psychiatric assessment, however, is mainly expert-based. If patient-based, it is limited to an individual account of symptoms, which is not identical to self-reported patient perception of health as a whole.²⁹ The first phase in assessing health-related quality in psychiatry was the development of interview schedules that unfortunately were relatively time-consuming for the patients and difficult to score.³⁰ More standardized approaches have subsequently been developed, firstly for schizophrenic patients³¹ and depression, and consecutively for other mental health conditions. Many of those have now been tested in different patient populations nationally and internationally.^32,33

Especially noteworthy is the development of patient-reported measures to assess well-being in schizophrenia,^34,35 such as the Subjective Well-being under Neuroleptic treatment (SWN) questionnaire.³⁶ Instruments have also been developed to assess QoL in depression,^37,38 for patients with anxiety disorders,^39,40 for those with obsessive-compulsive disorder^41,42 and substance-dependent patients.^43,44 More recently, questionnaires have been constructed for many more disorders including post-traumatic stress disorder (PTSD),⁴⁵ personality disorders⁴⁶ and for children with mental health problems.⁴⁷ While it is beyond the scope of the current paper to review these measures, individual papers in the current issue of Dialogues in Clinical Neuroscience provide more detailed information. Prigent and colleagues⁴⁸ have recently reviewed available measures and their implementation in research. They found them to be increasingly used also in randomized controlled trials and conclude that there is much convergence across measures. This suggests that the construction of item banks for specific assessment contexts and populations in psychiatry is worthwhile.¹⁴

QoL assessment in psychiatric research and practice

Health-related QoL instruments in mental health have been included in epidemiological and health economic studies. Epidemiological surveys have contributed to making the burden of psychiatric diseases publicly known^49,50 and health economics work has enumerated the cost of mental health problems from the perspectives of patients the health care system, and society.⁵¹ The majority of studies, however, identify patients QoL within or across mental health conditions, predominantly in cross-sectional designs, but increasingly also over time. As concerns evaluating effects of treatments, designs range from longitudinal studies and prospective assessments of QoL, before and after treatment to randomized clinical trials, with QoL indicators serving as a secondary or ancillary rather than primary outcome criteria.^52-56 Relatively rare are studies comparing psychopharmacological vs psychological interventions with regard to QoL end points.⁵⁷ Inclusion of QoL assessment in routine care or quality assurance is a relatively new field in need of increased research efforts. Documenting the effects of in- and outpatient psychiatric care is not only important for benchmarking across care providers, but also for the patients themselves. This is also true for the care they receive in private practice, where it is hoped that evidence-based information about treatment is embedded in care provision and the interaction between patient and physician.^58,59

Challenges of health-related QoL assessment in psychiatry

Specific challenges remain for the mental health field despite major developments in QoL measurement, especially regarding the precision of assessment, and despite increasing implementation in clinical research. These challenges relate to the differentiation of concepts, the patient reporting capacity, the role of observer based ratings, and the choice of purposeful measures.

Many studies have reported the high correlation between depressive symptoms and QoL ratings, suggesting that measuring depressive symptoms would make QoL assessment superfluous.⁶⁰ That this is not the case is impressively documented by many papers reporting that, although depression affects QoL response, other QoL domains vary independently.⁵⁷ Also, the capacity of the patients to reflect on and report about their QoL has been viewed critically. Reliability and validity of reporting has been questioned because of cognitive impairments and distortions that characterize several mental health conditions. It is important to note that while patients in acute psychoses might not be able to respond to health-related QoL questionnaires, this is not the case for patients in remission. Reliable and consistent reporting has been demonstrated in many studies of patients suffering from mental health problems.^61,62

The skepticism regarding patient self-report has led to considering observer-based reports by family or by medical staff as an alternative. It is clear, though, that observer-based ratings are not a simple reflection of patients' own perspectives and therefore cannot be taken to serve as a “proxy.” Nevertheless, caregiver information is valuable, both to understand the perception of family or staff and also to assess caregivers own QoL.⁶³

Concerning the choice of measures, much speaks for the selection of condition-specific instruments, as they are likely to reflect the patient concerns appropriately. Generic assessment is helpful when comparing patient groups with age- and gender-comparable national norms, or when aiming at health economic analysis. The widespread utilization of generic instruments such as the SF-36 health survey reflects this approach.²² Because of the different condition-specific instruments available for a given condition, it is not easy to settle for one of them.⁶⁴ Criteria for choice are psychometric quality and dissemination of the measure, but also compatibility with the target population and the specific research question under study, eg, descriptive, comparative, prognostic, and evaluative purposes.¹ Consensus in selecting a measure for a mental health condition in upcoming studies simplifies pooling of data across studies and facilitates evidence-based recommendations from meta-analyses.

The clinical use of information derived from QoL assessment in psychiatry is an important argument when critically evaluating the field. Knowledge about QoL benefits associated with treatments from clinical trials may shape care. An example for an innovative approach is the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) trial⁶⁵ with its systematic comparison of treatment regimens using patient-reported outcomes in large patient cohorts over time. Reconciling PRO assessment with the International Classification of Functioning is an innovative way to bridge the gap between mental health and other fields of medicine.⁶⁶

Since QoL assessment has become a research field in itself, the quality of instruments as well as the validity of its results is more highly valued. Recently, regulators, in line with the American Federal Drug Administration, have provided recommendations regarding the development of patient reported outcomes, especially health-related QoL.^67-69 Here, criteria are outlined, which an instrument has to fulfill in order to be acknowledged for claims regarding treatment-related QoL improvements in clinical trials. The necessity to develop these instruments according to specific guidelines and to provide dossiers to ascertain the methodological quality of the instruments underlines the importance of state-of-the-art development and testing of instruments. Such recommendations are also a consequence of an earlier inflationary use of the term “quality of life,” claiming QoL benefits from ancillary patient statements, interviews, or untested questionnaires.

Conclusion

Better care by including QoL indicators in clinical research and practice is a goal for the mental health field. Research has shown that therapeutic alliance, patient-physician interaction, and adherence is improved when patient concerns are respected and QoL is considered.⁷⁰ The question of whether QoL should be used as a screening tool and potential indication for treatment has been debated in medicine in general and specifically in psychiatry, but outcomes research still has priority.

Several institutional care providers have begun to routinely assess health-related QoL of their clients before, during, and after treatment, and have identified potentials for improvement of care.⁷¹ Although QoL is considered as an indicator of the quality of health care services in several countries, national regulations differ across countries in their request to document QoL as an outcome in clinical trials, health services research, and routine clinical care.

The precision with which QoL assessment can now be carried out, and the benefit of this information in terms of its impact on health care improvements, lead the way towards greater acceptance of patient reported outcomes. QoL indicators are ready to be included in the evaluation of health care in the individual patient, regarding the treatment approach and the care program.⁷² Models that relate clinical symptoms to QoL states, in addition recognizing the role of mediators such as coping or living conditions, are expected to be most helpful to assess and to understand the structure of subjective health in psychiatric patients. Potentials to improve QoL through specific interventions as well as to document these changes may also impact on reimbursement policy.

In contrast to somatic medicine, QoL research evolved with some delay in the mental health field. Despite early reflections on QoL in psychiatry, standardized approaches to assessment are relatively recent. While well-being and functioning of patients have always played a major role in individual patient-physician interaction, the challenge to measure and act upon patient well-being and functioning is likely to produce new accents for clinical research, in health care delivery, and in individual patient encounters. This is especially important for compliance: medication regimens associated with better QoL are also associated with higher adherence.⁷³

The evolution of the QoL field and the refinements of assessments methods make it possible to review the state of the art of QoL assessment in psychiatry. This seems especially timely when considering inclusion of patient-reported outcome assessment in psychiatric research and practice with accompanying guidelines and regulations from the national European and regulatory agencies.

Psychological interventions that are more traditionally accepted in psychiatry as compared with other disciplines are also helpful tools for improving patients' wellbeing. The rediscovery of the patient with an interest in understanding his or her concepts of disease and individual treatment goals is a major step forward toward providing care to persons with mental health problems.