Developing Accessible, Pictorial Versions of Health-Related Quality-of-Life Instruments Suitable for Economic Evaluation: A Report of Preliminary Studies Conducted in Canada and the United Kingdom

Abstract

A key component of the current framework for economic evaluation is the measurement and valuation of health outcomes using generic preference-based health-related quality-of-life (HRQoL) instruments. In 2015, a research synthesis reported the absence of conceptual and empirical research regarding the appropriateness of current preference-based instruments for people with aphasia—a disorder affecting the use and understanding of language—and suggested the development and validation of an accessible, pictorial variant could be an appropriate direction for further research. This paper describes the respective rationale and development process for each of three preliminary studies that have been undertaken to develop pictorial variants of two widely used preference-based HRQoL instruments (EQ-5D-3L and EQ-5D-5L). The paper also proposes next steps for this program of research, drawing on the lessons learned from the preliminary work and the demand for a pictorial preference-based instrument in the research community. Guidance for the use of the preliminary, pictorial instruments is also provided.

Electronic supplementary material

The online version of this article (10.1007/s41669-018-0083-2) contains supplementary material, which is available to authorized users.

Key Points for Decision Makers

Modifying questionnaires is sometimes necessary in order for them to be accessible to certain groups of respondents, such as individuals living with communication disorders or learning difficulties.

Developing pictorial versions of widely-used instruments is an alternative to ‘starting from scratch’. This approach has the potential to permit direct comparisons with data obtained from original, unmodified questionnaires.

This paper describes early, preliminary work in a line of research that could provide a significant step towards aligning the current cost-utility framework with clinical contexts comprising communication challenges.

Introduction

The measurement of health-related quality-of-life (HRQoL) is an important component in many areas of health research and policy, such as the routine collection of patient outcomes, administration of population-level surveys, and the clinical and economic evaluation of treatments and interventions. The purpose of a study is the primary consideration when selecting an appropriate HRQoL instrument to use. In economic evaluation, where there is a need for studies to allow for the comparison of findings with those reported across a broad range of different clinical contexts, generic preference-based HRQoL instruments are commonly recommended in national technology appraisal guidelines [1, 2] and recommendations [3]. These standardized instruments facilitate the estimation of health state values—often called utility scores or preference weights—that are used to calculate quality-adjusted life-years (QALYs) [4, 5]. In addition to permitting the comparability of study findings through the use of a common metric, this approach enables preferences from the general public to be incorporated into the measurement of health outcomes.

In the context of aphasia, recent attention has been paid to the use and suitability of standardized HRQoL instruments that are recommended for use in economic evaluation [6–8]. The US National Institutes of Health defines aphasia as a “neurological disorder caused by damage to the portions of the brain that are responsible for language production or processing” [9]. This language disruption affects speaking most obviously, but other language functions—speech comprehension, reading and writing—are also affected to varying degrees. There is a resulting critical impact on communication, given that the exchange of information and viewpoints that are essential to all human activities are often seriously disturbed [10]. Aphasia is most often caused by stroke, but it can also result from a brain tumor, infection, or head injury that damages language regions of the brain. Approximately one-third of individuals who suffer a stroke experience aphasia [11].

Whitehurst et al. [8] identified the absence of any conceptual or empirical research regarding the appropriateness of current preference-based HRQoL instruments in the context of aphasia. They concluded that, unlike aphasia-specific quality-of-life outcome measures that have been developed in recent years [12–14], current preference-based HRQoL instruments may not be accessible to individuals living with aphasia and proposed that the development and validation of a pictorial variant of an existing generic preference-based instrument was an appropriate research direction. The rationale for this approach drew from evidence showing that trained individuals using appropriate resources, such as pictographic material, can obtain important information from individuals with aphasia despite significant language barriers [15]. A considerable literature exists recommending the use of aphasia-friendly formatting (e.g. pictures, key words, and generous spacing of graphics) as a method of improving comprehension and expression of people with aphasia [16–19], although research that applies such aphasia-friendly formatting to preference-based HRQoL measures is in its infancy [6, 8].

The purpose of this paper is twofold. First, the paper describes the respective rationale and development process for each of three preliminary studies—performed, independently, by research groups in Canada and the UK—that were conducted with the objective of investigating the use of pictorial variants of two widely used preference-based HRQoL instruments, the EQ-5D-3L and EQ-5D-5L. Throughout the paper, the studies are described as ‘preliminary’ so as to emphasize the early-stage, exploratory nature of this instrument development work. Second, the paper proposes appropriate next steps for this line of research, drawing on lessons learned from the preliminary work and the demand for (and broader application of) a pictorial preference-based instrument. Guidance for the use of the preliminary pictorial instruments is also provided.

Before describing the rationales and methods adopted in the three preliminary studies, the following section provides a brief description of the key components of generic preference-based HRQoL instruments and specific details of the three-level (EQ-5D-3L) and five-level (EQ-5D-5L) instruments developed by the EuroQol Group. A more detailed explanation of preference-based instruments, including descriptions and comparisons of different instruments, can be found elsewhere [4, 5, 20].

Generic Preference-Based HRQoL Instruments

Preference-based HRQoL instruments comprise two components: a descriptive classification system and a valuation system. The descriptive classification system is made up of the dimension items and respective response options that allow a respondent to describe their health state as one of a finite number of possible health states. For example, a questionnaire with six dimensions (and one item per dimension), each with four levels of response, defines 4096 (4⁶) unique health states. In other words, there are 4096 different ways of completing the classification system. The valuation system is the procedure for scoring each health state defined by an instrument. For generic preference-based instruments, the numerical scores represent the relative value that society places on living in each health state. The set of single index scores are interpreted on a 0–1 scale, where 1 indicates ‘full health’ and zero represents a health state equivalent to being dead. Negative index scores are generated by some instruments, where the negative scores represent ‘worse than dead’ health state valuations. Country-specific sets of scores (known as value sets or national tariffs) are available for some preference-based HRQoL instruments, reflecting the fact that preferences for health states may differ across countries.

A number of preference-based HRQoL instruments have been developed, dating back to the Rosser–Kind Index from the late 1970s [4, 21]. Inevitably, some instruments are more prevalent than others in the field of economic evaluation. A review of 1663 studies that reported using a preference-based HRQoL instrument found that the EQ-5D-3L was the most widely used instrument (64% of studies) [20]. The descriptive classification system for the EQ-5D-3L comprises five dimensions (mobility, self-care, usual activities, pain/discomfort and anxiety/depression), with each dimension containing a single question that has three levels of response [see Appendix 1A in the Electronic Supplementary Material (ESM)] [22]. In 2005, a EuroQol Group Task Force was set up to investigate possible ways to improve the sensitivity of the EQ-5D-3L. This work resulted in the preservation of the same five dimensions, each with one item per dimension, but with five levels of response (see Appendix 1B in the ESM) [23]. As of 2016, there were 176 and 138 official language versions of the EQ-5D-3L and EQ-5D-5L, respectively [24]; details of country-specific value sets for the two instruments are available on the EuroQol website (http://www.euroqol.org/). In addition to the five questions of the descriptive classification systems, the EQ-5D-3L and EQ-5D-5L include a visual analogue scale (EQ VAS), where respondents are asked to self-rate their health state on a vertical 0–100 VAS. For the EQ-5D-3L, the EQ VAS endpoints are labelled ‘best imaginable health state’ (at 100) and ‘worst imaginable health state’ (at zero) (see Appendix 1C in the ESM), whereas the endpoints of the EQ VAS for the EQ-5D-5L are labelled ‘the best health you can imagine’ (at 100) and ‘the worst health you can imagine’ (at zero) (see Appendix 1D in the ESM). EQ VAS provides a quantitative measure of health outcome as judged by the respondent themselves; EQ VAS responses are not used in the derivation of EQ-5D-3L or EQ-5D-5L index scores.

Almost by definition, generic quality-of-life measures, whether preference-based or not, will be less sensitive than condition-specific measures because there is no tailored focus on aspects of quality-of-life that are specific to the condition under investigation. The value of using a generic instrument is in the comparability of findings across clinical specialties (enabling the comparison of cost-effectiveness findings across disease areas) and, typically, generic and condition-specific quality-of-life instruments should be viewed as complements rather than substitutes. In the words of Professor Alan Williams, in reference to instruments developed by the EuroQol Group, “The raison d’être of the EuroQol instrument is to provide a simple ‘abstracting’ device, for use alongside other more detailed measures of health-related quality-of-life …, to serve as a basis for comparing health care outcomes using a basic ‘common core’ of [quality-of-life] characteristics which most people are known to value highly” [25].

Preliminary Studies in the UK and Canada

Rationale

The UK and Canadian research groups became aware of their complementary interests while conducting a research synthesis [8], and a decision was made to form a single, international collaboration. The underlying rationale within both research groups was the same, i.e., current standardized instruments for measuring generic health outcomes for use in economic evaluation were likely to be inaccessible for many individuals living with aphasia. Without a solution to this issue, economic evaluation of interventions for individuals with aphasia could still be performed within a cost-effectiveness (or cost–benefit, or cost-consequence) framework, where health outcomes could be measured in several different ways [26]. However, to perform economic evaluation within a cost-utility framework (without relying on proxy assessments of HRQoL [27]), where outcomes are quantified using QALYs and societal preferences, there was a need to align widely accepted (and often mandated) practices in health technology assessment with the specific challenges of research in the context of aphasia. While the Canadian research group were interested in this general methodological void, the UK research group were motivated to include an accessible preference-based HRQoL instrument in the evaluation of a specific computerized aphasia therapy [28, 29].

Instrument Development Processes

The three ‘pilot’ instruments are presented in Appendix 2 in the ESM. Although the development of the Canadian and UK variants occurred independently, with teams bringing different experiences and orientations to the task, all versions aimed at getting around the language barrier by including pictures/pictographs of the key elements of each item in the questionnaires. For ease of reference, the following abbreviations are used to distinguish between instruments: CPI-3L (Canadian Pilot Instrument of the EQ-5D-3L), UKPI-3L (UK Pilot Instrument of the EQ-5D-3L) and the UKPI-5L (UK Pilot Instrument of the EQ-5D-5L). Abbreviations incorporating EuroQol (or ‘EQ’) terminology are avoided to prevent misconceptions about the official status of any of the pictorial instruments from the perspective of the EuroQol Group.

The CPI-3L was created as part of an exploratory investigation into the development of a communicatively accessible version of the time trade-off (TTO) preference-elicitation procedure, which is a method to directly measure the preferences of individuals. Initially, the primary objective was to create a pictorial version of the standard TTO exercise and map the pictorial TTO responses to a previously validated aphasia-specific quality-of-life measure, the Assessment for Living with Aphasia (ALA) [10, 30]. The CPI-3L was developed and included as a secondary consideration, as an alternative to TTO/ALA mapping for the estimation of QALYs, but the development and validation of a pictorial standardized instrument(s) has since become the primary research objective of the international research collaboration.

The CPI-3L was developed in line with the ‘Supported Conversation for Adults with Aphasia’ approach, an evidence-based method for reducing language barriers in various contexts [15]. Pictorial material for the CPI-3L came from the bank of images (or, where necessary, the creation of new images) developed and maintained by the Aphasia Institute (Ontario, Canada). The process included a series of steps beginning with rough pictorial approximations of EQ-5D-3L content (i.e. dimensions and response options) followed by iterations based on feedback from participant focus groups as well as clinicians. The Aphasia Institute has access to input from people with aphasia on a daily basis, and the development of pictographs, particularly those of a more abstract nature, often go through many revisions based on this input. As shown in Appendix 2A in the ESM, the CPI-3L displays one dimension per page (in portrait orientation), presented at the top of the page in clear emboldened text. Images are presented for each level of response. For the ‘self-care’ and ‘usual activities’ dimensions, further images are included (above the response options) to reflect particular activities and functioning expressed in the question and response options, such as dressing, bathing, housework or leisure activities. The exact wording of EQ-5D-3L response options is retained in the CPI-3L.

The UKPI-3L and UKPI-5L were created as part of a randomized controlled trial for self-managed computerized word-finding treatment for people with aphasia [28, 29]. The UKPI-3L was developed for the CACTUS (Cost-effectiveness of Aphasia Computer Treatment Compared to Usual Stimulation) pilot study [6, 28], whereas the UKPI-5L was developed for the full trial (Big CACTUS) [29]. In the CACTUS pilot study, two speech and language therapists used aphasia-friendly conventions to inform the modifications to the standard EQ-5D-3L (see Appendix 2B in the ESM), which included asking about one health dimension per page (in landscape orientation), presented at the top of the page in clear emboldened text; reducing the response options for each dimension to key words only; providing a picture to illustrate each response option; and a symbol (0, + or ++) to indicate the different levels of response. For example, the ‘mobility’ dimension is presented at the top of the first page with each of the three response options spaced out from left to right in order of ascending difficulty across a page. Using the first response option as an example of the wording reduction, ‘I have no problems in walking about’ was reduced to the emboldened words ‘no problems’. This response option appears beneath a line drawing of a man walking his dog and the symbol ‘0’. The illustrations, one per response option for each dimension, were developed through discussion between a speech and language therapist and an artist. Once drawn, these illustrations were taken to a group of four people with aphasia and their caregivers for feedback on how well the pictures represented the dimensions and response options, which informed any final modifications.

The work in the CACTUS pilot study was built on in preparation for the Big CACTUS trial with a group of four people with aphasia and their caregivers. The people with aphasia worked together in one group with a research speech and language therapist who used a ‘Total Communication’ approach to facilitate understanding of decisions to be made and expressions of preferences [31]. The caregivers, who were all relatives of the people with aphasia, worked with a second research speech and language therapist discussing their views on the extent to which their relatives with aphasia could complete the pictorial version in a meaningful way and their thoughts on design features that would facilitate independent completion of the tool. These two groups, working separately, co-produced the UKPI-5L (Appendix 2C in the ESM). The UKPI-3L and the standard EQ-5D-5L were presented to both groups alongside other aphasia-friendly tools to be used in the Big CACTUS trial to give ideas of a range of potential ways that the questions for each dimension could be presented. Both groups had a preference for a scale being developed in a similar style to that used by the Communication Outcome After Stroke (COAST) patient-rated outcome measure [32], with squares of increasing size and strength of colour to indicate the extent of difficulty with each dimension, and key words contained within each square (e.g. ‘unable’, ‘severe problems’ or ‘moderate problems’). The pictures designed for the UKPI-3L were reviewed for meaningfulness. Where group members identified features of the line drawings that could be improved, these changes were made. In the UKPI-3L, a line drawing was used to represent each of the three levels of response. As there are five levels of response in the EQ-5D-5L, the groups considered how best to use pictures to facilitate understanding of the five-level scale. The consensus was to have a picture at each end of the scale representing the extremes.

Contribution and Future Directions

From the outset, it is important to clarify that none of the instruments described in this paper (and presented in Appendix 2 in the ESM) should be regarded as a ‘validated’ tool—this paper describes preliminary methodological research only. As with any study intending to use a EuroQol instrument, researchers should consult the information provided on the EuroQol website (www.euroqol.org). Although the CPI-3L, UKPI-3L and UKPI-5L are not official variants of the EQ-5D, researchers interested in these preliminary, pictorial instruments are requested to follow the same process (see Acknowledgments for further details).

Awareness of the CPI-3L, UKPI-3L and UKPI-5L has steadily grown as a result of conference presentations (including the American Speech-Language-Hearing Association Convention, the Clinical Aphasiology Conference, and meetings of the EuroQol Group and the UK Health Economists’ Study Group) and the publication of related studies [6, 8]. Interest in a pictorial, preference-based HRQoL instrument also extends beyond application within the context of aphasia—for example, paediatrics and neurodevelopmental disorders (personal communications)—hence the focus on developing ‘accessible’ instruments as opposed to aphasia-specific instruments. The interest and awareness of our preliminary work was the catalyst for writing this paper. Despite the absence of formal validation (to date), documentation of our preliminary work provides clarity regarding the steps undertaken in this innovative line of research. The next stage is to develop a full-scale research program, with the objective of developing and validating official pictorial EQ-5D instruments. The development of the British Sign Language version of the EQ-5D-5L is an example of similar research, where a variant of a standardized instrument has been created to fit the needs of a particular subset of community-dwelling individuals [33].

To achieve our ultimate objective, further research will need to encompass more than the identification of suitable images to depict dimensions and response options of the EQ-5D-3L and EQ-5D-5L descriptive classification systems, and simplification of the EQ VAS. As with any new or modified instrument, the process of development and validation will require several steps. For example, early considerations that require the systematic review of available evidence and/or developmental pilot work include evaluating the relative merits of using (1) line drawings compared with photographs, (2) images for every response option compared with images at the ends of the scale only, and (3) the same images in different clinical contexts. Once a new pictorial descriptive classification system has been developed, examination of measurement properties will be required, such as the assessment of practicality, reliability, responsiveness and validity [34, 35]. The importance of comparability across studies and conditions was the primary reason for modifying EuroQol instruments rather than developing a new instrument. Accordingly, the merit of this research endeavour will rest heavily on the degree to which pictorial versions of EuroQol instruments provide the same health state descriptions as those elicited from the standard versions. One of the final steps in the process would be developing standardized scripts to accompany the pictorial instruments, which may vary across different modes of administration (e.g. researcher supported-completion, family/caregiver supported-completion and self-completion) and clinical contexts. Given the significant complexities of developing pictorial instruments compared with more conventional text-based instruments, a further important issue is whether there is a need for three-level and five-level pictorial variants. Such decisions will be influenced, to some extent, by the current research and debate surrounding the role of the EQ-5D-5L in health technology assessment [34, 35].

The generic measurement of health outcomes is a common feature at the intersection of health economics and healthcare decision making. This paper describes preliminary research and future directions with regard to improving the accessibility of EuroQol instruments through the development of pictorial variants. Such research will provide a significant step towards aligning the current cost-utility framework with clinical contexts comprising communication challenges.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Author Contributions

RP is the Principal Investigator for the CACTUS and Big CACTUS studies and was involved in the conception and development of the UKPI-3L and UKPI-5L. NRL is a named Co-investigator for Big CACTUS and also worked on the CACTUS study (including the conception and development of the UKPI-3L and UKPI-5L). AK, NS-M, JCV and JCH were involved in the conception and development of the CPI-3L. DGTW was responsible for leading the formation of the international comparison, drafted the original manuscript and is the overall guarantor. All authors were involved in the review of the draft manuscript and read and approved the final version prior to submission.

Funding

This paper presents independent research that has received funding from several sources. The UK studies (CACTUS and Big CACTUS) were commissioned by the National Institute for Health Research (NIHR) under its Research for Patient Benefit (RfPB) programme (grant reference number PB-PG-1207-14097) and its Health Technology Assessment (HTA) programme (grant reference number 12/21/01AB); the UK studies were also supported by the Tavistock Trust for Aphasia and the Stroke and Telehealth themes of the South Yorkshire Collaboration for Leadership in Applied Health Research and Care (CLAHRC). NIHR CLAHRC for South Yorkshire acknowledges funding from NIHR. The Canadian study described in this paper was funded by a grant from the Ontario Ministry of Health and Long-Term Care, administered and supported by the Ontario Stroke Network (grant reference number OSN1101-000120). A catalyst grant from the LivWELL Research Group (through Simon Fraser University’s Community Trust Endowment Fund) provided further funding to support the international collaboration. The views and opinions expressed herein are those of the authors and do not necessarily reflect those of any of the funders.

Conflict of interest

DGTW is a member of the EuroQol Group. DGTW, NRL, AK, RP, NS-M, JCV and JSH report no further potential conflicts of interest that go beyond the research funding/grants received and disclosed.

Ethical approval

The CACTUS study received approval from Bradford National Health Service Research Ethics Committee (reference number 09/H1302/20). The Big CACTUS study protocol was approved by Leeds West Research Ethics Committee (reference number 13/YH/0377) and Scotland A Research Ethics Committee (reference number 14/SS/0023). The Canadian study received approval from the Aphasia Institute Quality, Research and Ethics Committee (formerly the Aphasia Institute Advisory, Research and Ethics Board).