Saving time and effort: Best practice for adapting existing patient-reported outcome measures in hepatology

Abstract

It is increasingly recognised that collecting patient reported outcome measures (PROMs) data is an important part of healthcare and should be considered alongside traditional clinical assessments. As part of a more holistic view of healthcare provision, there has been an increased drive to implement PROM collection as part of routine clinical care in hepatology. This drive has resulted in an increase in the number of PROMs currently developed to be used in various liver conditions. However, the development and validation of a new PROM is time-consuming and costly. Therefore, before deciding to develop a new PROM, researchers should consider identifying existing PROMs to assess their appropriateness and, if necessary, make adaptations to existing PROMs to ensure their rigour when used with the target population. Little is written in the literature on how to identify and adapt the existing PROMs in hepatology. This article aims to provide a summary of the current literature and guidance regarding identifying and adapting existing PROMs in clinical practice.

Core Tip: In the last few years, there has been a rapid increase in the number of patient-reported outcome measures (PROMs) in hepatology and, therefore, the choice between which of these PROMs to use can be difficult. This paper aims to illustrate ways of identifying existing PROMs and outlines key considerations and good practice with respect to their adaptation in clinical practice or research in hepatology.

INTRODUCTION

What are patient reported outcome measures?

Patients are treated by healthcare providers with the primary goal of improving their health and well-being. Historically this improvement in health has been judged by improvement in biochemical, histological, radiological or clinical assessments. This approach does not always correlate with improvement from the patient perspective. From a patient perspective improving health is reflected in the documentation of their symptoms and experience of healthcare provision, which are more appropriately collected directly from the patient[1]. With a move towards shared-decision making and patient-centred care, there is growing recognition within the healthcare community of the importance of the patient perspective and the need to consider patient reported outcomes (PRO) as a key component of a holistic approach to patient care.

The U.S. Food and Drug Administration (FDA) defines a PRO as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else”[2]. The European Medicines Agency state that “Any outcome evaluated directly by the patient himself and based on the patient’s perception of a disease and its treatment(s) is called a patient-reported outcome (PRO)”[3]. Patient reported outcome measures (PROMs) can be broadly classified as generic or disease-specific instruments. Generic PROMs assess general aspects of health and can be applied across multiple conditions. Disease-specific PROMs, on the other hand, assess specific aspects that are related to a particular condition. PROMs are designed to measure aspects of health that can neither be directly observed or are not feasible to observe[4]. Broadly speaking, collection of PROMs from the patient can be classified into three main categories based on the outcomes measured: Health status and quality of life: patients’ health and well-being as indicated by patient report; Patient satisfaction: patient-reported satisfaction with their medical treatment or care; Resource use: patients’ reported use of health services and resources; Patient knowledge questionnaires: patients’ understanding of medical conditions and the treatment.

What is currently driving the use of PROMs?

PROMs were initially developed for research use and many regulatory authorities such as the European Medicines Agency (EMA) and the FDA advocate their use[2,5,6]. Recent consensus guidance also recommends the inclusion of PROMs in clinical trial designs[7]. The collection of PROMs aligns well with the increased drive within healthcare organisations for value based healthcare, whereby organisations aim to achieve the best possible outcomes for patients with the available resources[8]. As more clinicians recognise the benefit of collecting PROMs in addition to measuring clinical outcomes, PROMs have seen an increased use in routine clinical practice[9].

As a consequence of the drive to collect PROMs, there has also been an increase in the number of PROMs developed, validated, and used. The King’s Fund report reflects on this as ‘‘a growing recognition throughout the world that the patient’s perspective is highly relevant to efforts to improve the quality and effectiveness of health care’’ and that PROMs are likely to become ‘‘a key part of how all health care is funded, provided, and managed’’[10]. This has been illustrated in the United Kingdom when, in 2009, the United Kingdom Government implemented the routine collection of PROMs in England for four routine elective surgical procedures — hip and knee replacement, groin hernia repair, and varicose vein surgery (https://digital.nhs.uk/data-and-information/data-tools-and-services/data-services/patient-reported-outcome-measures-proms), in order to compare performance between providers. It is likely that routine PROMs collection will be extended to more conditions in the future.

LITERATURE SEARCH TO IDENTIFY BEST PRACTICE FOR THE ADAPTATION OF EXISTING PROMS

In order to identify relevant literature regarding best practice and guidance for the adaptation of existing PROMs we undertook a scoping review of the literature. This scoping review aimed to explore the extent of the literature within the PROMs field regarding best practice/guidance for PROM adaptation without describing findings in detail[11,12].

We undertook a review of the literature to identify key papers/guidelines for the adaptation of existing PROMs. We searched PubMed and the Cochrane database (https://www.cochrane.org/). In order to limit the search, we searched for literature in the English language published within the last 10 years. Reference lists of relevant identified publications were also hand searched to identify further relevant literature. We also undertook a Google^TM search to identify relevant publications.

Details of the search strategy are presented in Table 1. The searches were conducted on 17 February 2020. The inclusion and exclusion criteria are listed in Table 2.

Table 1.

Keyword combinations used in the literature search

Keyword combinations
Patient Reported Outcome Measures OR patient reported outcome measure1 OR PROM1OR PRO1 OR patient reported outcome1 OR patient outcome1 OR clinical outcomes assessment	AND	Guidance OR guidelineOR recommend1 OR good practice OR best practice OR instrument development OR adapt1 OR modif1 OR develop1 OR establish1 OR efficien1 OR standard1 OR measurement properties

¹It indicates where truncated versions of the wrord was used. Search was restricted to the last 10 yr and publications in the English lang.

Table 2.

Inclusion and exclusion criteria used when screening identified studies

Inclusion criteria	Exclusion criteria
(1) Articles presenting guidance and recommendations for use or adaptation of existing PROMs; and (2) PROMs methodology papers	(1) Papers detailing development and validation of new PROMs; (2) Papers not presenting guidance or recommendation for adaptation of PROMs; (3) Study protocols; (4) Papers that focus on implementation of PROMs in research or clinical practice; (5) Conference abstracts; (6) Not published in English; or (7) Published longer than 10 yr ago

PROMs: Patient reported outcome measures.

We carried out an initial title screening, then abstract screening to identify relevant papers that fitted the inclusion criteria, which we then reviewed fully. We identified specific themes related to the adaptation of existing PROMs which we regarded as recommendations/good practice and have structured the paper according to these identified themes.

FINDINGS

Supplementary Table 1 illustrates the publications identified as part of the scoping review of the literature. The guidance identified within these publications is organised under the specific headings of: defining the requirements of a PROM, identifying and appraising existing tools, adapting existing PROMs, issues of content validity and getting the right people involved.

Defining the requirements of a PROM

In order to provide meaningful information, PROMs need to be appropriately developed and validated according to robust criteria. The psychometric validation of PROMs can be complex and time-consuming and requires evidence of numerous facets including validity, reliability and responsiveness[13,14]. Given the growth in the number of available PROMs, even within the same condition, the old adage “don’t reinvent the wheel” should be the first principle applied before taking the decision to embark on the development of a new PROM. Consequently, to enable researchers to appraise the quality of existing measures with the aim of ascertaining whether a new measure is needed, researchers must first establish a clear definition of what is required of the PROM.

The requirements of the PROM need to be identified at the outset[15]. Consideration should include what the PROM aims to measure, whether the PROM should be generic or disease specific, the clinical condition of interest, the specific population for which the PROM will be applied and whether it will used as part of routine clinical care or research. These factors will help to determine whether existing PROMs are suitable or can be adapted. A useful overview and starting point for deliberations is provided by Luckett and King[16].

A generic PROM may allow comparison of patient outcomes across different conditions, however it will have less focus on specific symptoms relating to a condition. A disease-specific PROM will have a more defined focus on the condition itself and will be more sensitive to changes in the condition over time and its associated symptoms but may be longer and therefore the burden to the patient may be greater. If a disease-specific PROM is required, one needs to define the specific population. For example, a PROM developed to measure pruritus in primary biliary cholangitis may not be suitable for measuring pruritus in intrahepatic cholestasis of pregnancy as the patient experience of dyspnea may differ across these two clinical conditions[2,16]. It is also important to consider whether the PROM will be used within a routine clinical setting or a research setting[17]. In a clinic setting where time may be limited, the burden to the patient and the feasibility of completing the PROM need consideration. Within a research setting, time may not be as limited and longer, more detailed PROMs can be considered[17]. The proposed method of administration of the PROM is also important and authors planning on using a PROM should ensure that it has been appropriately validated for their proposed administration method[2]. Issues such as respondent and administrator burden — length, formatting, font size, instructions, privacy, literacy levels etc. also need to be considered[2]. Figure 1 provides an overview of the first steps required before choosing to develop a new PROM.

Figure 1.

Initial steps required prior to deciding whether to develop a new patient reported outcome measure. PROM: Patient reported outcome measure.

Luckett et al[16], have provided useful principles to consider when selecting a PROM: Selection of PROMs should be considered early during study design — selection should be driven by the research objectives, samples, treatment and available resources; For the primary outcome, choose as ‘proximal’ a PROM as will add to knowledge and inform practice — ‘proximal’ (symptoms) vs ‘distal’ (overall quality of life); Identify candidate PROMs primarily on the basis of scaling and content — which items/scales offer best coverage of the impacts of interest and which aspects of score distribution will be most meaningful to consider? Appraise the reliability, validity and ‘track record’ of candidate PROMs — look beyond articles that focus on evidence of validity and reliability; Look ahead to practical considerations — patient and staff burden, methods of administration, cost, availability of translated versions, guidelines for scoring and interpretation; Take a minimalist approach to ad hoc items — where content is similar, PROMs with proven psychometric properties are preferable to ad hoc measures developed by the researcher[17].

The EAPC[17] have provided guidance regarding the selection and measurement of PROMs in palliative care. This guidance reflects on the content of the PROM and the feasibility of its collection within the clinical environment. Some of the points made are particularly useful when considering adapting existing measures and can be usefully applied to other conditions: Use PROMs that have been validated with relevant populations and make sure these are sufficiently brief and straightforward; Use multi-dimensional measures; Use measures that have sound psychometric properties; Use measures that are suited to the clinical task being delivered and are also suited to the aims of your clinical work and the population you work with; Use valid and reliable measures in research that are relevant to the research question and consider patient burden when using measures[17,18].

Identifying and appraising existing tools

Once one has defined the scope of the PROM, possible candidate measures can be identified. This process will determine whether there is a need for a new PROM. It will also allow for the identification of PROMs that could, be adapted, shortened, translated or expanded.

Given the large number of available PROMs, there are several ways in which possible candidate PROMs may be identified. Identifying systematic reviews of PROMs for a particular clinical area may prove to be particularly fruitful as good reviews will assess the methodological quality[13-16] of the PROMs identified and provide a summary of the PROMs that offer the most promise. In addition to undertaking literature reviews, there are also databases and online resources that can be searched to identify existing PROMs. Some of these resources are generic and cover many conditions, whilst others provide a resource for disease-specific PROMs. Table 3 provides some examples of resources that can be used to identify candidate PROMs for adaptation.

Table 3.

Some examples of online resources that can be used to identify candidate patient reported outcome measures for adaptation

Resource name and web address	Resource Information
ePROVIDETM (https://eprovide.mapi-trust.org/)	This is an online service provided by Mapi Research Trust and is the official licensor and distributor of more than 450 clinical outcome assessments (or PROMs). This resource allows you to search for PROMs within a specific clinical area and presents: a summary of each tool; the authors of the tool; different version of the questionnaire; the copyright owner; the specific condition/disease in which the PROM has been used; the original language the PROM was developed in; references to the original PROM development publications; and a list of any validated translations of the original questionnaire. If a PROM is deemed appropriate but no valid translation exists, there is also an opportunity to submit a request to undertake a linguistic validation of the questionnaire
COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) (https://database.cosmin.nl/)	The COSMIN initiative (https://www.cosmin.nl/) aims to “develop methodology and practical tools for selecting the most suitable outcome measurement instrument). Their mission statement is: “to improve the selection of outcome measurement instruments of health outcomes by developing and encouraging the use of transparent methodology and practical tools for selecting the most suitable outcome measurement instrument in research and clinical practice”. The COSMIN website provides a link to the COSMIN Database for Systematic Reviews which can be searched to identify literature reviews that have been undertaken within specific clinical areas. The database provides a summary of the review and the PROMs that formed part of the review and links to the original publications. Examination of these reviews is useful in assessing whether an existing PROM may be appropriate to use. Many of these reviews will also present a synthesis of each PROM with an assessment of its methodological quality and validity according criteria outline in more or more of the guidance documents available[2,44,61-65]
International Consortium for Health Outcome Measurement (ICHOM) (https://www.ichom.org/)	As part of a wider initiative ICHOM publish Standard Sets. ICHOM Standard Sets are defined as ‘standardized outcomes, measurement tools and time points and risk adjustment factors for a given condition. Developed by a consortium of experts and patient representatives in the field, our Standard Sets focus on what matters most to the patient’
Measures for Person Centred Coordinated Care (http://p3c.org.uk/about)	Set-up as a result of an NHS England funded project. This online resource describes itself as providing information “about measures for Person Centred Coordinated Care (“P3C”) for people with long-term conditions (LTCs), multiple-LTCs, and those at the end of their life (EoL)”. It provides a compendium of measures — defined as PROMs and patient reported experience measures (PREMs) — that can be utilised within programs that aim to deliver or evaluate P3C in the target populations”
European Organisation for Research and Treatment of Cancer (EORTC) (https://www.eortc.org/tools/)	Amongst other resources, the EORTC website provides a list quality of life questionnaires that have been developed and validated for cancer patients that are available for academic use
Oxford University Innovation/University of Oxford Clinical Outcomes Assessments (https://innovation.ox.ac.uk/clinical-outcomes/patient-reported-outcomes-measures/)	The PRO portfolio is made up of condition-specific questionnaires aimed at assessing the outcome for patients being treated for a range of medical conditions

PROMs: Patient reported outcome measures.

If these strategies do not identify any PROMs, conducting a new systematic review may uncover PROMs for consideration or items/questions in existing PROMs that could be included in the development of a new PROM[14]. Prinsen et al[13], have formulated a useful ten step process for conducting such systematic reviews of PROMs[13,14]. Such a systematic review should be conducted in accordance with the guidelines outlined by internationally recognised COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN). COSMIN provide detailed information and tools to aid this process on their website (https://www.cosmin.nl/tools/guideline-conducting-systematic-review-outcome-measures/). This will ensure that the methodological rigor of the PROMs identified is appropriately appraised.

Once PROMs have been identified, the tools should be reviewed for their content and appropriateness for the desired application[13,14-22]. This process will also help to identify relevant questions/ items that could be used to develop a new PROM or adapt an existing one.

Adapting existing PROMs

Researchers need to consider the existing PROM literature to determine whether an adequate instrument exists to assess and measure the concepts of interest. If no PROM exists, a new PROM can be developed or in some situations, a PROM can be adapted by modifying an existing instrument[2].

Examples of instrument modifications include: (1) Making minor cultural/Language adaptations within the same source language; (2) Undertaking a cross-cultural adaptation that includes translation into a different language; (3) Including additional items/questions; and (4) Shortening the original instrument. Such PROM modification may be necessary to enable the PROM to be used with a different population or with a different population age group (for example, modification of an adult PROM for a paediatric population), to facilitate its use in a different language, for use in a different disease stage or treatment (for example cancer stage, or for a newly diagnosed condition rather than a pre-existing condition), or to reduce patient burden.

The FDA state that when a PROM is modified, evidence of adequacy for its new intended use should be provided and that “additional qualitative work may be adequate” to test such modifications[2]. Such changes include: Changing an instrument from paper to electronic format; Changing the application to a different setting, population or condition; Changing the order of items, item wording, response options, or recall period or deleting portions of a questionnaire; Changing the instructions or the placement of instructions within the PROM.

Snyder et al[21], outline some requirements to revalidate a PROM when changes such as these are made to an existing PROM.

The search for PROMs may identify existing instruments that have proven validity for the population being studied and can be applied without requiring any adaptation. Alternatively, a PROM may be identified that appears appropriate but requires modification. Before engaging in any adaptation, it is important first to contact the PROM developer/copyright holder to ask for permission to make changes to the original PROM. Wild et al[22], have recently published guidance from the International Society for Quality of Life Research (ISOQOL) Translation and Cultural Adaptation Special Interest Group (TCA-SIG) regarding copyright of PROMs. Failure to gain appropriate permissions for use and adaptation may result in legal challenges due to breaches in copyright. The authors present recommendations to prevent future conflict that includes: Protecting the copyright of the original PROM; Writing a contract; Taking care when publishing; Establishing rules; Making the copyright notice visible; Maintaining copyright of the PROM and any derivatives with the original author; Centralising distribution; Getting legal counsel; Clarifying the copyright situation with respect to legacy PROMs.

It is therefore prudent for researchers considering the adaptation (including the translation of existing PROMs) to identify and obtain agreement from the copyright holder prior to any adaptation[22].

CROSS CULTURAL ADAPTATION

If a suitable PROM is identified and has appropriate content validity for the population of interest, but was developed and validated in a different language, cross cultural adaptation represents an efficient way of adapting an existing PROM. Cross-cultural adaptation manages language translation and cultural adaptation issues with the aim of ensuring a PROM is sensitive to the linguistic and cultural needs of the target population[22]. A PROM that has undergone rigorous cross-cultural adaptation is suitable for use in multinational and multicultural studies.

It is important that any cross-cultural adaptations of PROMs are undertaken rigorously. Guidance regarding the process of cross-cultural adaptation has been described in a wide range of publications[22-25]. The lack of ‘gold standard’ guidance for cross-cultural validation prompted the Patient Reported Outcome (PRO) Consortium, to update and develop further guidelines for best practice in the translation process[26]. These guidelines are based on the ISPOR Task Force guidelines, updated with greater detail through a further consensus process[22].

The aim of cross-cultural adaptation is to provide equivalence between the source PROM and the adapted version. Equivalence has many definitions; however, most current guidelines follow the universalist approach proposed by Harachi et al[27], which gives consideration to the influence of culture on how people respond to any given item on a questionnaire. Questions therefore not only require linguistic translation, but they must also be adapted to fit culturally to the target country[26]. For example, a question about difficulty using a fork in eating may not be applicable in a country where a fork is not used in eating[28]. Equivalence can be divided into five categories plus a summary category[22,28] (see Table 4), and this has formed the basis of many guidelines for the cross-cultural adaptation of outcome measures.

Table 4.

Categories of equivalence

Categories
Conceptual equivalence	The domains of the questionnaire have the same relevance, meaning and importance in both cultures
Item equivalence	Individual items have the same relevance in both cultures
Semantic equivalence	The meaning of the items is the same in both cultures
Operational equivalence	The questionnaire can be used in the same way by the target population in both cultures
Measurement equivalence	The two versions have similar psychometric properties
Functional equivalence	This is meant as a summary category of the preceding five categories. It is an overall statement that identifies if both versions “do what they are supposed to do equally as well”

Ultimately, all of the available guidelines are broadly based on a core set of principles that need to be considered when cross-culturally adapting an existing PROM: (1) Preparation. The initial stage of the process is to identify the team that will be responsible for the work, identify suitable translators and gain permission from the original instrument design team to carry out a translation process; (2) Forward translation. Translation of the original language version into the new, target language. It is considered best practice for this to be performed at least twice by different translators from the target country; (3) Reconciliation. Comparison of multiple forward translations and merging them into one translated version; (4) Back translation. The newly translated version is translated back into the original source language; (5) Back translation review. The back-translated version is compared to the original version to assess for equivalence in text and meaning; (6) Harmonisation. All translated versions are reviewed for consistency in language and conceptual meaning; (7) Proofreading. All copies of the questionnaire are proofread to remove mistakes; (8) Cognitive interviewing. The newly translated questionnaire is piloted on a minimum of five people in the target population. Cognitive interviews are performed to identify problems with the questionnaire, difficulties in understanding and meaning of items and any other concerns; (9) Cognitive interview review. Results of the cognitive interviews are reviewed and changes made to the questionnaire if required; (10) Final review and publication. The final version of the translated questionnaire is agreed upon and published for use in the target population; and (11) Cross-cultural validation. Following the production of a culturally and linguistically valid and similar version of the original questionnaire, the adapted questionnaire must then be psychometrically validated against internationally recognised criteria[28,29].

The degree of cross-cultural adaptation required varies depending on the proposed use of the adapted PROM. The intended use of the PROM may influence the number of steps of the above that require completion[29]. Table 5 illustrates five different scenarios where differing adaptation needs are required[25,29]. These range from a situation in which no adaptation is required (i.e., the questionnaire used in the same population, in the same culture and language as originally designed), to full translation and cross-cultural adaptation (i.e., where the questionnaire is to be used in a different country and language).

Table 5.

Scenarios in which different degrees of cross-cultural adaptation are required[25]

	Results in a change in			Adaptation required
	Culture	Language	Country of use	Translation	Cultural adaptation
Use in same population. No change in culture, language or country	-	-	-	-	-
Use in established immigrants in source country	Yes	--	-	-	Yes
Use in another country, but same language	Yes	-	-	-	Yes
Use in new immigrants, not source language speaking but in the source country	Yes	Yes	-	Yes	Yes
Use in another country and another language	Yes	Yes	Yes	Yes	Yes

ADDING TO EXISTING PROMS

If an existing PROM is identified as largely meeting the requirement for the population of interest but following patient and expert consultation and/or exploration of the literature it is perceived to be missing in one or more key areas, there is the potential to adapt the PROM by adding new questions/items. There are various ways in which items can be sourced[17,28]: By asking patients. Patients can be asked to identify additional items and domains that do not exist in the current version of the PROM. Patients are essential to item generation, ensuring item content is both relevant and provides full coverage of the target construct. Qualitative methods such as patient focus groups, interviews and surveys are useful for generating potential new items[30-32]; By evaluating the PROMs identified as a result of reviewing the literature or online resources. This can be an efficient way to generate new items. There are benefits to sourcing items in this way, most notably that there are likely to be a limited number of ways to ask questions about a specific problem such as abdominal pain, vomiting, etc. Moreover, items in existing PROMs have been repeatedly used and validated in many studies and trials; By identifying possible items from clinical observations. These items can be derived by clinicians based on their experience; By asking experts. This is a commonly used approach to generate new items. Similar methods (for example interviews, focus groups and surveys) to those used with patients can be used for gathering information about possible items for inclusion. Although useful for generating items, expert involvement should be used in tandem with other methods and should not be used in place of patient input; By utilising item banks. Item banks are a source of validated items that can be added to existing PROMs. One such item bank, the Patient-Reported Outcomes Measurement Information System (PROMIS^TM) initiative was established in 2004, with the main goal of developing and evaluating, for the clinical research community, a set of publicly available, efficient and flexible measurements of PROs[33]. PROMIS^TM (http://www.healthmeasures.net/explore-measurement-systems/promis/intro-to-promis/List-of-adult-measures) provides item banks that offer the potential for PRO measurement that is efficient (minimizes item number without compromising reliability), flexible (enables optional use of interchangeable items), and precise (has minimal error in estimate) measurement of commonly-studied PROs[33]. The PROMIS group has developed and tested several hundred items measuring 11 health domains[33]. These core PROMIS domains reflect common, generic symptoms and experiences that are likely to apply to people in a variety of contexts or with a variety of diseases[33]. With additional validation, these banks may provide a common metric of represented constructs across a range of patient groups, thereby reducing the large number of different measures currently used in research and allowing researchers to compare these constructs across patient groups in different studies[33].

SHORTENING OF EXISTING PROMS

Although many single-item and short-form symptom measures exist, one reason for adapting an existing PROM is to shorten it and reduce the number of items included in it. This can result in reduced patient burden and facilitate the use of a PROM as part of routine clinical care. As with other aspects of adaptation, it is essential to ensure that a shortened PROM is comprehensible to patients, includes all the relevant items and is fit for purpose. Like cross-cultural adaptation and adding existing items to a PROM, shortening will require further psychometric testing according to recognised criteria[30].

Issues of content validity

Where any adaptation is planned, the PROM will still need to show evidence that it is ‘fit for purpose’ with the intended population[33]. In 2007, the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Health Science Policy Council recommended that an ISPOR Patient-Reported Outcomes taskforce on the use of existing instruments and their modifications be established. This resulted in the publication in 2009 of their report detailing good research practices[16]. A major aspect of this report related to the content validity of PROMs, and stated “evidence of content validity should be obtained from an analysis of the relationship between the instrument’s content and the construct it intends to measure”[16]. This report highlights the key issues relating to content validity issues that should be considered when selecting and modifying existing PROMs[16]: Name and define the concept; Target population and end point; Identify candidate PROMs; Identify or formulate a conceptual framework for the PROM; Assemble and evaluate information on development methods — elicitation focus groups and interviews; cognitive interviews; transcripts; Conduct any needed qualitative work; Assess adequacy of content validity for purpose; Determine the need for modifications or new PROM development.

This guidance has since been updated to include further recommendations from the ISPOR good practice task force[34]. Additional guidance regarding content validity and its consideration with respect to PROM development and adaptation have also been published and includes best practices for undertaking qualitative research to explore content validity, including differences between establishing content validity for new measures compared with existing measures[14,35].

Assessment of PROM content is an important process when adapting an existing PROM and this should involve engagement with, most importantly, patients and also clinicians.

Getting the right people involved

Having identified a candidate PROM for adaptation it is important to ensure that it is appropriate for the patient population being studied. This is particularly important to undertake if the PROM is being adapted for use with a new clinical population. Pre-testing the PROM with patients, clinicians, and subject-matter experts will provide evidence of the PROM’s content validity and help to ensure that any problems are rectified prior to applying the PROM in a large-scale study or implementing the instrument in routine clinical practice.

GETTING PATIENTS INVOLVED

In 2009 FDA guidance suggested that an important first step in establishing that a measure is fit for purpose is to develop a conceptual framework for the PROM and generate relevant items on the basis of direct input from patients with the clinical disease[2,34,35].

Recent guidance[36,37] highlights the various roles that patients and patient advocates can play in PROM studies. These include: PROM design and selection — bringing knowledge of the disease, symptoms and attributes of care with the greatest impact on patients’ lives; PROM implementation and administration — the patient can bring insights based on their experience to guide practical decisions around PROM administration and implementation; Linguistic and cultural input — patients can contribute to the language used in the PROM to ensure it is straightforward and understandable to patients.

Guidance regarding how the patients can be recruited to PROM studies, how to engage with patients, defining the role, provision of training and remuneration[38] has also been provided.

In addition, a framework for fully incorporating public involvement (PI) into PROMs has recently been published[38] which illustrates the extent to which patients can be involved in the adaptation process (see Table 6).

Table 6.

Stages within an patient reported outcome measure adaption process where patients may be involved

Stage	How can PI be involved
Establishing a need for a new or refined PROM	Review existing PROMs; Critique existing PROMs; Determine whether a new PROM is needed
Development of a conceptual model	Review of conceptual model to ensure validity
Identifying item content	Input on study design; Input on culturally appropriate issues; Input on participant facing documents; Input on ethics and governance issues
Item development	Analysis and interpretation of qualitative interviews; Advice and input on wording of potential items
Item reduction	Identify potentially redundant items; Identify items that could benefit from rewording; Input and advice on ordering of items
Pre-testing of items (cognitive interviews/debriefing)	Input on study design, methodology, recruitment, design and content of public facing documents and conducting the interviews; Analyse and/or interpret results
Selection of items for the PROM	Advice on final selection of items; Consideration of number of items to be included; Advice and input into how PROM may be used in clinical settings
Design of the PROM	Advice and input on format and layout of PROM; Advice on instructions of how to complete the PROM, framing of questions, wording of response options, and order of items

PI: Public involvement; PROMs: Patient reported outcome measures.

Existing measures can be reviewed to ensure they match the domains of interest and if further modification may be required[16]. Recent research that explored the level of involvement of patients in the development of PROMs has concluded that what patients consider important can differ from what health-professionals regard as important[30,31]. Content validity is often cited as a PROM’s most important measurement property as unless the PROM can be shown to be measuring the construct of interest from the patient perspective, all other measurement properties may be considered inconsequential[13]. This highlights the importance of engaging with patients as part of the PROM adaptation process.

A variety of qualitative methods can be used by researchers to engage with patients with the aim of maximising a candidate PROM’s content validity (relevance and comprehensiveness) and to pre-test an adapted PROM for comprehensibility and acceptability of instructions to respondents, its items and response format(s). In a recent study examining the developers’ perspective of including patients, the methods used were interviews and/or focus groups, cognitive interviews and feedback questionnaires[30,31]. Maitland and Presser advocate a diverse range of methods, both qualitative and quantitative, for appraising the quality of PROM items and the ability of the items to generate reliable and valid responses[39].

Interview and focus groups are often used to gain insight into the experiences of the target population in relation to the construct of interest and, therefore can be used to generate content for new or additional questions. Cognitive interviews, on the other hand, are normally used to refine item candidates and their response scales. Cognitive interviews capture problems with the cognitive processes associated with item response[40], thereby enabling the developers to evaluate the relevancy, comprehensiveness, comprehensibility and acceptability of the instrument’s items and response scales.

Feedback questionnaires can also provide patients insight regarding their experience of using a health status questionnaire. The QQ10 is one such validated, self-completed questionnaire. It is made up of 10-items scored using a 5-point Likert scale (0 = strongly disagree to 4 = strongly agree) covering two factors, “value” and “burden”. It contains specific items developed to assess a PROM’s content validity (i.e., relevance, comprehensiveness) from the patient perspective[41,42].

GETTING EXPERTS INVOLVED

The assessment of a candidate PROM from the expert clinical, researcher and academic perspective is also important. This can be achieved via focus groups and interviews, by questionnaire survey methods or by employing expert review panels[34]. Ideally, these panels should include clinicians with experience of treating the defined population, PROMs methodologists and researchers. The COSMIN standards recommend a minimum sample size of seven professionals for studies evaluating a PROM’s content validity[14].

Experts can also be utilised to calculate content validity indices (CVI) based on ratings of item relevance. A minimum of three experts is recommended for the purposes of calculating a CVI[43]. A CVI is a consensus indicator of the content validity of an item or scale[44]. It represents the proportion of reviewers who agree that an item is content valid, adjusted for chance agreement. If all reviewers are in agreement, the CVI value for an item (I-CVI) will be 1.00.

Employing different psychometric methods to PROMs development and adaptation

Traditionally the development and psychometric evaluation of PROMs has been based on classical test theory (CTT). CTT is probably still the most commonly applied method in validation studies[20,45]. CTT assumes that the expected value of all the random error will equal zero[46]. There are, however, some disadvantages with CTT, such as sample dependency. This is where the item and scale statistics can only in theory, apply to the specific group of patients who took the test and as such further validation is required for a different population[29]. There is also the assumption of item equivalence, where it is assumed that all items contribute equally to the final test score and no item weightings are applied[46,47].

As a result of the disadvantages of CTT, modern psychometric methods of item response theory (IRT) and Rasch measurement theory (RMT) have been developed[48]. Rather than considering the questionnaire as a whole, as in the case of CTT, these methods allow analysis at the individual item level[49]. They also provide sample-free measurements (i.e., the results are applicable to all similar groups once the validation process has occurred). In IRT, additional model parameters are used to model the relationship between the individual’s trait, the item property and the probability of endorsing an item. The assumption in IRT is that the “probability of answering any item in the positive direction is unrelated to the probability of answering any other item positively for people with the same amount of the trait” [28,29]. RMT differs somewhat in that the data are assessed to see if they fit the Rasch model. RMT allows for the creation of linear, interval-level measurement from categorical data. In the case of non-fitting data (items or persons), data can be further examined to understand why they do not fit or removed from the data set. Rasch analysis can be used to examine the properties of previously constructed scales as well as in the construction of new scales, and is important in making interval scales[50].

Although there has been a general shift towards using IRT in more recent years for developing and validating a PROM, there are some drawbacks to its use over CTT. One issue relates to the sample size required. It is recommended that sample sizes based on CTT should be large enough for the descriptive and exploratory pursuit of meaningful estimates from the data, starting with a sample of 30 to 50 subjects may be reasonable in some circumstances[51]. At later stages of psychometric testing, various recommendations have been given for exploratory factor analyses with recommendations of at least five cases per item and a minimum of 300 cases or to enlist a sample size of at least 10 times the number of items being analysed, so a 20-item questionnaire would require at least 200 subjects[51]. For IRT, sample sizes of a minimum of 150-250 patients has been proposed, with around 500 patients recommended for the latter stages of validation[51,52].

In addition to the inflated sample size recommendations for IRT, additional expertise in the study team is often required, and this may consequently result in greater development costs. Furthermore, strict assumptions in the model can mean that items may be rejected even when they have good content validity if they do not fit the IRT model. CTT should therefore not be disregarded and indeed, most authorities will agree that aspects of both CTT and IRT have a role to play in the validation process of a modern PROM.

CLINICAL USE OF PROMS IN HEPATOLOGY

Most health providers treat patients with the aim of improving radiological, biochemical, histological and clinical assessments[1]. Historically, health outcomes have focused on death or clinical indicators such as infection rates, readmissions, re-operations and adverse events[10]. Many of the symptoms that are experienced by patients in hepatology may be undetected by clinical tests or underestimated by clinicians[53]. The need to assess the impact of health treatment on patients and to demonstrate the value of the care provided to the patient by the provider is now recognised[9]. There is constant pressure on healthcare providers to improve the quality of healthcare provided and make it more patient-centered[54]. Given how much money is spend on treatment, it is important to assess if the treatment given offers value for the money. Clinical applications of PROMs can be divided into: Clinical research and trials: Health regulatory bodies such as the FDA and the National Institute for Health and Care Excellence (NICE) require PROMs to be incorporated into the assessment of new treatments, health technologies or medical devices; Quality improvement projects: PROMs can be very helpful in assessing the impact of a new service or project from the patient perspective. However, PROMs must be integrated into clinical practice with strong incentives to encourage their routine use in such quality improvement projects; Clinical practice: Measuring PROMs in clinical practice contributes to patient-centeredness and measures clinical effectiveness from a patient perspective.

There has been widespread adoption of PROMs use within the research field, especially since FDA and EMA recommended that PROMs should form part of the outcome assessment for new drug trials[2,5]. Reporting guidance for PROMs has also now been incorporated as an extension to CONSORT reporting for trials[7]. The value of collecting PROMs data routinely is now recognised as an important part of driving the delivery and organisation of healthcare and can thereby help to improve healthcare quality[9].

Although individual hospitals and clinicians have started to implement routine PROM collection, widespread adoption is largely restricted to England, Sweden and parts of the United States[9,55]. PROMs have now been implemented in England for the routine collection following some elective surgery (https://digital.nhs.uk/data-and-information/data-tools-and-services/data-services/patient-reported-outcome-measures-proms). Their potential to be used in other clinical areas, such as oncology[56,57], multiple sclerosis is now also recognised. The routine collection of PROMs is not without its challenges however[1,9,55-60].

Some of the practical challenges to routine integration include: the selection of the most appropriate tool; difficulties with patient completion (for example, lack of comprehension, elderly and frail or sick populations); clinical reluctance; achieving high rates of patient participation; operational difficulties; lack of clarity about the PROM; times pressure for patients and clinicians; lack of human resources; recognition of the three dimensions of quality (safety, effectiveness and experience); attributing outcomes to the quality of care; providing meaningful outputs from PROMs data for differing audiences; and avoiding misuse of PROMs[1,9,55,59-65].

McDowell and Jenkinson[66], have developed a series of key strategic priorities that should be considered when implementing PROMs in real-world situations: Ensure international collaboration across multiple stakeholders to agree on a standardised approach to PROM assessment; Develop a comprehensive standard set of recommendations, methods and tools that are applicable to the generation of real-world evidence; Formulate a clear governance process including an ethical framework for how patients should be consented, who selects patients, who has access to data and how data will be used; Establish standard sets of PROMs, electronic tools and administration schedules; Develop and use electronic PROMs where possible; Minimise workload and technical complexity for patients and clinicians; Consider the objectives of the PROM assessment, timings, length of follow-up, strategies for managing missing data and inclusion of diverse patient populations; Ensure data collection adheres to the FAIR (findable, accessible, interoperable and reusable) guidance; Provide guidance on interpretation and use of the data; Ensure both patients and clinicians gain value from PROM collection to tailor their needs.

FUTURE OF PROMS

With new treatment and technologies, mortality is reducing and more patients are living with their illness for longer. As such, there is a growing need to develop and implement PROMs to facilitate the translation of clinical research into practice and, in keeping with the principles of shared clinical decision-making as part of routine clinical practice.

The increased use of digital media presents an exciting opportunity for PROM capture and adaptation. By utilising new technologies to aid PROMs capture and support interpretation, more clinicians may be encouraged to use PROMs as part of their routine clinical care. For example, innovative delivery methods using app or web-based based technology [for example, through data platforms such as REDCap- Research Electronic Data Capture (https://www.project-redcap.org/)] are helping to streamline data capture from patients by facilitating PROM completion on tablets, mobile phones and the internet.

Employing digital media also allows novel methods such as ecological momentary assessment (EMA)[64-68] to be used. EMA refers to a collection of methods often used in behavioural medicine research where a patient repeatedly reports on, for example, their symptoms or quality of life close in time to when they experience them and in their own environment[64]. EMA data can be collected in various ways, including written diaries, electronic diaries and telephone. EMA using mobile phones, for example, could facilitate the collection of PROM data in real-time and overcomes some of the inherent problems of PROMs, such as patient recall accuracy.

The burden of data collection associated with the routine collection of PROMs data in practice can be reduced by simplifying data collection using techniques such as computerised adaptive testing (CAT). CAT involves using a computer to administer a PROM, one question at a time. The CAT then uses an algorithm to choose the subsequent question based on the previous answer given. For example if a PROM is assessing hand function and in response to the first question a patient answers that their hand function is ‘normal’, then there is little to be gained from asking increasingly granular questions about hand problems, which may be more appropriate to someone who has ‘non-normal’ hand function. By pre-selecting questions, the PROM score can be determined without having to ask all of the questions[61,62,69,70].

Another future technology is that CAT and electronic PROMs could be administered by virtual assistants (such as Siri or Alexa, or similar) using voice recognition software to avoid the need for manual form filling to further reduce the manpower required for data collection[59-62].

CONCLUSION

The process of developing a new PROM is often a complex and resource-intensive process. If possible, researchers should first consider whether any existing PROMs could be suitable candidates for use, or if they could be adapted. This review provides a general introduction to PROMs and some background regarding the recent drive to collect PROM data. It then reports on findings from a scoping review that identified good practice and issues that should be considered prior to adapting existing PROMs. These issues are organised under the specific headings of: defining the requirements of a PROM, identifying and appraising existing tools, adapting existing PROMs, issues of content validity and getting the right people involved. The review ends with some insights into different psychometric methods, clinical use of PROMs and future PROMs developments.