How to routinely collect data on patient-reported outcome and experience measures in renal registries in Europe: an expert consensus meeting

Abstract

Despite the potential for patient-reported outcome measures (PROMs) and experience measures (PREMs) to enhance understanding of patient experiences and outcomes they have not, to date, been widely incorporated into renal registry datasets. This report summarizes the main points learned from an ERA-EDTA QUEST-funded consensus meeting on how to routinely collect PROMs and PREMs in renal registries in Europe. In preparation for the meeting, we surveyed all European renal registries to establish current or planned efforts to collect PROMs/PREMs. A systematic review of the literature was performed. Publications reporting barriers and/or facilitators to PROMs/PREMs collection by registries were identified and a narrative synthesis undertaken. A group of renal registry representatives, PROMs/PREMs experts and patient representatives then met to (i) share any experience renal registries in Europe have in this area; (ii) establish how patient-reported data might be collected by understanding how registries currently collect routine data and how patient-reported data is collected in other settings; (iii) harmonize the future collection of patient-reported data by renal registries in Europe by agreeing upon preferred instruments and (iv) to identify the barriers to routine collection of patient-reported data in renal registries in Europe. In total, 23 of the 45 European renal registries responded to the survey. Two reported experience in collecting PROMs and three stated that they were actively exploring ways to do so. The systematic review identified 157 potentially relevant articles of which 9 met the inclusion criteria and were analysed for barriers and facilitators to routine PROM/PREM collection. Thirteen themes were identified and mapped to a three-stage framework around establishing the need, setting up and maintaining the routine collection of PROMs/PREMs. At the consensus meeting some PROMs instruments were agreed for routine renal registry collection (the generic SF-12, the disease-specific KDQOL™-36 and EQ-5D-5L to be able to derive quality-adjusted life years), but further work was felt to be needed before recommending PREMs. Routinely collecting PROMs and PREMs in renal registries is important if we are to better understand what matters to patients but it is likely to be challenging; close international collaboration will be beneficial.

INTRODUCTION

Established renal failure is a chronic disease with significant associated morbidity. Although it affects a small proportion of the general population (0.06–0.12% in countries across Europe in 2012) (Table A.4.3 in [1]), the quality of life of those affected is markedly lower than for those with most other chronic conditions and cancers [2]. Standard dialysis provides the equivalent of only 10% of kidney function so many patients are chronically tired, depressed and suffer pain; and many of these symptoms go unrecognized [3].

The success of treatments for renal failure has been historically assessed using measures considered important by doctors, such as phosphate level or urea clearance. Although instruments measuring the patient's perspective have been available for decades, their incorporation into routine clinical practice has until recently been slow. A number of patient-reported measures exist:

• Patient-reported outcome measures (PROMs) include any metric assessing health, illness or health care benefits from the patient's perspective; in general they take the form of a questionnaire and more specifically a quality of life or symptom questionnaire. In routine clinical practice, PROMs have the potential to highlight relevant symptoms and changes in symptoms, promote patient engagement in their treatment [4] and improve patient outcomes [5]. Summarizing PROM results across individual patients, for example at the level of treatments or hospitals, they could be used to inform a patient's choice of treatment or assess quality of care across different hospitals [6, 7]. The importance of PROMs as end points in clinical trials is also increasingly being recognized [8].

• Patient-reported experience measures (PREMs) capture information about the healthcare experience as perceived by the patient. They can refer to a variety of issues, ranging from cleanliness of facilities to information provision, and from timeliness of transport to family members′ access to health professionals [9]. In routine clinical practice, PREMs can be used to improve quality in clinical services [10].

• Measures reflecting aspects of patient involvement in their health care, including patient activation [11] and informed and shared decision-making [12].

Dialysis and transplantation are in the almost unique position of having an existing infrastructure of regional, national and international registries for collecting and reporting information on all patients receiving treatment. Quality of care, as measured against nationally and internationally agreed standards, can be publicly reported and compared between centres and between countries. These instruments were usually developed for measurement at the patient level, however, and caution will need to be exercised when comparing differences between centres or countries until more is known about their performance at these levels. To be able to report symptom burden and quality of life-adjusted survival alongside laboratory measures such as haemoglobin, calcium, phosphate and dialysis dose would be a major step forward in reporting what is important to patients. While some components of quality of life have been shown to be modifiable (e.g. the physical component summary score of the SF-36 in the Frequent Haemodialysis Network Short Daily Trial [13]), others reflect the broader social construct in which the patient functions which so could prove to be beyond the influence of the health care provider.

Part of this infrastructure is the ERA-EDTA Registry, which collects data on renal replacement therapy (RRT) via the national and regional renal registries in Europe: individual patient data is available from 31 national and regional registries in 17 countries and aggregated data from a further 14 national registries [1]. The data items collected vary by country but can include demographics, primary renal disease, RRT treatment history, date and cause of death, comorbidities, details of physical examination (e.g. weight, blood pressure), laboratory measurements (e.g. haemoglobin, albumin) and details of certain therapies given. At present, none of the national and regional registries report patient-reported outcome/experience measures to the ERA-EDTA Registry, but it is not certain what data are collected locally. This paper reports the results of a survey to capture existing experiences of European renal registries in PROM and PREM collection and a literature review of the facilitators and barriers to registries routinely collecting PROMs and PREMs. It then presents the discussions and conclusions of an international consensus meeting, funded by the QUEST initiative of the ERA-EDTA, aimed at promoting and harmonizing the routine collection of patient-reported data by European renal registries.

SURVEY OF EUROPEAN RENAL REGISTRIES

Prior to the consensus meeting, the organizers contacted all 45 European renal registries (using the ERA-EDTA Registry email contact list) asking for responses to the following questions:

1. Does your registry currently collect any quality of life or patient satisfaction/experience measures?

2. If so, what measures do you use and how are these data collected?

3. If not, have you tried to collect these measures in the past and/or do you have plans to collect these measures in the future?

The survey did not ask about patient involvement questionnaires, patient activation or shared decision making, as these had not been included in the original application for funding. Responses were received from 23 out of 45 registries. Only two registries (Austria and France) reported experience in the collection of PROMs/PREMs and a further three registries reported that they were actively exploring this possibility (Norway, Romania and Sweden). From these responses, there was no obvious consensus on the instruments or methods to use in the collection of PROMs and PREMs, although some barriers to the process were identified. These included low response rates, legal constraints and the burden on staff and patients.

LITERATURE REVIEW AND NARRATIVE SYNTHESIS

A systematic review of the literature was conducted with the primary aim of identifying facilitators and barriers to the routine collection of patient-reported measures by chronic disease registries. Medline and EMBASE databases were searched from 1981 to 2013 to identify papers reporting on the routine collection of patient-reported outcome, experience or quality of life data by chronic disease registries, including both renal and non-renal registries. Full details of the search terms are available in the Supplementary data. Abstracts of papers identified in the search were reviewed and full text versions obtained of those which appeared potentially relevant. Two reviewers (K.B. and F.C.) independently screened the full-text papers for relevancy, and—from the relevant papers—systematically identified any recurring themes across studies.

The search yielded 762 hits, of which 157 papers were deemed to be potentially relevant at the title and abstract screening stage based on pre-defined criteria; 9 papers were finally selected [14–22] and included in the narrative synthesis conducted according to guidelines provided by Popay et al. [23].

All nine of the selected papers were read by two independent researchers (K.B. and F.C.) and the two most relevant papers used to conduct a preliminary synthesis [20, 21]. Relationships between the data were then explored using all nine studies to develop a broad conceptual model in order to provide an appropriate framework for further exploration of relationships in the data (Figure 1). This framework comprised three stages: establishing the need for, setting-up and maintaining routine PROM/PREM data collection. In total, 13 themes were identified and agreed by two independent researchers (K.B. and F.C.) across the three stages of the conceptual model, each comprising a number of subthemes (Table 1). For each theme that resulted from the thematic analysis, we identified facilitators and barriers to implementation as reported in the selected papers, which we then translated into a series of recommendations (listed in Table 1).

Establishing the need for PROM/PREM data in registries was largely broken down into two main themes: recognizing the importance of PROMs and agreeing that, with all the other data they collect, registries provide a good backdrop for the collection of patient-reported data.

Several of the themes and subthemes were identified as important in setting up PROM/PREM collection by registries. Methodological issues associated with PROMs were identified, and recommendations developed based on the need for PROMs expertise, national and international support, stakeholder involvement (including patient and public involvement) and international standardization. As well as these themes, some specific issues relating to resources and practical aspects were identified.

More exclusive to maintaining a PROM/PREM programme were themes related to ensuring useful, high-impact output and the process of maintaining trust in the data. Several themes crossed the ‘set-up’ and ‘maintenance’ parts of the model, including subthemes under design and evaluation, and technological/information governance.

FIGURE 1:

Framework for narrative synthesis.

Table 1.

Summary of subthemes identified by the narrative synthesis process, organized by theme

Establishing need	Recognition of the importance of PROMs across all stakeholders: Highlight the potential clinical outcomes of PROMs Target evidence at all stakeholders Highlight recommendations from national/international organizations Identify research supporting the need for PROMs Capitalize on the increasing recognition of PROMs in other diseases Utilize the power of patient organizations to lobby policy makers
	Agreement that registries are a good way to collect PROMs: Registry-based collection may overcome some of the methodological limitations (e.g. larger samples, multiple time points, ability to account for interactions) Registries can be used as a sampling frame to target specific subgroups Provide evidence that registries are a cost-effective way to collect these data Highlight ability to describe equity of access to treatment/quality assurance Highlight ability to describe characteristics of responders/non-responders
Set up	PROM methodological issues Recognize methodological issues and how these will affect design and interpretation Recognize the importance of using properly translated and validated instruments for a population Consider undertaking a feasibility study when selecting the instruments Consider where the control data for measures came from
	PROM methodological expertise Involve experts at all stages of project design and development Provide staff with training in collecting, analysing and interpreting PROMs data
	National and international support Mandate or incentivise the collection of PROM data Obtain financial support from respected national and international organizations Coordinate expertise and infrastructure at a national level Understand laws and permissions in different countries with respect to PROMs
	Patient and public involvement Generate interest among the public and patients Involve patients in objective setting and the design of data collection and reporting Involve an international umbrella organization of patient associations where possible
	International standardization Agree on internationally standardized systems, definitions, data architecture and timings for data collection Aim for an internationally agreed core data set to enable international data pooling Design a system that can be easily adopted by subsequent countries wanting to join
	Stakeholder involvement in objective setting Involve all stakeholders in objective setting and design Aim to reach consensus of key objectives at/before the design stage Define relationships and responsibilities at the beginning Avoid having too many stakeholders
	Practical considerations including resources Obtain sufficient funding for staff, equipment etc Minimize the burden of administration at the clinic level Provide clear guidelines and training for staff administering PROMs Work out the most cost-effective way of collecting PROMs Consider using a PROM registry that is not disease specific so resources can be shared Set realistic timescales
	Design and ongoing evaluation/modification Understand current data collection status of existing registries Offer a range of modes of completion for questionnaires, including paper and electronic. Align the data collection to objectives set by stakeholders Ensure design has capacity for flexibility as the project develops Set criteria for evaluation of project in advance Aim to minimize the burden to patients Consider dividing up the project into a number of work packages, each with its own lead
Maintenance	Technological and information governance issues Ensure lack of familiarity with technology does not limit participation Ensure technology has the capacity for flexibility over time Consider issues of data security and information governance Establish and address the legal and ethical constraints of the country/state Develop a coordinated IT infrastructure Consider availability of technology across different participating countries Maintain database so that patient information is up to date Obtain legal advice on the data sharing agreements that may be necessary Get the technology working before rolling it out-slow technology can be a barrier
	Useful high-impact output Maintain interest by maximizing published output in a range of formats Target different stakeholder group with outcomes of PROMs work Present patient-level data in a readily understandable format Highlight the direct benefit to patients from participation Use ongoing nature of data to ensure frequent analysis and dissemination Data should aim to help improve patient care Make data as freely available as possible within the constraints of confidentiality
	Maintaining trust/faith in the data Ensure methodological rigour to maintain trust Be aware of the sensitivity of centres to publication of data that reflects poorly on their performance Ensure objectives and evaluation are transparent and set by stakeholders and not any group for example with a vested interest Be aware that PROMs viewed more positively if presented as a care management tool Report characteristics of responders and non-responders

A number of these themes, such as the need for stakeholder involvement, PROMs expertise and international standardization, were used when planning the consensus meeting. The results of the review were also presented to the delegates during the preparatory session of the consensus meeting.

CONSENSUS MEETING

In June 2014, experts from across Europe participated in a consensus meeting in Bristol, UK. Invited to the meeting were participants representing all European renal registries with an interest in collecting PROMs and PREMs routinely; experts on PROMs, PREMs and shared decision making; a representative from the National Cancer Registry Ireland with experience in routine PROMs collection among cancer patients, and; patient and carer representatives from a large UK kidney patient charity the National Kidney Federation. The objectives of this meeting were to (i) share any experience renal registries in Europe already have in this area; (ii) establish how patient-reported data might be collected by understanding how registries currently collect routine data and how patient-reported data is collected in other settings; (iii) harmonize the future collection of patient-reported data by renal registries in Europe by agreeing upon preferred instruments and (iv) identify the barriers to routine collection of patient-reported data in renal registries in Europe.

The opening sessions of the meeting comprised the presentation of the results of the narrative synthesis followed by a series of presentations aimed at addressing the first two objectives. Representatives from four of the five renal registries with PROMs/PREMs experience (France [24], Norway, Romania and Sweden) and the National Cancer Registry Ireland shared their experience of PROMs/PREMs data collection. The director of the ERA-EDTA Registry then described the current state of routine data collection in renal registries in Europe; this was followed by presentations from three experts in PROMs, PREMs and shared decision making. Summaries of the three methodological presentations follow.

EXPERT REPORTS

Patient-reported outcome measures—Elizabeth Gibbons, patient-reported outcome measurement group, University of Oxford, UK

PROMs capture patients' perceptions of their health and quality of life, and vary according to factors such as method of completion (e.g. paper-based or electronic) and content (ranging from general items to more disease-specific symptoms). PROMs are useful for research purposes, clinical monitoring and more recent applications include service improvement and national benchmarking (e.g. the National PROMs programme in England for elective procedures). In addition to this established programme, several PROMs pilots are in progress in the UK, looking at long-term conditions (LTCs) in primary care, depression in secondary care, cardiac revascularization and skin cancer.

Several challenges have been identified through such pilot studies including poor response rates and concerns about benchmarking and performance management. However benefits include the value of individual patient monitoring, plus the potential for service improvement.

Selection of a PROM should be informed by a systematic review of the literature reporting psychometric properties and consideration of the practicalities of data collection. In 2009 the PROM group, University of Oxford, was commissioned by the Department of Health in England to review PROMs for chronic kidney disease and established renal failure [25]. Recommendations based on the strength of evidence included the SF-36 [26], EQ-5D-5L [27] and KDQOL™-36 [28] measures.

When selecting a PROM, consideration needs to be given to the purpose of measurement and practicalities of data collection. Complexity of scoring may outweigh the benefits of precision whereas short versions of instruments with narrow focus and simple indices may not provide breadth of information. Data linkage and clinical information systems can support collection and feedback of the data but may be complex to develop and maintain.

Patient-reported experience measures—Dr Sabine van der Veer, Amsterdam Medical Center, the Netherlands

Patient experience has long since been acknowledged as an important dimension of quality of care [29]. Whereas patient satisfaction is the perceived discrepancy between the expected and experienced quality of care, the construct of patient experience attempts to exclude the former element from the equation [30]. However, when translating these two constructs into actual measurement instruments, the distinction between them often becomes less clear.

During the consensus meeting, eight instruments were discussed that were either available in English or on which an English publication was available in PubMed [31–38]. Most included items on a broad spectrum of care delivery aspects [31, 32, 35, 36, 38]; others focussed on capturing experiences with specific aspects like education, or with treatment in general [32, 33, 36]. The number of items varied widely between instruments, ranging from eleven in the Renal Treatment Satisfaction Questionnaire [33] to over 60 in the Scottish Renal Patient Experience Survey [36]. Only two instruments were applicable to any type of RRT [33, 37], while others were designed for one [32, 34] or more dialysis modalities [31, 35, 36, 38].

Almost all instruments were developed with input from patients: for example, focus groups or interviews which identified relevant aspects of care [31–33, 35, 38] or cognitive testing of a preliminary version of the instrument [32, 33, 38]. In most cases, developers evaluated the instruments' internal consistency [33–35, 38, 39]. Some also assessed construct validity by exploring the correlation between patient experience as measured by the instrument and global assessments of satisfaction [34, 38], clinical performance [31] or health-related quality of life [34]. Outside a development context, wide scale use has only been reported for the CHOICE and the CAHPS questionnaire [40, 41].

Shared decision-making—Dr Hilary Bekker, University of Leeds, UK

Measures used to assess shared decision making are predominantly self-report questionnaires designed to evaluate a decision support intervention's effectiveness [12, 42–45] and/or screen for decisional outcomes within usual care [46–51]. The substantial number of measures available reflects the complexity of these interventions in terms of their impact on different people within the process of delivering care and components needed to enhance people's active reasoning and engagement with others [52] (Figure 2).

FIGURE 2:

A framework representing informed, evidence-based and shared decisions with people and their healthcare roles.

Decision support intervention types include:

• Patient Decision Aids [12, 45] enabling people to make informed decisions (point 1, Figure 2) between options by consideration of accurate information about all options and their consequences without bias, evaluation of this information with their values and making a decision on trade-offs between evaluations [12, 53, 54]. There are several patient-reported informed decision outcome measures [37, 38, 46–50, 55–59]. Alternatively, proxy outcomes may be used to assess an aid's impact by capturing people's knowledge, risk perception, values, involvement, activation, usefulness intervention, value-choice consistency and/or decision quality [12, 45, 60–62].

• Professional Decision Support [63] enabling professionals to make evidence-based choices (point 2, Figure 2) by using the best evidence available, in consultation with the patient, to decide upon the option which best suits that patient [64].

• Shared Decision-Making Support [65–68] within patient-professional consultations enabling the process of choosing healthcare collaboratively (point 3, Figure 2) by exchanging information, preferences and values about treatments, explicit reasoning about choices and agreeing a choice and implementation plan. Some measures assess patient-reported shared decision making outcomes [48, 69, 70]. Proxy outcomes assess an aid's impact on the professional (e.g. provided option information, elicited values, awareness of patient experience, etc.), [71–73] the patient (e.g. asked questions, provided values, awareness of professional viewpoint, etc.) [61, 62, 74–76] and/or the concordance between patient-professional factors (e.g. SDM-Q-9; decisional conflict) [77–79].

When informed [56, 57, 59] and shared [60] decision outcomes are used in renal services, findings suggest that they are useful service-quality indicators [49, 50, 77, 80]. However, they may respond differently from application in other contexts, as they are not designed for decisions taking place across multiple consultations, health professionals and services, with delayed implementation and chronically ill, elderly and/or often frail patients [80].

REACHING A CONSENSUS

The second part of the meeting addressed the objective of harmonizing the routine collection of patient-reported data by renal registries in Europe by agreeing on preferred instruments and discussing some of the practicalities of this process. Attendees were divided into four groups and asked to discuss which of the PROM/PREM instruments on a list should be employed, how often they should be administered and in what format. Due to time constraints and as they had not been included in the original application for funding it was decided to exclude the measurement of Shared Decision Making from the consensus discussions. Groups were chaired by four of the attendees, who later reported back to the whole meeting. Discussions were also audio-recorded. Participants were also asked to vote on which PROMs/PREMs they thought would be most appropriate based on the evidence outlined throughout the course of the meeting.

1. PROM instruments. The overwhelming consensus was that any PROMs measurement programme should aim to support improvements in the quality of care for patients. In terms of the measures adopted, there was agreement that the programme should aim to include both generic and disease-specific measures if possible, whilst minimizing the overall length of questionnaires administered. Of all the instruments discussed, the KDQOL™-36 seemed to be preferred by delegates as it offers both generic and disease-specific outcomes. Of the generic instruments, the SF-12 [81] was the most preferred. The importance of capturing patient symptoms was recognized, especially if a generic health-related quality of life instrument was being used, but no preferred symptom burden instrument was agreed on. There was not complete agreement on whether a preference-based measure should be included, but consensus suggested that it would be useful to have a measure that would allow health economic evaluations provided that the length was not prohibitive. The EQ-5D-5L was the preferred instrument for this purpose but it was recognized that SF-6D [82], which could also provide the utility data necessary for calculating quality adjusted life years in health economic evaluations, could be derived from longer SF instruments (Table 2) [82, 83].

2. PREM instruments. Delegates were much less familiar with the various PREM instruments available. Therefore, although the strengths and weaknesses of the various instruments had been presented to them earlier in the meeting, there was a broad consensus that more work was needed to recommend specific PREMs for broader use.

3. Patient groups to be covered. There was a clear consensus that the aim should be to include all patients on RRT in any PROMs/PREMs programme, and that data should be collected on at least an annual basis. The possibility of extending coverage to pre-dialysis patients in the future, when these patients are captured by registries, was discussed. The importance of making patients aware of how the data were being used was emphasized if high response rates were to be achieved from a broad range of patients. Several issues relating to the timing of data collection were discussed, and the general view was that although it may be useful to collect data at specific time points (e.g. in relation to commencement of RRT), this may not be feasible. As regards timings, the patient representative suggested that patients may not wish to complete PROMs/PREMs questionnaires whilst attending for dialysis and PROM experts raised concerns that responses given while on dialysis may be sensitive to the current dialysis experience rather than ‘usual’ health-related quality of life. No clear preference for paper or web-based reporting emerged, but the group felt that data should be collected via unassisted self-reporting. Other issues that featured prominently in discussions were the need to consider the legal, data protection and consent constraints of participating countries, and the possibility of a pilot study to assess feasibility.

Table 2.

Summary of recommendations from consensus discussions for routine renal registry PROM collection

	Consensus
Which instruments
Generic	SF-12
Preference-based	EQ-5D-5L
Kidney specific	KDQOL™-36
Practical issues
Who?	All patients on RRT
When?	At least annually
	Preferably not during dialysis
How?	Unassisted self-report
	No clear preference for paper/web
Other issues	Consider ethics/consent/data protection
	Conduct initial pilot study

There are limitations to this report. There is a scarcity of robust data on measuring and reporting PROMs and PREMs in kidney patients. Further, as mentioned above, the instruments available were generally developed for measurement at the individual patient level, rather than the centre or national level. In some ways these limitations reflect the attempt to reach consensus across European renal registries before instruments and processes become established.

SUMMARY

From the survey of European renal registry representatives and discussions with those attending the meeting there seems to be a widespread acceptance of the need to extend renal registry data collection to capture patient-reported outcomes and experience. It is recognized that this represents a considerable challenge for renal registries and services if all the potential benefits from the information gathered—such as focussing consultations on what matters to patients, having real-life information on quality of life for decision-making, incorporating patient measures into the quality assurance of renal units, research and improving services—are to be realized. Establishing a dialogue between registries around PROMs/PREMs and agreeing on some preferred measures at this stage will hopefully enable the sharing of expertize and experience and stimulate the collection, and facilitate the collection and comparison of patients' outcomes and experiences across Europe in the future.

SUPPLEMENTARY DATA

Supplementary data are available online at http://ndt.oxfordjournals.org.

CONFLICT OF INTEREST STATEMENT

None declared.