Top research priorities in liver and gallbladder disorders in the UK

Abstract

Objectives

There is a mismatch between research questions considered important by patients, carers and healthcare professionals and the research performed in many fields of medicine. The non-alcohol-related liver and gallbladder disorders priority setting partnership was established to identify the top research priorities in the prevention, diagnostic and treatment of gallbladder disorders and liver disorders not covered by the James-Lind Alliance (JLA) alcohol-related liver disease priority setting partnership.

Design

The methods broadly followed the principles of the JLA guidebook. The one major deviation from the JLA methodology was the final step of identifying priorities: instead of prioritisation by group discussions at a consensus workshop involving stakeholders, the prioritisation was achieved by a modified Delphi consensus process.

Results

A total of 428 unique valid diagnostic or treatment research questions were identified. A literature review established that none of these questions were considered ‘answered’ that is, high-quality systematic reviews suggest that further research is not required on the topic. The Delphi panel achieved consensus (at least 80% Delphi panel members agreed) that a research question was a top research priority for six questions. Four additional research questions with highest proportion of Delphi panel members ranking the question as highly important were added to constitute the top 10 research priorities.

Conclusions

A priority setting process involving patients, carers and healthcare professionals has been used to identify the top 10priority areas for research related to liver and gallbladder disorders. Basic, translational, clinical and public health research are required to address these uncertainties.

Strengths and limitations of this study.

• A research prioritisation process involving clinicians, patients and carer, and public representatives was performed in the field of liver and gallbladder disorders. This will help to address the mismatch between research questions that are considered important jointly by patients, carers and healthcare professionals and the research performed in the field of liver and gallbladder disorders.

• A Delphi consensus method was performed. This prevented dominance of ‘loud voices’, a problematic issue with small and large group discussions.

• Because of the predominance of people with chronic liver disease on the Delphi panel, many of the top research priorities related to chronic liver diseases.

Introduction

Failure to address treatment uncertainties by research can lead to significant suffering and deaths.¹ It is important that research in any field of medicine takes into account the shared interests of patients, carers and clinicians.² However, there is a mismatch between research questions that are considered important jointly by patients, carers and healthcare professionals and the research performed in many fields of medicine.^{3 4} The James-Lind Alliance (JLA) exists to help ensure a patient-centred process and enables the limited research resources to be used in addressing the research questions that are considered important jointly by patients, carers and healthcare professionals² (‘top research priorities’). This is achieved by forming ‘priority setting partnerships’ (PSPs) between patients, carers and healthcare professionals.² Formal prioritisation of research topics jointly by patients and healthcare professionals can lead to increased research on the topic.^{5 6}

There has only been one formal research prioritisation process involving patients, carers and healthcare professionals in the field of liver and gallbladder disorders.⁷ However, the scope of that PSP was limited to alcohol-related liver disorders.⁷ The Non-Alcohol-Related Liver and gallbladder disorders Priority setting partnership (NARLIP) was established to address the prevention, diagnostic and treatment uncertainties related to the majority of liver disorders which were not covered by the JLA PSP on alcohol-related liver diseases⁷ and to include gallbladder disease.

The aims and objectives of the NARLIP were to work with patients, their carers and healthcare professionals treating them (‘stakeholders’) to identify uncertainties about the diagnostic tests and effects of prevention and treatments for non-alcohol-related liver and gallbladder disorders, to agree by consensus a prioritised list of those uncertainties for research, to publicise the results and process, and to take the results to research commissioning bodies to be considered for funding and researchers to encourage them to submit grant applications addressing these uncertainties.

Methods

The methods broadly followed the principles of the JLA guidebook.⁸ The broad steps involved the following and are summarised in figure 1.

Figure 1.

The major steps in the research prioritisation are shown in the figure. ^aThe protocol was registered with James-Lind Alliance (JLA) priority setting partnership (PSP). ^bThe final prioritisation was achieved by modified Delphi consensus method.

1. Formation of the partnership: organisations and individuals representing people affected by non-alcohol-related liver or gallbladder disorders, their carers and healthcare professionals treating people with non-alcohol-related liver and gallbladder disorders. A partnership was formed between KG representing University College London and the British Liver Trust initially, but following reorganisation in the British Liver Trust, PSC Support⁹ became the leading patient organisation partner of this process. A steering committee was formed. The members of the steering committee who participated in the complete process were KG, MW, BRD, CF, BF, AM, RM, SM, IS and ET.

2. Establishment of the scope: the steering committee members discussed and decided that the scope should include adult and paediatric liver and gallbladder disorders which required medical and surgical treatments. The protocol was registered with JLA PSP.

3. Identifying potential research questions: research questions were collected through online surveys and searching UK Database of Uncertainties about the Effects of Treatments (UK DUETs), research recommendations in high-quality systematic reviews and clinical guidelines, and registers of ongoing research.

4. Refining research questions: the research questions identified in the above step were reviewed and where necessary combined to result in a set of unique research questions. Research questions were considered ‘answered’ when recent high-quality systematic reviews (based on low risk of bias studies) concluded that further research was not required. Removal of such ‘answered’ research questions was planned. The remaining questions were ‘uncertainties’.

5. Interim prioritisation: to shortlist the set of questions to manageable levels for the final prioritisation process, the members of the steering committee ranked the uncertainties after stratifying the questions as medical and surgical questions. The members of the steering committee agreed that the interim prioritisation list should consist of 75% medical questions and 25% surgical questions. This decision was an arbitrary decision made by the steering committee based on the rationale that majority of individuals with liver and gallbladder disorders are treated medically but a minority require surgery which have a major impact on patients’ lives.

6. Final prioritisation by consensus: a modified Delphi consensus method was followed to identify the top priorities using methods described by Jones and Hunter.¹⁰ The Delphi was performed electronically using Excel for managing the process. The steps in the modified Delphi consensus method were as follows.

   1. A Delphi panel consisting of patients, their carers and healthcare professionals treating them was formed. The Delphi panel was formed by using ‘snowballing’ sampling methods and by contacting people through emails, online liver patient forums (British Liver Trust Health Unlocked forum) and newsletters. A total of 42 people expressed interest in joining the Delphi panel and 33 panel members completed all three rounds. Details of the Delphi panel composition and drop-outs are reported in the results section.
   2. A total of three rounds were conducted.
   3. Delphi panel members scored the shortlisted questions in the interim prioritisation process on a scale of 1–9 with 1 being considered least important and 9 being considered most important. Scores of 1–3 were categorised as ‘less important’, 4–6 as ‘moderately important’ and 7–9 as ‘highly important’. Panel members were requested to score the questions according to the importance of the question to them/the persons that they represent or treat and could leave questions that they were unable to score empty. Each Delphi panel member could add a maximum of two questions in the first round to ensure that the questions most important to the Delphi panel members were included in the prioritisation process even if they were not identified in the earlier steps. In the subsequent rounds, the panel members were shown the summary scores and their previous score for each question. They were able to retain or change their score in each of the rounds after the first round. For calculation of the summary scores and the proportion considering a question ‘highly important’, non-responses were excluded.
   4. Consensus about a specific research question being a top research priority was reached when 80% or more Delphi panel members considered the research question as highly important (allocated scores between 7 and 9).
   5. When fewer than 10 research priorities were obtained by consensus, the remaining priorities were completed by uncertainties based on the highest proportions of panel members agreeing that the research question was highly important (scores between 7 and 9).
   6. There was no restriction on the Delphi panel to consult others while scoring the questions. However, only one final response on the set of questions was accepted from each Delphi panel member.

When there were no recent high-quality systematic reviews on the research question, we have recommended high-quality systematic reviews. When recent high-quality systematic reviews recommended high-quality research, we have recommended randomised controlled trials (RCTs) for prevention and treatment, as such studies carry the lowest risk of bias if conducted well; we would have recommended well conducted diagnostic test accuracy studies for diagnostic uncertainties. All online surveys were completed using Google Forms designed by KG. The Delphi process was completed using Microsoft Excel and email.

In addition, we had full support of patient organisations with involvement of patient representatives throughout the whole process rather than patients visiting the hospitals.

Patient and public involvement

Patients and public were involved in all aspects of this project: they were part of the steering committee and were involved in the definition of the scope, methodology used for the prioritisation process, identification of further patients and public representatives, participation in the Delphi panel, interpretation and critical revisions of the draft report. They will be involved in the dissemination of the findings through patient websites, patient forums and to research funders.

Results

Identifying and refining of research uncertainties

A total of 126 patients, carers, and those at risk of developing non-alcohol-related liver and gallbladder disorders, and 13 healthcare professionals participated in the first survey which was conducted between July and December 2015. This survey resulted in a total of 209 unique research questions. In addition, 219 unique questions were identified from searching the UK DUETs, Pubmed and ClinicalTrials.gov on 2 January 2016. A total of 428 unique valid (ie, falling within the remit of this priority setting partnership) research questions (247 medical-related and 181 surgery-related) were identified from these sources. None of the research questions had been answered by recent high-quality systematic reviews based on low risk of bias studies which concluded that further research was not required. Therefore, all the 428 research questions were considered research ‘uncertainties’. The complete list of 428 unique valid uncertainties in no particular order is available in online supplementary appendix 1. This has been converted to the population, intervention, control and outcomes format whenever possible.

Interim priorities

To identify a shortlist of questions (from the list of 429 questions) that were to be considered for the next step, a total of 48 research priorities (36 medical questions and 12 surgical questions) were identified on the basis of being selected by at least one patient or carer and healthcare professional of the steering committee (24 questions) and obtaining the highest ranks among the members of the steering committee (additional 24 questions). The list of 48 questions identified as interim priorities in no particular order is available in online supplementary appendix 2.

Final priorities

A total of 42 people expressed interest in joining the Delphi panel and 33 panel members completed all three rounds. Five people dropped out before they returned the scores of the first round (all patients, carers and general public), three between first and second rounds (all healthcare professionals) and one between the second and third rounds (healthcare professional). Of the 33 panel members who completed all the three rounds, 17 were healthcare professionals and 16 were patients, carers and general public. Of the 17 healthcare professionals, six were hepatologists, four were surgeons, two were hepatology nurses and the remaining were general practitioner, hepato-pancreato biliary surgery nurse, organ preservation biologist, dietician and pharmacist (one each). Of the 16 patients, carers and general public, there was representation from general public and various liver diseases including autoimmune diseases such as primary sclerosing cholangitis (PSC), primary biliary cholangitis, autoimmune hepatitis, viral hepatitis, metabolic diseases such as non-alcohol-related fatty liver disease (NAFLD), and other diseases such as hepatocellular carcinoma and polycystic liver disease. There was also representation of liver transplanted patients in the Delphi panel. In total, 23 panel members were from England, seven were from Scotland and three were from Wales. There were no panel members from Northern Ireland despite attempts to include panel members from Northern Ireland.

A total of 22 additional questions were added by the Delphi panel members in the first round of the Delphi process. The Delphi panel achieved consensus that an uncertainty was a top research priority for six research questions. Four additional research questions with the highest proportion of Delphi panel members scoring the question as highly important (scores between 7 and 9) were added to constitute the top 10 research priorities. The list of the top 10 research priorities (in the order of proportion who agreed that the uncertainty is a very important research priority) is available is table 1. All the top 10 research priorities were prevention and treatment uncertainties, and none were diagnostic test uncertainties. None of the panel members thought the first two questions as least important (scores 1–3). For the remaining eight questions, 3%–6.5% of people considered the questions to be least important (scores 1–3).

Table 1.

Treatment uncertainties for which consensus that the uncertainty is a research priority was reached

Treatment uncertainty (research question)	Proportion who rated this question as highly important in the final round	Median (IQR) in the final round
What is the best treatment for people with early or very early hepatocellular carcinoma?	93.5%	8 (7,9)
What are the best treatments that cure or delay the progression (worsening) of primary sclerosing cholangitis (PSC)?	93.3%	8 (7,9)
What are the best treatments that cure or delay the progression (worsening) of non-alcohol-related fatty liver disease (NAFLD)?	90.3%	9 (8,9)
What is the best immunosuppressive regimen in adults undergoing liver transplantation?	90.3%	8 (7,9)
Should general public be educated about NAFLD with an aim to reduce the numbers of those that have it?	81.8%	8 (7,9)
What are the best treatments that cure or delay the progression (worsening) of autoimmune hepatitis?	80.6%	8 (7,9)
What are the best treatments that cure or delay the progression (worsening) of non-alcohol-related steatohepatitis?	76.7%	8 (6.75,9)
Prior to liver transplantation, is it better to transport the donor liver using a machine which pumps blood or preservation solution through the liver (machine perfusion) or is it better to transport it in the standard way of transporting it immersed in cold preservation solution (cold storage)?	74.2%	7 (6,9)
What are the best treatments that cure or delay the progression (worsening) of primary biliary cholangitis?	74.2%	7 (6,8)
Are there any treatments that reverse the liver damage in PSC?	72.4%	7 (6,9)

A summary of the availability of systematic review of RCTs on the topic of the individual questions, RCTs on the topic not included in the systematic review (if one exists), and the outcomes evaluated in these RCTs are listed in table 2. Table 2 also contains a suggestion for the next research steps. The list of the existing trials was compiled by searching ClinicalTrials.gov on 7 April 2018. The references to the trials not included in the systematic reviews is available in online supplementary appendix 3. As seen in table 2, a well-designed RCT is the next step for eight of these top 10 research questions. This is because it appears that the outcomes in those trials will not address the outcomes listed in the research questions.

Table 2.

Next step to address the top 10 research priorities based on current best evidence (summary)

Treatment uncertainty (research question)	High-quality systematic review*†	RCTs not included in the systematic review*‡	Patient-oriented outcomes assessed in trials not included in the systematic review§	Next step
What is the best treatment for people with early or very early hepatocellular carcinoma (HCC)?	13	8 trials	Survival (7 trials), recurrence (5 trials), morbidity (3 trials)	High-quality RCTs of interventions not covered in ongoing trials and comparison of health-related quality (HRQoL) in different treatments
What are the best treatments that cure or delay the progression (worsening) of primary sclerosing cholangitis (PSC)?	14	9 trials	None of the trials include survival, HRQoL as outcomes¶	High-quality RCTs with clinical outcomes (survival, HRQoL)
What are the best treatments that cure or delay the progression (worsening) of non-alcohol-related fatty liver disease (NAFLD)?	15 (Includes only pharmacological interventions)	More than 10 published trials on lifestyle interventions and more than 20 trials on nutritional supplementation with no recent high-quality systematic reviews Pharmacological interventions 44 trials	Lifestyle interventions and nutritional supplementation Not applicable as there are no high-quality systematic reviews Pharmacological interventions Health-related quality of life (2 trials), resolution of fatty liver disease (11 trials), mortality (2 trials), cirrhosis (2 trials), cardiovascular events (2 trials)¶	High-quality systematic reviews on lifestyle interventions (one review) and nutritional supplementation to cure or delay the progression of NAFLD and high-quality RCTs on pharmacological interventions with clinical outcomes (survival, HRQoL)
What is the best immunosuppressive regimen in adults undergoing liver transplantation?	16 (Covers only maintenance immunosuppression)	Induction immunosuppression: more than 20 published trials Maintenance immunosuppression 4 trials	Induction immunosuppression Not applicable as there is no high-quality systematic review Maintenance immunosuppression Graft survival (1 trial) Adverse events (1 trial) Hepatocellular carcinoma (1 trial)¶	High-quality systematic review on induction immunosuppressive regimen and high-quality RCTs on maintenance immunosuppression with important clinical outcomes (overall survival, HRQoL)
Should general public be educated about NAFLD with an aim to reduce the numbers of those that have it?	None	None	-	High-quality RCTs on education to prevent NAFLD
What are the best treatments that cure or delay the progression (worsening) of autoimmune hepatitis?	None	15 trials	Survival (1 trial), health-related quality of life (1 trial)¶	High-quality RCTs with clinical outcomes (survival, HRQoL)
What are the best treatments that cure or delay the progression (worsening) of non-alcohol-related steatohepatitis (NASH)?	The evidence related to this question is covered under NAFLD by performing a subgroup analysis of people with NASH
Prior to liver transplantation, is it better to transport the donor liver using a machine which pumps blood or preservation solution through the liver (machine perfusion) or is it better to transport it in the standard way of transporting it immersed in cold preservation solution (cold storage)?	None	5 trials	Overall survival (4 trials), graft survival (5 trials), health-related quality of life (2 trials)	Await results of the RCTs (all expected to complete by the end of 2019) and perform a high-quality systematic review
What are the best treatments that cure or delay the progression (worsening) of primary biliary cholangitis?	17	24 trials	Health-related quality of life (5 trials), relief of symptoms (5 trials)¶	High-quality RCTs with clinical outcomes (survival, HRQoL)
Are there any treatments that reverse the liver damage in PSC?	The evidence related to this question is covered under treatments for PSC. The systematic review did not include fibrosis as one of the outcomes. Nine of the trials included in the systematic review reported on fibrosis. Two of the trials not included in the systematic review (and listed above) reported on liver fibrosis

*Numbers indicate the reference number.

†Further well-designed randomised controlled trials (RCTs) using clinical outcomes were recommended by all these systematic reviews.

‡Ongoing trials, unpublished trials or trials published since the search date for the systematic review when a high-quality systematic review based on RCTs exists. If no systematic reviews based on RCTs exist, these are either published trials or ongoing studies.

§This information is reported to find out whether the important patient-oriented outcomes are reported in the trials not covered by high-quality systematic reviews. This is to help with deciding whether new RCTs are necessary on the topic.

¶The remaining trials reported treatment-related adverse events, composite outcomes and surrogate markers.

The complete list of questions in the Delphi process, the proportion of respondents who considered a research question as very important and the summary scores in each Delphi round is available in online supplementary appendix 4. This appendix also has the breakdown of the proportion of patients, carers and general public who considered a research question as very important and their summary scores in each Delphi round along with similar summary measures for healthcare professionals.

Discussion

This is the first priority setting partnership on non-alcohol-related liver and gallbladder disorders. This included a wide range of disease processes and a total of 428 unique research questions that met the scope of this priority setting partnership were identified. All the research questions were considered unanswered as there had been no high-quality systematic reviews which indicated that no further research is required, that is, all the research questions were uncertainties. Consensus that an uncertainty was a very important research priority was reached for six research questions. Four additional research questions with the highest proportion of Delphi panel members ranking the question as highly important were added to constitute the top 10 research priorities.

As evident from the online supplementary appendix 1, longevity of life and health-related quality of life are two major outcomes that appear important to patients, their carers and healthcare professionals. However, even when there are ongoing trials, it appears that the outcomes in those trials will not address the outcomes listed in eight of the top 10 research questions (table 2). Therefore, the next step in addressing these uncertainties is the design and conduct of RCTs. Such RCTs may involve qualitative studies to determine the design and should compare the treatments that improve the longevity of life and/or health-related quality of life.

It should be noted that uncertainties ‘what are the best treatments that cure or delay the progression (worsening) of NAFLD?’ and ‘what are the best treatments that cure or delay the progression (worsening) of non-alcohol-related steatohepatitis (NASH)?’ are related to each other. Although NAFLD includes NASH, most of the panel members felt that the research questions related to NAFLD and NASH should be kept separate uncertainties. While the same systematic review can cover both the uncertainties, the primary research study designed to address these two questions differ in terms of the setting, the outcomes used and the period of follow-up. Any primary research that tries to answer these two questions in a single RCT will be inefficient.

Similarly, for the uncertainties ‘what are the best treatments that cure or delay the progression (worsening) of PSC’ and ‘are there any treatments that reverse the liver damage in PSC?’, a single RCT will be inefficient and the preference of most of the panel members was to keep these uncertainties as separate uncertainties.

There are several limitations to our priority setting process. The first one is deviation from the original protocol. To select the final top priorities, the initial plan was to arrive at consensus by open small group and large group discussions of patients, carers, and healthcare professionals as suggested by the standard JLA process,⁸ which provides an opportunity for a knowledge exchange of viewpoints and experience. However, part of the steering committee with experience in a similar priority setting partnership felt that open discussions resulted in ‘loud voices’ being given more importance resulting in an unrepresentative list of top priorities. While this can be mitigated by facilitated group discussions by neutral JLA facilitators to ensure that all voices were heard in the discussions, this was considered by the steering committee as an important source of bias based on their prior experience in participating in open discussions. The steering committee therefore decided to follow the Delphi consensus method which is one of the major consensus methods.¹⁰ The advantages of Delphi consensus method over open discussions include anonymity of the response and the equal weight given to the opinions of all members.¹⁰ In addition, they are less costly to conduct without any limitation by geographical location compared with other methods of consensus¹⁰ because of the lack of necessity to travel and take time off regular work. However, there is considerable variability in the previous performance of Delphi processes with regard to the number of rounds and the criteria for achieving consensus.¹¹ Arriving at consensus depends on people revising their scores based on the other’s scores. Our initial plan was to extend the Delphi to four rounds if consensus on 10 top research priorities was not reached in three rounds. However, there was minimal change in scores between the rounds for most questions (online supplementary appendix 3) and the Delphi process was completed in three rounds. Consensus on a top research priority was achieved for six questions only. However, the proportion of Delphi panel members ranking a question as highly important was greater than 70% for the remaining four questions added to the list of top research priorities. Previous Delphi consensus processes have used various cut-off points for defining consensus: greater than 70% agreement among panel members is well within the definition of consensus used in previous Delphi consensus processes.¹¹

The other major limitation of our priority setting process is the representativeness of the people who completed the survey and took part in the Delphi process. The online survey was shared among clinicians and members of general and disease-specific patient organisations. Most questions resulting from the online survey relate to chronic liver disease (in particular, autoimmune liver diseases), perhaps reflecting the high motivation to support research from those groups. The Delphi panel also had a high representation of people related to chronic liver disease (in particular, autoimmune liver diseases) as patients, carers or healthcare professionals. While people affected by different liver and gallbladder disorders were actively sought through both general and disease-specific patient support groups and organisations, only a few responded and completed all three rounds of the Delphi process. The potential bias towards prioritising chronic liver diseases is evident as nine of the top 10 research priorities relate to chronic liver diseases (four relate to autoimmune liver diseases, three related to NAFLD and two related to liver transplantation). It was surprising that the uncertainties related to the treatment of chronic viral diseases such as chronic hepatitis B and chronic hepatitis C were not identified within the top 10 research priorities. This may be because of the perception by some of the panel members that the research questions related to the treatment of chronic hepatitis C were answered with the advent of directly acting antivirals (personal communication). The reason for non-prioritisation of chronic hepatitis B is not entirely clear. This may be because chronic hepatitis B may not have been considered as important as other chronic liver diseases or under-representation of chronic hepatitis B in the panel.

Cancer-related questions, childhood-related liver diseases and other benign disorders did not end up in the top research priorities (except for the treatment of very early hepatocellular carcinoma, which is managed by hepatologists and surgeons) probably for the reasons described above. We recommend that separate prioritisation processes are carried out for people with gallstones, a condition that affects approximately 5%–25% of the population,¹² for people with primary and secondary liver cancer, and childhood liver disorders where significant uncertainties remain on the effectiveness of different treatments in decreasing mortality and improving health-related quality of life.

As well as the above limitation, we are aware of the inherent limitations of using solely technology to carry out the Delphi exercise. These are limitations that can potentially lead to bias in any consensus-building method including that of face-to-face consensus methods normally used in a JLA PSP.

One solution which might address the limitations of this priority setting process and the standard JLA process may be to collect information routinely from patients visiting hospitals using paper forms and conduct online meetings (video conferencing and presentation) before the final round of the Delphi (or the standard face-to-face priority setting workshop used by the JLA. Some JLA PSPs do use methods such as face-to-face interviews and group discussions rather than solely online surveys). By collecting information on paper forms and conducting the meetings in hospitals, it is possible to engage with people who do not have access to or are not familiar with computers. It is also possible to engage with people who have concerns regarding data confidentiality with the use of computers or social media by collecting information using paper forms. Ethical and confidentiality issues will need to be considered prior to engaging patients attending hospital in the research prioritisation process.

Another limitation of our priority setting process is the drop-outs during the Delphi process. While some of the drop-outs may be related to the ability to complete online surveys and use Microsoft Excel, some patient representatives or clinicians may have dropped out because they did not find any research question to be of direct relevance to them. Other reasons include lack of understanding of the conditions, feeling that the process was too complicated, feeling that the process would not work, and the time commitment for the process. This is because of the broad scope of this research prioritisation process and may be overcome by choosing a narrower focus while defining the scope of the prioritisation process, and by better explanation of the disease processes through presentations.

It should also be recognised that the Delphi panel was constituted of representatives from England, Scotland and Wales. Therefore, the findings are applicable in only these countries. However, the findings are likely to be applicable throughout the NHS and in other European and Western countries with a similar spectrum of chronic liver diseases and similar treatment options available.

In summary, there are significant uncertainties in the management of liver and gallbladder disorders. Further high-quality research is necessary to address these uncertainties which may require programmes of basic, translational, clinical and public health research. For issues with diverse and unproven treatment options, RCTs may be the only mechanism for identifying the most effective treatment and the treatments that represent good value for money for the NHS. Such RCTs should assess the effect of different treatments in improving longevity of life and/or health-related quality of life.