Making a decision to wait for more evidence: when the National Institute for Health and Clinical Excellence recommends a technology only in the context of research

INTRODUCTION

New treatments tested in randomized trials are as likely to be inferior as they are to be superior to standard treatments.¹ The ethical implication of this empirical observation is that unevaluated new treatments should be assessed carefully in well-designed research before decisions are taken about whether or not they should become widely available. Indeed, the same principle applies when there are uncertainties about the effects of ‘standard’ treatments. The dangers of acquiescing when it is uncertain whether treatments do more good than harm has been illustrated dramatically by the results of the CRASH trial:² tens of thousands of iatrogenic deaths have occurred because of decades of failure to address uncertainties about the effects of systemic steroids for patients with acute traumatic brain injury

It is against this background that some clinicians believe that unproven treatments should only be offered within the context of appropriately designed research. In the perinatal field, for example, obstetricians decided that chorionic villus sampling should be used in low-risk pregnancies only within the context of a randomized comparison with amniocentesis.³ Similarly, neonatologists restricted the use of neonatal extracorporeal membrane oxygenation (ECMO) to a randomized comparison with standard respiratory support,^4,5 and they are currently restricting the use of prolonged whole-body cooling in term infants with perinatal asphyxial encephalopathy to a randomized comparison with standard treatment (the TOBY trial: Whole Body Hypothermia for the Treatment of Perinatal Asphyxial Encephalopathy)

Ethical considerations have been the main motivation for clinicians who have dealt with uncertainties about treatments in this way. But ethical considerations have also informed similar decisions made by agencies managing health service funds on behalf of defined populations to whom they are answerable for cost-effective use of resources.⁶ For example, the Centre for Medicare and Medicaid Services (CMS) in the USA decided that, until more was known about its relative merits and demerits, lung reduction surgery should only be supported if given within the context of a randomized comparison with non-surgical treatments.⁷ The Centre is currently formalizing the criteria for applying this principle,⁸ which has also been considered⁹ and implemented on a single occasion so far (personal communication, Professor Wing) by the Pharmaceuticals Benefit Scheme in Australia.¹⁰

The principle of recommending that when there are important uncertainties about a treatment's effects, that treatment should only be used in the context of research, was explicitly recognized by the National Institute for Health and Clinical Excellence (NICE) at its inception in 1999. NICE was set up to develop evidence-based guidance for the NHS and the broader public sector with the explicit objectives of promoting cost-effective use of resources while encouraging innovation, improving quality and reducing variation in practice.¹¹ To do so, NICE and its advisory bodies were given three decision options, set out by the Department of Health shortly before NICE's establishment in 1999.¹² When appraising health interventions (including health technologies, health promotion and disease prevention programmes, screening and diagnostic tests, surgical procedures, etc.), NICE would be able to recommend:

• The routine use of an intervention in the NHS, either for all or specific licensed indications or patient subgroups;

• That the intervention is not used in the NHS because of inadequate evidence of effectiveness, or more frequently cost-effectiveness; or

• The use of the intervention in the NHS only in the context of appropriate research. This would be appropriate in the case of promising interventions not yet supported by sufficiently robust evidence to justify an unqualified recommendation. In those cases, NICE would ‘recommend that further research is carried out to see whether the potential promise of the intervention can be realized, indicate in broad terms the questions this research should address and advise clinicians that, in the meantime, they should only use the new intervention as part of a well-designed programme of research intended to answer these questions’ (paragraphs 33-36 in Faster Access to Modern Treatment: How NICE Appraisal Will Work¹²)

This paper discusses NICE's third option, its characteristics, use and implications for the evidence base, and its future prospects as a viable policy option for NICE and the NHS. This seems to us a good time to promote discussion on the issue, given NICE's recent launch of the single technology appraisal to assess new technologies closer to licensing, with clear implications on the available evidence base, and the reorganization of the medical research infrastructure in the UK following the Cooksey review. To make the most of the ‘Only In Research’ (OIR) option for patients and the Health Service, this discussion needs to start as early as possible

HOW DOES NICE DEAL WITH UNCERTAINTY?

The evidence base on specific interventions or management strategies is rarely complete, so NICE and its advisory bodies are almost always faced with uncertainty when making their decisions. Despite this, NICE has to develop guidance on the clinical and cost-effectiveness of the interventions referred to it by the Department of Health, and it has to do so within specific timelines. Starting by explicitly acknowledging its existence, NICE has adopted a number of approaches to dealing with uncertainty. Throughout the topic selection process, NICE has always made a case against developing guidance in areas where the evidence is scarce;¹³ instead, these topics would be forwarded to NHS Research and Development Programmes. Furthermore, in 2003, NICE's Board approved the establishment of an R&D Programme supported by the NICE R&D Advisory Committee, with the explicit objective of improving the quality of the evidence base underpinning its guidance through promoting appropriate research.¹⁴ Over the years, NICE has encouraged the development of methodological tools to deal with uncertainty and has since adopted a number of them, including probabilistic sensitivity analysis,¹⁵ now explicitly referred to in the NICE Technology Appraisal Methods Manual.¹⁶ Consensus methods and elicited expert opinion are often used to supplement systematic reviews when developing clinical guidelines in areas where there is little good quality evidence,¹⁷ based on NICE's methods manuals.¹⁸ Finally, when important evidence gaps are identified during the guidance development process, NICE issues research recommendations, encouraging further research to help fill those gaps and inform future updates of its guidance. Calling for further research to inform clinical practice is an important NICE function, highlighted in both the NICE Framework document¹⁹ and the WHO Review of Technology Appraisals.²⁰

Nevertheless, there will always be cases where the available evidence is insufficient to support a positive—or negative—recommendation. In these cases, NICE advisory bodies can consider issuing a recommendation for the use of the intervention only in the context of research.

PUTTING IT INTO PRACTICE: NICE RECOMMENDATIONS FOR USE ONLY IN THE CONTEXT OF RESEARCH

Sixteen (approximately 4%) of NICE's technology appraisal recommendations published between 1999 and early 2007 advised the use of a technology only in the context of research (Table 1). For example, in 2000, NICE recommended that laparoscopic resection for colorectal cancer should only be undertaken in the context of a clinical trial.²¹ This decision encouraged recruitment to the then ongoing UK-based Medical Research Council CLASICC trial, which has since provided the necessary evidence to support a NICE recommendation that this intervention be introduced in routine NHS practice.²² NICE's advice on the use of liquid based cytology is another situation in which NICE recommended further research before subsequently issuing a positive recommendation for the adoption of this new technology across the NHS.²³ Five of the technology appraisal OIR recommendations became supportive recommendations when the original guidance was reviewed in light of additional research evidence (Table 1).

Table 1.

Examples of NICE Only In Research recommendations from Technology Appraisals (1999-2007) for which the research has been or is currently being undertaken and has informed reviews of the guidance

Topic	Original OIR guidance	Relevant research and subsequent reviews
Cervical cancer: cervical screening	2000 (TA005): The Committee recommends the undertaking of a series of pilot studies to investigate the feasibility of liquid-based cytology in terms of workload, productivity and detection rates.	The evaluation of the introduction of liquid-based cytology at these pilot sites informed important sections of the systematic review and modelling analysis underpinning the cost-effectiveness analysis of liquid based cytology. As a result, this technology was recommended by NICE in 2003 (TA069).
Colorectal cancer: laparoscopic surgery	2000 (TA017): Laparoscopic surgery should only be undertaken for colorectal cancer as part of a randomized controlled clinical trial.	The NICE guidance of 2000 accelerated recruitment into relevant trials, including the CLASICC trial; the new data generated by these studies were reviewed by the Committee and a positive recommendation for the use of this technology was issued in 2006 (TA105).
Advanced colorectal cancer: irinotecan, oxaliplatin	2002 (TA033): On the balance of clinical and cost-effectiveness, neither irinotecan nor oxaliplatin in combination with 5-fluorouracil and folinic acid (5FU/FA) are recommended for routine first-line therapy for advanced colorectal cancer.	In 2003, the available data did not support widespread use of irinotecan or oxaliplatin. At the same time, the Medical Research Council (MRC) established the FOCUS trial to address the uncertainties surrounding the value of irinotecan as a first-line therapy. NICE guidance actively encouraged oncologists to enrol patients (following discussion and consent) in this trial to ensure that patients would be treated appropriately and data collected to determine the true value of these treatments. The MRC FOCUS trial reported in 2004, and NICE guidance was reviewed in 2005. Both FOCUS and other studies were used to inform the review and support a positive recommendation for irinotecan and oxaliplatin (TA093).
Depression and anxiety: computerized cognitive behavioural therapy (CCBT)	2002 (TA051): The NHS should consider supporting an independent programme of research into Cognitive Behavioural Therapy (CCBT). This research should include looking at how CCBT could fit into a `stepped care' approach to treating anxiety and depression and the setting up of carefully monitored pilot projects. It should also include investigations into what users prefer, what their needs are, and what makes people suitable or unsuitable for treatment with CCBT (e.g. does cultural background or level of education affect whether CCBT is an appropriate treatment).	Given the uncertainty surrounding the various CCBT interventions, in 2002 NICE recommended independent research in order to gather more evidence on how the NHS might use this technology in the future. The Committee considered the evidence base on Beating the Blues and FearFighter, which had improved since the previous appraisal. On the basis of the new evidence, in 2006 the Committee recommended the routine use of the above two interventions, while the use of COPE and Overcoming Depression was recommended only in the context of research (TA097).
Macular degeneration (age related): photodynamic therapy	2003 (TA068): [Photodynamic therapy] is not recommended for the treatment of people with predominantly classic subfoveal choroidal neovascularisation [CNV] associated with wet age-related macular degeneration, except as part of ongoing or new clinical studies that are designed to generate robust and relevant outcome data, including data on optimum treatment regimens, longterm outcomes, quality of life and costs.	In preparation for the 2006 review, NICE carried out a search for information; it believes that it is appropriate for the review to go ahead and coincide with publication of the results of a relevant on-going trial (Verteporfin photodynamic therapy cohort study [UK]). The review is now pending awaiting the results of the above trial.

This type of recommendation is now found not only in appraisals of specific technologies but also in clinical guidelines, which deal with broader disease management strategies, and (since April 2005) in public health guidance, which looks at specific interventions or broader programmes of prevention and health promotion. Indeed, in one of the first pieces of public health guidance published by NICE, three of the four interventions considered were recommended only in the context of research studies²⁴ (Table 2). Almost half of NICE's recommendations in its interventional procedures guidance (which advises on the safety and efficacy of surgical procedures) fall in the category of ‘cautionary guidance’ and are recommended only with additional requirements for consent, audit and, occasionally, for more formal research.²⁵

Table 2.

Examples of Only In Research recommendations in NICE clinical guidelines and public health guidance

Guidance type	Title	Year of issue	NICE recommendation
Clinical Guideline CG013	Caesarean Section	2004	The risk of neonatal morbidity and mortality is higher with `small for gestational age' babies. However, the effect of planned [Caesarean section] in improving these outcomes remains uncertain and therefore CS should not routinely be offered outside a research context.
Clinical Guideline CG014	Familial Breast Cancer	2004	For women aged 30-39 years satisfying referral criteria for secondary or specialist care, mammographic surveillance should be carried out:
			• Only as part of a research study (ethically approved) or nationally approved and audited service; and
			• Individualized strategies should be developed for exceptional cases, such as
			○ Women from families with BRCA1, BRCA2 or TP53 mutations;
			○ Women with equivalent high breast cancer risk.
Clinical Guideline CG015	Type I Diabetes	2004	Metformin in combination with insulin is suitable for use only within research studies because the effectiveness of this combined treatment in improving glycaemic control is uncertain.
Clinical Guideline CG024	Lung Cancer	2005	Patients who are pathologically staged as II and III [non-small-cell lung cancer] following resection should not receive postoperative chemoradiotherapy unless it is within a clinical trial.
Public Health Intervention PHI002	Physical Activity	2006	Practitioners, policy makers and commissioners should only endorse exercise referral schemes to promote physical activity that are part of a properly designed and controlled research study to determine effectiveness. Measures should include intermediate outcomes such as knowledge, attitudes and skills, as well as measures of physical activity levels. Individuals should only be referred to schemes that are part of such a study. [The Public Health Interventions Advisory Committee] determined that there was insufficient evidence to recommend the use of exercise referral schemes to promote physical activity other than as part of research studies where their effectiveness can be evaluated.

Finally, NICE often recommends that interventions are used only if outcome data are assembled in appropriate registers or ‘databases’ to assess their safety as well as their clinical and cost-effectiveness (Table 3).

Table 3.

Examples of NICE recommendations for submitting data to clinical disease registers or databases

Document	Year of issue	NICE recommendation
Technology Appraisals
Arthritis (juvenile idiopathic): etanercept (TA035)	2002	It is strongly recommended that all clinicians prescribing etanercept should (with the permission of the child and/or parent) register the child with the Biologics Registry established by the British Paediatric Rheumatology Group (BPRG) [now British Society for Paediatric and Adolescent Rheumatology] and forward information on dosage, outcome and toxicity on a quarterly basis.
Hip disease: metal on metal hip resurfacing (TA044)	2002	When [metal on metal] hip resurfacing arthroplasty is considered appropriate, the procedure should be performed only in the context of the ongoing collection of data on both the clinical effectiveness and cost-effectiveness of this technology. Ideally, this data collection should form part of a UK national joint registry.
Obesity (morbid): surgery (TA046)	2002	Databases should be established by hospitals wanting to develop their [surgical services for morbid obesity], to enable the outcomes and complications of different procedures, including their impact on quality of life, to be monitored both in the short and long term.
Leukaemia (chronic myeloid): imatinib (TA070)	2003	It is strongly advised that a national registry should be set up to provide valuable information on the long-term effectiveness of imatinib treatment. It is also strongly recommended that data collection from original registration studies should be continued in order to:
		• Investigate the comparative long-term efficacy of imatinib in terms of [quality of life] and survival, for all phases of [chronic myeloid leukaemia];
		• Investigate the adverse effects and the potential for the development of treatment resistance of long-term imatinib use.
Public Health Guidance
Physical activity (PHI002)	2006	An immediate priority is to establish the best validated measure(s) to assess baseline and subsequent changes in physical activity levels following an intervention. A basic minimum data set also needs to be established to assess levels of physical activity in the population. This might include related physiological and psychological measures.
Interventional Procedures
Laser in situ keratomileusis (LASIK) for the treatment of refractive errors (IPG 102)	2004	Further research [on laser in situ keratomileusis (LASIK) for the treatment of refractive errors] will be useful and clinicians are encouraged to collect longer-term follow-up data.
Impedance-controlled bipolar radiofrequency ablation for menorrhagia (IPG 104)	2004	Information is lacking about the long-term results of [impedance-controlled bipolar radiofrequency ablation for menorrhagia]. Clinicians are encouraged to collect data on rates of recurrence and late complications, and they should ensure that patients are aware of the uncertainty about long-term safety and efficacy.
Laparoscopic nephrectomy (including nephroureterectomy) (IPG 136)	2005	Patient selection is important when [laparoscopic nephrectomy (including nephroureterectomy)] is being considered for the treatment of malignant disease. Long-term follow-up data are lacking, and clinicians are encouraged to collect data on rates of recurrence in patients with malignant disease.
Percutaneous disc decompression using coblation for lower back pain (IPG 173)	2006	Current evidence suggests that there are no major safety concerns associated with the use of percutaneous disc decompression using coblation for lower back pain. There is some evidence of short-term efficacy; however, this is not sufficient to support the use of this procedure without special arrangements for consent and for audit or research. Further research will be useful in reducing the current uncertainty, and clinicians are encouraged to collect long-term follow-up data. The Institute may review the procedure upon publication of further evidence.

SUPPORTING THE POLICY OPTION: UPSTREAM AND DOWNSTREAM MEASURES

Implementing the OIR option in the NICE and broader NHS context has been challenging.

Issuing an OIR recommendation

NICE operates within an environment of inevitable uncertainty.²⁶ Over-exercising the OIR option in this setting could result in indecisiveness and inaction, hamper innovation, and negate NICE's very objective: to provide advice based on the best available evidence.

To ensure the OIR option is used in a pragmatic way, NICE is exploring the decision-making trail leading to and the criteria underpinning these decisions. These criteria should ideally reflect how well NICE and the NHS tolerate uncertainty. A number of questions need addressing:

• How much uncertainty should there be before a recommendation is issued that a technology be used only in the context of research?

• How would this vary (if at all) when there is no ongoing or planned research?

• Should the focus be solely on the quality of the evidence or should the potential budgetary impacts and clinical importance of the practice be considered when issuing an OIR recommendation?

• How can these decisions be made in a transparent, consistent and methodologically sound way across NICE?

However challenging these questions may be, a methodological framework is already in place to help support the thinking in this area. Value of information principles already referred to in the NICE appraisals manual, supplemented by options theory,²⁷ could provide the methodological underpinnings for tackling some of these questions. At the same time, a qualitative checklist could also be developed to help guide the deliberations of the advisory bodies. For example, when further research is necessary and could help resolve uncertainty, advisory committee members could be asked to consider what the implications of making a positive recommendation would be:

• On any ongoing research and thus the development of the evidence base;

• On total spending, particularly if the recommendation is reversed in the future but the NHS has already incurred irretrievable fixed costs.

Implementing OIR recommendations

No formal arrangements exist between NICE, industry and the clinical research community, including the NHS and patient organizations, for pursuing the research recommended by NICE in a timely manner, so that uncertainties are reduced to the extent that definitive guidance can be developed. This contrasts with the active arrangements for promoting the uptake of positive NICE recommendations. As a result, OIR recommendations are hard to implement, particularly when there is no ongoing trial addressing the question. Indeed, they are often perceived to be negative recommendations by patients, clinicians and industry alike. In a recent BMJ paper reviewing NICE recommendations between 1999 and 2005, Raftery classifies all recommendations for the use of a technology in the context of research as ‘no’ decisions.²⁸

The development of a co-ordinated approach with other interested parties could encourage NICE advisory bodies to consider OIR recommendations in a more systematic and consistent way. For example, in guidance on the use of beta interferon and glatiramer in the treatment of multiple sclerosis NICE recommended that the Department of Health and the National Assembly for Wales, along with the manufacturers, consider what action could be taken so that the NHS could obtain these drugs in a way that would be cost-effective.²⁹ A national data collecting system was established to ascertain which sub-groups of patients are most likely to benefit. However, the lack of a coordinated approach has made it difficult to ensure that the information collected will ever inform an update of the NICE guidance

Furthermore, as the burden of evaluation lies with NICE, mandating research from the technology sponsors is unrealistic and wishful thinking, particularly in cases where head-to-head trials are needed. The prospect of an appeal, and the pressure generated by lobbying groups keen to ensure access to the new technology, make the OIR option even less attractive to NICE advisory committees.

As a result, the danger is that OIR recommendations turn into a means of saying no politely, instead of generating the additional evidence needed to make more confident decisions.

Indeed, the recently published Cooksey Review³⁰ recommends that special arrangements are set up and funding earmarked to ‘... implement NICE recommendations calling on the NHS to use health interventions in a research context’. This is an exciting development, directly addressing the need for co-ordination between publicly funded research and evidence-based policy making, as reflected in the NICE process of appraising new technologies. Key stakeholders, including NICE, research funders, patient organizations, industry, government and the NHS, will have to work together to implement Cooksey's recommendations. In the meantime, some have hastened to interpret the Cooksey proposals as a call for conditional coverage of technologies, across the NHS, long before phase III trials are completed.³¹ However important it is to accelerate access to new technologies, without the right processes in place, such moves could compromise patient safety and overall health outcomes in a financially restrained system.

Furthermore, the OIR option is now applied not only to new medical technologies with interested sponsors as originally anticipated,¹² but also to established clinical and public health strategies that almost always lack sponsorship. This has reinforced the implementation challenges described above but has also highlighted how important it is for NICE advisory bodies to be able to recommend data collection on existing, often well-established practices or interventions to inform recommendations for increased investment, or for disinvestment. The need to support the NHS by advising which ineffective, wasteful or even harmful practices should be discontinued² makes this evaluative dimension of the OIR option particularly relevant.

Linking research funding to NICE decision making is a pressing need; the recent single technology appraisal of Herceptin for early breast cancer is a case in point: in August 2006, only three months after the drug gained its license, NICE recommended the use of the drug for eligible women. The decision was supported by the available evidence at the time³² and recently published studies further reinforce this decision.³³ However, a positive decision based on the best available evidence may not be enough. NICE-mandated, publicly-funded research into alternative therapeutic regimes beyond the single licensed indication determined by the manufacturer could inform future updates of the guidance and ensure the Health Service is obtaining value.³⁴ Carrying out health technology assessment earlier on in the development of a technology in the context of the single technology appraisal poses a significant challenge if the evidential standards are to be maintained. Alternative approaches to the yes/no dichotomy, which encourage the development of the evidence base, will need to be applied to avoid widespread use of unproven technologies or denying access to promising treatments (Table 4).

Table 4.

Implications of getting it wrong

Saying no instead of Only In Research	Saying yes instead of Only In Research
• Patients are denied access to promising and potentially effective technologies	• Access to unproven and potentially harmful or ineffective interventions is promoted
• There are delays in building the evidence base in key areas, with a resulting negative overall impact on health outcomes	• Any ongoing or future research in the field is severely hindered. Important questions on effectiveness and cost-effectiveness may never be answered.
• Innovation that adds value is hampered and commercial R&D investment discouraged	• The NHS encourages `bad' value innovation
	• NICE increases inflationary pressures on the NHS and wastes limited resources
	• Having to reverse a yes decision in light of any future evidence compromises NICE's credibility and is difficult to implement

The views of the NICE Citizens' Council

NICE set up the Citizens' Council in 2002 (http://www.nice.org.uk/citizenscouncil) to advise NICE and its advisory bodies on the social value judgments that should underpin its guidance. Its 30 members reflect the demography of England and Wales and meet twice a year to debate issues put to them by NICE. Their reports inform the Social Value Judgments principles which constitute official guidance to NICE's advisory bodies.

In January 2007 the Council met to discuss NICE's approach to uncertainty. In their draft report (currently under public consultation) they unanimously endorse OIR as a valuable decision option for NICE and its advisory bodies, and go on to discuss the circumstances under which NICE should consider issuing such a recommendation. A number of key considerations are identified:

• The extent to which further research is likely to reduce current uncertainty;

• The value-for-money of the research;

• The implications of a positive recommendation on the evidence base and the NHS budget when this is made inappropriately in place of an OIR decision;

• The existence of an ongoing study or the feasibility setting one up study within a realistic time frame;

• Issues of patient access to the study across different geographical areas.

Having discussed the need for explicit criteria for OIR decisions and the difficulties of implementing the necessary research, the Council report concludes ‘... patients would be reassured to know that clinicians and the healthcare system in general could face up to uncertainty, and were confident enough to deal with it in a mature, scientific way, and avoid wasting money on unproven technologies.’³⁵

DISCUSSION

OIR recommendations have been statutory valid alternatives to yes and no decisions since NICE was set up by government in 1999. They represent the only rational way for addressing uncertainty, consistent with NICE's principles of transparency and methodological robustness. As demonstrated in a number of cases so far, OIR recommendations can be viable and workable decision options as long as there is some co-ordination between the research and decision-making communities.

Our study brings together all occurrences of such NICE decisions and describes how they have evolved over the years in response to the generation or absence of new evidence. We have not discussed here those cases where an OIR decision should have been made by NICE but was not, neither have we sought to identify any OIR that were made inappropriately given the circumstances. To do either of the above we would need to establish clear decision criteria as to when OIR recommendations are justified. To achieve this, additional work is underway to:

• Develop a clear set of criteria for recommending interventions be used only ‘in the context of research’. These would reflect how much uncertainty NICE and the NHS are willing to accept without hampering innovation, restricting access or compromising patient safety and wasting valuable resources. Clear criteria will increase the transparency and consistency of such decisions in the future and allow evaluation of past OIR recommendations;

• Establish clear arrangements for implementing NICE OIR recommendations in a timely, responsive and practical manner. These cover issues around who would bear the burden of proof, the role of manufacturers and of the NHS in co-ordinating and/or financing the research, and what happens when a technology has no sponsor, as is the case for many public health or established interventions.

SUMMARY BOX

NICE guidance is based on the best available evidence; however, there are several situations in which there may be significant uncertainties about the intervention under consideration. In these cases, recommending the use of the intervention in the context of research can help fill the evidence gaps, while ensuring the efficient use of resources and encouraging innovation that adds value. Making OIR recommendations has been a policy option for NICE since it was established in 1999. However, setting out clear and consistent criteria leading to such decisions and ensuring that the recommended research is undertaken in a timely and responsive manner are challenging tasks. Through reviewing the cases where NICE has issued OIR recommendations, this paper reinforces the importance of acknowledging uncertainty and highlights the difficulties in implementing this policy option within the NHS. With the recent publication of the Cooksey review of UK medical research and NICE introducing new streams of work, including faster technology appraisals closer to licensing and reviews of established treatments to reduce wasteful activities, the OIR option needs to be revisited.

In light of the recently published Citizens' Council report on OIR, we intent actively to engage with the broader public in this exercise so that any policy initiatives are not only consented upon but also driven by those of us who use the NHS.