How to formulate research recommendations

Short abstract

“More research is needed” is a conclusion that fits most systematic reviews. But authors need to be more specific about what exactly is required

Long awaited reports of new research, systematic reviews, and clinical guidelines are too often a disappointing anticlimax for those wishing to use them to direct future research. After many months or years of effort and intellectual energy put into these projects, authors miss the opportunity to identify unanswered questions and outstanding gaps in the evidence. Most reports contain only a less than helpful, general research recommendation. This means that the potential value of these recommendations is lost.

Current recommendations

In 2005, representatives of organisations commissioning and summarising research, including the BMJ Publishing Group, the Centre for Reviews and Dissemination, the National Coordinating Centre for Health Technology Assessment, the National Institute for Health and Clinical Excellence, the Scottish Intercollegiate Guidelines Network, and the UK Cochrane Centre, met as members of the development group for the Database of Uncertainties about the Effects of Treatments (see bmj.com for details on all participating organisations). Our aim was to discuss the state of research recommendations within our organisations and to develop guidelines for improving the presentation of proposals for further research. All organisations had found weaknesses in the way researchers and authors of systematic reviews and clinical guidelines stated the need for further research. As part of the project, a member of the Centre for Reviews and Dissemination under-took a rapid literature search to identify information on research recommendation models, which found some individual methods but no group initiatives to attempt to standardise recommendations.

Suggested format for research recommendations on the effects of treatments

Core elements

E Evidence (What is the current state of the evidence?)

P Population (What is the population of interest?)

I Intervention (What are the interventions of interest?)

C Comparison (What are the comparisons of interest?)

O Outcome (What are the outcomes of interest?)

T Time stamp (Date of recommendation)

Optional elements

d Disease burden or relevance

t Time aspect of core elements of EPICOT

s Appropriate study type according to local need

In January 2006, the National Coordinating Centre for Health Technology Assessment presented the findings of an initial comparative analysis of how different organisations currently structure their research recommendations. The National Institute for Health and Clinical Excellence and the National Coordinating Centre for Health Technology Assessment request authors to present recommendations in a four component format for formulating well built clinical questions around treatments: population, intervention, comparison, and outcomes (PICO).¹ In addition, the research recommendation is dated and authors are asked to provide the current state of the evidence to support the proposal.

Clinical Evidence, although not directly standardising its sections for research recommendations, presents gaps in the evidence using a slightly extended version of the PICO format: evidence, population, intervention, comparison, outcomes, and time (EPICOT). Clinical Evidence has used this inherent structure to feed research recommendations on interventions categorised as “unknown effectiveness” back to the National Coordinating Centre for Health Technology Assessment and for inclusion in the Database of Uncertainties about the Effects of Treatments (www.duets.nhs.uk).

We decided to propose the EPICOT format as the basis for its statement on formulating research recommendations and tested this proposal through discussion and example. We agreed that this set of components provided enough context for formulating research recommendations without limiting researchers. In order for the proposed framework to be flexible and more widely applicable, the group discussed using several optional components when they seemed relevant or were proposed by one or more of the group members. The final outcome of discussions resulted in the proposed EPICOT+ format (box).

Examples

A recent BMJ article highlighted how lack of research hinders the applicability of existing guidelines to patients in primary care who have had a stroke or transient ischaemic attack.² Most research in the area had been conducted in younger patients with a recent episode and in a hospital setting. The authors concluded that “further evidence should be collected on the efficacy and adverse effects of intensive blood pressure lowering in representative populations before we implement this guidance [from national and international guidelines] in primary care.” Table 1 outlines how their recommendations could be formulated using the EPICOT+ format. The decision on whether additional research is indeed clinically and ethically warranted will still lie with the organisation considering commissioning the research.

Table 1.

Research recommendation based on gap in the evidence identified by a cross sectional study of clinical guidelines for management of patients who have had a stroke

		Issues to consider	Example
Core elements
E	Evidence	What is the current evidence?	One systematic review3 dominated by a large randomised controlled study4 conducted in hospital setting
P	Population	Diagnosis, disease stage, comorbidity, risk factor, sex, age, ethnic group, specific inclusion or exclusion criteria, clinical setting	Primary care patients with confirmed stroke or transient ischaemic attack (mean age ≥75 years, female-male ratio 1:1, time since last cerebrovascular event ≥1 year)
I	Intervention	Type, frequency, dose, duration, prognostic factor	Intensive blood pressure lowering
C	Comparison	Placebo, routine care, alternative treatment/management	No active treatment or placebo
O	Outcome	Which clinical or patient related outcomes will the researcher need to measure, improve, influence or accomplish? Which methods of measurement should be used?	Major vascular events (stroke, myocardial infarction, vascular death); adverse events, risk of discontinuation of treatment because of adverse events
T	Time stamp	Date of literature search or recommendation	February 2006
Optional elements
d	Disease burden		Stroke is the most common cause of death and disability in most developed countries. It is a worldwide problem; about 4.5 million people die from stroke each year. Stroke can occur at any age, but half of all strokes occur in people over 70 years old. Risk factors for stroke include previous stroke of transient ischaemic attack (10% in the first year and about 5% each year after), increasing age, hypertension, diabetes, cigarette smoking, and emboli associated with atrial fibrillation, artificial heart valves, or myocardial infarction5 6
t	Timeliness	Time aspects of core elements:
		Mean age of population	Over 65
		Duration of intervention	Minimum 5 weeks
		Length of follow-up	Any length
s	Study type	What is the most appropriate study design to address the proposed question?	Randomised controlled trial.

Table 2 shows the use of EPICOT+ for an unanswered question on the effectiveness of compliance therapy in people with schizophrenia, identified by the Database of Uncertainties about the Effects of Treatments.

Table 2.

Research recommendation based on a gap in the evidence on treatment of schizophrenia identified by the Database of Uncertainties about the Effects of Treatments

		Issues to consider	Example
Core elements
E	Evidence	What is current state of the evidence?	One systematic review7 identified one small randomised controlled trial comparing compliance therapy with non-specific counselling which found no significant difference in adherence over 1 year8
P	Population	Diagnosis, disease stage, comorbidity, risk factor, sex, age, ethnic group, specific inclusion or exclusion criteria, clinical setting	People with schizophrenia or related disorders
			Suggested sample size >300 (powered to find 10% difference between groups for the primary outcome)
			Sex: men and women
			History: people in their first episode reported separately
I	Intervention	Type, frequency, dose, duration, prognostic factor	Compliance therapy administered according to manual of Kemp and David
C	Comparison	Placebo, routine care, alternative treatment or management	Non-specific counselling
O	Outcome	Which clinical or patient related outcomes will the researcher need to measure, improve, influence or accomplish? Which methods of measurement should be used?	Service use: bed occupancy (primary outcome)
			Compliance: clinical interview
			Other routinely recorded measures of mental state, quality of life, general functioning, adverse effects and service use
T	Time stamp	Date of literature search or recommendation	September 2006
Optional elements
d	Disease burden or relevance		Prevalence of schizophrenia worldwide is 2-4/1000. 1 in 100 people will develop schizophrenia9 10
t	Timeliness	Time aspect of core elements:
		Mean age of population	Working age adults
		Duration of intervention or comparison	5 sessions of 30-60 minutes
		Length of follow-up	2 years
s	Study type	What is the most appropriate study design to address the proposed question?	Randomised controlled trial
			Methods: concealment clear
			Blindness: patients and therapists not blind, assessors blind
			Setting: in hospital at start of study, community follow-up

Discussions around optional elements

Although the group agreed that the PICO elements should be core requirements for a research recommendation, intense discussion centred on the inclusion of factors defining a more detailed context, such as current state of evidence (E), appropriate study type (s), disease burden and relevance (d), and timeliness (t).

Initially, group members interpreted E differently. Some viewed it as the supporting evidence for a research recommendation and others as the suggested study type for a research recommendation. After discussion, we agreed that E should be used to refer to the amount and quality of research supporting the recommendation. However, the issue remained contentious as some of us thought that if a systematic review was available, its reference would sufficiently identify the strength of the existing evidence. Others thought that adding evidence to the set of core elements was important as it provided a summary of the supporting evidence, particularly as the recommendation was likely to be abstracted and used separately from the review or research that led to its formulation. In contrast, the suggested study type (s) was left as an optional element.

A research recommendation will rarely have an absolute value in itself. Its relative priority will be influenced by the burden of ill health (d), which is itself dependent on factors such as local prevalence, disease severity, relevant risk factors, and the priorities of the organisation considering commissioning the research.

Similarly, the issue of time (t) could be seen to be relevant to each of the core elements in varying ways—for example, duration of treatment, length of follow-up. The group therefore agreed that time had a subsidiary role within each core item; however, T as the date of the recommendation served to define its shelf life and therefore retained individual importance.

Applicability and usability

The proposed statement on research recommendations applies to uncertainties of the effects of any form of health intervention or treatment and is intended for research in humans rather than basic scientific research. Further investigation is required to assess the applicability of the format for questions around diagnosis, signs and symptoms, prognosis, investigations, and patient preference.

When the proposed format is applied to a specific research recommendation, the emphasis placed on the relevant part(s) of the EPICOT+ format may vary by author, audience, and intended purpose. For example, a recommendation for research into treatments for transient ischaemic attack may or may not define valid outcome measures to assess quality of life or gather data on adverse effects. Among many other factors, its implementation will also depend on the strength of current findings—that is, strong evidence may support a tightly focused recommendation whereas a lack of evidence would result in a more general recommendation.

The controversy within the group, especially around the optional components, reflects the different perspectives of the participating organisations—whether they were involved in commissioning, undertaking, or summarising research. Further issues will arise during the implementation of the proposed format, and we welcome feedback and discussion.

Summary points

No common guidelines exist for the formulation of recommendations for research on the effects of treatments

Major organisations involved in commissioning or summarising research compared their approaches and agreed on core questions

The essential items can be summarised as EPICOT+ (evidence, population, intervention, comparison, outcome, and time)

Further details, such as disease burden and appropriate study type, should be considered as required