Tackling treatment uncertainties together: the evolution of the James Lind Initiative, 2003–2013

At the end of 2003, the Journal of the Royal Society of Medicine published an article announcing the creation of the James Lind Initiative (JLI).¹ The article began by recalling the way that James Lind, an 18th century Scottish naval surgeon, had confronted uncertainty about how to treat scurvy by doing a controlled trial. This had shown that oranges and lemons were a very effective cure for an often lethal disease.^2,3 The 2003 article went on to suggest that therapeutic uncertainties today should also be confronted in research as an integral element of responsible healthcare, and that the JLI was being established to promote this principle. The current article describes the origins of the JLI and its work over the past 10 years.

The origins of the JLI

The idea for the JLI had its origins at a brainstorming meeting on 2 April 2001 convened by the UK Cochrane Centre to consider what might be done to increase general knowledge about why treatments need to be tested rigorously, and what rigorous testing of treatments entails. There were over 40 participants, from backgrounds including patient and public representation, education, lay media and medical journalism, government and parliament, medical ethics, history of medicine, medical practice, and research and research funding.⁴ Discussion and subsequent feedback generated a wide variety of suggestions, ranging from reducing research jargon in interactions with patients and the public, through more honest admission of uncertainties about the effects of treatments, to greater involvement of patients in shaping the health research agenda.⁵

The following year the Medical Research Council (MRC) established a working group to review the Council’s involvement in clinical trials. The section of the group’s report entitled ‘Promoting public engagement’ contained a number of recommendations, including a call for the creation of ‘a communications and discussion forum on randomized controlled trials, involving patients, practitioners, researchers, and others’, and stated that, with support from the Department of Health, the MRC had appointed Iain Chalmers to take this initiative forward.^6,7

After seeking views on how the challenge of engaging patients and clinicians in clinical trials might be pursued, it was decided that the JLI would take an indirect approach to the MRC’s brief: instead of promoting public and professional discussion of clinical trials directly, the JLI would instead promote discussion about how patients, clinicians and policy-makers should respond to uncertainties about the effects of treatments. The rationale for this theme was spelled out the in the JLI’s launch article in 2003 (Chalmers 2003a1), and in editorials published early in 2004.^8–10

Tens of thousands of uncertainties about treatment effects have never been investigated using systematic reviews of research evidence, and thousands of systematic reviews have made clear that many uncertainties about treatment effects have not been resolved by researchers. Furthermore, uncertainties reflected in the research choices of people in academia and industry cannot be assumed to address questions about the effects of treatments that are important to patients, clinicians and other users of research. For example, when patients, rheumatologists, physiotherapists and general practitioners were asked to identify their priorities for research on the management of osteoarthritis of the knee, there was little enthusiasm for yet more studies of the drugs that dominate the existing research on this condition. Patients and clinicians said they wanted more rigorous evaluation of the effects of physiotherapy and surgery, and better assessment of the educational and coping strategies that might help patients to manage this chronic, disabling and often painful condition.¹¹

Systematic reviews of available clinical research evidence prepared over the past two decades have made clear that this is not an isolated example of a mismatch between choices made by researchers and the priorities of patients and clinicians.¹² If the JLI was going to engage patients and clinicians in discussions about responding to uncertainties, it seemed sensible to begin by finding out which uncertainties mattered most to them.

Prioritizing uncertainties about the effects of treatments for further research

The James Lind Alliance (JLA): philosophy and early planning

To develop these ideas, Iain Chalmers sought help from Nick Partridge, chair of an organization promoting public involvement in health research (INVOLVE), and John Scadding, Dean of the Royal Society of Medicine, which has an education programme for clinicians. They agreed to establish a ‘James Lind Alliance’ to facilitate the identification of research priorities shared by patients, carers and clinicians.^13,14

The philosophy and values underpinning the JLA patient–clinician Priority Setting Partnerships (PSPs) were: (i) inclusion of a range of patient and clinician perspectives; (ii) transparency of the methods, outcomes and vested interests of those taking part in prioritization; and (iii) evidence-based knowledge – ‘known unknowns’ – on which to base prioritization. Research commissioned by the James Lind Alliance^15–17 suggested that identifying shared therapeutic research priorities by patient–clinician partnerships was rare, if not unique. To add momentum to the JLA’s ambitious plans, interested individuals and organizations were invited to become JLA ‘Affiliates’, and a website was established to promote the Alliance (www.lindalliance.org).

After ‘setting out its stall’ in articles and meetings, the JLA waited to be invited to facilitate the creation and work of PSPs. If people and organizations did not warm to the proposed concept and approach, the initiators were ready to acknowledge failure. In the event, there was no shortage of interest from patients and carers, although it was often difficult to engage clinicians.¹⁸

The UK Database of Uncertainties about the Effects of Treatments (UK DUETs)

Unsurprisingly, organizing the first PSP (for asthma) proved very challenging and it developed quite slowly. It was decided to create a public database in which uncertainties about the effects of treatments could be made explicit, as had been proposed at the brainstorming meeting in 2001.^19,20 This would provide a resource to help prioritize uncertainties for investigation in new research – either in the form of new, extended or updated systematic reviews; or in additional ‘primary’ research. ‘Treatment uncertainties’ were defined as ‘uncertainties that could not be resolved by reference to reliable, up-to-date systematic reviews of existing research evidence’. UK DUETs was conceptualized in 2005, developed with input from two advisory groups, and launched in 2006.²¹

After pilot work had been done, an infrastructure was needed to roll out UK DUETs, so that, as new uncertainties were identified, its coverage could be extended and maintained. In late 2006, it was agreed that the National Library for Health (NLH) would provide an appropriate infrastructure, particularly because of its commitment to address the needs of patients as well as clinicians. The following year NLH invitations to tender for specialist libraries specified that these would be expected to ‘identify and publish uncertainties about the effects of treatments, using agreed procedures, through DUETs’. This requirement was a clear recognition by the National Health Service that it is important to present information about what is not known as well as about what is known about the effects of treatments.

By the end of 2012, UK DUETs contained 4760 uncertainties, 67% derived from reports of systematic reviews and clinical guidelines, and 14% from protocols for systematic reviews (such as those published in the Cochrane Database of Systematic Reviews) or from registered information about ongoing clinical trials. Just under one-fifth (19%) of the treatment uncertainties came directly from patients or carers (14%), or from clinicians (5%).

James Lind Alliance Priority Setting Partnerships (JLA PSPs)

A review of published material on methods to achieve consensus and develop research priorities made clear that reports of existing experience were very limited.²² A JLA Strategy and Development Group and a Monitoring and Implementation Group debated how to formulate and implement JLA methods for identifying priorities shared by patients and clinicians.

UK DUETs proved to be an essential source of data for JLA PSPs, as well as being a repository for additional uncertainties revealed as a result of the surveys of patients and clinicians organized by PSPs. Figure 1 shows the process of refinement and prioritization, from ‘raw’ uncertainties to those prioritized for further research. Between 2005 and 2008 the JLA methodology was piloted in asthma and urinary incontinence. Reflection and discussion followed each key step of these initial PSPs, with refinements to the methods being used. Meetings and workshops were observed and recorded and both groups published their findings.^23,24 Subsequent PSPs added to an understanding of how to involve diverse groups in prioritization.

Figure 1.

Identifying and prioritizing uncertainties about the effects of treatment

Initially, the PSPs were based on specific health problems, but this focus was sometimes extended to cover a wide range of conditions (for example, ‘sight loss’), as well as to include treatment settings (for example, intensive care units). Whereas the first PSP had only one patient partner (Asthma UK) and one clinician partner (the British Thoracic Society), some later PSPs have had over 40 partner organizations. By March 2013 there had been 13 completed PSPs and a further 12 were at various stages of completion (Table 1). There are no signs yet of any let up in requests for JLA support for additional PSPs.

Table 1.

JLA priority setting partnerships

Completed	Current
Asthma	Acne
Urinary incontinence	Childhood disability
Vitiligo	Dementia
Prostate cancer	Dialysis
Schizophrenia	Inflammatory bowel disease
Type 1 diabetes	Multiple sclerosis
ENT aspects  of balance Life after stroke Eczema Tinnitus	Pre-term birth Sight loss and vision Skin-suppurative hidradenitis Hips and knee replacement  for osteoarthritis
Cleft lip and palate	Spinal cord injury
Lyme disease	Palliative care
Pressure ulcers
Epidermolysis Bullosa

Priority research themes emerging from JLA PSPs include emphasis on the need to assess long-term effects of treatments; safety and adverse effects of treatments; effects of complementary and non-prescribed treatments; and the effectiveness and safety of self-care;²⁵ and Table 2 shows the mismatch between the types of interventions that patients and clinicians wish to see evaluated compared with those being assessed by researchers.²⁶ Table 3 shows the implications for researchers of the priorities identified: additional primary research was needed to address under half of the priorities; new or updated systematic reviews of existing evidence were the appropriate response to the remainder.

Table 2.

Interventions mentioned in research priorities identified by James Lind Alliance patient–clinician Priority Setting Partnerships, and among registered trials, 2003–2012 (Crowe et al. in preparation)

Type of intervention	JLA patient–clinician Priority Setting Partnerships % (of total of 113 interventions mentioned)	Registered non-commercial trials % (of total of 1036 interventions mentioned)	Registered commercial trials % (of total of 798 interventions mentioned)
Drugs, vaccines and biologicals	19.4 (22)	37.2 (397)	86.3 (689)
Radiotherapy, surgery and perioperative, devices, and diagnostic	23.1 (26)	29.8 (332)	11.1 (89)
Education and training, service delivery, psychological therapy, physical therapies, exercise, complementary therapies, social care, mixed or complex, diet, other	57.6 (65)	31.9 (307)	2.6 (20)

Table 3.

James Lind Alliance: principal implications of top 10 research priorities

PSP	Prepare systematic review of existing evidence	Update or extend existing systematic review(s)	Design and do additional primary research
Asthma	4	3	4
Cleft lip and palate	5	0	7
ENT Balance	8	2	0
Eczema	4	2	8
Lyme disease	10	0	0
Prostate	0	0	11
Schizophrenia	3	4	3
Life after stroke in Scotland	3	3	4
Tinnitus	5	0	5
Type 1 diabetes	7	1	2
Urinary incontinence	4	1	5
Vitiligo	4	5	3
Total	57	21	52

Throughout its evolution, the JLA’s role has been as a neutral support/honest broker, and it encouraged flexibility as long as the founding principles of patient and clinician parity, transparency and systematic rigour were respected. The JLA Guidebook²⁷ has drawn on this experience, and has documented the variety of approaches adopted, showcased good practice, and encouraged PSPs to think independently. It describes in straightforward terms how to involve patients, carers and clinicians in research priority-setting. The Guidebook has been developed in consultation with its intended users because it had to be accessible to anyone – whether patient, clinician or researcher.

The mission, methods and output of the JLA have become quite widely reported and recognized during recent years and emulated in other countries. Furthermore, the MRC invited the JLA to consider how the JLA’s working principles might be applied to the Council’s support of pre-clinical research.

Mainstreaming UK DUETs and the JLA

An application in 2009 to extend support for the JLI for a further three years was approved on condition that efforts were made to ‘mainstream’ UK DUETs and the JLA. This process was facilitated by the support that UK DUETs had received from the National Library for Health, for which the National Institute for Health and Clinical Excellence (NICE) had become responsible, and the encouragement that plans for the JLA had received from its very first affiliate – the NIHR Health Technology Assessment Programme.

In 2010, it was agreed that formal responsibility for maintaining and developing UK DUETs should pass from the JLI to NICE. As far as we are aware, NICE thus became the first provider of health information to enable users to select information about uncertainties, and thus to identify gaps in knowledge and needs for additional research.^28,29 In 2012, the Swedish Council on Health Technology Assessment (SBU)³⁰ paid UK DUETs a compliment by announcing that it had established ‘a Swedish DUETs’. Finally, in 2011, it was agreed that, from April 2013, the NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC) would assume responsibility for managing the work of the JLA.

To mark the end of the JLI’s responsibility for UK DUETs and the JLA, it co-convened with NICE and the Association of Medical Research Charities a meeting to present information about national resources containing information about uncertainties and methods used for research priority setting.³¹ The responses from research charities new to these ideas ranged from scepticism to enthusiasm. A comment from one enthusiastic participant went to the heart of the matter:

‘A key lesson for us was to ask – always – “where is the patient in this uncertainty?” Charities don’t exist to give us jobs but to help those who give money. Asking them what to do with it should be normal.’

Enhancing general knowledge about testing treatments

Patients and members of the public more generally can play active parts in research addressing uncertainties about the effects of treatments. If patients and the public are to contribute effectively, however, they need help to enhance their general knowledge about how reliable information about the effects of treatments is generated.

This challenge has been addressed in the other element of the JLI’s programme of work – through articles, talks, interviews, radio and TV broadcasts, and committee and working party membership. The Initiative has also been involved in research showing that new treatments are as likely to be worse as they are to be better than existing treatments, evidence that is very relevant to creating an atmosphere in which research should flourish.^32–34

In addition, the JLI has: (i) coordinated the development of The James Lind Library (www.jameslindlibrary.org), a multilingual website introducing the reasons for and characteristics of fair tests of treatments; (ii) played a major part in co-authoring and making accessible two editions of Testing Treatments: better research for better healthcare, a book written for the public;^35,36 (iii) co-convened a meeting to establish an international Network to Support Understanding in Health Research (www.nsuhr.net); and (iv) established a new multilingual website called Testing Treatments interactive (www.testingtreatments.org).

The James Lind Library (www.jameslindlibrary.org)

The James Lind Library (JLL) is a website explaining and illustrating the evolution of fair tests of treatments. It was launched in the same year (2003) as the JLI, which also coincided with the 250th anniversary of the publication of Lind’s Treatise of the Scurvy.² The JLL contains historical material illustrating the evolution of fair tests of treatments, mostly provided by the Sibbald Library of the Royal College of Physicians of Edinburgh. Essays introducing the features of fair tests were added to the historical material, and this led Scientific American to select the JLL as one of only five medical websites to receive an award in 2003, and the only one based outside the USA.³⁷

The JLL has attracted widespread interest. In particular, the World Health Organization and the Pan American Health Organization funded the translation of the JLL explanatory essays into Arabic, Chinese, French, Portuguese, Russian and Spanish. The website has hosted for free download the texts of three books written for the public: Testing Treatments,³⁵ in six languages as well as in English; Smart Health Choices;³⁸ and Know your Chances.³⁹

Historical records and articles commissioned for publication in the JLL continue to be added to the website, and many of these articles have been republished in the Journal of the Royal Society of Medicine every month since October 2005. Copies of the website are being regularly ‘future proofed’ as part of the British Library’s Web Archiving Programme.

Testing treatments: better research for better healthcare

In December 2002, Imogen Evans, formerly an executive editor at The Lancet and then on the staff of the MRC, invited Iain Chalmers to co-author a book for the public on clinical trials. Planning this work began in earnest in July 2003, after IC had become responsible for taking forward elements of the MRC’s clinical trials strategy.⁶ IE and IC agreed that authorship would be strengthened by recruiting a ‘lay` viewpoint. On the basis of her decade of independent campaigning for citizen engagement in the research process, Hazel Thornton was recruited as a co-author.

The three co-authors all contributed to research and writing for the book, although the final drafting was done by Imogen Evans. The JLI co-ordinated the preparation of the manuscript for submission to the British Library. Testing Treatments was first published in 2006,³⁵ with a foreword written by the broadcaster and writer Nick Ross, and launched by the British Library at an event supported jointly by DH and the MRC.

The book was well received and the text was made available for free download from www.jameslindlibrary.org. After it had been downloaded 130,000 times, a new publisher – Pinter and Martin – published a second imprint, with an additional foreword by science journalist Ben Goldacre. Translations of the book were subsequently published in Arabic, Chinese, German, Italian, Polish and Spanish.

Because of the success of the first edition of Testing Treatments, discussions about a second edition began in 2008. Paul Glasziou, an Australian general practitioner, who had been director of the Centre for Evidence-Based Medicine at the University of Oxford, was recruited as a co-author of the new edition. The second edition of the book was published in print and ebook forms in October 2011 by Pinter and Martin.³⁶ As with the previous edition, the text of the new edition was made available for free download from a website www.testingtreatments.org, and, at the time of writing, it is one of only two books featured in the PubMedHealth ‘Bookshelf’ operated by the US National Library of Medicine. Translations seem likely to be freely available in at least a dozen languages other than English.

Testing treatments interactive

In 2010, consideration was given to upgrading the software of the James Lind Library (JLL), to take advantage of improvements in website functionality. Although the JLL will continue to assemble evidence about the evolution of fair tests of treatments, it became clear that, for technical reasons, it would be preferable to start from scratch with a new website designed primarily to enhance general knowledge about testing treatments. The JLI decided to base this around the text of the second edition of Testing Treatments. Accordingly, Testing Treatments interactive was established at www.testingtreatments.org. It is using video, audio, cartoons and other material to illustrate the messages and principles covered in the book.

Testing Treatments interactive (TTi) was piloted in late 2011, with sibling development sites established in Arabic, German and Turkish in late 2011 and early 2012. A complete version of TTi English was launched in August 2012, and a sibling site in Spanish – TTi Español – was launched in December 2012 (www.es.testingtreatments.org). In January 2013, a meeting was held to establish a TTi Editorial Alliance, bringing together editors involved in translating the second edition of Testing Treatments and in establishing and managing sibling interactive websites in Arabic, Turkish, German, Russian, Bahasa, Portuguese, Chinese, French, Norwegian, Croatian, Polish and Basque. The Editorial Alliance will provide a forum within which editors of all the sibling websites can exchange experiences and ways of evaluating the effectiveness of their work.

JLI in future

Enhancing general knowledge about testing treatments

Enhancing general knowledge about testing treatments will remain one of the JLI’s two main work themes in future. Testing Treatments interactive (TTi) will be a major component of this work, and will involve close collaboration with Paul Glasziou (Bond University, Australia), Douglas Badenoch (Minervation) and Amanda Burls (Oxford University).

TTi is already a key open learning resource for ECRAN⁴⁰ (the EU-supported European Communication on Research Awareness Needs Project), and SIHCLIC (Supporting Information for Health Care in Low Income Countries, funded by the Norwegian Government). To evaluate the impact of these learning resources, the JLI will collaborate with members of the TTi Editorial Alliance, especially colleagues at the Kunnskapssenteret in Oslo, and editors of the multilingual WHO Reproductive Health Library.

To remind people of the long history of efforts to obtain trustworthy evidence to inform and guide choices in healthcare,^41–52 the JLI will continue to maintain the James Lind Library,³⁷ with editorial input from Ulrich Tröhler (University of Bern, Switzerland) and Mike Clarke (Queen’s University, Belfast), in particular.

The JLI will continue to use opportunities to promote the principle that evaluative research addressing important uncertainties should be an expected element of usual clinical care,^53–59 as had been promoted by the GMC’s injunction that doctors ‘must work with colleagues and patients to help to resolve uncertainties about the effects of treatments’.⁶⁰ To encourage the application of this principle in practice, the JLI will continue to work with the Health Research Agency and others to reduce disproportionate regulation of clinical research.^61–69

Monitoring and reducing waste in research

Through a variety of mechanisms the public ends up meeting the cost of most medical research; but important questions remain about the value of the returns on this investment,⁷⁰ a matter of particular importance at a time of economic constraint. In a paper published in The Lancet in 2009, Iain Chalmers and Paul Glasziou⁷¹ estimated that 85% of the resources invested in medical research was being avoidably wasted. They have been involved in two important responses to their paper. First, they have had regular meetings with NETSCC to discuss and evaluate ways in which waste can be reduced in DH-funded research programmes, and to develop indicators for assessing progress. Second, after being asked by The Lancet to coordinate a series of articles on research waste to raise awareness of this problem, they have engaged co-authors to cover waste in pre-clinical as well as clinical and epidemiological research in a series of papers.

Monitoring and reducing waste in research will be the second of the two themes in the future work of the JLI. The Initiative will continue to focus on two ways to reduce waste in which it has an established interest: (i) ensuring that the design and interpretation of new research is informed by systematic reviews of existing evidence:^33,72–84 and (ii) working with others to expose and reduce the scandal of biased under-reporting of research,^{75,82,85–91} for example, through continued involvement in a campaign (alltrials.net) demanding that all clinical trials must be registered publicly at inception and reported publicly on completion.^92,93

In these various ways, the JLI will continue its mission to encourage acknowledgement of important uncertainties about the effects of treatments, and to promote better, more efficiently implemented research to address these.

DECLARATIONS

Competing interests

None declared

Funding

None declared

Guarantor

IC

Ethical approval

Not applicable

Acknowledgements

We are grateful to many individuals who have contributed to the progress we have described: to Joan Box (Medical Research Council) and John Pattison (Department of Health) for the key roles each played in shaping the early vision of the JLI; to Hazim Timimi, Mark Starr, Anne Brice, Jon Brassey and Muir Gray for their roles in the development of UK DUETs, and to Gill Leng, Alexia Tonnel and Keith Lloyd for mainstreaming the database within NICE; to John Scadding, Nick Partridge, Sandy Oliver, Sophie Petit-Zeman, Philippa Yeeles, Helen Hayes, Maryrose Tarpey and members of JLA Strategy and Development Group for developing the James Lind Alliance, and to Tom Walley, Lynne Kerridge and Pamela Young for supporting the JLA and fostering its incorporation by the NIHR Evaluation, Trials and Studies Coordinating Centre; to Imogen Evans, Hazel Thornton, Paul Glasziou, Douglas Badenoch and Amanda Burls for work on Testing Treatments and Testing Treatments interactive; and to the National Institute for Health Research and the Medical Research Council for supporting our work

Contributorship

All the co-authors contributed to the planning and preparation of the article

Provenance

Invited contribution from the James Lind Library