Dementia priority setting partnership with the James Lind Alliance: using patient and public involvement and the evidence base to inform the research agenda

Abstract

Background: the James Lind Alliance (JLA) created an approach to elicit the views of those under-represented in research priority exercises. Building on this, the JLA Dementia Priority Setting Partnership was set up as an independent and evidence-based project to identify and prioritise unanswered questions (‘uncertainties') about prevention, diagnosis, treatment and care relating to dementia.

Methods: a survey was widely disseminated to stakeholders with an interest in the needs of the older population. Thematic analysis was used to identify themes from the large amount of questions collected from which research questions were developed using PICO framework (Population, Intervention, Comparator, Outcome). Each question was checked against an extensive evidence base of high-quality systematic reviews to verify whether they were true uncertainties.

Findings: one thousand five hundred and sixty-three questionnaires were received, from people with dementia, carers/relatives, and health and care professionals; 85 uncertainties were identified from other sources. Questions were refined and formatted iteratively into 146 unique uncertainties. An interim prioritisation process involving diverse organisations identified the top 25 ranked questions. At a final face-to-face prioritisation workshop, 18 people representing the above constituencies arrived by consensus at the top 10 priority questions. The impact of patient and public involvement on the priorities is discussed.

Interpretation: the long (146 questions) and top 10 lists of dementia research priorities provide a focus for researchers, funders and commissioners. They highlight a need for more research into care for people with dementia and carers, and a need for high-quality effectiveness trials in all aspects of dementia research.

Background

The number of people living with dementia in the UK and worldwide is high, estimated at 800,000 people in the UK [1, 2] and 44 million people worldwide [3]. With an ageing population, these figures are predicted to increase in the UK to over a million people by 2021 [2] and worldwide to ∼135 million by 2050 [4], despite recent estimates that future prevalence and anticipated steep trajectory will be less profound [5].

The economic and societal impact of dementia places demands on health, social care and community services as well as family carers [1]. Recently, there has been acknowledgement of the need for governments to address dementia as a ‘public and health priority’ [6] and a call for national and international coordination [2, 7–10]. To maximise resources, there is a need to identify and prioritise unanswered questions (‘uncertainties’) about prevention, diagnosis, treatment and care relating to dementia from a wide societal perspective—one that integrates the views of all stakeholders [11].

The research agenda is often set by the research community or industry. However, this may not reflect the priorities of people with dementia or their carers' who live with the impact of dementia in their daily lives. Patient and public involvement (PPI) is increasingly being used to inform research and practice to ensure that views and outcomes relevant to service users and the public are included [12–16]. For dementia, inclusion of the views of people with dementia, their carers', clinicians, health and social care professionals and the wider public is needed.

The Alzheimer's Society Public Health Advisory Committee, with the University of Cambridge, initiated the Dementia Priority Setting Partnership (PSP) with the James Lind Alliance (JLA) to identify priorities for applied dementia research. The JLA is a not-for-profit organisation and is a part of the National Institute for Health Research (NIHR) Evaluation, Trials and Studies Coordinating Centre (NETSCC). JLA has been developing methods to identify and prioritise uncertainties since 2004 with a demonstrated impact on research calls (http://www.lindalliance.org/DementiaPSP.asp) [17–22].

The aim of JLA PSPs is to involve patients/service users, carers, and health and care professionals in identifying and prioritising the top 10 ‘uncertainties’ (unanswered questions) for research based on a rigorous, transparent and independent process. The objectives of the Dementia PSP were: (i) to identify unanswered questions around the prevention, treatment, diagnosis and care of dementia with the involvement of all stakeholders; (ii) to identify a top 10 prioritised list of uncertainties and (iii) to disseminate the results of the PSP to researchers, research funders, commissioners, stakeholders and the wider public.

Methods

The methods developed by the JLA for research priority setting and their methods for involving patients, carers, clinicians and other stakeholders are open and transparent, and the methodology followed was broadly that outlined in the JLA guidebook [23].

While the JLA provides guidelines and a framework for the methodology of PSPs, a pragmatic approach is taken so the process can be adapted to meet the differing requirements of PSP's. The methodology described below in particular focuses on those areas where the Dementia PSP may be differentiated from other PSPs. In particular, the Dementia PSP was underpinned by a public health and population perspective. A thematic analysis approach was developed to manage, refine and develop research questions from the very large amount of collected uncertainties. Moreover, each question was checked against an extensive and complex evidence base of systematic reviews, to verify whether they were true uncertainties.

The PSP process was conducted between April 2012 and June 2013. Ethical approval was not required as completion of the survey implied consent for submitted information to be included; however, principles of ethical research were followed such as confidentiality.

Management and scope

The Dementia PSP was guided and chaired by an independent representative of the JLA and overseen by a steering group (Supplementary data, Appendix, available in Age and Ageing online), which was actively engaged in the process. The steering group defined the scope of the project and agreed on a protocol—both were published on the JLA website at the start of the PSP process [24]. The scope included prevention, treatment, diagnosis and care of people with dementia; it did not include research on neurophysiological mechanisms or causes of dementia.

Involvement of potential partner organisations

Potential partner organisations were identified through the networks of the Alzheimer's Society and the steering group, ensuring representation from all stakeholders. Potential partners were invited to an initial awareness meeting to explain the project in April 2012. Those that signed up participated in the process by publicising the project, disseminating the survey, collecting completed surveys and prioritising questions for dementia research. Partner organisations and their involvement are shown in Supplementary data S1, available in Age and Ageing online.

Identifying uncertainties

A survey to collect uncertainties was designed using Survey Monkey (www.surveymonkey.com) (Supplementary data S2, available in Age and Ageing online). It included four questions, relating to prevention, diagnosis, treatment and care of dementia, with open text boxes and questions to collect demographic data about respondents. This meant there were no limits to the number, topic or format of questions that respondents could submit. Initial piloting was conducted among the Alzheimer's Society general practitioner steering group and Research Network volunteers, and issues raised were addressed in the design of the survey.

The survey was available on the Alzheimer's Society website from April to July 2012 and distributed through the Alzheimer's Society monthly magazine, local offices, Research Network volunteers, GP steering group, Facebook and Twitter pages and social media launches. It was also distributed through the PSP partner organisations (Supplementary data S1, available in Age and Ageing online), clinical, health and social care professional bodies, and internal and external contacts of the steering group (via meetings, conferences, emails, newsletters, bulletins, web and social media). Efforts were made in later dissemination rounds to increase responses from groups that proved to be more challenging to access and were initially under-represented (i.e. general practitioners, geriatricians and old age psychiatrists, people with dementia, people with no direct experience of dementia, and people from black and minority ethnic groups). The survey could be completed online or in paper format for reply to a Freepost address.

Uncertainties were also identified from existing research recommendations from Cochrane systematic reviews and NICE guidelines from 2005 onwards. Criteria for inclusion were that a specific new research question within the PSP scope was identified. Technical recommendations such as the need for better-conducted, larger scale trials, or different analysis, trial design or measurement methods were not included.

Additionally, uncertainties from a previous small-scale, in-house priority setting exercise conducted by the Alzheimer's Society's Research Network of volunteers were included (http://www.alzheimers.org.uk/site/scripts/documents_info.php?documentID=1109), and the UK Database of Uncertainties about the Effectiveness of Treatments was searched for previously registered uncertainties (http://www.library.nhs.uk/duets/, January 2013, date last accessed).

Question management and analysis

The raw submissions received ranged from one question to large tracts of questions and/or comments. Many of the entries into free text boxes contained multiple questions, and some single sentences contained multiple questions e.g. ‘Benefit (or otherwise) of different foods, or exercise and doing puzzles and mind exercises?’ Some submissions (or parts of submissions) were just comments with no underlying question, e.g. ‘had scans 7 years ago’, and these were excluded.

The ‘raw submissions’ were initially collated into an Excel spreadsheet and managed using NVivo (v9, QSR International). The data were categorised using themes identified in the raw data (predefined categories or ‘taxonomy’ were not used) [23] using a thematic analysis approach. This meant that there were no pre-conceptions about the question topics or type of information received, which allowed us to observe where questions were duplicated or similar questions were asked. All data were managed by one researcher to enable consistency in dealing with diverse but often interrelated questions. Ambiguities were discussed and agreed with the steering group.

The steering group reviewed the initial categorised questions and decided the following categories of question were out of scope: underlying cause of dementia (e.g. ‘what is the cause of dementia?’ or questions relating to underlying neurophysiological causes), genetics/inheritance (e.g. ‘if my parent had dementia am I likely to get it?’), media, stigma, policy, funding and training. These questions were communicated to the Alzheimer's Society for information and use in their programmes.

Questions that related to practical steps people could take to prevent or reduce risk of dementia such as diet, exercise or exposure to environmental toxins were included.

Through an extensive, iterative process, duplicate or similar questions were combined and where necessary (and possible) restructured according to the PICO (Population, Intervention, Comparator, Outcome) format. Questions that were not about interventions could not be fully converted into PICO format, but population and outcome were defined where available; nor were some questions about risk factors (e.g. ‘are viruses a risk factor for dementia?’) which remained in original format.

Verifying uncertainties

Each question was checked against existing systematic reviews to ensure it could not already be answered by up to date, high-quality, reliable evidence. Searching for systematic reviews was initially limited to the Cochrane Library and the databases of the York Centre for Reviews and Dissemination (York CRD; www.crd.york.ac.uk/crdweb) which include the Database of Abstracts of Reviews of Effects (DARE), NHS Economic Evaluation Database (NHS-EED), and Health Technology Assessment database, to meet the project timescale. These databases are sources of reliable systematic reviews which meet set methodological standards. Keywords used for the searches were ‘dementia’, ‘Alzheimer's', ‘cognition’ and ‘cognitive’ in the title and/or journal field. This identified 617 titles (March 2013) of which 185 were considered to be potentially relevant to the Dementia PSP questions and are shown in Supplementary data S3, available in Age and Ageing online.

To check questions about dementia risk factors, which are mainly addressed by reviews including prospective cohort studies, other systematic reviews were identified from a concurrent dementia mapping project. MEDLINE, EMBASE, CINAHL and PsycINFO were searched with the same keywords using a systematic review filter. A further 41 relevant reviews were identified. The combined total of 658 reviews was cross-checked against the Dementia PSP questions.

Questions were considered to be uncertainties if there was either: (i) no up-to-date, reliable systematic review, (ii) up-to-date systematic reviews of research evidence showed that there was not enough evidence available, (iii) the evidence available was not of sufficiently high quality to make robust conclusions about effectiveness or (iv) the available evidence (of any type) reported in a systematic review was inconclusive.

The data available from systematic reviews were diverse. Some systematic reviews included only randomised, controlled trials (RCTs), but some reviews included a range of study designs, such as a mixed methods approach or qualitative data. Few reviews had sufficient homogenous data to report meta-analysis. Consideration was given to: (i) study design of included studies in a systematic review, (ii) effect size and estimated precision where available, (iii) quality and consistency of the evidence and (iv) whether the review answered all or part of a question.

The GRADE system (www.gradeworkinggroup.org) for rating the quality of a body of evidence was used as a guideline wherever evidence was not clear. If a review only answered part of a question, only the ‘uncertain’ parts of the question were put through to the next stage of the process. Final decisions on included questions were taken by the steering group after consideration of the evidence and a clinical or professional perspective.

Interim prioritisation

An interim prioritisation stage was conducted initially among partner organisations to shorten the initial list of 146 questions to 25 questions to generate a manageable number for a final face-to-face prioritisation workshop. The questions were listed in a random order. Not all partner organisations took part, but care was taken to include a distribution of representatives (Supplementary data S1, available in Age and Ageing online). Each organisation was asked to rank their top 10 questions. They were free to generate their ranking according to the most appropriate method for their organisation, but were asked to report the process and number of people who participated [23]. Rankings from patients and carers representatives and health and social care professionals were combined to provide an overall interim ranking. The top 25 were presented to the final prioritisation workshop in random order (Supplementary data S5, available in Age and Ageing online).

Final prioritisation workshop

The aim of the workshop was to choose by consensus between representatives of patients, carers, clinicians and health and social care professionals, a final top 10 uncertainties from the list of 25 questions decided at interim prioritisation. This was held in London (June 2013). The method involved small-group sessions and ranking exercises based on nominal group techniques [23, 25]. The process encouraged open discussion and involvement of all group members guided by an independent JLA facilitator.

Results

The Dementia PSP received 1,563 completed surveys. Respondents are shown in Table 1. Of these, the ethnic composition was White 78.6%, Black or Asian 2%, other 0.2% and not stated 23.2%. As the survey covered four questions, this generated 4,116 ‘raw submissions’. Additionally, 21 research recommendations and 64 questions from the prior Research Network priority exercise were added. Figure 1 shows the flow of questions from the raw submissions to the final top 10 prioritised questions.

Table 1.

Dementia PSP survey respondents

Survey respondents	% (n)
People with dementia	4.1 (64)
Family carer/relative	76.0 (1,188)
Health and care professional (+retired)	14.4 (224)
Carer	7.2 (16)
Geriatrician	2.9 (6)
GP	2.4 (5)
Nurse	27.0 (61)
Old age psychiatrist	6.3 (14)
Physiotherapist	1.9 (4)
Occupational Therapist	8.7 (20)
Social worker	3.4 (8)
Other	28.5 (64)
Not answered	11.7 (26)
No direct experience	1.4 (22)
Other	2.9 (46)
Not answered	1.2 (19)

Figure 1.

Flow chart of Dementia PSP questions.

First round of combining and reformatting

The raw submissions were initially cleaned, combined and reformatted into a total of 1,106 questions (300 questions about prevention, 325 about treatment, 258 about care and 223 about diagnosis).

Second round of combining and reformatting

After further combining and reformatting, including merging similar questions across categories, 433 questions remained. Many of the questions that were submitted had been asked about by tens, or in some cases, hundreds of people. Questions that only one or two people asked were excluded (n = 149), after circulating to helpline staff at the Alzheimer's Society to identify any questions that were common queries. Four questions were identified as regular queries and retained (n = 4). Questions were organised across public health categories of primary prevention (n = 60), secondary and tertiary prevention (combined, as there were only a few questions about tertiary prevention) (n = 175), and questions about diagnosis (n = 49).

At this stage, a total of 284 questions were checked against systematic reviews. Twelve questions were considered to be ‘known, answerable’ questions by the steering group and were removed prior to the interim prioritisation stage. Three questions were considered to be requests for information rather than research questions. Excluded questions were removed and passed to the Alzheimer's Society for information.

Third round of combining and reformatting

There were too many remaining (n = 272) questions at this stage for interim prioritisation, based on the experience of the JLA with other PSPs (for consideration by groups representing patients, carers, and health and social care groups), so further merging and combining were conducted to reach 150 questions. The questions were checked by the steering group for clarity. Four questions were further merged, combined or reworded.

When questions were combined, as much detail as possible was included into the combined question. (For example: supplementary data S4, available in Age and Ageing online, Question 100, ‘Can the onset of dementia be prevented or delayed by dietary or nutritional factors?’—the question was asked by many people about different specific foods, diets, dietary patterns, components of diet and in different populations. So these factors were all included in the question that went forward for prioritisation).

Interim prioritisation

A list of 146 questions was sent to partner organisations in April 2013. Sixty-one organisations participated representing the views of people with dementia and their carers, and health and social care practitioners (Supplementary data S1, available in Age and Ageing online). The methods used by different organisations to rank the questions varied. Some organisations consulted with their members at meetings and conferences, and some used the collective decision of a group such as senior managers, but most were completed by an individual representing the organisation.

Final prioritisation workshop

Eighteen participants took part in the workshop: six were clinicians, five had a nursing and/or nursing management and/or professional care background, five people were family carers or former carers of people with dementia, and two were people with dementia. The list of 25 questions was sent out to participants before the meeting and participants were asked to initially rank questions before attending the workshop. Each participant was given the opportunity to express their views on which questions they thought should be prioritised and to hear other people's perspectives. Participants were free to debate and change their minds throughout the process.

During the final prioritisation workshop, participants considered some of the questions to be similar or related, and 7 of the 25 questions were further merged together by workshop participants. The research team observed that participants were keen to include as many topics as possible within the top 10 priorities.

Final top 10 priorities

Table 2 shows the final top 10 agreed priorities. It is notable that nine of these final questions were directly or indirectly about care of people with dementia and their carers; one was about diagnosis.

Table 2.

Dementia PSP top 10 priorities

1.	What are the most effective components of care that keep a person with dementia as independent as they can be at all stages of the disease in all care settings?
2.	How can the best ways to care for people with dementia, including results from research findings, be effectively disseminated and implemented into care practice?
3.	What is the impact of an early diagnosis of dementia and how can primary care support a more effective route to diagnosis?
4.	What non-pharmacological and/or pharmacological (drug) interventions are most effective for managing challenging behaviour in people with dementia?
5.	What is the best way to care for people with dementia in a hospital setting when they have acute healthcare needs?
6.	What are the most effective ways to encourage people with dementia to eat, drink and maintain nutritional intake?
7.	What are the most effective ways of supporting carers of people with dementia living at home?
8.	What is the best way to care for people with advanced dementia (with or without other illnesses) at the end of life?
9.	When is the optimal time to move a person with dementia into a care home setting and how can the standard of care be improved?
10.	What are the most effective design features for producing dementia friendly environments at both the housing and neighbourhood levels?

Impact of PPI

Of the 146 questions for interim prioritisation, most (n = 112) were asked by both healthcare professionals and people with dementia or their carers; 30 were asked only by people with dementia or their carers, and only one (drug-related) question was asked by healthcare professionals alone. No questions were asked solely by people with dementia. The questions submitted by people with dementia were also asked by other respondents. Of the 25 questions that went forward to the final prioritisation workshop, only one question was asked only by people with dementia or their carers. At the final prioritisation workshop that question was rated second in the top 10 priorities.

Of the top 10 questions voted for at the interim prioritisation stage by people with dementia and carers, 6 were included in the final top 10 priorities. Of these, three were related questions about care which were merged into one broad question; two were questions about non-pharmacological interventions which were also merged; and one question was about early diagnosis—ultimately yielding three questions in the final top 10. Of the top 10 questions voted for at the interim prioritisation stage by healthcare professionals, 7 were included in the top 10 priorities. Two questions about non-pharmacological interventions were merged, while ultimately 6 of the questions reached the final top 10. Overall, both groups were fairly well represented in the top 10 priorities.

Discussion

The Dementia PSP was conducted according to an extensive, independent and transparent process to identify and prioritise questions for research. To our knowledge, these are the first priorities for dementia in which patients, carers, health and care professionals, and the wider public have been involved at all stages of the identification and prioritisation process. One report that used a limited list of 15 researcher-defined questions to consult with people with dementia, carers and clinicians has been published [26].

While the broad areas of prevention, diagnosis, treatment and care of people with dementia were well represented in the ‘long list’ of 146 questions, only one question about prevention of dementia was ranked in the top 25 questions and none was voted into the top 10 priorities. The majority of questions that were voted through the interim and final stages related to issues around care and support of people with dementia and their carers. This may be a reflection that many of the responding organisations are currently involved in the daily care of people with dementia or their carers', or that these issues have not been adequately addressed in research to date. Also, as those who took part in the prioritisation stages were representatives of people with dementia, carers or established health and care professionals, there was little representation from younger people who may have more interest in longer term questions about the prevention of dementia. However, the ‘long list’ of 146 questions was identified through wide consultation with an extensive range of stakeholders in dementia and as such are a reflection of the concerns of people with dementia, their carers', clinicians and health and social care professionals.

All the questions will be available to others after being entered onto the UK Database of Uncertainties about the Effectiveness of Treatments (http://www.library.nhs.uk/duets) if in a suitable PICO format, or published on the Alzheimer's Society website. The full ‘long list’ of 146 questions used for prioritisation and the 25 top ranked questions considered at the final workshop are available as Supplementary data S4 and S5, available in Age and Ageing online. The questions and the details behind them have been submitted to NIHR and other funders for consideration in their funding programmes, and to date a number of NIHR funding calls have now been based around the questions identified in this process.

Until more effective treatments are found for dementia, the Dementia PSP priorities highlight that the issue of care for people with dementia, and how to enable people to live well with dementia, is also a research priority. Indeed, the G8 Dementia Summit and the ADI 2013 report have taken up the theme that in global societies dementia research efforts should include a focus on improving care and quality of life for people with dementia as well as on the search for disease-modifying therapies [4, 9].

Despite having scrutinised over 600 systematic reviews for this Dementia PSP, most of the questions that were verified against the evidence were still considered to be uncertainties. This was because there was often no available systematic review evidence, or where there was a systematic review, it identified a lack of relevant studies to answer the question. For some questions, there was evidence but the evidence available was inconclusive or not of sufficiently high quality to make robust conclusions. Responses to the identification of these uncertainties can include the belief that specific questions have been addressed, but the question here is about whether that evidence survives scrutiny on its strength. Thus, some questions may have been addressed by individual trials or study designs that were not included in a systematic review. However, individual trials may report differing results and vary in study quality, so systematic review data provide a consistent body of evidence that has also independently assessed the quality and design of any included trials. Few systematic reviews reported on the perspectives of people with dementia themselves and there is a need for more work in this area.

Overall, the Dementia PSP has been an extensive and wide-ranging process and has brought together a wide range of stakeholders in dementia to prioritise questions for research. PPI has provided a broader perspective to the research agenda and in particular highlighted the need for research into care and organisation of care and delivery of services for people with dementia.

As far as possible, we have reported the process and impact of public engagement according to the GRIPP checklist for PPI involvement [27]. While the questions, interim and final prioritisation process may reflect concerns and issues of importance to UK organisations and individuals, many of the questions, particularly in the ‘long list’ of 146 questions, are likely to be applicable in the international coordination of dementia research, as there were no country restrictions on the systematic review evidence base.

Issues for consideration by researchers, research funders and commissioners

The Dementia PSP priorities provide a focus for dementia research across the prevention, treatment, diagnosis and care of dementia. More attention needs to be put into well-designed studies, including RCTs to demonstrate effectiveness that address the range of populations, interventions and outcomes identified, including the perspectives and outcomes that are of importance to people with dementia and carers; such as quality of life, independence, management of behaviour, effect on progression of disease, effect of delaying moves to long-term care, effect on carers; and research relevant to all sub-types of dementia and in populations at higher risk of dementia, such as those with a family history or genetic risk factors for dementia.

Key points.

• Incorporating PPI, including the views of people with dementia, their carers' and other stakeholders, in research is important to ensure that issues of importance to them are included in the research agenda.

• The Dementia Priority Setting Partnership involved people with dementia, their carers', clinicians and other health and social care professionals in identifying, prioritising and disseminating unanswered research questions relating to dementia prevention, diagnosis, treatment and care.

• The top 10 priorities for dementia research identified by the process are reported, and the full list of questions identified is also available.

• The Dementia PSP has highlighted issues around care, organisation of care and delivery of services for people with dementia; a need for evidence from well-designed studies, relevant to all sub-types of dementia, including in populations at higher risk of dementia and inclusion of the perspectives and outcomes of importance to people with dementia themselves, and their carers'.

Conflicts of interest

None declared.

Funding

This work was funded by the Alzheimer's Society UK, with a grant to Professor Brayne and supported by the National Institute for Health Research Collaborations for Leadership in Applied Health Research and Care (CLAHRC) for Cambridgeshire and Peterborough. The project was coordinated by the Alzheimer's Society (by NH – research project officer); however, analysis and formatting of the questions, interpretation of the data and drafting of the paper were conducted independently by the University of Cambridge.

Supplementary data

Supplementary data mentioned in the text are available to subscribers in Age and Ageing online.