Skip to main content
Sage Choice logoLink to Sage Choice
. 2015 Sep 9;30(3):279–295. doi: 10.1177/0269216315605447

Development of a model for integrated care at the end of life in advanced dementia: A whole systems UK-wide approach

Louise Jones 1,, Bridget Candy 1, Sarah Davis 1, Margaret Elliott 1, Anna Gola 1, Jane Harrington 1, Nuriye Kupeli 1, Kathryn Lord 1, Kirsten Moore 1, Sharon Scott 1,2, Victoria Vickerstaff 1, Rumana Z Omar 3, Michael King 4, Gerard Leavey 5, Irwin Nazareth 6, Elizabeth L Sampson 1,7
Editors: Jeroen Hasselaar, Sheila Payne
PMCID: PMC4766969  PMID: 26354388

Abstract

Background:

The prevalence of dementia is rising worldwide and many people will die with the disease. Symptoms towards the end of life may be inadequately managed and informal and professional carers poorly supported. There are few evidence-based interventions to improve end-of-life care in advanced dementia.

Aim:

To develop an integrated, whole systems, evidence-based intervention that is pragmatic and feasible to improve end-of-life care for people with advanced dementia and support those close to them.

Design:

A realist-based approach in which qualitative and quantitative data assisted the development of statements. These were incorporated into the RAND/UCLA appropriateness method to achieve consensus on intervention components. Components were mapped to underlying theory of whole systems change and the intervention described in a detailed manual.

Setting/participants:

Data were collected from people with dementia, carers and health and social care professionals in England, from expert opinion and existing literature. Professional stakeholders in all four countries of the United Kingdom contributed to the RAND/UCLA appropriateness method process.

Results:

A total of 29 statements were agreed and mapped to individual, group, organisational and economic/political levels of healthcare systems. The resulting main intervention components are as follows: (1) influencing local service organisation through facilitation of integrated multi-disciplinary care, (2) providing training and support for formal and informal carers and (3) influencing local healthcare commissioning and priorities of service providers.

Conclusion:

Use of in-depth data, consensus methods and theoretical understanding of the intervention components produced an evidence-based intervention for further testing in end-of-life care in advanced dementia.

Keywords: Advanced dementia, end-of-life care, integrated care, complex interventions


What is already known about this topic?

  • The number of people worldwide dying with and from dementia is increasing and many have similar needs to those dying from cancer or major organ failure.

  • Care towards the end of life for people with advanced dementia is fragmented and many may die with unrecognised and untreated symptoms, in particular pain.

  • There are few evidence-based interventions available to improve care.

What this paper adds?

  • An in-depth approach to developing a complex healthcare intervention in advanced dementia as death approaches (the COMPASSION intervention).

  • The feasibility of undertaking systematic engagement with experts across the United Kingdom to enable wide consensus on intervention content and take account of context and new ideas for change.

  • An example of how theories underlying the professional behaviours required to implement the intervention and deliver an integrated approach to care can be explored.

Implications for practice, theory or policy

  • The COMPASSION intervention may facilitate integrated care at the end of life in advanced dementia; it includes training and support to enable this to occur.

  • COMPASSION has the potential to inform commissioners on processes, mechanisms and potential costs of an integrated multi-disciplinary model for end-of-life care in advanced dementia.

  • Use of rigorous methods, including theoretical perspectives, in developing this intervention may increase the likelihood of its feasibility and acceptability in practice.

Introduction

Dementia affects approximately 835,000 people in the United Kingdom and trends suggest that 70 million people will be living with the condition worldwide by 2020.1 Dementia typically involves a gradual, unpredictable decline in cognition, behaviour and function and may be considered a terminal neurodegenerative disease.2 It is estimated that one-third of people aged over 65 years in the United Kingdom will die with some form of dementia3 and 70% with advanced dementia reside in care homes.1

End-of-life care in dementia

Symptom management near the end of life in dementia is often sub-optimal, although symptom burden may be similar to cancer, chronic obstructive pulmonary disease (COPD) or advanced heart failure.47 In frail older people, dementia may exist with several co-morbid conditions, and death may occur from various causes. In the advanced stages, survival is unpredictable8 and death may be due to dementia or another inter-current illness. People may show challenging behavioural and psychological symptoms, and while these may lessen as dementia progresses, care requires skill, compassion, attention to detail and patience. Good person-centred care9,10 using simple measures may help. Quality standards for dementia and end-of-life care have been published in the United Kingdom;1114 a systematic review of international guidelines found another four documents (from United States, Australia, Singapore and Malaysia) that considered palliative care,15 and consensus from 23 countries and the European Association for Palliative Care16 highlights symptom control, comfort measures, communication, shared decision making, including families with care planning, timely recognition of dying, attention to continuity of care, and psychological and spiritual support.

Care homes

In England, there are 32,937 care homes currently registered with the Care Quality Commission, most privately owned. General practitioners have responsibility for the clinical care of residents but staffing in care homes varies, often with a few trained nurses supported by a larger pool of healthcare assistants. Use of the UK Department of Health Gold Standards Framework17 may enhance staff skills, although implementation and outcomes vary.1820 Relationships with external services such as community geriatricians and mental health and palliative care teams differ and depend on local commissioning priorities, interactions with care home owners and managers, and local clinicians’ levels of interest, knowledge and confidence in dementia care.2123

A feasible, context appropriate and sustainable intervention for end-of-life care in dementia is needed. Here, we describe how, in an iterative process, we used empirical data from our 3-year programme of research on end-of-life care in dementia24 and expert opinion to develop the COMPASSION intervention. We achieved consensus on core components using the RAND/UCLA appropriateness method (RAM)25 and mapped the components to theories of change in complex healthcare systems. Our methods reflect the importance of theoretical understandings and an iterative approach in devising an intervention to improve dementia care.26

Aim

We used a realist framework27 and rigorous processes, guided by the literature on development and implementation of complex healthcare interventions for whole systems change, to develop an intervention to enhance multi-professional detailed management of care home residents with advanced dementia who are approaching death and facilitate integrated care.

Methods

Theoretical basis of intervention development

We define a theory as ‘a system of ideas or statements held as an explanation or account of a group of facts or phenomena’.28 Our work was informed by Medical Research Council recommendations29 on the development and testing of complex interventions, and the MORECare guidance statements that were published to enhance methodological rigour in palliative care research.30 To develop an effective but feasible intervention, we were careful to include preliminary evidence both from empirical work and expert opinion and to consider theoretical principles underlying whole systems healthcare change.

Realist methods

Our 3-year research programme was based on a realist framework.24,27 This approach highlights barriers to good care but also identifies mechanisms that may improve care; it is consultative and iterative, taking into account local and contextual factors. We collected qualitative and quantitative data from multiple sources combining findings with evidence from reviews of the international literature and policy documents across the four countries of the United Kingdom. The generation of new ideas was enabled by an interactive process in workshops and interviews during which both researchers and participants contribute27 (see Figure 1). A final realist programme theory based on all elements of our wider research programme (National Institute for Health Research (NIHR) reference CRN-PCRN 12621; 12623) is in preparation.

Figure 1.

Figure 1.

Process for the development of the COMPASSION model of care for people with advanced dementia at the end of life and those close to them.

Theories affecting complex healthcare systems

We considered the general impact and process theories31,32 that might underlie potential components of our intervention to understand how these might operate in practice; these are described in Box 1. This provided a framework for our integrated care model.31

Box 1.

Description of theories underpinning complex healthcare interventions.

Impact theories31,32 Sub-theories Potential enablers and barriers of change
Individual (I) (patient, informal carer, health or social care professional) Cognitive, emotional and motivational factors Responses to a new model of care, a new way of thinking or new training may vary according to how a person currently thinks, their educational attainments or personal motivation.
Group or team including social interaction (G) Communication, leadership, professional development, team effectiveness and social learning Individual professionals are embedded in teams with members of varying seniority from within and across disciplines. How they respond to innovations and their motivation to accept change are affected by:
Attitudes, perceptions and expectations of all members of the team networks
Local social norms and personal autonomy within team structuresPersonal respect across peers and local opinion leaders.
Organisational context (O) Innovative organisations, organisational culture, organisational learning and knowledge management, total quality management, integrated care and complexity theory Teams are embedded in health and social care organisations. Organisations that are innovative:
Include specialised individual roles so decision making is decentralised and informed by knowledge
Encourage good communication both internally and externally, and receptiveness to innovative practice among leaders and managers
Keep the needs of patients and carers at the centre of care
Enhance quality by understanding that performance depends upon the whole system and attention to all voices within and across teams including the views of workers in the front line of care delivery.
To deliver integrated approaches to care, processes are needed to:
Promote new collaborations across established boundaries bringing in members of new disciplines
Allow different ways of allocating tasks, time management, new guidelines and referral pathways.
The interdependency of teams across boundaries reflects the complexity of whole systems and may be more important than the actions of individuals who may act independently or unpredictably depending upon context.
Political and economic context (P) Reimbursement contracting In healthcare systems, what an organisation can deliver is dependent upon local population needs, which services are prioritised within the local constraints on financial reimbursement and contracting of services.
Organisational plan: What is needed to plan the intervention to maximise effectiveness Linked to orientation (awareness, interest, involvement) and insight (understanding, insight into own routines) early phases of implementation (Grol et al.31)
Utilisation plan: How those required to accept and use the new practice can be influenced and might use the new practice Early involvement of those required to accept and use the new practice before manual is finalised. Linked to acceptance (positive attitude, decision to change), change (actual adoption, try-out, confirmation of value) and maintenance (new practice integrated into routines) phases of implementation (Grol et al.31)

Early evidence synthesis and derivation of statements on intervention components

We combined information from the following sources:24 (1) our rapid review of the literature and policy documents, (2) research team field observations, (3) advice from the programme expert steering group, (4) emerging results from our cohort study, (5) preliminary workshops and interactive qualitative interviews with professional and informal carers and (6) a workshop with people with early dementia. We used these data to generate case vignettes to illustrate emergent findings. Our team consisted of international experts in palliative care, dementia and complex intervention theory, and field researchers collecting data directly from care home residents.24 The expert steering group, including lay representation and national experts in health and social care, provided detailed feedback as our intervention developed.

Nominal groups

Within our realist framework, solutions were generated at a series of meetings attended by the whole research team at which we used our case vignettes as a basis for discussion. Using flip charts and field note recordings, possible solutions were noted and grouped for potential impact according to the four levels (individual, group, organisational or political/economic) at which system change operates (Box 1).31,32 This included consideration of the mechanisms required to implement the solution. Using nominal group techniques, in a three-round iterative process, solutions were ranked according to their feasibility within existing dementia care provision in the United Kingdom. The group considered how the solution might be operationalised and what staff or resources and which key decision makers might be required. This process generated a set of statements for consideration in a national consensus process.

Achieving national consensus on intervention content

We used the RAM25 to achieve consensus on which statements derived from our evidence synthesis should form the components of our intervention. The RAM process took place over three rounds (two online and one face-to-face in workshops) in which participants considered the statements and ranked them for appropriateness25 and necessity,25 irrespective of potential costs.

RAM workshops

To take account of context, RAM workshops were held in each of the four countries of the United Kingdom; two in England (London and Birmingham), and one each in Wales, Scotland and Northern Ireland. Acting as a local contact point, facilitators in Marie Curie hospices in Birmingham, Edinburgh, Belfast and Cardiff identified potential participants. Each liaised with a number of diverse agencies with an interest in people with advanced dementia and palliative care within their wider local health and social care system to include professionals from a range of disciplines and levels. In London, we accessed a network of contacts across 11 care homes. We aimed to include 10 locally based health and social care providers for people with dementia in each workshop; to achieve this, the research team circulated up to 30 invitations at each site. Interested individuals were sent an invitation and study information sheet to consider participation (via post or email). If they wished to take part, they were asked to sign and return a consent form either beforehand or on the day of the workshop. Each workshop lasted 2 h and was co-facilitated by two members of the research team (K.L., J.H. – London, Birmingham, Cardiff; S.S., S.D. – Edinburgh and K.L., S.S. – Belfast).

RAM process

The concept of appropriateness refers to the relative weight of the benefits and harms of a healthcare intervention. An appropriate component is one in which the expected health benefit exceeds the expected negative consequences by a sufficiently wide margin that the intervention is worth doing, exclusive of cost.25 Participants rate the benefit–harm ratio of each statement on a scale of 1–9, where 1 means the expected harms greatly outweigh the expected benefits and 9 that the expected benefits greatly outweigh the expected harms. A middle rating of 5 can mean either that the harms and benefits are about equal or that the rater is unable to decide.

The concept of necessity refers to opportunities for interventions that must be offered to patients fitting a particular clinical description. The RAM definition of necessity is that

the intervention is appropriate, i.e. the benefits exceed the risks by a sufficient margin to make it worthwhile; it would be improper care not to offer the intervention; there is a reasonable chance of benefit; the magnitude of the expected benefit is not small.25

Ratings are also made on a scale of 1 unnecessary to 9 necessary.

  • Round 1 responses completed online prior to the RAM workshops, rating for appropriateness. Invitees were emailed an online survey including the set of statements and asked to complete their ratings 2 weeks prior to each workshop. Participants rated each statement from 1 to 9 for appropriateness. The research team collated responses and ranked them as (1) ‘Appropriate’: ALL responses from the group rate that statement as 7–9, (2) ‘Uncertain’: At least ONE individual rated that statement 1–6 and (3) ‘Inappropriate’: At least TWO individuals rated that statement 1–3.

  • Round 2 responses completed in person at the end of the RAM workshops, rating for appropriateness. The local workshop was then held and all statements rated by the group as either ‘uncertain’ or ‘inappropriate’ were discussed. After the group discussion, all participants again rated all statements for appropriateness. They were instructed that if an individual felt their response remained the same they were under no obligation to change it. The research team collated responses as ranked appropriate, uncertain or inappropriate by the group.

  • Round 3 responses completed online 2 weeks after the RAM workshops, rating for necessity. Within 2 weeks of each workshop, participants completed a second online survey consisting of only those statements that had been ranked by that workshop as appropriate. Participants rated each of these statements for necessity. Responses were collated and ranked by the research team as either (1) ‘Necessary’: ALL responses (or all except one) from the group rated that statement as either 7–9, (2) ‘Uncertain’: At least TWO individuals rated that statement 1–6 and (3) ‘Unnecessary’: At least TWO individuals rated that statement 1–3.

Collating of results

Upon completion of Round 3, results from the five RAM process workshops were collated, and those statements rated as necessary by attendees at three or more of the workshops were retained.

Mapping to theory

Three senior members of the research team (L.J., E.L.S. and B.C.) not actively involved in data collection independently considered each statement retained after the RAM process and assessed (1) which of the four theoretical levels of impact (individual, social interaction, organisational context, political and economic context) were likely to be targeted by the statement and (2) which sub-theories might be operating (Box 1). Results were compared for consistency and statements on which two or more of the raters agreed were retained. Statements were then reviewed for overlap by the research team and combined to simplify language and clarify their operational aims.

Preparation of a written manual

We prepared a written manual describing in detail the content of the intervention and the steps needed to put it into practice, to maximise the potential for transferability and replication.33

Checking prior to implementation – orientation and insight phase

We developed a protocol for an exploratory naturalistic implementation study.34 We began implementation with a local focus group including clinical leads for care of the elderly, palliative care, old age psychiatry, social care and care home managerial staff. We discussed the content of the manual and adjustments were made as needed. This approach addressed process theories35 of how the intervention would be put into practice (Box 1) and forms part of the orientation and insight phases required for successful implementation.31

Results

First draft of ‘COMPASSION’ intervention components

Table 1 shows our early synthesis of existing evidence with data emerging from our wider research programme. After discussion in our nominal groups, we drafted 49 statements for consideration in the RAM process (Table 2).

Table 1.

Synthesis of evidence from wider programme of research NIHR portfolio reference CRN-PCRN 12621; 12623.

Levels at which intervention must operate Implications derived from rapid literature reviewa Evidence emerging from quantitative cohort study Emerging qualitative evidence: preliminary workshops and interactive interviews (healthcare professional, carers, people with early dementia) Expert steering group and research team experiences and expert knowledge leading to first draft of suggestions for Whole Systems Intervention component statements
Individual: person with dementia Key elements of care:comfort, pain, feeding, care planningOutcomes: patient centred rather than system-level dataPlace of death may not indicate quality of deathIncreased DNR orders/advance care planning discussions may not reflect improved experience Mean age 85 years (73% female and 27% male);
43% at fast stage 7d; 73% at 7d or above
Clinical problems:
Pain
pressure sores
Agitation
Swallowing
Weight loss
Lack of training/training for staff on very end of life and care after death
Learning from hospice model
Commissioning to agreed quality standards to include
Assessment of clinical status and outcomes
Use of advance care plans and preferred place of care records to assess whether patient and carer centred preferences achieved
Individual: family carers Carers need support with:
Anticipatory grief
Decision making
Mostly adult female children and spouses
Clinically significant anxiety and depression
Training needed for staff and carers on:
Difficult conversations
Care planning discussions
Highlight good practice (‘beacon’ type services)
Attention to comfort and existential needs of carers as patient deteriorates and in bereavement
Carer education and support borrowing from existing specialist models:
Communication
Natural history of dementia.
Referral pathways for carers if anxious and depressed.
Personal and culturally sensitive attitudes to death and bereavement in place
Group/team Address:
Knowledge of when a person is entering the dying phase
Focus on all aspects of clinical care
Advance care planning
Adequate expertise and skill mix for team to work with carers
Structures for on-going support of care home staff especially those who experience loss when long-term residents die
Fragmented care-little collaborative working across disciplines and care settings
Clinical staffing:
More trained nurses in care homes
Consider a medical model like hospice care/The Netherlands
Bereavement support/post death analysis and de-brief for all
Multi-disciplinary working across boundaries to include:
Geriatricians, mental health, palliative care and community specialists, primary care and district nurses, social in reach to care home
All care providers including hospitals, hospices, care homes and community.
New working patterns, for example, virtual rounds teleconferencing and case-based approach
Agree:
Assessment processes and triggers for action
Prescribing patterns for symptom control, management of delirium, behavioural disorders and the terminal phase
Strategies to manage personal, clinical and comfort care
Staffing levels: more trained staff who can administer drugs, make clinical decisions
Education to maximise clinical skills
Attention to comfort and existential needs of staff as patient deteriorates and in bereavement
Organisational A multi-component holistic intervention is needed:
Burdensome care transitions are common
Aspects of hospice care could be feasibly transferred
Care pathway data to follow in further data analysis Care home culture:
Fear of deaths in care home
Low pay
Low staff morale
High staff turnover
Consider:
Collaborative working between teams
Rotating staff across environments to bring new learning
Out of hours care from GP’s who know people with dementia
Outcomes reflect requirements of different audiences – commissioners, providers, service planners and voluntary/statutory sectors
Commission:
multi-disciplinary staff to work in care homes
mechanisms to join up care
Support to care homes from multi-disciplinary team:
To manage risk and increase willingness for deaths to occur on premises
For front line staff with follow up monthly deaths/discharge/case review meetings on a per patient basis
Political and economic Evidence on systems level change scanty
No economic data in evaluations
The structure of health and social care systems strongly influences the type of care given
Mortality and place of death data being collected On-going changes in commissioning for healthcare are disruptive and limit service planning
Local priorities for healthcare spending vary
Provision of social care rarely linked to healthcare needs
Multi-disciplinary team working for end-of-life care dementia should be part of commissioning process
Link to quality standards for dementia care and end-of-life care agreed nationally
Work within existing structures and avoid costly new approaches
Consider costs of:
Core components
Additional ‘bolt-on’ components
Variations between locality and context

NIHR: National Institute for Health Research; DNR: do-not-resuscitate; GP: general practitioner.

a

Report detailing the rapid review is available from the authors.

Table 2.

Statements considered in RAM process (N = 49).

Number Statements considered in RAM workshops Retained after RAM process Level operating (I, G, O, P) Sub-theories likely to be active
1 Establish a multi-disciplinary team of health and social care workers within one commissioning area No
2 There should be a multi-disciplinary team composed of geriatricians, mental health, palliative care, community care specialists, GP’s and district nurses Yes O Integrated care
Organisational learning and knowledge management
Innovative organisations
Complexity theory
G Communication
Team effectiveness
Social networks and influencing
Opinion leaders
Leadership
Social learning
Professional development
3 Regular review by members of the multi-disciplinary team of the physical health and social care needs of the person with advanced dementia No
4 Regular review by a member of the multi-disciplinary team of the social care needs of the person with advanced dementia Yes P Reimbursement
Contracting
I Cognitive
Motivational
5 The multi-disciplinary team should include statutory social care representation responsible for assessment and monitoring No
6 Regular reviews of health and social care by a member of the multi-disciplinary team going into the usual place of care of the person with advanced dementia Yes P Reimbursement
Contracting
O Total quality management
Innovative organisations
Organisational culture
Organisational learning and knowledge management
7 All multi-disciplinary team members (particularly care home staff) are trained to communicate and connect with person with dementia using simple methods, for example, tone of voice, touch Yes G Communication
Team effectiveness
Professional development
Social learning
I Cognitive
Educational
Motivational
8 On-going training for all members of the multi-disciplinary team, particularly care home staff, to begin difficult conversations with the family/friend carer of the person with advanced dementia to understand their wishes Yes G Communication
Team effectiveness
Professional development
Social learning
I Cognitive
Educational
Motivational
9 On-going training for all members of the multi-disciplinary team to improve understanding of what is meant by an advance care plan, and how such a plan might be worked out and used in practice Yes I Cognitive
Educational
Motivational
10 On-going training for all members of the multi-disciplinary team to recognise the needs of family/friend carers particularly being alert to anxiety and depression Yes I Cognitive
Educational
Motivational
11 Development and agreement across the multi-disciplinary team of clear referral pathways for family/friend carers who are identified as being in need No
12 It should be ensured that there is always someone on duty who can recognise the clinical needs of the person with advanced dementia and is qualified to respond, for example, administering pain relief Yes O Innovative organisations
Total quality management
I Educational
Cognitive
Motivational
13 The levels of staff available in care homes should be sufficient to allow appropriate clinical and personal care to be conducted for all people with dementia Yes P Reimbursement
Contracting
O Total quality management
Innovative organisations
Organisational culture
Organisational learning and knowledge management
14 Staff ratios and skill mix in care homes should be supported by the Care Quality Commission Yes P ReimbursementContracting
15 Prolonged shift working patterns (e.g. 12 h working days) in care homes should be discouraged No
16 On-going training and support for care home staff to ensure that they have the clinical skills to optimise the management of pain, acute medical events, behavioural disorders (e.g. agitation and delirium), feeding decisions, pressure sores and comfort measures for people with advanced dementia Yes O Innovative organisations
Total quality management
Integrated care
G Professional development
Communication
Social learning
Team effectiveness
I Cognitive
Educational
Motivational
17 Training in the natural history of dementia as a disease in its own right Yes O Innovative organisations
Total quality management
I Educational
Cognitive
Motivational
18 Training in the diagnosis of dying Yes I Cognitive
Educational
Motivational
19 Training in the use of clinical care pathways Yes G Communication
Team effectiveness
Professional development
Social learning
I Cognitive
Educational
Motivational
20 Support from the multi-disciplinary team to front line staff and managers in care homes to manage risk in people with dementia and avoid unnecessary place of care transfers, for example, to the acute hospital towards the end of life Yes O Integrated care
Organisational learning and knowledge management
Innovative organisations
Complexity theory
G Communication
Team effectiveness
Social networks and influencing
Opinion leaders
Leadership
Social learning
Professional development
21 Support for staff in managing their own losses and grief when people with dementia deteriorate and die No
22 Training on ethical and cultural issues in death and bereavement Yes G Communication
Team effectiveness
Professional development
Social learning
I Cognitive
Educational
Motivational
23 Set up simple mechanisms for the multi-disciplinary team to discuss cases – for example, teleconferencing No
24 Set up virtual wards in healthcare localities to provide complex care for people in the community, their own homes or care homes No
25 Monthly debriefing and discussion of difficult cases within the multi-disciplinary team shared with care home staff and managers No
26 Multi-disciplinary team facilitated by a named co-ordinator who would be a clinical nurse specialist trained in palliative care and dementia care No
27 A social care representative present at all multi-disciplinary team discussions No
28 There should be a commissioned person to assess and respond to carer need (borrowing from Admiral nurse and Alzheimer’s Society dementia care advisor models for signposting) No
29 Improved cross disciplinary and cross boundary communication, including out of hours support for people with dementia, their family/friend carers and professional carers in care homes Yes P Reimbursement
Contracting
O Integrated care
Innovative organisations
Total quality management
Organisational learning and knowledge management
G Communication
Leadership
Professional development
30 Use of volunteers to support people with dementia and their family/friend carers in their usual place of residence No
31 The multi-disciplinary team would agree processes for diagnostic assessment of people with advanced dementia within all settings in that locality No
32 The multi-disciplinary team would agree processes for guidelines for referrals for care from specialist or generalist clinical or social care services Yes O Integrated care
Organisational learning and knowledge management
Innovative organisations
Complexity theory
G Communication
Team effectiveness
Social networks and influencing
Opinion leaders
Leadership
Social learning
Professional development
33 The multi-disciplinary team would agree processes for the provision of holistic care Yes O Integrated care
Organisational learning and knowledge management
Innovative organisations
Complexity theory
G Communication
Team effectiveness
Social networks and influencing
Opinion leaders
Leadership
Social learning
Professional development
34 The multi-disciplinary team would agree processes for a case work approach to management of people with advanced dementia No
35 The multi-disciplinary team would agree processes for a single point of contact for family/friend carers of people with advanced dementia Yes O Integrated care
Organisational learning and knowledge management
Innovative organisations
Complexity theory
G Communication
Team effectiveness
Social networks and influencing
Opinion leaders
Leadership
Social learning
Professional development
36 The multi-disciplinary team co-ordinator will liaise with Gold Standards Framework meetings held in general practices Yes O Integrated care
Organisational learning and knowledge management
Innovative organisations
Complexity theory
G Communication
Team effectiveness
Social networks and influencing
Opinion leaders
Leadership
Social learning
Professional development
37 Training and monitoring to improve the sustainability and maximise the benefits of the Gold Standards Framework in care homes and in primary care No
38 Commissioning to quality standards for end of life and for dementia care Yes P Reimbursement
Contracting
39 Commissioning to improve clinical status and outcomes for people with advanced dementia Yes P Reimbursement
Contracting
40 Commissioning members of the multi-disciplinary team to work in care homes and liaise with Gold Standards Framework Co-ordinators: geriatricians No
41 Commissioning members of the multi-disciplinary team to work in care homes and liaise with Gold Standards Framework Co-ordinators: mental health professionals (psychiatrists and mental health nurses) Yes P Reimbursement
Contracting
O Total quality management
Innovative organisations
Organisational culture
Organisational learning and knowledge management
42 Commissioning members of the multi-disciplinary team to work in care homes and liaise with Gold Standards Framework Co-ordinators: palliative care specialist teams No
43 Commissioning members of the multi-disciplinary team to work in care homes and liaise with Gold Standards Framework Co-ordinators: community care specialists Yes P Reimbursement
Contracting
O Total quality management
Innovative organisations
Organisational culture
Organisational learning and knowledge management
44 Commissioning members of the multi-disciplinary team to work in care homes and liaise with Gold Standards Framework Co-ordinators: General practitioners and district nurses Yes P Reimbursement
Contracting
O Total quality management
Innovative organisations
Organisational culture
Organisational learning and knowledge management
45 There should be full buy in from local commissioners Yes P Reimbursement
Contracting
O Total quality management
Innovative organisations
Organisational culture
Organisational learning and knowledge management
46 Primary care provision for dementia should be linked to Quality Outcomes Frameworks for general practitioners and GP contracts No
47 Competitiveness and business models of independent care providers such as care homes should be linked to quality standards No
48 Expertise on dementia and end-of-life care will be shared between voluntary and statutory sectors including the NHS, for example, Alzheimer’s Society, Marie Curie and NHS Yes P Reimbursement
Contracting
O Integrated care
Innovative organisations
Total quality management
Organisational learning and knowledge management
G Communication
Leadership
Professional development
49 Beacon’ services and pockets of good practice for end-of-life care in advanced dementia should be highlighted and publicised, for example, Haringey (teleconferencing to discuss cases) and Croydon models (virtual wards) for dementia care Yes O Integrated care
Organisational learning and knowledge management
Innovative organisations
Complexity theory
G Communication
Team effectiveness
Social networks and influencing
Opinion leaders
Leadership
Social learning
Professional development

RAM: RAND/UCLA appropriateness method; I: Individual professional; G: Social interaction; O: Organisational context; P: Political and economic context, and sub-theories likely to operate; NHS: National Health Service.

Retained statements in italics (N = 29) mapped to levels at which complex healthcare systems operate.

RAM process, collation of results and mapping to theory

Table 3 shows characteristics of participants in each RAM workshop and the results of the ratings for appropriateness and necessity. Collation of data from Round 3 resulted in the retention of 29 statements which were mapped to their proposed theoretical levels of action (Table 2).

Table 3.

RAM workshops with healthcare professionals.

Round 1: online (appropriateness) (n = 41) Round 2: workshop (appropriateness) (n = 40) Round 3: online (necessity) (n = 26)
Job title (as described by participant) The number of statements agreed on by group in each round
Belfast (n = 7) n = 7 n = 7 n = 3
General Practitioner*, Project Manager*, Nursing Services Manager, Staff Nurse, Palliative Care Consultant*, Consultant Old Age Psychiatrist and Palliative Care Nurse Specialist 13 33 32
Edinburgh, n = 8 n = 8 n = 7 n = 7
Occupational Therapist, Service Manager, General Practitioner, PhD Student*, General Manager, Staff Nurse*, Clinical Nurse Specialist* and Consultant in Palliative Care 26 47 41
London (North), n = 10 n = 10 n = 10 n = 8
Health Services for Elderly People, Assistant Practitioner, Occupational Therapist Palliative Care, Speech and Language Therapist, Social Worker, Occupational Therapist Elderly Care, Head of Care, Clinical Nurse Specialist Palliative Care, Care Home Manager and General Practitioner 34 44 44
Wales (Penarth), n = 6 n = 6 n = 6 n = 2
Service Design Manager, Consultant Old Age Psychiatrist, Clinical Lead Nurse (Palliative Care), Registrar in Old Age Psychiatry and Nurse 21 28 28
Birmingham (Solihull), n = 10 n = 10 n = 10 n = 6
Consultant Geriatrician, Palliative Care Doctor, Ward Sister*, Staff Nurse, Staff Nurse, Clinical Nurse Specialist, Social Worker, Hospice Volunteer/Carer*, Admiral Nurse* and Admiral Nurse Team Leader 31 33 31

RAM: RAND/UCLA appropriateness method.

*

Also attended early workshops in wider programme.

In Rounds 1 and 2, the total number of statements considered at all sites is 49. In Round 3, the total number of statements considered at each site were Belfast = 33, Edinburgh = 47, London = 44, Wales = 28, Birmingham = 33. This was because only the ‘appropriate’ statements from each group following Round 2 were rated for ‘necessity’.

Combining statements to derive pragmatic components

The wider research team considered the retained statements finding that several could be combined, both on theoretical underpinning and stated practical aims. Since it is difficult to separate individual professional behaviours from interaction with colleagues and teams, we combined the levels of individual and social interaction under the pragmatic theme of ‘training and support based’. We retained the themes ‘organisational context’ and ‘political and economic context’. We describe three core components (Table 4):

Table 4.

The COMPASSION intervention version final draft for detailing in a written manual.

Main component Statements combined (numbers refer to statements listed in Table 2)
Component 1: influencing how local services are organised
(i) Improved cross disciplinary and cross boundary communication, including out of hours support for people with dementia, their family/friend carers and professional carers in care homes, assessment of social care needs 29, 4 (Statements 4 and 29 influenced mostly by the organisational context, but highly dependent on political and economic environment)
(ii) The integrated care team is composed of geriatricians, mental health, palliative care, community care specialists, GPs and district nurses and liaise with Gold Standards Framework meetings held in general practice. It agrees processes for guidelines for referrals for care from specialist or generalist clinical or social care services, provision of holistic care and single point of contact for family/friend carers of people with advanced dementia 2, 36, 32,33,35
(iii) Support from the integrated care team to front line staff and managers in care homes to manage risk in people with dementia and avoid unnecessary place of care transfers 20
(iv) ‘Beacon’ services and local pockets of good practice for end-of-life care in advanced dementia are highlighted and publicised, for example, in UK Haringey (teleconferencing to discuss cases) and Croydon models (virtual wards) 49
Component 2: providing training and support to formal and informal carers
(i) Training in the natural history of dementia. Someone is on duty in care homes who can recognise clinical needs and is qualified to respond, for example, administering pain relief 17, 12
(ii) On-going training and support for care home staff to ensure that they have the clinical skills to optimise the management of pain, acute medical events, behavioural disorders (e.g. agitation and delirium), feeding decisions, pressure sores, comfort measures. Includes training to maximise benefits of initiatives such as the UK Gold Standards Framework 16, 37
(iii) On-going training for all integrated team members improve understanding of what is meant by an advance care plan, and how such a plan might be worked out and used in practice; diagnose dying; to recognise the needs of family/friend carers particularly being alert to anxiety and depression 9, 18, 10
(iv) All integrated care team members (particularly care home staff) are trained to communicate and connect with person with dementia using simple methods, for example, tone of voice, touch, eye contact; begin difficult conversations with the family/friend carer of the person with advanced dementia to understand their wishes; understand and use appropriately clinical care pathways; understand ethical and cultural issues in death and bereavement 7, 8, 19, 22
Component 3: influencing the political and economic environment
(i) Commissioning to improve clinical status and outcomes for people with advanced dementia to national and international quality standards for end of life and for dementia care 39, 38
(ii) Staff ratios and skill mix in care homes should be supported by regulatory bodies (e.g. UK Care Quality Commission) 14, 34, 35, 13
The levels of staff available in care homes should be sufficient to allow appropriate clinical and personal care to be conducted for all people with dementia
(iii) There should be full buy in from local Clinical Commissioning Groups to enable members of the integrated care team (composed of geriatricians, mental health, palliative care, community care specialists, GPs and district nurses) to work in care homes, conduct regular reviews of health and social care needs for individuals and liaise with local care coordinators (e.g. Gold Standards Framework Co-ordinators) and with general practices to ensure people living at home with advanced dementia are included 45, 6, 41, 43, 44, 45 (Statements 4 and 29 influenced mostly by the organisational context, but highly dependent on political and economic environment)
(iv) Expertise on dementia and end-of-life care will be shared between larger provider organisations including voluntary and statutory sectors, for example, in UK Alzheimer’s Society, Marie Curie and NHS 48

Statements from theoretical mapping phase were reviewed and combined where overlap was found to devise pragmatic statements that were meaningful in practice. For simplicity, four finalised statements were prepared which may be thought of as operational sub-components for each main component of the intervention.

  1. Influencing how local services are organised;

  2. Providing education, training and support for formal and informal carers;

  3. Influencing the political and economic environment.

Components 1 and 2 require dedicated time from a professional, with experience of working with people with dementia in care home settings, to lead integrated approaches to care. They work within existing systems to (1) understand current service provision and practice; (2) develop joint working between providers across primary, secondary and tertiary care; (3) support local staff to establish processes for regular assessment of those progressing towards the end of life; (4) alert members of the multi-disciplinary team to those people with dementia who require additional care planning and management and (5) address educational and training needs (linked to nationally agreed core competencies) through shared working, use of online resources and formal topic-based teaching if required.

COMPASSION intervention manual

In the intervention manual, we provide detailed descriptions of processes to achieve these aims, to enable new working practices and to initiate and maintain an education training and support programme.34 Components 1 and 2 are both dependent upon, and will influence to varying degrees, the local political and economic context, including commissioning priorities of local service providers.

Discussion

Key findings

We used an iterative process to develop COMPASSION, an integrated intervention for end-of-life care in advanced dementia. We combined information from existing literature, empirical data, opinions of expert policy makers and those working in the field with our theoretical frameworks. A realist approach enabled interactive discussions in workshops and interviews so that rather than describing problems, potential solutions could emerge. These were considered in workshops, ranked for appropriateness and necessity in the RAM process, and combined in pragmatic statements for use in practice. Understanding the theoretical underpinning enabled us to consider who needs to do what differently, which barriers and enablers need to be addressed according to context, and which behaviours and models of delivery might influence these issues.36

We shall learn more about how COMPASSION operates as we complete our exploratory implementation study.34 This will provide information on how the integrated model of care works in practice, and preliminary data on costs.36 Describing sustainability is challenging and a number of theoretical models are available.37,38 We revisit the test sites twice within 12 months of implementation to assess which elements of the intervention have been incorporated into routine practice and look for ripple effects within the healthcare economy. An integrated approach to dementia care is needed as one discipline alone cannot manage the complex psychiatric, physical and social problems which occur.39 Integrated care may improve outcomes, in particular quality of life, through improved diagnosis, treatment and management of problems as they arise.40

Strengths and weaknesses

Our work concords with recent recommendations on research in dementia at the end of life, in particular gathering data from multiple sources, considering underlying theories and using an iterative approach to devise an intervention that is flexible and sensitive to context.26 We focus on care homes as we identified very few people with end-stage dementia who remained in their own homes, and we incorporate views from across the United Kingdom. The RAM process is more often used for achieving consensus on simpler healthcare interventions. However, it facilitates some idealism as participants rate for appropriateness and necessity irrespective of economic considerations. This is consistent with our realist approach which encourages discussion of new solutions to problems. Other consensus methods such as Delphi processes or nominal groups are available, but advantages of the RAM include incorporation of data from the literature, confidential ratings plus group discussion and multi-disciplinary panels including views from a range of perspectives. Disadvantages include multiple rating rounds with a nine-point scale, the need for face-to-face meetings, problems of dominant forces within groups and risk of bias from the research team in preparation of the statements for consideration.41

Additional bias may have occurred during our theoretical mapping phase which relied on choices of the research team. However, the benefits of gaining understanding from use of theories are well recognised in the field of healthcare innovations,36,42 and the incorporation of evidence on the views of recipients of an intervention at the design stage is increasingly understood.43,44

Conclusion

It is unusual for healthcare interventions to be developed using such thorough processes and our work provides a template of how this is possible. We hope, but cannot yet be sure, that COMPASSION has enhanced potential for feasibility, acceptability and effectiveness.

Through synthesising what may be most desirable in current practice, COMPASSION could be considered a ‘meta-intervention’. Rather than being innovative and suggesting radical changes to care, COMPASSION provides processes which maximise existing expertise. Care is enhanced through breaking down barriers between teams, placing the person with dementia at the centre of care, educating care providers and ensuring sensitivity to local context.

Acknowledgments

We thank professor Martin Marshall, Professor of healthcare improvement at UCL, for his methodological advice; Marie Curie Hospice Research Facilitators Kathy Armour, Anne Finucane and Noleen McCorry for assistance with workshops; Professor Steve Morris UCL as co-applicant in obtaining funding for the COMPASSION Programme and for advice on health economics; Mr Jeffrey Round UCL for advice on health economics; members of our expert steering group and all participants for their generosity in giving time to this programme of work.

Footnotes

Author contributions: Louise Jones and Elizabeth Sampson conceived the research and obtained funding for the COMPASSION Programme, managed all elements of the work and drafted the manuscript. Bridget Candy conducted the rapid evidence review and contributed to data interpretation. Sarah Davis, Jane Harrington, Nuriye Kupeli, Kathryn Lord and Sharon Scott collected data from participants and contributed to data analysis and interpretation. Margaret Elliott and Kirsten Moore finalised the COMPASSION intervention manual and contributed to analysis and interpretation of data. Victoria Vickerstaff and Anna Gola contributed to data analysis and interpretation. Rumana Omar, Michael King, Gerard Leavey and Irwin Nazareth (with Steve Morris) conceived and obtained funding for the COMPASSION research programme and provided expert advice throughout the programme. All authors contributed to the development of the manuscript and approved the final version.

Declaration of conflicting interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Ethics: Ethics approval was obtained from University College London (UCL) research ethics committee (ref 3578/001).

Funding: This work was supported by Marie Curie programme grant funding to Marie Curie Palliative Care Research Department, UCL, grant ref. MCCC-FPR-11-U.

References

  • 1. Alzheimer’s Society. Dementia 2014: opportunity for change. London: Alzheimer’s Society, 2014, www.alzheimers.org.uk/site/scripts/download_info.php?fileID=2317 (accessed 3 March 2015). [Google Scholar]
  • 2. Rait G, Walters K, Bottomley C, et al. Survival of people with clinical diagnosis of dementia in primary care: cohort study. BMJ 2010; 341: c3584. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Brayne C, McCracken C, Matthews FE. Cohort profile: Medical Research Council Cognitive Function and Ageing Study (CFAS). Int J Epidemiol 2006; 35(5): 1140–1145. [DOI] [PubMed] [Google Scholar]
  • 4. Addington-Hall J, Altmann D, McCarthy M. Variations by age in symptoms and dependency levels experienced by people in the last year of life, as reported by surviving family, friends and officials. Age Ageing 1998; 27(2): 129–136. [DOI] [PubMed] [Google Scholar]
  • 5. Houttekier D, Vandervoort A, Van den Block L, et al. Hospitalizations of nursing home residents with dementia in the last month of life: results from a nationwide survey. Palliat Med 2014; 28(9): 1110–1117. [DOI] [PubMed] [Google Scholar]
  • 6. Klapwijk MS, Caljouw MA, van Soest-Poortvliet MC, et al. Symptoms and treatment when death is expected in dementia patients in long-term care facilities. BMC Geriatr 2014; 14: 99. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. Perrar KM, Schmidt H, Eisenmann Y, et al. Needs of people with severe dementia at the end-of-life: a systematic review. J Alzheimers Dis 2015; 43(2): 397–413. [DOI] [PubMed] [Google Scholar]
  • 8. Mitchell SL, Miller SC, Teno JM, et al. Prediction of 6-month survival of nursing home residents with advanced dementia using ADEPT vs hospice eligibility guidelines. JAMA 2010; 304(17): 1929–1935. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Kitwood T, Bredin K. Towards a theory of dementia care: personhood and well-being. Ageing Soc 1992; 12: 269–287. [DOI] [PubMed] [Google Scholar]
  • 10. Kitwood T. Toward a theory of dementia care: ethics and interaction. J Clin Ethics 1998; 9(1): 23–34. [PubMed] [Google Scholar]
  • 11. National Institute for Health and Care Excellence. Quality standard for dementia [NICE QS1], https://www.nice.org.uk/guidance/qs1 (2010, accessed 3 March 2015).
  • 12. National Institute for Health and Care Excellence. Quality standard for end of life care for adults [NICE QS13], http://www.nice.org.uk/guidance/qs13 (2011. accessed 3 March 2015).
  • 13. Department of Health. End of life care strategy: promoting high quality care for all adults at the end of life. London: Department of Health, 2008. [Google Scholar]
  • 14. Department of Health. Improving care for people with dementia. London: Department of Health, 2013. [Google Scholar]
  • 15. Ngo J, Holroyd-Leduc JM. Systematic review of recent dementia practice guidelines. Age Ageing 2015; 44(1): 25–33. [DOI] [PubMed] [Google Scholar]
  • 16. van der Steen JT, Radbruch L, Hertogh CM, et al. ; European Association for Palliative Care (EAPC). White paper defining optimal palliative care in older people with dementia: a Delphi study and recommendations from the European Association for Palliative Care. Palliat Med 2014; 28(3): 197–209. DOI: 10.1177/0269216313493685. [DOI] [PubMed] [Google Scholar]
  • 17. Gold Standards Framework. The gold standards framework. NHS End of Life Care Programme, http://www.goldstandardsframework.org.uk/ (2008, accessed 2 March 2015).
  • 18. Kinley J, Stone L, Dewey M, et al. The effect of using high facilitation when implementing the Gold Standards Framework in Care Homes programme: a cluster randomised controlled trial. Palliat Med 2014; 28(9): 1099–1109. [DOI] [PubMed] [Google Scholar]
  • 19. Booth M, Nash S, Banks C, et al. Three approaches to delivering end-of-life education to care homes in a region of south east England. Int J Palliat Nurs 2014; 20(1): 27–35. [DOI] [PubMed] [Google Scholar]
  • 20. Kinley J, Hockley J, Stone L, et al. The provision of care for residents dying in UK nursing care homes. Age Ageing 2014; 43(3): 375–379. [DOI] [PubMed] [Google Scholar]
  • 21. Goodman C, Drennan V, Scheibl F, et al. Models of inter professional working for older people living at home: a survey and review of the local strategies of English health and social care statutory organisations. BMC Health Serv Res 2011; 11: 337. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22. Davies SL, Goodman C, Bunn F, et al. A systematic review of integrated working between care homes and health care services. BMC Health Serv Res 2011; 11: 320. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23. Gage H, Dickinson A, Victor C, et al. Integrated working between residential care homes and primary care: a survey of care homes in England. BMC Geriatr 2012; 12: 71. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24. Jones L, Harrington J, Scott S, et al. CoMPASs: IOn programme (Care Of Memory Problems in Advanced Stages of dementia: improving Our Knowledge): protocol for a mixed methods study. BMJ Open 2012; 2: e002265. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25. Fitch K, Bernstein SJ, Aguilar MS, et al. The RAND/UCLA appropriateness method user’s manual. Santa Monica, CA: RAND Corporation, MR-1269-DG-XII/RE, http://www.rand.org/content/dam/rand/pubs/monograph_reports/2011/MR1269.pdf (accessed 2 September 2015). [Google Scholar]
  • 26. Van der Steen JT, Goodman C. What research we no longer need in neurodegenerative disease at the end of life: the case of research in dementia. Palliat Med 2015; 29(3): 189–192. [DOI] [PubMed] [Google Scholar]
  • 27. Pawson R, Tilley N. Realistic evaluation. London: SAGE, 1997. [Google Scholar]
  • 28. Michie S, Abraham C. Interventions to change health behaviours: evidence-based or evidence-inspired? Psychol Health 2004; 19(1): 29–49. [Google Scholar]
  • 29. MRC framework for the development of complex interventions. http://www.mrc.ac.uk/documents/pdf/complex-interventions-guidance/ (2008, accessed 3 March 2015).
  • 30. Higginson IJ, Evans CJ, Grande G, et al. ; MORECare. Evaluating complex interventions in end of life care: the MORECare statement on good practice generated by a synthesis of transparent expert consultations and systematic reviews. BMC Med 2013; 11: 111. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31. Grol RPTM, Bosch MC, Hulscher MEJL, et al. Planning and studying improvement in patient care: the use of theoretical perspectives. Millbank Q 2007; 85(1): 93–138. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32. Ferlie EB, Shortell SM. Improving the quality of health care in the United Kingdom and the United States: a framework for change. Milbank Q 2001; 79(2): 281–315. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33. Greenhalgh T, Robert G, Macfarlane F, et al. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004; 82(4): 581–629. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34. Elliott M, Harrington J, Moore K, et al. A protocol for an exploratory phase I mixed-methods study of enhanced integrated care for care home residents with advanced dementia: the Compassion Intervention. BMJ Open 2014; 4: e005661. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35. Rossi PH, Lipsey MW, Freeman HE. Evaluation: a systematic approach. 7th ed. Thousand Oaks, CA: SAGE, 2004. [Google Scholar]
  • 36. French SD, Green SE, O’Connor DA, et al. Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implement Sci 2012; 7: 38. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37. Eccles M, Grimshaw J, Walker A, et al. Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. J Clin Epidemiol 2005; 58(2): 107–112. [DOI] [PubMed] [Google Scholar]
  • 38. Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci 2013; 8: 117. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39. Lloyd-Williams M, Abba K, Crowther J. Supportive and palliative care for patients with chronic mental illness including dementia. Curr Opin Support Palliat Care 2014; 8(3): 303–307. [DOI] [PubMed] [Google Scholar]
  • 40. Wolfs CA, Dirksen CD, Severens JL, et al. The costs and benefits of an integrated approach to dementia. Tijdschr Psychiatr 2011; 53(9): 657–665. (Dutch). [PubMed] [Google Scholar]
  • 41. Nair R, Aggarwal R, Khanna D. Methods of formal consensus in classification/diagnostic criteria and guideline development. Semin Arthritis Rheum 2011; 41(2): 95–105. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42. Brazil K, Ozer E, Cloutier MM, et al. From theory to practice: improving the impact of health services research. BMC Health Serv Res 2005; 5: 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43. Candy B, King M, Jones L, et al. Using qualitative synthesis to explore heterogeneity of complex interventions. BMC Med Res Methodol 2011; 11: 124. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44. Candy B, King M, Jones L, et al. Using qualitative evidence on patients’ views to help understand variation in effectiveness of complex interventions: a qualitative comparative analysis. Trials 2013; 14: 179. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Palliative Medicine are provided here courtesy of SAGE Publications

RESOURCES