Skip to main content
BMJ Open logoLink to BMJ Open
. 2020 Nov 24;10(11):e035988. doi: 10.1136/bmjopen-2019-035988

Improving regional care in the last year of life by setting up a pragmatic evidence-based Plan–Do–Study–Act cycle: results from a cross-sectional survey

Raymond Voltz 1,2,3,4,#, Gloria Dust 1,✉,#, Nicolas Schippel 5, Stefanie Hamacher 6, Sheila Payne 7, Nadine Scholten 5, Holger Pfaff 5, Christian Rietz 8, Julia Strupp 1; The CoRe‑Net Co‑applicants
PMCID: PMC7689073  PMID: 33234614

Abstract

Objectives

To set up a pragmatic Plan–Do–Study–Act cycle by analysing patient experiences and determinants of satisfaction with care in the last year of life.

Design

Cross-sectional postbereavement survey.

Setting

Regional health services research and development structure representing all health and social care providers involved in the last year of life in Cologne, a city with 1 million inhabitants in Germany.

Participants

351 bereaved relatives of adult decedents, representative for age and gender, accidental and suspicious deaths excluded.

Results

For the majority (89%) of patients, home was the main place of care during their last year of life. Nevertheless, 91% of patients had at least one hospital admission and 42% died in hospital. Only 60% of informants reported that the decedent had been told that the disease was leading to death. Hospital physicians broke the news most often (58%), with their communication style often (30%) being rated as ‘not sensitive’. Informants indicated highly positive experiences with care provided by hospices (89% ‘good’) and specialist palliative home care teams (87% ‘good’). This proportion dropped to 41% for acute care hospitals, this rating being determined by the feeling of not being treated with respect and dignity (OR=23.80, 95% CI 7.503 to 75.498) and the impression that hospitals did not work well together with other services (OR=8.37, 95% CI 2.141 to 32.71).

Conclusions

Following those data, our regional priority for action now is improvement of care in acute hospitals, with two new projects starting, first, how to recognise and communicate a limited life span, and second, how to improve care during the dying phase. Results and further improvement projects will be discussed in a working group with the city of Cologne, and repeating this survey in 2 years will be able to measure regional achievements.

Trial registration number

DRKS00011925.

Keywords: adult palliative care, quality in health care, change management


Strengths and limitations of this study.

  • This study presents a multidimensional and comprehensive assessment of care provided in multiple settings in the entire last year of life.

  • It describes a pragmatic template for determining regional improvement priorities based on patient experiences using a city-wide health services research and development structure (Plan–Do–Study–Act cycle).

  • The reports of relatives cannot be seen as a direct substitute for a self-assessment of patients, although a good agreement has been shown on service evaluations and observable symptoms.

  • The retrospective approach bypasses the difficult task of identifying the terminally ill, avoids putting an additional burden on very sick participants and minimises missing data due to poor functional status.

Introduction

Patients in their last year of life may be regarded as vulnerable because most of them need complex care provided by multiple health and social care practitioners.1 Their needs are often not sufficiently met, especially in settings outside specialist palliative and hospice care.2 3 Due to the fact that they are often dependent on others to meet their physical care needs, they can deteriorate unpredictably and rapidly, and their ability to make informed choices may be reduced due to cognitive impairment.4 Care transitions as well as non-beneficial treatments are common, which may compromise human dignity and quality end-of-life care.5 6 In Germany, the evidence-based guideline ‘Palliative care’ was published in 2015 to promote quality end-of-life care by all healthcare professionals. This aim may be achieved by ‘providing palliative care services in both a timely manner and in accordance with the affected persons’ needs, treating the common symptoms according to current scientific evidence and clinical expertise, enabling conversations with patients and their families to be held and treatment goals to be set together, ensuring that support in the dying phase can be appropriately and optimally given’.7

Evidence shows that most people prefer their home as place of death.8 This also applies to Germany. However, almost one person in two dies in hospital and one-third in care home.9 Fragmentation of multidisciplinary service provision is one major barrier to adequately addressing patients’ needs and preferences.10 Data on patients’ and their families’ perceptions are essential in order to understand these issues and to inform quality improvements. The last year of life can obviously only be objectively determined in retrospect, but it has to be managed prospectively. Adding palliative care in the last 12–24 months of life has proven to be greatly beneficial.11 12 Therefore, consideration also needs to be given to the introduction of these services. However, the majority of surveys focuses on the dying phase or the last weeks to months of life and only a few studies have compared end-of-life care across different settings.13–16 Care in the last year of life encompasses a wide range of services necessitating a multidimensional and comprehensive assessment approach.14 This comprises the measurement of patients’ experiences of the communication of a life-limiting disease (‘transition into the last year of life’), transitions between healthcare settings (‘transitions within the last year of life’) and the dying phase (‘transition into death and a new phase of life for bereaved relatives’). For Germany, data on patients’ perceptions of their experience of care in the last year of life that go beyond diagnoses and care settings are still rare.

Germany possesses a well-defined, government-led strategy for the development and promotion of national palliative care and performed well in the 2015 Quality of Death Index (position seven in the global rank of overall scores).17 Based on the patients’ needs, they may receive generalist palliative care (eg, provided in a general hospital ward, a nursing home, by a general practitioner (GP) or a nursing service at home) or specialist palliative care provided by specialist personnel with expert knowledge, skills and attitudes (eg, delivered in a hospital palliative care unit, a hospice or by a specialist palliative home care team). Hospice care in Germany refers to care provided in an inpatient facility. In addition, hospice services delivered by volunteers may support the patients and their families during illness and after a patient’s death. All services and accommodations in these facilities are provided at no extra cost to the patient and are part of the service of the statutory as well as private health insurance. As in many other countries, the national level indicates the general direction in which the healthcare system is developing. However, for the patients and caregivers, the regional healthcare situation matters most, and this shows a lot of heterogeneity in Germany.9 Due to this regional heterogeneity, this local level should be the driving force for innovation.18 19

Delivering improvements in the quality of healthcare remains an international challenge. Methods such as the Plan–Do–Study–Act (PDSA) cycle have been widely used in healthcare as an attempt to drive such improvements. The PDSA method is an iterative four-step cycle, which includes: (1) identifying a change aimed at improvement, (2) testing this change, (3) examining the success of the change and (4) identifying adaptations and next steps to inform a new cycle.20 The fundamental principle is iteration (figure 1).21 22 To inform practice, we need practical but still evidence-based solutions that lead to ‘satisficing’ decisions.23–26 Therefore, as a paradigm case for a 1 million inhabitant urban healthcare situation, this paper describes the set-up of a pragmatic quality improvement structure to improve regional end-of-life care. In particular, as the first step of the PDSA cycle, we analyse patient experiences across settings and providers and determinants of satisfaction with care in the last year of life as reported by bereaved relatives.

Figure 1.

Figure 1

Strategic learning process framed on the Plan–Do–Study–Act (PDSA) cycle Based on Pfaff and Zeike.22

Methods

Setting up the regional structure

Initiated by three cooperating faculties (Human Sciences; Medicine; and Management, Economics and Social Sciences) of the University of Cologne, Germany, a regional health services research and development structure has been established involving as many relevant partners as possible from medical and social care as well as from health services research. The ‘Cologne Research and Development Network’ (CoRe-Net)27 focuses on the further development of health and social care according to the concept of learning organisations based on a bottom-up approach. Its main aim is to analyse and continuously improve healthcare delivery for vulnerable patient groups by going through PDSA cycles. We invited national and international scientific experts, regional health and social care institutions, patient organisations and statutory health insurance providers to act as CoRe-Net collaboration partners. Within the network, three research projects have been initiated, one focusing on patients in their last year of life. For this project entitled ‘Last Year of Life Study Cologne (LYOL-C)’,28 all health and social care practitioners involved in the care of patients in their last year of life (ie, care homes, nursing services, hospices, bereavement cafés, doctors, hospitals, undertakers and the local public health department) as well as providers of grief work in Cologne were contacted for collaboration.

Patient involvement

Patient representatives were involved in the design and conduct of this research. They worked with us to refine the research question, choice of outcome measures and methods of recruitment. In collaboration with patient representatives, we will design a leaflet for dissemination to distributing to patient groups.

Sample and data collection

Data were collected in a postbereavement survey as part of the project LYOL-C. Participants were a purposive sample of relatives, friends and voluntary workers (all will be referred to as ‘informants’ hereafter). Inclusion criteria required informants to be aged 18 years and older and to have recently cared for a person who lived their last year of life in the Cologne area. Deaths of people under the age of 18 years as well as accidental and suspicious deaths were excluded. To ensure maximum variation, informants in this study were recruited in cooperation with health and social care practitioners from Cologne (partners from care homes, nursing services, hospices, bereavement cafés, doctors, hospitals, undertakers and local public health department). We applied two strategies between November 2017 and August 2018 to identify potential informants: (1) questionnaire distribution through cooperating care practitioners through client records by mail or personally and (2) self-selection through public media (newspaper articles, flyers and posters). Informants who were interested in taking part in the survey could request a postal questionnaire from one of the researchers (GD); one reminder was sent. An information sheet describing the study’s goals and a response form for opting out of the study were sent with the questionnaire. Informants gave written informed consent before taking part in the survey. All pseudonymously completed questionnaires were returned directly to the research team.

Questionnaire

The self-complete questionnaire is based on the Views Of Informal Carers – Evaluation of Services-Short Form (VOICES-SF).29 It is a validated questionnaire to assess the bereaved relatives’ perceptions of the patient’s care experiences with providers and services across care settings in the last 3 months of life. VOICES-SF was developed for a nationally representative cross-sectional survey to examine the quality of end-of-life care in England, conducted annually for 5 years (2011–2015),30 and has already been used internationally.3 31–37

We developed an adapted German version (‘VOICES-LYOL-Cologne’) that covers the last 12 months of life and additionally includes the communication of the diagnosis of a life-limiting disease as well as places of care with periods of stay. The questionnaire comprises 106 items and contains skip logic so that informants only respond to questions relevant to the care the patient received. For each specified setting/healthcare provider (home: nursing service, specialist palliative home care team, hospice service, GP and outpatient specialist physicians; care home; hospital [last admission] and inpatient hospice), informants could rate their experiences with care on a four-point scale with respect to the manner of communication, relief of pain, relief of other symptoms, coordination of care, care quality in the last 2 days of life (1=yes, 2=rather yes, 3=rather no and 4=no), respect and dignity (1=always, 2=most of the time, 3=some of the time and 4=never) and overall satisfaction (1=good, 2=rather good, 3=rather bad and 4=bad). The questionnaire further assesses communication of a life-limiting disease, carer support, unmet needs, financial needs, preferences and decision making, place of death, bereavement support, disease specific and sociodemographic data. Next the informants were asked to provide the settings in which the patients had received care during the last year of life (eg, home, hospital, nursing home, hospice and rehabilitation clinic) and the period of time spent per stay. Data were inserted in a table with chronological order. Objectivity, reliability and validity (content validity as well as divergent and convergent construct validity) were taken into consideration. The VOICES–LYOL-C survey has been shown to be useful to assess the quality of care in the last year of life (Dust et al in preparation).

Statistical analyses

Data were analysed descriptively, and results are presented as mean±SD and count (percentage), respectively. Differences in informants’ ratings of care were tested using Mann-Whitney U test for independent samples and Wilcoxon signed-rank test for paired samples.

Second, we performed a logistic regression analysis to explore factors associated with overall satisfaction with care provided in general hospital wards and intensive care units (ICUs). All variables were dichotomised as applied in previous VOICES studies.34 38–40 The dependent variable (‘Overall, do you feel that the care he/she got from the doctors in the hospital on that admission was: good, rather good, rather bad, bad?’) was merged into the most positive response to the question versus all other responses. The independent variables comprised sociodemographic variables of the deceased patients (age, gender, family situation and power of attorney) and variables concerning service provision (care by a specialist palliative home care team, treated with respect and dignity by the hospital doctors, relief of pain on last hospital admission, relief of other symptoms on last hospital admission and coordination of care on last hospital admission). First, univariable effect was tested. In a second step, all variables with p value ≤0.1 were entered into a multivariable logistic regression model using a stepwise, forward selection procedure. Results are presented as ORs, corresponding 95% CIs and p values.

Data were analysed using SPSS Statistics V.25 and RStudio V.3.5.1 (RStudio Inc, Boston, Massachusetts, USA). Analyses were restricted to individuals with complete data on all variables required for a particular analysis. All presented p values are two sided and considered significant if p<0.05.

Results

Regional structure: the ‘Cologne Research and Development Network’

CoRe-Net has been established as a long-term and sustainable network, making Cologne a model region for the improvement of health and social care in Germany. Its members consist of three research projects and their collaboration partners. The interdisciplinary cooperation enables the integration of different perspectives and methods. Around 100 partners from health and social care services in Cologne collaborated in the subproject LYOL-C. They represented care homes, nursing services, hospices, bereavement cafés, doctors, hospitals, undertakers and the local public health department. An initial meeting took place to consent the primary outcome and to discuss practical issues (eg, recruitment).

Survey sample

The questionnaire was returned by 365 informants. Response rate was 21.1% for postal distribution, 10.3% for personal distribution and 74.9% for the self-selection group. A total of 14 questionnaires did not meet the inclusion criteria and had to be excluded. Characteristics of the patients and their informants are presented in table 1. For the 351 deceased patients, the majority of informants were a spouse or a child (81.8%), female (71.5%) and aged between 50 and 79 years (77.5%). Point in time of participation in the survey was 7.6±4.9 months after the patient’s death. The sample of decedents comprises patients who died between December 2015 and July 2018. It was representative with respect to gender (47.9% male) and age (76.5±13.0 years) compared with full data from the city of Cologne (gender: 50% male,41 age: 77.7 years42). The main underlying diseases were cancer (59.5%) and cardiovascular (40.5%), neuropsychological (32.8%) and pulmonary (29.3%) diseases. One-third lived alone.

Table 1.

Demographics and characteristics of deceased patients and informants (N=351)

N (%)
Deceased age at death (years)
 18–29 1 (0.3)
 30–49 6 (1.7)
 50–64 65 (18.5)
 65–79 112 (31.9)
 80+ 167 (47.6)
Deceased sex
 Male 168 (47.9)
 Female 183 (52.1)
Deceased ethnic group
 German 340 (96.9)
 Other 11 (3.1)
Deceased family situation*
 Had a partner 163 (46.4)
 Lived together with partner 126 (35.9)
 Had children 168 (47.9)
 Lived together with children 32 (9.1)
 Lived together with someone else 21 (6)
 Lived alone 114 (32.5)
Someone else had a power of attorney
 Yes 311 (88.6)
 No 33 (9.4)
 Do not know 7 (2)
Illnesses in the last year of life*
 Cancer 209 (59.5)
 Cardiovascular disease 142 (40.5)
 Neuropsychological disease 115 (32.8)
 Disease of the respiratory system 103 (29.3)
 Liver or kidney disease 67 (19.1)
 Diabetes mellitus 45 (12.8)
 Decubitus ulcer 24 (6.8)
Informant relation to deceased
 Spouse 149 (42.5)
 Son/daughter 138 (39.3)
 Sibling 18 (5.1)
 Son/daughter-in-law 9 (2.6)
 Other relative 17 (4.8)
 Friend 12 (3.4)
 Other 8 (2.3)
Informant age (years)
 18–29 2 (0.6)
 30–49 46 (13.1)
 50–64 154 (43.9)
 65–79 118 (33.6)
 80+ 31 (8.8)
Informant sex
 Male 100 (28.5)
 Female 251 (71.5)

*Multiple responses were possible.

Transition into the last year of life

Two hundred and two (63.9%) of the informants reported that the patient had been told that the disease was leading to death; one-third (n=114, 36.1%) indicated that they had not been told. Of those who were told, 44 (22.8%) received this information less than a month before death, but one-third (n=63, 32.6%) more than a year before death. Hospital doctors were mentioned as breaking the news most often (n=112, 60.2%), followed by outpatient specialist physicians (n=29, 15.6%), close relatives (n=22, 11.8%), GPs (n=15, 8.1%), staff of the specialist palliative home care team (n=6, 3.2%), staff in the care home (n=1, 0.5%) and staff in a hospice (n=1, 0.5%).

Transitions within the last year of life

Regarding the last year of life, most informants (n=310, 88.6%) reported that the deceased person spent some time at home and 75 (22.4%) some time in a care home. GPs (n=305, 87.4%) and outpatient specialist physicians (n=270, 77.1%) mainly provided outpatient care. Furthermore, 157 (45.1%) received care from a home nursing service, 135 (38.8%) from a specialist palliative home care team and 23 (6.6%) were visited by a hospice service. For 224 (73%) patients, the informant reported that urgent care provided out of hours was needed. Furthermore, a total of 320 (91.4%) patients stayed in hospital at least once and almost one-fifth (n=64, 18.8%) in a hospice.

On average, patients had 3.72±3.1 transitions between care settings in the last year of life. Each of the five most frequent transitions included hospital care: home to hospital (47.2%), hospital to home (27.3%), care home to hospital (6.4%), hospital to hospital (6.4%) and hospital to care home (4.1%) (n=255). There was an increasing shift from home to the hospital as the main place of care near death. While 12 months prior to death patients spent 85.1% of their time at home and only 3.1% in hospital, in the final month of life, they spent 46.3% of time at home and 30.2% in hospital (figure 2).

Figure 2.

Figure 2

Care settings in the last year of life in Cologne (n=255).

Transition into death

A percentage of 42.2 (n=148) died in hospital, as opposed to only 4.2% (n=8) who wished to die there. One hundred and sixty-one (45.9%) relatives reported that the decedent did not express a preference for place of death or that they did not know the preference. The most preferred place of death was home (n=129, 67.9%), and this wish was only fulfilled for 27.6% (n=97) of patients. Nevertheless, the majority of informants stated that the decedent died in the right place (n=294, 87.2%), although one-third reported that the patient did not have enough choice about where they died (n=94, 32.1%).

Perceived quality of and satisfaction with care in the last year of life

Regarding informants’ views of professionals’ way of communicating a life-limiting disease, they were least satisfied with hospital doctors (table 2). Almost one-third stated that the hospital doctor did not communicate in a sensitive and caring way. Differences between ratings of the hospital doctor and non-hospital healthcare providers were statistically significant (p=0.003, n=156).

Table 2.

Informants’ views of professionals’ communication style, relief of pain, relief of other symptoms and coordination of care between care settings in the last year of life

Care setting N* Overall rating
Yes Rather yes Rather no No
n (%) n (%) n (%) n (%)
Communication of life-limiting disease in a sensitive and caring way
 Hospice staff 1 0 (0) 1 (100) 0 (0) 0 (0)
 Specialist palliative home care team 5 2 (40) 2 (40) 1 (20) 0 (0)
 Care home staff 1 1 (100) 0 (0) 0 (0) 0 (0)
 General practitioner 14 8 (57.1) 6 (42.9) 0 (0) 0 (0)
 Outpatient specialist 25 9 (36) 11 (44) 4 (16) 1 (4)
 Relative 19 16 (84.2) 3 (15.8) 0 (0) 0 (0)
 Hospital doctor 91 34 (37.4) 30 (33) 13 (14.3) 14 (15.4)
Relief of pain
 Hospice 60 48 (80) 10 (16.7) 2 (3.3) 0 (0)
 Hospital: palliative care unit 84 51 (60.7) 23 (27.4) 7 (8.3) 3 (3.6)
 Care home 66 33 (50) 26 (39.4) 7 (10.6) 0 (0)
 Homecare 246 112 (45.5) 91 (37) 33 (13.4) 10 (4.1)
 Hospital: general ward/ICU 188 82 (43.6) 78 (41.5) 21 (11.2) 7 (3.7)
Relief of other symptoms
 Hospice 58 39 (67.2) 15 (25.9) 3 (5.2) 1 (1.7)
 Hospital: palliative care unit 84 39 (46.4) 31 (36.9) 9 (10.7) 5 (6)
 Care home 60 28 (46.7) 24 (40) 8 (13.3) 0 (0)
 Homecare 255 81 (31.8) 113 (44.3) 41 (16.1) 20 (7.8)
 Hospital: general ward/ICU 180 58 (32.2) 82 (45.6) 31 (17.2) 9 (5)
Worked well together with other services
 Hospice 50 45 (90) 3 (6) 0 (0) 2 (4)
 Hospital: palliative care unit 54 35 (64.8) 12 (22.2) 4 (7.4) 3 (5.6)
 Care home 68 43 (63.2) 18 (26.5) 4 (5.9) 3 (4.4)
 Homecare 244 125 (51.2) 72 (29.5) 34 (13.9) 13 (5.3)
 Hospital: general ward/ICU 133 34 (25.6) 24 (18) 32 (24.1) 43 (32.3)

*Number of patients for whom the corresponding question was responded.

ICU, intensive care unit.

As examples of important quality indicators for care in the last year of life, table 2 further presents informants’ ratings of perceived relief of pain and relief of other symptoms, as well as perceived coordination of care differentiated between care settings. It is notable that for symptom control both the home as well as acute hospital setting is rated the most poorly. Only 45.5% of informants stated that pain relief was good at home and 43.6% during the last stay in a general hospital ward or ICU. Proportions were even lower for other symptoms with good relief at home reported by 31.8% and 32.2% in a general hospital ward or ICU. For perceived coordination of care, the general hospital wards and ICUs rank worst by far with only one-quarter of informants (25.6%) who said that the hospital worked well together with other services outside of the hospital. Hospice care was best rated for all three aspects. Eighty per cent of informants reported good pain relief, 67.2% good relief of other symptoms and 90% good coordination of care.

Accordingly, informants’ perceived overall satisfaction with patient care over the whole last year of life varied significantly depending on the care setting (table 3A). Informants were more likely to rate care provided by hospices (89.1%) and the specialist palliative home care team (86.8%) as ‘good’. Lower proportions of good ratings were given for the hospital palliative care units (68.5%), hospice services (68.4%), GPs (56.7%), care homes (50%), outpatient specialist physicians (47.6%) and nursing services (47.3%). They were least satisfied with the care provided in acute hospitals (general wards and ICUs), which was rated ‘good’ by 41.3% and received the most ‘bad’ (9.6%) ratings. We tested the differences in informants’ reports of satisfaction with acute hospital care versus all other health and social care practitioners (eg, hospital general ward/ICU vs hospice; table 3B). Differences were statistically significant in comparison with care provided by hospice (p=0.003, n=41), specialist palliative home care team (p<0.001, n=76), palliative care unit (p<0.001, n=297), home hospice service (p=0.033, n=12) and the GP (p=0.009, n=179).

Table 3A.

Informants’ overall satisfaction with care provided by health and social care practitioners in the last year of life

Care setting ‘Overall, how would you rate the care he/she got in the last year of life?’
Good Rather good Rather bad Bad
N* n (%) n (%) n (%) n (%)
Hospice 64 57 (89.1) 5 (7.8) 0 (0) 2 (3.1)
Specialist palliative home care team 129 112 (86.8) 13 (10.1) 2 (1.6) 2 (1.6)
Hospital (doctor): palliative care unit 89 61 (68.5) 22 (24.7) 3 (3.4) 3 (3.4)
Hospice service 19 13 (68.4) 6 (31.6) 0 (0) 0 (0)
General practitioner 282 160 (56.7) 75 (26.6) 36 (12.8) 11 (3.9)
Care home 72 36 (50) 26 (36.1) 10 (13.9) 0 (0)
Outpatient specialists 229 109 (47.6) 76 (33.2) 30 (13.1) 14 (6.1)
Nursing service 146 69 (47.3) 59 (40.4) 15 (10.3) 3 (2.1)
Hospital (doctor): general ward/ICU 208 86 (41.3) 76 (36.5) 26 (12.5) 20 (9.6)

*Number of patients for whom the corresponding question was responded.

ICU, intensive care unit.;

Table 3B.

Comparison of informants’ overall satisfaction with acute hospital care versus other health and social care practitioners

Care setting Acute hospital care (general ward/ICU) versus other care setting*
N P value†
Hospice 41 0.021‡
Specialist palliative home care team 76 <0.001‡
Hospital (doctor): palliative care unit 297 < 0.001§
Hospice service 12 0.268‡
General practitioner 179 0.069‡
Care home 52 0.543‡
Outpatient specialists 135 1.0‡
Nursing service 101 0.679‡

*Compares informant’s overall rating of the last hospital stay (doctor) in acute hospital setting (general ward or ICU) with the overall rating of every other care setting (eg, hospital general ward/ICU vs hospice).

†P values are adjusted using Bonferroni correction for multiple comparisons.

‡Wilcoxon signed-rank test for paired samples; comprises patients who experienced both forms of care.

§Mann-Whitney U test; comprises patients who either received care in a hospital general ward/ICU or in a palliative care unit.

ICU, intensive care unit.

Determinants of satisfaction with care in an acute hospital setting

Since care provided by general hospital wards and ICUs were rated worst compared with all other health and social care practitioners in the last year of life, we explored the determinants of satisfaction with care delivered in these hospital wards and units. Table 4 shows the results of the multivariable logistic regression analysis. Two variables were found to associate satisfaction with hospital care, while all other variables did not reach significance any more. First, the model indicated that the feeling that the patient was treated with respect and dignity by the doctor was associated with an almost 24-fold increase in the likelihood of being satisfied. Second, the impression that the hospital worked well together with other services outside the hospital was associated with an eightfold increase of being satisfied.

Table 4.

Logistic regression analysis of satisfaction with hospital care

Variable OR (95% CI) P value
Respect and dignity by the hospital doctor 23.80 (7.503 to 75.498) <0.001
Hospital worked well together with other services outside the hospital 8.37 (2.141 to 32.71) 0.002

Number of cases=124; cases correctly classified=85.5%; AUC=0.904

Discussion

Principal findings

This study is the first to our knowledge to set up a pragmatic PDSA cycle to improve regional care in the last year of life. Embedded in a regional health services research and development structure, we comprehensively analysed patient experiences with care in the last year of life from the bereaved relatives’ perspectives. For symptom control, both home as well as acute hospital setting was rated the most poorly. Hospitals were reported as the main player for diagnosing a progressive condition, for being involved in most transitions during the last year of life and being the most frequent place of death despite all home palliative and hospice services available in Cologne. Yet, satisfaction was lowest with care provided in an acute hospital setting, which was determined by the feeling of not being treated with respect and dignity and the impression that hospitals did not work well together with other services. Based on this first step of the PDSA cycle, two interventions were developed to continue the cycle.

Strengths and weaknesses

Most end-of-life care studies focus on a single care setting or are restricted to the dying phase, the last weeks of life or an underlying diagnosis. VOICES surveys include multiple settings to disaggregate findings according to specific setting type. Nevertheless, the instrument only evaluates the last 3 months of life.29 Within this study, we used an adapted German version of the VOICES-SF, which was expanded to the last 12 months of life and included the communication of a life-limiting diagnosis. This instrument enables the reconstruction of the entire last year of life of patients, irrespective of the underlying diagnosis, to assess the provision and quality of care of all health and social care practitioners involved in patients’ care. Our findings demonstrate the feasibility of collecting patient experiences reported by proxy-respondents across multiple care settings.

In Germany, there is no national register available that could be used to identify participants. Therefore, a population-based survey was not feasible, and the sample of decedents represented in this study was a purposive sample. Due to the recruitment strategy, patients from palliative care services are over-represented. Nevertheless, it appropriately reflected the age and gender distribution of people dying in the city of Cologne.41 42

We used a retrospective research design and asked bereaved relatives to act as proxies for the deceased patients. Relatives’ reports of care experiences after the death of a loved one are an important outcome measure to determine the quality of end-of-life care.14 This method is, of course, not without its limitations, especially in relation to memory, the impact of bereaved relatives’ feelings and the concordance between patient and proxy reports.43 44 A few studies examined these concerns and found that respondents were more accurate in recalling salient events and that satisfaction was mainly determined by service characteristics.45 46 A review on the validity of proxies’ responses found that the level of agreement was good on service evaluations and observable symptoms but lower for subjective symptoms such as pain.47 The retrospective approach has a long history in the conducting of end-of-life-care research. It bypasses the difficult task of identifying the terminally ill, avoids putting an additional burden on very sick participants and minimises missing data due to poor functional status. It furthermore creates a clear time frame for the purpose of comparing settings of care.48

The present study was conducted in Cologne, a city with 1 million inhabitants in Germany. It is not clear whether these results are transferable to other regions, rural as well as urban areas. Nevertheless, this study describes a pragmatic template based on patient experiences (PDSA cycle), which can be used to determine improvement priorities by other regions.

Comparison with existing literature

We found substantial differences in the reported satisfaction with care depending on the care setting. Informants were significantly more satisfied with quality of care provided in hospices compared with other care settings. Unfortunately, only a small number of patients will ever be admitted to hospice. In Germany, approximately 5% of patients, most of them cancer patients, die in hospice.49 In the present study, home was the place where people spent most of their time during their last year of life. Informants reported of insufficient symptom control when being cared for at home. Reasons may include a lack of home care provider expertise, limits on service allocation, fragmentation of multidisciplinary services and a considerable workload for many family practitioners with regards to home visit frequency and duration.3 10 50–52 Hospitals were the most frequent place of death, with the highest levels of dissatisfaction. These results are similar to those from other VOICES studies carried out in Western countries.1 3 32 34 Previous studies also identified psychosocial rather than physical factors leading to dissatisfaction with care. A lack of personal care and dignity has been reported to cause patients to feel ‘devalued’ or ‘dehumanised’.53 Being involved in decisions and discussing any worries were also described as predictors of satisfaction with hospital care.38 International studies have shown that there is still a discrepancy between current best practice recommendations and observed clinical reality, for example, with regard to the recognition of a patient’s transition into the last year of life, identifying a patient’s palliative care needs and aspects of shared decision making.54 Reasons for this may lie in the obvious life-saving culture of hospitals where the norm is to prevent death by whatever means are necessary. Furthermore, it may also be due to the increasing time and cost pressure put on hospitals without these institutions having established standards for patients who will die within the foreseeable future.55 Since almost 30% of all hospital patients are expected to be in their last year of life,56 our findings stress the importance of a further integration of palliative care into acute care hospitals. Strategies for integrating palliative care into a country’s healthcare system have been developed. In 2007, the WHO published a public health strategy that includes advice and guidelines for governments on priorities and how to implement national palliative care programmes.57 Furthermore, international experts in palliative care and cancer care formulated written statements regarding how integrated palliative care can be operationalised.58 In Germany, the evidence-based guideline ‘Palliative care’ was published in 2015 to promote quality end-of-life care by all healthcare professionals. This palliative care guideline presents the fundamental principles of palliative care which, in organ-specific guidelines, would be repetitive and/or not able to be dealt with in a comprehensive manner.7

Implications and future research

Based on the research of a regional working group, it was found that acute hospital care needs to be changed in order to improve care in the last year of life. Our regional priority for action now is improvement of care in acute hospitals, two new projects have started: first how to identify patients in their last year of life and initiate communication, and second, how to improve care during the dying phase according to quality indicators of our national guideline.7 59 Results and further improvement projects will be discussed in this regional working group, and repeating this survey in 2 years will be able to—hopefully—measure regional achievements. This study starting a PDSA cycle can therefore serve as an example for determining regional improvement priorities based on patient experiences.

Supplementary Material

Reviewer comments
Author's manuscript

Acknowledgments

This study was made possible by the collaborative efforts of health and social care practitioners in Cologne. We further thank everyone who contributed their time and expertise, in particular the study participants, the CoRe‑Net coapplicants and our research assistant Christian von Reeken.

Footnotes

RV and GD contributed equally.

Correction notice: This article has been corrected since it was published. The equal contributorship has been added in the paper.

Collaborators: The CoRe‑Net coapplicants: Professor Dr Christian Albus, Department of Psychosomatics and Psychotherapy, Faculty of Medicine (FM), University Hospital Cologne (UHC); Professor Dr Lena Ansmann, Department of Health Services Research, School of Medicine and Health Sciences, Carl von Ossietzky University Oldenburg; Professor Dr Frank Jessen, Department of Psychiatry and Psychotherapy, FM, UHC; Professor Dr Ute Karbach, Sociology in Rehabilitation, Faculty of Rehabilitation, Technical University Dortmund; Professor Dr Ludwig Kuntz, Department of Business Administration and Health Care Management, Faculty of Management, Economics and Social Sciences (FMESS), University of Cologne (UoC); Dr Ingrid Schubert, PMV Health Care Research Group, FM, UoC; Professor Dr Frank Schulz-Nieswandt, Institute for Sociology and Social Psychology, FMESS, UoC; and Professor Dr Stephanie Stock, Institute for Health Economics and Clinical Epidemiology, FM, UHC.

Contributors: RV and GD are joint first authors and act as guarantors. RV, JS and CR are principal investigators and responsible for the study design, project management, data analysis and dissemination. GD and NiS designed the survey instrument and collected data. GD cleaned and analysed data. SH, SP, NaS and HP contributed to study design, data analysis and dissemination. HP is principal investigator of CoRe-Net. NaS coordinates CoRe-Net. All authors read and approved the final manuscript.

Funding: This work was supported by the German Federal Ministry of Education and Research, grant number #01GY1606.

Competing interests: None declared.

Patient and public involvement: Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

Provenance and peer review: Not commissioned; externally peer reviewed.

Contributor Information

The CoRe‑Net Co‑applicants:

Christian Albus, Lena Ansmann, Frank Jessen, Ute Karbach, Ludwig Kuntz, Ingrid Schubert, Frank Schulz-Nieswandt, and Stephanie Stock

Data availability statement

Data are available on reasonable request.

Ethics statements

Patient consent for publication

Not required.

Ethics approval

All procedures for this study were approved by the Ethics Commission of the Faculty of Medicine of the University of Cologne, Germany (#17-188). Participants gave written informed consent before taking part.

References

  • 1. Hanratty B, Lowson E, Grande G, et al. Transitions at the end of life for older adults – patient, carer and professional perspectives: a mixed-methods study. Southampton, UK: NIHR Journals Library, 2014: 130. [PubMed] [Google Scholar]
  • 2. Addington-Hall JM, O'Callaghan AC. A comparison of the quality of care provided to cancer patients in the UK in the last three months of life in in-patient hospices compared with hospitals, from the perspective of bereaved relatives: results from a survey using the VOICES questionnaire. Palliat Med 2009;23:190–7. 10.1177/0269216309102525 [DOI] [PubMed] [Google Scholar]
  • 3. Bainbridge D, Seow H. Palliative care experience in the last 3 months of life: a quantitative comparison of care provided in residential Hospices, hospitals, and the home from the perspectives of bereaved caregivers. Am J Hosp Palliat Care 2018;35:456–63. 10.1177/1049909117713497 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4. Addington-Hall JM, Bruera E, Higginson IJ, et al. Research methods in palliative care. Oxford University Press, 2007. [Google Scholar]
  • 5. Bone AE, Gao W, Gomes B, et al. Factors associated with transition from community settings to hospital as place of death for adults aged 75 and older: a population-based mortality Follow-Back survey. J Am Geriatr Soc 2016;64:2210–7. 10.1111/jgs.14442 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6. Cardona-Morrell M, Kim J, Turner RM, et al. Non-beneficial treatments in hospital at the end of life: a systematic review on extent of the problem. Int J Qual Health Care 2016;28:456–69. 10.1093/intqhc/mzw060 [DOI] [PubMed] [Google Scholar]
  • 7. German Cancer Society, German Cancer Aid, AWMF . Palliative care for patients with incurable cancer short version 2.0: AWMF-registration number 128/001OL, 2019. Available: https://www.leitlinienprogramm-onkologie.de/leitlinien/palliativmedizin/ [Accessed 5 Nov 2019].
  • 8. Gomes B, Calanzani N, Gysels M, et al. Heterogeneity and changes in preferences for dying at home: a systematic review. BMC Palliat Care 2013;12:7. 10.1186/1472-684X-12-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Bertelsmann Stiftung . Palliative care: Available services do not (yet) meet patient needs - Expansion requires clear regulatory strategy: Spotlight Healthcare. Gütersloh: Bertelsmann Stiftung, 2015. [Google Scholar]
  • 10. Oishi A, Murtagh FEM. The challenges of uncertainty and interprofessional collaboration in palliative care for non-cancer patients in the community: a systematic review of views from patients, carers and health-care professionals. Palliat Med 2014;28:1081–98. 10.1177/0269216314531999 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Temel JS, Greer JA, Muzikansky A, et al. Early palliative care for patients with metastatic non-small-cell lung cancer. N Engl J Med 2010;363:733–42. 10.1056/NEJMoa1000678 [DOI] [PubMed] [Google Scholar]
  • 12. Smith TJ, Temin S, Alesi ER, et al. American society of clinical oncology provisional clinical opinion: the integration of palliative care into standard oncology care. J Clin Oncol 2012;30:880–7. 10.1200/JCO.2011.38.5161 [DOI] [PubMed] [Google Scholar]
  • 13. Bainbridge D, Seow H. Measuring the quality of palliative care at end of life: an overview of data sources. Health Aging Clin Care Elder 2016;8:9–15. [Google Scholar]
  • 14. Lendon JP, Ahluwalia SC, Walling AM, et al. Measuring experience with end-of-life care: a systematic literature review. J Pain Symptom Manage 2015;49:904–15. 10.1016/j.jpainsymman.2014.10.018 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15. Mayland C, Williams E, Ellershaw J. How well do current instruments using bereaved relatives’ views evaluate care for dying patients? Palliat Med 2008;22:133–44. 10.1177/0269216307085742 [DOI] [PubMed] [Google Scholar]
  • 16. Kupeli N, Candy B, Tamura-Rose G, et al. Tools measuring quality of death, dying, and care, completed after death: systematic review of psychometric properties. Patient 2019;12:183–97. 10.1007/s40271-018-0328-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17. The Economist Intelligence Unit . The 2015 quality of death index: ranking palliative care across the world, 2015. Available: https://eiuperspectives.economist.com/sites/default/files/2015%20EIU%20Quality%20of%20Death%20Index%20Oct%2029%20FINAL.pdf [Accessed 5 Nov 2019].
  • 18. Braithwaite J. Changing how we think about healthcare improvement. BMJ 2018;361:k2014. 10.1136/bmj.k2014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. Hämel K, Ewers M, Schaeffer D. Health care and regional disparities [Article in German]. Z Gerontol Geriatr 2013;46:323–8. [DOI] [PubMed] [Google Scholar]
  • 20. Taylor MJ, McNicholas C, Nicolay C, et al. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf 2014;23:290–8. 10.1136/bmjqs-2013-001862 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21. Pietrzak M, Paliszkiewicz J. Framework of strategic learning: the PDCA cycle. Management 2015;10:149–61. [Google Scholar]
  • 22. Pfaff H, Zeike S. Controlling Im Betrieblichen Gesundheitsmanagement: Das 7-Schritte-Modell. Wiesbaden: Gabler Verlag, 2019. [Google Scholar]
  • 23. Parker AM, Bruine de Bruin W, Fischhoff B. Maximizers versus satisficers: decision-making styles, competence, and outcomes. Judgment and Decision making 2007;2:342–50. [Google Scholar]
  • 24. Schwartz B, Ben‐Haim Y, Dacso C. What makes a good decision? robust Satisficing as a normative standard of rational decision making. J Theory Soc Behav 2011;41:209–27. 10.1111/j.1468-5914.2010.00450.x [DOI] [Google Scholar]
  • 25. Simon HA. Rational decision making in business organizations. The American economic review 1979;66:493–513. [Google Scholar]
  • 26. Simon HA. Making management decisions: the role of intuition and emotion. Academy Manage Perspect 1987;1:57–64. 10.5465/ame.1987.4275905 [DOI] [Google Scholar]
  • 27. Karbach U, Ansmann L, Scholten N, et al. [Report from an ongoing research project: The Cologne Research and Development Network (CoRe-Net) and the value-based approach to healthcare]. Z Evid Fortbild Qual Gesundhwes 2018;130:21–6. 10.1016/j.zefq.2017.11.005 [DOI] [PubMed] [Google Scholar]
  • 28. Strupp J, Hanke G, Schippel N, et al. Last year of life study Cologne (LYOL-C): protocol for a cross-sectional mixed methods study to examine care trajectories and transitions in the last year of life until death. BMJ Open 2018;8:e021211. 10.1136/bmjopen-2017-021211 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29. Hunt KJ, Richardson A, Darlington A-SE, et al. Developing the methods and questionnaire (VOICES-SF) for a national retrospective mortality follow-back survey of palliative and end-of-life care in England. BMJ Support Palliat Care 2019;9:e5. 10.1136/bmjspcare-2016-001288 [DOI] [PubMed] [Google Scholar]
  • 30. Office for National Statistics . National survey of bereaved people (voices): information paper. 13, 2016. [Google Scholar]
  • 31. Ross L, Neergaard MA, Petersen MA, et al. Measuring the quality of end-of-life care: development, testing, and cultural validation of the Danish version of views of informal carers’ evaluation of services-short form. Palliat Med 2018;32:804–14. 10.1177/0269216317740274 [DOI] [PubMed] [Google Scholar]
  • 32. Beccaro M, Caraceni A, Costantini M, et al. End-of-life care in Italian hospitals: quality of and satisfaction with care from the caregivers' point of view-results from the Italian survey of the dying of cancer. J Pain Symptom Manage 2010;39:1003–15. 10.1016/j.jpainsymman.2009.11.317 [DOI] [PubMed] [Google Scholar]
  • 33. Frey R, Williams L, Trussardi G, et al. The views of informal carers' evaluation of services (voices): toward an adaptation for the New Zealand bicultural context. Palliat Support Care 2017;15:67–76. 10.1017/S1478951516000146 [DOI] [PubMed] [Google Scholar]
  • 34. O'Sullivan A, Alvariza A, Öhlen J, et al. Bereaved family members’ satisfaction with care during the last three months of life for people with advanced illness. Healthcare 2018;6:E130. 10.3390/healthcare6040130 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35. Pungchompoo W, Suwan P, Kunapun S, et al. Experiences of symptoms and health service preferences among older people living with chronic diseases during the last year of life. Int J Palliat Nurs 2019;25:129–41. 10.12968/ijpn.2019.25.3.129 [DOI] [PubMed] [Google Scholar]
  • 36. Ingleton C, Morgan J, Hughes P, et al. Carer satisfaction with end-of-life care in Powys, Wales: a cross-sectional survey. Health Soc Care Community 2004;12:43–52. 10.1111/j.1365-2524.2004.00467.x [DOI] [PubMed] [Google Scholar]
  • 37. Donnelly S, Prizeman G, Coimín Diarmuid Ó, et al. Voices that matter: end-of-life care in two acute hospitals from the perspective of bereaved relatives. BMC Palliat Care 2018;17:117. 10.1186/s12904-018-0365-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Young AJ, Rogers A, Dent L, et al. Experiences of hospital care reported by bereaved relatives of patients after a stroke: a retrospective survey using the VOICES questionnaire. J Adv Nurs 2009;65:2161–74. 10.1111/j.1365-2648.2009.05081.x [DOI] [PubMed] [Google Scholar]
  • 39. Hunt KJ, Shlomo N, Addington-Hall J. End-of-life care and achieving preferences for place of death in England: results of a population-based survey using the VOICES-SF questionnaire. Palliat Med 2014;28:412–21. 10.1177/0269216313512012 [DOI] [PubMed] [Google Scholar]
  • 40. Hunt KJ, Shlomo N, Addington-Hall J. End-of-life care and preferences for place of death among the oldest old: results of a population-based survey using VOICES-Short form. J Palliat Med 2014;17:176–82. 10.1089/jpm.2013.0385 [DOI] [PubMed] [Google Scholar]
  • 41. Stadt Köln - Amt für Stadtentwicklung und Statistik . Kölner Statistische Nachrichten - 1/2019: Statistisches Jahrbuch Köln 2018. Köln, Stadt Köln, 2019. [Google Scholar]
  • 42. Stadt Köln - Amt für Stadtentwicklung und Statistik . Kölner Zahlenspiegel. Köln: Stadt Köln, 2018. [Google Scholar]
  • 43. Baddeley AD. Human memory: theory and practice. 515. Boston, MA: Allyn and Bacon, 1990. [Google Scholar]
  • 44. Bower GH. Mood and memory. Am Psychol 1981;36:129–48. 10.1037/0003-066X.36.2.129 [DOI] [PubMed] [Google Scholar]
  • 45. Fakhoury W, McCarthy M, Addington-Hall J. Determinants of informal caregivers’ satisfaction with services for dying cancer patients. Soc Sci Med 1996;42:721–31. 10.1016/0277-9536(95)00198-0 [DOI] [PubMed] [Google Scholar]
  • 46. Mathiowetz NA. Forgetting events in autobiographical memory: findings from a health care survey Rockville1988. Available: http://www.asasrms.org/Proceedings/papers/1988_029.pdf [Accessed 5 Nov 2019].
  • 47. McPherson CJ, Addington-Hall JM. Judging the quality of care at the end of life: can proxies provide reliable information? Soc Sci Med 2003;56:95–109. 10.1016/S0277-9536(02)00011-4 [DOI] [PubMed] [Google Scholar]
  • 48. Teno JM. Measuring end-of-life care outcomes retrospectively. J Palliat Med 2005;8:s-42–s-49. 10.1089/jpm.2005.8.s-42 [DOI] [PubMed] [Google Scholar]
  • 49. Dasch B, Blum K, Gude P, et al. Place of death: trends over the course of a decade: a population-based study of death certificates from the years 2001 and 2011. Dtsch Arztebl Int 2015;112:496–504. 10.3238/arztebl.2015.0496 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50. Bainbridge D, Brazil K, Krueger P, et al. Evaluating program integration and the rise in collaboration: case study of a palliative care network. J Palliat Care 2011;27:270–8. 10.1177/082585971102700403 [DOI] [PubMed] [Google Scholar]
  • 51. Bee PE, Barnes P, Luker KA. A systematic review of informal caregivers’ needs in providing home-based end-of-life care to people with cancer. J Clin Nurs 2009;18:1379–93. 10.1111/j.1365-2702.2008.02405.x [DOI] [PubMed] [Google Scholar]
  • 52. Pochert M, Voigt K, Bortz M, et al. The workload for home visits by German family practitioners: an analysis of regional variation in a cross-sectional study. BMC Fam Pract 2019;20:3. 10.1186/s12875-018-0891-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53. Rogers A, Karlsen S, Addington-Hall J. 'All the services were excellent. it is when the human element comes in that things go wrong': dissatisfaction with hospital care in the last year of life. J Adv Nurs 2000;31:768–74. 10.1046/j.1365-2648.2000.01347.x [DOI] [PubMed] [Google Scholar]
  • 54. Milnes S, Orford NR, Berkeley L, et al. A prospective observational study of prevalence and outcomes of patients with gold standard framework criteria in a tertiary regional Australian hospital. BMJ Support Palliat Care 2019;9:92–9. 10.1136/bmjspcare-2015-000864 [DOI] [PubMed] [Google Scholar]
  • 55. Gott M, Ingleton C, Bennett MI, et al. Transitions to palliative care in acute hospitals in England: qualitative study. BMJ 2011;342:d1773. 10.1136/bmj.d1773 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56. Clark D, Armstrong M, Allan A, et al. Imminence of death among hospital inpatients: prevalent cohort study. Palliat Med 2014;28:474–9. 10.1177/0269216314526443 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57. Stjernswärd J, Foley KM, Ferris FD. The public health strategy for palliative care. J Pain Symptom Manage 2007;33:486–93. 10.1016/j.jpainsymman.2007.02.016 [DOI] [PubMed] [Google Scholar]
  • 58. Payne S, Hughes S, Wilkinson J, et al. Recommendations on priorities for integrated palliative care: transparent expert consultation with international leaders for the InSuP-C project. BMC Palliat Care 2019;18:32. 10.1186/s12904-019-0418-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59. The Gold Standards Framework . Acute Hospital programme: summary of evidence of effectiveness, 2016. Available: http://www.goldstandardsframework.org.uk/cd-content/uploads/files/GSF%20Summary%20of%20EValuation%20%20Acute%20Hospitals%20Sept%202016%281%29.pdf [Accessed 5 Nov 2019].

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Reviewer comments
Author's manuscript

Data Availability Statement

Data are available on reasonable request.


Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES