Abstract
Objective
While there are limited studies addressing palliative care quality in China, the availability of an effective set of care quality indicators is scarce. This study aimed to develop a comprehensive set of quality indicators for palliative care in China.
Methods
Conducting a systematic literature search across databases and guideline websites from inception to October 2020, combined with qualitative interviews, we established a preliminary pool of indicators. Subsequently, two rounds of Delphi expert consultation surveys were administered to 19 multidisciplinary experts (specializing in clinical nursing/medicine, nursing/medicine management, and health care administration, as well as those engaged in teaching and research) from 12 provinces in Mainland China (three each from North, East, and South China, and four from Central China) via email from March to June 2021. The analytic hierarchy process was employed to determine indicator weights.
Results
Both rounds of expert consultation yielded a 100% positive coefficient, with expert authority coefficient values of 0.91 and 0.93, respectively. Kendall coefficient of concordance values for the two rounds were 0.148 and 0.253 (P < 0.001), indicating consensus among experts. Consequently, 71 quality indicators deemed important in the Chinese palliative care setting were identified, comprising 22 structure indicators, 35 process indicators, and 14 outcome indicators.
Conclusions
This study established an evidence-based set of quality indicators, addressing previously unmet needs and providing a novel approach to assessing and monitoring palliative care quality. Furthermore, ongoing refinement and integration with the evolving social context are warranted.
Keywords: Quality indicators, Palliative care, Delphi technique, Analytic hierarchy process
Introduction
Palliative care is an integrated approach involving physical, psychological, social, and spiritual support to help patients with serious illnesses and their families improve their quality of life.1 With an increasing number of people diagnosed with progressive diseases, palliative care has a high potential for demand.2 These patients’ physical, psychological, social, and spiritual needs differ from those of patients with general illnesses. Consequently, they need additional care services from others.
Quality indicators are standardized, evidence-based measures and can provide targeted care content3 to provide patients and health workers with relevant data to systematically monitor and improve the quality of medical care and support services.4, 5, 6, 7 The most commonly used theoretical framework in studies on quality indicators was developed by Avedis Donabedian, and it includes service structures, processes, and outcomes of care.8 The three aspects both complement and influence one another and can be more comprehensive and reflect care details. High-quality service structures and processes are likely to contribute to high-quality outcomes.9
In recent years, studies on quality indicators for palliative or end-of-life care have been increasing globally. However, these studies have some weaknesses. First, most focus on specific disease groups such as heart disease, cancer, end-stage liver disease, etc.,10, 11, 12, 13 which lack generality. Second, Virdun14 integrated national quality indicators and policies from 15 leading countries in adult end-of-life care; however, the studies particularly focus on outcome indicators (account for 62% of total quality indicators). Third, McCorry7 developed the first set of quality indicators (comprising seven structural, 21 process, and two outcome indicators) to evaluate the quality of palliative-care day service; however, this indicator set is not sufficiently comprehensive and some indicators are not suitable for China's national conditions and are difficult to implement in clinical practice.
In mainland China, Yang developed 36 indicators aimed at patients with cancer based on the theory of comfort care in 2018.15 However, these quality indicators were restricted to process, with no indicators for the necessary resources and how to evaluate patient-care outcomes. In 2019, Liu developed 38 indicators for palliative care in mainland China based on the structure-process-outcome theory.16 However, these indicators predominantly focused on outcomes. Therefore, more comprehensive quality indicators for palliative care in mainland China are required.
Despite the increasing attention paid to quality indicators for palliative care in China, studies on care-quality indicators for palliative care remain scarce due to the influence of traditional culture and the deficiency of relevant policies.17 As the health care professionals with the closest contact with patients and caregivers in palliative care, nurses are responsible for enhancing care quality. Integrating the Donabedian model with the Chinese context, this study aimed to develop a quality-indicator system for palliative care for hospitals and nurses in China.
Methods
Initial construction of potential indicators
A set of potential indicators was drafted according to the following steps: First, quality indicators were extracted from a literature review. The search terms were ‘palliative care/end-of-life/terminal ill/advanced cancer’ and ‘quality indicator/nursing quality/care quality/quality assessment/quality of life/quality management’. From inception to December 2020, a comprehensive search of the literature and guidelines for quality indicators in palliative care was performed, including on the Embase, PubMed, CINAHL, CNKI, WeiPu, and WanFang databases and those of the National Guideline Clearinghouse, Registered Nurses Association of Ontario, and National Institute for Health and Clinical Excellence. Second, qualitative interviews with six palliative-care managers (who were excluded from the Delphi consultation) and six clinical nurses in palliative wards were conducted to explore the understanding of quality indicators from a first-hand perspective and to obtain valuable advice for establishing an effective set of quality indicators. Third, before we began our Delphi process, we held a meeting with our research team to refine the potential quality indicators. Six interdisciplinary experts in nursing education, nursing quality control, and health education discussed the importance of the items, which were presented based on Donabedian's theory. Consequently, three primary (ie, structure, process, and outcome), 15 secondary, and 66 tertiary indicators were developed for the Delphi expert consultation.
Expert selection
We selected 19 experts (major in clinical nursing/medicine, nursing/medicine management, and health care administration, and those who specialized in teaching and research) from 12 provinces in China (three from North China, three from East China, two from South China, and four from Central China) for consultation. The inclusion criteria for experts were as follows: at least eight years of working experience in palliative or end-of-life care; a bachelor's degree or above; an intermediate title of nursing or above; willingness to provide ideas and advice about quality indicators for palliative care.
Data collection
The survey comprised three parts: (1) description of the questionnaire. In this part, we described the purpose of the research, purpose of expert consultation, and requirements for filling in the form, the time limit for returning questionnaires, and requirements for anonymity; (2) experts' basic information. In this part, we mainly collected data on the experts' gender, age, educational background, position, title, mentor qualification, and working field; and (3) evaluation questionnaire. This part primarily aimed to ask the experts to evaluate the importance and expression of each quality indicator to indicate its significance using 5-point Likert scales ranging from extremely important (5 points) to extremely unimportant (1 point).
The Delphi procedure
There were two ways to maintain anonymity. First, we instructed survey respondents to maintain anonymity by not writing their name or any special signs on the surveys. Second, the corresponding author distributed the survey to all experts and asked them to return the survey by email to the first author. The first author was only aware that the sample consisted of expert consultations and knew nothing about the individual experts themselves. Therefore, the first author could not recognize the experts through the basic information provided in the survey.
Two rounds of Delphi consultation took place from March to June 2021 to reach an agreement on the importance of each indicator. All 19 experts responded in both rounds. In the first round, the experts rated the importance of the 66 potential indicators on a five-point Likert scale. Each indicator was clearly defined to ensure that the experts fully understood them. We also advised the experts to suggest revising, adding, or deleting specific indicators. We compiled the results of the two rounds and the comments received.
Our research team comprised six interdisciplinary experts in nursing education, nursing quality control, and health education, who were excluded from the expert consultation. Following our research team's discussion and revision, the second round of indicators was developed. In round two, any modifications based on round one were highlighted and the reasons for modification were presented. The same 19 experts were invited to participate in the second round to complete the questionnaires.
Determining consensus
After each Delphi round, the mean, median, standard deviation (SD), and coefficient of variation (CV) were calculated for every indicator. The cutoff points for deletion were an importance rating below 4 (5-point Likert scale) and a CV greater than 0.25, and deletion was arrived at by a joint discussion of the experts. If an expert recommended deleting a quality indicator, it would be modified or deleted following discussion by the expert team.
Data analysis
The results of the surveys were double-checked for accuracy and inputted into Statistical Product and Service Solutions (SPSS) 22.0. Descriptive data were used to analyze the experts' demographic characteristics and authoritative coefficients as well as the concentration of and variations in their opinions.
Active coefficients
The response rate measured the experts’ active coefficients in each round. A higher active coefficient indicated that the expert was more concerned about the research topic.
Authoritative coefficients
The experts' familiarity with the problem (Cs) and basis for judgment (Ca) determined their authoritative coefficient (Cr), that is, Cr = (Cs + Ca)/2. The acquaintance level was categorized as five possible answers that were assigned scores of 0.9 (very familiar), 0.7 (relatively familiar), 0.5 (generally familiar), 0.3 (not very familiar), and 0.1 (not familiar).
Concentration and variation
The CV and Kendall's coefficient of concordance (Kendall's W) reflected the degree of coordination among the experts' opinions. A smaller CV value and a larger value of Kendall's W indicated a higher degree of coordination among the experts’ opinions.
Relative importance of quality indicators
The analytic hierarchy process (AHP) was used to determine the relative importance of each quality indicator based on the result of the second Delphi round. We conducted a pairwise comparison (Aij−Aji) of indicators for each level and then determined the corresponding values on the Saaty 1–9 scale (Table 1). The Saaty scale values for each level of indicators were entered into Yaahp 10.3, which is a software that calculates each indicator's weight and performs a consistency-test value. The combined weight is the result of multiplying the weight of the initial indicator by that of the upper indicator. If the random consistency ratio (CR) was less than 0.1, the comparisons were considered acceptable through a consistency test. A CR value greater than 0.1 indicated inconsistent judgments and suggested that the pairwise comparisons should be revised. A higher weighting demonstrated a more important indicator.
Table 1.
The Saaty scale.
| Scale | Mean difference | Interpretation |
|---|---|---|
| 1 | Aij−Aji = 0 | Both factors are equally important |
| 3 | 0.25 < Aij−Aji ≤ 0.5 | The first factor is slightly more important than the second |
| 5 | 0.75 < Aij−Aji ≤ 1 | The first factor is notably more important than the second |
| 7 | 1.25 < Aij−Aji ≤ 1.5 | The first factor is more important than the second |
| 9 | Aij−Aji > 1.75 | The first factor is considerably more important than the second |
Ethical considerations
This project was approved by the Ethics Committee of Hunan Cancer Hospital (IRB No. 2021-02). We informed participants of the information about the study and that they could withdraw whenever they wanted. We have obtained informed consent orally or in writing from all participants. Participants were assured of the confidentiality of their data and were granted anonymity in all documents related to the research.
Results
Experts’ characteristics
The 19 participants’ characteristics are shown in Table 2. Ten (52.6%) of them had worked for more than 30 years; 12 (63.2%) had a senior title; and 11 (57.9%) were master tutors. The experts were from 12 different provinces (or municipalities) and 16 different workplaces.
Table 2.
Characteristics of experts (N = 19).
| Characteristics | n (%) |
|---|---|
| Gender | |
| Male | 1 (5.3) |
| Female | 18 (94.7) |
| Age | |
| 31–40 years | 3 (15.8) |
| 41–50 years | 9 (47.4) |
| 51–60 years | 6 (31.6) |
| 60 years above | 1 (5.3) |
| Working experience | |
| Below 10 years | 1 (5.3) |
| 10–20 years | 3 (15.8) |
| 21–30 years | 5 (26.3) |
| 30 years above | 10 (52.6) |
| Title | |
| Intermediate level | 3 (15.8) |
| Associate senior level | 4 (21.1) |
| Senior level | 12 (63.2) |
| Educational background | |
| Bachelor's degree | 10 (52.6) |
| Master's degree | 7 (36.8) |
| Doctoral degree | 2 (10.5) |
| Mater's mentorship | |
| Not qualified | 8 (42.1) |
| Qualified | 11 (57.9) |
| Field of specialization | |
| Clinical nursing/medicine | 5 (26.3) |
| Nursing/medicine management | 12 (63.2) |
| Health care administration department | 1 (5.3) |
| Specialized in teaching and research | 1 (5.3) |
Experts’ active and authoritative coefficients
All 19 experts provided valid, complete responses in both Delphi rounds. Thus, the active coefficient was 100%. The Ca and Cs for the first round were 0.96 and 0.85, respectively, whereas those for the second round were 0.96 and 0.90, respectively. Consequently, the average Cr was 0.92.
Concentration and coordination in experts' opinions
In the first round, the mean score for importance ranged from 4.16 to 5.00, with CVs ranging from 0.00 to 0.26. The percentage of the full score for potential indicators ranged from 53% to 100%. In the second round, the mean score for importance ranged from 4.68 to 5.00, with CVs ranging from 0.00 to 0.16. The percentage of the full score for potential indicators ranged from 79% to 100%. Kendall's coefficient of concordance values for the two rounds were 0.148 and 0.253 (P < 0.001), respectively, reflecting consensus among the experts.
Indicator modification
After the first round, three primary, 15 secondary, and 71 tertiary indicators were generated for the second round of consultation. The results of the second round are presented in Table 3. One secondary and four tertiary indicators were revised after the second round. The final set included three primary, 15 secondary, and 71 tertiary indicators.
Table 3.
The result of the second round of Delphi consultation.
| Indicators | Median | Mean (SD) | Variation | Percentage of the full score | Outcome | Modified indicator |
|---|---|---|---|---|---|---|
| Structure | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| I-1 Human resource | 5.00 | 4.79 (0.69) | 0.14 | 90 | Retained | |
| I-1-1 Physician configuration meets national standards | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| I-1-2 Nurse configuration meets national standards | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| I-1-3 Assistant nurse configuration meets national standards | 5.00 | 4.89 (0.31) | 0.06 | 90 | Retained | |
| I-1-4 Appropriate staffing: pharmacists, dietitians, therapists, social workers, volunteers based on patient needs | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| I-1-5 Regular training for multidisciplinary teams on palliative care | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| I-2 Physical environment | 5.00 | 4.89 (0.31) | 0.06 | 90 | Retained | |
| I-2-1 Ward design meets fire safety and accessibility standards | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| I-2-2 Ward environment meets infection control standards | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| I-2-3 Ward environment satisfies sensory comfort needs | 5.00 | 4.89 (0.31) | 0.06 | 90 | Retained | |
| I-2-4 Ward environment satisfies psychological comfort needs | 5.00 | 4.79 (0.52) | 0.11 | 84 | Retained | |
| I-2-5 Ward environment addresses individual patient needs (eg, dining room, farewell room) | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| I-3 Instruments and equipment | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| I-3-1 Equipment meets national standards for basic medical care | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| I-3-2 Adequate analgesia pumps based on patient needs | 5.00 | 4.79 (0.41) | 0.09 | 79 | Retained | |
| I-3-3 Relevant comfort care equipment available (eg, bathing aids) | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| I-4 Management standards | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| I-4-1 Clear job descriptions and team regulations | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| I-4-2 Established informed consent process for ward admission | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| I-4-3 Evidence-based transfer criteria | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| I-4-4 Process & time frame for patient information | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| I-4-5 Dedicated nurse for coordination & referrals | 5.00 | 4.68 (0.73) | 0.16 | 79 | Retained | |
| I-5 Morality, ethics, and law | 5.00 | 4.89 (0.31) | 0.06 | 89 | Retained | |
| I-5-1 Respecting patient's wishes & decisions | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| I-5-2 Safeguarding patient privacy & rights | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| I-5-3 Respecting wishes & decisions of caregivers & surrogates | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| I-5-4 Professional ethics observed by a multidisciplinary team | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| Process indicators | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| II-1 Symptom management | 5.00 | 5.00 (0.00) | 0.00 | 100 | Revised | II-1 Symptom control |
| II-1-1 Dynamic pain assessment & management | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| II-1-2 Dynamic dyspnea assessment & management | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| II-1-3 Dynamic delirium assessment & management | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| II-1-4 Dynamic Gastrointestinal Symptom Assessment & Management | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| II-1-5 Dynamic sleep disorder assessment & management | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| II-1-6 Dynamic malnutrition assessment & management | 5.00 | 4.89 (0.31) | 0.06 | 89 | Retained | |
| II-1-7 Dynamic fatigue assessment & management | 5.00 | 4.89 (0.31) | 0.06 | 89 | Retained | |
| II-1-8 Dynamic cough & sputum assessment & management | 5.00 | 4.89 (0.31) | 0.06 | 89 | Retained | |
| II-2 Comfortable care | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| II-2-1 Assist the caregiver to change the patient's position regularly | 5.00 | 5.00 (0.00) | 0.00 | 100 | Revised | II-2-1 Assist caregiver in regular patient positioning |
| II-2-2 Maintain patient cleanliness (eg, bed hair wash, oral care) | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| II-2-3Proper patient pipeline care (eg, feeding tube, catheter) | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| II-3 Psychological care | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| II-3-1 Screen for patient psychological distress | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| II-3-2 Assess the degree of patient's psychological distress or not | 5.00 | 4.95 (0.22) | 0.05 | 95 | Revised | II-3-2 Assess the degree of patient's psychological distress |
| II-3-3 Psychological intervention for moderate/severe distress | 5.00 | 4.79 (0.61) | 0.13 | 89 | Retained | |
| II-3-4 According to the patient's condition, follow-up patients with psychological distress | 5.00 | 4.89 (0.45) | 0.09 | 95 | Revised | II-3-4 Follow-up for patients with psychological distress |
| II-3-5 Document & continuously improve psychological distress symptoms | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| II-4 Social support | 5.00 | 4.89 (0.31) | 0.06 | 89 | Retained | |
| II-4-1 Assess patient's social support status | 5.00 | 4.89 (0.31) | 0.06 | 89 | Retained | |
| II-4-2 Develop & implement a comprehensive social support plan | 5.00 | 4.74 (0.71) | 0.15 | 84 | Retained | |
| II-4-3 Guide patients to choose social support methods by disease stage | 5.00 | 4.74 (0.55) | 0.12 | 79 | Retained | |
| II-4-4 Guide patients/families to actively seek social support | 5.00 | 4.89 (0.45) | 0.09 | 95 | Retained | |
| II-4-5 Engage social workers & volunteers in patient care | 5.00 | 4.89 (0.45) | 0.09 | 95 | Retained | |
| II-4-6 Encourage family & friends to care for & accompany patients | 5.00 | 4.89 (0.31) | 0.06 | 89 | Retained | |
| II-5 Spiritual care | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| II-5-1 Assess patient's spiritual needs & support status | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| II-5-2 Respect patients’ beliefs and values | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| II-5-3 Assist patients in fulfilling their wishes | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| II-5-4 Guide patients to seek life's meaning via life review | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| II-6 Caregiver support | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| II-6-1 Assess the needs & support status of caregivers | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| II-6-2 Offer respite services for family members | 5.00 | 4.89 (0.45) | 0.09 | 95 | Retained | |
| II-6-3 Offer grief support for caregivers | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| II-6-4 Guide caregivers on patient care | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| II-6-5 Guide caregivers on self-health | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| II-7 Communication and information | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| II-7-1 Discuss patient's condition & prognosis with patients & family | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| II-7-2 Discuss the end-stage treatment plan with patients & family | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| II-7-3 Provide disease-related information to patients & family | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| II-7-4 Provide death education for patients and family members | 5.00 | 5.00 (0.00) | 0.00 | 100 | Revised | Ⅱ-7-4 Educate patients & family on end-of-life matters |
| Outcome indicators | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| III-1 Satisfaction rate | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| III-1-1 Patient satisfaction with nursing services | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| III-1-2 Family & surrogate decision-maker satisfaction with nursing care | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| III-1-3 Patient's willingness to recommend palliative care services | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| III-1-4 Multi-disciplinary team member satisfaction | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| III-2 Quality of life | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| III-2-1 Relief or elimination of patient's physical discomfort | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| III-2-2 Relief of patient's negative emotions | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| III-2-3 Effective social support received by patients | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| III-2-4 Fulfillment of patients' spiritual needs | 5.00 | 4.95 (0.22) | 0.05 | 95 | Retained | |
| III-2-5 Alignment of treatment & care with patient's wishes | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| III-3 Adverse events | 5.00 | 4.74 (0.55) | 0.12 | 79 | Retained | |
| III-3-1 No accidental extubation | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| III-3-2 No falls | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| III-3-3 No pressure ulcers | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| III-3-4 No incidents of suicide or self-harm | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained | |
| III-3-5 No complaints or other adverse events | 5.00 | 5.00 (0.00) | 0.00 | 100 | Retained |
SD, standard deviation.
The relative importance of quality indicators
Following the AHP procedure, the experts assigned each indicator a relative importance weight (Table 4). The structure, process, and outcome weights were 0.30, 0.40, and 0.31, respectively, with CR = 0.01. Among the secondary indicators, the composed weights ranged from 0.07 to 0.51, with CR ranging from 0.01 to 0.02. Among the tertiary indicators, the composed weights ranged from 0.00 to 0.04, with CR ranging from 0.00 to 0.01. The secondary indicators for the structure were as follows: human resources; physical environment; instrument and equipment; management standards; and morality, ethics, and law, among which ‘morality, ethics, and law’ carried the highest weight and ‘instrument and equipment’ the lowest weight. The secondary indicators for process quality comprised the following: symptom control; comfortable care; psychological care; social support; spiritual care; caregiver support; and communication and information. Of these, symptom control carries the highest weight. The secondary indicators for outcome comprised the following: satisfaction rate, quality of life, and adverse events, of which quality of life carried the highest weight. Of these, quality of life carries the highest weight.
Table 4.
The weight of each indicator.
| First-level indicators (initial weight) | Second-level indicators (initial weight) | Third-level indicators (initial weight) | Composed weight |
|---|---|---|---|
| I Structure quality (0.30) | I-1 Human resource (0.23) | I-1-1 Physician configuration meets national standards (0.25) | 0.02 |
| I-1-2 Nurse configuration meets national standards (0.25) | 0.02 | ||
| I-1-3 Assistant nurse configuration meets national standards (0.11) | 0.01 | ||
| I-1-4 Appropriate staffing: pharmacists, dietitians, therapists, social workers, volunteers based on patient needs (0.15) | 0.01 | ||
| I-1-5 Regular training for multidisciplinary team on palliative care (0.25) | 0.02 | ||
| I-2 Physical environment (0.13) | I-2-1 Ward design meets fire safety and accessibility standards (0.33) | 0.01 | |
| I-2-2 Ward environment meets infection control standards (0.21) | 0.01 | ||
| I-2-3 Ward environment satisfies sensory comfort needs (0.14) | 0.01 | ||
| I-2-4 Ward environment satisfies psychological comfort needs (0.11) | 0.00 | ||
| I-2-5 Ward environment addresses individual patient's needs (eg, dining room, farewell room) (0.21) | 0.01 | ||
| I-3 Instrument and equipment (0.10) | I-3-1 Equipment meets national standards for basic medical care (0.40) | 0.01 | |
| I-3-2 Adequate analgesia pumps based on patient needs (0.20) | 0.01 | ||
| I-3-3 Relevant comfort care equipment available (eg, bathing aids) (0.40) | 0.01 | ||
| I-4 Management standards (0.24) | I-4-1 Clear job descriptions and team regulations (0.23) | 0.02 | |
| I-4-2 Established informed consent process forward admission (0.08) | 0.01 | ||
| I-4-3 Evidence-based transfer criteria (0.23) | 0.02 | ||
| I-4-4 Process & time frame for patient information (0.23) | 0.02 | ||
| I-4-5 Dedicated nurse for coordination & referrals (0.23) | 0.02 | ||
| I-5 Morality, ethics, and law (0.30) | I-5-1 Respecting patient's wishes & decisions (0.25) | 0.02 | |
| I-5-2 Safeguarding patient privacy & rights (0.25) | 0.02 | ||
| I-5-3 Respecting wishes & decisions of caregivers & surrogates (0.25) | 0.02 | ||
| I-5-4 Professional ethics observed by a multidisciplinary team (0.25) | 0.02 | ||
| II Process quality (0.40) | II-1 Symptom control (0.28) | II-1-1 Dynamic pain assessment & management (0.18) | 0.02 |
| II-1-2 Dynamic dyspnea assessment & management (0.18) | 0.02 | ||
| II-1-3 Dynamic delirium assessment & management (0.18) | 0.02 | ||
| II-1-4 Dynamic gastrointestinal symptom assessment & management (0.12) | 0.01 | ||
| II-1-5 Dynamic sleep disorder assessment & management (0.12) | 0.01 | ||
| II-1-6 Dynamic malnutrition assessment & management (0.08) | 0.01 | ||
| II-1-7 Dynamic fatigue assessment & management (0.08) | 0.01 | ||
| II-1-8 Dynamic cough & sputum assessment & management (0.08) | 0.01 | ||
| II-2 Comfortable care (0.20) | II-2-1 Assist caregiver in regular patient positioning (0.40) | 0.03 | |
| II-2-2 Maintain patient cleanliness (eg, bed hair wash, oral care) (0.20) | 0.02 | ||
| II-2-3 Proper patient pipeline care (eg, feeding tube, catheter) (0.40) | 0.03 | ||
| II-3 Psychological care (0.12) | II-3-1 Screen for patient psychological distress (0.25) | 0.01 | |
| II-3-2 Assess the degree of patient's psychological distress (0.25) | 0.01 | ||
| II-3-3 Psychological intervention for moderate/severe distress (0.11) | 0.01 | ||
| II-3-4 Follow-up for patients with psychological distress (0.15) | 0.01 | ||
| II-3-5 Document & continuously improve psychological distress symptoms (0.25) | 0.01 | ||
| II-4 Social support (0.07) | II-4-1 Assess patient's social support status (0.20) | 0.01 | |
| II-4-2 Develop & implement a comprehensive social support plan (0.10) | 0.00 | ||
| II-4-3 Guide patients to choose social support methods by disease stage (0.10) | 0.00 | ||
| II-4-4 Guide patients/families to actively seek social support (0.20) | 0.01 | ||
| II-4-5 Engage social workers & volunteers in patient care (0.20) | 0.01 | ||
| II-4-6 Encourage family & friends to care for & accompany patients (0.20) | 0.01 | ||
| II-5 Spiritual care (0.11) | II-5-1 Assess patient's spiritual needs & support status (0.25) | 0.01 | |
| II-5-2 Respect patients’ beliefs and values (0.25) | 0.01 | ||
| II-5-3 Assist patients in fulfilling their wishes (0.25) | 0.01 | ||
| II-5-4 Guide patients to seek life's meaning via life review (0.25) | 0.01 | ||
| II-6 Caregiver support (0.09) | II-6-1 Assess the needs & support status of caregivers (0.33) | 0.01 | |
| II-6-2 Offer respite services for family members (0.11) | 0.00 | ||
| II-6-3 Offer grief support for caregivers (0.19) | 0.01 | ||
| II-6-4 Guide caregivers on patient care (0.19) | 0.01 | ||
| II-6-5 Guide caregivers on self-health (0.19) | 0.01 | ||
| II-7 Communication and information (0.13) | II-7-1 Discuss patient's condition & prognosis with patients & family (0.20) | 0.01 | |
| II-7-2 Discuss the end-stage treatment plan with patients & family (0.20) | 0.01 | ||
| II-7-3 Provide disease-related information to patients & family (0.20) | 0.01 | ||
| II-7-4 Educate patients & family on end-of-life matters (0.40) | 0.02 | ||
| III Outcome quality (0.31) | III-1 Satisfaction rate (0.29) | III-1-1 Patient satisfaction with nursing services (0.25) | 0.02 |
| III-1-2 Family & surrogate decision-maker satisfaction with nursing care (0.25) | 0.02 | ||
| III-1-3 Patient's willingness to recommend palliative care services (0.25) | 0.02 | ||
| III-1-4 Multi-disciplinary team member satisfaction (0.25) | 0.02 | ||
| III-2 Quality of life (0.51) | III-2-1 Relief or elimination of patient's physical discomfort (0.25) | 0.04 | |
| III-2-2 Relief of patient's negative emotions (0.25) | 0.04 | ||
| III-2-3 Effective social support received by patients (0.13) | 0.02 | ||
| III-2-4 Fulfillment of patients' spiritual needs (0.13) | 0.02 | ||
| III-2-5 Alignment of treatment &care with patient's wishes (0.25) | 0.04 | ||
| III-3 Adverse event (0.20) | III-3-1 No accidental extubation (0.20) | 0.01 | |
| III-3-2 No falls (0.20) | 0.01 | ||
| III-3-3 No pressure ulcers (0.20) | 0.01 | ||
| III-3-4 No incidents of suicide or self-harm (0.20) | 0.01 | ||
| III-3-5 No complaints or other adverse events (0.20) | 0.01 |
Discussion
Statement of principal findings
Evaluating the complex quality of palliative care is imperative for clinicians to adeptly and routinely perform quality improvement. Based on the learning experience from previous studies and local policies, we employed an evidence-based approach to develop a set of quality indicators for palliative care in the context of mainland China. This set of quality indicators included 22 structure, 35 process, and 14 outcome-quality indicators. These indicators cover all aspects of quality in palliative care and may apply to a palliative-care ward.
Interpretation within the context of the broader literature
The present study organized a systematic research, qualitative interviews, and two rounds of Delphi questionnaire surveys. Through a search of the literature and guidelines, we obtained a comprehensive understanding of the strengths and limitations of these quality indicators and a guide to building a more scientific set of quality indicators. The qualitative interviews revealed the required quality indicators from the perspectives of clinical nurses and nurse managers, who are key actors in palliative care. Using the structure-process-outcome theory as the framework, the research team divided the indicators extracted from the literature search and qualitative study into three types: structure, process, and outcome. Through two rounds of Delphi surveys, we summarized 19 experts' views, confirmed the phrasing and expression of indicators, and finally reached a consensus on 71 quality indicators that were deemed appropriate for evaluating and monitoring the quality of palliative care. These key quality indicators suit the Chinese socio-economic, legislative, and cultural contexts. The proposed indicators are intended to develop an environment to assure and improve the quality of palliative care in mainland China.
Ethical issues in palliative care include maintaining a patient's autonomy and dignity, respecting the patient's choice, and informing the patient of the true condition and prognosis.18 When nurses cannot appropriately address ethical issues such as when they must provide care that is inconsistent with their personal or professional values, moral distress may occur. Although some studies have explored how to develop moral education, moral distress remains a neglected issue in most palliative education programs19, 20, 21, 22; in the future, researchers can consider the scientific framework and, based on previous studies, develop an ethical education program that is appropriate for Chinese conditions.20
Indeed, symptom management plays a critical role in palliative care and is the basis for providing other aspects of services. However, it is often not systematically performed in nursing practice or routinely incorporated into the clinical decision-making processes.23 Andreas24 et al have conducted a systematic review to assess the efficacy and safety of pharmacological and non-pharmacological interventions for palliative symptom control, while clinical-decision support systems (CDSSs) have become increasingly crucial for assisting health workers in choosing the most efficient method to manage symptom.25 However, further research utilizing multi-disciplinary methods is required to develop guideline-based actions and verify the impact on patients’ symptom control.
Quality of life is a multidimensional concept used to evaluate the level of human-health indicators, and it involves physical, mental, and social health as well as environmental aspects.26 Quality of life has now become the most important outcome measure in studies, and many studies have explored intervention strategies to improve patients' quality of life in palliative care.27, 28, 29, 30, 31
Implications for policy, practice, and research
In 2017, to promote the development of Chinese palliative care, the Chinese government issued the Chinese Palliative Care Practice Guideline.32 It was a landmark in palliative-care quality management and provided significant reference value for our research. However, at present, currently on mainland China, there remains a lack of policies and regulations on palliative care. The biggest obstacle to the development of palliative care in China is that it is not included in the scope of medical-insurance reimbursement, and uniform charges and programmes have not been established.
On the one hand, most of the medical staff in a hospital are unwilling to work in the palliative-care ward because the patients usually have critical conditions, the medical staff have a heavy workload, and there are many humanistic-care services that cannot be charged. Consequently, the medical-staff salaries are lower than those in most of the departments. To develop palliative care, it must be included in the scope of health insurance; thus, policy support is urgently needed. On the other hand, increasingly more services in palliative care are not reimbursable (eg, hair washing in bed), which adds to patients’ financial burden. Thus, the patients usually choose to be discharged back to their homes to spend the last days of their lives if there is no hope of survival. At present, the prevalence rate of telemedicine and home care in China is low, the role of the community in the home care of terminal patients is not obvious, the family lacks the relevant knowledge and skills to take care of terminal patients, and the symptoms of patients are difficult to be effectively controlled, so the quality of life of terminal patients at home is not high. To address these issues, a palliative care quality committee could be established, as in the U.K., to develop a national palliative care quality management training program that offers, for example, qualifications for palliative care practitioners. Moreover, it is essential to develop and monitor the implementation of palliative care quality management protocols for hospitals, health care organizations, and the community. It would also ensure that palliative-care service items were included in the reimbursement scope for medical insurance to reduce patients’ economic pressure, increase medical-staff salaries, and improve patients’ enthusiasm and that of medical staff to participate in palliative care.
In terms of legislation, living wills and decision-making agents are crucial elements in palliative care. However, although some provinces have been experimenting with related matters, there is still a lack of identification at the national legal level, which may result in disputes or legal violations. Thus, national legislative support is the most important step to improve the quality of palliative care, and this study helps promote this legislative process.
Due to insufficient policies and laws, the current research on palliative care in China consists mainly of reviews; experimental research is extremely lacking, and much research needs to be conducted in the future. Directions for future research include the following: How can we promote the popularization of science education on hospice care? How can we improve the referral service between different hospice service organizations? How can we boost the accuracy of patient prognosis assessment? How can we enhance physical, psychological, social, and spiritual support for patients?
Strengths and limitations
This study possesses several notable strengths. Firstly, the Delphi experts, all recognized for their exceptional contributions to palliative care, offered valuable insights and guidance for our research. Secondly, the proposed indicators, derived from a systematic and comprehensive literature and guideline review, exhibit broad applicability and generality. Thirdly, given China's vast territory and large population, the inclusion of a substantial consultation panel enhances the scientific rigor of this study. However, it is imperative to acknowledge the study's limitations. Firstly, while the panel included a diverse range of stakeholders, certain groups, such as patients, family caregivers, and experts from underdeveloped provinces, were underrepresented. This could potentially limit the generalizability of the results. Further investigations are warranted to explore indicator integration in low-resource regions and collaboration across countries and regions.33 Secondly, owing to the impact of COVID-19, interviews were conducted via voice calls, preventing the precise observation of participants' facial expressions and body language. Lastly, the experimental analysis of the reliability and validity of the proposed palliative care quality indicators requires exploration in subsequent studies.
Conclusions
Based on the structure-process-outcome theoretical model, 71 evidence-based quality indicators established through a joint effort of our research team may serve as a reference to guide palliative care in China. Empirical studies are required to examine the quality indicators among in-patients receiving palliative care in the future.
CRediT author statement
Yongyi Chen and Youwen Gong conducted data collection; data analysis was performed by Hongling Zheng and Qinqin Cheng. The first draft of the manuscript was written by Hongling Zheng and subsequently revised by Qinqin Cheng, Xianghua Xu, Yixia Yan, and Ge Luo. All authors had full access to all the data in the study, and the corresponding author had final responsibility for the decision to submit for publication. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Declaration of competing interest
All authors have none to declare. The corresponding author, Prof. Yongyi Chen, is a member of the editorial board of the Asia-Pacific Journal of Oncology Nursing. The article underwent the journal's standard review procedures, with peer review conducted independently of Professor Chen and their research groups.
Funding
This study was supported by the Hunan Provincial Natural Science Foundation of China (Grant No. 2020JJ4417) and the Science and Technology Innovation Program of Hunan Province, China (Grant No. 2020SK51105). The funders had no role in considering the study design or in the collection, analysis, interpretation of data, writing of the report, or decision to submit the article for publication.
Ethics statement
This project was approved by the Ethics Committee of Hunan Cancer Hospital (IRB No. 2021-02). All participants received sufficient explanation about the study and voluntarily participated in the study. Informed consent was provided to all participants included in the study.
Data availability statement
Derived data supporting the findings of this study are available from the corresponding author on request.
Declaration of Generative AI and AI-assisted technologies in the writing process
No AI tools/services were used during the preparation of this work.
References
- 1.Cohen J., Hermans K., Dupont C., Van den Block L., Deliens L., Leemans K. Nationwide evaluation of palliative care (Q-PAC study) provided by specialized palliative care teams using quality indicators : large variations in quality of care. Palliat Med. Sep 2021;35(8):1525–1541. doi: 10.1177/02692163211019881. [DOI] [PubMed] [Google Scholar]
- 2.Chan K.S. Palliative care: the need of the modern era. Hong Kong Med J. Aug 2018;24(4):391–399. doi: 10.12809/hkmj187310. [DOI] [PubMed] [Google Scholar]
- 3.Shekelle P.G. Quality indicators and performance measures: methods for development need more standardization. J Clin Epidemiol. Dec 2013;66(12):1338–1339. doi: 10.1016/j.jclinepi.2013.06.012. [DOI] [PubMed] [Google Scholar]
- 4.Pooni A., Schmocker S., Brown C., et al. Quality indicator selection for the Canadian Partnership against Cancer rectal cancer project: a modified Delphi study. Colorectal Dis. Jun 2021;23(6):1393–1403. doi: 10.1111/codi.15599. [DOI] [PubMed] [Google Scholar]
- 5.Campbell S.M., Braspenning J., Hutchinson A., Marshall M. Research methods used in developing and applying quality indicators in primary care. Qual Saf Health Care. Dec 2002;11(4):358–364. doi: 10.1136/qhc.11.4.358. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Meissner W., Huygen F., Neugebauer E.A.M., et al. Management of acute pain in the postoperative setting: the importance of quality indicators. Curr Med Res Opin. Jan 2018;34(1):187–196. doi: 10.1080/03007995.2017.1391081. [DOI] [PubMed] [Google Scholar]
- 7.McCorry N.K., O'Connor S., Leemans K., et al. Quality indicators for palliative day services: a modified Delphi study. Palliat Med. Feb 2019;33(2):197–205. doi: 10.1177/0269216318810601. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Donabedian A. The quality of care. How can it be assessed? JAMA. Sep 23-30 1988;260(12):1743–1748. doi: 10.1001/jama.260.12.1743. [DOI] [PubMed] [Google Scholar]
- 9.Buja A., Rivera M., Baldo V., et al. Palliative care quality measures: an exploratory study. BMJ Support Palliat Care. Feb 14 2019 doi: 10.1136/bmjspcare-2018-001679. [DOI] [PubMed] [Google Scholar]
- 10.Mizuno A., Miyashita M., Hayashi A., et al. Potential palliative care quality indicators in heart disease patients: a review of the literature. J Cardiol. Oct 2017;70(4):335–341. doi: 10.1016/j.jjcc.2017.02.010. [DOI] [PubMed] [Google Scholar]
- 11.Nakazawa Y., Kato M., Yoshida S., Miyashita M., Morita T., Kizawa Y. Population-based quality indicators for palliative care programs for cancer patients in Japan: a Delphi study. J Pain Symptom Manage. Apr 2016;51(4):652–661. doi: 10.1016/j.jpainsymman.2015.11.011. [DOI] [PubMed] [Google Scholar]
- 12.Tsiamis E., Millar J., Baxi S., et al. Development of quality indicators to monitor radiotherapy care for men with prostate cancer: a modified Delphi method. Radiother Oncol. Aug 2018;128(2):308–314. doi: 10.1016/j.radonc.2018.04.017. [DOI] [PubMed] [Google Scholar]
- 13.Walling A.M., Ahluwalia S.C., Wenger N.S., et al. Palliative care quality indicators for patients with end-stage liver disease due to cirrhosis. Dig Dis Sci. Jan 2017;62(1):84–92. doi: 10.1007/s10620-016-4339-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Virdun C., Luckett T., Lorenz K.A., Phillips J. National quality indicators and policies from 15 countries leading in adult end-of-life care: a systematic environmental scan. BMJ Support Palliat Care. Jun 2018;8(2):145–154. doi: 10.1136/bmjspcare-2017-001432. [DOI] [PubMed] [Google Scholar]
- 15.Yang H.J., Yang X.W., Yang Z.X., Zhang C.Y. Developing the nursing services quality indicator system of hospice care for cancer patients. Chin J Nurs. 2018;53(12):1487–1491. [Google Scholar]
- 16.Liu Y., Peng W., Chen Q. Construction of hospice nursing quality evaluation index system based on the structure-process-outcome theory. Mod Clin Nurs. 2020;19(12):40–45. [Google Scholar]
- 17.Xu X., Chen Q., Wan Y., et al. Research progress in quality assessment of palliative care abroad. Chin Nurs J. 2019;54(10):1578–1583. [Google Scholar]
- 18.De Panfilis L., Di Leo S., Peruselli C., Ghirotto L., Tanzi S. “I go into crisis when …”: ethics of care and moral dilemmas in palliative care. BMC Palliat Care. Aug 9 2019;18(1):70. doi: 10.1186/s12904-019-0453-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.De Brasi E.L., Giannetta N., Ercolani S., et al. Nurses' moral distress in end-of-life care: a qualitative study. Nurs Ethics. Aug 2021;28(5):614–627. doi: 10.1177/0969733020964859. [DOI] [PubMed] [Google Scholar]
- 20.Zhang Q., Saharuddin N.B., Aziz N. The analysis of teachers' perceptions of moral education curriculum. Front Psychol. 2022;13 doi: 10.3389/fpsyg.2022.967927. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Assante G.M., Popa N.L., Grădinariu T. Evidence to the need for a unifying framework: critical consciousness and moral education in adolescents facilitate altruistic behaviour in the community. Behav Sci (Basel, Switzerland) Oct 2 2022;12(10) doi: 10.3390/bs12100376. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Cao Q. Entrepreneurial psychological quality and quality cultivation of college students in the higher education and moral education perspectives. Front Psychol. 2021;12 doi: 10.3389/fpsyg.2021.700334. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Kaasa S., Loge J.H., Aapro M., et al. Integration of oncology and palliative care: a lancet oncology commission. Lancet Oncol. Nov 2018;19(11):e588–e653. doi: 10.1016/s1470-2045(18)30415-7. [DOI] [PubMed] [Google Scholar]
- 24.Andreas M., Piechotta V., Skoetz N., et al. Interventions for palliative symptom control in COVID-19 patients. Cochrane Database Syst Rev. Aug 23 2021;8(8):Cd015061. doi: 10.1002/14651858.Cd015061. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Santos F.C.D., Snigurska U.A., Keenan G.M., Lucero R.J., Modave F. Clinical decision support systems for palliative care management: a scoping review. J Pain Symptom Manag. Aug 2023;66(2):e205–e218. doi: 10.1016/j.jpainsymman.2023.03.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.The World Health Organization Quality of Life Assessment (WHOQOL): development and general psychometric properties. Soc Sci Med (1982) Jun 1998;46(12):1569–1585. doi: 10.1016/s0277-9536(98)00009-4. [DOI] [PubMed] [Google Scholar]
- 27.Hoomani Majdabadi F., Ashktorab T., Ilkhani M. Impact of palliative care on quality of life in advanced cancer: a meta-analysis of randomised controlled trials. Eur J Cancer Care. Nov 2022;31(6) doi: 10.1111/ecc.13647. [DOI] [PubMed] [Google Scholar]
- 28.Candy B., Armstrong M., Flemming K., et al. The effectiveness of aromatherapy, massage and reflexology in people with palliative care needs: a systematic review. Palliat Med. Feb 2020;34(2):179–194. doi: 10.1177/0269216319884198. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Zeng Y.S., Wang C., Ward K.E., Hume A.L. Complementary and alternative medicine in hospice and palliative care: a systematic review. J Pain Symptom Manage. Nov 2018;56(5):781–794.e4. doi: 10.1016/j.jpainsymman.2018.07.016. [DOI] [PubMed] [Google Scholar]
- 30.Quinn K.L., Shurrab M., Gitau K., et al. Association of receipt of palliative care interventions with health care use, quality of life, and symptom burden among adults with chronic noncancer illness: a systematic review and meta-analysis. JAMA. Oct 13 2020;324(14):1439–1450. doi: 10.1001/jama.2020.14205. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Huo B., Song Y., Chang L., Tan B. Effects of early palliative care on patients with incurable cancer: a meta-analysis and systematic review. Eur J Cancer Care. Nov 2022;31(6) doi: 10.1111/ecc.13620. [DOI] [PubMed] [Google Scholar]
- 32.Galiana L., Oliver A., De Simone G., Linzitto J.P., Benito E., Sansó N. A brief measure for the assessment of competence in coping with death: the coping with death scale short version. J Pain Symptom Manage. Feb 2019;57(2):209–215. doi: 10.1016/j.jpainsymman.2018.11.003. [DOI] [PubMed] [Google Scholar]
- 33.Kamal A.H., Harrison K.L., Bakitas M., et al. Improving the quality of palliative care through national and regional collaboration efforts. Cancer Control. Oct 2015;22(4):396–402. doi: 10.1177/107327481502200405. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Derived data supporting the findings of this study are available from the corresponding author on request.