The development of local outpatient palliative care referral criteria for persons with advanced cancer at the University Hospital Centre Zagreb: a Delphi study

Abstract

Background

Despite recommendations for early integration, persons with cancer in Croatia are often referred to palliative care late in their disease course. This study aimed to develop standardized local referral criteria to facilitate earlier palliative care referral at University Hospital Centre Zagreb (UHC Zagreb).

Methods

We conducted a mixed-methods consensus process comprising two Delphi survey rounds with oncologists at UHC Zagreb, a facilitated consensus meeting to refine criteria, and a confirmatory survey. The initial item set drew on the Hui international consensus criteria and the Royal Marsden Hospital Trigger Tool. In both Delphi rounds, participants rated each criterion on a 9-point scale. Consensus was predefined as ≥ 70% rating 7–9. Quantitative data were summarised descriptively. Qualitative data were analysed using the content analysis method.

Results

37 oncologists participated. Of the initial 17 criteria, 10 met the Delphi consensus threshold. The consensus meeting accepted one criterion unchanged, rejected four, combined four overlapping psychosocial items, and revised eight for clarity and local applicability. The confirmatory survey endorsed a 10-item set: (1) symptomatic brain/leptomeningeal metastases or CNS tumour; (2) ECOG 2 and deteriorating; (3) patient request or when no further oncologic options remain; (4) severe physical symptom(s); (5) symptomatic spinal cord compression/cauda equina; (6) advanced/incurable cancer with estimated prognosis ≤ 1 year; (7) complicated social situation impacting treatment; (8) delirium related to cancer; (9) moderate–severe psychological/existential distress; and (10) frequent unplanned admissions or emergency visits.

Conclusions

We have developed local referral criteria to standardise and accelerate outpatient palliative care referrals at UHC Zagreb. Implementation and prospective evaluation should assess fidelity, timeliness, patient-reported outcomes, utilisation, and costs.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00520-026-10528-x.

Introduction

Prominent organizations, including the World Health Organization (WHO), the European Society for Medical Oncology (ESMO), and the American Society of Clinical Oncology (ASCO), recommended early integration of palliative care, alongside standard oncology care [1–3]. Early integration (from diagnosis of advanced cancer) improves quality of life, symptom burden, and satisfaction with care; it has also been shown to enhance communication and increase survival compared to standard oncology care alone [4–8]. Additionally, early palliative care has been shown to reduce the number of acute oncology admissions and emergency room visits [9], leading to reduced healthcare costs [10].

Despite this, in clinical practice, referrals to palliative care services for patients with advanced cancer remain variable and often occur late in the disease course when active cancer treatment has been discontinued [9, 11–13]. Questions surrounding the optimal timing of palliative care referral, as well as patient selection and the most effective models of palliative care integration, persist and vary based on local resources and practices. Additionally, oncologists differ widely in their attitudes, beliefs, and practices around referral to palliative care. Referral criteria have been developed in the United States and the United Kingdom to standardize referral to palliative care [9, 13]; the authors have acknowledged that modifications may be required for different healthcare systems based on local practices and service availability [14].

At University Hospital Centre Zagreb (UHC Zagreb), patients with advanced cancer are typically referred to the palliative care team only when active oncology treatment is no longer feasible, often in the advanced or terminal disease stages. Referrals are subjective and lack standardized criteria. Structured tools for assessing symptoms, psychological status, or social conditions are unavailable. These gaps lead to delayed referrals, diminishing the potential benefits of early palliative care. The aim of this study was to develop a set of local referral criteria for patients with advanced cancer, taking into consideration Croatia’s unique healthcare system, infrastructure, and specific patient population needs.

Methods

Study design

This study employed a sequential mixed-methods approach, combining quantitative and qualitative methodologies. Initially, two rounds of an online Delphi survey were conducted, followed by a consensus meeting, and, finally, a confirmatory survey to validate and adopt the developed local referral criteria. The consensus meeting was conducted with a subset of experts who had previously participated in the Delphi study. The purpose of the consensus meeting was to gain a deeper insight into the expert’s opinion about the feasibility of each criterion and the possibility of adapting the criteria to the local institutional context. The confirmatory survey examined the agreement of all experts with the criteria proposed based on the results of the Delphi study and focus group.

Reporting

The study was reported in accordance with the CHERRIES (Checklist for Reporting Results of Internet E-Surveys) [15] and COREQ (Consolidated Criteria for Reporting Qualitative Research) guidelines [16].

Ethics

The study was approved by the Ethics Committee of the University Hospital Centre Zagreb (UHC Zagreb; decision dated October 30, 2023; Classification no. 8.1–23.1/268-2; Ref. no. 02/013AG) and by the Ethics Committee of the University of Zagreb School of Medicine (decision dated October 29, 2024). All procedures were conducted in accordance with the ethical standards of the institutional and national research committees and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. Participation was voluntary; data were collected anonymously or anonymized prior to analysis; and no compensation was provided.

Participants

Eligible participants were oncologists from the Department of Oncology at UHC Zagreb, an academic centre and the largest hospital in Croatia, providing comprehensive oncology services. At the time of the study, the Department employed 40 oncologists. Eligible individuals were informed about the study in person during department meetings and via an emailed invitation containing a survey link. Reminders were sent via e-mail and WhatsApp. The study was not advertised outside the Department. Recruitment was done internally via direct communication.

Delphi surveys

Development and testing

Two Delphi surveys were developed based on two existing referral criteria for outpatient specialist palliative care, based on international consensus by Hui et al. [17] and Royal Marsden Trigger tool criteria [9]. The research team reviewed and pretested the survey items. Then, the research team developed the Google Form surveys. The lead author (IK) was responsible for designing the survey structure. The layout and logic of the questions were reviewed by two co-authors (LS and LP), both experienced in survey methodology, to ensure usability and content clarity. Prior to distribution, the surveys underwent internal pilot testing with two oncology residents not involved in the main study to assess question clarity, technical functionality, and completion time. Based on their feedback, minor adjustments were made to improve the clarity of item wording and optimize the order of questions. No technical issues were reported during the pilot phase.

Survey content

The survey consisted of two parts. The first collected sociodemographic data, including sex, age, years of oncology experience, type of specialization (radiation oncology or medical oncology), and specific area of oncology expertise (tumour site). The second part included 17 referral criteria extracted from two sets of existing criteria [9, 17]. In the first Delphi round, participants were asked to rate each criterion on a 9-point Likert scale (1 = not applicable at all, 9 = fully applicable). In the second Delphi round, participants reviewed the same 17 referral criteria together with the frequency and percentages of participants’ responses from the first round. Participants were asked to score each of the criteria again on the same 9-point Likert scale. Consensus was defined as 70% agreement within the 7–9 range (Key criteria), aligning with established Delphi study standards [18]. An English translation of the survey used in the first Delphi round is provided in Supplementary file 1. The survey used in the second round is provided in Supplementary file 2.

In accordance with the original framework by Hui et al., the criterion “Assistance with decision making or care planning” was understood to include situations in which the need for additional support is identified either by the patient or by the treating oncologist, particularly when clinicians recognize unmet needs related to cancer treatment decision making or advance care planning [17].

ECOG 2 and deteriorating was defined as ECOG performance status 2 accompanied by clinically observed functional decline over time, including patients whose status declined from ECOG 0 or 1 to ECOG 2 or who are ECOG 3 or 4 in line with the description of the criterion provided in the Royal Marsden Trigger Tool [9, 13, 19].

Complex social issues (e.g., limited social support, caregiver burden, financial or housing instability, safeguarding concerns, or social factors interfering with care) were identified as a criterion derived from the Royal Marsden Trigger Tool and subsequently refined during the consensus meeting [9, 13, 19]. In the original Royal Marsden trigger tool publications [9, 13, 19], the term “frequent” acute or unplanned oncology admissions or emergency department visits lacks a specific numerical definition; further refinement is planned in future national validation studies.

Surveys’ administration

Surveys were administered electronically. The first Delphi round was sent via e-mail on March 18, 2025, followed by a first reminder email on March 21 and a second on March 23. The second Delphi round was sent via email on March 29, 2025, followed by a first reminder email on April 1 and a second reminder e-mail on April 2. Additionally, oncologists were reminded either in person or via WhatsApp message to complete rounds 1 and 2. In both surveys, items were presented in a fixed, logical order and were not randomized. Adaptive questioning was not applicable, as all participants received the same set of questions. The surveys were presented across 2–3 screens, depending on the device used. Google Forms was set to require responses for all items before submission. Participants could review and change their responses before submitting the survey. Due to the internal and closed nature of the survey distribution, unique site visitor and view rate data were not applicable or tracked. All submitted surveys were completed in full, as forced-response settings in Google Forms ensured completeness. Registration was not required, as participation was based on a personalized invitation sent directly to eligible oncologists. We did not use any strategies to prevent multiple entries. However, we consider that multiple entries were unlikely, considering the nature of the study. IP address checks were not performed. Log file analysis was not applicable. Timing was not a factor in analysis, as data collection periods were short and tightly monitored.

Consensus meeting

A structured in-person consensus meeting was conducted as the qualitative component of the study, rather than a classical focus group. Its primary aim was the final refinement of referral criteria developed through the two Delphi rounds, rather than the exploration of new themes.

All oncologists who participated in the Delphi process were invited to attend the consensus meeting. The meeting was held in person at the University Hospital Centre Zagreb on April 8, 2025, and was moderated by IK in collaboration with LS. Eleven oncologists participated in the meeting (see Fig. 1). Non-participation was mainly due to organizational and scheduling constraints, including outpatient clinic duties, day hospital responsibilities, on-call duties, recovery after night shifts, or planned leave.

Fig. 1.

Study flowchart illustrating the progression of the study and outlining the phases

During the meeting, participants were presented with the results of the second Delphi round in both written and oral form. Criteria were discussed sequentially, starting with those that achieved the highest level of consensus and proceeding to those with lower levels of agreement. Participants were invited to comment on each criterion, propose wording modifications, suggest additional criteria that had not reached consensus in the Delphi process, or propose exclusion of criteria if clear justification was provided.

The meeting was audio-recorded, and extensive field notes were taken during and immediately after the meeting. A verbatim transcript was produced and verified against the audio recording and field notes. Given that the purpose of the consensus meeting was to validate and refine predefined criteria rather than to generate new concepts, data saturation was not sought.

Content analysis was conducted by three researchers (IK, LS, and LP), focusing on participants’ acceptance or rejection of individual criteria. Themes were derived inductively from participants’ comments. IK was supervised by LS and LP, who both have extensive experience in designing and conducting qualitative studies. A summary of the consensus meeting was subsequently presented at an expert meeting of 20 oncologists and served as a form of member checking. Further methodological details are provided in Supplementary File 3.

Expert meeting and confirmatory survey

Following the consensus meeting, an expert meeting with the oncologists who participated in the earlier phases of the study was organized on April 15, 2025. During this meeting, a brief summary of the consensus meeting was presented, and the results were presented to the oncologists for feedback. There were no new suggestions for revisions of the referral criteria. A confirmatory consensus survey was conducted via Google Forms to formally endorse the final set of referral criteria. An English translation of the confirmatory survey is attached in Supplementary file 4.

Quantitative data analysis

Sociodemographic data (e.g., gender, age, years of oncology experience, specialization type, and tumour site expertise) were summarized with descriptive statistics, using frequencies, percentages, means, standard deviations, and percentages. For the two-round Delphi survey, consensus was defined as agreement by at least 70% of participants rating a criterion in the same category within the 7–9 range (Key criteria), aligning with established Delphi study standards [18]. IBM SPSS Statistics software for Windows Version 22.0 (IBM Corp., Armonk, NY, USA) was used to perform statistical analyses.

Results

Participants’ characteristics

Among 40 oncologists, three were on medical leave at the time of the study and were excluded from participation. Of the remaining 37 eligible participants, 34 participated in the first Delphi survey, and 31 in the second. Characteristics of the participants are shown in Table 1. Participants’ flow in the study is shown in Fig. 1. There were 22 (65%) oncologists who had expertise in a single field, including breast cancer (N = 8), gastrointestinal tumours (N = 7), urogenital tumours (N = 5), and mesenchymal tumours (N = 2). The remaining 12 (25%) participants indicated having more than one field of oncological expertise (Table 1).

Table 1.

Demographic and professional characteristics of Delphi survey participants

Variable	Result
Total invited participants	37
Participated in round 1, N (%)	34 (92)
Women, N (%)	26 (76)
Men, N (%)	8 (24)
Mean age (years ± SD) K-S test: D = 0.167, p = 0.017 (Lilliefors correction)	45 ± 9,4
Years of experience working as an oncology specialist K-S test: D = 0.155. p = 0.038 (Lilliefors correction)	11.6 ± 9.8
Medical Oncology Specialist Clinical Oncology Specialist Oncological expertise in a single field Oncological expertise in more than one field Specific field of interest in gastrointestinal tumors Specific field of interest in urogenital tumors Specific field of interest in breast cancer Specific field of interest in mesenchymal tumors Specific field of interest in head and neck tumors Specific field of interest in skin tumors Specific field of interest in supportive and palliative care Specific field of interest in tumors of the central nervous system Specific field of interest in radiotherapy of the breast and lung tumors	24 (71) 10 (29) 22 (65) 12 (35) 12 (35) 10 (29) 9 (26) 6 (18) 4 (12) 3 (8.8) 1 (2.9) 1 (2.9) 1 (2.9)

Delphi survey results

In the first round of the Delphi survey, eight criteria out of seventeen were rated as critically important (scores 7–9) by ≥ 70% of the participants (Table 2). Of the 31 participants in the second round, 23 (74%) were women and 8 (26%) were men. Ten criteria of seventeen were rated as critically important (scores 7–9) by ≥ 70% of the participants (Table 2). These included all eight criteria selected as critically important in the first round, as well as criteria #10 and #15 (Table 2) as follows: severe physical symptom(s) (90%); assistance with decision making/advance care planning (77%); patient request for palliative care referral (94%); brain or leptomeningeal metastases (97%); spinal cord compression or cauda equina (87%); referral within 3 months of diagnosis of advanced/incurable cancer when prognosis is ≤ 1 year (84%); progressive disease despite second-line palliative systemic therapy (77%); ECOG performance status 2 with deterioration (97%); anorexia, hypercalcemia, or any effusion (71%); and complex social issues (74%).

Table 2.

Comparison of ratings for criteria scored as critically important in the first and second Delphi round

Criterion	Total score for range 7–9, N (%)
	Delphi 1	Delphi 2
1. Severe physical symptom(s) (e.g., severe pain, dyspnea, nausea or other physical symptoms with an intensity of ≥ 7/10 on the ESAS score)	29 (85)	28 (90)
2. Severe emotional symptom(s) (e.g., severe depression or anxiety, with an intensity of ≥ 7/10 on ESAS score)	20 (59)	18 (58)
3. Patient’s request for hastened death (suicidal intent or thoughts)	14 (41)	7 (23)
4. Spiritual or existential crisis—documentation of severe spiritual pain, such as an intensity of ≥ 7/10 on ESAS or description by chaplain)	15 (44)	11 (35)
5. Assistance with decision making/care planning – additional assistance with cancer treatment decision making or advance care planning	25 (74)	24 (77)
6. Patient request—patients request specifically for palliative care referral anytime prior to palliative care consultation	28 (82)	29 (94)
7. Delirium—clinical diagnosis of delirium or Memorial Delirium Assessment Scale (MDAS) > 13/30. The MDAS is a validated scale that assesses the severity of delirium [20, 21]. It consists of 10items based on clinical observations and interview of the patient. Each item is assigned a score from 0 to 3, with a higher score indicating greater severity. A total score of 13/30 supports the diagnosis of delirium	21 (62)	19 (61)
8. Brain or leptomeningeal metastases—radiologic or clinical diagnosis of cancer involvement in the brain or leptomeninges. Patients with central nervous system tumors were coded as yes	30 (88)	30 (97)
9. Spinal cord compression or cauda equina—radiologic or clinical diagnosis of impending or symptomatic spinal cord compression or cauda equine syndrome	25 (74)	27 (87)
10. Within 3 months of diagnosis of advanced/incurable cancer for patients with median survival 1 year or less (within 90 days from date of diagnosis of advanced cancer: metastatic lung cancer, metastatic non-colorectal gastrointestinal cancers, metastatic anaplastic thyroid cancer, metastatic hepatocellular carcinoma, metastatic renal cell carcinoma, metastatic head and neck cancer, metastatic cancer of unknown primary, locally advanced pancreatic cancer, locally advanced anaplastic thyroid cancer, and acute myeloid leukemia)	22 (65)	26 (84)
11. Metastatic cancer progressing after first line treatment	15 (44)	11 (35)
12. Progressive disease despite second line palliative systemic therapy	25 (74)	24 (77)
13. Performance status ECOG 2 and deteriorating	26 (76)	30 (97)
14. Acute oncology or unplanned admission	12 (35)	12 (39)
15. Anorexia, hypercalcemia, or any effusion	17 (50)	22 (71)
16. Moderate or severe psychological or existential distress	15 (44)	15 (48)
17. Complex social issues	24 (71)	23 (74)

Scoring explanation: value 1 (The criterion is not applicable at all to our patients) to 9 (The criterion is fully applicable to our patients)

Results highlighted in bold are those in which the respondents’ agreement in assessing the criteria with a score of 7 to 9 was achieved in a percentage greater than 70%

Consensus meeting

At the consensus meeting, participants reviewed the results of the second Delphi round—ranking the complete set of 17 criteria from the highest to the lowest level of expert agreement—using printed copies as presented in Supplementary File 5. This resulted in a final list of criteria that was further presented and examined in the Expert Meeting and Confirmatory Consensus Survey. Content analysis of the transcript from the consensus meeting resulted in four possible outcomes for each criterion: criterion accepted, rejected, combined, or revised. Detailed description of participants’ feedback and quotes are shown in Table 3 and Supplementary file 6.

Table 3.

Qualitative comments from the consensus meeting

Criterion outcome	Description of main results	Criterion	Selected participant quote
(1) Criterion accepted	Accepted in its proposed form	13th—Performance status ECOG 2 and deteriorating	Participant 1: “Someone with ECOG greater than 2 should be referred to palliative care.” Participant 2: “This refers to ECOG greater than 2 and ECOG 2. Patients with ECOG 2 are still eligible for systemic therapies.” Participant 3: “However, this does not exclude palliative treatment.” Participant 4: “Someone with ECOG ≥ 2 is certainly dependent on others to some extent, and I think they should be referred to palliative care, but let’s not confuse palliative care with necessarily excluding systemic oncological treatment.” Participant 5: “Those with ECOG 2 do not have to remain ECOG 2 under systemic therapy.” Participant 6: “We also need to consider the dynamics of ECOG changes. Someone can quickly go from 0 to 2.” Participant 3: “But they can also go from 3 to 0.” Participant 2: “For example, if someone has prostate cancer — you give castration and someone who was ECOG 3 might later be ECOG 1.”
(2) Criterion rejected	Rejected due to difficulties in implementation, vagueness and ambiguity, redundancy, insufficient comprehensiveness, and overlap with other criteria	3rd—Patient’s request for hastened death; suicidal intent or thoughts	Participant 4: “ I think everyone thought of euthanasia and automatically marked it as not applicable, because this does not apply in our context.” Participant 7: “Most of us reacted this way, because we are all against euthanasia, and therefore we do not want this to be a criterion at all.” Participant 4: “It is not common for our patients to express this…” Participant 8: “We cannot assess this criterion because we cannot ask patients whether they have a wish for death.” Participant 9: “What most people say is… basically, ‘I just don’t want to suffer. It does not matter how long it lasts, as long as I am not suffering. Families, of course, often say the same—just that he/she doesn’t suffer.”
		12th—Progressive disease despite second line palliative systemic therapy & 11th—Metastatic cancer progressing after first line treatment	Participant 10: “It is not applicable in itself, it is not the same second line in all sites, it is not the same second line for breast cancer or some other one. I would remove that criterion.” Participant 5: “This changes; it is dynamic.” Participant 4: “Some patients with colon cancer are on third-line treatment and are still working, fully independent, whereas it is very rare to see a person with pancreatic cancer progressing on second-line treatment with the patient still working—can anyone recall such a case? I cannot.” Participant 6: “Reformulate it as standard lines of treatment.” Participant 3: “I’m not sure how to reformulate this.” Participant 4: “This criterion should be removed.” Comment on the 11th criterion—Participant 4: “Just as we discarded Criterion 12—metastatic cancer progressing to second-line oncological treatment—we automatically discard this one as well. It makes even less sense than progression to second-line treatment.”
		15th—Anorexia, hypercalcemia, or any effusion & 1st—Chronic severe physical symptoms	Participant 2: “Not this criterion, but rather severe physical symptoms.” Participant 1: “I think this falls under severe physical symptoms.” Participant 4: “There is no need to split it so finely.”
(3) Criteria combined	Suggested merging two or more criteria based on recognition of boundaries between the responsibilities of the oncologist and the palliative care team, and recognizing the limits of professional competences for establishing differences between individual criteria	5th—Assistance with decision making/care planning) & 6th—Patient request for palliative care referral	Participant 4: “Assistance with decision making/care planning is similar to patient request for palliative care. I understood it as decision support, but not in terms of treatment—rather for planning care. I would remove ‘treatment’. All oncologists already do this, but it is not related to palliative care. The palliative care team should not be overloaded with this.” Participant 9: “I am concerned about the word ‘request,’ because someone who should request it may not actually do so. Yet we may clearly see that palliative care is needed.” Participant 4: “Perhaps instead of calling it a ‘requirement’, the criterion should be termed the patient’s need for palliative care. Either the patient explicitly requests it, or you think to yourself when the patient comes to the clinic, ‘oh dear, call the palliative care team.”
		2nd—Severe emotional symptoms & 4th—Spiritual or existential crisis and Criterion & 16th—Moderate or severe psychological or existential distress	Participant 7: “How are we supposed to know that? That is for a psychologist or a psychiatrist.” Participant 4: “If I refer a woman who is in emotional distress to the palliative care team, I am not sure whether I might further aggravate her emotional distress. The term ‘palliative care’ can worsen emotional distress. Maybe that’s for a psychologist.” Participant 3: “But at least one of those three psychological criteria should be included…” Participant 4: “Definitely keep one” Participant 5: “Perhaps Criterion 16 is the best: moderate to severe psychological or existential distress.” Participant 2: “We need to consider our local context. I am very much able to assess who is experiencing emotional distress.” Participant 6: “When it comes to severe depression, we are limited. That is in the domain of psychiatrists.” Participant 9: “If they also have other severe symptoms, then they are definitely for palliative care. But if this is the only issue, they may still be in an early stage of cancer. If we consider it as supportive care, then yes — and certainly for a psychologist” Participant 8: “This means that a severe emotional, moderately severe and spiritual or existential crisis can all be combined into one, yes.”
(4) Criteria revised	Reasons for the proposed changes included challenges with clinical applicability or the need to simplify the work in practice; a preference for broader, more comprehensive definitions; insufficient precision in the definition of terms in the criteria; and insufficient familiarity with the content of the criteria due to limitations in professional competences	1st—Severe physical symptoms	Participant 3: “Very frequent symptoms can be 7/10, for example fatigue.” Participant 2: “7/10 implies that they affect functioning.” Participant 4: “For example, dyspnea — if it occurs, it subsides; nausea can be a side effect of therapy, but usually they don’t feel sick.” Participant 5: “Chronic symptoms should be added.” Participant 4: “Maybe we should add what colleague said: chronic symptoms, meaning they persist for some time or are poorly controlled.”
		6th—Patient request for palliative care	Participant 4: “The criterion is good and acceptable, but in practice it rarely happens.” Participant 3: “Because patients and their families think that once palliative care is involved, it means the end of treatment — no more active therapy.” Participant 2: “Because active treatment is perceived, so to speak, as a kind of guarantee. It represents a sign that something proactive is still being done, whereas palliative care is often perceived as meaning that there is no longer any chance. The perception is different.” Participant 4: “Those who request it obviously do so for good reason. Maybe if it were introduced five months earlier, those five months could be of higher quality than the last two weeks They hear about the palliative team from neighbors.”
		7th—Delirium	Participant 9: “I think we do not encounter this very often, which may be why it received fewer votes.” Participant 7: “I think this falls within the domain of psychiatry.” Participant 2: “Not all patients who are confused or disoriented need to see a psychiatrist…I gave a score of 7 or 8, I’m surprised by such a small percentage, I think we should include this criterion.” Participant 4: “We certainly do not use this scale…If you have a patient who does not know what planet they are on, of course you will manage that.” Participant 5: “This was not clear to the colleagues.” Participant 8: “Perhaps this also belongs under severe symptoms” Participant 4: “This criterion seems very important to me.” Participant 9: “I would keep this.”
		8th—Brain or leptomeningeal metastases	Participant 4: “It seems clear by itself; these patients have severe neurological deficits and definitely meet criteria for palliative care, though not necessarily as the only care — also alongside treatment.” Participant 7: “Here we are referring, I think, to symptomatic brain metastases. That is not the same as brain metastases. That means with neurological deficit… ” Participant 5: “But it says clinical diagnosis, I understood it as symptomatic.” Participant 7: “In studies, this criterion is reported as symptomatic or asymptomatic brain metastases.” Participant 8: “They can undergo neurosurgery or radiation. Even a solitary brain metastasis can be symptomatic.” Participant 4: “I think we all understood that a clinical diagnosis refers to symptomatic brain metastases.”
		9th—Spinal cord compression or cauda equina	Participant 7: “If there is a radiological finding but no deficit, why would we refer the patient to palliative care?” Participant 4: “It’s not fully symptomatic today, but it will become so tomorrow. And then it will certainly be an indication for palliative care.” Participant 3: “It is important to distinguish between clinical (symptomatic) and purely radiological findings.”
		10th—Within 3 months of diagnosis of advanced/incurable cancer for patients with a median survival of 1 year or less	Participant 7: “I would not list specific diagnoses here. I would state a median survival of one year – remove diagnoses.” Participant 2: “There are fewer tumour types with such a short prognosis.” Participant 7: “Leave only an estimated survival of less than one year (without specifying tumour site), based on the judgement of the leading oncologist at the time of assessment, considering tumour burden, disease stage, and tumour biology.” Participant 10: “It is not realistic for the palliative care team to review all lung cancer patients within three months of diagnosis.”
		14th—Acute oncology or unplanned admission	Participant 5: “It may be redundant; an acute admission alone does not mean anything.” Participant 6: “The cause is what matters. Unplanned pneumonia.” Participant 4: “If we want to keep this criterion, there should be some threshold. It is not the same for someone who is admitted once every six months for febrile neutropenia versus someone who comes twice a week for nausea, weakness, or dehydration…Perhaps rephrase it as frequent visits to the emergency department.” Participant 5: “So, not hospitalizations, but emergency department visits.” Participant 9: “If someone comes to the day oncology unit every three weeks and, in between, presents several times to the emergency department, something is clearly not right.” Participant 5: “Perhaps we should activate the primary mobile palliative care team, the family doctor, and other services so that patients won’t need to go to the emergency department.”
		17th—Complex social issues	Participant 9: “There was no dilemma here. Because this is a very important criterion.” Participant 4: “In complicated social situations, he may even have an indication for palliative care before someone who is ECOG 2. … There is no one to cook for him, no one to wash his clothes, no one to go to the pharmacy…. There is no one to look after him… And they need palliative care earlier while they are still outpatients.” Participant 8: “And in fact, we often learn about their social situation quite late…” “When people are doing well, they do not speak objectively about their social situation—they say everything is fine, and then you realise it is not quite so.” Participant 2: “Definitely a very important criterion.” Participant 8: “Family support is extremely important. Can they even come to collect their medication? They may not be able to use the internet, or… Sometimes they cannot even obtain medical transport—because they live too far…” Participant 4: “You have people living in rural areas without hot water.” Participant 10: “It becomes clear once the problem manifests, but at the beginning, no one really asks about the social situation.” Participant 5: “It’s a very important criterion, but it’s not that we solve it ourselves, we need to involve someone else in these situations in order for the oncology treatment to be successful…” Participant 6: “We often don’t even have the mechanisms to solve it.”

(1) Criterion accepted

Only Criterion 13 (Performance status ECOG 2 and deteriorating) was accepted in its proposed form. Although Criterion 7 (Delirium) did not reach the predefined consensus threshold during the Delphi rounds, it was discussed in detail during the consensus meeting phase. Participants agreed that delirium represents a clinically important indicator of advanced disease complexity and unmet supportive needs in patients with advanced cancer. In line with the predefined study methodology, which allowed inclusion of criteria based on qualitative clinical rationale, and following endorsement in the confirmatory consensus survey, delirium was retained in the final referral criteria set.

(2) Criterion rejected

A total of four criteria were rejected. These were Criterion 12 (Progressive disease despite second-line palliative systemic therapy); Criterion 11 (Metastatic cancer progressing after first-line treatment), Criterion 3 (Patient’s request for hastened death; suicidal intent or thoughts), and Criterion 15 (Anorexia, hypercalcemia, or any effusion). Reasons for rejection included difficulties in implementation, vagueness and ambiguity, redundancy, insufficient comprehensiveness, and overlap with other criteria.

(3) Criteria combined

In two cases, participants suggested merging two or more criteria. This was suggested for Criterion 5 (Assistance with decision making/care planning) and Criterion 6 (Patient request for palliative care referral).

Despite Criterion 16 (moderate or severe psychological or existential distress) not reaching the predefined consensus threshold during the Delphi rounds, it was discussed during the consensus meeting phase. Participants agreed that at least one psychological criterion should be included in the referral list for early palliative care. Therefore, the final version incorporates a unified psychological criterion defined as “Observed moderate to severe psychological distress (e.g., pronounced emotional states or existential distress that interfere with patients’ normal functioning and psychological adaptation to the situation).” According to participants, this criterion integrates elements of Criterion 2 (severe emotional symptoms, such as depression or anxiety with an intensity of ≥ 7/10 on the ESAS), Criterion 4 (spiritual or existential crisis), and Criterion 16.

(4) Criteria revised

At the consensus meeting, experts recommended revising eight criteria (criteria 1, 6, 8–11, 14 and 17). The most common reasons for the proposed changes included challenges with clinical applicability or the need to simplify the work in practice; a preference for broader, more comprehensive definitions; insufficient precision in the definition of terms in the criteria; and insufficient familiarity with the content of the criteria due to limitations in professional competencies.

Expert meeting and confirmatory consensus survey

The final expert meeting, during which during which and following which the confirmatory survey was conducted, is described in more detail in the Methods section. The expert meeting was attended by 20 oncologists. Among 37 eligible participants, 33 (89%) took part in the confirmatory survey. The local criteria were confirmed by 32 (97%) oncologists, with one voting against. The final list of criteria, approved at the expert meeting and confirmatory consensus survey, are presented in Table 4.

Table 4.

Final local outpatient palliative care referral criteria for persons with advanced cancer at the University Hospital Centre Zagreb

1. Symptomatic brain or leptomeningeal metastases or a symptomatic tumour of central nervous system

2. Performance status ECOG 2 and deteriorating (ECOG 2 accompanied by clinically observed functional decline over time, including patients whose status declined from ECOG 0 or 1 to ECOG 2 or who are ECOG 3 or 4)

3. Patient request or oncological assessment of the need for palliative care referral (once no further oncological treatment options are available)

4. Severe chronic physical symptom(s) (e.g., severe pain, dyspnoea, nausea or other physical symptoms with an intensity of ≥ 7/10 on the ESAS score)

5. Symptomatic spinal cord compression or cauda equine syndrome caused by cancer or metastases

6. Diagnosis of advanced/incurable cancer for patients with median survival 1 year or less

7. Complex social issues (e.g., limited social support, caregiver burden, financial or housing instability, safeguarding concerns, or social factors interfering with care)

8. Clinical diagnosis or assessment of the presence of delirium symptoms related to cancer

9. Observed moderately severe and severe psychological distress (e.g., pronounced emotional states or existential distress that interfere with patients’ normal functioning and psychological adaptation to the situation)

10. Frequent acute or unplanned oncology admissions and emergency department visits due to cancer progression

Discussion

In this single-center Delphi study with a confirmatory consensus step, oncologists at the UHC Zagreb adapted two established frameworks into a pragmatic set of local referral criteria for outpatient specialty palliative care in advanced cancer. Ten criteria achieved consensus (≥ 70% rating 7–9) after two Delphi rounds and were endorsed in a confirmatory survey, reflecting domains of symptom burden, disease trajectory, performance status, patient preference, psychosocial complexity, and critical oncologic complications.

The agreed criteria emphasise: (i) high symptom burden (ii) early disease milestones (e.g., within 3 months of diagnosis for cancers with a median survival ≤ 1 year), (iii) declining performance status (iv) treatment trajectory (progression despite second-line therapy), (v) patient-centered triggers (patient request, assistance with decisions/advance care planning), (vi) complex social needs, and (vii) time-sensitive neurologic or metabolic complications (brain/leptomeningeal involvement, spinal cord compression, anorexia/hypercalcemia/effusions). This pattern aligns with contemporary concepts of timely palliative care that blend needs-based and disease-/time-based “automatic” triggers to minimise variation and delay [11, 17, 22]. The prominence of patient preference and decision-support is concordant with ASCO and ESMO guidance on early, concurrent palliative care integrated with routine oncology [1, 3], and with the WHO’s framing of palliative care as a holistic, longitudinal component of cancer care [3].

Notably, some items commonly included in international frameworks [17, 22], such as moderate/severe psychological or existential distress, delirium, and unplanned admissions, did not reach the ≥ 70% threshold locally. Several context-sensitive explanations are plausible, including perceived feasibility of systematic screening (e.g., routine ESAS/psychosocial assessment), uncertainties about care pathways for psycho-spiritual needs, and the local focus on outpatient referral mechanisms rather than hospital-based “acute oncology” triggers [9, 11, 17, 22]. These findings underscore why local adaptation is essential even when starting from well-validated international tools [11, 17, 22].

Our results are directionally consistent with the Hui international consensus, which prioritises high symptom severity, rapidly-evolving disease states, and declining performance status as referral prompts [17], and with subsequent work advocating automatic referral pathways to standardize access [22]. The inclusion of early time-based triggers (e.g., within 90 days of diagnosis in poor-prognosis cancers) is consistent with clinical trials demonstrating several benefits of early palliative care integration on quality of life, symptom control, communication, satisfaction, and survival [4–7]. System-level advantages, including lower healthcare utilisation, and economic benefits, are well documented and provide a rationale for institutional adoption and resourcing [9, 10]. International indicators for integration between oncology and palliative care also emphasize clear referral pathways and routine screening—elements that our criteria operationalize within the UHC Zagreb context [18].

Criteria implementation

For impact, criteria must be embedded into clinical workflows. We propose incorporating (i) simple electronic or paper-based prompts based on our criteria into oncology clinic and day-hospital templates; (ii) routine symptom screening; (iii) documentation and fast-track processes urgent referrals; and (iv) clear pathways for patient-initiated referrals and decision-aid/ACP support. A practical starter bundle of monitoring metrics could include: median days from advanced cancer diagnosis to first palliative consult and from consult to death; proportion referred within 90 days for poor-prognosis cancers; proportion with ECOG ≥ 2 referred; acute healthcare utilization and costs in the last 90 days of life; patient/caregiver-reported outcomes (e.g., symptom burden, satisfaction) [9, 10, 12, 18]. Regular audit and feedback to oncologists will reinforce use of the triggers [11, 17, 18, 22].

Beyond its immediate goal of developing locally applicable referral criteria, this study serves a broader purpose of enhancing the quality of palliative care by advocating for a biopsychosocial–spiritual model of care. Establishing clear, context-sensitive triggers for referral, will support earlier, more comprehensive, and more equitable access to care that respects the diverse and complex needs of patients with advanced cancer. Furthermore, the methodology used, combining expert consensus and Delphi rounds within a single institution, demonstrates how referral frameworks can be adapted to local healthcare contexts. This participatory approach enables standardization that remains sensitive to local realities, contributing to the systematization and transparency of referral decisions, while laying the groundwork for institutional models of collaborative, expert-informed criteria development in other areas of care as well.

The study has a number of strengths and limitations. The high engagement across both Delphi rounds is a strength, as is the mixed methods approach and adherence with CHERRIES and COREQ guidance [15, 16]. Local feasibility was specifically interrogated in a facilitated consensus meeting, increasing the likelihood of adoption. Limitations include the single-centre scope, reliance on oncologists without input from palliative care providers, patients, or caregivers; potential selection and social-desirability biases; and absence of prospective validation. This study involved oncologists only, reflecting their current central role in initiating palliative care referrals. The absence of specialist palliative care input represents a limitation of the health system and reflects the current organization of palliative care in Croatia, where no formal medical specialization yet exists. Future multidisciplinary studies may further refine and validate the criteria. The 70% consensus threshold is consistent with Delphi practice [18], but remains an operational choice. Finally, the criteria specify when to refer, not how to resource services. Their implementation may uncover capacity constraints that require phased scaling.

Future research

In Croatia, palliative care is still in its early stages, with no specialty or subspecialty training programs, and there may be a lack of education and specialized knowledge among oncologists regarding palliative medicine, pain management, the use of opioids, and palliative care in general. In primary palliative care teams, which provide home-based care, physicians are most often family medicine physicians or hospital specialists who have left the hospital system. This has undoubtedly shaped the attitudes of oncologists at UHC Zagreb and influenced their assessment of the importance of individual criteria. We are currently conducting a study on three years of experience with the first hospital-based palliative care team, where we observed that referrals are often made very late in the disease trajectory.

A future prospective implementation study, currently underway by our team, should evaluate fidelity, acceptability, and outcomes, especially the proportion of patients who would screen positive using these referral criteria, referral volume, time-to-referral, patient-reported outcomes, healthcare utilization, costs, and service demand. That study will be essential to inform sustainable implementation, particularly in healthcare systems where palliative care has traditionally focused on end-of-life care [4–7, 9, 10, 12]. Qualitative work with patients, caregivers, and the multidisciplinary team can refine pathways, especially around psychosocial/spiritual triggers that did not reach consensus locally. Comparative evaluation across Croatian centers could support national harmonization. A nationwide study in Croatia would be useful to confirm the applicability of the study findings to a national setting.

Conclusion

We developed a locally adapted, consensus-based set of ten referral criteria to trigger timely outpatient palliative care for adults with advanced cancer at UHC Zagreb. The criteria span symptom burden, disease/treatment trajectory, performance decline, early time-based triggers, neuro-oncologic and metabolic emergencies, patient preference/decision support, and social complexity, offering a pragmatic pathway to standardize and accelerate referrals. Embedding these triggers into routine oncology workflows should reduce variability and delays; a prospective implementation will evaluate fidelity, timeliness, patient-reported outcomes, utilization, and costs. This tool provides a transferable template for other centres seeking to harmonize palliative referral practices.

Supplementary Information

Below is the link to the electronic supplementary material.

Authors’ contributions

Study design: IK, LS, LP, SP, IB, BH.

Data collection, analysis, and interpretation: IK, LS, DG, LP, SP, IB.

Writing of the manuscript and revising the manuscript for intellectual content: IK, LS, DG, LP, SP, IB, BH.

Final approval of the manuscript: IK, LS, DG, LP, SP, IB, BH.

Funding

No extramural funding.

Data availability

The raw data collected in this study are available from the corresponding author upon request.

Declarations

Ethics approval and consent to participate

The study was approved by the Ethics Committee of the University Hospital Centre Zagreb (UHC Zagreb; decision dated October 30, 2023; Classification no. 8.1-23/268-2; Ref. no. 02/013AG) and by the Ethics Committee of the University of Zagreb School of Medicine (decision dated October 29, 2024). All procedures were conducted in accordance with the ethical standards of the institutional and national research committees and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. Participation was voluntary; data were collected anonymously or anonymized prior to analysis; and no compensation was provided. All participants received information about the study prior to participation. Informed consent was obtained electronically from all participants as part of the online survey process. The participant information sheet and consent text are provided in the Supplementary file 1.

Clinical trial registration

Not applicable. This study was not a clinical trial.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.