Identifying Nursing Home Residents with Unmet Palliative Care Needs: A Systematic Review of Screening Tool Measurement Properties

Abstract

Objectives:

Despite common use of palliative care screening tools in other settings, the performance of these tools in the nursing home has not been well established, therefore, the purpose of this review is to (1) identify palliative care screening tools validated for nursing home residents; and (2) critically appraise, compare, and summarize the quality of measurement properties.

Design:

Systematic review of measurement properties consistent with Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) guidelines.

Settings and participants:

Embase (Ovid), MEDLINE (PubMed), CINAHL (EBSCO), and PsycInfo (Ovid), were searched from inception to May 2022. Studies that (1) reported the development or evaluation of a palliative care screening tool and (2) sampled older adults living in a nursing home were included.

Methods:

Two reviewers independently screened, selected, extracted data, and assessed risk of bias.

Results:

We identified only one palliative care screening tool meeting COSMIN criteria, the NECesidades Paliativas (NEC-PAL, equivalent to palliative needs in English), but evidence for use with nursing home residents was of low quality. The NEC-PAL lacked robust testing of measurement properties such as reliability, sensitivity and specificity in the nursing home setting. Construct validity through hypothesis testing was adequate but only reported in one study. Consequently, there is insufficient evidence to guide practice. Broadening the criteria further, this review reports on three additional palliative care screening tools identified during the search and screening process but which were excluded during full-text review for various reasons.

Conclusion and Implications:

Given the unique care environment of nursing homes, we recommend future studies to validate available tools and develop new instruments specifically designed for nursing home use. In the meantine, we recommend that clinicians consider the evidence presented here and choose a screening instrument that best meets their needs.

Brief Summary:

This review identifies 1 palliative care screening tool but evidence for use with nursing home residents was of low quality. We recommend the development of a new instrument specifically designed for nursing homes.

Nursing home staff and clinicians frequently care for patients with advanced and end-stage illnesses that are accompanied by disability, functional limitations and multiple chronic comorbidities, including neurocognitive disorders. Moreover, nursing home residents are at increased risk for uncontrolled symptoms (e.g., pain), poor long-term outcomes (e.g., cognitive decline, psychosocial distress) and poor quality of life.^{1, 2} Palliative care focuses on preventing and relieving suffering for individuals living with serious illness and offers an opportunity to improve quality of life for nursing home residents. Research suggests that palliative care services in nursing homes can improve quality of life, resident and family satisfaction with care and management of distressing symptoms such as pain, while decreasing costs.^3–6 Despite positive outcomes associated with palliative care, nursing home residents do not receive palliative care services proportional to the high prevalence of advanced and end-stage illnesses.^{2, 7, 8} While hospice services and hospital-based palliative care services have seen increased usage,⁹ there remains a significant gap in access to and receipt of end-of-life and palliative care for individuals with serious illness residing in a nursing home, specifically those not requiring hospitalization and not eligible for hospice services.²

Barriers to palliative care for nursing home residents include insufficient access to providers with specialty palliative care training,^{2, 10} a nursing home workforce that is under resourced and without palliative care knowledge and skills to make appropriate referrals,^11–13 and disease trajectories with uncertain prognostication, such as dementia.¹⁴ Approaches to improving the integration of palliative care in nursing homes have focused on developing and testing innovative models of palliative care delivery (e.g., palliative care needs rounds),^15–18 embedding palliative care training into core curricula of all new health professionals (e.g., nursing, therapy, physicians),¹⁹ and developing nursing home specific palliative care quality indicators and practice guidelines.^{7, 20} A key impediment to nursing home staff and researchers in applying these innovative approaches is the difficulty with timely identification of residents and families with unmet palliative care needs. In a setting with high rates of serious and chronic illness, including cognitive illness, it may be challenging to identify the suitability of residents for palliative care. As such, standardized palliative care referral criteria could facilitate appropriate and timely entry to palliative care, but this is lacking in nursing homes.

One approach applied in other settings is the use of a validated palliative care screening tool.^21–25 A type of structured assessment, palliative care screening tools assist clinicians in early identification of patients who may benefit from palliative care. Typically, palliative care screening tools are designed to identify unmanaged pain, declining functional status, psychosocial distress, and family support needs. In hospitals, intensive care units, and oncology outpatient clinics, screening tools for structured evaluation of potential palliative care needs has been linked to improved symptom management, reduced in hospital death and acute care use, increased referral to palliative care specialists and improved advance care planning.^21–25 Despite common use of screening tools in other settings, the performance of these tools in the nursing home has not been well established, leaving no agreed upon ‘gold standard’ for use in nursing homes.

Nursing homes are unique care environments that may differ significantly from other settings thus limiting direct application of tools and criteria developed for other settings (primary care clinic, hospital). Nursing homes differ from primary care and outpatient palliative care clinics in the availability of onsite nursing staff around the clock and easy access to primary care clinicians for both routine and emergent care. Nursing home clinicians are typically skilled in caring for seriously ill older adults. They also have varying levels of general palliative care knowledge and skills in symptom management, facilitating goals of care conversations, discussing prognosis, advance care planning and providing anticipatory guidance. Thus, a resident in the nursing home may have the ability to receive primary palliative care from their primary care provider without the need for specialty palliative care consultation.^26–28 The high volume of projected palliative care needs in nursing homes coupled with available nursing home providers capable of providing some level of primary palliative care means that a minority of patients will need care from palliative care specialists. Palliative care screening tools and protocols in other settings are focused on identifying individuals for specialist level palliative care, which may limit the direct adoption of existing palliative care protocols from these settings.

Another challenge in application of screening tools designed for other settings is that most staff in nursing homes are not as skilled or educated as other settings (i.e., hospital) and most care is provided by nursing assistants and licensed practical nurses (LPNs). Registered nurses (RNs) have been found to have higher palliative care practice and knowledge scores than both LPNs and nursing assistants who have less clinical training.¹³ Moreover, some research suggests that social workers without training in the medical aspects of resident care are being expected to trigger the discussion and referral to palliative care.²⁹ Therefore, to be effective in the nursing home setting, a screening tool needs to be accessible to a wide range of staff and clinicians, easy for them to use regardless of their experience and expertise with palliative care. This further limits the application or adoption of existing palliative care screening tools that may not be understood by staff with limited medical or palliative care knowledge.

Therefore, the purpose of this systematic review is to: (1) identify palliative care screening tools that have been validated with nursing home residents; (2) critically appraise, compare and summarize the quality of the measurement properties for palliative care screening tools with nursing home residents, and (3) provide evidence-based recommendations in the selection of a palliative care screening tool for use in research and clinical practice in nursing homes.

METHODS

This systematic review was conducted in accordance with the Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) and the Joanna Briggs Institute (JBI) guidelines for systematic reviews of measurement properties.^30–32 The COSMIN criteria were developed to standardize terminology and definitions of psychometric properties and provide guidance on best methods for developing and validating instrument properties.^{32, 33} An a priori protocol was registered in PROSPERO (CRD42022345890).

Search Strategy and Study Selection

A comprehensive search of the literature was performed in May 2022 following collaboration with a health science librarian. Databases searched included Embase (Ovid), MEDLINE (PubMed), CINAHL (EBSCO), and PsycInfo (Ovid), with no date or language restrictions. The sensitive filter for measurement properties by Terwee et al, was used to identify articles reporting measurement properties of instruments.³⁴ The search strategy, including all identified keywords and index terms, was adapted for each included database. The full search strategies are provided in Supplemental Digital Content (SDC) A. Additional records were identified through backward citation searching and via reference lists of included articles. The authors used a structured program available at Covidence.org to organize the review process.³⁵

As recommended by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA),^{36, 37} two reviewers independently screened titles and abstracts against the inclusion criteria. Potentially relevant studies were retrieved and assessed in detail against the inclusion criteria by two independent reviewers. Reasons for exclusion of full-text studies that did not meet the inclusion criteria are reported in SDC B. Any disagreement between reviewers at any stage of the study selection and assessment process was resolved through discussions between the reviewers or with a third reviewer.

Eligibility Criteria

For this review, a palliative care screening tool was defined as an instrument that was developed or used to identify patients with possible palliative care needs intended to initiate primary or specialist level palliative care.

Inclusion criteria were: (1) studies that reported the development or evaluation of a palliative care screening tool; (2) included older adults living in a nursing home; and (3) published in peer-reviewed journals from database inception to May 2022. Studies conducted in mixed settings (e.g., community, hospital) were included if data for nursing home residents were reported separately.

Exclusion criteria were: (1) sample median age less than 65 years (unless data for older adults were reported separately); (2) prognostic tools (including the Surprise Question); (3) instruments with the purpose of describing or evaluating symptoms (e.g., pain assessment tools); (4) abstract only articles; (5) review articles; (6) studies that were conducted exclusively on patients already receiving palliative care; and (7) studies that only used the palliative care screening tool as an outcome measure (as per COSMIN guidelines).³⁰

Palliative care is appropriate for all patients beginning at the time of diagnosis with a serious illness and is not restricted to end of life, therefore tools that used only prognosis (i.e., the Surprise Question) were excluded because they are intended to identify patients nearing end of life and may not reveal palliative care needs (e.g., pain management).³⁸

We defined palliative care needs as the ability to benefit from palliative care which is not restricted to physical benefit but can also include emotional, social and spiritual support. Tools developed solely to measure single symptoms were excluded unless they were applied with the intent of initiating palliative care.

The sample age requirement was relaxed for studies reporting development and content validity of an instrument because the purpose of conceptualization is to ensure that the instrument measures what it purports to measure, and such articles may still provide evidence of content validity. Also, such studies are often qualitative in nature and may be limited to perspective of healthcare providers only.³⁰

Overview of Evaluation of Measurement Properties

The evaluation of measurement properties, conducted by two independent reviewers, proceeded in 4 stages. Any disagreement between reviewers was resolved through reviewer discussion. The process began with the evaluation of the methodological quality of each study using the COSMIN Risk of Bias checklist (stage 1). Following data extraction (stage 2), the result of each study on a measurement property was evaluated against the criteria of good measurement properties (stage 3). Finally, the evidence was summarized by measurement property and the quality of the evidence graded using a modified GRADE approach (stage 4).

Assessment of Methodological Quality (Stage 1)

Selected studies were assessed for methodological quality using the COSMIN Risk of Bias Checklist which contains standards for instrument development, content validity, structural validity, internal consistency, cross-cultural validity, reliability, measurement error, criterion validity, hypotheses testing for construct validity, and responsiveness.³⁹ The process began with assessment of the standards for instrument development and content validity. As a first step in evaluating the quality of instrument development and content validity, COSMIN recommends checking for existing ratings. If a rating of the instrument development and content validity exists, COSMIN recommends that this rating be used instead of repeating and duplicating the assessment.⁴⁰ Next, for all instruments with acceptable development and content validity, only the standards relevant to the measurement properties reported were evaluated. Each subsection of a standard was rated as ‘very good’, ‘adequate’, ‘doubtful’, ‘inadequate’ or ‘not applicable’, with the final scoring of each property based upon the lowest score. To meet the COSMIN methodological gold standard for measurement instruments, all subsections of a standard must be met.

Data Extraction (Stage 2)

Data were extracted following the assessment of methodological quality. The following information on each study was extracted: study (author and year), population (sample), setting, instrument description, measurement properties and psychometric values.

Assessment of Measurement Properties (Stage 3)

In stage 3, measurement properties were appraised using the COSMIN updated criteria for good measurement properties.³⁰ COSMIN provides guidance for assessing measurement properties with each result rated as either sufficient (+), insufficient (−), or indeterminate (?).

Grading Quality of Evidence (Stage 4)

To determine the overall quality of the instrument, we evaluated the quality of the evidence using the Modified Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach for systematic reviews of clinical trials recommended by COSMIN.³⁰ This approach grades the quality of the evidence as high, moderate, low or very low, providing an indication of the trustworthiness of the pooled or summarized results. Factors assessed included: 1) risk of bias (quality of studies); 2) inconsistency (of the results of studies); 3) imprecision (total sample size); and 4) indirectness (evidence from different populations).³⁰

RESULTS

The initial search produced 2184 results with one additional article identified through citation searching. After removal of duplicates, 1557 articles remained to be screened. After reviewing all abstracts, 1524 were excluded because they did not meet study inclusion criteria. The remaining 33 full-text articles were read and reviewed. Twenty-eight were excluded after full-text review due to ineligible study type (n=5), not measuring palliative care needs (n=10), not concerning the target population of interest (n=2), not concerning a palliative care screening tool (n=9), or lack of reporting on development or measurement properties (n=1) (SDC B). Five articles reporting data on four instruments met the criteria for quality assessment.

Articles describing the Gold Standards Framework Prognostic Indicator Guidance (GSF-PIG), Supportive Palliative Care Indicators Tool (SPICT) and Palliative Care Needs Rounds Checklist (PCNR) were excluded during assessment of methodological quality due to confounding of results (GSF-PIG;⁴¹ SPICT⁴²), lack of separate reporting for nursing home residents (GSF-PIG⁴¹) and lack of reporting on instrument development and initial content validity (GSF-PIG;⁴¹ PCNR⁴³) as reported in Table 1. Ultimately, two articles reporting on one instrument were included.^{44, 45} See PRISMA diagram Figure 1.

Table 1:

Studies excluded following assessment of methodological quality

Reference	Reason for exclusion
43 Forbat, L., M. Chapman, C. Lovell, W.M. Liu, and N. Johnston, Improving specialist palliative care in residential care for older people: a checklist to guide practice. BMJ Support Palliat Care, 2018. 8(3): p. 347–353.	“Inadequate” risk of bias Lack of data about initial content validity Palliative care needs rounds checklist
41 Grossman, D., Y. Grossman, E. Nadler, M. Rootenberg, J. Karuza, and A. Berall, Integrating Palliative Care Assessment Tools to Enhance Understanding of Illness Trajectory in Post-Acute Care and Long-Term Care. The American journal of hospice & palliative care, 2021: p. 10499091211018193.	Lack of data about initial content validity. Confounds results by combining GSF-PIG with PPS. Data for nursing home residents not reported separately.
42 Liyanage, T., G. Mitchell, and H. Senior, Identifying palliative care needs in residential care. Aust J Prim Health, 2018. 24(6): p. 524–529.	Evaluates the SQ as predictor of death at 1 year. Provides prevalence of conditions/criteria for SPICT items for SQ+SPICT+ residents – but no comparisons. SPICT measurement properties not reported.

GSF-PIG, Gold Standards Framework Prognostic Indicator Guidance; PPS, Palliative Performance Scale; SQ, Surprise Question; SPICT, Supportive Palliative Care Indicators Tool; +, positive

Figure 1:

PRISMA Diagram

Characteristics of Included Studies

Each of the final articles included in this systematic review describe studies that addressed the development or use of the NEC-PAL. Of the two included studies, one reported on the construction of the NEC-PAL tool for the purpose of identifying patients with advanced chronic conditions and provided preliminary prevalence of these patients in the general population.⁴⁴ The second study aimed to establish the most suitable indicators for identification of nursing home residents with palliative care needs and limited life expectancy.⁴⁵ Both studies originated in Spain. Publication dates ranged from 2013 to 2021.

Study samples included older adults with chronic conditions from diverse settings including urban, rural, and rural-urban primary care clinics, an acute bed hospital, a social-health center, and 11 nursing homes. The NEC-PAL was developed by physicians; however, other health care professionals including nurses, social workers, and psychologists participated in its testing.⁴⁴

Description of Included Instruments

NEC-PAL characteristics are described in Table 2 and measurement properties from included studies are described in Table 3. The purpose of the “NECesidades Paliativas” (NEC-PAL), which translates to “palliative needs” in the English language, is to identify patients who may benefit from palliative care supports in all settings, including primary care, hospital, and nursing home. It is not intended to determine prognosis or survival. The NEC-PAL was developed through translating and adapting questions from the GSF-PIG and SPICT but also included other dimensions (i.e., “demand” or “need” for palliative care). Content validity was tested with an expert interdisciplinary panel of physicians, nurses, social workers, and psychologists from acute hospital, cancer center, social-health center, and primary care with specialties in primary care, oncology, geriatrics, internal medicine, neurology, nephrology, and palliative care.⁴⁴ Final validity testing occurred in Spain with 11 primary care services, 160 bed hospital, 2 social health centers, and 22 nursing homes.

Table 2:

Characteristics of Identified Instruments – Revised entire table

	NEC-PAL	SPICT	GSF-PIG	PCNR Checklist
Surprise question	Yes, 12 months	No	Yes, 12 months	Yes, 6 months
Global Indicator	Patient, family, or team request for palliative care	Patient request for palliative care		Transferred to facility for end-of-life care
General Indicators
Functional Status	Functional decline (Karnofsky or Barthel score <30%. Loss of >2 ADLs) Severe dependence (Karnofsky <50 or Barthel <20)	In bed >50% of the day Performance status is poor or deteriorating, with limited reversibility. Depends on others for care	Functional performance status decline, increasing dependence and need for support In bed or chair >50% of day	Physical decline in the last month
Cognitive status	Cognitive decline (>3 Pfeiffer)	Not addressed	Not addressed	Cognitive decline in the last month
Weight loss	>10% in 6 months	Progressive weight loss, remains underweight, or low muscle mass	>10% in 6 months Serum albumin <25g/l	Not addressed
Hospital admissions	≥2 urgent or unplanned admissions in the last 6 months	Unplanned hospital admission(s)	Repeated unplanned crisis admissions	Not addressed
Other	≥2 Geriatric syndromes (recurrent or persistent): Falls, dysphagia, pressure ulcers, delirium, recurrent infections ≥2 Persistent or refractory symptoms (pain, weakness, anorexia, digestive, etc.) Emotional distress, severe social vulnerability	Persistent symptoms despite optimal treatment Chooses to reduce, stop, or not have treatment, wishes to focus on quality of life	Significant comorbidities Unstable, deteriorating, complex symptom burden Decreasing response to treatments Decreasing reversibility Desires no further active treatment, wishes to focus on quality of life	Exacerbation of symptoms in last month No plans in place for last 6 months of life No advance care plan Conflict within family around treatment and care options
Disease-specific or clinical indicators	Cancer, COPD, chronic heart disease, liver, renal, stroke, dementia, neurodegenerative diseases, AIDS, other advanced illnesses	Cancer, dementia/frailty, neurological disease, heart/vascular disease, respiratory disease, kidney disease, liver disease Multiple conditions and/or complications that are not reversible	Cancer, Organ failure (heart disease, COPD, kidney disease, liver disease, neurological disease, Parkinson’s disease, Motor Neuron Disease, Multiple Sclerosis); Frailty, dementia, multi-morbidity, stroke	Not addressed
Setting Use	Primary care Hospital	Primary care Hospital	Primary Care Hospital	Nursing Home
Purpose	To identify persons with palliative care needs and life-limiting prognosis to actively improve the quality of their care.	To help identify people whose health is deteriorating and assessment them for unmet supportive and palliative care needs.	To enable earlier identification of people nearing the end of life who may need additional supportive care.	To support the integration of specialist palliative care into nursing homes to improve quality care.

NEC-PAL, NECesidades Paliativas; SPICT, Supportive and Palliative Care Indicators Tool; GSF-PIG, Gold Standards Framework Prognostic Indicator Guidance; PCNR, Palliative Care Needs Rounds; ADL, activities of daily living; COPD, chronic obstructive pulmonary disease; AIDS, Acquired Immune Deficiency Syndrome.

Table 3.

Description of measurement properties of included studies

Instrument	Study Design and authors	Measurement properties reported Study population, N Country	Age mean (SD, range)	Gender % women Setting % NH	Results of measurement properties	Proportion of patients in need of palliative care
NECPAL	Development and prevalence study (Gomez-Batiste et al)44	Development and content validity Participants: doctors, nurses, social workers, and psychologists, experts in palliative care, N=18 Spain	NR	NR	Development & content validity Translation, cultural and clinical adaptation of relevant items of GSF PIG and SPICT were accomplished with three rounds of expert consultations. Appropriateness, comprehensiveness, and feasibility were assessed through interviews.	N/A
		Face validity & comprehension Participants: doctors, nurses, social workers, and psychologists, experts in palliative care, N=18 Spain	NR	NR	Face validity & comprehension 18 interviews performed for comprehension and face validity in primary care settings with five versions of the tool	N/A
		Prevalence & Feasibility Patients with advanced chronic conditions from nine different care services (3 primary care services, 4 nursing homes, 1 hospital, 1 social-health center), N=1,064 Spain	81.7 years (SD 12) for the SQ+ patients	61.5% of the SQ+ sample were women 22.1% of the SQ+ sample were NH residents	Feasibility Assessed through two focus groups of eight multidisciplinary healthcare professionals	Prevalence 70.5% of patients with chronic conditions were surprise question positive 64.3% of patients with chronic conditions were NECPAL+ indicating unmet palliative care needs. 91.2% of SQ+ patients with chronic conditions were NECPAL+ 7% of population >65 years of age were NECPAL+.
	Prevalence and correlational study (Esteban-Burgos et al)45	Hypothesis testing for construct validity Participants: Complex chronic patients with cancer, dementia, specific organ failure, N=149, (7 nursing homes) Spain	84.5 years (SD 9) total sample	67.1% women total sample 100% of the total sample were NH residents	Hypothesis testing for construct validity Case Complexity Index (CaCI) Spearman correlation 0.133 (NS) with NECPAL+	N/A
		Hypothesis testing for construct validity Participants: Complex chronic patients with cancer, dementia, specific organ failure, N=149, (7 nursing homes) Spain	84.5 years (SD 9) total sample	67.1% women total sample 100% of the total sample were NH residents	Hypothesis testing for construct validity Frail-VIG Spearman correlation +0.405 (p<.01) with NECPAL+	N/A
		Hypothesis testing for construct validity Participants: Complex chronic patients with cancer, dementia, specific organ failure, N=149, (7 nursing homes) Spain	84.5 years (SD 9) total sample	67.1% women total sample 100% of the total sample were NH residents	Hypothesis testing for construct validity IDCPal+ Spearman correlation +0.375 (p<.01) with NECPAL+	N/A
		Hypothesis testing for construct validity Participants: Complex chronic patients with cancer, dementia, specific organ failure, N=149, (7 nursing homes) Spain	84.5 years (SD 9) total sample	67.1% women total sample 100% of the total sample were NH residents	Hypothesis testing for construct validity PPS Spearman correlation −0.374 (p<.01) with NECPAL+	N/A
		Hypothesis testing for construct validity Participants: Complex chronic patients with cancer, dementia, specific organ failure, N=149, (7 nursing homes) Spain	84.5 years (SD 9) total sample	67.1% women total sample 100% of the total sample were NH residents	Hypothesis testing for construct validity PPI Spearman correlation +0.444 (p<.01) with NECPAL+	N/A
		Prevalence Participants: Complex chronic patients with cancer, dementia, specific organ failure, N=149, (7 nursing homes) Spain	84.5 years (SD 9) total sample	67.1% women total sample 100% of the total sample were NH residents	Prevalence Assessed with positive surprise question	Prevalence 53% of complex chronic residents were SQ+ Prevalence of NECPAL+ not reported

SD, standard deviation; NR, not reported; SQ+, surprise question positive; NECPAL+, positive screen for palliative care needs; ICDPal+, positive screen for palliative care complexity

The NEC-PAL contains 10 items, structured as a checklist, including the surprise question (Would you be surprised if the patient died in the next 12 months? “No” is considered a positive response). For the NEC-PAL to be considered positive, indicating potential palliative care needs, the surprise question must be positive in addition to the presence of one or more other indicators. Developed in Spanish, the NEC-PAL, has been culturally adapted to Brazil (Portuguese),⁴⁶ Czech Republic (Czech)⁴⁷ and Chile.⁴⁸ It has been translated into English for publication but has not been culturally or linguistically adapted or validated.⁴⁹

Assessment of Methodological Quality (Risk of Bias)

Following the aforementioned COSMIN guidance regarding the use of prior ratings, we identified a prior review of measurement properties of palliative care screening tools for use in the hospital setting which rated the quality of the development and content validity of NEC-PAL as “doubtful.”⁵⁰ As recommended by COSMIN this would lead to exclusion from further evaluation, however, Luthi et al believed this to be related to the construct of palliative care being poorly defined and did not exclude the NEC-PAL for this reason.⁵⁰ For consistency in application of COSMIN criteria, these ratings were adopted for this review and the NEC-PAL was not excluded.

Structural validity, internal consistency, cross-cultural validity, reliability, measurement error, criterion validity and responsiveness were not assessed because these were not reported in the identified studies. Assessment of methodological quality (risk of bias) for construct validity was rated as adequate and is presented in Table 4.

Table 4.

Table on results of studies on measurement properties of instruments for the identification of patients in need of palliative care in the nursing home setting

Instrument	Country (language) in which the instrument was evaluated	Structural validity			Internal consistency			Cross-cultural validity/measurement invariance			Reliability
		n	Meth qual	Result (rating)	n	Meth qual	Result (rating)	n	Meth qual	Result (rating)	n	Meth qual	Result (rating)
NECPAL	Spain	0			0			0			0
Pooled or summary result (overall rating)		0			0			0			0

Instrument	Country (language) in which the instrument was evaluated	Measurement error			Criterion validity			Hypotheses testing (Construct validity)			Responsiveness
		n	Meth qual	Result (rating)	n	Meth qual	Result (rating)	n	Meth qual	Result (rating)	n	Meth qual	Result (rating)
NECPAL	Spain	0			0			149	Adequate	Results in line with 6 hypo’s (6+)
Pooled or summary result (overall rating)		0			0			149		6+ Overall +

Assessment of Measurement Properties

Data on measurement properties for the NEC-PAL were limited to six hypotheses tests for construct validity as reported by one article.⁴⁵ None of the studies reported on structural validity, internal consistency, cross-cultural validity, reliability, measurement error, criterion validity or responsiveness. Measurement properties from included studies are described in Table 4.

Hypotheses Testing for Construct Validity

Spearman correlations for the Case Complexity Index, Frail VIG, Diagnostic Instrument for Complexity in Palliative Care (IDC-Pal), PROFUND index, Palliative Performance Scale (PPS), and Palliative Prognostic Index (PPI) in patients with a positive NEC-PAL were reported. COSMIN recommends correlations with instruments measuring similar constructs should be ≥ 0.50.³⁰ Correlations with related but dissimilar constructs such as the Frail VIG (0.405), Case Complexity Index (0.375), IDC-Pal (0.375), PPS (−0.374), and PPI (0.444) should be lower (i.e., 0.30–0.50).³⁰ Unrelated constructs, such as the PROFUND index (0.148), should have correlations <0.30.³⁰ All six correlations were positive and in line with predetermined hypotheses (higher scores on NEC-PAL correlated with higher scores on the comparator instrument) with the exception of the PPS. The PPS is scored with lower scores indicating increased mortality risk and palliative care needs, inversely related to the NEC-PAL scoring with higher scores indicating increased palliative care needs. Therefore, an inverse correlation was expected.

GRADE Levels of Evidence

Whereas the assessment of methodological quality focuses on the quality of single studies of individual measurement properties, GRADE levels of evidence focus on the quality of the instrument as a whole.³⁰ This approach generally begins with the pooling of results from individual studies, however, due to each study reporting different measurement properties (content validity, construct validity) this was not possible.

Using the GRADE approach, all instruments are initially considered of high quality and subsequently downgraded by one or two levels per factor when there is risk of bias, inconsistency (among studies), imprecision (low sample size), or indirect results (evidence from different populations).³⁰ Following this approach, the NEC-PAL was rated as having low quality evidence for use in nursing homes after downgrading for “doubtful” risk of bias rating for content validity and the indirectness of the evidence, (only part of the study population consisted of nursing home residents). A low-quality GRADE level of evidence indicates that confidence in the measure is limited and may be substantially different from the true measurement property.³¹

Discussion

This systematic review aimed to identify palliative care screening tools with robust development, comprehensive content and strong methodological quality, applicable to nursing home residents, using COSMIN criteria. Out of the four instruments identified, three were excluded during quality assessment due to various reasons reported in Table 1. Ultimately, one instrument, the NEC-PAL met COSMIN criteria for inclusion in the final analysis. Evaluation of the NEC-PAL revealed lack of robust testing of measurement properties in nursing home residents, leading to the conclusion that there is no ‘Gold Standard’ screening tool for recommended use in nursing homes at this time.

To improve the clinical relevance of this review, the section that follows reports on the screening tools identified during the full-text review including those that failed to meet COSMIN quality assessment criteria and were excluded from the analysis presented earlier.

Existing Tools

Four screening tools were identified during full-text review: the NEC-PAL (discussed above), the GSF-PIG, SPICT, and PCNR Checklist. All instruments were intended to support the clinical judgement of multidisciplinary teams seeking to identify patients who might benefit from palliative care assessment and care planning. Table 2 presents a comparison of these screening tools.

The SPICT was developed in Scotland for use in all care settings to facilitate early identification of patients with advanced life-limiting conditions. The SPICT was developed based on a literature review, peer review and a prospective case-finding study of patients with advanced kidney, liver, cardiac, or lung disease.⁵¹ The SPICT is free to use and is available in English, Danish, Spanish, Italian, and German.⁵² A comprehensive website of SPICT resources is available at https://www.spict.org.uk/. In the reviewed study, Liyanage et al, evaluated the accuracy, feasibility and acceptability of the surprise question, followed by application of the SPICT for those at risk of death in the next 12 months.⁴² This article was excluded from this review due to methodological confounding and lack of reporting on measurement properties of the SPICT.

The GSF-PIG was developed for use in the United Kingdom to improve earlier identification of patients in the last year of life. GSF-PIG is free to use and is available in English and Italian.⁵³ A comprehensive website of GSF-PIG resources is available at https://www.gsfinternational.org.uk/pig-tool and is currently under revision.⁵⁴ The GSF-PIG was originally developed for community use but has since been validated in the acute hospital setting.⁵⁵ In the reviewed study, Grossman et al, evaluated the GSF-PIG combined with the Palliative Performance Scale (PPS) in a sample of 40 patients from an academic geriatric center which included 10 residents from a long-term care unit.⁴¹ Interprofessional staff reported the combined GSF-PIG/PPS improved their awareness of palliative care goals for patients at the study sites and was easy to use. Use of the GSF-PIG combined with the PPS into a single screening instrument prevented evaluation of the GSF-PIG. Moreover, the data for nursing home residents was not reported separately. For these reasons this article was excluded from this review.

The PCNR checklist was developed in Australia to guide palliative care needs rounds which is described as a mechanism for triaging residents with palliative care needs in nursing homes. The PCNR checklist is used by nursing home staff during monthly clinical meetings with a palliative care specialist conducted at the nursing home.⁴³ The checklist begins with a brief list of triggers for identifying residents to discuss. The remainder of the checklist guides the flow of the meeting with open ended bullets including recommendations for further action. The checklist was developed based on a literature review and a grounded theory ethnography.⁴³ In the reviewed study, Forbat et al, describe the development of the PCNR checklist.⁴³ It was excluded due to “inadequate” risk of bias for lack of reporting on initial content validity and lack of reporting on measurement properties.

Based on this review, we identified one palliative care screening tool (NEC-PAL) that met COSMIN criteria but evidence for use with nursing home residents was of low quality. The relatively limited number of palliative care screening tools identified in our review (1 tool) is lower than reports from reviews in other settings such as primary care (6 tools⁵⁶ and 10 tools⁵⁷), hospital (4 tools⁵⁰) and general practice (4 tools⁵⁸), indicating the lack of study in this setting.

Despite lack of a ‘Gold Standard’ it is clear that clinicians practicing in this environment need some guidance on selecting a tool. Table 2 presents an overview of the four identified instruments and may be used by clinicians in selecting an instrument that best meets their needs. The NEC-PAL, SPICT and GSF-PIG are more comprehensive in scope than the PCNR Checklist, however the PCNR Checklist was developed specifically for use in nursing homes. The NEC-PAL uses complex criteria such as Karnofsky, Barthel, and Pfeiffer scoring which may make application by nursing home staff with limited medical knowledge difficult. Overall, the SPICT and GSF-PIG may be appropriate for nursing home use until further evidence can be gathered.

An additional criticism of the NEC-PAL, GSF-PIG, and PCNR Checklist is the use of the surprise question as a decision point for completing the instrument. The surprise question has been shown to perform poorly to modestly as a predictive tool for death, with worse performance in patients without cancer.³⁸ Moreover, prior reports have shown that clinicians are inaccurate at prognostication and may lead to residents with palliative care needs going unrecognized.⁴⁵ In one study that evaluated the use of the surprise question as the decision point to complete other palliative care tools, authors found that its use excluded a substantial proportion of patients who had palliative care needs with a longer life expectancy.⁴⁵The SPICT is the only reported instrument that does not use a limited prognosis (<6–12 months) as a decision point for completing the instrument which may broaden the application beyond end of life.

Based on findings in other settings it is likely that one can differentiate between individuals with palliative care needs versus those who are not strong candidates for palliative care through use of a palliative care screening tool.^21–25 While palliative care screening tools exist, our review shows that none have been robustly tested in nursing homes. This represents a major gap in the literature as there are unique characteristics of nursing home care that may be particularly relevant for identifying palliative care needs that differ from other settings.

Finally, an ideal palliative care screening tool has many meaningful uses in nursing homes, such as triggering discussions about care goals and setting treatment priorities. Moreover, such an instrument my facilitate timely referral to primary palliative care by nursing home clinicians, many of whom have rigorous training in geriatrics and are well-versed in the unique needs of nursing home residents. Lastly, an additional use may be to identify individuals for enrollment in quality improvement and research projects.

Implications for Research and Practice

Based on our results there are various implications for future research and practice. First, the existing evidence is limited and consequently, we are unable to provide a ‘gold standard’ recommendation for use in nursing homes. The only palliative care screening tool that met criteria, the NEC-PAL is limited by the use of prognosis, inadequate testing with nursing home samples, and lack of cultural and linguistic adaptation to U.S./English.

Research to validate available instruments, such as those identified in this review, for nursing home use is sorely needed. However, given the unique features of the nursing home environment, designing and testing a palliative care screening tool specifically for nursing home use is recommended. Once tools are developed, future research should aim to determine how palliative care screening tools are used by nursing home staff who are typically untrained in palliative care and particularly how these instruments could enable nursing home staff to determine unmet palliative care needs in a timely manner within a population that experiences a high prevalence of functional, sensory and cognitive deficits. Until available instruments are tested, or new instruments developed, clinicians may wish to consider the evidence presented here and choose a screening instrument that best meets their needs.

Strengths and Limitations

A strength of this systematic review was the use of COSMIN quality criteria, which is seen as a methodological gold standard for this type of evaluation.³² In addition, this review benefited from an extensive search without date or language restrictions, including consultation with a health sciences librarian. Still, this review is not without limitations. While thorough in our review process by searching several databases and handsearching citations, we only included papers and instruments that were published as a full report of the study. In eliminating studies only reported in abstract format we may have missed a relevant study or instrument not published in the scientific literature with a full report. However, this enabled us to fully evaluate the instruments.

One of the challenges in this review was what precisely constitutes a palliative care screening tool. Checklist type instruments that indicate a patient might have an unmet palliative care need and may warrant further evaluation was defined as a practical means of screening which excluded instruments that simply measured symptoms such as pain or depression.

Finally, when using COSMIN to assess methodological properties, it should be noted that we are evaluating development studies reported prior to release of the COSMIN guidelines, and this may account, in part, for the low assessed quality. Moreover, COSMIN tends to result in low ratings due to the use of the lowest score method.

Conclusion and Implications

Based on this systematic review, we identified only one palliative care screening tool meeting COSMIN criteria, the NEC-PAL, but evidence for use with nursing home residents was of low quality. The NEC-PAL lacked robust testing of measurement properties such as reliability, sensitivity and specificity in the nursing home setting. Construct validity through hypothesis testing was adequate but only reported in one study. Consequently, there is insufficient evidence to guide practice. Broadening the criteria further, this review reports on three additional palliative care screening tools (SPICT, GSF-PIG, PCNR Checklist) identified during the full-text review process but which were excluded during quality assessment for various reasons. Given the unique care environment of nursing homes, we recommend the development of a new instrument specifically designed for nursing home use. Until that time we recommend that clinicians consider the evidence presented here and choose a screening instrument that best meets their needs.