Palliative Care Planner: A Pilot Study to Evaluate Acceptability and Usability of an Electronic Health Records System-integrated, Needs-targeted App Platform

Abstract

Rationale: The quality and patient-centeredness of intensive care unit (ICU)-based palliative care delivery is highly variable.

Objective: To develop and pilot an app platform for clinicians and ICU patients and their family members that enhances the delivery of needs-targeted palliative care.

Methods: In the development phase of the study, we developed an electronic health record (EHR) system-integrated mobile web app system prototype, PCplanner (Palliative Care Planner). PCplanner screens the EHR for ICU patients meeting any of five prompts (triggers) for palliative care consultation, allows families to report their unmet palliative care needs, and alerts clinicians to these needs. The evaluation phase included a prospective before/after study conducted at a large academic medical center. Two control populations were enrolled in the before period to serve as context for the intervention. First, 25 ICU patients who received palliative care consults served as patient-level controls. Second, 49 family members of ICU patients who received mechanical ventilation for at least 48 hours served as family-level controls. Afterward, 14 patients, 18 family members, and 10 clinicians participated in the intervention evaluation period. Family member outcomes measured at baseline and 4 days later included acceptability (Client Satisfaction Questionnaire [CSQ]), usability (Systems Usability Scale [SUS]), and palliative care needs, assessed with the adapted needs of social nature, existential concerns, symptoms, and therapeutic interaction (NEST) scale; the Patient-Centeredness of Care Scale (PCCS); and the Perceived Stress Scale (PSS). Patient outcomes included frequency of goal concordant treatment, hospital length of stay, and discharge disposition.

Results: Family members reported high PCplanner acceptability (mean CSQ, 14.1 [SD, 1.4]) and usability (mean SUS, 21.1 [SD, 1.7]). PCplanner family member recipients experienced a 12.7-unit reduction in NEST score compared with a 3.4-unit increase among controls (P = 0.002), as well as improved mean scores on the PCCS (6.6 [SD, 5.8]) and the PSS (−0.8 [SD, 1.9]). The frequency of goal-concordant treatment increased over the course of the intervention (n = 14 [SD, 79%] vs. n = 18 [SD, 100%]). Compared with palliative care controls, intervention patients received palliative care consultation sooner (3.9 [SD, 2.7] vs. 6.9 [SD, 7.1] mean days), had a shorter mean hospital length of stay (20.5 [SD, 9.1] vs. 22.3 [SD, 16.0] patient number), and received hospice care more frequently (5 [36%] vs. 5 [20%]), although these differences were not statistically significant.

Conclusions: PCplanner represents an acceptable, usable, and clinically promising systems-based approach to delivering EHR-triggered, needs-targeted ICU-based palliative care within a standard clinical workflow. A clinical trial in a larger population is needed to evaluate its efficacy.

There is a substantial need for high-quality, multidisciplinary palliative care in an intensive care unit (ICU) setting (1). Many of the 5.7 million annual ICU patients suffer from serious physical and emotional symptoms in a setting often marked by the progressive accumulation of dehumanizing invasive life support technologies (2) and represent a population constituting nearly 20% of all U.S. deaths, a frequency that is climbing despite increasing hospice use nationwide (3–5). Family surrogate decision makers commonly report poor communication, unsupported decision making, and low medical comprehension that can promote discordance between patients’ care preferences and the treatments they actually receive (6–8). Palliative care interventions led by ICU teams and palliative care specialists can help address these patient and family member needs (9). However, the current delivery of ICU-based palliative care is highly variable by hospital (10–14), among physicians in the same ICU (15, 16), and even within individual physicians based on daily ICU characteristics (17), and often of lower quality compared with that delivered in other settings (18–20).

Improving the delivery of ICU-based palliative care to such a large population with unmet needs is challenging. There is a very limited (n = 5,500) U.S. palliative care specialist workforce, and many ICU clinicians report a lack of confidence in their ability to address patients’ palliative care needs comprehensively (21, 22). Process-of-care barriers also exist with identifying patients and family members with actual, and not assumed, unmet palliative care needs (23); engaging them as accepting partners in palliative care uptake (24); and efficiently coordinating care between ICU teams and palliative care specialists (25).

To address these common barriers to efficient and patient-centered care delivery, we sought to develop an electronic health record (EHR)-integrated app platform prototype to proactively identify patients and their family members with unmet palliative care needs. This manuscript describes the development and pilot evaluation of Palliative Care Planner (PCplanner), a web app platform that aims to use technology to improve the humanistic approach to ICU-based palliative care (26). Our specific aims were to understand clinicians’ and family members’ perceptions of PCplanner and to explore early evidence of clinical effect on the delivery of ICU-based palliative care and family member stress.

Methods

This project includes two phases, a development phase and an evaluation phase, together conducted during a 2-year period. In the development phase, we developed and validated EHR-based palliative care triggers; adapted the needs of social nature, existential concerns, symptoms, and therapeutic interaction (NEST) palliative care needs scale for use in an ICU setting; and then built the EHR-integrated PCplanner prototype. In the evaluation phase, we prospectively piloted PCplanner in a clinical setting and compared it with historical control populations of both ICU patients and ICU family members. This project was funded by a Duke Institute for Healthcare Innovation grant and conducted with Duke Institutional Review Board approval (Protocols 00071161 and 00069031). This manuscript has an online supplement. Portions of these data were presented at the 2017 American Thoracic Society International Meeting in Washington, DC.

Phase 1: Development of the EHR-integrated PCplanner App Platform Prototype

The development phase included three key tasks: establishing EHR-based e-triggers, adapting the NEST instrument for the ICU setting, and building the PCplanner web app platform itself.

EHR-based triggers

Our aim was to identify and evaluate palliative care triggers that were valued by clinicians (27), had capacity for uniform temporal ascertainment, and had evidence of association with disability, readmission, symptoms, and mortality (28). On the basis of feedback from local ICU physician and nursing leaders, we focused on five triggers relevant to elderly patients who were either receiving mechanical ventilation or were in shock more than 48 hours after ICU admission: dementia, declining health status, poor functional status, severe acute illness, or severe acute stroke (see Table 1 and online supplement, for more details).

Table 1.

E-triggers and their definitions

E-Triggers Piloted in PCplanner Mobile App
E-Triggers: Mechanical Ventilation or Shock: 48 Hours Plus One of These:	Data Sources Defining E-Triggers
Dementia (49–52):	E-triggers were built from Meaningful Use Common Data Set standards (53):
Alzheimer’s
Multi-infarct
Other dementia etiology
Declining health status (54–57):
≥2 hospital admissions in 3 months or	Vital signs
>1 ICU admission in 3 mo
Poor functional status (49, 51):	Demographics
Admit from SNF or LTAC…or…
≥3 ADL limitations at admission	Diagnoses (via real-time ICD-10–linked physician billing)
Severe acute illness (58):
Cardiac arrest...or...
Multisystem organ failure (≥3 of: lung, kidney, blood, brain, cardiac, liver) that has worsened over 48 h	Procedures (via SNOMED CT; ICD-10 crosswalks)
Severe acute stroke: acute intracranial hemorrhage, ischemic stroke, or traumatic brain injury	Laboratory values (via LOINC)
	Medications (via RxNorm NLM standards)

Definition of abbreviations: ADL = activities of daily living; ICU = intensive care unit; ICD-10 = International Statistical Classification of Diseases and Related Health Problems, 10th Revision; LOINC = Logical Observation Identifiers Names & Codes; LTAC = long-term acute care facility; NLM = National Library of Medicine; PCplanner = Palliative Care Planner; SNF = skilled nursing facility; SNOMED CT = SNOMED Clinical Terms.

NEST adaptation to ICU setting

We adapted NEST for the ICU setting, using a validated methodology used previously to adapt it for an emergency department setting (29), ensuring that all eight National Consensus Project domains of palliative care quality were represented (see online supplement) (30).

PCplanner app platform

We built the clinician- and family-facing PCplanner mobile web app prototype between early 2015 and mid-2016, guided by a previously described conceptual framework (26). PCplanner has three key data inputs including the EHR (e-trigger lists), family members (self-reported needs), and clinicians (approval of family member approach for study; i.e., palliative care consult). The PCplanner web app provides different content for different users (see online supplement). Family members can view video and text information about the purpose of palliative care and complete a need survey. After using university credentials to log in, clinicians can access a dashboard that allows them to view a list of e-trigger-positive patients, approve a palliative care consult for any patient on the list, and review family-completed NEST item scores. Before clinical deployment, we informally evaluated the usability of the family and clinician web app among programmers (n = 3), user experience experts (n = 1), ICU and palliative care clinicians (n = 8), and content-familiar clinical research coordinators (n = 4). Minimal user interface edits were made as a result. We also interrogated the integrity of the database, finding no disagreement between manually entered data and recorded data.

Phase 2: Evaluation of PCplanner and Comparison with Patient and Family Member Controls: Subjects and Setting

Intervention: screening and eligibility

Before enrollment, extensive discussions were held with ICU physician, ICU nurse, and palliative care leaders to ensure approval. PCplanner was used to automatically screen adult medical, surgical, cardiac, and neurologic ICUs at Duke University Medical Center each weekday to identify consecutive potential research subjects between June 6 and September 30, 2016, using the workflow shown in Figure 1. Of note, no study ICU had a protocolized approach to family communication or palliative care. Patients who appeared on PCplanner’s daily list were automatically screened by the app to ensure each met inclusion criteria including age 18 years or older, admission to an ICU for at least 48 hours, and presence of at least one of five e-triggers shown in Table 1. Each day, a nurse champion in each study ICU would open PCplanner, evaluate the list of e-trigger positive patients, discuss the case with the ICU attending, and record approval (or denial) for study participation in the PCplanner user interface. Each clinician-initiated change in the patient list would generate an automated email to the study clinical research coordinators (CRCs). CRCs discussed each approved patient with the ICU team before approaching them for informed consent. Patients were excluded if the ICU attending expected death within 24 hours, the patient was a prisoner, or the patient possessed decisional capacity, as this would substantially change the family member’s role. CRCs attempted to enroll at least three persons for each eligible patient: a family member, the attending ICU physician, and the bedside ICU nurse on the day of family member consent. Any family members aged at least 18 years and defined as related or unrelated persons self-identified as participating directly in health care decision-making or planned postdischarge supportive care for a patient were invited to participate. We excluded family members who needed translation assistance because of poor English fluency. Family members were compensated $25 for each study interview completed.

Figure 1.

Study flow for Palliative Care Planner (PCplanner) intervention. The intensive care unit team (ICU) reviewed PCplanner’s list of patients who met e-triggers and approved the study team approach of the family. After the first family data collection (including the needs of social nature, existential concerns, symptoms, and therapeutic interaction needs scale), the care teams were alerted and at least one palliative care-led family meeting was held. CRC = clinical research coordinator.

PCplanner operations

During the initial family member interaction, CRCs showed the prospective participant a short video (2 minutes) produced by our group, explaining the study. After obtaining informed consent, the family member completed a preintervention survey on a tablet computer. Family members were given login codes so they could access the PCplanner video content. After survey completion, study staff sent a text page to the palliative care team to alert them of a new consultation. Before conducting the palliative care consultation meeting with the family, the palliative care team used PCplanner to view the patient’s name and location, as well as the self-reported needs of the family member or members. After completion of at least one palliative care team/family meeting, and within 4 days after consent, family members then completed a digital postintervention survey. Nurse champions, palliative care specialists, and ICU physicians involved with the intervention completed a single brief survey after completion of all family enrollment.

Palliative care intervention

The content of the palliative care team/family meeting was unscripted, although practitioners were encouraged to explore the family-reported needs (NEST scale items) they viewed in PCplanner. At least two palliative care specialists conducted each family meeting. The ICU team was invited to attend all meetings as well. Only one meeting was required, although subsequent meetings were allowed per the palliative care team’s standard approach.

Controls

To provide a general context for the magnitude of any observed PCplanner effect on outcomes, we included both historical patient and family member control cohorts (Table E1). For Control A, we abstracted clinical outcomes from medical charts of all patients treated in adult medical and surgical ICUs at Duke University Medical Center who received an initial palliative care specialist consultation in the ICU in the 2 months preceding the intervention. For Control B, we prospectively enrolled consecutive family members of adult patients who received mechanical ventilation for more than 48 hours in the Duke medical ICU in the 4 months preceding the intervention period. Family exclusions and the timing of surveys they completed (day of consent and then 4 days later) were identical to those used for the PCplanner cohort; however, ICU clinicians were not provided with any survey results (e.g., unmet needs).

Measures and data collection

For all intervention and control patients, CRCs recorded sociodemographics as well as patient clinical data including APACHE II score on the day of enrollment, admission diagnosis, duration of ICU care and ventilation, and hospital disposition. For Control A patients, the timing of palliative care specialist consultation and the relationship of code status change was recorded. Family members in both intervention and Control B groups completed the 10-item NEST (0 = no need, 100 = greatest need) (31), the 4-item Perceived Stress Scale (0 = no stress, 16 = high stress) (32), the 12-item Patient-Centeredness of Care Scale to assess the degree to which families perceived ICU care to be focused on the patient (12 = lowest, 48 = highest) (33), and single-item metrics of goal concordance of treatment with patient values (“Do you feel that the medical care your loved one is getting fits with their values”?; strongly agree/agree = yes, disagree/strongly disagree = no), as well as quality of communication (scale of 0 [very worst I could imagine] to 10 [very best I could imagine]) (34). Intervention family members and ICU clinicians also reported PCplanner acceptability, with a 4-item version of the Client Satisfaction Scale (4 = worst, 16 = best) (35), and usability, with a 5-item version of the Systems Usability Scale (5-point Likert scale; 5 = worst, 25 = best) (36).

Statistical analyses

Analyses for this exploratory pilot study were primarily descriptive. Summary statistics are reported including number (frequency), mean (standard deviation), and median (interquartile range). Comparisons between intervention and control outcomes were performed using t tests and Wilcoxon rank-sum tests for continuous variables and Pearson’s chi square and Fischer’s exact tests for categorical variables. P < 0.05 was considered statistically significant. Stata version 14 was used for analyses.

Results

Among intervention participants, from 423 ICU patients screened, 385 were excluded, 24 refusals were experienced (14 physician, 10 legal decision maker), and 42 (14 patients, 18 family members, and 10 clinicians) participants were enrolled (Table E2). Seven (50%) physician refusals occurred because patient death was imminent, although just outside the window of the 24-hour expected death exclusion criterion. There were four family refusals within the first few weeks, although after introduction of our self-produced study video, only six further refusals occurred during the next 3 months. Characteristics of patients and family members from the intervention and control groups are shown in Table 2; clinician characteristics are shown in Table E3. Intervention patients were generally older adults and met e-triggers defined by advanced age plus either acute respiratory failure or shock. Intervention family members were primarily patients’ spouses and adult children. Control patients and family members were late-middle-aged to older adults. Septic shock and pneumonia predominated as ICU admission diagnoses; patients’ illness severity was high in all groups, based on APACHE II scores on the day of enrollment.

Table 2.

Characteristics of patients and family members

Characteristic	Intervention Patients (N = 14)	Intervention Family Members (N = 18)	Control A: Palliative Care ICU Patients (N = 25)	Control B: Medical ICU Family Members (N = 49)	Control B: Medical ICU Patients (N = 39)
Age, mean (SD), yr	77 (6.9)	59.1 (13.9)	66.3 (12.0)	53.7 (15.4)	55.3 (16.7)
Female sex, n (%)	6 (43)	9 (50)	9 (36)	37 (75)	20 (74)
Race, n (%)
American Indian/Alaskan Native	1 (7)	0	2	0	0
Black	9 (64)	12 (67)	6	10 (20)	6 (21)
White	4 (29)	6 (33)	17	39 (80)	21 (79)
Hispanic ethnicity, n (%)	0	0	0	0	0
Comorbidity scale, mean (SD)	2.6 (1.3)	—	2.5 (1.9)	—	2.1 (1.7)
Activities of daily living limitations, mean (SD)	4.3 (3.3)	—	—	—	—
APACHE II, mean (SD)	34.1 (6.3)	—	23.1 (7.4)	—	30.1 (8.6)
Trigger criterion met, n (%)*
Age ≥70 + ventilator for ≥48 h	10 (71)	—	5 (20)	—	8 (21)
Age ≥70 yr + ≥3 activities of daily living limitations	2 (14)	—	7 (28)	—	9 (23)
Dementia + ventilator for ≥48 h	2 (14)	—	2 (8)	—	4 (10)
Nursing home resident before ICU admission	2 (14)	—	1 (4)	—	1 (3)
Stroke or intracranial hemorrhage + ventilator for ≥48 h	2 (14)	—	4 (16)	—	6 (15)
Cardiac arrest + ventilator for ≥48 h	1 (7)	—	3 (12)	—	1 (3)
Insurance
Medicare	12 (86)	—			—
Commercial	2 (14)	—			—
ICU
Medical	6 (43)	—	14 (56)		39 (100)
Neurological	3 (21)	—	—		—
Surgical	2 (14)	—	11 (44)		—
Cardiac	1 (7)	—	—		—
Mixed medical-surgical	1 (7)	—	—		—
Primary ICU admission diagnosis
Septic shock, n (%)	6 (43)	—	3 (12)	—	8 (21)
Intracerebral hemorrhage, n (%)	2 (14)	—	2 (8)	—	3 (8)
Cardiac arrest, n (%)	1 (7)	—	—	—	1 (3)
Hemorrhagic shock, n (%)	1 (7)	—	2 (8)	—	2 (5)
Pneumonia, n (%)	1 (7)	—	9 (36)	—	18 (46)
Cardiogenic shock, n (%)	1 (7)	—	1 (4)	—	3 (8)
Postperative, n (%)	1 (7)	—	4 (16)	—	0
Seizure, n (%)	1 (7)	—	—	—	0
Cancer complications, n (%)	—	—	4 (16)	—	4 (10)
Relationship to patient, n (%)
Spouse or partner	—	10 (56)	—	28 (57)	—
Child	—	6 (33)	—	5 (10)	—
Parent	—	1 (6)	—	7 (14)	—
Other	—	1 (6)	—	9 (18)	—
Live with patient	—	8 (44)	—	28 (57)	—
Education
<High school	—	1 (6)	—	0	—
High school or trade school	—	4 (22)	—	12 (24)	—
<College	—	4 (22)	—	9 (18)	—
College	—	5 (28)	—	17 (35)	—
Graduate school	—	4 (22)	—	11 (22)	—

Definition of abbreviations: APACHE II = Acute Physiology and Chronic Health Evaluation II; ICU = intensive care unit.

^*Some patients met more than one trigger criterion.

There were fewer mean ICU days before palliative care consultation in the intervention group compared with the Control A palliative care patient group (3.6 [2.7] vs. 6.9 [7.1]), although this was not statistically significant (P = 0.21). Intervention patients had a lower mean hospital length of stay (20.5 days [9.1]) compared with Control A patients (22.3 days [16.0]) and Control B medical ICU patients (29.7 days [16.1]; Table 3 and Table E5). Intervention patients had shorter postconsultation mean hospital length of stay compared with Control A patients (7.9 [6.2] vs. 9.7 [7.9] days), although this was not statistically significant (P = 0.48). Hospital mortality was lower in the intervention group (n = 4; 29%) compared with Control A patients (n = 14; 56%), although intervention patients more frequently received hospice care (n = 5 [36%] vs. n = 5 [20%]) overall, and home hospice (14% vs. 0) in particular.

Table 3.

Patient clinical outcomes

Variable	Intervention Patients (N = 14)	Control A: Palliative Care ICU Patients (N = 25)	Control B: Medical ICU Patients (N = 39)
Hospital length of stay, total
Mean (SD)	20.5 (9.1)	22.3 (16.0)	29.7 (16.1)*
Median (IQR)	17.5 (14.8, 26.7)	18 (8.3, 35.8)	29 (17, 36)†
Hospital length of stay after palliative care consultation, d
Mean (SD)	7.9 (6.2)	9.7 (7.9)	—
Median (IQR)	6 (3, 12.5)	8 (2.5, 15.3)	—
Intensive care unit length of stay, total
Mean (SD)	16.1 (8.1)	11.5 (12.9)	15.1 (13.1)
Median (IQR)	15.5 (9.5, 25.3)	7 (4.3, 13)†
Intensive care unit length of stay before palliative care consultation, d
Mean (SD)	3.6 (2.7)	6.9 (7.1)	—
Median (IQR)	3 (1, 5.3)	4.5 (1, 7.8)	—
Intensive care unit length of stay after palliative care consultation, d
Mean (SD)	4.4 (4.2)	5.1 (7.1)	—
Median (IQR)	3 (1,7)	2 (1, 8)	—
Duration between eligibility and palliative care consultation, d
Mean (SD)	1.5 (2.4)	—	—
Median (IQR)	1 (0, 1)	—	—
Mechanical ventilation
n (%)	9 (64)	15 (60)	27 (100%)
Duration, days, mean (SD)	15.5 (6.6)	12.2 (12.0)	17.3 (12.0)
Duration, days, median (IQR)	14.3 (11.6, 18.8)	8 (4, 15)†	15.9 (7.8, 20.4)
Mechanical ventilation duration after palliative care consult, d
Mean (SD)	7.0 (6.2)	9.0 (10.4)	—
Median (IQR)	5.6 (3.0, 8)	7 (1, 12.5)	—
Tracheotomy, n (%)	4 (29)	4 (16)	3 (11)
CPR preference for full care
Preintervention, n (%)	13 (93)	25 (100)	—
Postintervention, n (%)	4 (29)	10 (40)	—
Hospital discharge disposition, n (%)
Home	1 (7)	2 (8)	8 (21)
Inpatient rehabilitation facility	0	0	4 ()
Skilled nursing facility	2 (14)	1 (8)	3 (11)
Long term acute care hospital	2 (14)	2 (16)	4 ()
Transfer to other acute care hospital	0	1 (4)	0
Hospice	5 (36)	5 (20)	2 (5)
Home hospice	2 (40)	0	0
Inpatient hospice	3 (60)	5 (100)	2 (100)
Died	4 (29)	14 (56)	18 (46)
Withdrawal of treatment	4 (100)	12 (42)	7 (39)

Definition of abbreviations: CPR = cardiopulmonary resuscitation; ICU = intensive care unit; IQR = interquartile range.

^*P < 0.05 for t test with intervention group.

^†P < 0.05 for rank-sum test with intervention group.

Intervention family members experienced a mean decrease of 12.7 (13.3) units in the NEST total unmet needs score in comparison with Control B family members, who actually had an increase in NEST score of 3.4 (15.0) units (P = 0.002; Table E4). The largest reductions in need in the intervention group were seen in domains of communication (7.5 [3.3]), social support (5.5 [3.2]), and patient symptoms (1.6 [3.5]), with smaller changes seen in psychological and spiritual needs (Figure 2). No notable changes were reported in domains associated with patient values, information, cultural concerns, decisions, trust, and finances. Intervention group improvements were also seen in mean Patient-Centeredness of Care Scale scores (difference, 6.6 [5.8]), Perceived Stress Scale scores (difference, −0.8 [1.9]), and the quality of communication (difference, 0.9 [1.6]). Compared with preintervention, family members more frequently reported that patients were receiving goal-concordant treatment (79% vs. 100%). The frequency of goal-concordant care did not change over time (80% vs. 80%) among Control B family members.

Figure 2.

The needs of social nature, existential concerns, symptoms, and therapeutic interaction (NEST) score changes between pre- and postintervention. The NEST total score is shown in bold; each NEST item is displayed beneath it. The vertical line represents 0. Bars extending to the right of the vertical line show a net reduction in needs between pre- and postintervention. Bars extending to the left represent an increase in need over time. Orange bars represent intervention family members (n = 18); gray bars represent family members of mechanical ventilated patients (control B group; n = 49).

The feasibility of the intervention was supported by excellent rates of completed daily PCplanner screens (90% by ICU nurse leaders during 84 days of screening), clinician approval (78%), and palliative care consult completion (93%; mean, 1.5 days [median, 1 day] after consent). Acceptability was excellent among family members and clinicians, based on high postintervention Client Satisfaction Questionnaire scores, whereas mean Systems Usability Scale scores reflected “good” usability (Table 4).

Table 4.

Acceptability and usability of PCplanner app system

Characteristic	Intervention family members (N = 18)	Clinicians (N = 10)
Client Satisfaction Questionnaire, mean (SD)*	14 (1.4)	12.4 (2.8)
Mean (SD)	14 (1.4)	12.4 (2.8)
Median (IQR)	14 (13, 15)	12 (11.3, 14.5)
Systems Usability Scale, mean (SD)†	21.1 (1.7)	15.2 (1.8)
Mean (SD)	21.1 (1.7)	15.2 (1.8)
Median (IQR)	22 (20, 22.5)	15.5 (14, 16)
PCplanner program quality was excellent/good, n (%)	18 (100)	9 (90)
PCplanner program met my all/most of my needs, n (%)	18 (100)	8 (80)
I would generally/definitely recommend PCplanner, n (%)	18 (100)	8 (80)
I am very/mostly satisfied with PCplanner, n (%)	18 (100)	8 (80)
I was able to, n (%)
Log in	16 (89)	9 (90)
Complete the program	15 (83)	9 (90)
I was satisfied/very satisfied with PCplanner, n (%)	15 (83)	8 (80)
I strongly agree/agree that I would like to use PCplanner if I had a loved one in the ICU, n (%)	17 (94)	8 (80)
I thought PCplanner was easy to use, neutral/agree, n (%)	18 (100)	6 (60)
What did you like most about PCplanner?‡	-It was very user friendly.	-Accessible
	-The bridge illustration was sooooo me.	-Easy to make necessary changes and comments
	-Ease of use and good description of the PCplanner.	-Helps focus conversation based on needs
	-It was simple.
	-It was very well thought out, you could understand oh so well, and the questions was very well thought out.
	-Easy to understand
	-Easy to find
	-Very easy to use
How could we improve PCplanner?	-Post information about it in patient rooms	-Logging into web app took too many steps

Definition of abbreviations: IQR = interquartile range; PCplanner = Palliative Care Planner.

^*Client Satisfaction Questionnaire: range 0 (worst) to 16 (best)

^†Systems Usability Scale: range 0 (worst) to 25 (best)

^‡Written feedback from users.

Discussion

This pilot evaluation of the PCplanner app platform prototype provides early support for its feasibility, acceptability, usability, and clinical impact. In comparison to historical ICU controls, PCplanner use more frequently reduced family members’ unmet needs, particularly communication and social support, and was also associated with improved patient-centeredness of care and patient receipt of goal-concordant treatment. In comparison with a palliative care control population, PCplanner-augmented ICU-based palliative care was delivered sooner and was associated with both a shorter subsequent hospital length of stay and a higher frequency of post-ICU hospice use. Importantly, our empirically validated e-triggers resulted in an intervention group that was primarily elderly, reflecting a population with a particularly great palliative care need (1).

Exploration of EHR-integrated, need-targeted interventions such as PCplanner is timely because these platforms can serve as the reliable framework for proactively and consistently delivering high-quality palliative care. It is estimated that perhaps half of ICU patients and their family members have a high burden of unmet palliative care needs (28, 37, 38). However, these needs are not being reliably and systematically met in the consultative model that dominates current practice (9). Intensivists order palliative care consultations with great variability and typically late in the course (39, 40), whereas the limited and geographically skewed palliative care workforce cannot fill the gap alone. And although the integrative care model that emphasizes palliative care concepts within routine ICU care is aspirational, the evolution of easily accessible educational programs will take time and expense. Yet fundamental change is needed, as indicated by a recent multicenter study of 303 ICU clinicians in which only 6% reported that they were satisfied with their current ICU-based palliative care model (27).

This change is likely to come through the continued evolution of the collaborative model, in which ICU clinicians deliver basic palliative care while palliative care specialists assist with more complex issues (41, 42). This pragmatic approach could accommodate structural barriers (workforce size, clinician skills) and efficiently use the skills of physicians, nurses, social workers, and clergy (11, 43, 44). However, a successful and replicable model must overcome both structural (e.g., staffing, technological resources) and process (e.g., patient/family identification, workflow integration, patient/family engagement) barriers to care delivery. PCplanner was specifically developed to address these barriers; in particular, by combining automated, data-driven screening with a patient-centered needs-targeted approach.

Health systems are increasingly activating palliative care specialist consultation by screening for clinical characteristics associated with poor outcome, called triggers. However, the unknown accuracy of triggers for actual need is concerning when applying them to large patient populations, given the limited number of palliative care specialists (26, 38). A novel aspect of PCplanner is its ability to automate EHR screening for patients with often difficult-to-recognize conditions such as declining health status and poor functioning, while also identifying the true positives with the use of family-reported needs. At the very least, given the enormous patient population with unmet need, PCplanner could help direct limited palliative care specialist bandwidth away from false-positives (i.e., patients with low need), thereby accommodating structural care delivery barriers (45). The app system could even work as a needs assessment tool in its present format without EHR integration, perhaps the purest and most unbiased form of patient-centered palliative care.

A second opportunity PCplanner provides clinicians is a way to move beyond assumptions of unmet need based on patient characteristics by rapidly gaining insight into family members’ self-report of need across all eight core domains of palliative care quality (30, 31). The intervention is the first to our knowledge both to employ a validated self-reported needs instrument in an ICU population and to demonstrate a reduction in unmet palliative care needs, as reported by family members themselves. These results appear to be explained in part by observed improvements in communication quality and symptoms management (9). However, family members and clinicians reported good, although not excellent, usability, based on mean Systems Usability Scale scores. To reach this target, future PCplanner iterations could build in prompts, hints, and advice for how to bring up needs and help guide discussions directed toward addressing them. More broadly speaking, specific needs could in fact activate care from a host of providers (e.g., social work, nursing, chaplain) and direct palliative care specialists to the most challenging cases.

Electronic health data systems are at a turning point, as the focus is now turning from meaningful use toward interoperability; that is, the capacity for programs and data sharing to work across different types of EHRs, devices, and settings (46). Although PCplanner’s EHR integration is not currently enabled for cross-EHR interoperability, future iterations can be quickly re-engineered, using increasingly accepted programmatic technologies (47). We expect that regulatory and economic pressures will pressure EHR vendors to facilitate direct integration of health apps into EHRs to allow clinicians to perform sensible tasks that currently are difficult within often unwieldly EHR user interfaces and data architectures. This would in turn promote scalability and enhance the pace of innovation.

A key strength of our approach is that it is one of the first to evaluate an automated, needs-targeted, ICU-based palliative care intervention. However, this study has numerous limitations. First, the relatively small sample size from a single center and the use of historical controls should prompt caution bout interpretations of the study’s statistical tests and the intervention’s effectiveness. Second, we evaluated an imperfect prototype that requires further programmatic refinement to enhance usability and more rigorous testing before introduction into clinical care. However, an automated version of PCplanner applied to a real-world clinical setting could greatly improve its future feasibility and scalability. Third, PCplanner was targeted to family members because patients often cannot voice their needs. Because many patients may lack an engaged family member to act as their surrogate, future efforts should focus on innovations directed toward empowering patient involvement in care (48). Fourth, the e-triggers tested capture a relatively narrow patient spectrum. Yet by expanding the number of e-triggers, the benefits to other populations are potentially sizable.

Conclusion

The PCplanner app platform demonstrated evidence of feasibility, acceptability, usability, and clinical effect among ICU patients and their family members. The likely mechanisms of action appear to be through accurate identification of those with unmet needs, engagement of family members in the palliative care process, and assisting clinicians in a coordinated effort to address patient and family member needs. A randomized clinical trial conducted in a larger, more diverse population is required to provide more compelling evidence of effectiveness.