Palliative Care Always: Hepatology—Virtual Primary Palliative Care Training for Hepatologists

Abstract

Palliative care (PC) benefits patients with serious illness including end‐stage liver disease (ESLD). As part of a cluster randomized trial, hepatologists were trained to deliver primary palliative care to patients with ESLD using an online course, Palliative Care Always: Hepatology (PCA:Hep). Here we present a multimethod formative evaluation (feasibility, knowledge acquisition, self‐efficacy, and practice patterns) of PCA:Hep. Feasibility was measured by completion of coursework and achieving a course grade of >80%. Knowledge acquisition was measured through assessments before and throughout the course. Pre/post‐course surveys were conducted to determine self‐efficacy and practice patterns. The hepatologists (n = 39) enrolled in a 12‐week online course and spent 1‐3 hours on the course weekly. The course was determined to be feasible as 97% successfully completed the course and 100% passed. The course was acceptable to participants; 91.7 % reported a positive course experience and satisfaction with knowledge gained (91.6%). The pre/post knowledge assessment showed an improvement of 6.0% (pre 85.9% to post 91.9%, 95% CI [2.8, 9.2], P = 0.001). Self‐efficacy increased significantly (P < 0.001) in psychological symptom management, hospice, and psychosocial support. A year after training, over 80% of the hepatologists reported integrating a variety of PC skills into routine patient care. Conclusion: PCA:Hep is feasible, acceptable, and improves learner knowledge and confidence in palliative care skills. This is a viable method to teach primary PC skills to specialists caring for patients with ESLD.

Abbreviations

COVID‐19

coronavirus disease 2019

ESAS

Edmonton Symptom Assessment Scale

ESLD

end‐stage liver disease

MOOC

massive open online course

PAL‐LIVER

Introducing Palliative Care within the Treatment of End‐Stage Liver Disease

PC

palliative care

PCA

Palliative Care Always

PCA:Hep

Palliative Care Always: Hepatology

PHQ‐9

Patient Health Questionnaire

RAB

research advisory board

End‐stage liver disease (ESLD) is one of the 10 leading causes of death in the United States^{( 1 )} and the seventh leading cause of death in persons aged 25‐64 years.^{( 2 )} ESLD is chronic, complex, progressive illness resulting from the decline of the structure and function of the liver due to cirrhosis or hepatocellular cancer. It is commonly associated with high symptom burden (both somatic and psychological), frequent hospitalizations, and pain similar to that of patients with colon and lung cancer.^{( 3} , ⁴ , ^{5 )} These factors lead to significant deterioration in patients’ quality of life with immense burden on caregivers.^{( 6 )}

Early integration of palliative care (PC) for patients with ESLD and their family caregivers is recommended,^{( 7} , ⁸ , ⁹ , ^{10 )} yet remains underutilized.^{( 11} , ¹² , ¹³ , ¹⁴ , ^{15 )} One of the reasons for this is a limited specialty PC workforce.^{( 16 )} This critical gap can potentially be addressed by developing primary PC skills in hepatology providers.

As part of a Patient‐Centered Outcomes Research Institute–funded clinical trial, Introducing Palliative Care Within the Treatment of End‐Stage Liver Disease (PAL‐LIVER) study,^{( 17 )} we developed an online course, Palliative Care Always: Hepatology (PCA:Hep), to teach primary PC skills to hepatologists. Participants were taught new PC including goals of care conversations, complex communication skills, and physiological and psychological symptom management. To provide practice and preparation for conversations with patients and caregivers, PC specialists facilitated virtual roleplay scenarios and regular refresher courses, allowing the hepatologists to interact as a group and provide each other feedback. We sought to improve self‐efficacy of PC skills by balancing web‐based learning with the interactive practice sessions, as this approach has been shown to be acceptable^{( 18 )} and aligns with preferred training practices of health care providers learning PC.^{( 18} , ^{19 )} Additionally, we explored whether the trained hepatologists would translate these skills outside the PAL‐LIVER trial, thereby providing an extra layer of support to all patients. The purpose of this paper is to describe the formative and summative evaluations of PCA:Hep.

Experimental Procedures

Parent Trial

The PAL‐LIVER study is a comparative effectiveness trial of two PC delivery models for patients with ESLD: PC provided by (1) PC specialists or (2) PC provided by hepatologists trained in primary PC skills (hepatologist‐led model). The details of the PAL‐LIVER trial design are described by Verma et al.^{( 17 )} Briefly, of the 18 participating clinical centers, 10 were randomized to the hepatologist‐led model. The hepatologists at the 10 sites were trained in PC and asked to incorporate the following as part of routine care with study patients: (1) Assess patient/caregiver understanding of diagnosis, illness, and prognosis; (2) assess and manage symptoms both physical and psychological, including psychosocial; and (3) discuss goals of care and advance directives. The components of the PC intervention were decided by a leadership panel of hepatology and PC specialists and a research advisory board (RAB) consisting of patients, caregivers, and patient advocates. This paper reports the development and implementation of the primary PC skills training provided to the clinicians in the hepatologist‐led arm of the trial.

Course Participants

Course participants included a mix of hepatologists and advanced practice providers. Overall, 39 hepatology providers were enrolled in the course (approximately three per site). Successful course completion was defined by viewing all modules over a 12‐week period and achieving a cumulative grade of 80% or higher on the combination of graded assessments (described in Table 1). Once this goal was achieved, the site was permitted to begin enrolling patients in the parent trial.

TABLE 1.

Exercises and Assessments

	Name	Description
Graded	Knowledge checks	Multiple‐choice questions based on the didactic videos and lessons; the number of questions asked range from 4 to 9
	Reading reflection	Short write‐up reflecting on an assigned reading
	Forum discussion	Short posts addressing module‐specific questions; participants are required to answer the question and reply to a colleague’s post
	Communication practice	Written scenario describing an appropriate dialog between patient and clinician to demonstrate communication skills taught in the course and proper symptom management
	Final exam	20‐question multiple‐choice assessment; the same questions are asked in the pretest
Not graded	Pretest	20‐question multiple‐choice assessment; the same questions are asked on the final exam
Optional exercises	Submit and compare	Free‐text question about managing the caregiver; once the participant submits their response, an answer written by the course team is displayed
	Virtual meeting	Weekly virtual meeting to discuss questions related to the course content
	Practice communication revision	Participants are given the opportunity to redo their communication after receiving feedback from the course team
	Patient scenarios	Participants can write up patient–clinician practice communications; for each communication scenario submitted, a knowledge check score is dropped

Course Development and Formative Evaluation

PCA:Hep is an online course for hepatologists teaching a PC skill set that includes physical and psychological symptom assessment and management, goals of care discussions, spiritual care, and caregiver support (learning goals described in Supporting Table S1). The course is hosted on Stanford University’s version of the edX platform and spans 12 weeks. PCA:Hep was adapted from an online course focused on oncology, Palliative Care Always.^{( 20} , ^{21 )} The original course was developed by a team including PC physicians, nurses, advanced care practitioners, chaplains, psychologists, social workers, and hospice physicians. The course content was reviewed by clinical faculty with expertise in PC. The ESLD‐specific content for PCA:Hep was developed by hepatologists and palliative care clinicians on the PAL‐LIVER leadership panel. Patients and caregivers from the PAL‐LIVER RAB contributed to case studies, reviewed PCA:Hep course content, and provided feedback. After a primary review by the PAL‐LIVER leadership panel and the RAB, PC and liver specialists outside of the trial reviewed the course to identify any content gaps and the feedback was incorporated.

Course Content

PCA:Hep has 11 modules covering primary competencies in PC (Supporting Table S1). Modules include didactic lectures, reading and reflections, patient and caregiver vignettes, interactive forums, and virtual discussions. Lectures include studio‐filmed faculty videos (e.g., https://youtu.be/ZE6j‐jufqtk). Readings and reflections focus on online news articles or videos. Modules highlight case studies to illustrate core palliative care principles for patients with liver disease. Short videos and written vignettes of provider, patient, and caregiver interactions follow patients from diagnosis to hospice. “Care for the Caregiver” provides caregiver’s thoughts with suggested guidelines for responding to their needs. The online forum and weekly virtual meetings provided opportunities for interactive discussions.

Course Implementation

The initial group of hepatologists moved through the course week‐by‐week in a moderated approach. When three new sites joined the study, the new cohort of hepatologists proceeded through the course asynchronously with the requirement to complete and pass the course within 12 weeks. Both groups had virtual office hours and instructor‐led discussions.

To facilitate course engagement, the hepatologists were required to reflect on the readings, participate on the online forum, and answer module‐specific questions each week. Comments on the “Care for the Caregiver” guidelines and participation in the weekly virtual meetings were encouraged but optional (Table 1). The hepatologists had access to the course director and faculty (via direct email and office hours) to ask questions on course content and practice their skills with role play scenarios.

Baseline Demographics

Participant demographics were collected by an anonymous precourse survey. The survey gathered data on participant demographics (age, sex, area, and years of clinical practice, and prior training in PC).

Summative Evaluation Measures

Feasibility

The feasibility of PCA:Hep was defined as completion of all assigned coursework within the allotted 12‐week interval with a passing grade of 80% or above. We recorded attendance at virtual discussions and completion of optional content. We explored additional metrics of feasibility and acceptability, including time spent in the course as well as learner feedback, through an anonymous postcourse survey.

Knowledge Acquisition

Learning was measured through questionnaire assessments and written exercises (described in Table 1). A 20‐question multiple‐choice, precourse assessment served as a baseline of PC knowledge. The participants did not receive correct answers for questions that were answered incorrectly on the pretest. The same questions were asked for the course final exam, taken 12 weeks later, to allow comparison between precourse and postcourse knowledge. Modules include multiple‐choice knowledge checks. Written exercises include reading reflections, online forum discussions, and a communication practice. Of those exercises, the communication practice was most comprehensive, involving the creation of a patient/clinician dialog. Participants were graded on their demonstration of communication skills and proper symptom management; written feedback was provided with resubmissions allowed.

Table 1 describes each exercise and assessments. The composite course score was based on the assessments labeled “Graded” (Table 1). A certificate was awarded with a minimum passing score of 80% to ensure uniformity of training.

The postcourse survey collected data on learning satisfaction, course experience, and self‐reported learning by topic.

Self‐Efficacy

The participants’ self‐efficacy of their PC skills was measured before and after the course using a 5‐point Likert scale (1 = strongly disagree, 5 = strongly agree) to a variety of “I feel comfortable…” statements (Table 5) in the precourse and postcourse surveys.^{( 22} , ^{23 )} Additional surveys are sent periodically to assess any changes in self‐efficacy throughout the trial.

TABLE 5.

PC Understanding and Practice Patterns

	Strongly Agree	Agree	Neither Agree nor Disagree	Disagree	Strongly Disagree
I have a strong understanding of palliative care and can explain what it is to my patient	2.8%	36.1%	47.2%	13.9%	0%
I regularly provide palliative care to my patients	5.6%	41.7%	16.7%	33.3%	2.8%
I regularly ask about physical symptoms with my patients in clinic	55.6%	36.1%	8.3%	0.0%	0.0%
I regularly address goals of care with my patients in clinic	22.2%	50.0%	19.4%	8.3%	0.0%
	Once a week	Once a month	Once in 6 months	Once a year	Less than once a year
I refer patients to palliative care approximately	2.8%	47.2%	27.8%	5.6%	16.7%
	On diagnosis	When they are no longer eligible for transplant	When they are no longer ambulatory	When they request it	When they are at the end of life
I refer my patients to palliative care at the following time points	2.8%	36.1%	16.7%	13.9%	30.6%

Data from precourse survey (n = 35‐36).

Practice Patterns

In the precourse survey, participants were asked about their pretraining practice of PC skills, such as the regularity of key PC conversations and frequency of referrals to social workers, psychology services, or chaplains. Every 6 months following course completion, the hepatologists report current practice patterns in a follow‐up survey.

Statistical Analyses

Paired Wilcoxon signed‐rank test was used to evaluate pre/post change in knowledge assessment. Two‐sample Wilcoxon test was used to analyze the significance of the pre/post change in self‐efficacy; here, P values do not account for correlation within a responder as in the pre/post responses because of the anonymous nature of the questionnaire, but P values are conservative because a positive correlation of pre/post measures is likely.

Surveys were created using Qualtrics^{( 24 )} survey software.

Stanford University’s institutional review board reviewed and approved the course surveys and knowledge assessments (protocol number IRB‐47235).

Results

Demographics

We had 39 participants who were part of the hepatologist‐trained arm in our trial. Approximately half of the hepatologists had a medical degree or osteopathic medicine degree, while the other half consisted of advanced practice providers. Participants ranged in age from 25‐70 years and approximately half were female. Most often their primary practice was hepatology, and more than half have been practicing for 0‐9 years. Eighty percent of hepatologists reported no prior formal training in PC (Table 2).

TABLE 2.

Course Participants

Age (years)	25‐40	46%
	40‐55	46%
	56‐70	8%
Gender	Female	54%
	Male	46%
Primary area of practice	Hepatology	83%
	GI	9%
	Primary care	3%
	Other*	6%
Clinical classification	M.D./D.O.	58%
	APP	42%
Years in practice	0‐9	63%
	10‐19	26%
	>20	11%
Prior training in PC	CME course	6%
	Online course	3%
	In‐person training	11%
	None	80%

Data from precourse survey (n = 35) and course enrollment statistics (n = 39).

Abbreviations: APP, advanced practice provider; CME, continuing medical education; D.O., doctor of osteopathic medicine; GI, gastrointestinal; and M.D., doctor of medicine.

^*Includes infectious diseases and oncology.

Feasibility

Ninety‐seven percent of the hepatologists successfully completed the coursework within the allotted period of time and scored a passing course grade (>80%). Ninety‐two percent used the optional content (i.e., completing nongraded exercises, attending virtual sessions). The hepatologists reported spending less than 5 hours a week on the course, with most spending between 1 and 3 hours.

Overall, the hepatologists had a positive course experience. Notably, 92% of hepatologists had a positive course experience; of those, 14% reported an extremely positive experience. Moreover, 92% were satisfied with their learning, and 19% were extremely satisfied.

Knowledge Acquisition

Knowledge Assessments

All but one hepatologist, who decided they were not able to participate in the PAL‐LIVER study due to clinical responsibilities, passed the course (Table 3). The hepatologists performed better on the final exam (91.9%) than the pretest (85.9%), with an average improvement of 6.0% (95% confidence interval [CI] 2.8, 9.2; P = 0.001). The average composite score (knowledge check, reflection, forum discussion, and practice communication scores) for the course was 87.5%.

TABLE 3.

Course Completion and Grades

Number of Successful Course Completions		38
		Mean	Range
Grades	Pretest	85.9%	70.0%‐95.0%
	Final exam	91.3%	75.0%‐100.0%
	Knowledge checks	82.1%	65.4%‐92.3%
	Composite course	87.3%	82.1%‐93.5%

Assessment grades are presented as the mean with bottom and top score range. Composite score includes all graded assessments described in Table 1. Course analytics, n = 37.

Self‐Reported Learning

One hundred percent of the hepatologists felt they increased their knowledge on goals of care/advanced directives (Fig. 1). Participants felt their knowledge was “greatly increased” in three topics: caregiver support (64.1%), psychosocial support (53.8%), and goals of care/advanced directives (53.8%). Only a small percentage of the hepatologists felt they did not learn anything new about communication skills (2.6%), psychosocial support (2.6%), physical symptom management (15.4%), psychological symptom management (5.1%), spirituality (2.6%), hospice (10.5%), survivorship (7.7%), or caregiver support (2.6%).

FIG. 1.

Self‐reported learning. Data from course completion survey, n = 39.

Self‐Efficacy

We asked the hepatologists to rate their self‐efficacy on key PC topics both before the course and at course completion. The hepatologists reported an increase in self‐efficacy for all PC topics (Table 4), with a statistically significant increase (P < 0.001) for conversations with patients about psychological health, hospice, and psychosocial needs. The hepatologists also rated their self‐efficacy at various intervals postcourse completion. Although the increases from baseline (precourse) were generally not statistically significant, there was a positive trend that self‐efficacy remains increased over time.

TABLE 4.

Self‐Efficacy Before Versus After the Course

		Mean	P Value (vs. Precourse)
I feel comfortable asking about physical symptoms	Before course	4.58	—
	Course completion	4.59	0.94
	6 months after course	4.67	0.61
	12 months after course	4.67	0.66
	24 months after course	4.71	0.48
I feel comfortable talking about psychological health	Before course	3.67	—
	Course completion	4.28	<0.001
	6 months after course	4.15	0.006
	12 months after course	4.28	0.004
	24 months after course	4.18	0.034
I feel comfortable addressing goals of care with my patients in clinic	Before course	4.17	—
	Course completion	4.44	0.18
	6 months after course	4.33	0.58
	12 months after course	4.44	0.25
	24 months after course	4.47	0.20
I feel comfortable talking about hospice and hospice benefits	Before course	3.75	—
	Course completion	4.49	<0.001
	6 months after course	4.30	0.007
	12 months after course	4.17	0.081
	24 months after course	4.35	0.011
I feel comfortable talking about the psycho‐social needs of my patient and family	Before course	3.64
	Course completion	4.26	<0.001
	6 months after course	3.89	0.17
	12 months after course	3.89	0.22
	24 months after course	4.00	0.12

Providers were asked to categorize their level of comfort with common PC discussions with patients before the training, at course completion, and at 6, 12, and 24 months after the training. Mean response as indicated on Likert scale: 1 = Strongly Disagree, 2 = Disagree, 3 = Neither Agree or Disagree, 4 = Agree, and 5 = Strongly Agree. Data from precourse survey, n = 36; course completion survey, n = 39; 6 months after course survey, n = 27; 12 months after course survey, n = 18; and 24 months after course survey, n = 17. P value < 0.05 is considered significant.

Practice Patterns

Precourse

In the precourse survey, 38.9% of the hepatologists reported they “Strongly Agree” or “Agree” that they have a strong understanding of PC and can explain it to their patients; 47.3% “Strongly Agree” or “Agree” that they provide PC to patients; and 72.2% “Strongly Agree” or “Agree” that they regularly address goals of care with their patients (Table 5). A total of 91.7% of hepatologists “Strongly Agree” or “Agree” that they ask about physical symptoms; however, 91.4% reported never using a symptoms screen such as the Edmonton Symptom Assessment Scale (ESAS) (Supporting Table S2). Screens for patient distress were also not frequently used, as 65.7% never used a screen like the Patient Health Questionnaire (PHQ‐9) or National Comprehensive Cancer Network's Distress Thermometer. A total of 82.9% of the hepatologists never assessed caregiver needs using the Brief Assessment Scale for Caregivers or a similar screen.

The hepatologists most commonly referred patients to a PC specialist when they are no longer eligible for transplant or are at the end of life. Almost half of the hepatologists referred a patient to PC at a frequency of once a month (Table 5).

Post‐Course

The hepatologists use the skills they learned in the course training with patients outside the PAL‐LIVER study (Table 6 and Supporting Table S3). One year after completing the course, 84.6% of the hepatologists reported they provide PC, and 88.5% address goals of care more frequently with patients than before the training. Approximately one third reported they use communication strategies to connect with my patients, screen for distress using the National Comprehensive Cancer Network’s Distress Thermometer or the PHQ‐9, and have a greater sense of meaning than before the training. Around a quarter of the hepatologists reported to use screens such as the ESAS, practice self‐care, and refer non‐study patients to PC more than they did before training.

TABLE 6.

Changes in Practice Patterns

	12 Months After Course
I use NURSE (Name, Understand, Respect, Support, Explore) and SPIKES (Setting, Perception, Invitation, Knowledge, Empathy, Summarize) communication strategies to connect with my patients more frequently than I did prior to the training	30.8%
I provide palliative care to my patients more frequently than I did prior to the training	84.6%
I refer my patients to palliative care more frequently than I did prior to the training	23.1%
I address goals of care more frequently than I did prior to the training	88.5%
I refer patients to hospice more frequently than I did prior to the training	19.2%
I screen for distress using the NCCN’s Distress Thermometer or the Patient Health Questionnaire (PHQ‐9) more frequently than I did prior to the training	38.5%
I use screens such as the Edmonton Symptom Assessment System (ESAS) to assess my patients’ symptoms more frequently than I did prior to the training	26.9%
I practice self‐care more frequently than I did prior to the training	26.9%
I have a greater sense of meaning	38.5%

Data represent the percentage of providers who incorporated course learnings into general practice, 12 months after course survey (n = 26).

Abbreviation: NCCN, National Comprehensive Cancer Network.

Discussion

PCA:Hep is a feasible method for training hepatology providers in core PC skills, as assessed by successful course completion and learner feedback. A total of 97% of our hepatologists passed the course with an average score of 87.3%. The hepatologists spent less than 5 hours a week on the course; most spent between 1 and 3 hours. The hepatologists reported improved self‐efficacy after completing the course in the areas of providing caregiver support, conducting goals of care discussions, and delivering psychosocial care; these are traditional gaps in specialist training.^{( 25 )} Additionally, many of the hepatologists incorporated their course learnings into their interactions with non‐study patients. We conclude that PCA:Hep was successful in training a cohort of hepatology providers with the primary PC skills required to deliver the intervention in the PAL‐LIVER study.

The online course environment enabled busy clinical practitioners to learn and practice core PC skills. Learners benefited from the flexible learning environment and demonstrated that they were able to complete the course (both the required and optional content). A qualitative interview study by Williams et al.^{( 19 )} revealed that (1) being actively engaged, (2) having the opportunity to interact and network, (3) finding meaning and relevance, and (4) exercising reciprocity are common training preferences for learning PC. For this reason, PCA:Hep, similar to other online trainings in communication and PC,^{( 26} , ²⁷ , ^{28 )} integrated roleplay scenarios in addition to didactic lectures and patient‐provider demonstrations. PCA:Hep included weekly synchronous virtual sessions in addition to instructor availability via phone or email over the entire period of the training, to provide active learning opportunities for the hepatologists. This interactive, but fully virtual approach, which predated the pandemic, came to be of even greater value over the last year. Furthermore, virtual roleplay scenarios helped providers become more comfortable with providing PC care remotely—a key adaptation of the trial due to coronavirus disease 2019 (COVID‐19).

The decrease in self‐efficacy in the PC skills 6, 12, and 24 months after course completion is concerning but not unexpected. The study by Pelayo‐Alvarez et al.^{( 29 )} measuring the effectiveness of their PC online training for primary care physicians^{( 27 )} found that participant confidence levels were not significantly increased after 18 months, whereas confidence levels did increase following online training completion. Taken together, this may imply that regular refresher courses that consist of practice‐based exercises and simulations may be necessary to ensure provider confidence. Additionally, the COVID‐19 pandemic impacted recruitment, leading to the inability to see PAL‐LIVER patients on a regular basis, and thereby decreasing opportunities to practice PC skills and negatively affecting self‐efficacy. At trial maturity, we will be able to ascertain whether PC training improves patient‐level outcomes such as quality‐of‐life and end‐of‐life use.

The hepatologists not only learned PC skills but also implemented them into their practice beyond the PAL‐LIVER study. A year after the training, almost half of the providers increased their use of communication skills and did routine distress screening. Additionally, most of the providers incorporated PC into their practice and goals of care discussions with non‐study patients. This is likely due to the reported increased self‐efficacy in conducting goals of care discussions and managing distress. Interestingly, these practice patterns changed when the same group of providers was resurveyed a year later. This may be due to the extreme demand on health care providers due to COVID‐19 and the challenges of remote health care.

Current training guidelines do not recommend PC training for hepatologists. Concordant with this, 80% of our learners had no prior PC training. None of our hepatologists have had formal training in PC (>6 months), which aligns with a national survey of hepatologists and gastroenterologists.^{( 30 )}

Patients with ESLD have complex PC needs that include symptom management, care coordination, and support at end of life. Integration of PC into these patients’ care continuum has the potential to improve quality of life and survival, as has been shown in other trials in oncology.^{( 31} , ³² , ³³ , ^{34 )} Given the fact that hepatology providers are well versed on prognosis and the trajectory of ESLD, they are often the primary care providers for these patients and could potentially be the personnel who can deliver PC to this high‐need population. Hepatologist‐provided PC may be even more important under the pressure of COVID‐19, as patients may not be willing to see an additional provider to get specialized PC. Additionally, the impact of sheltering in place and minimal in‐person contact can lead to a variety of emotional responses. Hepatologists who are trained in distress screening are vital, especially with the documented increase in alcohol consumption during the current pandemic.^{( 35 )}

Study Limitations

One limitation in our evaluation of knowledge acquisition is that the pretest and the final exam included the same questions. However, the final exam was done 12 weeks after the pretest, minimizing the chance for the participants to remember their prior answers. As a result, we believe it is reasonable to conclude that the final exam results reflect knowledge acquisition from the course experience.

We also recognize that participant survey data may be biased by multiple factors including courtesy bias, demand characteristics, and social desirability bias. However, we made every effort to ensure the hepatologists knew that the surveys were anonymous, with the goal of decreasing the potential for bias. Additional data on clinician performance are being assessed quantitatively and qualitatively as part of the parent study and will be reported after the study conclusion.

Due to the small number of participants, we are unable to make further conclusions on which learning activities may have had the greatest impact on acquisition of knowledge. We also are unable to fully assess the utility of some of our optional learning opportunities, as they were not used consistently (such as the low attendance at our virtual meetings). The virtual format of the course may have decreased the effectiveness of the education, as there were no in‐person interactions, which are helpful for practicing communication skills.

The clinical trial itself may be a limitation in determining the feasibility and effectiveness of PCA:Hep. This was a self‐selected cohort of clinicians who recognize the importance of PC and are more likely to participate in a clinical trial on the subject. Passing the training course is a requirement of the trial; therefore, we do not know whether the average clinician would be as motivated to complete the entire course. Additionally, clinicians who recognize the importance of PC likely already understand the basics of the specialty. This could explain why the hepatologists performed well on the precourse assessment, even though 80% had no prior PC training. Finally, as the primary focus of the PAL‐LIVER trial is to compare patient quality of life between patients of PC specialists compared with patients of PC‐trained hepatologists, we do not have a control group of untrained hepatologists for additional evaluation of training effectiveness.

Future Directions

We continue to evaluate the efficacy of PCA:Hep through postcourse surveys, as well as within the larger trial, where we will assess quality of life of our participants who receive PC from their hepatologists. Postcourse surveys are sent out at 6‐month intervals and focus on the self‐efficacy of the hepatologists’ PC skills and changes in practice patterns. We will also ensure continuing educational competency through semi‐annual refresher courses.

in conclusion, the PCA:Hep course is feasible for busy clinicians. The course improves provider confidence in several PC skills, including difficult conversations with patients and caregivers, and can lead to changes in practice patterns at least 24 months following course completion.

Supporting information

Table S1‐S3