Post-doctoral research training in palliative care: lessons learned from a T32 program

Abstract

Our aging population and advances in chronic disease management that prolong the time that patients live with a chronic illness have combined to create an enormous need for improved palliative care research across diverse diseases. In this article, we describe the structure and processes of an NIH-funded T32 post-doctoral research fellowship at the University of Washington and our experiences in developing and implementing the program. We recognize a broad definition of palliative care research, including research focused on improving quality of life, minimizing symptoms, providing psychological and spiritual support, and improving communication about patients’ values and goals of care, all in the context of a serious illness. We describe our four core principles for post-doctoral training in palliative care research, each with a number of specific approaches: 1) mastering a set of essential content and research skills; 2) structured mentoring and academic career development; 3) creating and supporting early success; and 4) interdisciplinary training and team science. In addition, we also describe our framework for the essential competencies necessary for a palliative care research training program, our methods for identification and selection of applicants, our outcomes to date and our processes of continuous quality assessment and improvement. Our goal is to describe our successful post-doctoral research training program in palliative care in order to promote development of new programs and share information between programs to continue to build the field of collaborative and interdisciplinary palliative care research.

Introduction

Attention to the field of palliative care has dramatically increased over the past 20 years. This increase is particularly notable in the development of palliative care clinical programs. In 2007, over 50% of U.S. hospitals had a palliative care program¹ and by 2019, 72% of U.S. hospitals with fifty or more beds reported a palliative care team, serving 87% of all hospitalized patients in the U.S.^2,3 Palliative care is rapidly becoming a routine part of healthcare, spurring the development of evidence-based research on the effectiveness of palliative care interventions.^4–11 This growing focus on palliative care has similarly extended to schools training clinicians, including physicians and nurses.^12,13 Despite this increased attention to the importance of palliative care in healthcare, important shortcomings remain in access to and delivery of effective palliative care for patients with serious illness and their families.^6–11 The science of palliative care is relatively new and evolving. Although federally funded research in this important field has increased over the past decade, fewer than 1% of all grants awarded by the National Institutes of Health were awarded to palliative care researchers between 2011 and 2015.¹⁴ Funding for these researchers has failed to keep pace with the need to expand the efficient implementation of effective, evidence-based palliative care to patients and their families.^14,15 Additionally, unlikely many other subspecialty medicine fellowship training programs for physicians, palliative medicine fellowship training generally includes one year of clinical training and no dedicated time for research training. For these reasons, it is important to foster the development and continued success of palliative care research training programs. We provide this article as an example of one such program’s approach in order to promote development of new programs and share information between programs.

We recognize a broad definition of palliative care research, including research focused on improving quality of life, minimizing symptoms, providing psychological and spiritual support, and improving communication about patients’ values and goals of care, all in the context of a serious illness. We also recognize that a training program built upon such a broad definition of palliative care research risks being diffuse. Therefore, it is important for a training program to identify and focus on specific competencies in palliative care research. Examples of palliative care research competencies include clinical epidemiology, health services research and biostatistics; social and behavioral sciences; health disparities research; healthcare communication; biomedical and health informatics; qualitative methods; health economics; bioethics and research ethics; and implementation and dissemination science.

There are four key reasons for increasing and improving post-doctoral training programs focused on palliative care research. First, although the science of palliative care is rapidly developing, research training opportunities lag behind, leaving important gaps for the science and the research workforce of the future.^14,15 Second, the conduct of palliative care research faces distinctive challenges that result from studying vulnerable populations of seriously ill patients and their families. Approaches that appropriately address these challenges require specialized training. For example, measurement and outcome assessment, as well as recruitment and retention of seriously ill participants, pose unique challenges for effective research; training in ways to address these challenges advances the science of palliative care. Third, existing palliative care research across diverse diseases is limited and in need of ongoing development.^14–17 Palliative care research often involves care of patients with multiple comorbidities and complex social needs that may not fit disease-focused and biology-focused research training programs. Finally, palliative care is an inherently interdisciplinary field.¹⁸ We believe it is valuable for a post-doctoral training program to be inclusive of diverse disciplines including Medicine, Nursing, Psychology, Pharmacy, Social Work, and Public Health. As such, we designed a training program that offers a variety of research opportunities necessary to meet the broad interests and needs of aspiring palliative care scientists. We also believe that interdisciplinary co-training and co-location of MDs, RN/PhDs, and other PhDs allows important cross-pollination of ideas and experience with team science, with the clinician-scientist providing clinical perspectives and the PhD post-doctoral trainee bringing cutting-edge research methodologies and interdisciplinary perspectives.

Four core principles for post-doctoral training in palliative care research

We have implemented our research training program based on four core principles to prepare researchers for independent careers: 1) Trainees must master a defined set of essential content and research skills necessary to become independent yet collaborative investigators; 2) Structured research mentoring and academic career development are critical to retaining trainees in science careers; 3) Research projects must be tailored to each trainee and designed to be training vehicles that facilitate early academic productivity and success while also providing a framework to support future grants; and 4) Training in palliative care research should provide real opportunities for interdisciplinary interactions and promote team-science approaches to address this interdisciplinary field of research. These core principles were based, in part, on a successful T32 program in pulmonary and critical care, and were developed by the program directors in consultation with the program faculty.

These four core principles form the foundation of our structured approach to training post-doctoral clinicians and scientists. These principles also provide direction during the conduct and continuous quality improvement of the program. The core principles, along with their implementation and evaluation metrics are outlined in Table 1. We have defined several approaches to each principle, which aid in guiding changes and improvements to the program.

Table 1.

Core principles of the T32 training program, their implementation, and metrics for evaluation

Core Principle	Implementation	Formal Evaluation Metrics
1. Defined set of research and writing skills	• Develop and implement with mentor(s) an explicit training plan to learn research skills • Attain advanced degree in Nursing, Social Work, Public Health, or Medical Ethics, when appropriate. • Attend NIH Summer Courses when appropriate • Attend grant & manuscript writing workshops • Receive mentored review of grants and manuscripts • Participate in research works in progress	• Number of trainee publications & abstracts • Number of trainee grants submitted & funded • Number of trainees obtaining advanced degrees • Review of IDP at mentor committee meetings • Presentations and participation in monthly research works-in-progress sessions • Annual evaluations of works-in-progress • Attendance at research & writing workshops
2. Structured team mentoring	• Primary and secondary mentor-trainee partnerships • Mentoring committees meet twice yearly and update mentoring contracts and career plans • Structured mentoring opportunities • Seminars on mentoring and career development • Coaching the mentor to improve mentoring	• Mentoring feedback from trainees • Mentoring feedback from faculty colleagues • Completion of mentor coaching sessions • Faculty feedback on coaching • Review of mentoring plan at each meeting • Attendance at mentoring workshops
3. Research projects tailored to trainee and to early productivity	• Initial project selection targeting early success • Initial projects reviewed by Steering Committee • Trainee and mentor encouraged to submit abstract to scientific meeting early in training (Year 1 and 2) • Generate preliminary data for grant applications	• Number of publications and abstracts in each year • Formal review of trainee progress at each mentoring committee meeting • Submission of mentored grants during T32 • Presentations at national meetings
4. Interdisciplinary team science interactions	• Active participation in team science projects • Encourage cross-discipline collaborations	• Number of publications with authors from different disciplines • Involvement of scientific mentors from separate disciplines

Core Principle 1: Mastering a Set of Essential Content and Research Skills

With guidance from their primary mentor and mentoring committee, trainees master a specific set of research skills. The methods to identify and implement this training vary depending on the area of research. This core principle is accomplished through four key approaches.

Approach 1 - Skills Training in Healthcare Research Methods

Palliative care researchers need a background in biostatistics, epidemiology, and health services research. We offer a fundamental curriculum that includes a Master of Science degree in Epidemiology or Health Services for trainees with limited prior experience in these disciplines. In some cases, trainees enter with a PhD or significant prior research training; these fellows benefit from an individualized approach to achieve the most productive research training. Trainees in either category can choose additional training in specific disciplines like health equity, implementation science, social and behavioral sciences, informatics, bioethics, advanced biostatistics, or qualitative research, based on their research interests, projects, and training plan.

Most of physician-trainees without a PhD pursue a Master’s degree in Epidemiology or Health Services because these degree programs provide systematic and thorough training in these disciplines. The curriculum includes courses in biostatistics that describe the foundations of statistical inference and the types of statistical tests. Because a significant proportion of clinical research is observational (particularly research done by trainees) and must address issues of bias and confounding, trainees also need an in-depth understanding of multivariate modeling including linear regression, logistic regression, and survival analyses. Many of our trainees also work with repeated measures and clustered datasets requiring additional skills and training. In addition, trainees should have an understanding of the complexities of survey design, the role of patient- and family-reported outcomes, and the use of qualitative methods in palliative care research, including the rigorous conduct and analysis of focus groups and one-on-one interviews, ethnography, discourse analysis, and grounded theory.

Approach 2 – Research Works-in-Progress Sessions

A successful training environment requires an intellectual environment formed by a critical mass of excellent clinical investigators, usually both faculty and trainees, to provide a network of colleagues. This critical mass facilitates conduct of a seminar in which trainees and faculty present their Palliative Care Research Works-In-Progress (PC-WIP). The main purpose of the PC-WIP is for the presenter to receive feedback on the specific aims, project design, analytic plan, or the interpretation and presentation of results of a project. Therefore, these conferences are relatively small (generally 8 to 15 people) and provide a supportive atmosphere for the presentation of works-in-progress. To enhance discussion and interaction, the group sits around a table and presenters are asked not to use slides unless the goal of the session is to practice a formal research talk. In addition, this conference is an opportunity for trainees to learn to critique the work of their colleagues in a constructive way through modeling by faculty who attend and present.

Approach 3 – Teaching grant and paper writing

Another necessary skill set includes scientific writing and grant writing. All trainees are introduced to these topics in an annual “Orientation to the Research Years” course offered by the University of Washington (UW) Department of Medicine as well as scientific writing workshops offered by the Clinical and Translational Science Award (CTSA) at UW. As trainees begin to write grants, they submit specific aims and grant drafts to members of their mentoring committee for feedback. We also conduct mock study sections for all trainees submitting grants, such as individual post-doctoral NIH Ruth L. Kirschstein National Research Service Awards (NRSA), K awards, and K99-R00 awards. In addition, this skill set includes presentations through posters or talks. Trainees are expected to present their research at national meetings, but first preview their talk or poster at a PC-WIP session where they receive feedback on the presentation.

Approach 4 – Essential content in palliative care

Because trainees vary in prior exposure to palliative care (i.e., palliative medicine fellows who have completed clinical training, as well as cardiology or other sub-specialty fellows or PhD post-doctoral trainees without comprehensive training in palliative care), we invite trainees needing exposure to palliative care clinical content to attend a weekly one-hour lecture series conducted by the UW Palliative Medicine Fellowship that covers core content in palliative care. The goal of attending this lecture series is not to provide clinical training in palliative care, but rather to have exposure to the principles of palliative care in order to enhance the quality of palliative care research.

Core Principle 2: Structured Mentoring and Academic Career Development

Mentoring is one of the most important determinants of whether a trainee develops into a successful scientist. The Association of Professors of Medicine¹⁹ called out the need for team-based mentoring by groups of mentors who have complementary skills and insights into various aspects of a scientist career. Andrew Schafer’s commentary on the ‘vanishing clinician-scientist’ stated “the role of effective mentoring will become more important than ever in promoting successful physician-scientist careers in the future. But, just as the ‘lone ranger’ model of individual investigators is giving way to biomedical research as a team sport, contemporary models of mentoring for each individual developing physician-scientist will often require cohesive teams of trans-generational, dedicated mentors who can provide committed time to this vital function.”²⁰ We embrace these recommendations and believe it is important to create a strong mentoring environment through: 1) support of the primary mentor-mentee relationships, 2) team-based mentoring, 3) structured mentoring opportunities, and 4) education and evaluation of mentors. We focus on creating outstanding mentors of the future through two programs: a “mentor-in-training” program and a “coach-the-mentor” program.

We believe that a team-based approach to mentoring has many advantages. To facilitate excellent and consistent mentoring of trainees, our program emphasizes 5 specific approaches to mentoring.

Approach 1 – Utilize primary and secondary mentors

In our experience, the pairing of primary and secondary mentors can be of great benefit to both the trainee and the mentors. The primary mentor is responsible for direct supervision, training, and support of the trainee. The secondary mentor provides perspective and connections outside of the primary mentor’s network, acting as a second sounding board for research ideas and career development. Secondary mentors often provide a complementary interdisciplinary scientific approach or clinical area of expertise. We often pair junior and senior faculty together so the junior faculty member can learn mentoring skills from the senior faculty member as a “mentor-in-training”.

Approach 2 – Create and implement structured independent development plans

After selecting a mentor, each primary mentor and trainee work through an independent development plan (IDP) together, ensuring they address the important elements of their mentoring relationship. This becomes the IDP required for T32 programs. The IDP is not intended to rigidly prescribe how a mentor should train their trainee, but rather to facilitate the conversation and make expectations explicit. The IDP is signed by both the primary mentor and trainee, and subsequently reviewed and updated at each mentoring committee meeting. The IDP is also reviewed and approved by the T32 Directors after each mentoring committee meeting. As trainees progress and their careers develop, the IDP is revised, making it a living document. The IDP also includes issues of job placement. The elements of the IDP are listed in Table 2 and the template we use is included as Online Supplement A.

Table 2.

Elements of Independent Development Plan for all research trainees

Element	Description
Trainee’s long- and short-term career goals	• Long term goals: general, focus on the type of career the trainee would like to pursue • Short term goals: specific and measurable; focus on the specific research training experiences as well as a timeline for the completion of each goal including abstract, manuscript, and grant submissions
Plan for trainee’s development and acquisition of discipline-specific knowledge and skills	• Brief description of mentee’s planned research project(s) and dissemination activities, i.e., selection of meetings and journals for abstracts/manuscripts specific to each project • Identification of specific research skills needed to complete research projects • Structured plan on how these skills are to be acquired: may include individual coursework or obtaining a Masters or PhD
Communicate and disseminate science	• Identification of presentation opportunities (local and national venues) • Specific plans to learn to give and receive constructive criticism and to work in teams
Plans for trainee’s career development in professionalism and leadership	• Plans for how the mentor will provide instruction on a regular basis in professional practices such as: 1) ethical conduct of science, 2) protection of human subjects and interactions with Institutional Review Boards, and 3) standards of professionalism • Plans to develop the mentee’s leadership skills, and how the mentor will aid in the mentees’ development of these skills - this might include: 1) supervised mentorship by mentee of residents and students in research projects; 2) teaching mentees to effectively manage research staff through participation in team meetings; 3) mentor’s promotion of mentee’s involvement in professional societies
Resources provided to support trainee	• Resources include: 1) office space, 2) computer, 3) access to research staff support, 4) access to statistical or database support, and 5) support for travel to present work at meetings
Schedule for ongoing mentoring meetings	• Confirms that committee members have the mentoring meetings on their calendars and describes the frequency of the meetings and the methods to be used to schedule the meetings
Formal review of CV and discussion of job placement	• Review of the trainee’s CV to refine it for the best possible presentation • Discuss job placement opportunities • Discuss and support job interview process
Formal evaluation of the trainee	• Mutually develop an assessment of the trainee’s progress in meeting his or her training goals • Ensure the trainee receives feedback when presenting to works-in-progress sessions, and research conferences as well as in individual meetings with the mentor • Progress toward meeting training goals is reviewed at each mentoring committee meeting
Formal evaluation of the mentor	• Mentee fills out a confidential evaluation of mentors on an annual basis • Data from multiple mentees to be reviewed by program directors, and general principles will be shared with all mentors at regular intervals to provide constructive feedback while protecting trainee confidentiality

Approach 3 - Develop and support mentoring committees

Once trainees select primary and secondary mentors, the mentors and trainee—with input from the T32 directors–will identify 2 to 4 additional members of their mentoring committee. In addition to these mentors, the mentoring committee always includes at least one of the T32 Directors. The charge of the mentoring committee is to oversee the trainee’s professional development and review all of the elements of the IDP. In addition, committees facilitate academic job placement.

Approach 4 - Support peer mentoring

We have found that peer mentoring can be very effective, not only for those in the first year of training, but also for those in the second year and beyond to develop their own mentoring skills. We have created a monthly career development seminar as one opportunity for trainees to meet together to discuss their projects and progress, to identify opportunities to work together, and to share experiences.

Approach 5 - Coaching mentors and mentors-in-training

The centerpiece of this effort is individualized feedback to mentors by select members or “coaches” from our senior training faculty. We have chosen this approach because of the challenges for mentees to provide effective “feed-up” to their mentors given the power imbalance that can be present in a mentor-mentee relationship. We solicit feedback from a number of different sources and collate this information to provide suggestions to individual faculty members. Feedback is solicited through an electronic survey system. These surveys are used to gather 360° feedback from trainees and faculty (See Online Supplement B). We conduct a survey every other year for each primary mentor. The coaches are faculty who are respected by others for their ability to provide honest but supportive feedback.

Core Principle 3: Creating and Supporting Early Success

One of the key features of a productive research training program is to help trainees identify early projects that are high yield and that are likely to be successful in a relatively short time frame. Early success not only provides positive reinforcement for trainees, but facilitates early publications and generates pilot data for grant applications. Proper selection of trainees’ early research projects is a challenging component of any training program. We have developed a series of criteria for trainees’ initial projects as well as a review process to ensure these projects achieve our training program’s goals. This core principle is implemented with three specific approaches.

Approach 1 - Explicit goals for initial trainee projects

We have developed specific goals for each trainee’s initial project. First, the project must be feasible for the trainee given the resources available and the mentor must be willing to commit the necessary resources. Second, the skills needed to complete the project must be within the reach of the trainee, with an explicit plan for the trainee to obtain those skills. Third, the mentor and trainee must have an explicit discussion regarding early abstract submission and early publication from each project. Ideally, at least one project for each trainee should be relatively safe for early abstract submission and publication. Finally, the project must be innovative and of interest to the trainee. To make sure the project meets these goals, the program leadership reviews the initial project for every trainee. Within the first 2 months of the research fellowship, all trainees are required to submit a brief project proposal (1–2 pages) with the following format: 1) background and significance (bullet points limited to a half page); 2) research questions or aims; 3) approach and analysis; and 4) proposed timeline for completion. We ask that trainees develop at least two projects for several reasons: 1) to provide some security in case the primary project is delayed; 2) to increase early productivity; and 3) to provide ongoing productivity even when one project is temporarily on hold, such as when a mentor or statistician is reviewing it. Each proposal is reviewed by a subcommittee including the Program Directors and selected faculty. The goal of this review is not to accept or reject these proposals, but rather to provide direction and feedback to the trainee and their mentors on the selection of initial projects.

For their first two research projects, we encourage trainees to develop projects utilizing already available research databases. Most mentors have access to databases that can support trainee’s first research project. The early projects should help the trainee learn biostatistics and epidemiology course material in the context of their project, and allow the trainee to complete the first research year with a finished project. For those obtaining a Master’s degree, one of these early projects should result in a manuscript that can be used for their Master’s thesis and can also be submitted for peer-reviewed publication.

Approach 3 - Early transition planning to mentored awards and independence

One of the most important markers of success of a T32 award is transitioning trainees to individual post-doctoral NRSA awards, K-level mentored awards, and ultimately independent funding. We are intentional in our plans to structure an environment that succeeds at supporting trainees in this area. We include seminars about NRSA, K awards, and other mentored funding during the first research training year, devote a series of WIP and career development seminar sessions to grant writing including the development and review of specific aims, and work with trainees to develop grant submissions late in the first or early in the second research training year. Other activities include a mock study section conducted 6–8 weeks prior to a trainee’s grant submission and access to a grant writing workshop held quarterly. Transition to independence is a later phenomenon, but is a topic for discussion at the mentoring committee meetings in order to begin to plan ahead for independent funding and map out timelines.

Core Principle 4: Interdisciplinary Training and Team Science

Interdisciplinary research training is an important focus for palliative care research training because of the inherently interdisciplinary nature of palliative care research that often encompasses physical, psychological, emotional, and spiritual domains. Growing evidence suggests that true innovation in research derives from interdisciplinary teams that combine existing knowledge and scientific approaches to address major problems and overcome existing roadblocks.²¹ Single-discipline teams are more likely to use “sequential thinking” to produce incremental advances that are limited in scope and innovation. Interdisciplinary teams are more likely to use “connective thinking” leading to major advances in understanding health and illness. Our program facilitates interdisciplinary collaboration through two approaches.

Approach 1: Capitalize on the breadth of current interdisciplinary research

Our ability to form productive interdisciplinary mentoring teams is enhanced when diverse scientists are using a variety of scientific methods. Our program benefits from training leaders and faculty across diverse clinical and research disciplines.

Approach 2 - Develop collaborative interactions between trainees

Faculty promote interdisciplinary team science by encouraging trainees to include not only training faculty but also other T32 trainees on their projects. In a training program with interdisciplinary trainees, this approach offers several advantages: a) it brings diverse skills to bear on the trainees’ projects; b) it increases the productivity of trainees asked to collaborate on peer trainee projects; and c) it engenders a culture of collaboration and interdisciplinary team research.

Identifying the Core Competencies for a Palliative Care Research Training Program

Although we define palliative care research broadly, we focus on specific competencies that are particularly important for palliative care research. Not all trainees will need to become experts in each of these, but each trainee needs an understanding of these competencies and should pick at least one for more extended and focused study. These competencies are taught in research seminars, mentoring meetings, through individual trainee research projects, and through formal coursework.

Clinical epidemiology, health services research, and biostatistics

These skills form the basics of clinical and healthcare research and provide a foundation for high quality palliative care research. Because of the foundational nature of these disciplines for palliative care research, all trainees need skills in these areas.

Implementation and dissemination sciences

Although the pace of discovery from basic and biobehavioral research has led to fundamental changes in our understanding of disease and approaches to treatment, its application to benefit diverse patient populations has failed to keep pace and its public health impact has been limited.^22,23 Dissemination and implementation science, the study of methods that influence the integration of evidence-based interventions into practice settings, has grown, in part, in response to this challenge. Its systematic approach to defining, addressing, and working within the complexities of healthcare in order to support the widespread adoption of evidence-based interventions that have been shown to benefit patients is particularly important in palliative care. Despite a growing evidence base for the efficacy of palliative care interventions, progress in quality measurement has been slow and implementation and scalability of high-quality palliative care interventions have been lacking. It is important to meet this challenge by training a new generation of researchers focused on the science of implementation and dissemination applied to palliative care research.

Social and behavioral sciences

Social and behavioral sciences are critical if we are to change patient, clinician, and public behaviors, reduce the complications of disease, and understand and modify the social determinants of health. The social and behavioral sciences are especially relevant for palliative care research with its focus on the social and psychological aspects of illness and patient- and family-centered outcomes.

Health disparities research

Understanding and reversing health disparities is an important priority. Health disparities are well-documented in palliative care, yet poorly understood. Importantly, preferences for care (particularly care received at the end of life) vary by race, ethnicity, and culture, but not all these differences represent disparities.²⁴ An important challenge in palliative care research is to develop an understanding of which differences represent the informed values and preferences of patients and families, and which represent disparities in how care is discussed or delivered. Differences based on values and preferences of informed patients and their families must be supported; differences based on disparities in health, communication, or healthcare must be eliminated. These complicated differences create an imperative in palliative care research for which we need researchers to be trained to address this challenge.

Healthcare communication and decision-making

Communication and decision-making are central components of palliative care and areas in need of innovative research and research methods.

Biomedical and Health Informatics

There are a number of key opportunities within palliative care research to incorporate health informatics, a field which both develops innovative applications of health information to improve health and healthcare and evaluates the impact of those interventions.

Qualitative methods

Qualitative methods provide an opportunity to comprehensively examine patients’ and families’ experiences with serious illness and healthcare; these methods also allow researchers to develop a richer understanding of factors that determine the effectiveness of healthcare interventions.

Health economics

Health economics is a broad discipline that not only assesses costs and cost-effectiveness, but also provides an approach for identifying value in healthcare. Health economic research is driving important innovations to ensure that quality care is delivered efficiently. The economic effect of palliative care has generated a considerable amount of interest because of its potential to simultaneously increase quality while reducing costs by ensuring patients receive care that is consistent with their informed preferences.

Identification and Selection of Applicants

Defining Post-Doctoral Fellows

Our program is funded by the NIH and therefore we use their definition of post-doctoral to consider eligible applicants. Post-doctoral fellows are individuals with a doctoral degree (PhD, MD, DDS, or the equivalent) who are engaged in a temporary period of mentored research in order to acquire the skills needed to pursue their career.

Post-doctoral Palliative Care Scientists

Our program focuses on training post-doctoral scientists focused on palliative care research, including clinician scientists (physicians, nurses, psychologists, pharmacists, etc.) as well as non-clinician scientists (epidemiologists, ethicists, health economists, bio-informaticists, etc.). We seek to identify and support such aspiring scientists as early as possible in their post-graduate training. In particular, we are committed to ensure that women, racial/ethnic minorities, persons with disabilities, and other individuals who have been traditionally under-represented in science are well represented in our program.

Selection criteria

The goal of our T32 is to train future independent investigators in palliative care research. Therefore, we use rigorous selection criteria to identify candidates who are likely to be successful in a career as an independent investigator. These selection criteria include evidence-based predictors of academic success^25–27 (prior research experience, publications, and grant funding) and also include the selection committee and references’ assessment of the applicant’s talent, drive, and passion for a research career. Applicants are asked to submit a formal application which includes the following components: cover letter, curriculum vitae, 3-page research proposal, mentor’s support letter and NIH-biosketch, and two additional reference letters. Selected applicants are invited to interview with members of the T32 selection committee, which is comprised of the program directors and other members of the training faculty. The program directors and training faculty confer to determine which applicants will be offered positions.

Forces creating post-doctoral research fellowship applicants and jobs for palliative care research trainees

Post-doctoral palliative care research training is important because the rise to prominence of palliative care and palliative care research has resulted in a large number of excellent applicants with an interest in palliative care research at a time when there is a focus on increasing funding for palliative care research within the NIH and Patient-Centered Outcomes Research Institute (PCORI).^14,15 In addition, with our aging population, rising multimorbidity, and rising costs of healthcare, there is an increasing realization of the importance of incorporating palliative care into the care of patients with chronic life-limiting illness to simultaneously improve quality and reduce the intensity and costs of care at the end of life.^4–11 Palliative care research trainees will also have access to research and quality improvement positions in healthcare systems and accountable care organizations interested in maximizing quality and value of healthcare for patients with serious illness.

National organizations

Like many of the post-doctoral training programs that focus on palliative care research, our program accesses three national resources.

1. Palliative Care Research Cooperative: The National Institute of Nursing Research has funded a the Palliative Care Research Cooperative Group (PCRC) directed by Drs. Jean Kutner of the University of Colorado, Christine Ritchie of Massachusetts General Hospital, and Kathryn Pollak of Duke University (https://palliativecareresearch.org/). The PCRC includes an “Investigator Development Center” directed by Dr. Ritchie. The PCRC is a national organization with the mission of advancing palliative care research and research training. The PCRC and the Junior Investigator Training Center have access to large numbers of trainees, researchers, and mentors throughout the US, and are committed to advertise and disseminate training opportunities.

2. National Palliative Care Research Center: The Kornfeld Foundation, along with other funders, have funded this national center, directed by Dr. Sean Morrison of the Icahn School of Medicine at Mount Sinai, to promote palliative care research and research training (http://www.npcrc.org). The NPCRC provides an administrative home to promote intellectual exchange, sharing of resources (e.g., biostatisticians) and access to data from ongoing studies to plan and support new research. The Center also offers career development awards for palliative care research.

3. National Post-Doctoral Palliative Care Research Training Collaborative: This group represents a consortium of 8 postdoctoral palliative care research training programs (University of Colorado, Duke University, University of Pittsburgh, UCSF, University of Washington, Harvard University, University of Utah and University of Alabama Birmingham). This group collaborates on fellow recruitment and curriculum development, with monthly leadership conference calls to develop, support and further training materials and collaborative efforts. Recognizing that each program has a different focus and that trainees may have specific geographic preferences, the Collaborative has developed a joint recruitment flier that is distributed via national and international meetings and to academic palliative medicine fellowship programs nationally (http://www.npcrc.org/content/62/National-Post-Doctoral-Palliative-Care-Research-Training-Collaborative.aspx). The consortium conducts quarterly virtual conferences for all T32 trainees on research and career development topics.

Trainees and Outcomes

In the first 5 years of our program (2014–2019), we have trained 11 post-doctoral fellows; 7 have graduated from our program and 4 of are currently enrolled. Table 3 provides a summary of the demographics of these 11 individuals as well as key outcomes including publications and subsequent grants received.

Table 3.

Cambia Palliative Care Center of Excellence T32 Trainees, 2014–2019

Demographics	N = 11	Percent (%)
Female	9	82%
Underrepresented minority	1	9%
MD	8	73%
PhD	3	27%
Current status	N = 10	Percent (%)
Currently on this T32	4	36%
Clinician-scientist in faculty position	7	64%
Industry researcher	0	0
Academic Clinician Educator	0	0
Private Practice	0	0
Grant Applications/Success	Applied	Received (% success)
F32 fellowship funding	2	1 (50%)
K award funding	5	4 (80%)
R level funding	2	1 (50%)
Publications	Number	Average/trainee
Publications while on T32*	27	4.5
Research Degree (PhD or Masters)
Conferred	7	64%
Enrolled in graduate school	3	27%
Conferred or planned	10	91%

^*Includes 6 trainees who spent at least 1 year on T32; updated as of October 2019

Continuous Quality Assessment and Improvement

We endeavor to create a culture of continuous assessment and improvement with an evaluation program that includes formal evaluations of individual mentors, rigorous assessment of each trainee’s progress toward their educational goals, and a formal exit interview.

Evaluating Mentoring

Our T32 faculty provides mentoring in a number of different forums: mentoring one-on-one, participating on mentoring committees, providing career advice to fellows and junior faculty, and providing feedback during group discussions in our works-in-progress seminars, to name just a few. To fully evaluate all of their mentoring roles, mentors are assessed using a 360° evaluation system by many individuals and not just their mentees. We use a secure web-based tool that allows administrators to create, distribute, and analyze surveys. We use this program to seek feedback about each mentor from multiple sources, including from primary and secondary mentees, trainees who participate in the monthly PC-WIP meetings, junior faculty members, and peers.

Evaluating Trainee Progress and Performance

Evaluation metrics include the number of presented abstracts, published research papers, submitted grants, and funded grants. Table 4 provides an example of our minimum expectations for trainees. In addition, we evaluate progress using the mentoring committee structure. We distribute minutes from each mentoring committee meeting, as well as the IDP that is updated at each meeting, to the mentoring committee as well as the Program Directors. The minutes include an explicit evaluation of the trainee’s performance and specific action items for the trainee to accomplish between mentoring committee meetings. Progress on these items is reviewed at each meeting.

Table 4.

Minimum Expectations for T32 Trainees

Minimum Expectations

• Attend the monthly career development seminar for post-doctoral fellows

• Attend the monthly Palliative Care Research Works-in-Progress sessions and present 1–2 times per year

• Formal presentation of research at a different research seminar (besides the PC-WIP) at least once per year

• Develop at least 1–2 research projects per year with mentor(s), with timelines and productivity benchmarks in Mentoring Plan

• Meet with mentor(s) at least twice-monthly and meet with Mentorship Committee at least twice yearly

• Attend other relevant conferences as agreed on with mentor(s) and described in the Mentoring Plan

• Completion of required courses and other didactic training as determined by the Mentoring Plan

• Presentation of 1–2 abstracts each year at relevant national professional meetings

• Completion and submission of at least 2 peer-reviewed, first-authored research manuscripts each year

• Completion and submission of at least 2 peer-reviewed, collaborative research manuscripts each year

• Plan and complete a mentored grant application during the second year

• Maintain contact with the training program following training for collection of data on presentations, publications, grants, and positions

• Attend the UW Biomedical Research Integrity Series each summer

Annual and Exit interviews

We have implemented a formal process of annual and exit interviews as trainees complete each year of their research training. We ask all trainees to provide feedback on their training experience and to give us suggestions for improving our program. We request feedback via an annual interview with one of the Program Directors (the Program Director with the least direct involvement with the specific trainee). We have used this mechanism successfully and have found this input extremely helpful, resulting in substantive improvements in this program. Table 5 describes the key themes identified by graduating fellows and our programmatic response to this feedback.

Table 5.

Summarized feedback from fellow exit interviews (n=7)

Strengths of program

Areas for improvement and changes made

• Overall training environment • Protected time for research activities • Committed mentorship • Support for grant submissions including mock reviews • Connection to broader palliative care research community • Networking opportunities at local and national levels • Rigorous research training, including qualitative and behavioral science methods • Inter-professional training environment

• Desire for palliative care specific WIP meetings – initiated in 2016 • More focus on career development and planning for life after fellowship –added to monthly career development seminars • More structured process of onboarding new fellows – implemented summer series didactics and early meetings with Program Directors • Create more opportunities for fellow interaction – started monthly career development seminars, participating in National Post-Doctoral Palliative Care Research Training Collaborative • Create a repository of resources for fellows to reference – Provide access to relevant resources and creating space for palliative care specific resources on Cambia PCCE website

Conclusions

As the recognition of the need for high quality palliative care increases, so too does the need for high quality palliative care research to guide the assessment and implementation of evidence-based palliative care practice. We have described the structure and processes of our NIH-funded T32 post-doctoral research fellowship for palliative care research with the goal of promoting development of new programs and information exchange across programs in order to continue to build the field of collaborative and interdisciplinary palliative care research.