Are Pediatric Critical Care Medicine Fellowships Teaching and Evaluating Communication and Professionalism?

Abstract

Objective

To describe the teaching and evaluation modalities utilized by pediatric critical care medicine (PCCM) training programs in the areas of professionalism and communication.

Design

Cross sectional national survey.

Setting

PCCM fellowship programs.

Subjects

PCCM program directors.

Interventions

None.

Measurements and Main Results

Survey response rate was 67% of program directors in the United States, representing educators for 73% of current PCCM fellows. Respondents had a median of 4 years experience, with a median of 7 fellows and 12 teaching faculty in his/her program. Faculty role modeling or direct observation with feedback were the most common modalities used to teach communication. However, 6 of the 8 (75%) required elements of communication evaluated were not specifically taught by all programs. Faculty role modeling was the most commonly utilized technique to teach professionalism in 44% of the content areas evaluated, and didactics were the technique utilized in 44% of other professionalism content areas. Thirteen of the 16 required elements of professionalism (81%) were not taught by all programs. Evaluations by members of the healthcare team were used for assessment for both competencies. The use of a specific teaching technique was not related to program size, program director experience, or training in medical education.

Conclusions

A wide range of techniques are currently utilized within PCCM to teach communication and professionalism, but there are a number of required elements that are not specifically taught by fellowship programs. These areas of deficiency represent opportunities for future investigation and improved education in the important competencies of communication and professionalism.

Introduction

Since 2001, graduate medical education (GME) programs have been responsible for implementing processes to teach and evaluate the six general competencies established by the Accreditation Council for Graduate Medical Education (ACGME).(1) Recent emphasis by the ACGME, American Board of Pediatrics (ABP), Association of Pediatric Program Directors, and others involves a movement toward a more competency-based educational model.(1-4) These collaborative efforts among pediatric and graduate medical education leadership groups led to the recent release of the Pediatrics Milestones Project, further shifting the educational focus toward the “development of learners across the education, training, and practice continuum.” (4)

Within this continuum of education, interpersonal communication and professionalism are two competencies that have traditionally been difficult to both teach and evaluate. For each of these competencies, there are specific content areas that must be taught and assessed, but the actual implementation of the education and evaluation process is often challenging. There are a wide range of teaching methods for both communication and professionalism described in the literature, including the use of standardized patients or standardized families, peer role play, small group problem-based learning, simulation, and critical incidents. (5-10) Each of these methods has its own advantages and disadvantages, and no single method has been demonstrated to be superior.

Evaluation of these competencies is perhaps even more challenging. The Kalamazoo II Report, summarizing the conclusions from a conference focused on the state of the art in teaching and evaluating competence in communication and interpersonal skills, lists three general methods for assessing communication and interpersonal skills. (11) These three general methods encompass a wide range of specific evaluation methods described or recommended in the literature. (12-29) However, there is no single method that is accepted as being reliable and valid for the assessment of either communication or professionalism, and it has been argued that multiple methods must be combined to produce a reliable assessment of these multidimensional competencies. (12, 27, 28, 30)

Communication and professionalism are especially important in pediatric critical care medicine, where providers are responsible for critically ill patients with complex disease processes and multifaceted teams. Despite the importance of these two competencies, it is currently unknown what techniques are being utilized in Pediatric Critical Care Medicine (PCCM) fellowship programs to teach and evaluate communication and professionalism. It is also unclear whether formal instruction and assessment is being provided in the necessary and ACGME mandated areas of professionalism and communication. This study seeks to address potential gaps in knowledge and programmatic deficiencies that may exist in PCCM fellow education and evaluation in professionalism and communication.

Materials and Methods

A multi-center education research collaborative, the Education in Pediatric Intensive Care (E.P.I.C.) Investigators, developed a 50 question survey using the modified Delphi technique to be administered to PCCM fellowship program directors. The initial version of the survey was based on ACGME and ABP requirements for teaching of communication and professionalism in graduate medical education. (1, 31) (Table 1) After review and editing, the final survey elements included 17 questions regarding specific elements of teaching and evaluating communication, 20 questions regarding specific elements of teaching and evaluating professionalism, 9 demographic questions, and 4 qualitative questions regarding perceptions of teaching and evaluating communication and professionalism. The survey was then piloted among 7 program directors to further address the clarity of the questions, along with the content, format, and length of the survey. To improve consistency of responses, specific definitions of the teaching techniques were included with the survey. (Table 2) The anonymous survey was then sent electronically to the 66 program directors of all accredited PCCM fellowship programs in the United States. After original electronic mailing, two subsequent reminders were sent.

Table 1. Required Elements of Communication and Professionalism (1).

Required Elements of Communication

Communicate effectively with patients, families, and the public, as appropriate, across a broad range of socioeconomic and cultural backgrounds

Communicate effectively with physicians, other health professionals, and health related agencies

Work effectively as a member or leader of a health care team or other professional group

Act in a consultative role to other physicians and health professionals

Maintain comprehensive, timely, and legible medical records, if applicable

Required Elements of Professionalism

Compassion, integrity, and respect for others

Responsiveness to patient needs that supersedes self-interest

Respect for patient privacy and autonomy

Accountability to patients, society and the profession

Sensitivity and responsiveness to a diverse patient population, including but not limited to diversity in gender, age, culture, race, religion, disabilities, and sexual orientation

Table 2. Definitions of Teaching Methods Utilized.

Division didactic: teaching in a classroom setting in which session is organized and led by the Division. These can be formal presentations or interactive sessions.

Department/Institution/GME didactic: teaching in a classroom setting in which session is organized and led by the Department, the Institution or the Graduate Medical Education Office. These can be formal presentations or interactive sessions.

Simulation: teaching activities based on simulated exercises including those using mannequins, standardized patients/parents, and roleplaying.

Web-based module: internet based teaching. Modules may be predominately didactic or interactive.

Journal Club: teaching related to the in depth review and discussion of preselected articles.

Clinical case review including M and M conferences: teaching related to the review and analysis of a specific case. This can occur in several forums, including Case conferences and Morbidity and Mortality conferences.

Quality improvement activities: teaching from participation in activities devoted to quality improvement including a specific QI project or service on a QI committee.

Formal faculty-fellow mentorship: one on one education provided by a formally identified mentor.

Faculty role modeling: observation by the fellow of a faculty member in performing his/her duties. Modeling may be implicit or explicit and not necessarily include specific discussion/feedback.

Direct observation of fellows by faculty: direct observation of fellow performance with immediate feedback. Examples include feedback after observing a procedure performed, a fellow directed family conference, or a fellow's interaction with another healthcare professional.

Experiential learning with feedback: retrospective review of an event with feedback. Examples include providing feedback to the fellow after being made aware of a difficult interaction that he/she had with another healthcare professional, providing feedback on rounds regarding the autonomous activities performed during a prior overnight call, or after a complaint about his/her behavior in relation to a specific event.

Statistical Methods

Questions were categorized into those relating to teaching and evaluation of either communication or professionalism. Standard summary statistics and graphing techniques were used to describe answers in each category. The Wilcoxon Rank-sum test was used to compare answers to questions across various program categories. Data are reported as median [range].

Results

Survey response rate was 67% (44/66). Respondents represented educators for 73% of PCCM fellows currently in training. Program Directors had a median of 4 [1-27] years of experience, with 7 [2-12] fellows and 12 [4-45] teaching faculty in his/her program. The total median number of ICU beds, including those dedicated to cardiac patients, was 36 [14-90]. Forty-three percent of programs had no one with formal training in medical education involved in the training program and only 6% of programs had more than two people with this formal training (median 1 [0-4]). Eighty-five percent of program directors had received no formal training in directing a fellowship program, and over two-thirds of respondents had participated in an ACGME review of his/her program at the time of survey completion.

Faculty role modeling and direct observation by faculty with feedback were the most common modalities used to teach communication (Figure 1), with these two modalities being most commonly used to teach 7 of the 8 (88%) required communication areas tested. However, six of the 8 (75%) required elements of communication were not specifically taught by all programs. (Table 3) The most prevalent deficiencies were a lack of mechanism to teach communication in the context of a non-clinical group in 32% of programs, and no specific mechanism to teach communication and consultation outside of the ICU environment in 25% of programs.

Figure 1.

Comparison of the modalities utilized to teach communication and professionalism in pediatric critical care medicine.

Table 3. Survey Questions Addressing Communication.

Teaching Communication Survey Elements	n = 44
How do you teach fellows:	Programs with no formal teaching (%)
Effective communication and collaboration with patients and families?	0 (0)
Effective communication across a broad range of socioeconomic and cultural backgrounds?	5 (11)
How to handle difficult conversations/interactions with patients, families, and colleagues?	1 (2)
Effective communication and collaboration with other physicians and health care professionals?	3 (7)
To provide consultation outside the ICU environment?	11 (25)
To communicate and work effectively as a member or leader of a healthcare team?	0 (0)
To communicate and work effectively as a member or leader of a non-clinical group or committee?	14 (32)
To maintain and communicate through comprehensive, timely, and legible medical records?	8 (18)

Faculty role modeling and didactics were the most common modalities used to teach professionalism. (Figure 1) Didactics were the most common method utilized to teach 7 of 16 (44%) required professionalism areas tested, and faculty role modeling was the most commonly utilized modality in 7 of 16 (44%) required topics, but thirteen of the 16 required elements of professionalism (81%), were not taught by all programs. (Table 4) The most prevalent deficiencies were that 25% of programs had no mechanism to teach the maintenance of professional standards, 18% no method to teach peer review, and 16% no mechanism to teach fellows the impact of grief and loss on themselves.

Table 4. Survey Questions Addressing Professionalism.

Teaching Professionalism Survey Elements	n = 44
How do you teach fellows:	Programs with no formal teaching (%)
To be accountable to patients, families, physicians, society, and the profession of medicine?	1 (2)
Sensitivity and responsiveness to diverse patient populations, cultures, sexual orientations, religions, and patient disabilities?	3 (7)
Patient confidentiality/HIPPA?	1 (2)
Appropriate resource allocation?	4 (9)
Recognize risk factors and signs of stress, sleep deprivation, impairment, burnout, depression, drug and alcohol abuse and mental health disorder in his/her self or others and its impact on patient care?	0 (0)
Consider the potential conflict between personal beliefs/circumstances and professional values?	4 (9)
Maintain appropriate professional boundaries?	6 (14)
Effectively conduct peer review?	8 (18)
Recognition and disclosure of conflict of interest in research or clinical practice?	1 (2)
Legal and ethical issues in end of life decision-making?	0 (0)
Impact of grief and loss on his/her self?	7 (16)
Function of institutional ethics committees?	2 (5)
Ethical issues in research?	0 (0)
About informed consent?	1 (2)
Importance of maintenance of professional standards?	11 (25)
Importance of life-long learning and strategies to develop this practice?	3 (7)

Faculty role modeling was perceived by Program Directors to be the most useful modality to teach both communication and professionalism. (Figure 2) There were no differences (p > 0.05) in the techniques utilized to teach or evaluate communication or professionalism related to program size, duration of Program Director experience, Program Director participation in an ACGME review, or departmental/divisional availability of a person with advanced education training.

Figure 2.

Comparison of the modalities perceived by pediatric critical care program directors as most useful to teach communication and professionalism.

In regard to evaluation, the 6 questions assessing techniques utilized for evaluation of communication with families and members of the multidisciplinary team demonstrated that a median of 37 [12 – 42] (84%) of programs use evaluations by nurses and other health care professionals. A median of 29 [18-32] (66%) of programs use self-evaluations and 11 [2-14] (25%) use peer evaluations for the assessment of communication with patients, families and the multidisciplinary team. Twenty-three percent of programs (median 10 [0 – 34]) used evaluations by parents to assess communication. Multi-rater evaluations were felt to be the most useful mechanism to evaluate communication by 63% of respondents. The question addressing evaluation of professionalism demonstrated that almost all (91%) programs used evaluations by nurses and other health care professionals. Self-evaluations were used to evaluate professionalism in 23 (52%) of programs, peer evaluations were used in 10 (23%) of programs and 27 (61%) used evaluations by parents. Fifty-six percent of respondents felt that multi-rater evaluations were the most important mechanism to evaluate professionalism.

Discussion

Interpersonal communication and professionalism are vital to the practice of medicine regardless of discipline, level of training, or specific responsibility in caring for patients. Competency in interpersonal communication consists of a number of complex elements involving communication with patients, families, and other members of the multidisciplinary team, all of which may require varying approaches and techniques. In addition, communication within PCCM involves a range of complex, emotional, and potentially difficult discussions which include neglect, child abuse, end of life, and many others. Similarly, professionalism includes a wide range of concepts and behaviors that are critical to being an effective health care provider.

In this study, we demonstrated that despite the requirements to teach multiple specific components of both communication and professionalism, substantial deficiencies exist among PCCM training programs, with almost 80% of the required communication and professionalism elements not being taught by all programs. While the ACGME and ABP have established required elements that should be taught by all programs, the ‘ideal’ curriculum is not yet established. Despite the challenges that exist in teaching and evaluating these competencies, a comprehensive curriculum in communication and professionalism is attainable and should be no less rigorous than mechanical ventilation, pathophysiology, or any other competency within PCCM. The components of a curriculum in communication and professionalism should include clear goals and objectives, a range of teaching techniques to address the needs of adult learners, and validated evaluation tools. While there is no ‘one size fits all’ model for curriculum development, as duty hours continue to become more restrictive and the educational system moves toward a more outcomes-based approach, increasing emphasis is being placed on how all of the competencies, including communication and professionalism, are taught and evaluated.

The barriers to teaching these competencies are not completely clear, and data within the medical literature are largely limited to single center reports of programs and curricula to teach communication or professionalism (5-10, 32, 33). Given the traditional challenges associated with teaching these two complex and multifaceted competencies, it is likely that similar deficiencies exist across other disciplines, specialties, and subspecialties. Data regarding education in this area are limited, with reports ranging from 10% of surgical residency programs to 35% of pediatric residency programs without professionalism curricula, (34-36) and it is likely that programs with explicit curricula have areas that are not adequately taught and evaluated.

While deficiencies are present, there are a number of components of communication and professionalism that are explicitly taught, but programs rely heavily on faculty role modeling as a primary teaching method. Historically, faculty role modeling has been utilized within medical education as a common teaching method, but this approach may not necessarily provide the explicit education that is necessary for a given situation (37). Faculty role modeling also often places a significant burden on the learner's ability to distinguish ‘good’ from ‘bad’ examples and apply this experience appropriately. Some trainees may not distinguish these differences as well as others, and there may be circumstances in which the faculty role models also may not provide the necessary insight to optimize the educational experience. Evidence suggests that to be truly effective, role modeling must be explicit rather than implicit as is typically used. That is, the learner must be specifically told to observe how a professional or communication task was performed and participate in active reflection on the experience (38, 39). Faculty role modeling is unquestionably an important element of the graduate medical education process (40, 41) but further data are needed to determine if this widely used approach is the most effective technique in the current climate of medical education.

As an example, this survey demonstrated that 75% of program directors stated that they have specific mechanisms to teach ‘communication and collaboration across a broad range of socioeconomic and cultural backgrounds,’ but faculty role modeling was the primary mechanism used to teach this concept in 82% of programs. While it is reassuring that three quarters of programs provide teaching in this area, a topic of tremendous importance in healthcare, an unanswered question for future investigation is the effectiveness of faculty role modeling as the main instructional method.

This study also demonstrated that an alternative approach used by many PCCM programs to teach professionalism is the didactic teaching session. In many circumstances, these didactic experiences are part of a larger lecture based curriculum that includes all subspecialty trainees, not just PCCM, at a given institution. Teaching in this manner allows for programs to address many of the necessary professionalism elements, but the effectiveness of this technique is not known. An important consideration is the balance between program efficiency to teach the multitude of required and necessary topics versus the optimal teaching method to provide the most valuable and productive experience for the learner. In the busy critical care training program, these two goals may be in conflict and require careful consideration. Another factor is the growing evidence that interactive educational experiences are more effective than traditional lectures, a technique that is more conducive to small groups. (42-44)

Examples of interactive experiences demonstrated in this study to be utilized by PCCM programs to teach communication and professionalism are quality improvement activities, journal club, mentorship, experiential learning, clinical case reviews (including morbidity and mortality conferences), and simulation-based experiences. While the majority of programs use these interactive methods to teach both communication and professionalism, program directors rarely feel that these techniques are most effective (Table 1 and 2). While perceived effectiveness of these techniques is low among program directors, the overall impact of any single technique remains unclear. It is possible that interactive techniques represent an opportunity for curricular development and improvement, but further investigation is needed to determine the effectiveness of the various methods identified in this survey.

Regarding evaluation, the majority of programs also use a wide range of methods for both communication and professionalism, but substantial variation exists in the use of peer and parental evaluations. The majority of program directors in this survey felt that the use of multi-rater evaluations was the best method to assess both communication and professionalism, but the impact and effectiveness of these techniques on trainees is uncertain. Evaluation of these competencies represents an area for potential intervention and improvement within PCCM training.

While this survey demonstrates that there are numerous areas for potential investigation and improvement in the areas of communication and professionalism education, it is not without limitations. The elements included in the survey are based on criteria and requirements set forth by the ACGME and ABP, but there are areas that may be important and relevant for practice that were not included. Additionally, recall bias may also have contributed to the results that were reported by program directors in this study. Also, while two thirds of program directors responded, it is possible that there may be differences in teaching and evaluation techniques among those who did not respond to the survey.

This survey focused on how program directors teach and evaluate PCCM fellows in the areas of communication and professionalism, and an important next step is the assessment of fellow experience and what they are learning in these areas. Based on the information from this survey, it is clear that there are numerous areas for both intervention and improvement in the education of communication and professionalism within PCCM, but identification of gaps and inconsistencies between fellow perceptions and those of program directors is a key aspect of improving the educational process as graduate medical education continues to evolve. Another mechanism to potentially improve fellow education is through more formal education training for program directors. This survey demonstrates that access to formally trained educators within PCCM is limited, and in the current environment, there are increasing opportunities for program directors to enhance their training.

As a next step, we plan for multicenter intervention and investigation using a collaborative and cooperative approach among programs through a newly formed education network, the Education in Pediatric Intensive Care Investigators. This collaborative approach within education-based research will allow for a more robust assessment of impact, effectiveness, and patient-centered outcomes when compared to a single center intervention. This prioritization of trainee education is an important responsibility as we attempt to train the next generation of pediatric intensivists.