Clinical Competency Committee: Tools for Developing Tomorrow’s Physician Leaders

Introduction

Many practicing physicians participated in residency training in the Twentieth Century and remember the era of “see one, do one, teach one.” Residents seldom interacted with the program director and graduate medical education staff (many institutions did not even have designated staff for the purpose of graduate medical education). A chord of fear shot through the residents’ hearts in response to the thought of bothering an attending physician after hours. Interns sought guidance from upper-level residents, who sometimes waited until patients were in dire straits before approaching their attending physicians with panicked requests for assistance.

The primary training tool for the resident was the autonomous management of patient care. Exposure to a wide variety and number of patients was assumed to be adequate to produce competent physicians.¹ A written examination to determine core medical knowledge was the primary method of assessing graduate readiness. This education system assumed that a sense of duty to patients and core medical knowledge would produce good patient outcomes and effective physicians.¹

Patient Safety Concern

As patients acquired better access to medical information, they began to demand more of their physicians. Concern for the safety of patients under the care of residents in a hospital setting came to the forefront. On November 29, 1999, the Institute of Medicine (IOM) released a report called To Err is Human: Building a Safer Health System² recommending improved safety and oversight within a hospital system. In response to these concerns, it became necessary to evaluate residents on their ability to work effectively and safely within a health care system.

At the same time, burgeoning technological advances and employer sponsored health insurance transformed the practice of medicine from a solo endeavor into a complex multifactorial business. The solo physician became a relic of the past, most physicians becoming salaried physicians within a larger group connected with multiple resources. Payment reforms shifted the focus away from acute care fee-for-service payment systems to capitated payments requiring accountability towards providing care for populations of patients with chronic health conditions.³ These changes required resident physicians to learn to work as effective team members, exhibit exceptional communication skills and to demonstrate the highest levels of professionalism. Novel methods of evaluation had to be developed to assess new skills.

As scientific advances continued to accelerate, the need for cost containment and equity in health care delivery became evident. The Affordable Care Act stressed the need for improved access to care, and focused the public eye on health care. The public demanded increasing transparency and accountability in health care⁴ and the government started critically examining the funding of residency training.⁴ Requirement for continuous practice improvement became another area in which residents needed to be assessed.

Core Compentencies

The Accreditation Council for Graduate Medical Education (ACGME) began to respond to changes in the medical practice climate early on. In 1999, the ACGME introduced its Outcome Project which defined medical competency as a core group of six competencies.⁵ These core competencies formed the new basis for residency curricula and evaluation of resident graduation readiness. Outcome project core competencies included the traditional areas of evaluation, patient care and medical knowledge, and added requirements to achieve competence in systems-based practice, interpersonal and communication skills, professionalism, and practice-based learning and improvement. Much latitude was given to individual programs to define the criteria for competency in each of the core areas.⁵

In 2007, after nearly a decade of experience with the six core competencies, the ACGME Outcomes Project required delineation of specific anchor points or milestones to mark development of knowledge, skills and attitudes expected of a resident during training.¹ Each of these milestones was developed to anchor a subcategory within one of the six core competencies. Milestones were designed with the hope of measuring growth, stagnation or deterioration of resident performance.⁶ These milestones provided program directors and evaluators with observable goals and behaviors designed to assess a learner’s progression towards competency.⁶ These assessment milestones known as Next Accreditation System (NAS) Reporting Milestones were initiated in 2012 for pilot specialties and were enacted for all residency program specialties in 2014.⁷

Many practicing physicians participated in residency training in the Twentieth Century and remember the era of “see one, do one, teach one.”

The milestones were developed along a continuum of education which includes behaviors expected at the level of a novice advancing up to aspirational goals that practicing physicians may not meet.⁸ Level 1 milestones measure the knowledge, skills and attitudes expected of a recent medical school graduate upon entry into a residency program. Level 2 milestones reflect ongoing progress expected in the first and second years of residency. Level 3 milestones reflect basic competence and the ability to practice independently, although oversight may be required. Level 4 is the graduation target and represents behavioral anchors expected of a competent physician ready to enter independent practice. Level 5 is an aspirational level for residents attained by professional leaders and mentors within the specialty.⁹

The Current State of Evaluation for Family Medicine Residents

Each specialty has individual milestones to evaluate residents and fellows. The ACGME Family Medicine milestones contain between two and five subcategories for each of the six core competencies. Within each of these subcategories, there are the five levels of milestones. For family medicine residents, there are 22 subcategories with a total of 225 milestones to evaluate.¹⁰ Family medicine milestones reflect that during residency a developing physician moves from a dependent learner to an independent practitioner. Recurring themes within the milestones reflected that resident physicians are expected to develop an appreciation of the need for teamwork and incorporation of resources outside of self. The learner is also required to develop self-awareness, seek feedback, understand his or her limitations and know when to ask for help. Also, the learner must develop strategies to manage ambiguity.¹

To assess achievement in areas such as this, our residency program coordinator and a member of hospital administration who runs compliance reports are important members of our committee.

As the number of requirements for observing resident performance expanded, the ACGME perceived the need for a more formal system of resident evaluation. As part of NAS, they required the formation of a clinical competency committee (CCC) to meet regularly and assess resident performance.⁷

Individual milestones capture observable behaviors for a specified time and context. The CCC does not just check off individual milestones, rather the committee reviews repeated behaviors over time and in different clinical settings to determine if a resident is developing a habit of consistent performance within the competency domain.⁸ The CCC is expected to synthesize information from multiple sources such as rotation evaluations, 360 degree evaluations completed by hospital and clinical staff, patient surveys, in-training examination scores, objectives structured clinical examinations (OSCEs) using standardized and simulated patients, direct observation of residents and others.⁹

Every resident is evaluated twice a year on the 225 milestones. Therefore, lack of achievement or growth in a specific milestone can be identified by the CCC. The committee recommends interventions that assist residents in their growth towards graduation and independent practice. One of the primary goals of the CCC is to identify struggling learners and provide support early to insure proper progress towards the ultimate goal of independent practice which signifies graduation readiness. In this way, the process of assessing graduation readiness begins on the first day of orientation as a new family medicine resident.

A major task undertaken by the CCC in evaluating each resident is taking information from across a variety of settings and deconstructing this information into behaviors applicable to graduation milestones.⁶ The committee uses written and verbal evaluations from multiple sources to confirm that the learner possesses the knowledge, skills and attitudes needed for the safe, effective and compassionate practice of medicine within their specialty.¹¹

One barrier to evaluating residents was the fact that each evaluator brought their standards and criteria to a general milestone such as “effectively delivers bad news to patients.”¹¹ The initial task of our CCC was to look at each milestone and more clearly define how that milestone was going to be evaluated in the context of our family medicine residency program. The committee also had to look at the opportunities available in our program to decide what level of milestone was attainable. The next step was to rewrite our global rotation evaluations to correlate with the milestones. Several milestones cannot be evaluated in the context of a global rotation evaluation, for example “uses systematic improvement method (e.g. PDSA cycle) to address an identified area of improvement.”¹⁰

To assess achievement in areas such as this, our residency program coordinator and a member of hospital administration who runs compliance reports are important members of our committee. They provide essential information regarding resident participation in hospital safety and quality initiatives and progress toward meeting academic requirements. Resident self-evaluation is also vital to the process. The subcategory “Maintains emotional, physical and mental health; and pursues continual personal and professional growth”¹⁰ is measured by completion of a resident self-evaluation form, and by input from our behaviorist. The psychologist is a vital part of the CCC and provides valuable insight into resident behavior and progress toward independent practice.

To meet the momentous requirement of evaluating 14 residents on 225 milestones every six months, we have developed a system that we repeat every four weeks. Our family medicine residents enjoy the most autonomy on inpatient obstetrical and inpatient medicine services. Each of these services has a third year resident acting in a leadership role as the resident in charge (RIC). This resident supervises a team of first and second year residents. Two weeks into the rotation, each RIC provides a verbal report to the CCC regarding their assessment of the strengths and potential areas for improvement of each of their team members. The RIC is evaluated by faculty members supervising the service. The milestones are incorporated into our electronic rotation evaluations and the software then generates a numerical level from 1–5. Using this number as a guide, the CCC takes into account the information gathered from all other sources as previously mentioned and then assigns a numerical value to each milestone. A remediation strategy is recommended if the resident fails to progress at the level expected for their point in training.

During the final month of the residency program, all the information gleaned from the multiple sources outlined above is compiled for each of the graduating residents. The members of the CCC meet and review the information. Each resident has been discussed regularly throughout his or her residency program and interventions were introduced when needed to insure success. Criteria for successful graduation include achievement of a minimum of level 3 on all milestones, and achievement of level 4 on the majority of milestones. The resident meeting these criteria can then be recommended as a candidate for successful completion of our residency program. The program director then provides the American Board of Family Medicine with a completion letter attesting that “Dr X has demonstrated sufficient competence to enter practice without direct supervision and is qualified to practice medicine competently and independently as a family physician.”

Biography

Laura Hempstead, DO, (top left), Program Director, AOA Internship and Associate Professor; Margaret E. Gibson, MD, (top right), MSMA member since 2010, Associate Professor; Todd Shaffer, MD, (bottom left), Professor and Program Director Family Medicine Residency; and Jennifer Groner, DO, (bottom right), Assistant Professor, are all at the University of Missouri - Kansas City, Department of Community and Family Medicine and Truman Medical Center-Lakewood.

Contact: hempsteadl@umkc.edu