Competencies: From Deconstruction to Reconstruction and Back Again, Lessons Learned

Abstract

I address the potential impact of the Association of Schools of Public Health’s development of a competency model for the graduate Master of Public Health. I reflect on the model in relation to the Accreditation Council for Graduate Medical Education’s adoption of a competency-based model for medical education.

Six lessons learned by the Accreditation Council for Graduate Medical Education that the Association of Schools of Public Health might consider in moving forward are how learning outcomes can be enhanced by using competency models, the effect of competency development processes in “creating a common language” among educators, the benefits and challenges of numerous competencies within a model, the usefulness of the Dreyfus model for progressive competency development, the need for multiple assessment tools used over time, and the value of learning portfolios.

This committee is treating this issue the way a dog does an old shoe; it’s too big to swallow, but they can’t let it go.

—Marvin Dunn, MD, former director, Department of Accreditation Committees, Accreditation Council for Graduate Medical Education, written communication, June 2000

DEBATES ABOUT THE benefit of deconstructing competence into component elements (competencies) have been going on for a long time. One of the more notable of these debates occurred in Victorian England between Robert Lowe, vice president of the country’s Education Department, and the famous Victorian poet, Matthew Arnold. Lowe, who gave us the “three R’s,” favored a payment system in which schools would receive funds based on examination of each student—full government payment would be awarded only if each student could demonstrate competence in reading, writing, and arithmetic.1 Arnold, who found it impossible to support his family on poetry alone, spent his entire working life as a school inspector. He visited as many as 300 schools each year and examined about 20 000 students annually.2 This experience gave him opinions about the use of educational outcomes in schools:

It turns the inspectors into a set of registering clerks, with a mass of minute details to tabulate . . . [and will] necessarily withdraw their attention from the religious and general instruction, and from the moral features of the school. In fact the inspector will just hastily glance round the school, and then must fall to work at the “log-books.” And this to ascertain the precise state of each individual scholar’s reading, writing, and arithmetic! It is as if the generals of an army . . . were to have their duties limited to inspecting the men’s cartouch-boxes. The organization of the army is faulty: -inspect the cartouch-boxes! The camp is ill-drained, the men are ill-hutted, there is danger of fever and sickness. Never mind; inspect the cartouch-boxes! But the whole discipline is out of order, and needs instant reformation: no matter; inspect the cartouch-boxes!3

Arnold was convinced that the result of the three-R’s approach would be the inevitable decline in the education of the people.

In this issue of the Journal, Calhoun et al. report on the Association of Schools of Public Health’s (ASPH’s) national study to identify core competencies for the Master of Public Health degree. They also discuss the process used to develop these competencies. With this study, the ASPH has responded to calls for reform in health profession education and has joined other health professions in moving to competency-based education.

In 1997 the Accreditation Council for Graduate Medical Education (ACGME) moved to a competency-based model and is using it to accredit the nation’s 8000 residency programs that in aggregate teach 100 000 residents. In this commentary I seek to celebrate and congratulate the ASPH on its work and also to offer the ASPH six lessons learned by ACGME over the past 10 years.

LESSON 1: BOTH ARMS OF THE PARADOX MUST BE HONORED

Both Lowe and Arnold were right. There is tremendous power in focusing on particular competencies, and yet, competence is a habit, and learning good habits is helped by a context that supports and models good habits. Competence is de-constructed into competencies to measure and to improve the elements of competence, and yet, the whole of competence is greater than the sum of its parts. Conforming to the truth requires that both arms of the paradox, competence and competencies, be honored. Judging learners requires both deconstruction and reconstruction of their competence.

It is clear that the ASPH is not advocating an “either/or” approach. By fostering a national dialogue and identifying competencies, they have clarified and focused the efforts of both teachers and learners. The experience of ACGME would suggest that this process will enhance the graduate medical education community’s awareness of competence as a whole.

LESSON 2: NAMING THE COMPETENCIES IS A GOOD WAY TO BUILD COMMUNITY

The ASPH engaged in an iterative series of conversations with more than 400 individuals as it developed the final list of competencies. The ACGME did likewise: we started with 84 competencies that had been identified in the literature as important for physicians, clustered them into 13 categories, and sent them out to the world of experts, residency program directors, deans, chief executive officers, and others with two queries: Is this competency important? Is it feasible to measure it? An advisory committee received the input from these surveys and recommended the final list to the ACGME board for adoption. The board adopted six competencies: patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice.

The ACGME accredits 122 specialties and subspecialties. Naming competencies offered the potential to either bring together or separate farther these different communities. Seeking the former, the conversations focused on things that all physicians, independent of specialty, had in common. This proved to be a good strategy. It enabled wide acceptance by the umbrella groups of medicine and their components: all of the certifying boards, the specialty societies, the apparatus for continuing medical education, the American Osteopathic Association, and others. Eventually the competencies were incorporated into the Joint Commission’s credentialing standards.

The ASPH focused on the broad practice of public health. Of the 12 core domains, five reflect traditional core disciplines and seven reflect crosscutting competencies. The crosscutting competencies may have the effect of both reinforcing the public health community and linking public health more closely with the other health professions that have adopted similar competencies. Competencies have the effect of creating a common language with which we can have conversations about the work of health care.

LESSON 3: OUR FRIENDS COULD CONSISTENTLY REMEMBER SIX THINGS, BUT NOT SEVEN

The ACGME resisted the opportunity to deconstruct physician competence into an almost infinite set of competencies. Six competencies could be recalled by a group of people interested in and working on the project without referring to a document and those volunteers and employees of ACGME who were working on the problem were not certain that would be true for seven. If conversations about competencies had to begin with looking for documents, we thought that much would be lost. This is especially true for residents and clinician–educators engaged primarily in experiential learning.

The ASPH settled on 12 competency domains that would provide the basis for an organized curriculum in the classroom and in formal didactic settings. One hundred nineteen individual competencies were quite daunting and provided a framework for assessment and deconstruction of the 12 domains, rather than for conversation. Conversations will be needed between the National Board of Public Health Examiners and the ASPH about how competencies will be used in national examinations.

The ACGME used “quadrads” for these assessment conversations. Four representatives of each of the main specialties met in retreat settings and developed broader approaches to assessment than simply multiple-choice examinations. Each quadrad (the relevant residency review committee, the certifying board, the relevant program directors’ group, and a resident from that specialty) considered the best assessment tools for its discipline, and together, the 26 quadrads came up with four general categories of assessment tools (multiple-choice examinations, focused observation and evaluation of particular skills, 360-degree assessments, and a learning portfolio). It was recognized that the use of different types of assessment tools over time provided a more robust assessment system than had been in place.

The ACGME learned to distrust any expressions that included the word “covered,” as in “we covered that” or “how can we cover this?” Knowledge, skills, and attitudes have depth as well as breadth. Teachers delude themselves and their students if they claim that anything is ever “covered.” If one looks under those covers there is not much there. Rather than a list of topics to be covered, competencies offer a vocabulary for conversations about the work. In the case of residents, learning does not begin with goals and objectives; it begins with experience and reflection on experience. Competencies are very helpful in organizing those reflections.

LESSON 4: THE ALTERNATIVE TO COMPETENCE IS NOT NECESSARILY INCOMPETENCE

Knowledge, skills, and other attributes of learning are not simply present or absent, there is a continuum that proceeds in an orderly fashion as described by Dreyfus.4 The learner first learns the rules of the discipline (novice), begins to apply the rules in certain contexts (advanced beginner), becomes more engaged in context and becomes accountable (competent), then learns to read contextual cues quickly and reliably (proficient), and integrates learning into a personal style (mastery). The ACGME has found it very useful to use the Dreyfus model in following the progress of resident physicians. Because the same six competencies have been adopted by the certifying boards, continuing medical education apparatus, and credentialing system, it is possible to have depth as well as breadth for each of the competencies.

In making public health competencies more explicit, the ASPH will improve the quality and accountability of public health education and training. Learners in graduate school may be at the novice or advanced beginner stage of development for many of the competencies. It would be helpful to apply the Dreyfus model in their self-assessments after graduation.

LESSON 5: ASSESSMENT OF COMPETENCIES REQUIRES DIFFERENT TOOLS APPLIED OVER TIME

As mentioned above, ACGME settled on four general types of assessment tools: cognitive tests, 360-degree evaluations, direct focused observation of skills, and caselogs or portfolios of experiences. Some of the competencies lend themselves to one or another of the assessment tools. For example, medical knowledge is best tested by cognitive exams; professionalism and interpersonal and communication skills are best examined by 360-degree evaluations, direct observation, and computer-based vignettes; and practice-based learning and improvement by a portfolio of experiences. Looking at the competencies with different instruments and looking at the same competency over time has proven to be useful.

The cognitive tests include not only board certification exams, but also annual standardized national in-training exams. Various 360-degree instruments have been developed. Instead of just asking the resident’s supervisor about the resident’s performance, now patients, nurses, peer residents, medical students, and others contribute to the resident’s evaluation. Direct focused observation of residents has included standardized patients and simulations, but also includes an approach as simple as having an experienced faculty member observe a resident interviewing and examining a patient. Caselogs provide a record of a resident’s experience in procedures and, more recently, with a variety of clinical cases. Each year, residents enter about four million cases on ACGME servers, and these logs serve as a record for them and, with their consent, are transferred to the certifying board for admission to the certifying examination.

LESSON 6: LEARNING PORTFOLIOS ARE PROBABLY QUITE USEFUL

Graduates of MPH programs follow very diverse career pathways and will have very different experiences. Competence may be thought of as the demonstrated habit of reflective practice. An interactive tool is needed to support professional development during and after graduate school. The ACGME accredits 122 specialties and subspecialties. As with public health graduates, physicians follow very diverse pathways into their careers. In February 2007, the ACGME board approved development of a learning portfolio for residents.5 The portfolio went into alpha testing in July 2007. Portfolios have been widely used in grade school, high school, and undergraduate communities, and learners frequently come into residency having had experience with portfolios.

In the case of residency, learning begins with experience rather than with goals and objectives. Residents do not know what they are going to learn until after they have seen the patient, or opened the abdomen, or viewed the film. Accordingly, ACGME’s portfolio begins with a drop-down menu of educational experiences, offers an opportunity to capture reflection on things learned, and offers evaluation that uses standardized assessment tools that, in turn, are aligned with the competencies. It may be of use for lifelong learning to use a similar portfolio for graduates of schools of public health.

CONCLUSION

One hundred years ago, medical education needed reform and a system was put into place to distinguish the “quacks” from the qualified. Board certification, accreditation of medical education, and more-rigorous credentialing resulted. Being qualified proved helpful but inadequate; given the high risk associated with health care, it became appropriate to provide evidence that individual physicians were competent as well as qualified (hence the emergence of competency-based learning and educational programs).

The social trend, however, is broader and more vigorous than that. There is a strong desire and need for accountability and transparency at all levels of society. The ASPH has responded to this trend by facilitating a broad dialogue about the competencies relevant for schools of public health. Use of the identified competencies will have anticipated and some unanticipated effects, but in my view, is clearly the right thing to do. The real goal is to build a learning community that is based on data and enables the broad community interested in health care to improve the health of the public. Competencies allow a common language for the conversations. Both Lowe’s and Arnold’s views are correct—we need the equivalent of the “three R’s” and also to pay attention to the habit of competence. Both arms of the paradox are strengthened by the ASPH’s effort.