Communities of Practice and Learning: Disseminating Their Work

What is the use of a book, without pictures or conversations?

—Lewis Carroll, Alice's Adventures in Wonderland

Research published in the biomedical and/or education literature constitutes the “gold standard” of scientific evidence to enact change, and the Accreditation Council for Graduate Medical Education's (ACGME) standard-setting process is informed and profoundly influenced by scientific evidence. At the same time, a large portion of the current standards and processes for the accreditation of graduate medical education originate from the consensus of education and medical practice communities, with input from stakeholders and the general public. There are many reasons for using multiple sources to shape accreditation standards. Scientific data are often not available for an issue that needs a resolution, and large-scale, randomized, controlled educational trials to determine the effectiveness of approaches prior to implementation are usually not feasible or fundable. Resident and fellow education occurs in a complex system made up of various interconnected parts in which scientific research can neither produce evidence to support all aspects of change under consideration nor offer information on the likelihood of the intended outcome, or the nature and likelihood of any unintended effects. The design of traditional educational research projects, where a single intervention or a group of interventions is tested within an educational system while maintaining all other factors constant, is a complex undertaking. Furthermore, the generalizability of results from such single-institution trials is often questioned.

“Communities of practice” can contribute to the acquisition of new knowledge in areas where science may be hampered by the issues discussed above. The concept of communities of practice initially was studied in occupations with an apprenticeship model, in which the community acts as a living curriculum for the apprentice; it is now recognized in a range of settings, including organizations, government, education, and health care safety and quality, among others.¹ The article by Green et al² in this issue of the Journal describes a 2-year effort by the internal medicine community of practice to construct a framework based on the ACGME competencies that establishes specific, observable developmental milestones for internal medicine residents for use by programs in resident assessment and in the accreditation process. Green and colleagues present a rich and meaningful summary of the internal medicine community's effort to develop competency milestones; the article also suggests the need for a new format for reporting the process and outcomes of community consensus-building efforts, particularly when these are instituted to formulate complex interventions to advance education and practice. The article highlights that creating measurable competency milestones for residents as a tool for resident assessment and program evaluation in the accreditation process is a complex intervention.

Just what makes an intervention complex? One definition is that a complex intervention is made up of a number of separate elements essential to its proper functioning, in which the “active ingredient” that makes it effective is difficult to identify or specify.”³ Competency milestones have a number of active ingredients—clear expectations, behavioral observation, faculty development, resident assessment, data collection, and comparisons to national and peer data. Research in these areas is ongoing, and a number of articles in this issue of the Journal present important recent additions to this work. Thus, the work of setting standards through the work of communities of practice is not removed from scientific inquiry. It incorporates and interprets research conducted on assessment and clinical competence while not being bounded by the limits of current evidence.

The significance of consensus and community input regarding important changes in graduate medical education and its accreditation standards suggests a need for a well-developed, formal approach to disseminate the results of consensus-building efforts on new approaches in communities of practice and learning, and to use these as drivers of change in graduate medical education. It could be a means for disseminating to the educational community the results of the specialty community's consensus on residency milestones in pediatrics and surgery, the other specialties that are currently completing their milestone formulation process. It also could have broader applicability in reporting and vetting the recommendations of the ACGME's Duty Hour Task Force when it completes its work in 2010.

The effort to establish Standard for Quality Improvement Reporting Excellence (SQUIRE) has created a viable approach for academic reporting and dissemination of quality and safety improvements in clinical settings, giving this important work a place in the biomedical and education literature.⁴ SQUIRE is an additional resource for researchers and practitioners seeking an accepted and respected way to report academic literature findings, and it could serve as a model for formalizing reporting on the progress and results of consensus-building in the education community that is informed by science but may extend beyond where science can offer clear guideposts.

What might the steps be in developing a similar reporting methodology for community consensus? It could start with draft guidelines and obtaining informal feedback on their usefulness and limitations, beginning with authors who have written about efforts to create new processes through community involvement. Creating such guidelines may itself be organized around a community of practice, with a shared domain of interest in advancing transparency and “scientific” reporting on setting and revising accreditation standards and, more broadly, elements of clinical and educational practice. Additional steps may entail feedback from editors and experts on publication guidelines, wide dissemination to the medical education community for comments and revisions and, ultimately, beta testing and refinement using real manuscripts.

The availability of an accepted method for reporting this type of academic research may benefit the wider dissemination and understanding of the value of this work, along with its limitations. The aim is not to supplant or cheapen the role of scientific evidence, akin to applying Gresham's law (“Bad money drives out good”) to the formulation of accreditation standards. Community consensus cannot replace scientific evidence in the process of setting and revising standards. Wherever possible, accreditation standards and processes must represent scientifically validated, state-of-the-art medical and educational practice. The role of consensus efforts is to supplement science in the process of setting standards only in areas where research has not been done or cannot be done because of time, financial, ethical, and other considerations. The process is analogous to consensus-based guidelines in clinical practice. In addition, because scientific evidence by nature often is narrow, community consensus can assist in weighing its merit, relevance, and applicability in the learning environment. Contrary to Gresham's notion, scientific evidence and community consensus, as coin of the realm of changing accreditation practice, can circulate together.

It is our hope that the new Journal of Graduate Medical Education will create an opportunity for the educational community to disseminate its work, with the shared goal of enhancing the patient care quality in the United States and around the world through the enhancement of educational outcomes for our residents and fellows.

Note: The Internal Medicine Milestone Group invites residency program faculty, coordinators, medical team members, and any other stakeholders of the quality of care and education in internal medicine, including patients and members of the public, to comment on the draft milestones. Stakeholders and other interested individuals can view the milestones framework at http://www.abim.org/milestones/public/ and offer feedback through a survey linked to this site.