Systems-Based Practice: Time to Finally Adopt the Orphan Competency

Systems-based practice (SBP) is 1 of 6 core competencies from the Accreditation Council for Graduate Medical Education (ACGME).¹ Residents demonstrating competency in SBP “understand complex systems and the physician's role in them, navigate them for the benefit of patients, and participate in continually improving them.”² The definition, performance, training, and evaluation of SBP is considered by many to be the most challenging among the 6 competencies.^3–5

Despite the ACGME's introduction of SBP as a core competency more than 20 years ago, with the requirement for SBP teaching and assessment, the published literature suggests that program directors continue to struggle with implementing curricula and assessing residents' competency in this domain.⁴ Teaching of the various subcompetencies of SBP is inconsistent and variable across training programs.^6–8 Many programs rely on passive learning approaches and indirect observation without objective criteria for demonstration of residents' knowledge and skill.⁹ As a result, many residents graduate with a persistent knowledge and skill gap in SBP. This outcome defines both a gap and an opportunity within and across institutions to develop and share feasible and generalizable SBP curricula and assessment methodology. An explicit focus on SBP subcompetencies during training is foundational to professional accountability; understanding both individual and team responsibility helps ensure high-functioning, cost-effective, quality-driven work environments that optimize patient care delivery. Review of the current literature identifies barriers to adoption of the SBP competency in graduate medical education, SBP teaching and assessment methods and tools available for use or adaptation, as well as processes to guide programs in the development of SBP curricula and assessment methods.

Barriers to Adoption

Since implementation, residency programs have modified existing curricula to incorporate the ACGME competencies. While most integrate nicely within established models of medical education, incorporation of the SBP competency has been challenging and not readily adopted.^3,4,10,11 The examination of barriers is an important step in identifying solutions. The reasons programs have failed to fully adopt this “orphan” competency are multifactorial, including but not limited to, conceptual difficulty, inadequate faculty understanding of SBP, lack of formalized curricula (eg, coding, insurance authorization), poorly defined teaching and assessment methods, few opportunities for direct observation of trainees, and challenges finding room in a time-limited training program.^8,12–14 The conceptual problem is exemplified by the broad variation in the SBP Milestone subcompetencies defined across specialties (Table provided as online supplementary data). “This variability may create differential expectations of residents across specialties, complicate faculty development, and make sharing assessment tools difficult.”¹⁵

Table.

Systems-Based Practice Teaching and Assessment Methods and Tools

Citation	Teaching Method	Assessment Method/Tool
Papademetriou M, Perrault G, Pitman M, et al. Subtle skills: using objective structured clinical examinations to assess gastroenterology fellow performance in system based practice milestones. World J Gastroenterol. 2020;26(11):1221–1230. doi:10.3748/wjg.v26.i11.1221.	Teaching through formative feedback based on OSCE assessment.	OSCE Milestones assessment with validated checklists used for formative feedback. Domains assessed: “Works effectively within an interprofessional team” “Recognized system error and advocated for system improvement” “Transitions patients effectively within and across health delivery systems”
Dolansky MA, Moore SM, Palmieri PA, Singh MK. Development and validation of the systems thinking scale. J Gen Intern Med. 2020;35(8):2314–2320. doi:10.1007/s11606-020-05830-1.	N/A	Validated 26-item scale to assess systems thinking
Samala RV, Hoeksema LJ, Colbert CY. A qualitative study of independent home visits by hospice fellows: addressing gaps in ACGME milestones by fostering reflection and self-assessment. Am J Hosp Palliat Care. 2019;36(10):885–892. doi:10.1177/1049909119836218.	Reflection and self-assessment during a designated rotation/experience (hospice home care) Self-identified knowledge and skill gaps Implemented self-improvement plans Enhanced teamwork	Reflection and self-assessment provide a qualitative assessment of SBP knowledge and skills, including teamwork.
Williamson K, Moreira M, Quattromani E, Smith JL. Remediation strategies for systems-based practice and practice-based learning and improvement milestones. J Grad Med Educ. 2017;9(3):290–293. doi:10.4300/JGME-D-16-00334.1.	Strategies for remediation of learners struggling with performance in the SBP Milestone domain	N/A
Prince LK, Little DJ, Schexneider KI, Yuan CM. Integrating quality improvement education into the nephrology curricular milestones framework and the clinical learning environment review. Clin J Am Soc Nephrol. 2017;12(2):349–356. doi:10.2215/CJN.04740416.	Presents a curriculum design with SBP components: Systems-based challenges Design and implementation techniques for system changes	Assessment of SBP Milestones through performance of multidisciplinary quality improvement projects (knowledge, skills, teamwork)

Abbreviations: OSCE, objective structured clinical examination; SBP, systems-based practice.

A significant obstacle to teaching and assessing SBP is the fact that its foundational cornerstone, “systems thinking,” is commonly absent from medical education curricula.^16–19 “Systems thinking is defined as the ability to analyze systems as a whole, including the recognition of essential interrelationships within the system and between subsystems, and any changes and patterns that arise out of the networks of relationships and interactions,” reported Colbert and colleagues.¹⁶ Resident education has traditionally focused on acquiring medical knowledge and individual patient care skills. Consequently, residents often do not develop awareness of and integrate into care the boundaries, functions, stakeholders, or critical interfaces within the greater health care system.^16,20 One survey of program directors and residents documented their perception that SBP is the least important resident competency.¹² Physicians need to be cognizant of the system around them and how their actions can positively or negatively affect patient care in the broadest sense. For example, a physician working slowly in clinic can create a bottleneck in the system with negative effects on other areas, including other health care workers and patients; if the physician works too quickly, the system (eg, nurses, medical assistants, laboratory staff, and checkout personnel) may not be able to accommodate the patient flow.

In order for residents to develop and apply systems thinking skills, their teachers must be versed in SBP themselves. Many faculty find the concept of SBP unclear and know little about how it applies to their individual practices or about the interrelationships within their own clinical systems; as a result, residents' cognitive and behavioral skill acquisition in SBP is compromised.²¹ Our complex and everchanging health care system is increasingly difficult to navigate and understand. This complexity adds to faculty reticence to serve as SBP educators and assessors of resident knowledge and skills. In a study of orthopedic educators, faculty reported frustration with and poor understanding of SBP: “I never learned this. How can I teach it?”⁸ Despite their discomfort with this competency domain, observation during routine patient care is a method commonly used by faculty for SBP assessment.⁸

The framework for assessing SBP Milestone levels is lacking in specific tools and defined methods. Because competence in SBP requires multidimensional complex behaviors that are skills-based, performance-based assessment is of critical value. There is little published literature on SBP assessment, especially for subcompetency areas other than quality improvement (QI) and patient safety. Published assessment methods include 360-degree evaluations from peers, allied health staff, and patients^8,12,22,23; self-assessment¹¹; performance evaluations of the design, implementation, and analysis of QI projects^13,24,25; surveys¹¹; objective structured clinical examinations^26,27; simulated cases with examinations²⁸; web-based tools²⁹; direct observations with real time assessments and feedback³⁰; and participation in systems improvement.³¹ There is limited validity evidence to support the usefulness of these assessment methods; it is unclear whether these methods predict performance in this domain. Validated tools have not been developed for the assessment of physician behavior within the larger systems of health care.

Solutions From the Literature

Residents should routinely engage in systems thinking as it relates to their role in the various health care delivery settings and systems relevant to their specialty. However, this cannot be assured: Systems thinking must be formally taught.¹⁶ Knowledge of systems theory, tools, and techniques enables physicians to advance their understanding of system attributes and the environment within which each system exists, help identify system structures and improvement processes, and enable performance monitoring over time. Systems thinking can be viewed as a cognitive prerequisite for SBP behaviors, and there is likely a level of practical experience required before one can engage in true systems thinking behavior. One approach is to teach trainees how to ask relevant questions on systems improvement and change, starting them on the path to thinking and learning about SBP.

Beyond simple assessment of knowledge, demonstration of systems thinking is required. Ideally, there should be agreement across programs defining the essential knowledge, skills, and attitudes, and how achievement can be measured. Comprehensive assessment should be interprofessional, multimodal, and include direct observation by expert clinicians in the workplace. Assessment tools for systems thinking in medicine are rare; interprofessional health models and those that exist outside of the health professions may also be applicable.^32,33 Systems thinking is routinely applied by the aviation industry for safety and error reduction, natural resource allocation for land management, and event analysis by the nuclear power industry.^34–36

In 2018, in response to stakeholder dissatisfaction with the lack of consistency in Milestones across specialties in several competency domains, the ACGME published the results of a “competency crosswalk,” which identified common themes across all specialty Milestones.¹⁵ Using this information as a starting point, the ACGME brought together an interprofessional and interdisciplinary workgroup to develop a set of SBP Harmonized Milestones.³⁷ The Harmonized Milestones were based on “common and essential” themes and could be used by all specialties, creating a more uniform basis for assessment and faculty development, and potentially a foundation for curriculum development. The 3 core themes defined for the SBP Harmonized Milestones are patient safety and QI, system navigation for patient-centered care, and the physician role in health care systems.

Published research on teaching and assessing SBP in residency and fellowship training has primarily focused on the following: curriculum, assessment methods, mapping assessments to Milestones, feedback, program improvement, systems change, and clinical competency committee performance (list provided as online supplementary data). Several models of SBP teaching and assessment have been proposed, ranging from self-directed study to comprehensive multidisciplinary and interprofessional programs.¹⁴ Examples can be found in the published literature which programs can use or adapt to meet program needs (Table). There are published processes that can guide development of SBP assessment tools.^19,38 Online education should generally not be the sole method of teaching SBP, as it is an individualized and often isolated approach to learning a domain that is, at its core, team- and systems-based. Competency requires engagement with a broad variety of team members and administrative aspects of a large and diverse system. Experiential learning is critical; competence is achieved through active participation with the system itself.⁸

SBP group projects, including those that identify inefficiencies in the system and potential solutions, can strengthen residents' competency as members of the interdisciplinary and interprofessional health care team.^25,39 Simulation and real world experiential learning can provide opportunities to interact with non-physician providers and other members of the health care team leading to an increased appreciation of non-physician members of the health care system, awareness of care opportunities, and efficiency in various patient care tasks.^40,41

Faculty Development

There is a dearth of literature regarding faculty development via SBP knowledge, teaching, and assessment.^16,21 Faculty education on the basic tenets of systems thinking and recognition of SBP elements within daily practice is critically important. This will enhance faculty's ability to teach and assess SBP, pointing out systems interactions relevant to routine clinical care and providing guidance for residents' self-reflection on interactions with the broader health care system. Educators across disciplines and specialties can and should participate in faculty development opportunities and resident education in SBP.

Already Doing It—Recognize and Document

Systems-based practice underlies everything we do as medical providers. It is foundational to our daily activities and work. SBP is also intrinsic to residents' everyday activities but may not be appreciated. The list of SBP learning activities already present in residency training and physician practice is exhaustive. Examples include continuous QI programs, bedside cost/benefit patient management discussions, preauthorization processes, prescribing using the patient's insurance list of preferred prescriptions, multidisciplinary inpatient rounds, and root cause analyses. The educational efficacy of these activities may be enhanced by labeling them as SBP experiences, thereby increasing awareness, acceptance, sustainability, and dissemination.

Gaps and Future Needs

Systems-based practice is happening all around us; it is critical that educators make residents aware and incorporate SBP education into daily practice. Current evidence regarding teaching and assessment of SBP during training suggests that appropriate tools and methods have not yet been identified and disseminated.¹⁷ An educational framework that provides the tools for achieving competency in SBP is critical to success. This should include knowledge and skills in systems thinking, QI skills to implement change, and progressive opportunities to engage in systems change, starting at the clinical microsystem level (eg, in their residency program). To help embed systems thinking within residency curricula, the ACGME should consider including language related to systems thinking and the associated cognitive behaviors into the SBP competency language and SBP Milestones for all specialties.