Assessing for Practice-Based Learning and Improvement: Distinguishing Evidence-Based Practice From Reflective Learning

Practice-based learning and improvement (PBLI) Milestones focus on 2 themes: evidence-based and informed practice (PBLI-1) and reflective practice and commitment to personal growth (PBLI-2).¹ The Harmonized Milestones 2.0 were developed with an understanding that graduate medical education programs need to have methods to assess trainees' development in these complex areas. The conceptual framework of self-directed lifelong learning is a key aspect of medical professionalism and an important skill to develop to maintain proficiency in the ever-advancing field of medicine.^2,3 The American Board of Medical Specialties values meaningful participation in PBLI that includes aspirational continuing learning expectations.^4,5This article intends to provide guidance on assessing PBLI Milestones and provide resources for trainee development.

PBLI-1: Evidence-Based and Informed Practice

The subcompetency of evidence-based medicine and informed practice highlights the principles of evidence-based medicine, with a focus on integrating the best available evidence into clinical decisions, informed by patient values, and clinical expertise (Figure 1).¹ All clinicians need these skills as a foundational component to lifelong learning. Several articles examine how to assess the use of evidence-based medicine in a variety of trainee types (Table 1). Some authors have conducted a needs assessment or summarized the available tools.^6,7 The majority of the tools focus on multiple-choice and/or short-answer tests to evaluate the learner's knowledge of the tenets of evidence-based medicine.^8–12 The most studied of these tests is the UCSF-Fresno Medical Education tool, a 7-question written test evaluating how to ask a clinical question, assess the hierarchy of evidence, and understand basic statistical and methodological concepts.^9,10 The Fresno tool has been validated in several populations of learners and could be used to assess if a trainee has met Level 2 for the PBLI-1 Milestones.

Figure 1.

Harmonized Evidence-Based Medicine and Informed Practice Milestones

Table 1.

Summary of Literature to Assess Learners in Evidence-Based Medicine

Author(s), (y)	Target Audience	Assessment
Bhutiani et al (2016)8	Third-year medical students	Objective structured clinical examination
Bougie et al (2015)9	Obstetrics and gynecology residents in all programs in Canada	Self-assessment; standardized written questions
Epling et al (2018)6	Family medicine program directors in all programs in the United States	Program director's needs assessment
Haspel (2010)13	Transfusion medicine residents in a university-based program	Journal club curriculum
Lentscher and Batig (2017)14	Obstetrics and gynecology residents in a military program	Structured journal club
Patell et al (2020)10	Internal medicine residents in both university- and community-based programs	Multiple-choice evidence-based medicine test
Smith et al (2018)11	Third-year medical students	Fresno evidence-based medicine test
So et al (2019)15	Foot/ankle residents in 2 community-based programs	Structured review instrument for journal club
Thomas and Kreptul (2015)7	Family medicine residents	Meta-analysis of available tools
Tilson (2010)12	Physical therapy doctorate students	Validation of Fresno test

Fewer published approaches evaluate the day-to-day clinical application of evidence-based medicine and informed practice. Structured journal clubs are one approach; however, this tactic is retrospective and does not assess an individual trainee's ability to apply findings to a specific patient's needs.^13–15 While direct observation and chart-stimulated recall have not been studied, they are potential approaches to assess if a trainee has met Level 3 or 4 for this Milestone. Level 5 can be assessed based on a portfolio of evidence-based guidelines that the trainee creates or documentation of mentoring others in evidence-based medicine.

PBLI-2: Reflective Practice and Commitment to Personal Growth

An important goal of medical educators is to foster the development of reflective lifelong learners.³ The subcompetency “Harmonized Reflective Practice and Commitment to Personal Growth” acknowledges this critical skill and uses 3 streams of behavior to differentiate seeking feedback, addressing gaps, and intentionally developing a learning plan (Figure 2).¹ While assessment tools that provide formative and summative assessment of this complex framework are lacking, a number of studies described best practice characteristics and implementation strategies to utilize learning plans.

Figure 2.

Harmonized Reflective Practice and Commitment to Personal Growth Milestones

Individualized Learning Plans

Physicians do not always effectively incorporate an individualized learning plan (ILP) and learning goals into their daily work. Lack of time and understanding of the skills needed to be a self-directed learner are barriers to self-directed learning and use of ILPs.^16–19 Residents can struggle with identifying specific goals and formulating an effective plan to achieve them.¹⁸ Literature exists regarding how to best teach this skill and how to make the process meaningful to the learner.^20–22 However, the mechanism to most effectively develop self-directed lifelong learning skills is not fully known at this point. Previous research indicates individual characteristics are more strongly associated with self-directed learning than program characteristics.²³ A conceptual model for self-directed learning based on the I-SMART mnemonic (Important, Specific, Measurable, Accountable, Realistic, Timeline) emphasizes specific aspects of the goal itself that lead to success.²⁴ There is initial evidence that the type of learning goal identified by residents is associated with success in achieving that goal.^25,26 However, it is suggested that a supportive learning environment is crucial.²⁷ Residents identified key aspects of the learning environment in the longitudinal block that facilitated their success: (1) flexibility to closely align learning goals with clinical activities; (2) adequate time to work on goals; and (3) faculty oversight and support of their learning goals.^24,26

Performance Dashboards

Performance dashboards can display a trainee's progress along the Milestones and identify areas of concern and strength. Multiple elements can be pulled from the electronic health record, learning management systems, and administrative data to create performance dashboards (Table 2). These dashboards have been developed for multiple specialties to track progress, provide real-time feedback, and document operative autonomy for surgical cases.^28–30 Using performance data can help assess if the trainee achieves Level 2 or 3 of the first PBLI-2 Milestones stream. Performance data, if on a dashboard or as individual elements, is necessary for reflective practice, but by itself is insufficient. Data alone does not drive improvement. Performance data create transparency about expectations and actual performance, which should lead to analysis, reflection, and ultimately improvement. There are limitations to performance dashboards. Using performance metrics compared to standard feedback showed emergency medicine residents had improved satisfaction with the feedback process when the performance metrics were included, but there was no change in clinical productivity or efficiency.³¹ Performance metrics may not be easily attainable or reflect outcomes that are under the trainee's control. For example, the emergency department metric of “left without being seen” has a number of causes, many of which are unrelated to trainee performance. Using dashboard metrics that will be tracked when the trainee is in autonomous practice may make the data seem more applicable and better prepare them for the future.

Table 2.

Sources of Trainee Data for Performance Dashboards

Clinical Data	Educational Data
Chart audits	Online module completion
Procedure/case logs	Direct observation evaluations
Medical record completeness and deficiencies	Attendance data
Case volumes; appointment volumes	Scholarly output
Quality/safety indicators (readmission rates, complication rates)	Rotation evaluations
Patient evaluations/patient experience scores	Semiannual program evaluations

Assessing PBLI Along the Developmental Continuum

The PBLI Milestones echo qualities of reflection and insight that are fundamental to self-directed lifelong learning, and to clinical practice at any stage of professional development. Trainees who do not incorporate constructive feedback into practice can be frustrating to program directors. One way to help a trainee become a self-directed learner is to work with them to develop and implement an ILP with supervision and assistance from a faculty coach. The faculty can act as guide, resource, and expert to provide feedback and course-correction.²¹ The ILP can be used as a tool—a litmus test—for the CCC to gauge the trainee's insight.

Eva and Regehr drew the distinction between self-assessment (ability), assessment and reflection (pedagogical strategies), and self-monitoring (immediate and contextual responses).³² When given this assigned task, some trainees will refuse (“I don't need to do this” or “I don't have the time to do this”), some will deflect (“I'm being treated unfairly”), and some will concede (“I can't do this”). These responses help gauge the trainee's capability of improving and ability to achieve competence in PBLI. Trainees who are incapable of effectively demonstrating insight are unlikely to achieve the academic standards of the program and to be self-directed lifelong learners. The CCC can use this information to inform their recommendations to the program director.

Conclusions

Use of evidence-based medicine, ILPs, and performance dashboards help clinicians apply evidence to patient care, recognize areas of improvement, and identify when gaps are closed. Programs must evaluate the feasibility of implementing PBLI assessment tools and the impact of the clinical learning and working environment on PBLI.³³ Continued research is needed to develop and test assessment strategies for PBLI to create a robust set of tools.