Preceptor Development: Providing Effective Feedback, Part 2

Abstract

An integral part of providing effective feedback to pharmacy residents occurs during the evaluation process. Residency evaluation involves measuring and documenting performance as it relates to standardized residency outcomes, goals, and learning objectives. Evaluations may be formative or summative and include the preceptor’s evaluation of the resident’s performance, the resident’s self-assessments, and the resident’s evaluation of the preceptor and learning experience. Evaluations are more structured than feedback, and they involve documentation of the verbal feedback that was provided throughout the learning experience. This article will focus on the preceptor’s role in providing effective resident evaluations based on specific learning activities.

Many factors impact preceptors’ ability to provide quality feedback.¹ In verbal feedback and written evaluation, communication is critical. Effective communication is a key component to a successful pharmacy practice,² and much of the residency year involves training residents to develop communication proficiency in multiple practice situations. Because the intention in residency training is for learners to model the preceptors’ skills, attitudes, and behaviors, the way that preceptors communicate in evaluations is important in the residents’ overall development.

Along with communication, professionalism, role-modeling, coaching, and evaluation skills have been identified as key elements of effective pharmacy precepting.^3,4 Colleagues in medicine and nursing have also identified preceptor characteristics associated with proficiency in providing feedback.^5–7 These include completing preceptor training, showing empathy for learners, establishing mutually agreed upon learning goals, challenging learners, managing conflict, and documenting and reviewing professional goals annually. Imparting these skills to residents, including the skills needed for effective evaluation, is not only important for individual resident growth, but is also important for the profession’s growth as new and future residency graduates will need to be confident and competent preceptors to advance residency training.⁸

Challenges of Evaluation

All preceptors, including the residency program director (RPD), can evaluate residents more effectively by becoming more competent and confident with the systematic process. Other than lack of time, lack of confidence (or training) is often cited by preceptors when asked about their ability to be most effective at precepting.^9,10 In a survey of student preceptors, 73% of the respondents desired more formal training; those who had been trained possessed greater confidence and ability to communicate and evaluate.¹¹ These data emphasize the importance of effective preceptor training and development, particularly in regard to precepting residents who require longitudinal training and the evaluation of their developing skills, abilities, and attitudes.

American Society of Health-System Pharmacy (ASHP) accreditation data indicate that many preceptors are overwhelmed by the evaluation process¹² and find it extremely challenging. Throughout this article, we encourage readers to consider resident evaluation as one important piece of a large process.

The use of a more structured approach in residency training may be especially helpful, considering that the latest generation of residents (commonly referred to as “millenials”) may require and desire increased structure and coaching.^13–15 In a recent article on “Managing Underperformers,” the authors discussed the importance of creating a highly structured process in the training of new pharmacists to improve their performance and retention.¹⁶ The benefits included improved assimilation of new employees, departmental communication, and employee retention; the opportunity to reinforce expectations; and the means for acknowledging and recognizing employees for their commitment to training others.¹⁶

Training Structure

With the involvement of educational and pharmacy practice experts, ASHP has developed standards that include outcomes, goals, and objectives for postgraduate year 1 (PGY1) and postgraduate year 2 (PGY2) residencies.¹⁷ Residency outcomes provide a vision for programs and residents and vary depending on the type of residency. Outcomes and goals are broad in scope and describe actions that require consistent demonstration of progressive learning and professional development. A number of learning objectives support each goal statement. Objectives provide a more specific description of the intended performance, and programs utilize objectives to develop activities that make up a learning experience in the particular residency. Well-developed and well-defined learning activities provide preceptors with the means for directly observing and measuring resident performance and progress toward fulfilling the residency outcomes. Improving learning activities may also help facilitate the systematic process.

To develop a structure for training, programmatic decisions must be made on what the specific learning experiences will be and how they will be set up. According to ASHP accreditation standards for PGY1 pharmacy residency programs under principle 4, the residency program should focus on developing resident “training that will be designed, conducted, and evaluated using a systems-based approach.”¹⁷ Specific to evaluation, the program will create a “competency-based approach to evaluation of resident performance of the program’s educational goals and objectives, resident self-assessment of their performance, and of the program.”¹⁷ Finally, this strategy must be employed uniformly by all preceptors. The 3-part approach should contain the following¹⁷:

1. Summative assessment by the preceptor of the performance of each resident on the selected goals and objectives assigned to the learning experience based on the documented criteria.

2. Formative self-evaluation by the resident, and encouraged by the preceptor, of aspects of routine performance based on the criteria.

3. Completion of the evaluation of the preceptor and learning experience by the resident.

Learning Activities

Residency evaluation is a significant part of an effective training structure for the overall program. For the individual learning experience, preceptors must develop and plan specific activities that support the associated learning objectives. Through this process, preceptors are determining how the residents will be evaluated during the learning experience. Therefore, effective learning activities may be the most important part of the evaluation structure, and their development may be one of the most challenging tasks for preceptors.^12,18

Learning activities may be referred to as resident-specific duties and responsibilities; however, well-designed learning activities will also help communicate and facilitate understanding of the preceptor’s expectations of the residents during the particular learning experience.^5,7 Activities that are linked directly to goals and objectives provide residents with “what’s most important” without the sometimes technical jargon included in the goals and objectives. For the evaluation process to function properly, learning activities should be as specific as possible and measurable. If the objective is behavior-based, then the activity should be observable.

A basic premise of patient care goals and objectives is that they should provide the opportunity for the residents to eventually function as independent and highly competent patient care providers. When developing activities for a particular objective, preceptors should ask themselves what abilities, skills, and attitudes residents need to demonstrate prior to working independently with patients. In other words, what must preceptors observe or measure prior to resident autonomy?

Every preceptor has an idea of what the “basics” are, and these may differ depending on the priorities of the learning experience and the patient population being served. It may be a big challenge for preceptors to connect activities to the assigned learning objectives and then draft them into a learning experience description. Programs should consider that this may be a productive preceptor development activity. Appendix 1 outlines examples of learning experience activities.

Preceptors should consider additional factors in developing learning activities. For example, activities may need to be modified depending on the learner and the time of the year (eg, August vs March). Attainment of residency outcomes and generation of the residency product is a longitudinal, year-long effort. Many programs choose a “divide and conquer” approach when determining how to manage evaluation of goals and objectives, particularly those related to patient care. However, there should be routine assessment of the residents’ ability to perform what the preceptors would consider basic responsibilities.

Effective development and preceptor knowledge of the residents’ customized plan or modified plan are also critically important in this process.¹⁹ The residents’ plan should guide preceptors in choosing the goals and objectives on which to focus their coaching and evaluation time. Additional goals and objectives, skills, and abilities may be taught and learned during an experience, but preceptors should ensure that qualitative assessment and documentation occur in evaluating the areas of focus specified in the residents’ plan.

An important step in appropriate evaluation is a meeting between the preceptor and the resident to review rotation learning activities, goals, and objectives. When residents understand the required rotational activities and the preceptors’ expectations, they can reflect on their performance upon completion of the learning experience.

Assessment Process

Formative Precepting

Once learning activities are developed and communicated, they must be taught, observed, measured, and evaluated (as well as self-assessed by the resident). The assessment of individual activities during the experience is referred to as formative precepting or evaluating (taking a “snapshot” of resident performance). This is documentation of the day-to-day verbal coaching interaction between resident and preceptor. Even though preceptors may routinely provide verbal feedback, some documentation of formative skills assessment should occur during the experience. Formative skill assessment documentation:

• Focuses on the residents’ identified weaknesses (or helps the preceptor identify additional or future areas on which to focus work),

• Focuses on the skills that the residents need to demonstrate early in training,

• Provides plan-associated data for the summative evaluation, which occurs at the end of the experience, and

• Teaches residents about the value of self-assessment.

Self-assessment

The ability to accurately and adequately self-assess performance is the key to becoming a successful clinician, practice leader, and preceptor.²⁰ Self-assessment can be viewed as the development of a professional “conscience.” Not all incoming residents will possess this level of professionalism. Self-assessment skills often need to be learned and require ongoing development.

As with other skills, preceptors must provide effective role-modeling and coaching for effective self-assessment. As emphasized earlier, preceptors can help residents obtain these skills by developing learning activities that are clearly defined and are measurable and observable. Establishing an objective measure by which to self-assess will bring greater awareness to the process. This enables the residents to focus on specific targets (ideally, these would be identified and documented in the residents’ plan), exposes areas for improvement, and/or reinforces areas of strength and talent.

Documentation of the residents’ ability to self-assess is a required element of residency standards.¹⁷ The focus on learning activities (or targets for resident skill) will aid preceptors in qualitatively documenting residents’ progress. The preceptors’ commentary provides an example to the residents for how they should be documenting; therefore, as the residents learn, their self-assessments should improve. Similar assessments (whether in a formative or summative context) between preceptor and resident may show that the resident has not only progressed in his or her performance of an activity (and therefore the particular goal and objective), but is also growing toward mastery of self-assessment, indicating readiness for autonomous practice.

Summative Evaluation

On-going verbal feedback is valuable to resident growth. Qualitative documentation of verbal feedback is required by accreditation standards, and it establishes a record of progress (or lack of it), which provides direction to the residents and vital information for the RPD and future preceptors.

Summative evaluations, provided at or near the end of a learning experience, summarize how the residents performed (or are performing) based on the intended goals and objectives. The evaluation typically involves 2 steps. The first is the preceptors’ rating of the residents’ performance on learning objectives based on a standard rating scale (ie, satisfactory progress, achieved, or needs improvement). Interpretations of the rating scale may vary among preceptors; at a minimum, it may be helpful for preceptor groups to agree on the rating system to achieve greater consistency in the message being sent to residents in regard to progress. Residency programs should also determine which of the ratings require qualitative commentary from the preceptors.

The second step in the summative evaluation process involves preceptor comments. It is important for preceptors to remember that comments are not required for all objectives, nor should they be. With summative evaluations, quality is much more important than quantity, but some criteria-based commentary should be required. For example, areas that have been rated as “needs improvement” should require commentary. In this case, the preceptors should document how the residents could improve (based on a predefined “target”) or what action should be taken by the residents to help them get closer to “hitting the target.” In addition, the preceptors should comment on previously known areas for improvement that were noted in the residents’ plan as well as any newly identified weaknesses. The preceptors should also use the comments section to reinforce specific skills in which the residents excelled. In this way, even residents who are performing at a high level can be challenged to continue to demonstrate strong skills.

Effective commentary highlights another important facet of the evaluation process. The language that is used in documentation or the way words are used to describe performance and give qualitative feedback can make a significant difference. Through effective documentation (or commentary) on evaluations, preceptors should be able to answer several questions, which are listed in the box, “Questions Answered by Effective Preceptor Evaluation Comments.”

These thought questions can serve as a checklist (or self-assessment activity) for preceptors when they are documenting feedback on evaluations. RPDs may ask similar questions to stimulate discussion in preceptor development meetings that are focused on improving evaluation. Some programs conduct annual peer-review of evaluation comments in which questions such as these are used to guide the exercise.

Appendix 2 includes comments that preceptors commonly use on written evaluations with suggested improvements; a related suggested preceptor development exercise is provided in Appendix 3. Residency programs are encouraged to make this activity more program-specific by replacing these comments with evaluation comments that are used by their program’s preceptors. It may be helpful to include a date in the residency year when the comment was documented and reference the associated objective for which the comment was intended. Furthermore, residents should be involved in this type of exercise. It fulfills their desire to be a valued member of the team, and it is important for them to learn that effective evaluation involves ongoing practice and self-assessment. This will help them develop proficient skills as future preceptors (ie, this could be considered effective role-modeling).

Questions Answered by Effective Preceptor Evaluation Comments.

• What skills were focused on during the experience and how were these connected to the resident’s customized plan?

• In regard to these skills, what qualitative information should be documented (not “what was done” but “how it was done”)?

• How did the resident improve on known weaknesses? How can he/she continue to improve or what actions should be taken (eg, on the next rotation) to stimulate future improvement?

• What new areas for improvement were identified (if any)?

• How can observed strengths be reinforced?

• What skills should be the focus for future rotations and learning experiences?

• Can the documentation lead to future improvement in resident skill? Does the commentary (or the language used) direct future improvement in skills, attitudes, and abilities of the resident?

Effective Evaluations

Falcone²¹ offers several tips for providing effective evaluations:

1. Limit the surprises: Sharing concerns for the first time during an evaluation should be an exception, not the rule. Nothing is more demotivating for residents than finding out that their overall performance did not meet expectations when they thought they were doing fine. If the preceptor is faced with documenting issues in an evaluation, Falcone²¹ recommends acknowledging (in writing) that the matter has not been formally brought to the resident’s attention beforehand. This may help prevent or alleviate a difficult situation. For example, the preceptor could say, “I recognize that we haven’t formally discussed … however, I felt it appropriate to bring this issue to your attention during this evaluation because ….” This open and honest approach will make the preceptor’s intentions appear to be more objective and evenhanded.

2. Don’t give everyone an “A”: If preceptors are routinely giving residents and students the highest grade rating, they are probably doing something wrong. Preceptors who award As to everyone often have a problem distinguishing between genuine superior performance and overall good work. They may also be avoiding the conflict that comes with providing more down to earth, albeit realistic, performance scores.

3. Strengthen the overall message: Preceptors should avoid using the term “attitude.” Instead, they should describe the objective behaviors that create a negative perception of the resident. In addition, preceptors should use the phrase “for example” at least 3 times in the evaluation if at all possible. It is easy to make comments without documenting the facts that justify the points of view. Stating that a performance “needs improvement” is not the same as saying that “it does not meet my expectations” or it is “unsatisfactory.” When documenting future development goals and residency areas of improvement, preceptors can strengthen their message by applying the following: “I expect you to [perform a specific learning activity or meet a particular goal] by [required action].” Preceptors should make it a habit to clearly document and provide examples of times when performance was unacceptable, unsatisfactory, or failed to meet expectations.

Conclusion

Preceptors may provide feedback to residents through 2 primary methods: verbal interaction or written commentary (or documentation in online programs such as Resi-Trak). Most residency preceptors, many of whom have completed a residency, understand the challenges of ensuring effective communication within the evaluation process. From an accreditation perspective, the issues associated with resident evaluation are not unlike those faced by pharmacists in managing pharmacotherapy. These are ongoing challenges that require routine and focused management to produce optimal outcomes.

Preparing residents to meet training standards is not easy, and ASHP will continue to raise the bar in regard to accreditation to ensure that residency graduates are able to function effectively in a complex health care system. Preceptors must strive to improve their teaching abilities and their knowledge and competency with the training processes within which current and future residents must learn and progress to proficiency. The development of basic and measurable learning activities for each learning experience may be the most important part of the evaluation process, but it may also be the most challenging to preceptors. Thus, these activities should be emphasized in ongoing preceptor development plans.

Appendix 1.

Examples of Potential PGY1 Learning Activities²²

Outcome R2: Provide evidence-based, patient-centered medication therapy management with interdisciplinary teams.

Goal R2.1: As appropriate, establish collaborative professional relationships with members of the health care team.

Objective R2.1.1: (Synthesis) Implement strategy that effectively establishes cooperative, collaborative, communicative working relationships with members of interdisciplinary health care teams.

Potential activities:

1. Demonstrates reliability, availability to team; present and prepared for rounds; responds to team during off-hours (if appropriate or required)

2. Interacts with and engages team; knows team members by name; understands role of each

3. Demonstrates assertiveness on rounds (eg, asks questions or intervenes appropriately during rounds; consistently discusses drug-related issues with team; makes appropriate verbal/written interventions)

4. “Checks out” with team or covering pharmacist for evenings, weekends, and/or holidays

5. Follows up and follows through on patient and/or team-related tasks and projects

Goal R2.4: Collect and analyze patient information.

Objective R2.4.1: (Analysis) Collect and organize all patient-specific information needed by the pharmacist to prevent, detect, and resolve medication-related problems and to make appropriate evidence-based, patient-centered medication therapy recommendations as part of the interdisciplinary team.

Objective R2.4.3: (Analysis) Using an organized collection of patient-specific information, summarize patients’ health care needs.

Potential activities:

1. Uses an organized method for managing patient data

2. Demonstrates “triaging” ability

3. Demonstrates understanding of “pertinent” and identifies pertinent information in one, some, all patients on a service

4. Uses all available patient data collection methods, including patient and patient family, if available

5. Demonstrates understanding of patient data (eg, subjective versus objective)

6. Explains how medication therapeutics relate to specific patient’s needs (eg, Why is medication being used the most appropriate for the particular patient?)

7. Interviews patients; displays assessment skills; completes medication reconciliation forms, clarifies allergies, conducts medication histories

8. Consistently presents organized patient cases (for one, some, all patients on service) with identified patient medication issues and associated recommendations

9. Demonstrates effective use/assessment of evidence in medication-related recommendations

Author comments:

• Goals R2.1 and R2.4 are PGY1 pharmacy goals on which programs typically spend a lot of time.

• Note that the objectives for each goal (“Synthesis,” “Analysis”) are of higher level Bloom’s Learning Taxonomy, meaning that demonstration of mastery will require the resident to use a higher level of skill. Preceptor role-modeling and coaching may be most critical for this type of learning.

• It’s also important to note that some of the associated objectives seem to overlap. In these situations, basic and observable activities can support multiple objectives.

• Some activities (eg, resident reliability) may be a challenge to measure, but they can be observed. This emphasizes the importance of preceptor competency with goals and objectives, because resident progress and preceptor ability to document progress can hinge on the effectiveness of learning activities. It may be necessary for preceptors to define or model certain activities (eg, how appropriate “assertiveness” will be defined), specify what types of projects and/or tasks will be expected and to what quality, and/or provide helpful examples. In reviewing these learning activities with the resident, initial coaching might include determining resident understanding of an “organized” method for maintaining patient data. Examples may be helpful as residents will often want and need insight on how preceptors “do it.” Other helpful questions might include:

• What is “triaging?”

• What constitutes “pertinent” info?

• How can “pertinent” info be collected and presented in an organized way for one, some, and eventually all patients on the service? In other words, what is an “organized” system for maintaining/managing patient data?

Appendix 2.

Table A1. Productive versus nonproductive evaluation commentary.

Instead of…	Try…	Or…
Resident needs to do better	Resident identified most medication problems but often forgot to assess compliance. This should be a point of focus on future rotations.	How should/could/will you assess compliance in future patient interactions?
Resident is too quiet on rounds	It is important that the health care provider on the team be assertive and take the initiative to prevent problems and adequately address drug issues. How could you improve your skill in this area in the future?	My suggestion (to gain confidence with assertiveness) would be to start by addressing one issue per day, then build from this. What are some common issues you have noticed on rounds?
Resident needs to learn how to more effectively utilize the literature	Resident did well identifying problems in cardiology clinic but sometimes medication recommendation conflicted with current guidelines. Remember to always consult the most current evidence/practice guidelines prior to making future recommendations.	What do the latest published guidelines state about your medication recommendation in patients with systolic heart failure? What alternative therapies should be considered in these cases? Important to consider benefits and risks; encourage continued focus on this skill on future rotations.
Resident needs to improve time management	You seem to be struggling with multitasking. Make a list of your responsibilities and research some time management strategies, and we will discuss. Future preceptors may have additional insight; I encourage you to discuss with them.	Resident completed the weekend consults but left problem orders unresolved. What is needed so that this is prevented in the future? Have discussed with the resident and provided him with patient-focused strategies but recommend ongoing monitoring of this for now. Resident has been open to coaching on this skill to his credit.
Resident successfully completed this objective.	Resident made a lot of progress over the past month on designing and implementing therapeutic regimens. She is now incorporating the endpoints to therapy along with rationale for her recommendations. I would encourage her to continue to identify disease states in which she is less familiar and target them for additional review as this will enable her to more effectively recommend the most appropriate regimens.	The resident performed exceptionally with the team. I especially liked the passion with which she approached daily preparation for rounds. I observed this consistently and the attendings all commented positively that she was effective in anticipating problems. I would strongly encourage her to hold onto this skill and positive attitude. The team picks up on this and will immediately welcome you onto the team whatever the service.
Great job!	The resident consistently made effective interventions/recommendations to the team. For the adverse drug reaction identified earlier this month, please consider writing up a case report on this issue and submitting for publication or poster presentation. Please continue these efforts on future rotations. Why would it be important to contribute information such as this to our profession?	You engaged the students quite effectively in pharmacotherapy. Your cases were well-developed and your follow-up questions were well-conceived as they generated a lot of discussion. You provided effective coaching on thought processes and also remembered to provide plenty of time for the student teams to interact and reflect on their learning. Please aim for this level of detail and preparation for future classes.

Appendix 3. Preceptor Development Exercise: Using More Effective Written Evaluation Commentary

Please review (actual) comments below and rate as either “P” or “NP”:

Productive (P) comments provide information that could be used by resident to improve future performance in an area of professional practice (ie, comment is specific, qualitative).

Non-productive (NP) comments do not provide specific, qualitative insight on actions and/or future methods for improvement. (If rated “NP,” how would it become “P”?)

________Resident should continue to hone her team-building skills by being more assertive.

________Resident did well during this rotation; she completed all assignments.

________Resident has improved in her ability to summarize patient care needs but her assertions to the team could be more concise, organized, and to the point; she tends to ramble at times. This emphasizes the importance of preparation and determining good methods for maintaining patient information. I have encouraged her to continue to practice with this aspect. Future preceptors should monitor this skill closely.

________Resident does a good job prioritizing patient problems, but she should focus on effective “triaging” or “prioritization” of her sickest patients. The sickest patients will often have greater medication-related needs/problems. V—in what ways can you determine who is the “sickest” on your service?

________Resident needs to work on time management.

________Resident has improved immensely in working independently; however, there were too many distractions this month from interviews and meetings.

________Resident is very good at retrieving drug information and evaluating the literature.

________Consider what you would do if the SCr value drops (and renal function improves). At what level would adjustments need to be made and with which agents? What is the most effective way to make dosage adjustment recommendations?

________Resident recommended continued K+ supplementation.

________Resident was assigned to develop a treatment protocol for patients with health-care associated pneumonia. She did not have time to complete it.

________Resident makes patient care a priority in her daily work. She was almost always in the neonatal intensive care unit before me, reviewing charts and collecting patient data.

________To improve organization in order processing, resident should prioritize physician orders and focus on finishing one task prior to moving on to another so as to not miss potentially important information (leading to an error).

________Resident was generally prepared for rounds; however, M seemed to struggle on Mondays. Because census and patient loads change often over weekends, I would suggest he arrive 1 hour earlier on Mondays on future inpatient rotations to ensure all patients are reviewed effectively prior to rounds.

________Resident is improving in problem identification, but I would encourage him to be more patient-centered in his recommendations. What factors do we need to consider in improving adherence in patients with hypertension? What does literature say about this?

________Your presentation was very good; you appeared comfortable and confident.

________Resident did a great job collecting patient information prior to rounds and organizing the data so she can trend the information. She was also thorough with reviewing her patient’s current medications and has caught a couple of discrepancies related to medication reconciliation this month.

________Resident did not make appropriate recommendations during rounds.