Structured feedback: A teaching and learning tool for the postgraduate medical students in transfusion medicine

Abstract

CONTEXT:

Structured Feedback is a learning and assessment tool designed to provide feedback to students and educators to adjust learning and teaching during the training. Lack of provision of structured feedback to postgraduate (PG) medical students prompted us to plan a study to introduce a structured feedback module into the existing monthly assessment schedules in the Department of Transfusion Medicine.

AIM:

This study aims to introduce a structured feedback module and evaluate its efficacy after incorporation into the existing monthly assessment schedules for the PG students in the Department of Transfusion Medicine.

DESIGN AND SETTING:

A quasi-experimental study was commenced after obtaining clearance from the Institutional Ethics Committee in the Department of Transfusion Medicine for the students pursuing postgraduation in Transfusion Medicine.

METHODOLOGY:

A peer-validated feedback module was designed and implemented for MD students by the core team faculty. The students underwent the structured feedback sessions after each monthly assessment for of 3 months. One on one, verbal feedback was conducted using Pendleton's method, for monthly online assessment for the learning that happened during study period.

DATA COLLECTION AND STATISTICAL ANALYSIS:

The data were collected from the open-ended and closed-ended questions using Google form-based Student/Faculty perception and students’ pre-post self-efficacy questionnaires on 5-point Likert Scale and the quantitative data analysis was done using percentage of Likert scores, median values for each item for pre-and post-responses and comparison using nonparametric test – Wilcoxon signed-rank test. The qualitative data analysis was done using thematic analysis from the open ended questions.

RESULTS:

All (n = 9; 100%) the PG students strongly agreed and agreed (median score of 5 and 4) that the feedback they received made them aware of their learning gaps, enabled them in bridging those gaps and provided ample opportunity to interact with faculty. Both students and faculty agreed that the feedback session should be an ongoing and continuous process in the department.

CONCLUSION:

Both the students as well as faculty were satisfied with the implementation of the feedback module in the department. Students reported awareness about the learning gaps, identification of appropriate study resources, and ample opportunity to interact with faculty, after taking the feedback sessions. The faculty felt satisfied on the acquisition of new skill for delivering structured feedback to students.

Introduction

Feedback in the medical education context is a personalized information provided to a learner, based on direct observation and delivered in a way so that the learner can utilize the information to achieve their best potential. Feedback can reduce discrepancies between the current understandings and desired performance of a learner.[1] It is considered central in medical education for promoting learning and ensuring that standards are met.[2] The feedback aims at refining the clinical skills and knowledge of students, correcting what is done wrongly and reinforcing the task to be done correctly, and encouraging them in clinical skill development for better healthcare delivery.[3]

Structured feedback per se is a learning and assessment tool created to impart feedback to students as well as to the educators to adjust learning and teaching during the semester.[4] The close working relationship between the faculty and postgraduate (PG) students offers an excellent opportunity to share the assessment of learners’ strengths and weaknesses and help further develop their skills.[5] In the medical setting, the feedback helps the student develop accurate self-assessment skills and the lack of feedback may extend beyond self-improvement and ultimately can impact patient care. The students, if not provided with adequate feedback for their performance, may miss the chance to improve on and self-reflect.

There was an insufficient practice of provision of feedback to PGs in our department. Hence, a feedback module was developed to provide the desired information related to the concept of structured feedback to enable the stakeholders including faculty and students to get equipped for delivering and receiving feedback, respectively. The study was planned to introduce a structured feedback module to PG students in the Department of Transfusion Medicine and evaluate its effectiveness through feedback questionnaires.

Methodology

A prospective, quasi-experimental study was conducted in the Department of Transfusion Medicine at an academic tertiary care hospital in the Northern India as illustrated in Figure 1. The participants were nine PG students (Year 1, 2 and 3) pursuing MD Transfusion Medicine. The study duration was 1 year starting from February 2020 to January 2021. The ethical approval was obtained from Institutional Ethics Committee vide letter No.-xxxxx/IEC/20/209.

Figure 1.

Flow chart for study methodology

Written Informed consent form and Participant Information Document were shared with the participants as per the requirement of the Institutional Ethics Committee. Preparation of student and faculty feedback questionnaires were done after an extensive literature search and the same were validated by taking the opinion of 4 subject experts (internal) and 32 external experts including faculty and fellows in medical education (through an online platform-listserv). The suggestions so received were duly incorporated in the questionnaires before administration. The structured and validated feedback module was created and implemented for 9 MD students by 4 faculty of the core team. The PG students and faculty were sensitized on the feedback process and about conducting a one-to-one feedback session using the Pendleton's method through departmental meeting. The monthly topics were identified and the learning happened through online seminars as the study period corresponded to the COVID-19 pandemic times. The assessment for knowledge happened through administration of multiple-choice questions (MCQ) as clinical vignettes to students through an online assessment software application (Classmarker) each month. Three assessments each of 1 h consisting of 50 clinical vignette MCQs per month for the topics scheduled for respective months, were included for the feedback sessions as part of this study.

After receiving the assessment result, a detailed and verbal, one-to-one feedback session using Pendleton's Method was employed to conduct feedback sessions where the student and teacher, sequentially, described what went well, followed by areas of improvement.[5] A faculty checklist and a student matrix sheet were used during the session for recording the responses. Three sessions (June 2020 to August 2020) of one-to-one, verbal feedback based on monthly assessment were conducted, as per a fixed schedule with an opportunity of rotating the student-faculty to avoid any bias [Table 1].

Table 1.

Schedule of feedback sessions provided during the study period

Study period	A	B	C	D	E	F	G	H	I
Month I	2	1	1	2	3	3	4	2	4
Month II	1	2	2	3	4	1	2	4	3
Month III	3	4	3	1	2	4	3	1	2

Receivers (Postgraduate students) = A to I; Givers/Providers (Faculty) = 1,2,3 and 4. PG: Postgraduate

Following completion of the third feedback session, the students and faculty were administered Feedback questionnaires through Google forms, consisting of both close-ended and open-ended questions. Responses to retrospective pre-post self-perceived efficacy, students, and Faculty perception questionnaires were stored in Google spreadsheets. The perceptions were assessed quantitatively using 5-point Likert scale and percentage of scores for each response received for all the items and median values with Interquartile range of Likert scale scores. Satisfaction index (SI) for the item scores obtained from students’ and faculty perception questionnaire was calculated using the following formula:[6] ([n₁ × 1] + [n₂ × 2] + [n₃ × 3] + [n₄ × 4] + [n₅ × 5]) × 20/(n₁ + n₂ + n₃ + n₄ + n₅) where, “n” represents the total number of students attaining the score mentioned in the subscript.

Qualitative data analysis for open-ended questions from students and faculty perception for evaluation of the structured feedback sessions was done using thematic analysis. Several themes were generated from the open-ended questions.

Results

A total of 09 PG students in the department of transfusion medicine, participated in the study. Out of the total, five (55.5%) were male and four (44.5%) were female students. One student was pursuing 2^nd year of M. D. (11.2%) and 4 each were in first (44.4%) and third (44.4%) year of postgraduation residency program respectively.

The perception questionnaires consisting of twelve closed-ended items were administered to students and faculty and were evaluated with the help of 5-point Likert scale. All (100%) the PGs strongly agreed/agreed (median score of 5 and 4) that the feedback they received made them aware of their learning gaps and performance hindering factors as well as enabled them in bridging those gaps. The students strongly agreed (55.6%) or agreed (44.4%) that feedback sessions should be an ongoing and continuous process in the department [Figure 2a].

Figure 2.

(a) Students’ Perception about the feedback sessions as percentage likert scores. (b) Perceptions of faculty for structured feedback sessions

Among faculty, three (75%) agreed that they have become skilled in delivering the feedback to PG students. Half (n = 2) of the faculty agreed that structured feedback sessions were time consuming while 25% of total felt that incorporation of one-to-one sessions have put extra burden in their routine working schedule. All (100%) of the faculty members strongly agreed (median score of 5) that the feedback sessions should be an ongoing and continuous process in the department of transfusion medicine, should be taken up by other departments of the institute and should be incorporated in the transfusion medicine curriculum at other institutes [Figure 2b].

Median scores of knowledge and awareness were found to be higher for all the variables for postfeedback than prefeedback sessions, as perceived by the students [Table 2]. Perception for pre and post feedback was reported on the same set of assessment topics. The comparison was done using Wilcoxon Signed-ranks Test which indicated that there was statistically significant difference in self-assessed knowledge and awareness (P < 0.05) in 9 and 4 variables respectively, before and after participating in the feedback sessions.

Table 2.

Student’s prepost self-perceived efficacy

Knowledge-assessment topics	Before		After		Z	P
	Median	IQR	Median	IQR
Quality management system in blood transfusion services	2	1	3	4	−2.86	0.004*
NABH standards	3	2	4	3	−2.4	0.01*
Regulatory bodies for blood transfusion services	3	2	4	3	−1.69	0.09
Evidence based practice in transfusion medicine	2	2	4	3	−2.22	0.02*
Convalescent plasma	3	2	4	2	−1.62	0.1
Molecular testing for COVID-19	3	2	3	4	−2.53	0.01*
Bleeding disorder work-up	3	2	4	3	−2.34	0.01*
Regulatory requirements for set up of a blood center	3	2	4	3	−2.1	0.03*
Quality control of blood components	2	2	3	4	2.39	0.01*
Work up for alloimmunized patient	3	2.5	4	2	−2.74	0.006*
Iron absorption and metabolism	3	3	4	3	−1.86	0.06
Hemophilia-management	3	2	4	3	−2.41	0.01*
Patient blood management	3	2	4	3	−2.05	0.04*
Attributes-awareness
Awareness about your own learning gaps for the topics covered	3	2	4	2	−2.68	0.007*
Desire to bridge those learning gaps	3	2	4	5	−2.89	0.003*
Confidence level to attempt the questions during assessment	3	2	4	3	−1.69	0.09
Retention of knowledge of topics covered during assessment	2	2.5	4	3	−1.86	0.06
Continued motivation to study your subject	3	2	4	2	−2.38	0.01*

*P significant <0.05. IQR=Interquartile range, NABH=National Board for Hospitals & Healthcare Providers

As regards the Satisfaction Indices, before and after attending structured feedback sessions, the students’ SI ranged from 66.6 to 84.4 (after) versus 48.9–64.4 (before) [Figure 3a]. Among faculty, the SI for 9 items were between 80 and 100 while for remaining 3 items, it ranged from 55 to 70 [Figure 3b].

Figure 3.

(a) Student Satisfaction Index- Pre-post Self-Efficacy (b) Satisfaction Index of Faculty

The themes were generated from the two open-ended questions – (1) Mention two good things about the structured feedback sessions. (2) Kindly give two suggestions for improvement in conducting feedback sessions in future. Both faculty and students agreed on Identification of Learning gaps and clarity of scope of improvement and suggested including senior residents and provision of more time for content specific discussion [Table 3 and Figure 4].

Table 3.

Themes generated from the qualitative feedback by students

Open-ended questions	Themes	Corresponding verbatims
1. Mention two good things about the structured feedback sessions	Identification of learning gaps Instilling motivation to study Opportunity for introspection Identification of right study resources Better chances of revision possible Helps building clearer concepts Introspection of one’s performance Increased one to one student-faculty interaction Improvement in future performance	“Helps to identify the lacunae in the knowledge”
		“Get motivated to fill those gaps”
		“It helps to identify the weak areas in the topics studied and helps to understand how such deficiencies can be overcome and what proper resources to be used”
		“One to one interaction and guiding about the right material for good learning”
		“Make chance to revise”
		“Cleared concept, make me understand my knowledge gap” “Detailed and relevant”
		“One to one session with faculty, introspection of exam performance”
		“Interaction with faculty, understanding weak areas in the subject”
		“It helps to know my knowledge deficits”
		“The interactive session helps to perform better in future assessments”
2. Suggest two things about structured feedback sessions which require improvement	Provision of more time for discussion for high difficulty concepts Provision of viva question banks Provision of answer key through email/hard copy for MCQs assessment Approach to attempt practical cases	“Adequate time should be given to all students and common topics with queries by all students can be taken up by SR’s or faculty for better understanding”
		“Also, more time may be provided to the student to properly go through the Qs before the mentor sits with the student, in case gap between exam and feedback is more”
		“Provision of potential viva questions”
		“Provision of high yield topics from exam point of view”
		“It will be greatly beneficial if we can get answers in form of hard/soft copy, so that we can make our own notes for future tests”
		“Approach to practical and OSCE questions”

MCQs=Multiple-choice questions, OSCE=Objective structured clinical examination

Figure 4.

Thematic analysis of the responses generated from the faculty perception questionnaire

Discussion

The present study was conceptualized on the grounds of enabling the PG students to self-assess and seek self-improvement for better performance in future with the agents being department faculty. Monthly assessment is a regular feature for PG students in our institution. The scores obtained by the students are added to their internal assessment and has 50% weightage for final exit examination. The students, if not provided with feedback for their performance, miss the chance to improve on and self-reflect. A systematic review of 17 studies found that the self-assessments of practicing doctors and residents correlated poorly with external assessments and their level of training was not associated with better self-assessment accuracy.[7]

A study by Srinivasan et al. reinforces findings on self-assessment and states that without intervention and perspective, medical students self-reflect inaccurately.[8] Hence, the current intervention was planned in our department to incorporate structured feedback sessions in the ongoing monthly assessment schedules.

The literature says that effective learners are self-regulating and feedback is integral to develop and fortify the process that constitutes self-regulation such as goal setting, strategies to progress learning, task selection, and monitoring.[9] As reported by Aggarwal et al., a well-directed verbal feedback after each assessment can enhance confidence in a student to attempt a task and self-analyze. They also emphasized that one-to-one feedback may be even more helpful in eliciting the learning gaps and motivating more accurate self-analysis by weaker students.[10] They found a statistically significant difference in mean test scores (P = 0.000) between batches with written feedback compared to batches with verbal feedback in second, third, and fourth tests. Their findings corroborate with our study for better elicitation of learning gaps and motivation for future learning after one-on-one verbal feedback as observed through the students’ perceptions.

Wojcikowski and Kirk observed that students receiving detailed feedback after online assessment performed significantly better in the final biomedical exam than those who just received corrected answers only (mean score 74.80% vs. 70.22%; P = 0.011).[11] This is in accordance with our study where we found a statistically significant (P < 0.05) higher percentage of scores for perceived knowledge in 69.2% of the variables after delivering detailed feedback sessions.

In the present study, about 75% of the faculty reported acquiring new skill of delivering structured feedback during the study period. Jug et al. mentions that giving feedback is an art and the feedback conversations should be bidirectional and explicitly labeled. They also emphasized that feedback should be descriptive and based on direct observation.[5]

The ability to give and receive feedback is key for trainees as an integral component of their professionalism competency. The future roles of transfusion medicine residents may vary including teacher, learner, mentor, colleague, administrator, and director. Each such role may often seek, to assess and provide feedback to others in the laboratory environment.[12] Any such training generally is lacking during residency. Effective assessment is inclusive of an assessment of performance, self-reflection, and attainment of desired competency. Hence, provision of effective feedback is regarded as a critical teaching skill in medicine.

Feedback is considered time-consuming, more so when it is meant for students on a one-to-one basis after each assessment. Half of the faculty in our study reported that structured feedback sessions are time consuming. Furthermore, 25% of the faculty perceived that incorporation of one-to-one structured feedback sessions has put extra burden in their routine working schedule. Hence, it is imperative to take certain steps which can reduce the time duration taken to conduct sessions, as well as to make them manageable with the routine schedule. During regular feedback sessions with the same students, they can be taught self-assessment and analysis and hence effective learning strategies. Subsequently, the same students will become confident, start internalizing the feedback and attain improvement-seeking behavior. At this stage, the exhaustive task feedback may not be required as a routine after each assessment.[10]

To gain an in-depth understanding of the students’ perceptions, we used qualitative methodology and various themes were generated. The current study findings show that the PG students were satisfied with the implementation of structured feedback module and 100% of them are in agreement for its continuous use in the department. The students stated that incorporation of structured feedback sessions have enabled them to identify their learning gaps. Few of the verbatims as stated by the students were: “Cleared concept, make me understand my knowledge gap.” “It helps to know my knowledge deficits. “Helps to identify the lacunae in the knowledge. Get motivated to fill those gaps.” “It helps to identify the weak areas in the topics studied and helps to understand how such deficiencies can be overcome and what proper resources to be used.” ”Interaction with faculty, understanding weak areas in the subject” [Table 3]. The excerpts from the participating students clearly indicate that students value feedback conversations with faculty. The perspectives of the participants align with the findings of research by Price et al. who found that students need opportunities to discuss the feedback and ask questions.[13]

The thematic analysis of the faculty perspectives highlighted that the implementation of a structured feedback module is a step forward in the learning curve of PG students. During their residency program in the department, constructive feedback can enhance their learning by acknowledging the learning gaps. The interpersonal relationship between faculty and student also has a bearing on the future success of the module, as emphasized by Ramani and Krackov ensuring that a committed and caring interpersonal relationship is of paramount importance for effective feedback to take place.[14] As per Watling and Ginsburg the successful blend of assessment and feedback demands clarity of purpose, support for learners, and a system and organizational commitment to a culture of improvement rather than a culture of performance.[15]

A timely, formal, credible, specific, and detailed feedback on performance is crucial for competency-based and outcome-based medical education. Our study had a limitation of small sample size and hence the findings cannot be generalized. COVID-19 outbreak was another setback during the study period as the feedback could only be provided for the knowledge domain and the psychomotor skill could not be addressed due to physical distancing in the peak pandemic. Reassessment of the monthly test schedule, after the administration of structured feedback to the students, could not be considered during the study period while only self-perceived efficacy of their knowledge and awareness, pre and post feedback session, was undertaken.

Long-term use of the structured feedback sessions in departments running MD (postgraduation) in Transfusion Medicine in India, may contribute to successful implementation of enhanced feedback culture resulting in effective assessments. This might eventually result in promotion of professional development among PG student and faculty for better healthcare delivery.

Ensuring adequate feedback for the PG students should be an important consideration for curriculum development or revision.[16] The faculty, working with their PG students for the entire stretch of residency, get to know their learners through one-to-one interaction. Introduction and incorporation of structured feedback in the teaching schedule of PG students for the formative assessment, can improve their knowledge. A planned reassessment of the topics not performed well by the student, can be reinforced to the students after provision of structured feedback. Re-assessment on the same concept can further motivate the learner for a time bound learning for better performance in future. Self-assessment through regular and formal structured feedback will motivate students to introspect and seek improvements and become life-long learners thereby contributing to enhancement in better patient care. Hence, there is a need of incorporation of continual, structured feedback sessions into existing assessment schedules for the PG students.

Conclusion

The study highlights the utility of structured feedback module among PGs in the department of Transfusion Medicine. Both the students as well as the faculty were satisfied with the implementation of the feedback module in the department. Students reported awareness about the learning gaps as well as motivation to bridge the same, identification of appropriate study resources and ample opportunity to interact with faculty, after taking the feedback sessions. Faculty felt satisfied on acquisition of new skill for delivering structured feedback to students. This study augments the learning opportunities of PG students by implementation of structured feedback module in their teaching schedule wherein they identify their learning gaps by self-reflection. The faculty identify their strengths and weaknesses and work in tandem for future improvements.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.