Pharmacy Students’ Views of Faculty Feedback on Academic Performance

Abstract

Objective. To investigate students' views on and satisfaction with faculty feedback on their academic performance.

Methods. A 41-item survey instrument was developed based on a literature review relating to effective feedback. All pharmacy undergraduate students were invited via e-mail to complete the self-administered electronic questionnaire relating to their views on feedback, including faculty feedback received to date regarding their academic performance.

Results. A response rate of 61% (343/561) was obtained. Only 32.3% of students (107/331) agreed that they were satisfied with the feedback they received; dissatisfaction with examination feedback was particularly high. The provision of faculty feedback was perceived to be variable in terms of quality and quantity.

Conclusions. There are some inconsistencies relating to provision of feedback within the MPharm degree program at Queen's University Belfast. Further work is needed to close the gap between student expectations and the faculty's delivery of feedback on academic performance.

INTRODUCTION

Feedback has been defined as “information about how successfully something has been or is being done.”¹ Within education, feedback is considered important for performance improvement and is widely accepted as a fundamental aspect of teaching and learning in any environment.^1,2 A review of over 250 studies found that feedback had positive effects on learning and achievement.³ Indeed, learners who receive a response indicating whether or not their work is correct are more likely to maintain interest in the subject being taught.⁴

Some researchers advocate feedback within medical education for developing effective communication skills.⁵ Others consider that if feedback is not provided within healthcare disciplines “mistakes go uncorrected, good performance is not reinforced and clinical competence is achieved empirically or not at all.”⁶ A systematic literature review found that feedback alone had a positive influence on doctor performance in 74% (32) of the included studies.⁷ Furthermore, a Cochrane systematic review (n=118 studies) investigating the effects of audit and feedback on the practice of healthcare professionals and patient outcomes concluded that they can be effective in improving professional practice. The authors cautioned that the effects are generally small to moderate and effectiveness is likely to be greater when baseline adherence to recommended practice is low and when feedback is delivered more intensively.⁸ With specific reference to pharmacy, Watson and colleagues conducted a randomized controlled trial in community pharmacies and found that using simulated patients and giving immediate feedback was an acceptable way of improving the quality of consultations for nonprescription medicines.⁹

Despite its recognized importance, many pharmacy students are not satisfied with the quality and quantity of feedback they receive from their teachers.^10,11 Student dissatisfaction with feedback is a well-documented problem that is not unique to pharmacy.^12,13 In National Student Surveys (NSS), students have consistently rated questions on assessment and feedback lowest of all categories, with Northern Ireland students’ ratings significantly lower than those of students in other parts of the United Kingdom.¹² As the NSS results are published to help UK students make an informed choice when deciding where to study, they are held with high regard by universities.

Providing effective feedback to students is a complex process with no single correct approach. Moreover, students may not understand the feedback that is given¹⁴ and if it does not require active engagement by the students, it is less likely to have an impact on their learning.¹⁵ Effective feedback is constructive, explicit, and non-judgmental.¹⁶ However, as Sadler pointed out, “…even when teachers provide students with valid and reliable judgments about the quality of their work, improvement does not necessarily follow. Students often show little or no…development despite regular, accurate feedback.” ¹

There has been limited research conducted on pharmacy students’ opinions on faculty feedback. The students in this study routinely received feedback on their performance in formative aspects of assessment, such as assignments and other coursework, whereas limited feedback was provided on summative aspects, such as examinations. The objectives of this study were to investigate students' views of, expectations for, and satisfaction with feedback on their academic performance in all modules they had completed to date.

METHODS

A questionnaire was developed based on findings from a review of the literature relating to effective feedback.^6,17-19 Five validated questions regarding assessment and feedback from the NSS (ie, developed by the Higher Education Funding Council for England) also were incorporated.²⁰ The questionnaire consisted of 6 sections covering questions (largely closed, with pre-formulated answers) on the following areas: perception of feedback; usefulness of feedback; examination feedback; overall impression and influence of feedback; type, quality, and timing of feedback; and demographic data (but no identifiable information). Most questions (32) contained attitudinal statements followed by responses given on a 5-point Likert scale (“strongly agree” to “strongly disagree”). Additionally, 1 question (4 parts) used a scoring system of 1 to 5, where 1 equaled “most useful” and 5 equaled “least useful” and 1 question (5 parts) used a scoring system of 1 to 5, where 1 equaled “most important” and 5 equaled “least important.”

The School of Pharmacy Ethics Committee reviewed and approved the proposed research. The questionnaire was internally reviewed for content validity by an expert in the field and assessed for face validity by colleagues. Undergraduate students (n=10) from various degree programs (ie, students not enrolled in the MPharm degree program) pilot tested the questionnaire, and as a result, minor changes were made (eg, the estimated completion time was increased from 5 to 8 minutes).

The study population was students enrolled in the 4-year MPharm degree program at Queen's University Belfast (n=561). SurveyGizmo (www.surveygizmo.com; Boulder, Colorado) was used to create an electronic version of the questionnaire. In February 2011, all MPharm students were invited by e-mail to participate in the study. The e-mail contained a unique link to the questionnaire which allowed each student to complete the questionnaire once only. Students were given 14 days from the time the invitation was sent to complete the questionnaire with 2 follow-up e-mails sent to nonrespondents after 7 days and again after 12 days. Several methods were used to maximize the survey response rate including highlighting the deadline in the invitation e-mail, stating in follow-up e-mails that other students had already responded, ensuring the questionnaire was relatively short, and designing the questionnaire with a simple header and a white background.²¹

Responses were coded and entered into SPSS for Windows, version 18 (SPSS, Inc, Chicago, IL). As the data were non-normally distributed and were ordinal or nominal in nature, non-parametric tests (chi-square, Mann Whitney U test, and Kruskall-Wallis) were used to test for association between responses. Subanalyses were performed by gender and by program level. An a priori level of less than 0.05 (p < 0.05) was set as significant and differences meeting this criterion were reported. Missing data were not estimated or used in analyses. The free response questions were analyzed independently by 2 of the researchers using thematic analysis.²²

RESULTS

A response rate of 61% (343/561) was obtained; 70.6% (242/343) of the respondents were female and 29.4% (101/343) were male. This gender ratio was similar to that of the entire population of students enrolled in the pharmacy degree program (67.4% [378/561] female; 32.6% [183/561] male). Out of all respondents who completed the questionnaire, 24.2% (83/343) were first-year students; 23.6% (81/343) were second-year students; 21.9% (75/343) were third-year students and 30.3% (104/343) were fourth-year students. The proportion of respondents from each year of the program differed significantly, with participants more likely to be fourth-year students than any other year (P = 0.002). While there were 343 respondents, the number of respondents who answered each question varied.

Less than a third of the 343 student respondents (32.3%) strongly agreed or agreed that they were satisfied with the feedback they received. Third-year students (n = 71) appeared to be least satisfied (21.1% strongly agreed or agreed) whereas fourth-year students (n = 102) were most satisfied (39.2% strongly agreed or agreed). Almost a third of respondents (31.1%) strongly agreed or agreed that they had been made aware of what to expect regarding feedback throughout the program. Even fewer respondents (9.8%) strongly agreed or agreed that they were normally informed of the class average and range in assessed work. Additionally, when asked if the criteria used in grading was clarified in advance, only 32.0% of the students strongly agreed or agreed with this statement.

Students were invited to provide general comments, via an open response question, about the feedback they had received during the degree program. Students’ comments centered on inconsistencies in provision of feedback; other remarks related to the quantity and quality of feedback received (particularly in relation to examinations) and the timing of feedback.

Most of the 343 student respondents (98.0%) strongly agreed or agreed that receiving feedback was an important part of their degree program. Indeed, 91.8% strongly agreed or agreed that feedback should tell them how good their work is. The majority of respondents (99.4%) strongly agreed or agreed that feedback should tell them how to improve performance. Furthermore, most students (80.1%) strongly agreed or agreed that the feedback they received helped to improve performance and 72.2% strongly agreed or agreed that they performed best in the modules (particular components of the pharmacy degree program) where they received the most feedback. Most (87.4%) strongly agreed or agreed that they should receive feedback on every piece of completed work with over 90% strongly agreeing or agreeing that feedback was most useful in areas where they performed badly. There was less agreement surrounding the amount of feedback obtained in each module with 68.7% strongly agreeing or agreeing that feedback should be of a similar quantity in every module. Many students thought that the reason why they received a particular grade was more important than the grade itself; 71.6% strongly agreed or agreed with this statement. Furthermore, students in lower levels were more likely to strongly agree or agree with this statement than students in higher levels (78.3% of first-year students strongly agreed or agreed compared with 65.4% of fourth-year students; p = 0.020). Over 60% (62.7%) of all respondents strongly agreed or agreed that feedback helped clarify things they did not understand.

Students were asked if they considered grades for submitted work, verbal, and written comments as types of feedback (Figure 1). There was almost unanimous agreement across the 4 groups that written comments were a type of feedback. However, there were significant differences across the 4 years about grades for submitted work (P = 0.003) and verbal comments (P = 0.002), as indicated by an asterisk (Figure 1).

Figure 1.

What pharmacy students consider to be methods of feedback.

When asked about individualized versus generic feedback, the majority of respondents (91.0%) strongly agreed or agreed that individual feedback was of more benefit to learning than generic feedback (general feedback provided to the class). Male students were more likely to strongly agree with this statement than female students (68.3% of male students strongly agreed compared with 47.9% of female students; p = 0.002). However, several open responses provided by students indicated that they welcomed a combination of both individual and generic feedback. Only 33.6% of respondents strongly agreed or agreed that they had received sufficient personalized feedback. The majority of students (78.7%) strongly agreed or agreed that it was useful to evaluate your own performance using the class average mark and range. About three-quarters of respondents (76.1%) thought that most of the feedback they had received had been mixed (ie, both positive and negative). Similarly, 73.4% strongly agreed or agreed that they had received more written than verbal feedback. Two-thirds (66.1%) strongly agreed or agreed that they normally did not receive any justification for why they received a particular grade. Most students (97.6%) strongly agreed or agreed that receiving model answers would be beneficial to learning.

When asked specifically about examination feedback, 30.3% strongly agreed or agreed that a grade was sufficient to assess their performance on examinations. Furthermore, 72.4% strongly agreed or agreed that receiving generic examination feedback would improve their performance. Female students were more likely to strongly agree or agree with this statement than male students (78.3% vs. 58.1% males; p < 0.001). Only 14.7% (49/333) of all the respondents strongly agreed or agreed that examination feedback had been satisfactory.

Many students (71.1%) strongly agreed or agreed that feedback was only useful when received promptly. More than half of the students (56.6%) strongly agreed or agreed that feedback on their work had been provided promptly. About 40% of the respondents (41.9%) strongly agreed or agreed that feedback was given in sufficient time to allow suggested changes to be implemented in the next piece of related work. The students were also asked to rate the value of feedback being given during the class, at the end of the class, at the start of the next class, and at the end of the module (Table 1). This question used a scoring system of 1 to 5, where 1 equals most useful and 5 equals least useful. There was a significant difference in opinion between the year groups in relation to feedback being given during the class (p = 0.010) (Table 1).

Table 1.

Pharmacy Students' Views of the Usefulness of Feedback Given at Various Times (n=343)

Students were asked to consider various aspects of feedback (promptness, tailored to the individual, how easy it is to understand, how detailed, how focused) and rate them in terms of their importance on a scale of 1 to 5 on which 1 represented the most important and 5 represented the least important (Table 2). The variable “How easy the feedback is to understand” received the top mean score of 1.5, whereas “How focused the feedback is” received the lowest mean score of 2.2. No significant differences were found in the responses of male and female students or between students in different years of the program.

Table 2.

Student Ratings of the Importance of Various Aspects of Feedback (N=333)

Students were asked which module they considered to be the best in terms of feedback and to explain their rationale. The most frequently reported modules were practice-based, namely, Extemporaneous Dispensing (second year), Proprietary Dispensing (third year), and Responding to Symptoms (fourth year). Various reasons were reported including ease of understanding, timing of feedback, a mix of both written and verbal feedback, both positive and negative feedback, and feedback that outlined how to improve performance.

DISCUSSION

Student dissatisfaction with feedback is an almost insurmountable problem.^11,13 Less than a third of the students in this study were satisfied with the general feedback they had received from teachers, with even fewer students satisfied with examination feedback. These findings echo work carried out by Brown,²³ who reported a general discontentment with examination feedback and Archer,²⁴ who found that feedback is seldom given for high-stakes written examinations. The difference in satisfaction with general feedback was marked between third-year and fourth-year students, but may be related to the publication of poor grades in a third-year module examination prior to the questionnaire study.

Almost a third of respondents agreed or strongly agreed that they were aware of what to expect regarding feedback. This indicates that the majority of students are generally unaware of when feedback will be given and the form it will take. A similar picture emerged when views on assessment were sought, with only 32% of students strongly agreeing or agreeing that grading criteria had been made clear in advance. This appears to support Yorke's concerns that assessment criteria for complex tasks are normally not sufficient to convey all their meaning.² If students are unsure as to what is required of them, they will be less able to align their performance to that expected of them by teaching staff. A number of improvements could be made, from using better definitions of requirements to providing examples of work with feedback attached.¹⁸

Almost all students (98%) strongly agreed or agreed that receiving feedback was an important part of their degree program. Indeed, there are many studies in the literature with similar findings.^25-27 Students mostly (91.8%) agreed or strongly agreed that feedback should tell them how good their work is, implying that they saw feedback as a judgment of how their work compared to some preset standard. There was a high level of agreement that feedback should tell students how to improve performance and that it was most useful in areas where they had performed badly. Hattie and Timperley¹⁹ state that feedback given to improve performance can have a powerful influence on learning. Additionally, 62.7% of respondents strongly agreed or agreed that feedback had helped to clarify understanding. Improving performance and correcting misunderstandings are crucial for students within any medicine-related discipline as they impact both future quality of care and patient safety.

Students’ expectations were found to be largely in line with what the literature considers to be good feedback practice.¹⁸ Most (87.4%) expected to receive feedback on all work, which may imply that students see providing feedback as the responsibility of the teaching staff and see their responsibility ending with the submission of their work. Students have high expectations with respect to the quantity of feedback they receive – expectations that academic staff members may not share nor be able to meet. Interestingly, the majority of students (71.6%) thought that the reason for their grade was more important than the grade itself and rated “how easy the feedback is to understand” as more important than other aspects. These results suggest that students are positively engaged in the feedback process and may indicate that most students’ approach to interpreting grades is learning oriented.

Students preferred that feedback relate specifically to their work and not to that of the group, although they thought that group feedback could still be helpful. Indeed, 72.4% were in agreement that generic examination feedback would help improve their performance, whereas 91% strongly agreed or agreed that individual feedback was more beneficial to their learning than generic (class) feedback. This mirrors research by Perera²⁸ who found that students wanted more individualized feedback. While the level of detail also was deemed by students to be an important consideration, it can be a 2-edged sword. Enough detail must be provided to address performance issues and show how improvement can be made; however, if too much information is provided, some will be disregarded.¹⁶ From a pharmacy practice context, this means prioritizing feedback that relates to patient safety issues and dealing with other issues later.

Over 70% of students were in agreement that feedback is only of use if it is received promptly, with less than 40% strongly agreeing or agreeing that feedback had been given in enough time to allow changes to be implemented before the next piece of related work was presented. Indeed, there was a significant difference (p < 0.05) between the ratings achieved for feedback delivered during the module and that given at the end of the module, indicating that students felt this to be summative and of less use to their learning. This view is backed up by a plethora of literature sources, which state that feedback must be “timely,” ie, given when it is needed and therefore able to influence future performance.^25,29

There was general consensus among students that written comments were considered a feedback method. However, when the responses for whether verbal feedback and receiving grades for submitted work were analyzed, different views emerged. While most first-year students (85.5%) considered grades to be a form of feedback, this dropped to 64.1% among fourth-year students. Conversely, 69.9% of first-year students considered verbal comments a method of feedback compared with 91.1% of fourth-year students. These results can be explained in light of the program structure. First-year students are mainly exposed to foundational aspects of pharmaceutical science, which traditionally uses more summative feedback in the form of grades. Fourth-year students, on the other hand, have completed more practice modules where verbal feedback is commonly given before, during, and after class. Therefore, fourth-year students may have higher expectations regarding what constitutes feedback.

Pharmacy practice modules appear to be best suited to fulfill the feedback expectations of the students in this study and, perhaps even raise student expectations of what feedback should be provided in other modules. Feedback in modules is given through weekly face-to-face interaction in formats ranging from individual role-plays to class-level discourse. After this verbal feedback has been provided, grades for work are given along with detailed written comments. This approach is supported by Butler,³⁰ who found that students who received grades were less attentive to comments, which lessened the impact of the feedback on future performance. These modules are taught predominantly by pharmacist teaching fellows, who have limited research responsibilities. Yorke noted that various pressures, such as faculty involvement in research and high student to staff ratios, make the provision of formative assessment difficult.²

In terms of limitations to this study, student opinions were captured at one point in time, and due to time constraints, data were collected during the second semester, which did not allow students to assess feedback provision for the full academic year. This may have been more of an issue for first-year students who had only completed 2 full modules at the time of the questionnaire. Also, there was an overrepresentation of fourth-year students (final year students) in the study population. The fourth-year students may have been more interested in the topic as they had the most exposure to teaching and feedback.

The response rate (61%) of the student population (n=561) was reasonable³¹; however, the possibility of bias due to nonresponse cannot be ignored. Similarities in findings were noted between this work and other studies documented in the literature,^25,28,29 which enhanced the validity of this study. The study sample was similar to the overall MPharm student population in terms of gender, which enhances the generalizability of the findings. Furthermore, while the research was conducted with pharmacy students, many of the findings are relevant and transferable to other higher education and healthcare settings.

CONCLUSION

Pharmacy students considered feedback to be a crucial part of learning and assessment throughout the program. However, this work exposed gaps and inconsistencies relating to the provision of feedback throughout the degree program, and in general, students were not satisfied with the feedback received. To address this issue, academic staff members must be made aware of students’ expectations and opinions related to feedback provision and reminded of the documented importance and benefits of feedback to teaching and learning.¹⁸ Moreover, as provision of feedback on various components of the pharmacy program were identified as acceptable (and in some cases exemplary), there is an opportunity for dissemination of good feedback practices between modules within the school. However, the impending increase in student tuition fees,³² reductions in funding of higher education,³³ and other priorities such as conducting high quality research, undoubtedly will present barriers for those trying to close the gap between student expectation and staff delivery of feedback.