Using Reflective Writing as a Predictor of Academic Success in Different Assessment Formats

Abstract

Objectives. To investigate whether reflective-writing skills are associated with academic success.

Methods. Two hundred sixty-four students enrolled in a pharmacy practice course completed reflective statements. Regression procedures were conducted to determine whether reflective-writing skills were associated with academic success in different assessment formats: written, oral, and video tasks.

Results. Reflective-writing skills were found to be a predictor of academic performance in some formats of assessment: written examination; oral assessment task and overall score for the Unit of Study (UoS). Reflective writing skills were not found to predict academic success in the video assessment task.

Conclusions. Possessing good reflective-writing skills was associated with improved academic performance. Further research is recommended investigating the impact of reflective skill development on academic performance measures in other health education.

INTRODUCTION

Research has shown that fostering reflective skills in health professional education can assist students to improve their clinical decision-making skills and enhance academic performance.^1,2 Furthermore, enhancing reflective writing within health education has been at the forefront of the published literature in recent years.^1,3-5 The ability to reflect on a deeper level is a desirable attribute for all health professionals.^5-11 Reflective capacity is regarded as a skill to enhance learning from previous knowledge and experience in order to improve future clinical practice and clinical reasoning ^2,5,7,10,12 In particular, reflective writing can be used as a tool to enhance reflective capacity.^9,13

Furthermore, in the medical field, awareness of the patient and of the health professional’s own mental processes can be enhanced through the reflective-writing process. This can potentially improve clinical decision-making ability.¹⁴ In many cases, academic performance in health professional education is an indicator of effective clinical decision-making skills. Therefore, fostering reflective-writing skills may be a skill associated with improvement in clinical decision-making skills and academic performance.

Scholars posit that developing reflective-writing capacity will foster deeper learning and reflective thinking, which may lead learners to make better informed judgments and increase their ability to make optimal clinical decisions.^{1,5,8,9,15,16} Previous research has shown that using the skills of reflective writing improves academic and/or clinical performance^17,18 and is perceived by students as a valuable exercise to enhance reflective learning, thus potentially improving their performance of future clinical tasks.^3,19 Furthermore, reflective narratives used in medical education have been shown to improve clinical-reasoning skills by nurturing the skills of observation, analysis, and interpretation.^1,20 Thus, effective reflective-writing skills may be beneficial for educating the pharmacy student and are fundamental for developing not only the reflective practitioner, but may assist with clinical reasoning and academic performance.

Reflective writing may take the forms of reflective statements, essays, portfolios, journals, diaries, or blogs.⁹ Evidence of support for the use of reflective writing in portfolios and journals in health professions are numerous.^14,21-30 These tools are used to foster students’ thinking to challenge their biases, assumptions, and firmly held beliefs.¹⁴ Moreover, reflective-writing instruments in the form of statements have been used as an effective educational tool to assess medical and surgical resident learning in the medical curriculum.³¹ As some have advocated, using tools of reflective writing and working towards becoming a reflective practitioner should be considered a core attribute for professional development.^{1,7,18,20,25,32-34} Health professions are now insisting on students developing reflective-writing abilities to document continuing professional development and competencies.^1,35

Despite the importance placed on reflective-writing skill development in health professional education, some researchers report “tension” between what students will write as an honest reflection and what they will write if a grade is associated with the task. This was illustrated in a Belgian study that investigated the perceptions of 142 midwifery students toward completing reflective writing exercises during their clinical training sessions. While most respondents perceived the reflective writing exercises as an effective learning tool, others reported that they wrote reflections that they believed would gain them a better assessment score.³ This was also illustrated in another qualitative study involving eight focus groups of second-year UK medical students, which reported similar issues regarding the “tension” caused by grading written reflections.³⁶ Nevertheless, students from both studies perceived the exercises as a valuable learning experience that could enhance their performance of future clinical tasks and facilitate deeper understanding and learning.^3,36 Similarly, a study introducing reflective-writing processes into an undergraduate pharmacy curriculum indicated that students perceived this type of learning as challenging.¹⁹ However, the majority of the students did perceive this learning exercise as valuable and indicated that the skill of reflective practice would be beneficial for improvement in other essential skill development including counseling and clinical decision-making.¹⁹

Therefore, it can be argued that effective reflective-writing skills can be used as a tool to promote deeper understanding and learning, and thereby enhance clinical decision-making skills and improve academic performance. Research suggests that reflective ability is not necessarily an inherent skill, but can be taught through various strategies, models, and guidelines.^37-44 Moreover, the development of reflective capacity has been described as necessary for experiential learning in the practice setting.³ Thus, developing and evaluating this fundamental skill should be a consideration in undergraduate and graduate health professions educational programs,⁹ and further scaffolding these skills in practice settings may benefit immediate and long-term reflective learning.^{3,5,8,9,11,19}

Although reflective practice activities may enhance clinical reasoning skills,^2,45 there is little published research investigating the effect of reflective-writing skills on particular types of assessments. The aim of this research was to explore the relationship between reflective writing and academic performance in a cohort of undergraduate pharmacy students.

METHOD

The sampling frame for this research was 264 second-year undergraduate pharmacy students (92 male students, 172 female students) enrolled in a pharmacy practice course. The classroom or laboratory structure used in this course involved 10 groups of 26-27 students. Each group was facilitated by a practicing pharmacist clinical educator affiliated with the university. The pharmacy practice unit of study was an introductory unit that explored disease states and management options, including nonpharmacological recommendations. It focused on methods of delivering patient care to individual patients as well as to the wider community with an emphasis on primary care.

The data from this project were analyzed as part of a larger research project⁵ in which approval for the study was sought and granted by the University of Sydney Human Research Ethics Committee. Approval for integrating reflective activities into curriculum was also received from the school’s Learning and Teaching Committee. As students were required as part of their regular course assessment to complete a reflective-writing task, all students in the second-year curriculum participated in this study.

In 2014, a reflective-writing module was introduced into a second-year undergraduate pharmacy curriculum as part of the Reflective Ability Clinical Assessment (RACA).⁵ We have described the process of the integration of reflective practice activities into the curriculum, and the structure of the Reflective Ability Clinical Assessment (RACA) and its evaluation in more detail in our previous paper.⁵ Briefly, The RACA aimed to enhance the reflective-thinking capacity of students and involved three components: a clinical scenario; a video podcast, and a reflective writing component. The RACA tasks required students to develop a role play, counsel another student, video tape the counseling session (for a maximum of five minutes), and then reflect on the experience through an introspective reflective-writing task. The reflective-writing task took the form of a reflective statement (Appendix 1).

Students were assigned one of 10 health problems that could be treated with nonprescription medication or nonpharmacological advice. These health problems included heartburn, sunburn, impetigo, worms, scabies, insomnia, head lice, traveler’s diarrhea, chicken pox, and ticks. The students were then asked to research the problem (signs, symptoms, required referrals, advice), develop a typical clinical discussion/counseling session between a pharmacist and a patient, counsel (as the pseudo pharmacist) another student (serving as the pseudo patient) and write reflectively about the task. As part of the assignment, students were also encouraged to view and reflect on each other’s videos to enhance self- and peer-reflection (Figure 1). The objective of the end-of-semester oral examination was to assess effective counseling and clinical decision-making skills.

Figure 1.

Utilization of video podcasts to enhance reflective capacity (self and peer reflection) of pharmacy students.⁵

While the video podcast task (which counted 8% of the course grade) provided students with the opportunity to reflect on their counseling and clinical competency skills and those of a peer, it did not involve clinical decision-making skills as the student had prior knowledge of the clinical scenario. The main purpose of the video podcast task was to provide an opportunity for counseling practice prior to the end-of-semester oral assessment. No limit was placed on the number of podcasts recorded before uploading the best one to the Blackboard Learning Management System (LMS). Thus, this activity provided an opportunity for students to counsel each other several times. The video podcast counseling task was submitted mid-semester, along with the reflective statement.

The reflective statement consisted of guided questions (Appendix 1)⁵ and was assessed by one external assessor to ensure grading consistency. The external assessor had expertise in the area of reflective writing and possessed extensive clinical pharmacy experience. The statements were assessed via a reflective rubric which was developed from Boud and colleagues’⁴⁰ stages of reflection and Mezirow’s⁴⁶ categories of reflection. Elements of the rubric were also drawn from Wetmore and colleagues’ reflective rubric, used in dental education (Table 1).⁴⁷ The reflective statement counted 7% of the student’s overall grade for the unit of study.

Table 1.

Reflective Rubric to Assess Reflective Writing in Pharmacy Education^11,a

The oral assessment was a high-stakes examination completed at the end of the semester that accounted for 30% of the overall grade for the unit of study. Students were required to pass this assessment in order to pass the unit of study. While the topic area is provided earlier, the assessment does not provide students with prior knowledge of the case, patient background, signs and/or symptoms. In contrast to the video podcast assessment task, the oral assessment required the student to make appropriate clinical decisions based on gathering patient information during a five-minute timeframe.

After the oral assessment, all students from the cohort completed the end of semester written examination, which included both short-answer and multiple-choice questions (single best answer). This assessment accounted for 55% of the total grade for the unit of study.

Scores from all assessment and examination tasks were analyzed using SPSS, version 20 (IBM Corp., Armonk, NY, USA). Descriptive analyses were performed to provide an overview of the scores allocated for the four dependent variables (1) overall marks for the unit of study excluding the reflective statement scores; (2) written examination scores (including short answer and multiple-choice); (3) video podcasts score and (4) end of semester oral examination score. The predictor variable used the reflective-writing task scores. Simple regression procedures were employed to determine if reflective-writing skills predicted academic success in these different assessment formats. The reflective statement scores were excluded in the regression analyses, as these scores were used as a predictor variable. An adjusted overall score that excluded the reflective statement scores was used for this research. Significance was set at p<.05.

RESULTS

From a sample size of 264 undergraduate students enrolled in the Pharmacy Practice course, 259 (98%) students completed the video podcasts counseling activity and a reflective writing task. Two hundred fifty-eight (98%) students completed the written end-of-semester examination (Table 2).

Table 2.

Descriptive Statistics: Scores for Different Formats of Assessment in a Second Year Undergraduate Pharmacy Curriculum

The results from simple regression analyses explaining 14% of its variance (R²= 0.14) with overall academic achievement are as follows:

• (i) Are reflective writing skills a predictor of academic success in the Overall Marks for one unit of study?

Reflective writing skills was found to be a predictor of academic success in the Overall marks for the Unit of Study, F(1,256)= 25.2, p<0.01

• (ii) Are reflective writing skills a predictor of academic success in the end of semester written examination?

Reflective writing skills was found to be a significant predictor of academic success in the end of semester written examination, F (1, 256) = 21.6, p< 0.01

• (iii) Are reflective writing skills a predictor of academic success in the video counseling format of an assessment?

Reflective writing skills did not predict academic success in the video podcast counseling task, (p=0.08).

• (iv) Are reflective writing skills a predictor of academic success in the end of semester oral assessment?

Reflective writing skills was found to be a predictor of academic success in the end of semester Oral examination, F(1,257)= 8.5, p<0.01

DISCUSSION

In this study we examined the predictive relationship between reflective-writing skills and academic performance in four formats of assessment. The significant findings suggest that possessing reflective-writing skills is a predictor of academic success on the written examination and in the end-of-semester oral assessment. Interestingly, reflective-writing skills were not a predictor of academic success in the video podcast assessment task. There are two possible explanations for this. First, research has shown that reflective-writing skills may enhance decision-making skills.⁴⁸ Assessing the video podcast did not require the student to make any “on the spot” clinical decisions as the student had thoroughly researched the case prior to the counseling session. Conversely, the end-of-semester oral assessment did require students to demonstrate their clinical decision-making skills. Second, the video podcast assessment task was completed midsemester, prior to writing the reflective statement, and perhaps this contributed to the nonsignificant result, as reflection on that task was completed later. This study reported reflective writing had a stronger association with the written component compared to the oral assessment task. This may have been due to the fact that written skills were involved in completing the reflective task. In contrast, there were no written skills involved in completing the oral assessment.

These results, although significant, do not indicate a particularly strong effect size (14% of its variance with overall academic achievement). It is possible that the traditional assessment modalities that are employed in pharmacy education are by and large not directly assessing reflective skills. If educators wish to develop this skill in the health professions, we need to address this in our assessment practices.¹¹ Furthermore, the low effect size could be due to more specific details of the assessment strategies and to grades being allocated to content knowledge, written, and verbal communication skills compared to grades allocated specifically for reflective ability. The limitations of this study were that it involved only one course in an undergraduate curriculum at a single university and that it only involved pharmacy students.

This study advances understanding from previous research in the area of reflective learning of health professionals and the importance of considering the integration of reflective-writing skills into medical and other health professions educational curricula. Pharmacy students are not a homogenous group of future health professionals and this is also the case with the medical and other health professions. The use of reflective tools has been shown to improve decision-making skills. ⁴⁵ Furthermore, it has been suggested that tools which encourage the reflective process allow for continuous evaluation of learning and development ⁴⁸ as it encourages critiquing of practice,⁴⁹ thus developing the health professional into a reflective practitioner.⁴⁹ The findings from this study support the integration of reflective writing into health professional education.

Perhaps the greatest pedagogical challenge for educators of health professionals is how we can help them to write reflectively. Fostering students’ reflective writing skills may be one solution to further developing their critical-thinking and problem-solving skills so they will be able to make better informed clinical judgments. Reflective-writing skills are not necessarily inherent and therefore may be developed through inclusion in health care education programs.^20,38 For this to occur, a reflective framework or model may need to be further developed by health professional education systems to provide a springboard for fostering effective reflective writing.⁹ Ideally these skills should develop over the course of the degree; therefore, time allocated within the curriculum to foster these skills may be a necessary consideration for educators.⁹ Moreover, evaluating the effectiveness of such a fundamental skill through reflective rubrics may further enhance reflective capacity, particularly if reflective rubrics are provided to students prior to reflective-writing activities.¹¹ Also, well-designed reflective rubrics may assist in providing appropriate guidance for desired learning outcomes, such as enhanced reflective capacity. ^11,50

Evidence from this study supports the notion that reflective-writing skills may be associated with improvement in pharmacy students’ clinical decision-making capacity and academic performance. It therefore stands to reason that further research into the effects of various reflective activities in other areas of health education, where effective clinical-reasoning ability is also an important attribute, should be considered.

CONCLUSIONS

This study suggests that developing reflective-writing skills in undergraduate pharmacy students may be associated with improved academic performance. As these results may be specific for one educational setting, namely, pharmacy, we must be mindful that these findings may not translate to other learning environments or health professions. However, as reflective activities and tools appear to embed several health educational programs worldwide, further research in how reflective writing impacts on academic performance measures in other health disciplines is recommended. If reflective writing encourages a broader thinking process, it stands to reason that all health professional education programs should consider this element across curricula and into clinical environments.

Appendix 1. Reflective Statement⁵