Effect of Metacognitive Prompts on Undergraduate Pharmacy Students’ Self-regulated Learning Behavior

Abstract

Objective. To elucidate the ability of pharmacy students to self-regulate their learning, and to determine the impact of their self-regulatory behaviors on their learning outcomes.

Methods. This study took a mixed methods approach that used “meta-learning” assessment tasks to identify the learning strategies chosen and relied on by 139 second-year pharmacy students, and to determine the relationships between the quality of strategies and academic achievement.

Results. Although students had previously tried a wide range of learning strategies, they only consistently rely on a few of them. When prompted to use more advanced strategies, the quality of strategies reported were significantly related to academic achievement, with significant relationships between achievement and goal setting, self-efficacy, self-satisfaction, self-evaluation, and adaptive reactions.

Conclusion. These findings suggest that high-achieving students use higher-quality forethought and self-reflective strategies than do poor-achieving students. Potentially, prompting students to engage in higher-quality strategies may increase students’ awareness of their own learning and improve student learning outcomes.

INTRODUCTION

Self-regulation is a critical skill for students at all levels, particularly university students.¹ During the transition from secondary to tertiary education, students leave a prescriptive but supportive learning environment to enter a tertiary environment that values independence in time management, particularly regarding students’ ability to plan their own study.² However, new university students are often inadequately prepared to overcome the challenges faced in this new environment,³ and managing time efficiently is only one of many skills required for success. Students also need to develop proactive-learning qualities, such as initiative, perseverance, and adaptation, traits which are influenced by positive motivational feelings and beliefs.⁴ Altogether, student success in learning at university relates to their ability to use effective strategies to regulate their own learning, ie, their self-regulation.¹

Self-regulated learning behaviors can be defined as the thoughts, feelings, and actions that are planned and adapted by an individual to attain a goal.⁵ It is more than simple attempts to gain knowledge, as self-regulation implies that learners become aware of their learning, and make motivational and behavioral adjustments to attain and implement knowledge more effectively. Research on self-regulation over the past 20 years has led to the development of a well-established, three-phase cyclical model of self-regulation that includes forethought, performance, and self-reflection phases.^5,6 While other models of self-regulation have been proposed, with variations in descriptors and complexity, all share similarities in that they are cyclical and suggest that learning occurs through distinct phases.^7-11 Within the Zimmerman model, the planning or forethought phase includes task analysis processes, involving goal setting and strategic planning, and self-motivation processes, which relate to student’s beliefs bout learning.^12-14 The performance phase involves behaviors implemented in response to the learning process,¹⁵ with the main processes being self-control and self-observation.⁵ Finally, the self-reflection phase, occurring after learning has taken place, involves self-reaction and self-judgement processes, with self-evaluation and causal attributions being part of the latter process.^5,16

Poor self-regulation of learning has the potential to negatively affect many aspects of a student’s academic career, and consequently it is an issue that is not discipline-specific. However, poor self-regulation is likely to be more prevalent in students who are required to study outside their area of expertise or interest. For students enrolled in degree programs that cater to a specific career outcome, career goals and interests play an important role in determining the relevance of the subject area and therefore motivation.^17,18 Such scenarios are relatively common for students in professional programs such as allied health, where they are required to undertake “foundation” science courses. In addition, students within these programs may exhibit learning characteristics that make them less likely to self-regulate effectively. For example, students undertaking pharmacy degree programs are known to have a strong vocational focus, a preference for external sources of regulation, and lower preferences for the deep-processing learning strategies that are involved in self-regulation.^19-22

For students to both recognize and modify their self-regulatory behavior, they must develop an understanding and awareness of their own learning processes, their “metacognition,” and then use that knowledge to control their own learning. This metacognition of learning, often called “meta-learning,” occurs when students are aware of their motives, abilities, and the demands of the task, and can control their behavior to achieve desired outcomes.^23,24 It involves self-awareness, self-identity as a learner, and reflection as a key aspect of self-awareness,^25,26 and is therefore closely associated with self-regulation of learning.²⁷

All students are capable of self-regulating their learning, but the quality and quantity of the self-regulatory processes differ between learners.¹ In evaluating the types of strategies employed by students, researchers believe that those within the forethought and self-reflection phases are more advanced.^12,28 Strategies used during the performance phase are those used in direct reaction to the learning process that is occurring.^1,12 In contrast, strategies from the other two phases involve the application of more complex metacognitive processes requiring self-awareness, and are therefore inherently more difficult for students to master and implement.²⁹ We postulated that highly self-regulated learners would set clearer and more specific goals during the forethought phase, and self-evaluate more often, prompting better adaptations during the self-reflection phase.^30,31

This study reports on the use of meta-learning assessment tasks, which were developed primarily as a tool for students to reflect on and evaluate their own learning of biomedical sciences, to gain insight into students’ learning strategies and processes.³² Specifically, student responses to meta-learning questions were used to identify the repertoire of learning strategies previously employed by second-year undergraduate pharmacy students studying biomedical science; determine the strategies they predominantly rely on for their study, and (3) determine the inter-relationships between the quality of students’ forethought and self-reflection strategies used during the semester and their academic performance.

METHODS

Participants for this study were undergraduate Bachelor of Pharmacy students undertaking a Human Physiology and Pharmacology course in the second semester of the second year of their degree program at the University of Queensland, Australia. The course aimed to develop students’ understanding of human body functions and the use and effects of drugs in disease-oriented contexts. The course had three contact hours a week consisting of lectures only, and covered an integrated approach to physiology and pharmacology in the endocrine, reproductive, neuronal, and gastrointestinal systems. Students were required to complete seven assessment items across the 13-week semester: a mid-semester examination worth 25%, an end of semester examination worth 60%, and five equally weighted meta-learning tasks collectively worth 15%.

This study was approved by the University of Queensland Human Experimentation Ethical Review Committee and all participants (n=139, 59% of the cohort) provided their informed consent. Demographic data of the cohort and consenting students (Table 1) showed strong similarities. A slightly higher percentage (74%) of female students were in the group that consented to participate in the study than the percentage of female students in the whole cohort (64%). However, a Fisher exact test showed the percentages were not significantly different, and that there was no difference in the proportion of international and domestic students that agreed to participate. Furthermore, unpaired t tests showed that there were no significant differences in prior performance or age between the groups (Table 1). To further confirm that these groups were academically equivalent, the end of semester examination results of consenting students (n=139) and those of the whole cohort were subjected to an unpaired t test.

Table 1.

Demographic Data of Cohort and Consenting Student Groups

Students completed meta-learning tasks at approximately two-week intervals during the semester. The meta-learning tasks and their implementation and impact have been described in detail previously.³² Briefly, each task was comprised of six open-ended questions designed to help students develop an understanding of their own learning. Questions within the first two meta-learning tasks were developed to prompt students to reflect on recent lecture content, to articulate their goals and motivations for the course³³ to consider the study strategies they had used in the past and those they most often relied on,⁶ and to identify factors influencing their learning process.²⁴ The third task, completed two weeks prior to the mid-semester examination, was developed with the aim of prompting students to use strategies from the forethought phase and promote effective study for the midsemester examination.^5,24 Students completed the fourth meta-learning task after receiving midsemester examination results and feedback. This task aimed to encourage students to reflect on strategies they had used for study, to determine whether they were effective, and consider how they could improve their study process for future examinations.²⁴ The final meta-learning task, completed just prior to the end of the semester, prompted students to again reflect on their goals, assess their motivation for the end of semester examination, and propose strategies they would use to assist their studies.^5,6,24 Students were awarded a small number of grade points (marks) for participation in the meta-learning tasks.

As the meta-learning tasks are assessment items, all students completed them; however, only responses from consenting students were used for detailed analyses. Responses to meta-learning questions were subjected to deductive thematic analysis with the qualitative analysis software NVivo 10 (QSR International, Melbourne, Australia). To identify the types of learning strategies that students had previously used, responses to the question, “What learning strategies have you employed in the past?” were analyzed. Students could select as many strategies from the provided list of 14 strategies as they wished, with an additional option of “other.” The list of strategies was based on the classification system of self-regulatory study strategies created by Nota and colleagues⁶ who identified the most commonly used self-regulation strategies among high-school students. The strategies were adapted to suit the university setting and expressed in lay language. Students were then asked to identify which of those strategies they relied on most during study and why. As students were in their second year, the answers to these questions were likely to reflect their study strategies at the university level but, as this was the first meta-learning task, should not have been influenced by any effect the tasks may have had on their learning. The hierarchy of themes against which the student responses to the latter question were coded was based on the phases and processes in Zimmerman’s model,⁵ further sub-coded using the strategy classifications adapted from Nota and colleagues (Table 2).⁶ To determine coding reliability, a second researcher independently coded 25% of the data against these themes and performed a coding comparison that found coding agreement was above 99%.

Table 2.

Strategy Classifications and Examples from Student Responses

To evaluate the forethought and self-reflection processes the students engaged in, responses of consenting students to questions from the third- and fourth meta-learning tasks were analyzed. Within these tasks, delivered prior to and after the midsemester examination, questions prompted students to engage in subprocesses of the forethought and self-reflection phases. Rather than simply identifying whether students did or did not engage in specific strategies when prompted, an analysis of the quality of self-regulatory processes they reported, and the strength of their perceptions was undertaken. To aid this analysis, a set of rubrics was developed through a reiterative, data-driven process (Tables 3, 4 and 5). The standards for each criterion were developed based on the supposition that the most self-aware and highly self-regulated students were more likely to provide detailed descriptions of their self-regulation strategies, to display greater insight into the relevance and practicality of their plans, and to display stronger convictions regarding their performance. Standards were based on either the strategy quality the strength of the beliefs students expressed; or the attributions students made regarding their achievement. For example, in setting goals, a student giving a high-quality, detailed response may mention an actionable goal [G] with increased detail relevant to the assessment task, and further mention why [W] and how [H] this goal could be achieved (Table 3). One student’s detailed response stated: “I've set myself to read and go over all the lecture notes [G] four times [H] to make sure that everything has sunk in [W]. I've also planned on writing small summaries on each lecture [G] and answering the short answer questions that were put up on blackboard [G] and memorizing them for the examination [H]. I have also planned study sessions with pharmacy friends [G] so we can quiz each other and discuss certain concepts [W]”. In contrast, a vague response may mention a goal, but not how or why it could be achieved, such as: “Yes I have. My study goal consists of self-deadlines which fit my BIOM study into what I want done by a certain date while still leaving time for other things I have to do [G].” The researcher was blinded to student academic performance throughout the analysis. To determine the reliability of the analysis, 25% of the data was independently graded by a second researcher. The initial agreement between researchers was 87%. For discrepant responses, the two researchers discussed and reached consensus, and then revised the coding accordingly.

Table 3.

Criteria for Forethought and Self-Reflection Processes, Strategies, Questions and Meta-Learning (ML) Tasks Used to Elicit Student Responses for Each Process. Standards Reflect Whether Responses were “Vague,” “Intermediate,” or “Detailed”

Table 4.

Criteria for Forethought and Self-Reflection Processes, Strategies and Questions Used to Elicit Student Responses for Each Process. Standards Reflect Students’ Self-Perceptions and Satisfaction Levels, Ranging from “Low” to “High”

Table 5.

Criteria for Causal Attributions Based on Whether Students’ Responses were Attributable to “Uncontrollable” or “Controllable” Factors or a Combination of Both, and Question Used to Elicit Student Responses

Statistical analyses were conducted using Prism (GraphPad, San Diego, CA), with a significance level of p<.05. Throughout this article results are expressed as means and standard deviations. A non-parametric Spearman rank correlation was performed using data from all consenting students to identify relationships between the quality, strength or attribution of students’ self-regulatory strategies and academic achievement, calculated from students’ grades in mid-semester and final examinations, and strategies in the forethought and self-reflection phases. These data were further subjected to a permutation test to exclude effects of order on the outcomes.

RESULTS

Performance on the end of semester examination by consenting students, 78% (1.2) was not significantly different from that of the whole cohort at 75% (1.0), suggesting that the consenting students were academically representative of the cohort.

Responses to meta-learning questions indicated that students had used between three and 14 strategies, with an average of 8.3 (2.8) strategies in the past. The most frequently used strategy reported by 193 students was reviewing records, and the least frequently used strategies were active reappraisal of records and strategic planning, each reported by 83 students (Figure 1A). Forty-one percent of students also chose “other,” but always in combination with one or more of the listed strategies. All the performance strategies were nominated more frequently than forethought and self-reflection strategies, except goal setting (Figure 1A).

Figures 1a and 1b.

Self-Regulated Study Strategies

Student responses to the open-ended question “Which learning strategies [from the list] do you rely on the most?” were collated to determine which strategy and phase of self-regulation was relied on the most. Students could mention more than one strategy, and 67% did so. Responses provided by students (n=129) were categorized into one of the self-regulation categories proposed by Nota and colleagues⁶ (Table 2). Keeping records and reviewing records were the most frequently relied on strategies, while self-evaluation was least relied on (Figure 1B).

Responses were further analyzed to determine the proportion of students who used or relied on strategies from one, two, or three phases of the self-regulation cycle, and what combination of phases these represented. When the strategies students reported previously using were allocated into phases of the self-regulation cycle, it was found that all students had previously employed at least one strategy from the performance phase, and many had previously used at least one strategy from the forethought phase (n=122) and the self-reflection phase (n=116), respectively, such that 88% of students had previously used strategies from more than one phase (Figure 2). This suggests that the majority of students had previously used strategies from all three phases of the self-regulated learning cycle (n=101, 73%), with a smaller proportion having previously used strategies from just two phases of the cycle (n=37, 27%) and only one student using strategies from just one phase of the self-regulation cycle (Figure 2). However, when students were asked to identify the strategies they relied on, 61% reported strategies from only one phase of the self-regulation cycle. Of these students, the majority (n=64) relied solely on strategies from the performance phase, 10 students reported self-reflection strategies, and just a few students identified forethought strategies. An additional 44 students relied on strategies from two phases of the cycle: 20 students relied on performance and forethought strategies, 20 students relied on performance and self-reflection strategies, and four students relied on forethought and self-reflection strategies. Finally, in marked contrast to the response on strategies previously used, only seven students said that they relied on strategies from all phases (Figure 2).

Figure 2.

Students’ responses (n=139) regarding which strategies they had used (hatched bars), or responses (n=129) regarding which they relied on (black bars) were categorized into the Forethought (F), Performance (P) or Self-reflection (S) phases of the self-regulation cycle, with the combination of letters (FP, FS, PS, FPS) reflecting the phases reported

Students were profiled on the quality of their forethought and self-reflection strategies using the generated rubrics (Tables 3-5). Responses were categorized across standards of quality, from vague to detailed; strength of the beliefs expressed, from low to high; or attributions made to factors which were controllable, uncontrollable, or a combination of both (Figure 3). The majority of students (52%) expressed high levels of satisfaction with their performance, and nearly three-quarters (71%) attributed the success or failure of their study to controllable factors (Figure 3). Students tended to take responsibility for their own performance, whether good or bad. For example, one student said “My organization and preparation leading up to the midsemester exam allowed me to complete the exam with confidence,” while another said, “One criticism which I have of myself is that during the semester before the exam I did not review the material as frequently as I could have.” Most students had moderate to high self-efficacy regarding their capabilities in performing assessment tasks, with responses such as “I believe with the study and effort I have put into revising for this exam I should be able to perform fairly well.” However, some (9%) students had low belief in their capability (Figure 3). For example one said, “I think it will be difficult for me and I don't expect to do very well.”

Figure 3.

Responses of students (n=139) to questions regarding their Forethought and Self-Reflective behaviors were categorized across standards of quality (Tables 3-5), with standards ranging from highest (hatched bars), moderate (black bars) to lowest (white bars).

The quality of some strategies from the forethought and self-reflection phases was positively correlated with academic achievement (Table 6). Specifically, there were significant positive relationships between the quality of students’ (in descending order) self-satisfaction, self-efficacy, goal setting, self-evaluation, and adaptive reactions regarding the course and their academic achievement. Adaptive reactions also had a significant positive correlation to goal setting and strategic planning (p<.05). Self-evaluation was positively correlated with the greatest number of forethought and self-reflection strategies, being significantly correlated with goal setting (p<.001), strategic planning (p<.01), and adaptive reactions (p<.01). Interestingly, there was a moderate but significant negative correlation between self-satisfaction and perception of ease (p<.05), suggesting that students who thought the examination would be easy were somewhat less satisfied with their results. The causal attributions students made regarding their performance in this course were not related to their academic achievement or to any other strategy evaluated.

Table 6.

Correlation Matrix of Achievement and Quality of Self-Regulation Strategies or Strength of Students’ Perceptions from Forethought and Self-Reflection Phases (n=139)

DISCUSSION

Educators expect that students will acquire higher levels of independence in their study habits as they progress through their education, and in doing so, will acquire a wider repertoire of self-regulation strategies.^13,34 This cohort demonstrated that they were familiar with a wide range of self-regulation strategies and had previously tried many, reflecting that, as second-year undergraduate students, they were at an advanced level of education. Using a variety of self-regulation strategies is associated with high achievement and the ability to overcome learning difficulties,^35,36 and highly self-regulated learners are thought to use strategies from all phases of the self-regulatory cycle,^12,28,31 but the optimal type and number of strategies required to succeed at university is not known.³⁷ Despite students reporting that they had used a breadth of strategies, when asked to identify which strategies they relied on, a very different pattern emerged. Very few students relied on strategies from all three phases (Figure 3), with most relying solely on strategies from just one phase, primarily the performance phase.

These findings give rise to the question: if most students had tried strategies from the forethought and self-reflection phases, why did students predominantly resort to strategies from the performance phase? Potentially, students may find the strategies from the forethought and self-reflection phases difficult to use. Research suggests that planning for learning is cognitively demanding as it requires learners to understand the area of expertise they wish to acquire, have insight into their existing knowledge, and possess the pedagogical knowledge to make informed decisions.³⁸ In addition, Pintrich²⁸ and Granville and Dison²⁹ state that self-reflection is a difficult task for most students to master. When under academic pressure, students may revert to strategies they find easier rather than strategies that they find more difficult to use. Indeed, when asked why they found the strategies they relied on to be effective, students said: “I have found this extremely helpful in the past” and “this is a strategy that I find easy,” implying that their preference for specific strategies may have been because of the ease of and/or familiarity with using them. Educators should be concerned that only 5% of students were relying on strategies from all phases, as it indicates that the majority of these students are not engaging in effective study habits and exhibit signs of poor self-regulation.³⁹ While these findings represent a snapshot of second-year pharmacy students studying biomedical science, they do corroborate the findings of Smith and colleagues.⁴⁰ and of other researchers^19,22 who emphasized the propensity for pharmacy students from all years to display poor self-regulatory skills.

The learning strategy preferences of pharmacy students that are described here imply that one potential factor might separate high-achieving students from poor-achieving students and this may be the regular use of forethought and self-reflection processes. Profiling individual students demonstrated that, when students were prompted, there were significant positive relationships between the quality of learning strategies in both the forethought and self-reflection phases and their academic achievement in this course (Table 6). Specifically, the strength of student’s perceptions regarding self-efficacy and self-satisfaction, and quality of their goal setting, self-evaluation, and adaptive reactions regarding this course were all related to academic achievement.

Within the forethought phase, the two main subprocesses in task analysis, goal setting and strategic planning, demonstrated differing relationships with achievement.¹² Specifically, a significant correlation was found between the quality of student’s goal setting and their achievement, whereas strategic planning was not related to achievement. Goals can vary in type, with students who set mastery goals focusing on processes to attain competence or understanding,⁴¹ while those who set performance goals emphasizing the final products of learning, such as examination results.³⁰ Previous studies suggest that setting a combination of mastery and performance goals has the most positive effects on student performance.^42,43 Our findings support this, suggesting that students who have well-articulated, relevant goals with clear plans for reaching them are more likely to perform well than those who do not. Within the self-motivational beliefs,¹² students’ self-efficacy was significantly related to achievement (Table 6). This relationship is well established, with self-efficacy also considered important for motivation and task persistence.^44-46 There was also a relationship between self-efficacy and perception of ease, suggesting that students who considered the examination would be easy also expected to do well, but perception of ease was not related to academic achievement.

Within self-reflection processes, the ability to react adaptively was significantly correlated with academic achievement (Table 6). Zimmerman suggests that highly self-regulated learners monitor their strategy use and modify strategies if task demands change.⁵ In a study of first year students, Heikkilä and colleagues⁴⁷ revealed that students who demonstrated effective adaptive reactions were more likely to have a deep understanding of a task, critically evaluate their progress, and demonstrate high academic achievement. Our findings corroborate those results, as high-achieving students in our study demonstrated a better understanding of the potential impact of their strategies on understanding and performance. One high-achieving student stated, “In order to improve my performance for the end of semester exam, I need to manage my time better. This involves establishing a strict schedule so that I can cover the required lecture content efficiently…” Sandars and Cleary⁴¹ postulate that the extent to which students use strategic plans loses value if students are not proficient at making adaptive changes when faced with challenges. The significant correlation shown in this study between adaptive reactions and strategic planning support this statement. There was also a positive relationship between self-evaluation and adaptive reactions. Frequent evaluations of the effectiveness of one’s study strategies and understanding are essential for academic success.^1,5,12 Students are able to retain knowledge for longer and apply skills more accurately when they engage in self-evaluation¹⁶ When students self-evaluate, it implies that they are prepared to identify factors that influence their performance and are willing to modify ineffective strategies.⁴¹ Importantly, this study has shown relationships between self-evaluation and academic achievement, and a number of processes from the forethought phase (Table 6). This key finding has important implications for education; it suggests that influencing students’ self-evaluations may not only influence their adaptive reactions, but also impact on their goal setting, strategic planning, and academic achievement for future learning.

Interestingly, there were no significant correlations between causal attribution and achievement (Table 6), with only nine students attributing their performance to uncontrollable factors (Figure 3). The effect of causal attributions on academic success has been well documented in the past.³⁶ For example, Soric investigated primary school students to determine their causal attributions after receiving their grades, finding that successful students attributed their successes to controllable factors, while unsuccessful students attributed their failure to uncontrollable or fixed factors.⁴⁸ Similar findings were reported in a study involving first-year undergraduate students.⁴⁹ However, much of the previous research has evaluated school students, and those studies on undergraduates focused on first-year students.^35,36,49 Research suggests that as individuals progress in their studies, their ability to positively attribute successes and failures to controllable factors increases.⁵⁰ Perhaps these second-year students reflect a more mature approach, as the majority attributed their performance to controllable factors and took responsibility for their own performance.

A limitation of this study was the self-reported nature of the data obtained. While students did provide rich and detailed responses to the meta-learning questions regarding their use of self-regulatory learning strategies, they did not provide information on the extent to which they use them. Therefore, distinguishing between students who spent considerable time on study and those who did not was difficult. In addition, as all students in the cohort completed the meta-learning tasks, identifying any influence the tasks themselves may subsequently have on students’ learning behaviors is difficult. For example, having been prompted to reflect on their learning strategies,³² students may have been better able to articulate them. However, the findings regarding strategies students used and relied on were drawn from the first meta-learning task, which should have reduced the impact of this issue. In addition, students who consented may have been more likely to provide detailed answers, but this effect may have been limited by the nature of the meta-learning tasks, given that they are assessed, with grades awarded for complete answers. Finally, while these findings represent a snapshot of second-year pharmacy students studying biomedical science, they may not be representative of pharmacy students at all year levels, although they do corroborate findings from Smith and colleagues⁴⁰ and other researchers^19,22 who emphasized the propensity for pharmacy students from all year levels to display poor self-regulatory skills. Further, the findings may be unique to pharmacy students and not reflect students in other professional degree programs, although we have observed similar findings in other allied health students.⁵¹

CONCLUSION

Self-regulation is a crucial skill for university students as they need to independently gain knowledge in a short period of time, often while juggling additional commitments. There is considerable pressure on students to perform to an acceptable standard, and on educators to facilitate positive results. In order for educators to facilitate effective learning strategies, they must be cognizant of the current level of self-regulation in their students, and use this information to guide methods for cultivating more effective self-regulation skills. The findings from this study provide evidence of the range of self-regulatory strategies used by undergraduate pharmacy students studying biomedical science, and importantly, identify the interrelationships between strategies from different phases of the self-regulatory cycle and with academic achievement. Additionally, while their knowledge and experience of self-regulatory strategies is broad, these students relied on a narrow range of strategies, suggesting that these students may benefit from interventions to improve their self-regulation. Potentially, the meta-learning assessment tasks used in this study can act as prompts for the students to engage in more diverse strategies, particularly those from the forethought and self-reflection phases, and increase students’ awareness of their own learning processes.