Achievement Goal Orientation and Situational Motivation for a Low-Stakes Test of Content Knowledge

Abstract

Objective. To determine the extent of the relationship between students’ inherent motivation to achieve in a doctor of pharmacy program and their motivation to achieve on a single low-stakes test of content knowledge.

Method. The Attitude Toward Learning Questionnaire (ATL) was administered to 66 third-year pharmacy students at the beginning of the spring 2011 semester, and the Student Opinion Scale (SOS) was administered to the same group immediately following completion of the Pharmacy Curricular Outcomes Assessment (PCOA).

Results. Significant differences were found in performance approach and work avoidance based on situational motivation scores. Situational motivation was also found to be directly correlated with performance and mastery approaches and inversely correlated with work avoidance. Criteria were met for predicting importance and effort from performance and mastery approaches and work avoidance scores of pharmacy students.

Conclusions. The ability to predict pharmacy students’ motivation to perform on a low-stakes standardized test of content knowledge increases the test’s usefulness as a measure of curricular effectiveness.

INTRODUCTION

Students’ successful learning and performance in individual courses and across all aspects of a curriculum are to a great extent dependent on their level of motivation to achieve overall and in specific situations.¹ Motivation to achieve is a complex construct where motivation and achievement are naturally interconnected and codependent.² Individuals enter into activities with varying levels of competence and interest. They formulate ideas about the value of an activity relative to the types of consequences they may experience in order to decide how much effort the activity is worth. In some instances the motivation to achieve is intrapersonal; in others, normative. It is not unusual to find situations where the motivation to achieve is a combination of intrapersonal interest and the desire to perform better than others. These affective, behavioral, and outcome-based actions are delimited in 3 theoretical perspectives that collectively help researchers understand the dynamics behind motivation and achievement: expectancy-value theory, interest theory, and achievement goal theory.

The reasons for engaging in a particular activity and the amount of effort put into achieving competence are the focus of the expectancy-value theory. Motivation to achieve is rooted in one’s expectation of success and belief in the level of the relative importance or value of the task in question. Individuals using the expectancy-value approach may ask, “is the task important to me and do I believe I can successfully achieve a positive outcome?”^3-5

Expectancy and value beliefs influence achievement behavior. The balance between competence (the ability to succeed) and consequences (the relative cost of engaging) can impact the extent to which one is motivated to engage and pursue competence. Understanding motivation to achieve in this way assists educators in planning instructional methodologies and assignments, and in assessing achievement. Assignments that generate value and afford opportunity to demonstrate competence are more likely to be embraced and successful, and more accurately measure ability.^1,3,4

The process of determining the relative importance of engaging in a task assumes a level of interest in the topic or activity. According to researchers, interest is a motivational variable that describes the desire to engage with events, ideas, or objects over time.⁶ Interest theory attempts to define how interest develops in individuals as they engage in activities or situations in a variety of environments. Situational or initial interest is a spontaneous response to something in the environment about which an individual may or may not have prior knowledge. Individual interest requires some prior knowledge about a topic or event and a desire to learn more. The distinction between arousing interest and maintaining interest is determined by the level of value and meaning that an event, object, or idea has to an individual. Thus, a situational or initial interest can evolve into an individual or deeper interest if there is sufficient value placed on knowing more. This process is divided into 4 phases (Table 1).

Table 1.

Interest Theory Phases

For more than 20 years, educational psychologists and their colleagues in related disciplines have studied students’ reasons for seeking knowledge and competence in a particular topic or area of study. These reasons or motivations are important determinants of an individual’s achievement goal orientation.^7-11

Achievement goal orientation theory suggests there are 2 broad thought processes and behaviors used to achieve competence in either assigned or self-directed learning goals (performance and mastery). Performance goal orientation is dominated by those who seek to gain competence by performing as well as possible relative to others. These individuals look for challenging goals and competitive environments with articulated achievement standards. Mastery goal orientation is dominated by those who gain competence by learning as much as possible about a particular topic through immersion. They tend to seek challenging tasks and use adaptive behaviors to gain knowledge apart from or in addition to articulated expectations. These behaviors can be further divided into approach and avoidance goals. Researchers posit that these classifications are related to an individual’s intrinsic or inherent motivation to engage and are defined as follows:¹⁰

• Performance approach (positive): strives to achieve competence by performing as well as possible relative to others (normative competence).

• Performance avoidance (negative): avoids situations where achieving competence relative to others is uncertain.

• Mastery approach (positive): strives to achieve competence by learning as much as possible about a topic. Achievement is defined by the subject matter and is less influenced by external expectations about what is relevant.

• Mastery avoidance (negative): avoids situations where barriers to learning impact competence.

• Work avoidance (negative): strives to minimize effort in achievement situations. It is a fifth measure of achievement motivation that provides a measure of insight into an individual’s interest in a task, assessment of the consequences for not engaging, and decision about the value of being successful.^12-14

The approach-avoidance achievement goal orientation is predictive of achievement.^8-11,15,16 Mastery goals were found to be positive predictors of depth of study, effort, and persistence. Performance goals were positive predictors of cursory or surface study, effort, and persistence, and performance avoidance was found to be a positive predictor of disorganized studying. Additionally, researchers determined that performance approach was a positive predictor of examination performance, whereas mastery approach was a positive predictor of inherent motivation. Finally, performance avoidance was a negative predictor of both examination performance and inherent motivation (Table 2).

Table 2.

Achievement Goal Orientation as a Predictor of Academic Activity of Pharmacy Students

Because certain measures of achievement orientation are predictive of performance and intrinsic motivation, they have the potential to inform how competence is measured. This research builds on previous research that measured pharmacy students’ situational motivation to achieve on the Pharmacy Curricular Outcomes Assessment (PCOA).¹⁷ The PCOA is a relatively new national test of content knowledge intended to inform programs about their curriculum and individuals about their academic preparation for pharmacy practice. Wilkes University’s School of Pharmacy chooses to administer the PCOA as a low-stakes assessment. Because performance on low stakes examinations is frequently unreliable, the previous study looked at the impact of low-motivation effort on PCOA test performance and attempted to compute a more realistic aggregate performance score to better inform curricular decisions. This study sought to determine the extent to which students’ motivation to perform on the PCOA (situational motivation) is related to their achievement goal orientation toward their pharmacy coursework (inherent motivation).

METHODS

The primary objective of this study was to determine the extent of the relationship between a student’s inherent motivation to achieve in the major, and his or her motivation to achieve on a single standard test of content knowledge that was administered as a low-stakes assessment. Specifically, the author hoped that the study would indicate the potential to use students’ inherent motivation scores as a predictor of their situational motivation to achieve on the PCOA. The purpose was to improve the reliability of PCOA results to inform decisions concerning curricular effectiveness by finding ways to control for situational motivation.

This study used 2 standardized survey instruments to measure and compare inherent and situational motivation to achieve. Sixty-six third-year pharmacy students were surveyed to determine their achievement goal orientation for the pharmacy major (inherent motivation) and their motivation to achieve on the PCOA (situational motivation).

The entire third-year class of pharmacy students was recruited to participate. This study was approved by the Wilkes University Institutional Review Board.

Attitude Toward Learning Questionnaire

The Attitude Toward Learning (ATL) questionnaire is a 16-item Likert-type survey instrument that is designed to measure aspects of inherent motivation. It was administered to all third-year pharmacy students at the beginning of the spring semester. The PCOA took place several weeks after the administration of the ATL.

The ATL scale used in this study combines the revised Achievement Goal Questionnaire (AGQ) developed by Elliot and McGregor¹⁸ with the ATL questionnaire, which adds the fifth measure of achievement motivation, work avoidance.¹² The AGQ-R (revised) is a 12-item scaled-response survey that is a reliable and validated standard measure of achievement goal competence for the study of motivation in achievement settings.¹⁰ The ATL questionnaire was introduced by Pieper in 2003¹³ and used in subsequent research by Miller and Sundre.¹⁴

As it is not related to a particular task, administration of the ATL is straightforward, does not require connection to a course or course of study, and can be given individually or in a group setting. The questionnaire takes approximately 10 minutes to complete. Items are grouped by achievement goal designation and aggregated and scored on a 7-point scale where 1= “not at all true of me” and 7 = “very true of me.”

The Student Opinion Scale

The Student Opinion Scale (SOS) is a 10-item Likert-type survey instrument that measures perceptions of task importance and motivational effort for a particular task (situational motivation). It was administered immediately following the PCOA and before students left the testing site.

The SOS was developed by Sundre¹⁹ and was influenced by expectancy-value theorists. It is specifically designed to be used with instruments or tests that measure other constructs (eg, PCOA) at the time the other constructs are being tested. As such, it can be considered a measure of situational motivation. It quantifies the level of participants’ motivation relative to the other instrument, and Sundre states it is particularly useful when conducted with low-stakes tests.

As the purpose of the SOS is to provide information about student motivation during a testing situation, the SOS is intended to be administered at the end of a test or series of tests. As students complete the test or tests, they are directed to complete the 10-item scale. Paper and pencil or computer-based test administration is equally acceptable. Each subscale, importance and effort, has 5 items that can yield a total of 25 points for each subscale.

All students were expected to complete the ATL and SOS as part of ongoing curricular assessment. However, each student was given the option of disallowing their data to be used in research and scholarly publication with no consequences to their standing in the pharmacy program.

RESULTS

Sixty-six students in their third-year completed the ATL at the beginning of the spring semester and completed the SOS immediately following the PCOA.

Descriptives & Reliability

There was a normal distribution for importance and effort scores and for 3 of the achievement goal orientations: performance, mastery approach, and work avoidance. Performance and mastery avoidance distributions were bimodal and no further statistical tests were performed on them.

Tests for reliability suggested a fairly high level of reliability for importance (Cronbach alpha = 0.77) and effort (Cronbach alpha = 0.79). Reliability of ATL scores were also calculated by achievement goal orientation: performance approach (alpha = 0.88), performance avoidance (0.84), mastery approach (0.88), mastery avoidance (0.71), and work avoidance (0.85). The total possible score for performance approach, mastery approach, and performance avoidance is 21 each; and for work avoidance, 28.

Variability

Building off the normal distribution of the 3 achievement goal orientations, analysis of variance was calculated to determine differences in performance and mastery approach and work avoidance scores based on levels of reported situational motivation. Importance and effort were each divided into quartiles and became the groups for the one-way ANOVAs.

There was a significant difference among importance and effort quartile groups for performance approach and work avoidance. There was no significant difference among quartile groups for mastery approach (Table 3). Post-hoc tests (Tukey HSD [honestly significant differences] test) determined that the significant difference occurred between the first and fourth quartiles for both importance and effort. The means plots for both importance and effort quartile groupings indicated a marked increase in performance approach scores and decrease in work avoidance scores as situational motivation increased (Table 3).

Table 3.

Differences Among Pharmacy Students’ Achievement Goal Orientations Based on Quartile Groupings for Importance and Effort

Regression

A correlation matrix was produced to measure the extent to which situational and inherent motivation to achieve were related. The Pearson product moment or Pearson r test was used: r (degrees of freedom). Both components of situational motivation (importance and effort) were significantly correlated to the same 3 achievement goal orientations: mastery approach, performance approach, and work avoidance. There was a moderate direct correlation between importance and performance approach (r(64) = 0.31, p < 0.05) and importance and mastery approach (r(64) = 0.31, p < 0.05). There was a moderate indirect correlation between importance and work avoidance (r(64) = -0.31, p < 0.05). Effort was moderately correlated with performance approach (r(63) = 0.31, p < 0.05) and weakly correlated with mastery approach (r(63) = 0.25, p < 0.05). There was a moderate indirect correlation between effort and work avoidance (r(63) = -0.36, p < 0.01). There was no significant relationship between either subscale for situational motivation and mastery, or for performance avoidance.

Multiple linear regression was calculated to predict the students’ motivation to achieve in a specific situation based on their achievement goal orientation (mastery approach, performance approach, and work avoidance). A significant regression equation was found for importance: (p < 0.05) with an R² of 0.15, indicating that 15% of the variation in students’ perceptions of the value of a task can be explained by the variation in their reported achievement goal approach and work avoidance. The resultant equation states: students’ predicted level of importance equaled 13.306 = 0.215 (performance approach) + 0.101 (mastery approach) - 0.134 (work avoidance) +/- 6.354 (standard error of the estimate = 3.2). The negative direction of the work avoidance coefficient supports the indirect relationship. Thus, a student who scores 15 on performance approach, 17 on mastery approach, and 7 on work avoidance, could be expected to score between 11 and 25 on the SOS scale of importance. The SOS scale does not exceed 25 on either importance or effort.

A significant regression equation was also found for effort: (p < 0.01) with an R² of 0.178 indicating 18% variability. In this equation, both mastery approach and work avoidance have negative values. The predicted level of effort equals 18.537 + 0.220 (performance) -0.085 (mastery approach) - 0.177 (work avoidance) +/- 5.142 (standard error of the estimate = 2.571). Accordingly, a student who scores 21 on performance approach, 20 on mastery approach, and 5 on work avoidance would be expected to score between 15 and 25 on the SOS scale of effort.

DISCUSSION

This study attempted to determine to what extent a student is motivated to achieve on the PCOA when it is administered as a low-stakes assessment, based on his or her self-reported inherent achievement motivation towards the pharmacy major. Knowing students’ motivation level is important given that Wilkes University School of Pharmacy wants to use the results of the PCOA to inform decisions regarding the pharmacy curriculum. As motivation is not believed to be correlated with ability, optimum motivation to achieve does not ensure a successful outcome on the PCOA for a student, but it does suggest that the outcome represents a student’s level of competence at that point in time.

Understanding the type of inherent motivation with which students engage in achieving competence provides a window into how they prefer to learn and how they advance toward competence. For example, those students who champion a performance approach are competitive and prefer clear parameters regarding instructor expectations. Their frame of reference is normative and they measure competence based on how well they perform relative to others. Conversely, those using a mastery approach are focused more on depth of learning about a topic. Apart from what others are doing, they seek competence by learning all they can about a subject and they measure competence by how they perform relative to the task. Thus, they tend to be less influenced by a situation and more influenced by self-interest.

The data support this in that differences in situational motivation had an impact on performance approach and work avoidance but not on mastery approach. Those who reported low (first quartile) situational motivation toward the PCOA also had low performance approach goal orientation and high work avoidance goal orientation. Conversely, those who reported a high situational motivation toward the PCOA also reported high performance approach goal orientation and low work avoidance. As expected, mastery approach was not impacted by situational motivation.

There was a significant relationship between both components of situational motivation (importance and effort) and achievement goal approach (performance approach and mastery approach), and work avoidance. This is not particularly surprising in that they measure similar constructs. However, the relationships are not so strong as to suggest colinearity and thus the need to either collapse or eliminate a variable.

The predictive qualities of achievement goal orientations stated in the literature provide a strong reason for measuring students’ approach-avoidance orientations to create a baseline for understanding how students prefer to learn and what to expect in terms of performance. Knowing that the research supports performance approach as a positive predictor of examination performance and mastery approach as a positive predictor of inherent motivation provides insight into what to expect on tests of content knowledge. When those tests are low-stakes assessments, the modulating impact of motivation to achieve in a particular situation may improve the ability to predict overall competence. Additionally, there is perhaps less reason to be concerned about those who score high on mastery approach as they are not as influenced by a particular situation.

This study had some limitations. The results were calculated based on students’ performance on 1 low-stakes test. Additionally, sample size was limited to 1 class at a single point in time, and as such, generalizability is severely limited. Testing needs to be expanded to include other programs that use the PCOA as a low-stakes test of content knowledge.

Multiple regression, although significant, accounts for a relatively small variability in perceived value and effort applied to the PCOA based on achievement goal orientation (15% and 18% respectively). As a result, predictions should be made with caution.

Finally, the PCOA is norm referenced and, as such, dependent on the number of schools participating in the test in order to calculate weighted scores. The pharmacy program participation rate in the PCOA hovers around 20%, which is not necessarily 20% of the total number of enrolled students at the colleges and schools represented. Increasing the number of participants or modifying the test to criterion referenced would lessen the impact of low participation and potentially increase the use of the test, even for those programs that choose to administer it as a low-stakes assessment.

CONCLUSION

Measuring a student’s achievement goal orientation has value to inform instruction (how students learn) and to manage the assessment of learning in order to achieve a more accurate measure of performance. In particular, this study provides pharmacy programs that administer the PCOA as low stakes a way to manipulate the impact of low situational motivation by using a students’ achievement goal orientation to predict the level of importance and effort a student is likely to attribute to this test of content knowledge. A thoughtful approach to assessment of student learning that considers various aspects of the motivation to achieve should improve the ability to decipher individual and aggregate measures and impact the use of these measures to make decisions about teaching and learning.