Monetary versus grade incentives depending on personality traits: A field experiment on undergraduate students’ performance

Abstract

This study aims to examine the role of personality on the effectiveness in improving students' performance of two extrinsic incentives: monetary and grade incentives. To achieve this goal, we conducted a randomized field experiment in which students in a Microeconomics course were offered the opportunity to participate in a practice test program, with no effects on the grade of the course itself. In the call to participate, students were informed that participants would be randomly assigned to one of two groups. Whereas in the control group students would not be monetarily incentivized, participants assigned to the treatment group would be paid according to their performance in the practice tests. In addition, we elicited the big five personality and risk aversion traits of the participants (168 undergraduates). All subjects received grade incentives in the later official course exam, in which no monetary incentives were offered. We used non-parametric tests to carry out both between-subjects and within-subjects performance comparisons. Controlling for potential confounding factors like students' gender and academic record, our OLS regressions indicate that although monetary incentives are effective in improving students’ performance in practice tests, their effect does not carry over to the course exam. Furthermore, we find that the effectiveness of grade incentives (used in the course exam) on improvement as a substitute for monetary incentives (adopted in practice tests), is higher the more conscientious the students are.

Highlights

• •Students' practice test scores are improved by monetary incentives.
• •This effect is not carried over to their academic performance in the course exam.
• •Grade incentives as a substitute for monetary ones are more effective for more conscientious and lower extraverted students.

1. Introduction

Students’ motivation has a significant role in academic performance [1]. According to self-determination theory (SDT), there are two types of motivation: extrinsic and intrinsic motivation [2]. Whereas intrinsic motivation manifests itself in genuine interest and enjoyment from learning, extrinsic motivation refers to goal-oriented behavior to achieve the objective sought.

In this study, we focus on two types of external regulations: monetary and grade incentives. Although both are inherently extrinsic, they give different possibilities of being internalized by students. Whereas money is a reward unrelated to academic activity, grades can be internalized as an engagement factor if it makes sense to students. Contrary to other social sciences like psychology, economists (and especially experimental economists) are convinced that monetary incentives enhance subjects’ motivation, leading to greater effort and performance.1 In the context of education, Barrow and Rouse [3] suggest that students are short-sighted when faced with the short-term effort of performing well as opposed to the delayed potential future benefits. Thus, Wallace [4] proposes monetary rewards linked to academic performance as an effective mechanism to make the benefits from studying more short-term and prominent.

In fact, many studies using financial rewards as an extrinsic motivator suggest that paying students has been shown to result in better performance.2 Although part of the literature has taken into account some individual characteristics3 as potential confounding factors, students' personality traits have not been related to the effectiveness of monetary incentives in improving academic performance. There is a gap in the empirical literature concerning how students respond to extrinsic incentives depending on their personality. Thus, it is necessary to consider the role of the students' personality in the effect of money and grades as engaging mechanisms. This shortcoming motivates our research question: How two types of extrinsic incentives like grades and money, can help students to achieve a better performance depending on their personality traits? We contribute to the literature by evaluating their effectiveness in improving students’ performance conditional to the big five personality domains.

2. Literature

Our paper involves two separate lines of research. The first one is the literature concentrating on the influence of monetary incentives on academic performance. Empirical evidence on the effectiveness of monetary incentives on academic achievement is mixed depending on the type of student affected and the incentive schemes offered [5]. Regarding the former, Leuven et al. [6] and De Paola et al. [7] find significantly positive treatment effects for high-ability undergraduate students. Moreover, Le [8] conducts a meta-analysis using programs offering monetary incentives in elementary and secondary education. His results show a moderate positive effect for overall achievement as well as for mathematics achievement, but none for reading/language arts achievement.

As for the type of payment scheme, the literature has tested different incentives systems. Fryer [9] and Angrist et al. [10] increase the payment linearly based on students' grades, with zero or little effectiveness respectively. Elsewhere, Behrman et al. [11] in a secondary school and Levitt et al. [12] and List et al. [13] in primary and secondary education pay students who improve on their past grades, finding small and substantial effects of the treatment respectively.4 Other studies use exogenous5 or endogenous6 thresholds to be met in order to get paid. In relation to cash sensitivity, larger monetary rewards do not seem to produce greater effects on university students [14]. Contrary to Herranz-Zarzoso and Sabater-Grande [15] and Sabater-Grande et al. [16] in which students' rewards were based on self-chosen-goals following a quadratic scoring rule, in this study we use a simpler incentive system, where students' earnings increase linearly with their scores. As a novelty with respect to our previous work, in the present experiment monetary incentives are implemented in absence of grade incentives. Specifically, we apply rewards on students' performance in an ungraded practice test program7 mimicking the later course exam. Although practice tests are a tool to help students to prepare for and pass the course exam, appraising the present costs of studying can be easier than evaluating potential future benefits. Following Wallace [4], given that monetary incentives offset students’ present costs implementing closer benefits, we expect that monetarily-incentivized students will score higher in practice tests than those not rewarded with money (Hypothesis 1, referred to as H1 hereafter). In addition, from a theoretical perspective, positive feedback on practice tests has the potential to enhance future performance [17] by increasing students' perceived self-efficacy [18]. Thus, we predict an indirect effect on exam grades, given that money will incentivize students to give a better practice performance, and this positive feedback could empower them to attain superior academic achievement. So, previous monetary incentive effects (H1) would be transferred to the course exam. In consequence, we hypothesize that students scoring higher in practice tests due to the influence of monetary incentives will obtain higher grades in the course exam (Hypothesis 2, referred to as H2 hereafter).

The second line of research is focused on how students’ personality affects their motivation and their academic achievement. A limited number of studies have analyzed the relationship between personality (generally in the context of the Big Five model) and academic motivation. From these five personality traits, conscientiousness is the factor that has been directly related to motivational behavior by Costa and McCrae [19]. Specifically, Komarraju and Karau [20] find that openness and conscientiousness factors are directly linked to positive aspects of academic motivation, such as persistence and achievement, indicating that students who are more responsible and open to experiences are more intrinsically motivated. Similar results are obtained by Watanabe and Kanazawa [21],8 McGeown et al. [22] and Ariani [23] 9 regarding conscientiousness. The role of conscientiousness as a positive significant predictor of intrinsic motivation is also confirmed by Cardoso et al. [24] and Tomsik [25] for students and teachers alike.

Reports by Hart et al. [26] and Watanabe and Kanazawa [21] suggest that conscientiousness (plus extraversion for the former) is positively associated with extrinsic motivation. Other studies like Bidjerano and Dai [27], Komarraju et al. [28], Clark and Schroth [29] and Ariani [23] also report that neuroticism has a positive effect on extrinsic motivation. Additionally, Komarraju et al. [28] suggest that students scoring high in conscientiousness are most likely to be motivated both by enjoyment of learning and the accomplishment of goals. In this vein, Payne et al. [30] shows that subjects scoring high in conscientiousness and extraversion are more goal-oriented in learning. Finally, Saleh et al. [31] claims that, whereas openness is the factor commonly linked to intrinsic motivation, extraversion is associated with extrinsic motivation, and conscientiousness with both.

Moreover, empirical evidence10 suggests that personality traits and academic motivation are related variables explaining academic performance. Following this finding, Hazrati-Viari et al. [32] argue that motivation plays a mediating role in the relationship between personality traits and academic performance, concluding that whereas conscientiousness explained both intrinsic and extrinsic motivation, openness predicted only intrinsic motivation. Related to this finding, Kaufman et al. [33] states that academic performance improves for students with high scores in intrinsic motivation and responsibility, while there are low scores in their extrinsic motivation. Although the relationship between personality traits and academic performance may differ because of the diversity of mechanisms used for measuring it, most studies11 suggest that conscientiousness is the strongest predictor of academic achievement. The reason for this is that this personality trait is associated with self-discipline, goal-orientation and continuous effort, all basic factors of academic achievement.12

Along these lines, John et al. [34] claims that, with the exception of a minority of studies,13 the remaining four personality traits show weak positive associations (or even negative ones in the case of neuroticism) with academic performance.

This second line of literature gives us a deeper insight into the potential role that certain personality traits have on how money and grade (extrinsic) incentives improve students' performance. Specifically, whereas grades can be seen by students as a representation of the quality of their learning, money is not connected with knowledge. Thus, we expect that money could be a more effective incentive than grades in order to improve the performance of students scoring high in personality traits related exclusively to extrinsic motivation. This assumption has potential consequences for the two types of subjects of our sample: controls and monetarily incentivized students. Specifically, we expect that, for the controls, the addition in the course exam of an extrinsic incentive like grades (with respect to previous practice tests) will help improve performance among students scoring high in personality traits related to extrinsic motivation. In consequence, we hypothesize that in the control group, the effectiveness of grade incentives in improving performance will be positively affected by students’ scores in conscientiousness and extraversion (Hypothesis 3, H3 hereafter).

Regarding the monetarily-incentivized students, we expect that the substitution of an extreme extrinsic incentive (money) by another one with possibilities of being internalized (grade), will improve performance for students scoring low in personality traits exclusively related to extrinsic motivation, or those scoring high in traits related to non-extrinsic motivation. Supporting this assumption, the findings of Fulmer and Walker [35] and Harper et al. [36] suggest that conscientious subjects might value money less and thus exert less effort to attain it. Thus, we hypothesize that, in the monetarily-incentivized group, the effectiveness of grade incentives on improving performance will be positively affected by the students’ conscientiousness score, and negatively by their extraversion score (Hypothesis 4, referred to as H4 hereafter).

Summing up, we aim to fill the gap between these two strands of literature in order to analyze the effectiveness of grade and monetary incentives on student performance according to their personality traits.

The rest of the paper is structured as follows: first, we present the material and methods used; after that, we analyze the empirical evidence collected, present our results and discuss them; and lastly, we draw conclusions, pointing out the implications and limitations of our study.

3. Material and methods

3.1. Research design and procedure

A randomized field experiment was conducted in order to compare the effectiveness of two alternative extrinsic incentives—grades and monetary incentives— on improving performance. In the call to participate, students in a Microeconomics course at a Spanish university in 2019 were offered the possibility of taking part in a monetary incentive program, asking them through an invitation call about their willingness to participate.14 We opened our call at the beginning of the course. In the call, students were informed that those responding affirmatively would participate biweekly in five practice tests with no implications on the final grade of the course itself. Each test involved ten four-choice questions corresponding to the course topic recently taught. Each right answer was rewarded with 1 hypothetical point and each wrong answer deducted 0.4 hypothetical points. The practice tests were designed to mimic the format and difficulty15 of the subsequent course exam. In addition, the participants were informed that they would be randomly assigned to one of two groups: a control group or a monetarily-incentivized (MI) group. Students in the control group took a series of non-incentivized and ungraded practice exams, followed by the standard, graded course exam. Students in the MI group took the same series of exams, but the practice exams were monetarily incentivized. Specifically, participants assigned to the MI group were informed that they would receive €5 for each hypothetical point obtained in one of the five practice tests, randomly selected.

In addition, since we were interested in testing whether personality traits modulate the effectiveness of monetary incentives compared to grade incentives, we used the NEO-PI-R Five Factor Model [37] to elicit students' personality facets.16 Moreover, given the multiple-choice question format implemented, one might argue that risk aversion could influence students' behavior when it comes to choosing one (or none) of the four options offered in the test. In consequence, we elicited incentivized subjects’ risk attitudes using the standard Holt and Laury (HL, hereafter) [38] lotteries.

3.2. Participants

From 168 voluntary students, 85 were randomly assigned to the control group, and 83 to the MI group. After discarding students not attending the course exam, our sample was reduced to 62 subjects in the control group and 72 in the MI group. Both control and monetarily-incentivized students were subject to identical grade incentives in the course exam (in which no monetary incentives were offered). Thus, whereas in the MI group, the grade incentives in the course exam were perceived as a substitute for previous monetary incentives (i.e., students received monetary incentives in the practice tests but only grade incentives in the course exam), in the control group grade the incentives were perceived only as an addition with respect to the practice tests (i.e., students did not receive any incentives in the practice test, but they got grade incentives in the course exam).

3.3. Data collection instruments

Subjects participated in the experiment following the usual recruitment and ethical clearance protocols used in the LEE, and it was approved by the ethics committee at the Universitat Jaume I (Castellón, Spain).

3.3.1. Personality instrument

We used Costa and McCrae's Five Factor Model [37], the Spanish NEO-PI-R. This test is a 240-item self-report measurement for quantifying 30 specific traits or facets that define the five personality factors or domains: Neuroticism, Extraversion, Openness, Agreeableness and Conscientiousness. Items are responded to on a 5-point Likert scale ranging from 0 (strongly disagree) to 4 (strongly agree). Example sentences are: “I am not a person who worries a lot” (Neuroticism); “I like most of the people I meet” (Extraversion); “I have a very active imagination” (Openness); “I tend to be cynical and skeptical about the intentions of others” (Agreeableness); “I am known for my prudence and common sense” (Conscientiousness).

3.3.2. Risk attitude elicitation instrument

Following Zhou and Hey [39] and Friedman et al. [40], Holt and Laury's Lotteries [38] is perhaps the most popular approach for measuring risk tolerance in the lab. Subjects are given Table 1, and in each row they are asked to make binary choices between a safe or a risky lottery. In each one of them, the probability associated with the high payoff in both lotteries (safe or risky) increases and the probability associated with the low payoff decreases as subjects go down Table It is expected that when the probability of the high payoff increases enough, a subject switches from the safe to the risky lottery at some point and never switches back. When implementing the HL method, participants are informed that after making all their decisions, one decision will be selected at random and the chosen lottery will be played for real. Then, subjects are paid according to that outcome.

Table 1.

Holt and Laury (2002) lottery.

Option A	Option B
1/10 of €2.00, 9/10 of €1.60	1/10 of €3.85, 9/10 of €0.10
2/10 of €2.00, 8/10 of €1.60	2/10 of €3.85, 8/10 of €0.10
3/10 of €2.00, 7/10 of €1.60	3/10 of €3.85, 7/10 of €0.10
4/10 of €2.00, 6/10 of €1.60	4/10 of €3.85, 6/10 of €0.10
5/10 of €2.00, 5/10 of €1.60	5/10 of €3.85, 5/10 of €0.10
6/10 of €2.00, 4/10 of €1.60	6/10 of €3.85, 4/10 of €0.10
7/10 of €2.00, 3/10 of €1.60	7/10 of €3.85, 3/10 of €0.10
8/10 of €2.00, 2/10 of €1.60	8/10 of €3.85, 2/10 of €0.10
9/10 of €2.00, 1/10 of €1.60	9/10 of €3.85, 1/10 of €0.10
10/10 of €2.00, 0/10 of €1.60	10/10 of €3.85, 0/10 of €0.10

3.4. Statistical analysis

Our study is designed to conduct both between and within-subjects comparisons. Whereas between-subjects comparisons assess group differences between monetarily incentivized and non-monetarily incentivized students, within-subjects comparisons contrast related performances from the same students between different conditions. Given that the variables presented in our study are not normally distributed,17 both types of comparisons will be performed by means of non-parametric tests.

Our experiment allows us to conduct between-subjects comparisons for two outcomes: students’ performance in practice tests and grades obtained in the course exam. The former allows us to analyze the effectiveness of monetary incentives in the absence of grade incentives in the practice tests (control vs. monetarily-incentivized students). The latter favors studying whether the aforementioned effect can be carried over to the course evaluation, in which no monetary incentives are offered. Both comparisons between independent and not normally distributed samples are achieved by means of a Mann-Whitney test. When the two distributions have the same shape, this test allows us to determine if there are differences in medians between groups. However, when samples are differently distributed, only mean rank comparisons can be obtained.

In addition, two within-subjects comparisons are carried out. Regarding the control students, a performance comparison between the scores obtained in practice tests (in which no incentive is offered) and the course exam allows us to analyze the effectiveness of adding grade incentives. In relation to the monetarily-incentivized students, comparing students’ performance in practice tests (in which monetary incentives are offered) and the course exam allow us to study the effectiveness of implementing grade incentives (in the course exam) as a substitute for monetary incentives. Both comparisons are carried out by means of a Wilcoxon test, given that we were examining two sets of scores coming from the same pool of participants in two time points.

4. Results and data analysis

4.1. Descriptive statistics and tests

Descriptive statistics of students’ scores in the five practice tests and course exam grades, splitting the sample in the control group and the MI group, are shown in Table 2. Moreover, one can see additional descriptive statistics on: university academic record, personality trait scores in each facet of the NEO-PI-R test, and the number of safe choices per pair of options in the HL task. Additionally, we present p-values from a Mann-Whitney (M − W) test, comparing the control group and the MI group for all these variables.

Table 2.

Descriptive statistics and treatment comparisons of scores in practice tests, course exam grades, academic record, personality traits in the NEO-PI-R test, and safe choices in the HL task. CG: control group; MIG: monetarily-incentivized group;

**p < 0.01, *p < 0.05

	Mean CG N = 62	Mean MIG N = 72	Median CG N = 62	Median MIG N = 72	S.D. CG	S.D. MIG	M − W test
Practice Test 1	4.69	5.38	5.25	5.50	2.51	1.79	−1.586
Practice Test 2	2.30	3.49	2.00	3.75	2.20	2.27	−3.084**
Practice Test 3	3.15	4.34	3.00	4.00	2.59	2.51	−2.603**
Practice Test 4	2.63	4.02	2.00	4.00	2.69	2.37	−3.423**
Practice Test 5	2.50	3.62	2.75	3.75	2.17	2.27	−2.930**
Course exam	2.99	2.75	2.86	2.50	2.01	2.00	0.959
Academic record	6.44	6.42	6.36	6.21	0.65	0.71	0.394
Agreeableness	116.02	111.74	115.00	114.00	26.38	28.06	1.076
Conscientiousness	121.90	115.92	123.00	117.00	19.93	19.30	1.877
Neuroticism	92.17	89.24	93.00	86.00	16.92	17.12	1.360
Extraversion	113.71	117.22	113.00	118.00	18.85	19.50	−1.253
Openness	115.30	111.26	116.00	108.00	16.15	17.76	1.495
Risk aversion	4.88	5.14	4.50	5.00	2.16	1.90	−0.837

We can observe that students' mean rank performance in the practice tests is significantly higher when monetary incentives are introduced.18 Moreover, regarding course exam grades, academic record, personality traits or risk aversion, no statistically significant differences in mean rank scores are found between controls and monetarily incentivized students.

The mean rank performance result is confirmed by comparing the standardized average score obtained in practice tests between students monetarily-incentivized and those who were not. Fig. 1 shows standardized grades for the practice tests and the course exam for the control students and the monetarily-incentivized ones. Focusing on the practice tests, we can detect a shift to the right in the distribution of standardized grades of monetarily-incentivized students with respect to controls. This observed shift is proof of the monetary incentives’ effectiveness: monetarily-incentivized students obtain significantly higher grades than those without monetary incentives.19 However, although monetary incentives improved scores in the practice tests, their effect is not carried over to the course exam given that when they are not longer present,20 no statistically significant differences between groups are detected.21

Fig. 1.

Box plot comparing the distribution of standardized practice tests and exam grades between the CG and the MIG. CG: control group; MIG: monetarily-incentivized group.

In addition to the above between-subjects analysis, Fig. 1 allows us to compare within-subject performance for each treatment group. For the control group, the addition of grade incentives does not significantly improve students’ performance.22 Nevertheless, when grade incentives substitute previous monetary incentives (implemented in the MI group), their performance is significantly diminished.23

4.2. Regression analysis

The main purpose of this section is to isolate the causal relationship between incentives (money or grades) and academic performance controlling for potential driving factors. Students’ performance is analyzed through the practice test scores, course exam grades and the difference between them. Furthermore, the effect of monetary incentives is included in the next OLS models through a dummy variable that takes the value 1 if the student belongs to the MIG and 0 if not. These potential driving factors used are university academic record, gender (a dummy variable that takes the value 1 if the student is a woman and 0 if the student is a man),24 risk aversion and the five facets of personality.

First, in the model presented in Table 3, we explore the factors explaining students' performance in the practice tests. In this task, the students who belong to the MI group are monetarily incentivized, but the control students do not receive any incentive. The main result is that monetary incentives and students' academic record play a crucial role in explaining subjects' performance in practice tests. Specifically, we find that both variables positively and significantly affect students' scores in practice tests. On the contrary, students’ gender, risk aversion and personality traits have no significant effect on their performance on practice tests.

Result 1

When grade incentives are not present, we find a direct relationship between monetary incentives and subjects' performance in practice tests, confirming H1.

Ancillary result 1: In the absence of grade incentives, students' academic record positively affects practice test performance. However, the latter is not affected by students’ personality traits, risk aversion or gender.

Now, we focus on the students' academic performance in the course exam. In this task, no monetary incentives are provided and the incentive offered for all students (control and monetarily-incentivized students) is their grade. At this point, the practice tests' average score is included in the model because it can be a potential driving factor. Additionally, an interaction term between practice tests and monetary incentives is also included to analyze whether there is a carry over effect of monetary incentives (provided in the practice tests) to the course exam. The regression shown in Table 4 indicates that, in the presence of grade incentives, students' final exam score is explained by their academic record and their performance in practice tests, regardless of whether they have been rewarded monetarily for it or not in the practice tests. Moreover, personality traits, risk aversion and gender are not significant predictors of students’ academic performance.

Result 2

Students' course exam grades are positively affected by their performance in practice tests, regardless of whether they were monetarily rewarded or not. Although practice test scores are improved by monetary incentives (result 1), this effect is not carried over to their academic performance in the course exam. Thus, this result rejects H2.

Ancillary result 2: In the absence of monetary incentives, students’ academic record is a significant predictor of their course exam grades. Nevertheless, such grades are not affected by personality traits, risk aversion or gender.

Table 5 explains performance differences between the course exam and practice tests according to whether monetary incentives were implemented in practice tests (MI group) or not (control group). Specifically, the first model corresponds to the case in which grade incentives implemented in the course exam are simply added with respect to the practice tests,25 and the second model refers to the case in which monetary incentives offered in the practice tests are substituted by grade incentives in the course exam.

Finding no support for H3, the first model shows that, when grade incentives in the course exam are an addition with respect to previous practice tests, personality traits do not explain the differences in students’ performance. Nonetheless, the second model shows that, when monetary incentives offered in practice tests are substituted by grade incentives in the course exam, there is a direct relationship between conscientiousness and differences in performance. That is, the more conscientious a student is, the more grade incentives (as a substitute for monetary incentives) help them to improve their performance, partially confirming H4. This finding can be explained by the fact that individuals scoring high in conscientiousness are associated with successful academic achievement under grade incentives. Nevertheless, our results do not support the hypothesized indirect relationship between extraversion and the difference in academic performance under alternative incentives (grade vs. monetary).

Result 3

The improving effectiveness of grade incentives as a substitute for monetary incentives is higher the more conscientious students are.

Table 3.

OLS regression for students’ performance in practice tests.

	Practice tests
Monetary Incentive	0.951** (0.272)
Academic Record	1.120** (0.211)
Gender	−0.285 (0.295)
Risk aversion	−0.0450 (0.0715)
Agreeableness	0.000312 (0.00915)
Conscientiousness	−0.00900 (0.00744)
Neuroticism	0.000118 (0.00829)
Extraversion	0.00164 (0.00794)
Openness	−0.00161 (0.00881)
Constant	−2.635 (2.381)
Observations	134
R-squared	0.300

Robust Standard errors in parentheses.

**p < 0.01, *p < 0.05 (Bonferroni-adjusted significance levels for personality traits).

Table 4.

OLS regression for course exam grades.

	Final exam
Monetary Incentive	−1.315 (0.685)
Practice tests	0.265* (0.132)
Practice test*Monetary incentive	0.157 (0.171)
Academic Record	1.363** (0.214)
Gender	−0.415 (0.323)
Risk aversion	0.0430 (0.0748)
Agreeableness	−0.0137 (0.00852)
Conscientiousness	0.0105 (0.00728)
Neuroticism	0.00231 (0.00888)
Extraversion	−0.00236 (0.00784)
Openness	−0.0166 (0.00935)
Constant	−4.351* (2.133)
Observations	134
R-squared	0.489

Robust Standard errors in parentheses.

**p < 0.01, *p < 0.05 (Bonferroni-adjusted significance levels for personality traits).

Table 5.

OLS regression for the difference between students’ performance in the course exam and the practice tests.

	T1	T2
Academic Record	0.318 (0.419)	0.570* (0.261)
Gender	0.439 (0.569)	0.860 (0.529)
Risk aversion	0.0826 (0.110)	0.108 (0.119)
Agreeableness	−0.0345 (0.0160)	−0.00579 (0.0126)
Conscientiousness	0.00463 (0.0137)	0.0249* (0.00914)
Neuroticism	0.0178 (0.0159)	−0.0169 (0.0153)
Extraversion	0.0214 (0.0123)	−0.0265 (0.0116)
Openness	−0.0254 (0.0168)	−0.00293 (0.0140)
Constant	−0.591 (4.130)	−2.738 (3.022)
Observations	62	72
R-squared	0.164	0.304

Standard errors in parentheses.

**p < 0.01, *p < 0.05 (Bonferroni-adjusted significance levels for personality traits).

5. Discussion

Monetary incentives are typically used in the real world as a mechanism to maximize performance by creating an extrinsic motivation for subjects [41]. Our findings support previous literature in higher education26 suggesting the effectiveness of monetary incentives as an effective intervention to improve students' performance. However, in line with Zeiske et al. [42], we observe that monetary incentives’ effectiveness is not sustained over time, once the stimulus is removed. Although we expected that previous monetary incentive effects could be transferred to the course exam, it is arguable that a counter effect might be produced by the fact that low grades seem to incentivize students to improve their following course performance [43]. Thus, educators should be aware that the monetary incentive has to be applied directly to the task in which student performance is intended to be improved, given that it will only be effective while in place.

Given that motivation is influenced by personality traits [44], it would seem reasonable to analyze how people respond to different extrinsic incentives depending on their individual characteristics. In addition, Grant and Green [45] suggest that most studies supporting the effectiveness of grade incentives in improving academic performance can be potentially subject to omitted variable bias. In fact, Jalava et al. [46] find that the motivational strengths of grade incentives are highly dependent on students skills and their gender. In the same vein, Schinske and Tanner [47] suggest that the effectiveness of grade incentives rely on students' ability and could even enhance anxiety among struggling students. These results contrast with our findings given that our study does not support a significant effect of grade incentives improving controls’ performance, independently of their gender, academic record, risk aversion and personality traits.

The empirical evidence about the influence of personality traits on the effectiveness of monetary incentives is scarce and it has usually been generated in the lab. Specifically, the lab experiments by Fulmer and Walker [35] and Harper et al. [36] suggest that conscientious subjects might be less responsive to piece-rate monetary incentives. We found that the effectiveness of grade incentives as substitutes of previous monetary incentives is higher the more conscientious the students are. This role of the conscientiousness trait can be interpreted as a result of its relationship with both intrinsic and extrinsic motivation. This is because, although both incentives are classified as extrinsic, grade incentives can be considered by students to be a well-internalized form of extrinsic motivation. For this reason, money can be a less effective incentive than grades on improving the achievements of students scoring high in conscientiousness. Our findings suggest the importance of knowing our target population in order to design an effective intervention aimed at improving their performance. However, for the purpose of robustness, the results obtained should be replicated in different academic environments. Taking into account that field experiments provide less control over the environment than lab experiments, hindering their replicability, all protocol conditions regarding the procedure, experimental instructions and methods used should be maintained in order to ensure compatibility.

6. Conclusion

To the best of our knowledge, this study is the first randomized field experiment analyzing the mediating role of personality traits on the effect of grades and monetary incentives as tools to improve undergraduates' performance. Our findings indicate that although monetary incentives are effective in improving students’ scores in ungraded practice tests, their effect does not carry over to following course exam, in which only grade incentives are offered.

In addition, this experiment offers preliminary evidence regarding the moderating role of students' personality in the effect of monetary and grade incentives on improving academic performance. Controlling for students' gender, risk aversion and academic record, empirical evidence suggests that conscientiousness is the only personality trait with a significant role in the effectiveness of the extrinsic incentives implemented in our field experiment. Precisely, the effectiveness of grade incentives (based on the differences in performance between the course exam and the practice tests) can be explained by conscientiousness only when they substitute previous monetary incentives. Specifically, in the monetarily-incentivized group, highly conscientious students improve more between practicing and the course exam than less conscientious ones. However, when grade incentives are included simply as an addition, their effectiveness does not rely on students’ conscientiousness.

7. Theoretical, practical and policy implications

From a theoretical standpoint, the present study makes two contributions to the empirical body of knowledge about the effectiveness of extrinsic incentives in improving students' performance. First, we obtain that monetary incentives are effective in improving undergraduates' scores in a given task, but this effect fades out in subsequent related tasks in which students perform. Second, students’ conscientiousness plays a significant positive role in the effectiveness of grade incentives when they substitute previous monetary incentives.

A practical implication of our study focuses on the long-term effects of monetary incentives on improving students' performance. Interventions aiming to implement temporary monetary incentives as tools to encourage ongoing behavior are not supported by this study, however. Our findings suggest that monetary incentives are effective in promoting students’ performance only while they are in place. Thus, educators should carefully examine and select which critical tasks deserve a financial effort by policy makers given that they should not expect students to continue engaging in the targeted behavior once the monetary incentive is removed.

Another policy implication of our study is related to the role of personality traits in the success of an extrinsic incentive intervention. Students' motivation is a critical driver of achievement, and exploring heterogeneity in this driver by students' personalities is useful when thinking about targeting policies. Given that the implementation of monetary incentive programs is costly, it is crucial to be better positioned to assert under what conditions certain interventions work. Students’ personality can influence how certain extrinsic incentives are perceived, driving the type of target population impacted by a monetary incentive intervention. Therefore, being able to assess the personality traits of our target population can be an essential factor so as to evaluate in advance the success of this kind of intervention in improving academic achievement.

8. Limitations and potential areas of future studies

Our study is limited by specific features that would restrict the generalizability of its findings: treatment effects depending on context like the type of exam (multiple choice test) and course (Microeconomics, where advanced math skills are important), where the experiment was implemented, and uncontrolled factors which may be partially responsible for some of the differences reported here. Regarding the first limitation, extrinsic achievement motivation is predicted worse by personality traits than intrinsic achievement because, whereas the latter is relatively unaffected by specific situational factors, the former generally relies on context characteristics. This could explain the empirical inconsistency in the relationship between extrinsic incentives and personality traits. Thus, further research is needed to study the aforementioned relationship depending on different types of course assessments and students. Regarding the second limitation, other uncontrolled factors may partially account for some of the effects reported here, but no further data was available. Additional research is needed to control for individual aspects potentially clouding our results, like students’ socioeconomic status. Thus, our results have to be interpreted cautiously when establishing causal relationships between extrinsic incentives and academic performance.

Author contribution statement

Gerardo Sabater-Grande: Conceived and designed the experiments; Performed the experiments; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Noemí Herranz-Zarzoso: Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding sources

This work has been supported by Project PID2021-123053OB-I00 funded by MCIN/AEI/10.13039/501100011033/ and by ERDF A way of making Europe; and by the Universitat Jaume I [UJI-B2021-23].

Data availability statement

Data included in article/supp. material/referenced in article.

Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Gerardo Sabater-Grande reports financial support was provided by Spain Ministry of Science and Innovation. Gerardo Sabater-Grande reports a relationship with University Jaume I that includes: funding grants.

Appendix A. Supplementary data

The following are the Supplementary data to this article: