Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 Jul 27.
Published in final edited form as: Appl Cogn Psychol. 2011 Jul-Aug;25(4):528–535. doi: 10.1002/acp.1725

SAT Performance: Understanding the Contributions of Cognitive/Learning and Social/Personality Factors

BRENDA HANNON 1,*, MARY MCNAUGHTON-CASSILL 1
PMCID: PMC3144549  NIHMSID: NIHMS300597  PMID: 21804694

SUMMARY

This study identifies a number of sources of individual differences in SAT performance by examining the simultaneous contributions of factors from two otherwise disparate research areas, namely cognition/learning and social/personality. Preliminary analysis revealed that just the cognitive/learning measures accounted for 37.8, 41.4 and 21.9% of the variance in SAT, V-SAT and Q-SAT performance, respectively while just the social/personality measures accounted for 21.4, 18.2 and 17.3% of the variance. When combined, cognitive/learning and social/personality factors accounted for even larger amounts of variance in performance; specifically 43.4, 44.6 and 28% for the SAT, V-SAT and Q-SAT, respectively. Finally, the results revealed that three measures consistently predicted performance on the SAT, V-SAT and Q-SAT; two measures were the learning/cognitive factors of working memory and integration of new text-based information with information from long-term memory and one measure was the social/personality factor, test anxiety.

Each year tests of academic achievement, like the SAT (i.e. Scholastic Assessment Test), are completed by thousands of students in the United States. For many students the SAT is one of the most, if not the most, important measures of academic achievement because it plays a dominant role in the high stakes decisions of college admissions. For this reason the SAT is of great interest to students and the public in general. However in spite of its importance, the SAT is frequently subjected to intense scrutiny; especially with issues concerning construct validity. Indeed Richard Atkinson, well-known cognitive psychologist and former president of the University of California, proposed dropping the SAT as a requisite for admissions to the University of California because, as he exclaimed in a speech, ‘Who knows what they measure?’ (Cloud, 2001, p.62). Although Atkinson’s concerns were more to do with the relative values of general measures of aptitude versus measures of specific content areas, his comments reflect the construct concerns frequently voiced about the SAT.

The present study addresses some of the construct concerns about the SAT by examining the simultaneous contributions of factors from two research areas, namely cognition/learning and social/personality; see Credé and Kuncel (2008) who describe these separate research areas as cognitive and non-cognitive. These two otherwise disparate research areas were selected because few studies have examined their factors simultaneously even though studies have shown that a number of factors from both of these research areas contribute to SAT performance. The value of this approach is that it not only reveals the relative contributions of factors from each of these research areas but it also might lead to a comprehensive theory of SAT performance and academic achievement that considers both cognitive/learning and social/personality factors.

The specific cognitive/learning factors we selected were working memory, higher-level cognitive processes and epistemic belief of learning. These cognitive/learning factors were selected because: (i) they allow us to build on previous research examining the relationships between the SAT, V-SAT and/or V-SAT passages and working memory (e.g. Daneman & Carpenter, 1980; Daneman & Merikle, 1996; Turner & Engle, 1989), higher-level cognitive processes (e.g. Hannon & Daneman, 2006) and epistemic belief of learning (Daneman & Hannon, 2001); (ii) each of these factors accounts for considerable amounts of variance in SAT, V-SATand/or V-SAT passage performance (e.g. Daneman & Carpenter, 1980; Daneman & Hannon, 2001; Daneman & Merikle, 1996; Hannon & Daneman, 2006; Turner & Engle, 1989) and (iii) these three factors fall within the expertise of the authors.

With respect to social/personality factors, we selected academic self-efficacy, achievement motivation, test anxiety and academic locus of control. These factors were selected on the basis of research examining academic achievement. For example, Zeidner (1998) recommends that academic self-efficacy, achievement motivation and test anxiety are three factors that should be considered in order to develop a ‘reasonable’ model of academic achievement. Similarly, Borkowski and colleagues (Borkowski, Carr, Rellinger, & Pressley, 1990; Borkowski, Chan, & Muthukrishna, 2000) include achievement motivation, academic self-efficacy and academic locus of control in their model of academic achievement.

BACKGROUND

Over the years, a number of studies have documented factors that influence or explain individual differences in performance on the SAT (e.g. Cassady & Johnson, 2002; Coyle, 2006; Coyle & Pillow, 2008; Daneman & Carpenter, 1980; Daneman & Hannon, 2001; Daneman & Merikle, 1996; Elliot & Church, 1997; Elliot & McGregor, 1999; Engle, Tuholski, Laughlin, & Conway, 1999; Frey & Detterman, 2004; Hannon & Daneman, 2006; Turner & Engle, 1989; Zwick & Green, 2007). However the factor or factors under investigation have often varied from study to study. According to some cognitive psychologists, measures of general intelligence or g are strongly related to SAT performance. For instance, correlations as high as .83 have been observed between SAT performance and measures of intelligence, such as the Armed Services Vocational Aptitude Battery, Ravens’ Advanced Progressive Matrices, and the Wonderlic Personnel Test (e.g. Coyle, 2006; Frey & Detterman, 2004; see Dodrill & Warner, 1988 for more information about the Wonderlic). Indeed, research suggests that SAT scores load highly on the first principal factor of a factor analysis of cognitive measures; a finding that strongly suggests that the SAT is g loaded (Frey & Detterman, 2004).

Given that measures of general intelligence and working memory are strongly related (e.g. Engle et al., 1999), it is not surprising to find that some cognitive psychologists have shown that measures of working memory—a cognitive resource shared by many cognitive processes—correlate with performance on both the V-SAT (i.e. verbal SAT) and the Q-SAT (i.e. quantitative SAT), the two major subsections of the SAT (e.g. Daneman & Carpenter, 1980, 1983; Daneman & Hannon, 2001, 2007; Daneman & Merikle, 1996; Engle et al., 1999). Indeed Engle et al. (1999) showed that the aggregate of multiple measures of more complex processing + storage measures of working memory, like the reading and operation spans, accounts for a considerable amount of variance in V-SAT and Q-SAT performance (range of r from .25 to .50) than does the aggregate of simple storage-only measures, like the word span and digit span (See Daneman and Carpenter (1980) and Daneman and Merikle (1996) for more on working memory). From a theoretical perspective, these researchers argue that measures of working memory presumably reflect one’s ability to keep a mental representation active, especially in the face of interference or distraction (Engle et al., 1999). With respect to the SAT, this ability is particularly important because large amounts of information are continually processed, assimilated, and/or discarded within a limited amount of time.

More recently, researchers have suggested that other cognitive factors influence SAT performance. For instance, recent research in Hannon’s lab has shown that a number of higher-level cognitive processes that are used to learn and understand text predict SAT performance (e.g. memory for explicit information from a text, text-based inferencing, the process for accessing prior knowledge from long-term memory, and the integration of new information with prior knowledge). Especially relevant to the present study was the finding that a student’s ability to integrate prior knowledge from long-term memory with new information acquired from text (i.e. knowledge integration) accounted for as much as 20% of the variance in SAT performance (See Hannon & Daneman, 2006 who also showed a similar finding using VSAT reading passages). Presumably knowledge integration is a good predictor of SAT performance because it ‘draws not only on text-based processes and knowledge access processes but also on processes involved in integrating the text information with prior knowledge (Hannon & Daneman, 2001, p. 111)’. These integration processes are the types of processes that are used to reason about reading passages, analogies or word problems (Hannon & Daneman, 2001). In other words they are used to process the types of text found in the SAT.

Still other cognitive researchers suggest that when students have good metacognitive awareness or beliefs about the complexity of knowledge and the value of integration during learning they perform better on the comprehension passages of the V-SAT than do students who have poor or weak metacognitive awareness (Daneman & Hannon, 2001). For the purposes of the present study we refer to this metacognitive awareness about learning as epistemic belief of learning (See Daneman & Hannon, 2001; Rukavina & Daneman, 1996 for more on epistemic belief of learning.). From a theoretical perspective, presumably more mature beliefs about learning engage the types of processes that lead to greater acquisition of knowledge. On the other hand, naïve beliefs about learning tend not to engage these types of processes and consequently, less knowledge is acquired (e.g. Rukavina & Daneman, 1996).

Besides cognitive factors, researchers suggest that other non-cognitive factors influence individual differences in performance on the SAT (e.g. Coyle & Pillow, 2008); although the findings supporting these other factors are quite limited. Indeed, the findings related to non-cognitive factors, like social and personality factors, are frequently the secondary results of research that had other goals besides determining the non-cognitive factors that influence individual differences in SAT performance. For instance, although the goal of Robins, Lauver, Le, Davis, Langley, and Carlstron’s (2004) study was to determine whether psychosocial and study skill factors predict college outcomes, they also showed that academic self-efficacy, characterized as a self-evaluation of one’s success at academic performance, was related to ACT/SAT performance, r = .22. Theoretically speaking, self-referent thoughts/beliefs play a central role in behaviour and study skills which, in turn, influence motivation and performance (i.e. academic self-efficacy → study skills → motiva-motivation and performance). Supporting this theory are studies that show that increases in academic self-efficacy are related to increases in motivation and performance (e.g. Gore, 2006).

Performance on the SAT may also be related to one’s performance-avoidance goals, a dimension of achievement goals that is characterized as one’s desire to not perform poorly. According to Elliot and Church (1997) motive dispositions (e.g. fear of failure) influence performance-avoidance goals and these avoidance goals then negatively influence achievement-relevant behaviour (i.e. motive dispositions → performance-avoidance goals → achievement-relevant behaviour). Indeed, research suggests that one’s performance-avoidance goals is inversely related to one’s performance on an exam, such that the greater the performance avoidance goals the lower the exam performance (e.g. Elliot & Church, 1997).

It should be noted, however, that the negative influence that performance-avoidance goals exert on test performance is mediated by test anxiety (e.g. Elliot & McGregor, 1999); a social/personality factor that is typically characterized as a personality state that includes cognitive, emotional, behavioural and bodily reactions (McIllroy, Bunting, & Adamson, 2000; See Hembree, 1988 for more on the influences of test anxiety on performance and Cassady & Johnson, 2002 for cognitive test anxiety). According to Elliot and McGregor (1999), although performance-avoidance regulation involves attempting to avoid a negative outcome, in a test taking situation this type of regulation will probably elicit anxiety; especially when a test taker begins to focus on attaining normal performance in the face of possible failure (p. 629).

Finally, although little to no research has examined the relationship between locus of control and SAT performance, it would not be that surprising to observe a positive relationship between these two constructs given that locus of control influences GPA (e.g. Robbins et al., 2004) and GPA has a strong relationship with SAT performance (e.g. Coyle & Pillow, 2008; Robbins et al., 2004). Presumably students with high levels of internal locus of control understand that their own personal positive choices and behaviours result in positive outcomes (Borkowski et al., 2000).

SUMMARY AND PRESENT STUDY

In summary, previous research suggests that cognitive factors, like working memory, higher-level cognitive processes (i.e. knowledge integration) and epistemic belief of learning predict SAT performance. Additionally, there is some basis to argue that social/personality factors, like academic self-efficacy, performance-avoidance goals, test anxiety and locus of control also predict SAT performance. However, because previous research tended to examine many of these factors in isolation it is unclear whether each factor contributes uniquely to performance on the SAT. The present study addresses this shortcoming by examining the simultaneous contributions of cognitive/learning and social/personality factors to SAT performance. The value of this research is that it reveals the relative contributions of factors from each of these research areas, which in turn might lead to a comprehensive theory of SAT performance.

All the measures in the present study are frequently used and/or have good psychometric properties. For instance, as our measure of specific cognitive processes we selected Hannon and Daneman’s component processes task (2001; 2006; 2009); a task that provides estimates of a reader’s ability to learn new text-based information, to draw text-based inferences, to access prior knowledge from long-term memory and to integrate prior knowledge with new text-based information. The component process task (CPT) was selected over other tasks because its multi-component nature not only eliminates the need to administer multiple measures but it also saves time. In addition, each component process has a high degree of construct validity and reliability (e.g. Hannon & Daneman, 2001, 2006). Indeed, the internal structure of the CPT has been validated using correlations, factor analysis and structural Equation modelling. In addition, variants of Daneman and Carpenter’s (1980) reading span task and Turner and Engle’s (1989) operation span task were used as measures of working memory. These working memory measures were selected over other measures like the word span and digit span because an aggregate of more complex processing + storage measures of working memory capacity, like the reading and operation spans, accounts for more variance in SAT-V and SAT-Q performance than does the aggregate of simple storage-only measures, like the word span and digit span (e.g. Engle et al., 1999). Finally, as our measure of epistemic belief of learning we selected the version of the measure that was administered by Daneman and colleagues (e.g. Daneman & Hannon, 2001; Rukavina & Daneman, 1996) because it has been shown to correlate well with VSAT passages (e.g. Daneman & Hannon, 2001) as well as mastery of mathematics (e.g. Schommer, Crouse, & Rhodes, 1992).

With respect to our measures for the social/personality factors, we administered Elliot and Church’s (1997) measure for approach and avoidance achievement motivation because it is a short but widely used measure (i.e. only 18 items) that has high Cronbach αs (i.e. .77+).1 Furthermore, scores on this measure are also correlated with test anxiety (e.g. Elliot & McGregor, 1999); another social/personality factor that is considered in the present study.

We also administered three measures of test anxiety, namely the Sarason (1978), the Benson and El-Zahhar (1994) and the Hodapp and Benson (1997). These three measures are relatively short measures that are frequently used in studies examining test anxiety. Further each measure has been found to have high levels of validity and reliability. Indeed the Cronbach αs observed in the present study for these three measures were .86 or higher.

Finally we administered McIllroy et al. (2000) measures of academic self efficacy and academic locus of control. Both of these measures are relatively short measures (i.e. 10 items each) and have been shown to have good Cronbach αs (i.e. .713+). Furthermore, scores on these measures are also correlated with test anxiety (McIllroy et al., 2000); another social/personality factor that is considered in the present study.

METHODS

Participants

The participants were undergraduates of the University of Texas at San Antonio who received $40.00 for their participation in a large three-year study investigating the relationships among social-attitudinal beliefs, cognitive abilities and test anxiety in European-American and Hispanic students (grant # 5R24MH070636). The specific subset used in the present study were the 253 students who were native English speakers, mean age = 19.43, std = 1.71. Sixty-eight of the students were of Hispanic descent and 185 were of European descent. All students were free of any known learning disability.2

Measures

For each measure, we provide a brief description below. We also provide references of papers that describe the measures in full.

Cognitive Measures

Higher-level processes were assessed using Hannon and Daneman’s (2001, 2006, 2008, 2009) component processes task. Briefly, in this measure students study three-sentence paragraphs that describe relationships among two real and three nonsense terms; for instance: A MIRT resembles an OSTRICH but is larger and has a longer neck. A COFT resembles a ROBIN but is smaller and has a longer neck. A FILP resembles a COFT but is smaller, has a longer neck, and nests on land. Next students answer true-false statements that assess four higher-level component processes: text memory (e.g. A COFT resembles an OSTRICH.), text inferencing (e.g. A COFT is smaller than a ROBIN.), knowledge access (e.g. An OSTRICH has a longer neck than a ROBIN.), and knowledge integration (e.g. AMIRT has a longer neck than a ROBIN.). Accuracy on each type of test statement are the dependent variables of primary interest; however, a speed measure was calculated by averaging the reaction time on all correct test statements. See Hannon and Daneman (2001) for a similar approach.

Students also completed two measures of working memory, namely variants of the reading and operation span (Daneman & Carpenter, 1980; Daneman & Hannon, 2007; Turner & Engle, 1989). For each of these tasks the dependent measure was the total number of words recalled. In order to verify that each working memory measure assessed the same construct, a factor analysis with a promax rotation (correlated solution) was completed. The factor analysis confirmed that both measures loaded on the same factor. This single factor had an eigenvalue of 1.64 that accounted for 82.2% of the variance. Because both measures loaded on a single factor, a composite working memory score was calculated by summing the products of each measure’s factor loading with a student’s score for that same measure (i.e. (reading span factor loading × reading span score) + (operation span factor loading × operation span score)).

Finally, students responded to 12 items selected from two subsets of Schommer’s (1990) epistemology questionnaire (See Hannon & Daneman, 2001; Rukavina & Daneman, 1996 for an identical administration.). As mentioned earlier, for the purposes of the present study this task is called a measure of epistemic belief of learning. A sample item from this measure is the best thing about science courses is that most problems have only one solution. For each statement students identify their level of agreement using a 5-point Likert scale. Lower scores on this measure represent mature beliefs about learning whereas higher scores represent naïve beliefs.

Social/Personality Measures

Each of the measures described below were presented on a computer. For each of these measures, students selected their answer and the research assistant typed in the response. In order to protect the privacy of the participant, the research assistant could not see the computer screen.

Students completed the academic self-efficacy scale (e.g. I am confident that I can achieve good exam results if I really put my mind to it.) and the academic locus of control scale (e.g. No matter how well I prepare for my exams, I have no guarantee of being successful.) created by McIllroy et al. (2000). Each of these scales includes ten items and each item is answered with a 7-point Likert scale. A high score on the academic self-efficacy scale indicates high self-efficacy. A high score on the academic locus of control scale indicates greater internal locus of control.

In addition, students completed three measures of test anxiety: (i) Sarason’s (1978) test anxiety scale, which consists of 37 true-false statements,3 (ii) Benson and El-Zahhar’s (1994) revised test anxiety scale, which consists of 20 items that were answered using a 4-point Likert scale and (iii) Hodapp and Benson’s (1997) Measure of Test Anxiety, which consists of 21 items that were also answered using a 4-point Likert scale. A factor analysis with a promax rotation (correlated solution) confirmed that all three measures loaded on the same factor. Specifically, the single test anxiety factor had an eigenvalue of 2.58 that accounted for 85.9% of the variance. Because each measure loaded on a single factor a composite test anxiety score was calculated by multiplying each measure’s factor loading with a student’s score and then summing the three products (i.e. (Sarason factor loading × Sarason score) + (Hodapp & Benson factor loading × Hodapp & Benson score) + (Benson & Zahhar factor loading × Benson & Zahhar score)). A high score on the composite measure of test anxiety indicates greater test anxiety.

Finally, students completed Elliot and Church’s (1997) measure of achievement motivation goals. This measure includes three scales: mastery goals, performance-approach goals and performance-avoidance goals. For the purposes of this study, however, we only considered the scales for performance-approach goals and performance-avoidance goals because previous research suggests that the mastery orientation scale is not related to either exam or SAT performance. An example item from the performance-approach goals scale is It is important to me to do better than the other students and an example item from the performance-avoidance goals scale is I just want to avoid doing poorly in this class. There are six items for each scale and students select their answer for each item using a 7-point Likert scale. High scores on each of these scales indicate a greater propensity to that orientation for achievement.

SAT, V-SAT and Q-SAT scores

SAT scores were obtained from university records after students consented to their release. SAT scores were the sum of the verbal and quantitative sections (i.e. V-SAT and Q-SAT respectively).

RESULTS

Data screening, descriptive statistics and correlations

SAS was used to screen the data for (i) outliers (studentized residuals, DFITTS, DFBETAS and Cook’s D), (ii) data points that exerted excessive leverage (hat-values), (iii) linearity (bivariate scatterplots), (iv) normality (normal probability plots) and (v) multicollinearity (tolerance test via regression analysis). Preliminary regression analyses, which included all of the measures as predictors, revealed that no single data point was an outlier exerting excessive leverage. Further, inspection of bivariate scatterplots, normal probability plots and tolerance tests revealed fairly normal data that did not have excessive multicollinearity. Finally, multivariate normality was assessed using Mardia’s PK, a statistical test that is based on skew and kurtosis. Our Mardia’s PK was 1.16; a value that is well below the 1.96 limit (Tabachnick & Fidell, 2007). Thus it appears that our data has multivariate normality.

Descriptive statistics and correlations

Table 1 shows the means, standard deviations, ranges, skew and kurtosis for each of the measures and Table 2 shows the correlations among all of the measures. As Table 1 shows there is good variability for most of the predictors (i.e. large ranges) and the values for skew and kurtosis suggest that the distributions for most of the predictors do not deviate too far from normal (i.e. all values except one was under ±3). The only exception is that the kurtosis for low-knowledge access is a little high, which suggests a distribution that has a larger than expected concentration of values.

Table 1.

Means, standard deviations, ranges, skew and kurtosis for SAT, V-SAT, Q-SAT, measures of higher-level processes, working memory, epistemic belief, academic self efficacy, academic locus of control, test anxiety, performance-approach goals and performance-avoidance goals (n = 253)

M SD Range Skew Kurtosis
Measures of academic achievement
 SAT (max = 1600) 1083.36 157.42 690–1540 0.11 −0.26
 V-SAT (max = 800) 545.55 90.32 340–800 0.35 −0.16
 Q-SAT (max = 800) 537.81 87.02 320–800 0.06 −0.18
Measures of higher-level processes (i.e. component processes task)
 Text memory (max = 84) 67.74 10.87 40–84 −0.63 −0.43
 Text inferencing (max = 36) 25.97 4.29 12–34 0.02 −0.64
 Low-knowledge integration (max = 24) 20.89 2.58 13–24 −0.90 0.15
 High-knowledge integration (max = 36) 27.17 5.17 14–36 −0.42 −0.61
 Low-knowledge access (max = 36) 33.59 2.14 24–36 −1.62 3.81
 High-knowledge access (max = 24) 22.53 1.59 15–24 −1.44 2.47
  Speed (in ms) 3749.00 653.21 1978–6964 0.56 2.00
Measures of working memory
 Reading span (max = 100) 57.62 10.76 33–85 0.29 −0.58
 Operations span (max = 100) 73.11 11.95 46–98 −0.14 −0.47
 Composite working memory 0.01 1.81 −4.29–4.36 0.16 −0.62
Epistemic belief of learning (max = 60) 34.92 5.04 20–47 −0.15 −0.15
Academic self efficacy (max = 70) 52.22 7.89 29–69 −0.46 −0.07
Academic locus of control (max = 70) 54.32 7.19 30–70 −0.41 0.23
Test anxiety 93.65 24.31 54–170 0.73 0.04
Performance-approach goals (max = 7) 5.00 1.35 1–7 −0.68 −0.14
Performance-avoidance goals (max = 7) 4.50 1.18 1–7 −0.47 −0.07
Open in a new tab

Table 2.

Correlations among SAT, V-SAT, Q-SAT, measures of higher-level processes, working memory, epistemic belief, academic self efficacy, academic locus of control, test anxiety, performance-approach goals and performance-avoidance goals (n = 253)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
1. SAT — .89 .88 .35 .37 .32 .45 .29 .24 −.10 .47 −.40 .24 .06 −.41 −.04 −.39
2. V-SAT — .58 .36 .34 .33 .42 .34 .27 −.15 .47 −.42 .23 .11 −.40 −.07 −.36
3. Q-SAT — .27 .32 .23 .37 .16 .15 −.02 .35 −.28 .20 −.01 −.33 −.00 −.34
4. Text memory — .73 .68 .68 .44 .21 .09 .36 −.23 .12 .22 −.24 −.11 −.23
5. Text inferencing — .54 .60 .35 .13 −.13 .37 −.15 .09 .15 −.24 −.12 −.26
6. Low-knowledge integration — .65 .43 .15 −.05 .29 −.19 .10 .15 −.21 −.09 −.18
7. High-knowledge integration — .42 .28 .02 .33 −.23 .18 .18 −.30 −.10 −.31
8. Low-knowledge access — .38 −.04 .16 −.20 .15 .13 −.28 −.02 −.23
9. High-knowledge access — −.07 .12 −.04 .10 .08 −.19 −.03 −.21
10. Speed — −.08 .03 .02 .08 −.00 .02 .04
11. Composite working memory — −.30 .12 −.07 .20 −.03 −.20
12. Epistemic belief of learning — .21 .01 −.25 .11 −.32
13. Academic self efficacy — .47 −.50 .16 −.28
14. Academic locus of control — −.29 .08 −.15
15. Test anxiety — .22 .58
16. Performance-approach goals — .26
17. Performance-avoidance goals —
Open in a new tab

Note: For r <.13, p >.05.

As Table 2 shows both the cognitive/learning (i.e. working memory, higher-level cognitive processes and epistemic belief of learning) and social/personality measures (i.e. academic self efficacy, academic locus of control, test anxiety and performance-avoidance goals) predicted individual differences in performance on the SAT, V-SAT and Q-SAT. Indeed, the magnitude of the correlations between the measures of the cognitive/learning factors and the SAT, V-SAT and Q-SAT were highly comparable to the magnitude of the correlations between the measures of social/personality factors and the SAT, V-SAT and Q-SAT (i.e. max r = .47 versus −.41). This new finding suggests that measures of social/personality factors are just as predictive of SAT, V-SAT and Q-SAT performance as are measures of cognitive/learning factors.

Further, looking at the correlations in Table 2 it appears that the measures for working memory and the higher-level process of high-knowledge integration were the best two cognitive predictors of SAT, V-SAT, Q-SAT performance (r = .47 and .45, respectively) while the measures for test anxiety and performance-avoidance goals were the best two social/personality predictors (r = −.41 and −.39, respectively). Together, the measures of working memory and high-knowledge integration accounted for 31.4, 30.8 and 19.9% of the variance in SAT, V-SAT, and Q-SAT performance while together, the measures of test anxiety and performance-avoidance goals accounted for 21.4, 18.2 and 15.8% of the variance respectively. However, as Table 2 shows, performance-approach goals did not predict SAT, V-SATor Q-SAT performance, r ≤ −.07. Because of its poor predictive power of the SAT, V-SAT and Q-SAT, the measure for performance-approach goals was excluded as a predictor in the subsequent regression analyses.

Regression analyses

We also were interested in determining the total amount of variance in SAT, V-SAT and Q-SAT performance that was accounted for by just the cognitive/learning factors and just the social/personality factors. In order to make these determinations, we completed two sets of regression analyses. One set of regression analyses allowed the measures for the cognitive/learning factors to enter freely into three regression equations; one regression predicted SAT performance, one regression predicted V-SAT performance, and one regression predicted Q-SAT performance. The other set of regression analyses allowed the measures for the social/personality factors to enter freely into three other regression equations; again, one regression predicted SAT performance, one regression predicted V-SAT performance, and one regression predicted Q-SAT performance.

The results of these six regressions revealed that a considerable amount of variance in SAT, V-SAT and Q-SAT performance was accounted for when just the cognitive/learning factors or just the social/personality factors were considered. In other words, cognitive/learning factors accounted for a considerable amount of variance and so did social/personality factors. Specifically, the measures of the cognitive/learning factors accounted for 37.8% of the variance in overall SAT performance while the measures of the social/personality factors accounted for 21.4% of the variance. With respect to V-SAT performance, the cognitive/learning factors accounted for 41.4% of the variance while the social/personality factors accounted for 18.2% of the variance. Finally, cognitive/learning factors accounted for 21.9% of the variance in Q-SAT performance while the social/personality factors accounted for 17.3% of the variance.

Of course, we were also interested in the total amount of variance in SAT, V-SAT and Q-SAT performance that was accounted for by measures of both the cognitive/learning and the social/personality factors. Therefore, in the final three regression analyses (i.e. one for SAT performance, one for V-SAT performance and one for Q-SAT performance) we allowed measures of both the cognitive/learning and the social/personality factors to enter freely into the regression equations. Table 3 depicts the results of these three regressions.

Table 3.

Regression analyses of SAT scores, V-SAT scores and Q-SAT scores using cognitive/learning factors and social/personality factors as predictors (n = 253).

Variable R R2 ΔR2 F
(a) Overall SAT scores
1. Working memory .467 .218 .218 69.89*
2. Test anxiety .569 .324 .106 39.46*
3. High-knowledge integration .616 .380 .056 22.21*
4. Epistemic belief of learning .643 .414 .034 14.48*
5. High-knowledge access .650 .422 .008 3.52a
6. Academic locus of control .654 .428 .006 2.52a
7. Performance-avoidance goals .658 .433 .005 2.20a
(b) V-SAT scores
1. Working memory .471 .222 .222 71.42*
2. Test anxiety .564 .318 .096 35.29*
3. Epistemic belief of learning .612 .374 .056 22.36*
4. High-knowledge integration .640 .409 .035 14.84*
5. High-knowledge access .654 .428 .019 7.99*
6. Speed .663 .440 .012 5.45*
7. Low-knowledge access .668 .446 .005 2.49a
(c) Q-SAT scores
1. High-knowledge integration .372 .138 .138 40.10*
2. Working memory .446 .199 .061 18.99*
3. Performance-avoidance goals .495 .245 .046 15.15*
4. Test anxiety .507 .257 .012 4.15*
5. Academic locus of control .523 .273 .016 5.55*
6. Academic self efficacy .529 .280 .007 2.41a
Open in a new tab
*

p <.05;

a

p <.15.

As Table 3 reveals, measures from each set of factors accounted for variance in SAT, V-SAT and Q-SAT performance. Further, the total amount of variance accounted for by both sets of factors was considerable. Specifically, when measures of both cognitive/learning and social/personality factors were combined within a single regression analysis three cognitive/learning factors (i.e. working memory, high-knowledge integration and epistemic belief of learning) and one social/personality factor (i.e. test anxiety) accounted for 43.4% of the variance in SAT performance. This 43.4% variance is greater than either the 37.8% variance accounted for by just the cognitive/learning factors or the 21.4% variance accounted for by just the social/personality factors.

With respect to V-SAT performance, five cognitive/learning factors (i.e., working memory, epistemic belief of learning, high-knowledge integration, high-knowledge access and speed) and one social/personality factor (i.e., test anxiety) accounted for 44.6% of the variance in V-SAT performance. This finding is analogous to that of the regression analysis for SAT performance inasmuch as the 44.6% variance in V-SAT performance is greater than either the 41.4% variance accounted for by just the cognitive/learning factors or the 18.2% variance accounted for by just the social/personality factors.

Finally, two cognitive/learning factors (i.e. high-knowledge integration and working memory) and three social/personality factors (i.e. performance-avoidance goals, academic locus of control, and academic self efficacy) accounted for 28.0% of variance in Q-SAT performance. This 28.0% variance is larger than the 21.9% variance accounted for by just the cognitive/learning factors as well as the 17.3% variance accounted for by just the social/personality factors.

As a final observation we would like to point out that three measures were consistently significant predictors in all three regressions (i.e. the regression for SAT performance, the regression for V-SAT performance, and the regression for Q-SAT performance). Two of these measures were the cognitive/learning factors of working memory and high-knowledge integration while the third measure was the social/personality factor of test anxiety.

DISCUSSION

This study identifies a number of sources of individual differences in SAT performance by examining the simultaneous contributions of factors from two otherwise disparate research areas, namely cognition/learning and social/personality. As a starting point, the zero-order correlations showed that measures of social/personality factors were just as predictive of SAT performance as were measures of cognitive/learning factors. In addition, the regression analyses revealed that the simultaneous contributions of both measures of cognitive/learning factors and measures of social/personality factors accounted for more variance in SAT, V-SAT and Q-SAT performance than did just the measures of cognitive/learning factors or just the measures of social/personality factors. Finally, the regression analyses revealed that working memory, high-knowledge integration, and test anxiety were unique and consistent predictors of SAT performance, V-SAT performance and Q-SAT performance. Because of this uniqueness and consistency, theories of the SAT and academic performance should include these three factors.

The present findings are consistent with previous research that suggests that some of the shared variance between SAT performance and GPA is a consequence of non-cognitive factors (e.g. Coyle & Pillow, 2008). Specifically, in Coyle and Pillow’s (2008) study a significant amount of shared variance between SAT performance and GPA was not accounted for by general cognitive abilities (as measured by a battery of cognitive tasks). Consistent with Coyle and Pillow’s suggestion, the present study showed that non-cognitive factors, like test anxiety, academic self-efficacy, academic locus of control and performance-avoidance goals account for unique variance in SAT performance. Of course, the present study did not examine the shared variance between SAT performance and GPA like Coyle and Pillow. However, given that non-cognitive factors, like the ones tested in the present study, do account for variance in SAT, V-SAT and Q-SAT performance, it certainly is possible that these factors also account for some of the variance that is shared between SAT performance and GPA. For this reason, one avenue for future research might be to consider the non-cognitive factors investigated in the present study as potential predictors of some of this shared variance.

There are also other non-cognitive factors that might account for unique variance in SAT performance that were not tested for in the present study. Coyle and Pillow (2008), for instance, suggested (i) ability tilt, which is the performance difference between mathematical versus verbal measures and (ii) personality factors, such as openness and conscientiousness, which have been shown to correlate with college grades. Research also shows that a student’s approach to achievement, characterized as a student’s desire to achieve high scores for the sake of appearances, predicts SAT performance (Rose, Hall, Bolen, & Webster, 1996).

Of course the present study also has a number of limitations. One obvious limitation is that we used a population of students at one university rather than several populations at multiple universities. A second limitation is that for most of the predictors there was only one measure rather than multiple measures. And so, because of this limitation it is unclear whether the present findings are limited to just those measures used in the present study. A third limitation is that although the present study reveals a number of unique predictors of the SAT it does not establish causality; after all the DV SAT performance was measured before any of the predictors in the present study were measured. Finally, future research should consider the relationships among the predictors. For instance, future research might wish to model the relationships among the cognitive/learning and social/personality factors in order to provide a theoretical framework for describing SAT performance.

In summary, the present study showed that both cognitive/learning and non-cognitive social/personality factors contribute to unique variance in performance on the SAT, V-SAT, and Q-SAT. Indeed, the cognitive factors of working memory and high-knowledge integration as well as the social/personality factor of test anxiety predicted both global SAT performance as well as performance on the two subtests, the V-SAT and Q-SAT.

Acknowledgments

This research was funded by a NIMH grant to Dr. Brenda Hannon and Dr. Mary McNaughton-Cassill. We thank Jill Argus, Sarah Frias, Veronica Torres, Laura Flores, and Elaine Tamez for their invaluable assistance with booking participants, running participants and scoring data.

Footnotes

1

The Elliot-Church measure of approach and avoidance achievement motivation goals has been updated; see Elliot and Murayama (2008). This new updated version was not available when the data for the present study were collected; however, the updated version is highly similar to the Elliot-Church measure used in the present study.

2

All participants were first pre-screened in order to assess availability and qualifications; for example, age, gender, ethnicity, dominant language, physical disabilities that might impede learning, or learning disabilities such as ADHD or dyslexia. A few weeks after this pre-screening those participants that were dominant English speakers who were free of any known learning disability were invited to participate.

3

Although the standard version of Sarason’s (1978) test anxiety measure is composed of 37 statements because of a computer malfunction students only answered 36 statements.

References

  1. Benson J, El-Zahhar N. Further refinement and validation of the revised test anxiety scale. Structural Equation Modeling. 1994;1:203–221. [Google Scholar]
  2. Borkowski JG, Carr M, Rellinger L, Pressley M. Self-regulated cognition: Interdependence of metacognition, attributions, and self-esteem. In: Jones B, Idol L, editors. Dimensions of thinking and cognitive instruction. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc; 1990. pp. 53–92. [Google Scholar]
  3. Borkowski JG, Chan IKS, Muthukrishna N. A process orientated model of metacognition: Links between motivation and executive functioning. In: Schraw G, Impara J, editors. Issues in the measurement of metacognition. Lincoln, NE: Buros Institute of Mental Measurements, University of Nebraska; 2000. pp. 1–42. [Google Scholar]
  4. Cassady JC, Johnson RE. Cognitive test anxiety and academic performance. Contemporary Educational Psychology. 2002;27:270–295. [Google Scholar]
  5. Cloud J. Should SATs matter? Time. 2001;157:62–76. [Google Scholar]
  6. Coyle TR. Test-retest changes on scholastic aptitude tests are not related to g. Intelligence. 2006;34:15–27. [Google Scholar]
  7. Coyle TR, Pillow DR. SAT and ACT predict college GPA after removing g. Intelligence. 2008;36:719–729. [Google Scholar]
  8. Credé M, Kuncel NR. Study habits, skills, and attitudes: The third pillar supporting collegiate academic performance. Perspectives on Psychological Science. 2008;3:425–453. doi: 10.1111/j.1745-6924.2008.00089.x. [DOI] [PubMed] [Google Scholar]
  9. Daneman M, Carpenter PA. Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behaviour. 1980;19:450–466. [Google Scholar]
  10. Daneman M, Carpenter PA. Individual differences in integrating information between and within sentences. Journal of Experimental Psychology: Learning, Memory, and Cognition. 1983;9:561–584. [Google Scholar]
  11. Daneman M, Hannon B. Using working memory theory to investigate the construct validity of multiple-choice reading comprehension tests such as the SAT. Journal of Experimental Psychology: General. 2001;130:208–223. doi: 10.1037//0096-3445.130.2.208. [DOI] [PubMed] [Google Scholar]
  12. Daneman M, Hannon B. What do memory spans like reading span really measure? In: Osaka N, Logie RH, D’Espisito M, editors. The cognitive neuroscience of working memory: Behavioral and neural correlates. New York: Oxford University Press; 2007. pp. 21–42. [Google Scholar]
  13. Daneman M, Merikle PM. Working memory and language comprehension: A meta-analysis. Psychonomic Bulletin and Review. 1996;3:422–433. doi: 10.3758/BF03214546. [DOI] [PubMed] [Google Scholar]
  14. Dodrill CB, Warner MH. Further studies of the Wonderlic Personnel Test as a brief measure of intelligence. Journal of Consulting and Clinical Psychology. 1988;56:145–147. doi: 10.1037//0022-006x.56.1.145. [DOI] [PubMed] [Google Scholar]
  15. Elliot JA, Church MA. A hierarchical model of approach and avoidance achievement motivation. Journal of Personality and Social Psychology. 1997;72:218–232. doi: 10.1037//0022-3514.76.4.628. [DOI] [PubMed] [Google Scholar]
  16. Elliot JA, McGregor H. Test anxiety and the hierarchical model of approach and avoidance achievement motivation. Journal of Personality and Social Psychology. 1999;74:628–644. doi: 10.1037//0022-3514.76.4.628. [DOI] [PubMed] [Google Scholar]
  17. Elliot AJ, Murayama K. On the measurement of achievement goals: Critique, illustration, and application. Journal of Educational Psychology. 2008;100:613–628. [Google Scholar]
  18. Engle RW, Tuholski SW, Laughlin JE, Conway ARA. Working memory, short-term memory, and general fluid intelligence: A latent-variable approach. Journal of Experimental Psychology: General. 1999;128:309–331. doi: 10.1037//0096-3445.128.3.309. [DOI] [PubMed] [Google Scholar]
  19. Frey MC, Detterman DK. Scholastic assessment or g? The relationship between the scholastic assessment test and general cognitive ability. Psychological Science. 2004;15:373–378. doi: 10.1111/j.0956-7976.2004.00687.x. [DOI] [PubMed] [Google Scholar]
  20. Gore P. Academic self-efficacy as a predictor of college outcomes: Two incremental validity studies. Journal of Career Assessment. 2006;14:92–115. [Google Scholar]
  21. Hannon B, Daneman M. A new tool for measuring and understanding individual differences in the component processes of reading comprehension. Journal of Educational Psychology. 2001;93:103–128. [Google Scholar]
  22. Hannon B, Daneman M. What do tests of reading comprehension ability such as the VSAT really measure?: A componential analysis. In: Mittel AV, editor. Focus on educational psychology. New York, NY: Nova Science Publishers Inc; 2006. pp. 105–146. [Google Scholar]
  23. Hannon B, Daneman M. Individual differences in cognitive abilities: A component processes account. Poster presented at International Congress; Berlin, Germany. July.2008. [Google Scholar]
  24. Hannon B, Daneman M. Age-related changes in reading comprehension: An individual-difference perspective. Experimental Aging Research. 2009;35:432–456. doi: 10.1080/03610730903175808. [DOI] [PubMed] [Google Scholar]
  25. Hembree R. Correlates, causes, and treatment of test anxiety. Review of Educational Research. 1988;58:47–77. [Google Scholar]
  26. Hodapp V, Benson J. The multidimensionality of test anxiety: A test of different models. Anxiety, Stress, and Coping. 1997;10:219–244. [Google Scholar]
  27. McIllroy D, Bunting B, Adamson G. An evaluation of the factor structure and predictive utility of a test anxiety scale with reference to students’ past performance and personality indices. British Journal of Educational Psychology. 2000;70:17–32. doi: 10.1348/000709900157949. [DOI] [PubMed] [Google Scholar]
  28. Robbins SB, Lauver K, Le H, Davis D, Langley R. Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin. 2004;130:261–288. doi: 10.1037/0033-2909.130.2.261. [DOI] [PubMed] [Google Scholar]
  29. Rose RJ, Hall CW, Bolen LM, Webster RE. Locus of control and college students approaches to learning. Psychological Reports. 1996;79:163–171. [Google Scholar]
  30. Rukavina I, Daneman M. Integration and its effect on acquiring knowledge about competing scientific theories from text. Journal of Educational Psychology. 1996;88:272–287. [Google Scholar]
  31. Sarason IG. Test anxiety, stress, and social support. Journal of Personality. 1978;49:101–114. [Google Scholar]
  32. Schommer M. Effects of beliefs about the nature of knowledge on comprehension. Journal of Educational Psychology. 1990;82:498–504. [Google Scholar]
  33. Schommer M, Crouse A, Rhodes N. Epistemological beliefs and mathematical text comprehension: Believing it is simple doesn’t make it so. Journal of Educational Psychology. 1992;84:435–443. [Google Scholar]
  34. Tabachnick BG, Fidell LS. Using multivariate statistics. 5. New York, NY: Pearson; 2007. [Google Scholar]
  35. Turner ML, Engle RW. Is working memory capacity task dependent? Journal of Memory and Language. 1989;28:127–154. [Google Scholar]
  36. Zeidner M. Test anxiety: The state of the art. New York, NY: Plenum Press; 1998. [Google Scholar]
  37. Zwick R, Green JG. New perspectives on the correlation of SAT scores, high school grades, and socioeconomic factors. Journal of Educational Measurement. 2007;44:23–45. [Google Scholar]

RESOURCES