Assessment of Adolescents’ Motivation for Educational Attainment

Abstract

The Adolescent Motivation for Educational Attainment Questionnaire is a 32-item questionnaire (we drew 20 items from 3 subscales of the Educational Motivation Questionnaire; Murdock, 1999) that was developed to measure multiple potential dimensions of adolescents’ motivation to complete high school and enroll in post-secondary education, including competence and effort beliefs; perceived value of education; and peer, teacher, and parent support for educational attainment. We assessed a multiethnic sample (N = 569) of low-achieving students who started 1st grade together in 1 urban and 2 small city school districts. Participants were assessed over 2 consecutive years (Grades 8 and 9 given prior grade retention, or Grades 9 and 10 if not retained). Exploratory factor analyses identified 4 correlated dimensions underlying the questionnaire responses. Subsequent confirmatory factor analyses provided support for a bifactor model, which includes a general factor of students’ basic educational motivation, and specific factors of (a) teacher educational expectations, (b) peer aspirations, and (c) value of education. Measurement invariance of the bifactor model was established across students’ gender and ethnicity (Caucasian, African American, and Hispanic) and year of testing. Criterion-related validity of the general and specific factors with students’ school belonging, student–teacher warmth and conflict, disciplinary practices, letter grade, conduct problems, and behavioral engagement was examined. Practical implications of the measure are discussed.

A substantial proportion of students who begin high school do not earn a high school diploma. In 2009, 8.1% of 18- through 24-year-olds had not received a high school diploma or alternative credential and were not currently enrolled in high school (Chapman, Laird, & KewalRamani, 2011). Individuals who lack a high school degree are far more likely to spend their lives periodically unemployed, on government assistance, or cycling in and out of prison. Dropouts also have worse health than adults who complete high school (Pleis, Ward, & Lucas, 2010). Over his or her lifetime, each individual who drops out of school is estimated to cost the nation approximately $260,000 in lost taxes and increased government spending (Alliance for Excellence in Education, 2007). Of those individuals who do complete high school or an alternative credential, many do not pursue formal post-secondary education, greatly reducing their opportunities for economic mobility (Haskins, Holzer, & Lerner, 2009). The focus of this article is to develop and evaluate a measure designed to predict early indicators of high school dropout and ultimately high school dropout itself.

Consistent with ecological theories, empirical evidence suggests that individual, social, family, cultural, and school factors operate in dynamic ways across time to affect students’ interest in and identification with academic success (Janosz, Archambault, Morizot, & Pagani, 2008; Rumberger & Thomas, 2000; Skinner, Furrer, Marchand, & Kindermann, 2008). Dropping out of school before graduating is viewed as a gradual process of increasing disengagement and alienation from school that begins in the elementary school years and increases in secondary school (Alexander, Entwisle, & Horsey, 1997; Janosz et al., 2008). Although academic failure is the strongest predictor of dropping out of school (Battin-Pearson et al., 2000), some students who are making adequate academic progress leave school prematurely (Alexander, Entwisle, & Dauber, 2003). Drawing from diverse theoretical perspectives, motivational theorists have identified a number of factors that energize and direct students’ educational effort and aspirations (for reviews, see Eccles & Roeser, 2009; Pintrich, 2003).

Conceptualization of Motivation for Educational Attainment

Why do some students strive to continue to meet the demands of graduating, persisting in academic tasks in the face of challenges, choosing to invest their energies and time in academic tasks rather than in other activities? Academic motivation and the factors that give rise to it have been the topic of extensive investigation for decades (e.g., Deci & Ryan, 2000; Pintrich, 2003; Wigfield, Cambria, & Eccles, 2012). Evidence accumulated over the past several decades (e.g., Deci & Ryan, 2000; Pintrich, 2003; Wigfield et al., 2012) documented the critical role of students’ self-system beliefs (e.g., perceptions of academic ability, value of academic achievement, and academic goals) that underlie students’ achievement-related behaviors, such as effort and persistence in a course and choice in coursework. We expect these self-system beliefs are also relevant to academically at-risk students’ motivation to attain a high school diploma and to pursue post-secondary education. These self-system variables are shaped in important ways by students’ relationships with key social agents (Deci & Ryan, 1991; Eccles, 1993; Legault, Green-Demers, & Pelletier, 2006; Skinner et al., 2008). Interactions with these social partners provide critical scaffolding of the students’ motivation that helps maintain positive self-system beliefs (academic competence and effort beliefs, valuing of educational attainment, and educational aspirations) that are proximal influences on achievement behaviors. Although motivational context is traditionally considered a “cause” of self-system beliefs (Eccles, 1993), contemporary theorists view context and beliefs as influencing each other over time, as part of a dynamic system (Legault et al., 2006; Wigfield, Eccles, Schiefele, Roeser, & Davis-Kean, 2006). For example, students with high academic motivation, who value achievement and make good grades, select academically motivated friends who support and encourage their academic goals (Altermatt & Pomerantz, 2005; Kindermann, 2007; Kiuru, Aunola, Vuori, & Nurmi, 2007). With respect to teacher–student relationships, a cross-lag panel study across Grades 1–3 found that student behavioral engagement in the classroom and teacher support for students’ learning influenced each other in a reciprocal manner (Hughes, Luo, Kwok, & Loyd, 2008).

The overall purpose of this study is to investigate the psychometric properties and the structure of a relatively brief, multidimensional measure of secondary students’ motivational context and beliefs relevant to completion of high school and pursuit of post-secondary education. Because prior research finds that dimensions of educational motivation and motivational context are correlated (Eccles & Wigfield, 1995; Legault et al., 2006; Murdock, 1999), it is important to understand whether correlated dimensions or a general dimension of motivation for educational attainment best captures the commonality of educational motivation dimensions. Our study has an important applied purpose: A psychometrically strong measure of motivation for educational attainment administered to students at the beginning of high school is key to efforts to identify students at highest risk of not completing high school for selection into preventive interventions.

Supporting Educational Attainment: Dimensions of Motivational Beliefs and Their Scaffolding

Competence and Effort Beliefs

Self-efficacy theory (Bandura, 1986), expectancy value theory (Wigfield, 1994), and self-determination theory (Skinner, Wellborn, & Connell, 1990) all posit a key role for academic self-efficacy beliefs in school success. Self-efficacy refers to whether a person perceives himself or herself as capable of mobilizing and maintaining effort needed to achieve a goal (Bandura, 1986). Extensive research has explored the role of general and domain-specific self-efficacy beliefs in self-regulated learning and achievement (see Denissen, Zarrett, & Eccles, 2007; Pajares, 1996; Wigfield et al., 2006; Zimmerman, Bandura, & Martinez-Pons, 1992). When students hold positive expectations about their ability to successfully execute academic tasks, they exert greater effort, withstand challenges and setbacks, and achieve more, relative to students with lower academic expectations (Bandura, Barbaranelli, Caprara, & Pastorelli, 1996; Wigfield & Eccles, 1994). Conceptually, an individual’s perceived academic competence can be distinguished from one’s perceived efficacy for self-regulated learning (i.e., ability to plan and organize academic activities; Patrick, Skinner, & Connell, 1993; Skinner et al., 1990; Zimmerman & Clearly, 2006). Both academic competence beliefs and academic self-regulatory beliefs predict adolescents’ intention to complete school and actual dropping out of school (Alivernini & Lucidi, 2011; Caprara et al., 2008; Legault et al., 2006).

Value of Education

Both self-efficacy theory and expectancy value theory posit that perceived competence is not sufficient to energize and direct behavior; one must also expect that the behavior will result in outcomes that are valued or desired (Bandura, 1986; Eccles, O’Neill, & Wigfield, 2005). Students who value educational attainment and believe that academic success will result in social or economic benefits work harder in school, make higher grades, and expect to attain higher levels of education (Colón & Sánchez, 2010; Murdock, 1999; for review, see Eccles et al., 2005). Importantly, in a study of different aspects of adolescent lack of motivation, Legault et al. (2006) found that students’ valuing of education predicted students’ reported intention to drop out of school. Similarly, Eccles and colleagues have found that both competence beliefs and values predict career choices (see Eccles, 2005).

Peer, Teacher, and Parent Support for Educational Attainment

According to self-determination theory (Deci & Ryan, 1991) and social motivation theory (Furrer & Skinner, 2003), social support for academic success is a key component of students’ educational motivation. Extensive research indicates the importance of social support for education from different relational partners (parents, peers, teachers) on academic effort and outcomes across Grades 1–12 (Bokhorst, Sumter, & Westenberg, 2009; Cole, Maxwell, & Martin, 1997; Furrer & Skinner, 2003; Murdock, 1999). In adolescence, peer, teacher, and parent support for academic success each contribute to students’ academic motivation (Cole et al., 1997; Legault et al., 2006; Murdock, 1999; Vitoroulis, Schneider, Vasquez, del Pilar Soteras de Toro, & Gonzales, 2012). As adolescence proceeds, youths report increasing reliance on social support from peers, with the importance of peer support exceeding that of parent support among 16- to 18-year-olds (Bokhorst et al., 2009). Academically successful adolescents who hold high educational expectations tend to affiliate with peers who are academically successful and motivated (Kiuru et al., 2007). Having academically motivated friends predicts increased academic effort and achievement (Altermatt & Pomerantz, 2005; Vitoroulis et al., 2012), presumably due to peer approval and reinforcement for academic behaviors and the perceived pressure to conform to peers’ expectations, as well as the provision of support with academic tasks. In a large national study of students in Grades 9–11, persistence in math course taking was associated with the achievement of their close friends (Crosnoe, Riegle-Crumb, Field, Frank, & Muller, 2008).

Parental educational expectations also matter. In adolescence, parent educational expectations are more predictive of students’ academic performance than are other forms of parental involvement (You & Nguyen, 2011). Furthermore, adolescents’ perceptions of parents’ educational expectations are highly predictive of their educational motivation and aspirations. For example, in a sample of African American adolescents, the relation between parent educational expectations and youths’ educational aspirations and expectations was mediated by youths’ perception of parents’ expectations (Wood, Kurtz-Costes, & Copping, 2011). Tynkkynen, Tolvanen, and Salmela-Aro (2012) found that adolescents’ perceptions of their parents’ educational expectations for them predicted their own 5-year trajectories for educational expectations.

Although students report declining levels of social support from teachers as they transition from elementary to middle school (Bokhorst et al., 2009), adolescents’ perceptions of teachers’ educational expectations and support for their autonomy continue to be important throughout secondary school (Roeser & Eccles, 1998; Skinner et al., 2008; Wang & Holcombe, 2010). Interestingly, in a study of adolescents’ perceptions of competence support from teachers, parents, and peers, only perceived teacher competence support (i.e., educational expectations) predicted students’ academic self-efficacy beliefs (Legault et al., 2006).

Structure of Motivational Beliefs and the Context for Educational Attainment

Although there is considerable consensus that educational motivation is a multi-dimensional construct, motivational dimensions have typically been studied separately (Caprara et al., 2008; Vitoroulis et al., 2012; Wentzel, 1998). Defining and examining the dimensions of motivation separately may not attend to the distinctiveness of constructs or to the commonality or interrelationships among constructs. Notable prior efforts to identify dimensions of educational motivation in adolescence include those by Eccles and Wigfield (1995); Legault et al. (2006); Murdock (1999); and Vallerand, Pelletier, Blais, and Brière (1992). However, none of these scales assesses both competence and effort beliefs as well as the motivational context. Furthermore, only Murdock’s scale includes items pertaining specifically to motivation to complete high school. In a study of middle school students, Murdock investigated the relationships among four motivational context factors (i.e., perceived teacher educational expectations, teachers’ emotional support, peers’ educational aspirations, and the economic value of educational success) and teacher-rated engagement and behavioral conduct. Perceptions of teachers’ educational expectations and peers’ educational aspirations uniquely predicted engagement, over and above socioeconomic status. Although Murdock’s findings support the role of teacher and peer academic support for school engagement, she did not test the dimensionality or structure of the four scales, and she did not include measures of perceived academic competence beliefs or educational aspirations. The degree to which each of these constructs contribute jointly through a more general underlying construct versus uniquely to school engagement remains an open question.

The aforementioned multi-dimensional measures of motivational beliefs and context find that dimensions of motivation, including motivational beliefs and contexts, are moderately to strongly correlated. For example, scales assessing beliefs concerning the value of academic achievement and competence or effort beliefs correlate between .38 and .77 (Eccles & Wigfield, 1995; Legault et al., 2006). There has long been discussion of how best to represent highly correlated multi-dimensional constructs. One traditional approach captures the correlations of each construct with the criteria, ignoring the correlations between pairs of predictors that may represent a more general factor. Another traditional approach uses multiple regression to identify each construct’s unique prediction of the criteria, again de-emphasizing prediction from a common factor underlying the set of constructs. A final traditional method aggregates across constructs, focusing on a composite score as a predictor of the criteria that represents the common factor, but losing information about unique prediction from each of the individual factors (see Carver, 1989; Hull, Lehn, & Tedlie, 1991). In the present study, we consider three different factor models that may be used to represent the hypothesized motivation of educational attainment construct (Brunner, Nagy, & Wilhelm, 2012; F. F. Chen, West, & Sousa, 2006). These three models have recently been used in several research areas to help represent the structure of multi-dimensional constructs and clarify their general and unique contributions to the prediction of important criteria.

Panel A in Figure 1 presents the correlated-factor model in which each of the educational motivation dimensions are considered to be separate, correlated factors (labeled η₁–η₄). Panel B presents the second-order factor model in which at the topmost (second) level, each of the four specific educational motivation factors (η₁–η₄) loads on a general educational motivation factor (ξ). At the lower level, each item loads on its corresponding specific educational motivation construct. Each pair of specific educational motivational factors (η) is not directly correlated; they are only indirectly related through the general factor. Panel C presents the bifactor model in which each item loads on a general educational motivation factor (ξ). In addition, each item also loads on its corresponding specific educational motivation factor. Each of the specific factors is uncorrelated with the general factor and the other specific factors (Rindskopf & Rose, 1988). Although not obvious, the second-order factor model is nested within the bifactor model given certain model constraints (F. F. Chen et al., 2006; Yung, Thissen, & McLeod, 1999). In the second-order factor and bifactor models, the general factor captures the commonality of educational motivation dimensions. Each specific factor represents the specific effect of the corresponding items over and above the general factor or other specific factors. The second-order and bifactor models offer one resolution to the classic dimensionality problem of whether to represent highly correlated constructs as a general factor or specific, correlated factors. The bifactor model is particularly attractive because of its clear interpretation: It partitions the relationships into a single general factor plus specific factors representing the independent contribution of each construct over and above the general factor.

Figure 1.

Illustrative path diagrams of (A) correlated-factor, (B) second-order factor, and (C) bifactor models. Y₁–Y₁₂ are 12 items; η₁–η₄ are the four factors.

Development of Adolescent Motivation for Educational Attainment Questionnaire

We sought to develop a measure of high school students’ motivation for completing high school and enrolling in post-secondary education. One goal was to assess pertinent aspects of the hypothesized motivation for educational attainment, which we define as motivation to persist in high school and post-secondary education and not drop out. Of particular importance are three dimensions related to the students’ competence and effort beliefs and their scaffolding: (1) students’ academic competence and effort beliefs, (2) perceptions of the utility of educational attainment, and (3) perceived academic support from teachers, parents, and peers for educational attainment. A second goal was to keep the measure relatively brief so that it could be used in practice. For our initial scale, we drew 21 items from three subscales of the Educational Motivation Questionnaire (Murdock, 1999) that had the strongest relation to teachers’ reports of student behavioral engagement and that assess students’ perceived economic value of an education (related to Dimension 2), teachers’ competence support (related to Dimension 3), and peers’ educational aspirations (related to Dimension 3). We further developed 15 new items to assess other pertinent aspects of the hypothetical dimensions: (a) students’ perceived competence and effort specifically for graduation from high school and enrollment in post-secondary education (nine items; related to Dimension 1), (b) students’ subjective economic valuing of a high school diploma (two items; related to Dimension 2), (c) students’ perceived parent educational expectations and valuing of education (two items; related to Dimension 3), and (d) friends’ expectations for high school graduation and post-secondary attainment (two items; related to Dimension 3).

Study Purpose

The overall purpose of this study is to test the dimensionality, structure, measurement invariance, and construct validity of a measure of adolescents’ motivation to complete high school and pursue post-secondary education. The availability of a relatively brief measure of motivation for high school completion that has demonstrated good psychometric properties could assist schools’ efforts to identify students early in their high school careers who are at greatest risk of not earning a high school diploma. We pursue these aims with an ethnically diverse sample of students who were originally selected because they were educationally at risk in first grade. For these low achieving students, high school graduation and post-secondary education represent particularly challenging, but important life milestones. Based on prior research, we expected to identify factors pertaining to students’ competence beliefs related to high school completion and post-secondary education, valuing of educational attainment, and social support. With respect to social support, based on the research finding that peers and teachers provide different types of social support for academic attainment (Furman & Buhrmester, 1992; Legault et al., 2006; Murdock, 1999), we expected that items assessing perceptions of one’s peers’ educational aspirations and items assessing perceptions of teachers’ educational expectations would load on separate factors. Because the parent educational support items assessed both perceived valuing of education and expectations for high school and postsecondary education, we did not have a strong basis for expecting them to load on a single factor or to be distributed across other factors.

We began with exploratory factor analysis (EFA) to determine the number of factors and the tentative (configural) pattern of the factor loadings. We randomly split our sample into two halves, conducted parallel EFAs on each half, and compared the two solutions. Based on the EFA results, confirmatory factor analyses (CFAs) were conducted on the full sample to determine the factor structure of the Adolescent Motivation for Educational Attainment Questionnaire. We considered correlated-factor, second-order factor, and bifactor CFA models. We sought to cross-validate the CFA results by examining the longitudinal measurement invariance of the solution across 2 years of the study. Following the recommendation of Knight and Zerr (2010), we also examined measurement invariance across students’ gender and ethnicity. Establishing measurement invariance provides the essential foundation for the direct comparison of means, variances, and covariances of latent constructs across ethnic and gender groups.

Finally, we collected several criterion measures as a first step in establishing the validity of the measure. Importantly, criterion measures included not only other self-report measures, but also key teacher report and institutional measures for this at risk sample including grades and disciplinary actions that would not be affected by potential method bias associated with self-report. We measured criterion measures in middle school or the beginning of high school that have been found in previous research to predict student intention to drop out or actual dropping out of high school. Specifically, student-reported liking for and sense of inclusion and support at school predict subsequent completion of high school (Barile et al., 2012; Fall & Roberts, 2012; Ream & Rumberger, 2008). Conduct problems also predict dropping out of school (Archambault, Janosz, Fallu, & Pagani, 2009; Fall & Roberts, 2012; Finn & Rock, 1997; Newcomb et al., 2002; Rumberger & Larson, 1998). One of the strongest predictors of non-school completion is involvement in behaviors that result in school exclusionary sanctions, such as in-school and out-of-school suspensions (Finn & Rock, 1997) and arrests (Finn & Rock, 1997; Hirschfield, 2009). Low behavioral engagement in learning (i.e., lack of academic effort and persistence) predicts not only student-reported intention to drop out of school (Caprara et al., 2008) but also actual dropping out of school (Fall & Roberts, 2012; Finn & Rock, 1997). Academic failure—defined as poor performance on standardized achievement tests (Fall & Roberts, 2012), low grades (Bowers & Sprott, 2012; Suh & Suh, 2007), or grade retention (Goldschmidt & Wang, 1999; Rumberger & Larson, 1998)—is highly predictive of failure to complete high school. In a large, longitudinal study that examined a range of factors predicting drop out, academic performance (as measured by achievement test scores and grade point average [GPA] at Grade 8) was the most important unique predictor of dropping out of school (Newcomb et al., 2002). Finally, as suggested by an anonymous reviewer, given the results of measurement invariance of the measure across gender, we investigated gender differences on the measure. Consistent with prior research reporting that high school girls, relative to boys, show higher academic competence, sense of school belonging, and effortful engagement in learning (Benner & Graham, 2009; Caprara et al., 2008; Gillen-O’Neel & Fuligni, 2013) and are more likely to complete high school (Chapman et al., 2011), we expected that girls would obtain higher scores on the measure of motivation for educational attainment.

Method

Participants

Participants were drawn from a sample of students recruited into a larger longitudinal study. Participants in the longitudinal study were originally recruited when they were in first grade in the Fall of 2000 or 2001. Data for the current study were collected during Years 9 and 10 of the larger study. Participants in the current study were 569 students at Years 9 and 10. In the original longitudinal sample, 784 students were enrolled in one of three school districts (one urban and two small city districts) in the South central United States and were selected into the study on the basis of scoring below the median on a district-administered test of literacy administered in the spring of their kindergarten year or the fall of their first grade year. Additional inclusionary criteria for the larger study included speaking English or Spanish, not receiving special education other than speech and language services, and not having been previously retained in first grade. More details on recruitment of the 784 participants are reported in Hughes and Kwok (2006).

At the end of the first 5 years of participation in the study, parental consent for continued participation was received for 569 participants. Almost all non-consent was due to parental non-response despite repeated attempts to contact the parents. Relative to active participants, the attrited participants were more likely to have been enrolled in a bilingual class, were more likely to be Hispanic or “other ethnicities” rather than African American or Caucasian, and scored lower on math achievement at Year 1, but they did not differ on a wide range of other variables (e.g., literacy scores, reading achievement, school engagement, economically disadvantaged status, parent education level). Among these 569 participants, there were 54.7% males; the ethnic composition was 35.1% Caucasian, 24.6% African American, 36.2% Hispanic, and 4.0% other ethnicities. At Year 1, 60.4% of participants received free or reduced lunch, and 89.3% of parents had a high school degree or above. At Year 9, participants were enrolled in 107 schools in 77 school districts. At Year 10, participants were enrolled in 115 schools in 79 school districts. At Year 9, 63.3% students were enrolled in Grade 9, 29.4% in Grade 8, and 0.9% in Grade 7. At Year 10, 58.8% students were enrolled in Grade 10, 34.2% in Grade 9, and 1.8% in Grade 8.

Of these 569 students, 491 and 509 students had data on the Adolescent Motivation for Educational Attainment Questionnaire at Years 9 and 10, respectively. We probed the type of missingness using Little’s (1988) missing completely at random test. We compared the means of the Adolescent Motivation for Educational Attainment Questionnaire items, students’ gender and ethnicity, and our criterion-related validity measures at Years 9 and 10 across all missing data patterns. Little’s test was not significant, Δχ²(604) = 586.06, p = .69, consistent with the interpretation that the data were missing completely at random so that no adjustment in the results would be needed.

Procedures

At Year 9, students were administered study measures in individual sessions conducted at school between October and May. Two student report criterion measures (perceived school belonging and conduct problems) were administered at Year 9 but not Year 10. At Year 10, students were given the option of completing an online or a paper version of the questionnaire, with 18.3% completing the online version and 81.7% completing the paper version. The two versions did not differ on the educational motivation items (mean absolute Cohen’s, 1988, d = 0.08).

Each year in the spring semester, teachers completed questionnaires, which included questions regarding students’ behavioral engagement. Although it would have been preferable to have all of a student’s teachers complete the questionnaire, we did not have the financial resources to do so. The student’s language arts teacher was asked to complete the mailed questionnaire, and received compensation for completing it. Because all students in Grades 8, 9, and 10 take a language arts course, selecting the language arts teacher as the respondent increased the probability that students were evaluated in a similar instructional context. In approximately 7% of the cases, the questionnaire was completed by another teacher named by the language arts teacher as having greater knowledge of the student.

Focal Measure: Adolescent Motivation for Educational Attainment Questionnaire

As described above, the initial 36-item student-report questionnaire includes 21 items drawn from Murdock’s (1999) measure of perceived motivational context that assesses perceived social support for educational achievement (i.e., teachers’ competence support and peers’ educational aspirations) and students’ perceived economic value of an education. Six new items were developed that assess the two hypothesized educational motivation dimensions, but the content is specific to high school completion and post-secondary enrollment rather than to general academic success—for example, “Graduating from high school is not as important to me as getting a good paying job” (reversed; related to value of education), “My friends expect me to graduate from high school” (related to peer aspirations), and “My parents expect me to go to college” (related to parent educational expectations). Nine new items were developed by the researchers to assess the hypothesized dimension of competence and effort beliefs to graduate from high school and pursue post-secondary education (e.g., “I am confident I will graduate from high school”; “I know what courses and grades it takes to get into the vocational school or college I want to enter”). Students were asked to indicate the degree to which they agreed or disagreed with each of the statements using a 5-point scale (1 = strongly disagree, 5 = strongly agree). The Appendix presents the full set of items used in the scale.

Criterion-Related Validity Measures

School belonging

Participants completed the Psychological Sense of School Membership Scale (Goodenow, 1993) by indicating their agreement on 18 items measured on a 5-point Likert-type scale. The items assessed students’ perceived acceptance, feelings of inclusion, respect, and encouragement for participation. Higher school membership scores have been found in previous research to be associated with greater school attendance, higher grades, more positive self-concept, greater time spent on homework, and better social-emotional adjustment (Goodenow, 1993; Hagborg, 1998). Cronbach’s α for the current sample at Year 9 was .90. Scores on the 18-item scale have been found in the larger sample to be moderately correlated with students’ perceived level of teacher support and academic self-efficacy (Hughes, 2011).

Teacher-rated student–teacher relationship warmth and conflict

The Teacher Network of Relationships Inventory (TNRI; Hughes & Kwok, 2006) was developed from the child version of the Network of Relationships Inventory (Furman & Buhrmester, 1985). Teachers reported their provision of warmth (13 items) and the level of conflict (six items) in their relationship with a given student on a 1–5 Likert scale. Example warmth scale items (Year 9: scale α = .95; Year 10: scale α = .95) include “I enjoy being with this child,” and “This child gives me many opportunities to praise him or her.” Example conflict scale items (Year 9: scale α = .92; Year 10: scale α = .93) include “This child and I often argue or get upset with each other,” and “I often need to discipline this child.” Scores on the TNRI have demonstrated good construct, factorial, and predictive validity across the elementary and middle school grades (Hughes, 2011; Hughes et al., 2008; Li, Hughes, Kwok, & Hsu, 2011; Wu & Hughes, 2012).

Teacher-reported disciplinary practices

Teachers indicated whether the student had experienced one or more of the following five actions: (1) sent to the office for disciplinary reasons, (2) assigned to in-school-suspension, (3) assigned to disciplinary alternative education program within the school district, (4) assigned to a judicial placement outside school district, and (5) expelled from school. These five actions were reduced to three ordered categories: did not experience any disciplinary actions (Year 9: 77.1%; Year 10: 77.5%); experienced mild disciplinary actions [(1) or (2) only] (Year 9: 18.0%; Year 10: 17.7%); and experienced more substantial disciplinary actions [(3), (4), or (5)] (Year 9: 4.9%; Year 10: 4.8%). If the teacher reported more than one type of disciplinary action, the student was assigned a score corresponding to the highest ordinal category.

Teacher-reported grades

Teachers were asked to report the letter grade (from A to F, with A = 4 and F = 0) that the student received in the teacher’s class for the most recent grading period. If the student had more than one subject with the teacher, the teacher listed grades for each subject, and the mean letter grade was computed. The language arts teacher reported the student’s grades in the instructor’s language arts class in over 90% of the cases (Year 9: 92.8%; Year 10: 93.0%); in the remaining small percentage of the cases, another teacher who knew the student better reported the student’s grades.

Youth-rated conduct problems

Students completed the Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997), a brief screening measure for psychopathology for children and adolescents. Each item is rated on a 0–2 scale (i.e., not true, somewhat true, certainly true). The SDQ yields five scales comprised of five items each. Only the Conduct Problems scale (example items: “Often fights with other children or bullies them”; “Often lies or cheats”) was used in the current study. Scores on the conflict scale demonstrate adequate internal consistency for a short five-item scale (scale α = .60 in Goodman, 2001) and have also demonstrated adequate construct validity in adolescents ages 11–17 (Goodman, 2001; Muris, Meesters, & van den Berg, 2003; Roy, Veenstra, & Clench-Aas, 2008). Scale α at Year 9 was .63.

Teacher-rated classroom engagement

Teachers rated students’ classroom engagement using an 11-item questionnaire. Items were adapted from both the teacher and the student ratings of students’ engagement (Connell & Wellborn, 1991; Skinner, Zimmer-Gembeck, & Connell, 1998). Items assess effort, persistence, concentration, and interest. Example items include the following: “Tries hard to do well in school,” “concentrates on doing work,” “tries to look busy” (reverse scored), and “participates in class discussion.” Teachers were asked to indicate the extent to which each statement was true of the student on a scale ranging from 1 (Not true at all) to 4 (Very true). These 12 items demonstrate good factorial validity (Q. Chen, Hughes, Liew, & Kwok, 2010) and internal consistency (scale α at Years 9 and 10 was .93 and .92, respectively).

Results

Exploratory Factor Analysis

Using the items measured at Year 9, maximum likelihood exploratory factor analysis (EFA) was initially used to determine the number of constructs and the factor patterns underlying these items. The participants were randomly split into two subsamples. EFAs were conducted separately in each subsample. Figure 2 shows the scree plot (Cattell, 1966) and the parallel analysis (Horn, 1965) of the educational motivation items of the two subsamples, which aid in the determination of the number of factors (Zwick & Velicer, 1986). The scree plot suggested that the eigenvalues of the two subsamples were virtually identical. The eigenvalues flattened from the fifth factor. The parallel analysis showed that four factors had eigenvalues larger than the intersection point with the scree plot. These results suggested that up to a maximum of five factors might underlie the data.

Figure 2.

Scree plot and parallel analysis (Year 9). The dotted line represents the eigenvalues for a randomly generated correlation matrix having the same sample size in which there is no relationship between variables in the population.

In the first subsample, we used oblique geomin factor rotation (ε = .5) to facilitate the interpretation of factor loadings¹ (Browne, 2001). A detailed investigation of the geomin rotated factor patterns from one to five factors and the factor loading patterns suggested further consideration of the four-factor solution. Factor 1 was termed Competence and Effort Beliefs. Items included “I am confident that I will graduate from high school.” Factor 2 was termed Teacher Educational Expectations. Items included “My teachers expect that I will do well in the future.” Factor 3 was termed Peer Aspirations. Items included “Most of my good friends plan to go to college.” Factor 4 was termed Value of Education. Items included “Many of the things we do in school seem useless to me” (reversed item). The replicability of this factor pattern was investigated using the second subsample. In the second subsample, we used Procrustes factor rotation, which is based on the four-factor solution obtained from the first subsample (Browne, 2001). We calculated Tucker’s (1951) congruence coefficient, which is a standardized measure of the degree of replicability of the loadings of each factor between the two subsamples (0 = no replicability, 1 = perfect replicability). According to Lorenzo-Seva and ten Berge (2006), the four factors had fair similarity (Factor 1 = .89; Factor 2 = .89; Factor 3 = .83; Factor 4 = .88).

To obtain more stable estimates of the factor loadings (MacCallum, Widaman, Zhang, & Hong, 1999), we combined the two subsamples and reran the four-factor EFA with geomin factor rotations (ε = .5) with the full sample. Table 1 shows the rotated factor patterns. We identified and deleted the items that had low factor loadings on all factors or substantial cross loadings on secondary factors for the subsequent CFAs. One item was deleted because it had a minimal loading (maximum = .18) on all four factors. Three items were deleted because of their substantial cross loadings on one or more secondary factors. The resulting final scale had 32 items.

Table 1.

Standardized Factor Loadings of Geomin (ε = .5) Rotated Four-Factor EFA Model

Item	Factor 1	Factor 2	Factor 3	Factor 4
	Competence and Effort Beliefs	Teacher Educational Expectations	Peer Aspirations	Value of Education
1	0.49	0.15	0.18	−0.12
2	0.53	0.17	0.08	−0.04
3	0.42	0.29	0.17	−0.03
4	0.53	0.17	0.18	−0.08
Aa	0.07	−0.05	−0.05	0.18
5	0.00	−0.22	0.01	0.49
6	0.41	0.08	0.16	−0.13
Ba	0.36	0.23	0.31	−0.04
7	0.52	0.14	0.26	−0.18
8	0.42	−0.01	0.23	−0.19
Ca	0.39	0.17	0.33	−0.14
9	0.57	0.07	0.18	−0.21
10	0.61	0.11	−0.07	0.01
11	0.59	0.19	−0.09	−0.02
12	0.56	0.05	−0.04	0.13
13	0.00	0.79	0.05	−0.04
14	−0.03	0.87	−0.01	−0.02
15	0.06	0.65	0.21	0.01
16	0.01	0.77	−0.03	−0.01
17	−0.02	−0.46	−0.16	0.17
18	0.11	0.04	−0.74	0.08
19	0.21	0.11	0.56	−0.03
20	0.09	0.13	0.57	0.09
21	0.16	0.18	0.60	0.10
22	0.07	−0.04	−0.71	0.14
23	0.06	−0.05	−0.62	0.14
24	−0.02	−0.05	−0.15	0.52
25	−0.03	0.01	−0.07	0.67
26	−0.11	−0.16	0.07	0.50
27	0.01	−0.08	−0.04	0.61
28	−0.02	−0.04	−0.08	0.69
29	0.07	0.02	−0.14	0.48
30	−0.10	−0.09	−0.15	0.59
31	0.19	0.20	−0.03	−0.30
32	0.19	0.16	0.02	−0.35
Da	0.19	0.21	0.01	−0.24

Note. Items were not reverse coded. Bold factor loadings were the highest factor loadings; these items were permitted to load on the corresponding factor in the confirmatory factor analysis (CFA). A list of the items is presented in the Appendix. EFA = exploratory factor analysis.

^aItems A–D were removed in the CFA because of low loadings or loadings on more than one factor. Item D was from Murdock’s (1999) Educational Motivation Questionnaire, so 20 items from Murdock’s questionnaire were used in our final set of items.

Confirmatory Factor Analysis

For ease of interpretation, the score on each educational motivation item was recoded so that high scores on items meant higher levels on its underlying primary factor. Throughout the CFAs, the Satorra–Bentler correction (Satorra & Bentler, 2001; Yuan & Bentler, 2000) to the standard errors and χ² statistic in maximum likelihood estimation was used to address moderate degrees of non-normality (Year 9, skewness ranged from −2.47 to 1.25, kurtosis from −1.10 to 8.72; Year 10, skewness ranged from −2.44 to 1.09, kurtosis from −1.00 to 8.98).

Three CFA models presented in the introduction and illustrated in Figure 1 that represent a multi-dimensional construct were investigated. Model 1 was a correlated-factor model corresponding to the EFA results. For each item, the factor loading was only estimated for the factor corresponding to the highest loading factor in the four-factor EFA solution. Correlations between each pair of the four factors were estimated. Model 2 was a second-order factor model in which the four first-order factors loaded on a second-order factor. Model 3 was a bifactor model in which each item loaded on a general factor and a specific factor that corresponded to the highest loading in the EFA solution. Each confirmatory factor model was tested separately at Years 9 and 10. The Year 10 test provides a replication of the Year 9 solution.

In the correlated-factor model at Years 9 and 10, the standardized factor loading pattern was consistent with the EFA solution (not depicted). In addition, the factor correlations were moderately to highly positive (Year 9: median r = .58, range = .53–.67; Year 10: median r = .52, range = .43–.66). These correlations provided support for the development of second-order factor and bifactor models (Brunner et al., 2012, p. 836). Table 2 shows the χ² statistic and the global model fit indices of the three hypothesized CFA models. The χ² statistic of each factor model is testing the null hypothesis that the factor model fits the observed data within sampling error. At both Years 9 and 10, the root-mean-square error of approximation (RMSEA) showed the bifactor model fitted the data better than the correlated-factor and second-order factor models, whereas the correlated-factor and second-order factor models had similar fits. The second-order factor model is nested within the correlated-factor and the bifactor models (F. F. Chen et al., 2006). However, the correlated-factor model is not nested within this bifactor model (Reise, 2012). Satorra and Bentler’s (2001) χ² difference test showed that the correlated-factor model did not fit the data significantly differently from the second-order factor model: Year 9, Δχ²(2) > 1.33, p = .51; Year 10, Δχ²(2) = 0, p > .99.² The bifactor model fitted the data significantly better than the second-order factor model: Year 9, Δχ²(28) = 263.89, p < .001; Year 10, Δχ²(28) = 361.01, p < .001. The standardized factor loading patterns between the correlated-factor and the second-order factor models were similar in magnitude (not depicted). In the bifactor model, nearly all standardized factor loadings on the general factor were substantial (Year 9: Mdn = .51, range = .25–.83; Year 10: Mdn = .49, range = .18–.77). However, many of the absolute standardized loadings of Specific Factor 1 (Competence and Effort Beliefs) at both Years 9 and 10 were small, and some were in an unpredicted direction, suggesting that this specific factor could be deleted from the model (F. F. Chen et al., 2006).

Table 2.

Global Model Fit Indices of CFA Models

Model	χ2		df	RMSEA
	Y9	Y10		Y9	Y10
Correlated-factor	1,548.4	1,417.1	458	.070	.064
Second-order factor	1,547.8	1,418.0	460	.069	.064
Bifactor	1,210.8	997.9	432	.061	.051
Modified bifactor	1,458.3	1,244.7	443	.068	.060
Modified bifactor (correlated uniqueness)	1,138.8	987.0	440	.057	.049

Note. CFA = confirmatory factor analysis; RMSEA = root-mean-square error of approximation; Y9 = Year 9; Y10 = Year 10.

The bifactor model was re-estimated with Specific Factor 1 (Competence and Effort Beliefs) deleted from the model. Modification indices at both Years 9 and 10 suggested that three pairs of correlated uniqueness reflecting very similarly worded items be added to the model: Item 10 with Item 11, Item 6 with Item 8, and Item 31 with Item 32. The first pair was two items concerning student’s competence and effort beliefs for college. The second pair was two items concerning parent educational expectations. The third pair included the only two positively worded items in the Specific Factor 4 (Value of Education). The global fit index RMSEA agreed that this modified bifactor model with correlated uniqueness fitted the data best among all tested models (see Table 2). In this model, the RMSEA at Years 9 and 10 met rule of thumb criteria for adequate fit (RMSEA < .06; Hu & Bentler, 1999). The standardized factor loadings for this model are shown in Table 3. In the section below on measurement invariance, we present a direct comparison of the results of the bifactor model for the Year 9 and our Year 10 replication data.

Table 3.

Standardized Factor Loadings of Modified Bifactor Model With Correlated Uniqueness

Item	Year 9				Year 10
	Specific Factor 2	Specific Factor 3	Specific Factor 4	General factor	Specific Factor 2	Specific Factor 3	Specific Factor 4	General factor
	Teacher Educational Expectations	Peer Aspirations	Value of Education		Teacher Educational Expectations	Peer Aspirations	Value of Education
1				0.75				0.60
2				0.66				0.57
3				0.73				0.68
4				0.77				0.75
5			0.41	0.38			0.32	0.43
6				0.57				0.50
7				0.81				0.77
8				0.57				0.59
9				0.76				0.76
10				0.52				0.51
11				0.56				0.53
12				0.40				0.44
13	0.60			0.56	0.56			0.52
14	0.68			0.54	0.70			0.51
15	0.46			0.62	0.49			0.55
16	0.58			0.49	0.59			0.42
17	0.32			0.51	0.30			0.59
18		0.66		0.35		0.66		0.36
19		0.42		0.60		0.51		0.50
20		0.43		0.46		0.53		0.40
21		0.41		0.57		0.49		0.49
22		0.60		0.46		0.66		0.39
23		0.54		0.43		0.53		0.38
24			0.46	0.38			0.34	0.35
25			0.60	0.37			0.67	0.29
26			0.44	0.39			0.41	0.39
27			0.57	0.35			0.57	0.30
28			0.64	0.40			0.75	0.29
29			0.45	0.25			0.49	0.19
30			0.51	0.52			0.43	0.49
31			0.19	0.42			0.22	0.46
32			0.25	0.43			0.26	0.44

Note. Items were reverse coded.

As a second method of evaluating the model, Table 4 presents McDonald’s (1999) coefficient ω as a measure of reliability calculated for each of the factors at Years 9 and 10 (see also Reise, 2012; Zinbarg, Yovel, Revelle, & McDonald, 2006). The ω of the specific factors is the proportion of variance accounted for by each specific factor (ignoring other specific factors), over and above that by the general factor and the error variance. The ω of the general factor is the proportion of variance accounted for by the general factor over and above those by the specific factors and the error variance. The ω of the whole model is the proportion of variance accounted for by the general factor and all specific factors over and above the error variance. The great majority of the reliable variance was accounted for by the general factor, with smaller amounts of reliable variance accounted for by each of the three specific factors.

Table 4.

Coefficient ω of Modified Bifactor Model With Correlated Uniqueness

Factor	Year 9	Year 10
Specific Factor 2 (Teacher Educational Expectations)	0.02	0.03
Specific Factor 3 (Peer Aspirations)	0.03	0.04
Specific Factor 4 (Value of Education)	0.07	0.07
General factor	0.84	0.81
Whole model	0.95	0.94

Measurement Invariance

Following the recommendations of Knight and Zerr (2010), we investigated the measurement invariance of the selected bifactor model across time (Years 9 and 10), students’ gender, and students’ ethnicity³ (Caucasian, African American, and Hispanic). Invariance tests of students’ gender and ethnicity were investigated separately at Years 9 and 10. We followed the sequence of tests of measurement invariance originally developed by Meredith (1993) and discussed and extended by Millsap (2011), Millsap and Cham (2012), and Widaman and Reise (1997). In sequence, we tested configural invariance (same factor pattern),⁴ metric invariance (configural + same factor loadings), scalar invariance (metric + same latent intercepts), and strict invariance (scalar + same unique factor variances and covariances). We used the following procedures to test the invariance models: (1) The configural invariance model should have acceptable fit to the data based on RMSEA index (see Table 5). (2) Satorra and Bentler’s (2001) χ² difference test of the two nested invariance models in the sequence should ideally be non-significant. The null hypothesis of the χ² difference test is that the more restricted invariance model (e.g., metric) fits the data equally well as the less restricted invariance model (e.g., configural). Given that the χ² difference test can show significant differences when there is only trivial lack of invariance, we followed up significant differences on this test with an examination of the RMSEA. Small differences of the RMSEA between the more restricted and less restricted invariance models support the retention of the more restricted invariance model (see Table 5; F. F. Chen, 2007; Cheung & Rensvold, 2002). (3) When the χ² difference test is statistically significant at α = .05, a potential partial invariance model was considered based on modification indices (Millsap, 2011).

Table 5.

Global Model Fit Indices of Measurement Invariance Tests

Invariance model	χ2	df	χ2 difference test			RMSEA
			Comparison	χ2(df)	p
(A) Invariance across time (Years 9 and 10)
Configural	3,531.0	1868				.041
Metric	3,580.5	1917	Configural	53.5 (49)	.305	.040
Scalar	3,661.9	1945	Metric	85.4 (28)	<.001	.041
Partial Scalar	3,634.8	1942	Metric	55.4 (25)	<.001	.040
Strict	3,696.5	1980	Scalar	50.7 (35)	.042	.040
(B) Invariance across students’ gender
Year 9
Configural	1,758.6	880				.064
Metric	1,808.2	929	Configural	54.9 (49)	.261	.062
Scalar	1,848.7	957	Metric	38.7 (28)	.086	.062
Strict	1,853.3	992	Scalar	38.0 (35)	.333	.059
Year 10
Configural	1,573.1	880				.056
Metric	1,624.7	929	Configural	56.5 (49)	.215	.054
Scalar	1,674.7	957	Metric	50.0 (28)	.006	.054
Strict	1,722.5	992	Scalar	54.8 (35)	.018	.054
(C) Invariance across students’ ethnicity
Year 9
Configural	2,558.5	1320				.077
Metric	2,643.6	1418	Configural	106.0 (98)	.274	.074
Scalar	2,783.4	1474	Metric	143.4 (56)	<.001	.072
Strict	2,816.2	1544	Scalar	85.0 (70)	.107	.072
Year 10
Configural	2,276.3	1320				.067
Metric	2,352.9	1418	Configural	93.5 (98)	.611	.064
Scalar	2,454.0	1474	Metric	101.3 (56)	<.001	.064
Strict	2,491.1	1544	Scalar	80.6 (70)	.182	.061

Note. RMSEA = root-mean-square error of approximation.

Invariance across time

The RMSEA (= .041) of the configural model suggested acceptable fit to the data. For metric invariance, the χ² difference test versus the configural invariance model was non-significant, Δχ²(49) = 53.5, p = .31. For scalar invariance, the χ² difference test versus the metric invariance model was significant, Δχ²(28) = 85.4, p < .001. The RMSEA (change ≤ .01) did not differ noticeably between two models. For strict invariance, the χ² difference test versus the scalar invariance model was significant but small in magnitude, Δχ²(35) = 50.7, p = .04. The RMSEA (change ≤ .01) did not increase noticeably between the two models. These results suggested that the bifactor model obtained at Year 9 was replicated at Year 10 and that there was not an appreciable change in the measurement structure with an additional year of age of the student.

Invariance across students’ gender

The RMSEA (Year 9: .064; Year 10: .056) of the configural invariance model suggested an acceptable fit to the data. For metric invariance, the χ² difference tests versus the configural invariance model were non-significant: Year 9, Δχ²(49) = 54.9, p = .26; Year 10, Δχ²(49) = 56.5, p = .22. For scalar invariance, the χ² difference test versus the metric invariance model was non-significant at Year 9, Δχ²(28) = 38.7, p = .09, but reached statistical significance at Year 10, Δχ²(28) = 50.0, p < .01. At Year 10, the RMSEA (change ≤ .01) did not differ noticeably between two models, and modification indices did not suggest a potential partial scalar invariance model. For strict invariance, the χ² difference test versus the scalar invariance model was non-significant at Year 9, Δχ²(35) = 38.0, p = .33, but reached statistical significance at Year 10, Δχ²(35) = 54.8, p < .05. The RMSEA (change ≤ .01) did not change noticeably between the two models. The bifactor model of educational motivation appeared to have the same measurement structure in males and females at Years 9 and 10.

Invariance across students’ ethnicity

The RMSEA (Year 9: .077; Year 10: .067) of the configural invariance model suggested an acceptable fit to the data. For metric invariance, the χ² difference tests versus the configural model were non-significant: Year 9, Δχ²(98) = 106.0, p = .27; Year 10, Δχ²(98) = 93.5, p = .61. For scalar invariance, the RMSEA (change ≤ .01) did not differ noticeably between the two models, and modification indices did not suggest a potential partial scalar invariance model. For strict invariance, the χ² difference tests versus the scalar invariance model were non-significant: Year 9, Δχ²(70) = 85.0, p = .11; Year 10, Δχ²(70) = 80.6, p = .18. The bifactor model of educational motivation appeared to meet the criteria for measurement invariance in Caucasian, African American, and Hispanic students at Years 9 and 10.

Criterion-Related Validity

Table 6 shows the maximum likelihood estimated correlations, means, and standard deviations of the measures that assess the criterion-related validity of the selected bifactor model at Years 9 and 10 (except for the categorical measure teacher-reported disciplinary practices; see the Method section for its descriptive statistics). The correlations of teacher-rated student–teacher relationship warmth and conflict, teacher-reported disciplinary practices, teacher-reported grades, and teacher-rated behavior engagement between Years 9 and 10 ranged from .26 to .54, which indicated low to moderate stabilities of these measures. Note that the ratings were performed by different teachers in Years 9 and 10 attenuating the stability relative to its true value (Campbell & O’Connell, 1982). The means and standard deviations of teacher-rated student–teacher relationship warmth and conflict, and teacher-rated behavior engagement, were about the same between Years 9 and 10, which was also supported by non-significant t-test results (p > .05). The mean of teacher-reported grades at Year 9 was slightly higher than that at Year 10 (p = .03).

Table 6.

Correlations, Means, and Standard Deviations of Criterion Measures

Criterion measure	1	Year 9						Year 10
		2	3	4	5	6	7	8	9	10	11
1. Gender	1
Year 9
2. School belonging	−0.10	1
3. Teacher-rated warmth	−0.18	0.21	1
4. Teacher-rated conflict	0.21	−0.10	−0.49	1
5. Teacher-reported grades	−0.22	0.21	0.47	−0.33	1
6. Youth-rated conduct problems	0.15	−0.43	−0.20	0.19	−0.25	1
7. Teacher-rated behavior engagement	−0.24	0.20	0.71	−0.52	0.67	−0.28	1
Year 10
8. Teacher-rated warmth	−0.11	0.28	0.26	−0.26	0.34	−0.27	0.37	1
9. Teacher-rated conflict	0.11	−0.18	−0.20	0.38	−0.25	0.28	−0.31	−0.54	1
10. Teacher-reported grades	−0.06	0.20	0.26	−0.16	0.45	−0.25	0.39	0.59	−0.34	1
11. Teacher-rated behavior engagement	−0.14	0.25	0.34	−0.28	0.42	−0.36	0.53	0.73	−0.51	0.66	1
M	0.55	3.84	3.53	1.55	2.55	0.37	3.29	3.50	1.60	2.38	3.23
SD	0.50	0.67	0.91	0.79	1.25	0.33	0.82	0.92	0.84	1.31	0.84

Note. Gender is dummy coded (1 = male, 0 = female).

Table 7 shows the correlations of the general and specific factors with each of the criterion measures at Years 9 and 10. Polychoric correlations were estimated between the teacher-reported disciplinary practices (ordered-categorical measure) and the general and specific factors. The general factor was significantly positively correlated with school belonging (Year 9: r = .56), teacher-rated warmth (Year 9: r = .23; Year 10: r = .26), teacher-reported grades (Year 9: r = .30; Year 10: r = .25), and teacher-rated behavior engagement (Year 9: r = .31; Year 10: r = .32), whereas it was significantly negatively correlated with gender (dummy coded: 1 for male and 0 for female; Year 9: r = −.15; Year 10: r = −.06), teacher-reported disciplinary practices (Year 9: r = −.65; Year 10: r = −.56), and youth-rated conduct problems (Year 9: r = −.39).

Table 7.

Correlations Between the General and Specific Factors and the Criterion Measures

Criterion measure	Year 9				Year 10
	Specific Factor 2	Specific Factor 3	Specific Factor 4	General factor	Specific Factor 2	Specific Factor 3	Specific Factor 4	General factor
	Teacher Educational Expectations	Peer Aspirations	Value of Education		Teacher Educational Expectations	Peer Aspirations	Value of Education
Gender	−0.049	−0.086	−0.021	−0.154**	0.000	−0.028	−0.016	−0.059*
School belonging	0.411**	0.092	0.053	0.558**
Teacher-rated warmth	0.152*	0.063	0.014	0.228**	0.154*	0.050	−0.012	0.259**
Teacher-rated conflict	−0.058	−0.052	0.073	−0.117*	−0.077	−0.072	0.058	−0.191**
Teacher-reported disciplinary practices	0.217	−0.205	−0.093	−0.646**	0.003	−0.436**	0.149	−0.555**
Teacher-reported grades	0.085	0.161**	−0.016	0.296**	0.089	0.214**	−0.086	0.252**
Youth-rated conduct problems	−0.065	−0.186**	−0.092	−0.386**
Teacher-rated behavior engagement	0.114	0.152*	−0.061	0.307**	0.106	0.113	−0.026	0.315**

Note. Specific Factor 2 is Teacher Support. Specific Factor 3 is Peer Support. Specific Factor 4 is Value of Education. Gender is dummy coded (1 = male, 0 = female).

^*p < .05.

^**p < .01.

The correlations of the three specific factors with the criterion measures were interpreted as the partial correlations adjusted for the effect of the general factor. Teacher Educational Expectations (Specific Factor 2) was significantly positively correlated with school belonging (Year 9: r = .41) and teacher-rated warmth (Year 9: r = .15; Year 10: r = .15). Peer Aspirations (Specific Factor 3) was significantly correlated with teacher-reported grades (Year 9: r = .16; Year 10: r = .21) and youth-rated conduct problems (Year 9: r = −.19). Peer Aspirations was also significantly correlated with teacher-rated behavior engagement at Year 9 (r = .15), and was correlated in the same direction at Year 10 (r = .11), but did not attain statistical significance. Value of Education (Specific Factor 4) was not significantly correlated with any criterion measure at Years 9 and 10.

Discussion

Dimensionality of Motivation for Educational Attainment

The primary purpose of the current study was to investigate the dimensionality, structure, and construct validity of a measure of adolescents’ motivation to complete high school and pursue post-secondary education. Results support the multi-dimensionality of high school students’ motivation to attain a high school degree and post-secondary education. The EFA found that students’ motivation for educational attainment consists of at least four highly related but distinct dimensions: educational competence and effort beliefs, economic valuing of educational attainment, teacher educational expectations, and peer educational aspirations. These findings are consistent with expectancy value theory (Wigfield, 1994), which posits self-efficacy to engage in a behavior and valuing of a behavior are distinct motivational constructs. Also, as expected, based on the hypothesized effect of competence beliefs on behavior (Zimmerman & Cleary, 2006), items assessing competence beliefs for educational attainment (e.g., “I am confident I will go to college”; Item 7) and items assessing effort toward attainment (e.g., “I know how much it cost to go to vocational school or college”; Item 12) loaded on the same factor. Additionally, consistent with other measures of social support for education (Legault et al., 2006; Murdock, 1999), teachers’ and peers’ support for educational attainment comprise distinct aspects of the motivational context. A separate factor assessing perceived parent support for educational attainment was not identified, perhaps due to the fact that only three items assessed perceived parent support; these items were divided between parental educational expectations and economic valuing of education.

It is important to note that the four dimensions of educational motivation were highly related (Year 9: factor correlations from 0.53 to 0.67; Year 10: factor correlations from 0.43 to 0.66), likely due to their inter-relationships with each other that develop over time in a dynamic system of influence. For example, teachers’ communication of high expectations for students likely enhances students’ competence beliefs, which engender higher levels of academic effort, which, in turn, influence teachers’ expectations (Hughes et al., 2008). Similarly, peer educational aspirations likely influences the student’s choice to affiliate with peers who have similar goals, who, in turn, likely influence the adolescent’s motivation to pursue higher levels of educational attainment. Thus, although distinct dimensions of educational motivation can be identified, the dimensions likely constitute a system of reciprocal influences, which accounts for the pattern of high inter-correlations between the dimensions.

Structure of Motivation for Educational Attainment

Based on the high correlations between each pair of the four factors in the EFA, we examined correlated-factor, second-order factor, and the bifactor models (Brunner et al., 2012; F. F. Chen et al., 2006) with simple, independent cluster structure in our confirmatory factor analyses. The high correlations between each pair of the latent factors in the correlated-factor model provided statistical support for further investigation of the second-order factor and bifactor models. The χ² difference tests and model fit indices supported that the bifactor model fitted the data better than the correlated-factor and second-order factor models. In the bifactor model, the general factor reflects a student’s basic, fundamental motivation for educational attainment.

It also captures the commonality of all the measured items. The specific factors were constrained to be uncorrelated with each other and with the general factor, following the recommendations of F. F. Chen et al. (2006) and Rindskopf and Rose (1988). This solution resulted in an extra interpretational advantage that each specific factor represented its unique contribution over and above other specific factors and the general factor. Further investigation revealed that the standardized factor loadings of the specific factor Competence and Effort Beliefs were very small in magnitude and did not form a coherent pattern, which led to the conclusion that this specific factor was unnecessary and could be removed in the final bifactor model. The disappearance of the specific factor Competence and Effort Beliefs in the bifactor model does not mean that this hypothesized dimension had no importance to motivation for education attainment. Rather, it indicated that this hypothesized dimension was part of the fundamental, basic motivation for educational attainment and did not provide unique information over and above that dimension.

Measurement Invariance

Measurement invariance across age, gender, and often ethnic group is routinely established in standardized tests of achievement and measurement invariance. Measurement invariance is increasingly being established across age and gender on key psychosocial measures in prevention research. Establishing measurement invariance insures that group differences in means, variances, and correlational relationships of the latent constructs can be directly interpreted. Otherwise, group differences are confounded with potential differences in the performance of the instruments in the two groups—there is no guarantee that the same constructs are being measured. Knight and Zerr (2010) have called for the routine use of tests of measurement invariance in developmental research with diverse populations. F. F. Chen (2008) has illustrated the serious difficulties that arise in cross-cultural research when measurement invariance cannot be established.

Few investigators to date have tested gender invariance of measures of adolescents’ educational motivation (for an exception, see Green-Demers, Legault, Pelletier, & Pelletier, 2008). To the best of our knowledge, no study has investigated the invariance of a measure of adolescents’ motivation for educational attainment over time or invariance across gender and ethnic membership. Our findings showed that the present bifactor structure of our motivation for educational attainment measure is invariant across gender, the three major ethnic groups (Caucasian, African American, Hispanic), and over 2 years during the early adolescence period. Our measurement invariance studies showed that strict invariance was attained for each comparison. According to Widaman and Reise (1997),

the common factors [here, the general and specific factors] are entirely responsible for any group differences in means and variances on the measured variables [items]; all group differences on the measured variables are captured by, and attributable to, group differences on the common factors. (p. 296)

As each level of measurement invariance in the hierarchy (configural, metric, scalar, and strict in ascending order) is attained, factors that potentially confound the interpretation of group or temporal differences are ruled out. The present results showing measurement invariance across year of study also serve as a replication of the CFA results for the original bifactor model.

Criterion-Related Validity

Results support the criterion-related validity of the interpretation provided by the bifactor model of the Adolescent Motivation for Educational Attainment Questionnaire. The use of the bifactor model permits us to study the contribution of the general factor and the specific (unique) factors separately. Because the specific factors were uncorrelated with each other and with the general factor in the final bifactor model, the correlations of the specific factors with the criterion measures can be interpreted like partial correlations, whose effects have been adjusted for the effects of other factors. The correlations involving the general factor can be interpreted as the relationship between the common covariance shared by all of the items and the criterion measures. For this reason, it is not surprising that the general factor was consistently predictive of each of the criterion variables, including gender. The strongest correlation was between the general factor and teacher-reported disciplinary practices (r = −.65). The magnitude of the correlation is noteworthy not only because the two variables are obtained from different sources (e.g., student vs. teacher) but also because in-school and out of school disciplinary practices are the strongest predictor of dropping out of school (Finn & Rock, 1997). In a study of inner-city youths attending Chicago schools, students arrested in ninth or 10th grade were six to eight times more likely than non-arrested students to drop out of high school. The general factor was moderately related both to teacher-reported behavioral engagement and to students’ letter grades (rs = .31 and .30, respectively).

Above and beyond the effect of the general factor, adolescents’ reports of teacher educational expectations predicted students’ perceived school belonging and teacher-reported relationship warmth. Consistent with research demonstrating the importance of students’ perceptions of teacher competence beliefs to their academic motivation (Legault et al., 2006), these findings suggest that students who believe their teachers have confidence in their educational ability feel more connected to school and more emotionally supported, above and beyond students’ general educational motivation. Similarly, consistent with research demonstrating the role of one’s peer group’s academic motivation on students’ academic effort and achievement (Altermatt & Pomerantz, 2005; Kiuru et al., 2007; Vitoroulis et al., 2012) as well as conduct problems (Murdock, 1999), students’ perceptions of their peers’ educational aspirations make a unique contribution to the prediction of students’ grades, teacher-reported engagement, and student-reported conduct problems, above the general factor. These results show that the specific factors make unique contributions to specific outcomes, independent of the general factor of educational motivation.

Surprisingly, valuing of education did not make a unique prediction, above general motivation for educational attainment, to any of the criterion measures. This factor’s contribution to the criterion measures appears to be redundant with that of general educational motivation.

Limitations and Implications of Findings for Research

We had access to a single relatively large longitudinal sample of children who were studied beginning in Grade 1. We randomly split our sample into two subsamples to cross-validate our EFA results. We cross-validated our final CFA bifactor model using data collected from the same students collected during another year. Despite the strength of our at risk sample in which gender and three major U.S. ethnic/racial groups were approximately equally represented, further replication with other samples is warranted.

Our longitudinal analysis was limited to 2 consecutive years. Future research on longitudinal invariance across a wider time frame would increase the generalizability of current findings, enhancing the usefulness of the scale for a wider age range of students. Furthermore, the investigation of criterion-related validity was limited by the cross-sectional nature of the data. When the data are collected in future years in our ongoing longitudinal study, we will assess the ability of the motivation for educational attainment measure to predict not only proximal precursors (surrogate endpoints) of school completion (or dropping out of school) but whether students eventually earn a high school diploma and whether students pursue post-secondary education.

Our findings also demonstrated that the low coefficient ω of the specific factors in the bifactor model does not necessarily indicate that these factors could be discarded. In this analysis, the Teacher Support and Peer Support specific factors were significantly correlated with the criterion measures.

The assessment of parent support for educational attainment is a limitation both in terms of few parent support items and in terms of the content of the parent support items. With respect to content, previous research has suggested that adolescents’ perceptions of parents’ warmth and closeness may be important to students’ commitment to educational attainment, via their impact on students’ valuing of educational attainment (Legault et al., 2006). In addition, budget limitations allowed us to only assess teacher ratings from one teacher, typically the language arts teacher. Aggregation of ratings from multiple teachers of subject matter courses common to all students in these grades would enhance the reliability of the teacher ratings.

Practical Implications of Findings

The existence of a relatively brief, psychometrically strong measure of early high school students’ motivation to complete high school and pursue post-secondary education serves an important applied goal: It permits the early identification of students at elevated risk for dropping out of school. Early identification efforts, however, are beneficial only if interventions with strong evidence of efficacy are available. One such intervention is the Check and Connect mentoring program (Christenson & Reschly, 2010). This program has proven effective in engaging academically at-risk adolescents in school through building relationships that support educational engagement, monitoring attendance and academic performance, and encouraging association with academically oriented activities and peers. The finding of a strong general factor that captures the common variance across the specific factors means that a total score on the Adolescent Motivation for Educational Attainment Questionnaire provides a good measure of an adolescent’s overall motivation to attain a high school diploma and further vocational or academic education.

Appendix

Adolescent Motivation for Educational Attainment Questionnaire Items

No.	Item
1	I am confident that I will graduate from high school.
2	I know what courses I need to take to graduate from high school.
3	I am on track to graduate from high school.
4	Nothing will get in the way of my graduating from high school.
5 (R)	Graduating from high school is not as important to me as getting a good paying job.
6	My parents expect me to graduate from high school.
7	I am confident that I will go to college.
8	My parents expect me to go to college.
9	Nothing will get in the way of my going to college.
10	I have started gathering information about vocational schools or colleges.
11	I know what courses and grades it takes to get into the vocational school or college I want to enter.
12	I know how much it costs to go to vocational school or college.
13*	My teachers expect that I will do well in the future.
14*	I am one of the students teachers believe will be successful.
15*	My teachers believe that I will graduate from high school.
16*	My teachers consider me to be pretty smart.
17* (R)	My teachers don’t think I’ll go to college.
18* (R)	Most of my good friends will quit high school when they are old enough.
19*	Most of my good friends plan to go to college.
20*	Most of my good friends won’t drop out.
21*	Most of my good friends will get a high school diploma.
22* (R)	I don’t think many of my friends will graduate from high school.
23* (R)	Lots of my good friends won’t be able to go to college.
24* (R)	I don’t think education will guarantee that I get paid well.
25* (R)	I can make good money without an education.
26* (R)	Many of the things we do in school seem useless to me.
27* (R)	If I get bad grades, I can still get a good job.
28* (R)	I could be successful in life without an education.
29* (R)	I know many people who have done well in life with little education.
30* (R)	School is not that important for future success.
31*	I will make more money someday if I do well in school.
32*	If I work hard in school, I will get a better job than the kids who don’t try hard.

Note. These 32 items comprise the final scale. An asterisk after the item number indicates that the item was extracted from the Educational Motivation Questionnaire (Murdock, 1999). An “R” in parentheses indicates reverse-scored items. Students were asked to indicate the degree to which they agreed or disagreed with each of the statements using a 5-point scale (1 = strongly disagree, 5 = strongly agree).