Self-Report Measures of Individual Differences in Regulatory Focus: A Cautionary Note

Abstract

Regulatory focus theory distinguishes between two independent structures of strategic inclination, promotion versus prevention. However, the theory implies two potentially independent definitions of these inclinations, the self-guide versus the reference-point definitions. Two scales (the Regulatory Focus Questionnaire, Higgins al., 2001, and the General Regulatory Focus Measure, Lockwood, Jordan, & Kunda, 2002) have been widely used to measure dispositional regulatory focus. We suggest that these two scales align respectively with the two definitions, and find that the two scales are largely uncorrelated. Both conceptual and methodological implications are discussed.

Regulatory focus theory (Higgins, 1997) distinguishes between two modes of motivational regulation, termed promotion and prevention focus, which respectively emphasize attention to desires and potential gains and attention to obligations and potential losses. This framework of motivational processes has been highly influential, and has been implicated in such widely ranging effects as the complexity of language use (Semin, Higgins, de Montes, Estourget, & Valencia, 2005), the accuracy of estimates of probability (Brockner, Paruchuri, Idson, Higgins, 2002), and the initiation of goal related activity (Freitas, Liberman, Salovey, & Higgins, 2002).

Despite the ubiquity and utility of regulatory focus theory, questions remain about both this construct and its measurement. Conceptually, the seminal statement of regulatory focus (Higgins, 1997) presented two distinct conceptualizations of regulatory focus: the self-guide definition, based on whether goals are derived from an attention to desires versus obligations, and the reference-point definition, based on the end-state to which current goal progress is compared. Do these definitions in fact represent a single, unitary construct? Methodologically, two distinct measures of regulatory focus have been used in previous investigations: the regulatory focus questionnaire (RFQ, Higgins et al., 2001) and the general regulatory focus measure (GRFM, Lockwood, Jordan, & Kunda, 2002). Are these scales in fact measures of the same construct? The present research is the first to directly examine these questions.

One conceptualization of regulatory focus, which we term the self-guide definition, distinguishes promotion and prevention in terms of the degree to which two possible “self-guides” are used for regulation. In this definition, promotion emphasizes internal standards, whereas prevention emphasizes external or socially based standards. Thus, in the self-guide definition, promotion focus is defined in terms of a focus on achieving personally important aspirations, ideals, and ambitions (i.e., an “ideal-self” guide). By contrast, prevention is defined in terms of a focus on fulfilling duties, obligations, and responsibilities that are conveyed from parents or other authority figures, or are intrinsic to the adherence to social roles, such parenting or teaching (i.e., an “ought-self” guide).

The second definition of regulatory focus, which we term the reference-point definition, distinguishes promotion and prevention focus on the basis of which of two possible end-states is used in goal regulation. In this definition, promotion focus is defined as regulation centering on the positive reference point of a “gain” (i.e., a goal to reach a desirable or pleasurable end-state and avoid the absence of these states), whereas prevention focus is defined as regulation centering on the negative reference point of a “loss” (i.e., a goal to steer clear of an undesirable or unpleasant end-state and attain an absence of these states). A promotion goal is thus accomplished when the current state matches the desired state of a gain, whereas a prevention goal is accomplished when a state of non-loss has been attained. Higgins (1997) emphasized that regulatory focus was orthogonal to an alternative framework of motivation, approach and avoidance motivation. Both promotion and prevention were argued to encompass approach of desired end-states (gains in the case of promotion, and non-losses in the case of prevention) and avoidance of undesired end-states (non-gains and losses, respectively). Similarly, whereas approach and avoidance are closely related to positive and negative affect, respectively, promotion and prevention were theorized to be independent of affective valence (Idson, Liberman, & Higgins, 2000).

Just as two definitions of regulatory focus exist, two scales, the regulatory focus questionnaire (RFQ, Higgins et al., 2001) and the general regulatory focus measure (GRFM, Lockwood, Jordan, & Kunda, 2002), have dominated the literature. The RFQ was derived from a factor analysis of items assessing the history of individuals’ success at promotion and prevention tasks over the course of their lives (see Higgins et al., 2001 for a complete discussion of the development and validation of this measure). In contrast, the GRFM, which originated as an ad hoc scale in research on motivation, is tailored to the participant population of undergraduate students, emphasizing success and failure at academic goals. These two scales have been used in a variety of recent investigations, but with little apparent overlap in topic area. The RFQ has been shown to predict emotional outcomes, such as guilt and coping (e.g. Camacho, Higgins, & Luger, 2003), cognitive outcomes, such as persuasion and language use (e.g. Semin et al., 2005) and both mental and physical health outcomes (e.g., Strauman et al., 2006). The GRFM, in contrast, has primarily been used in investigations examining role models (e.g., Lockwood, Chasten, & Wong, 2005), and in applied research (e.g., Yeo & Park, 2006). Despite their differences in content, development, and applications, both scales share the limitation of being self-report measures and thus are limited by the degree to which participants possess insight into their own motivational state and experiences. Nonetheless, these scales represent the dominant approach to measuring regulatory focus on the trait level, and the range of findings with each scale suggests they possess adequate predictive validity.

An examination of the items in these two measures suggests that the RFQ primarily centers on the self-guide conceptualization of ideals versus obligations, with a significant portion of items dealing with parental interaction and other past self-guide experiences capturing the obligation aspect of prevention focus. In contrast, the GRFM closely follows the reference-point definition, with items emphasizing academic achievement in the present. Thus, the key theoretical question of this paper—whether the self-guide and reference-point definitions are empirically unitary—is paralleled by the methodological question of how closely these two major self-report measures are related In Study 1, we present a principal components analysis of these two scales that shows that they load on distinct factors, and that rather than being orthogonal to approach and avoidance (measured with the BIS/BAS scale; Carver & White, 1994), the reference point definition seems to overlap it. In Study 2, we investigated whether this overlap was simply an artifact of the structure of the GRFM, and, in Study 3, whether it extended to affect.

Study 1

Methods

Five hundred and four University of Illinois undergraduates enrolled in an introductory social psychology course participated in mass testing sessions in exchange for course credit.

A pool of 48 items, representing the RFQ (Higgins et al., 2001), the GRFM (Lockwood, Jordan, & Kunda, 2002), and the BIS/BAS scale (Carver & White 1994), were presented using 9-point Likert scales. Items were anchored with endpoints “strongly disagree/strongly agree” or with “never or seldom/very often,” as appropriate based on item wording. Four forms of the questionnaire were created to remove concerns about order effects on the items.

Results

Eigenvalues indicated that 3 factors adequately accounted for the correlation matrix (eigenvalues = 9.57, 7.58, and 4.42). A principal components analysis was performed with a 3-factor solution specified, and the results subjected to varimax rotation.

The first two factors corresponded to gain and loss. Sixteen items, deriving from all three of the contributing scale sources, had loadings greater than .50 on the first factor. These items made reference to positive outcomes, ambition, achievement, and success (e.g., “In general, I am focused on achieving positive outcomes in my life;” “I typically focus on the success I hope to achieve in the future;” “I feel like I have made progress toward being successful in my life”). Conversely, the 12 items showing loadings greater than .50 on the second factor made reference to failure, loss, obligation, and anxiety (e.g., “I often worry that I will fail to accomplish my academic goals;” “I frequently think about how I can prevent failures in my life;” “I feel pretty worried or upset when I think or know somebody is angry at me”). The third factor had the strongest loadings by the subset of RFQ items involving obedience (e.g., “Growing up, would you ever ‘cross the line’ by doing things that your parents would not tolerate?”)

This pattern of relationships was supported by the simple correlations between the three component scales. The GRFM promotion subscale showed a higher correlation to the approach subscale of the BIS/BAS (r = .55, p < .001) than to the RFQ promotion subscale (r = .39, p < .001), t(497) = 3.56, p < .001. The correlation between the BAS and RFQ promotion subscale was .32, p < .001. Similarly, the prevention subscale of GRFM was more strongly related to the avoidance subscale of the BIS/BAS (r = .54, p < .001) than to the prevention subscale of the RFQ (r = −.15, p < .01), t(497) = 21.7, p < .001. As might be expected given that the RFQ prevention items loaded on a third, orthogonal factor, the correlation between the BIS and the RFQ prevention subscale was .05, n.s..

Discussion

Items corresponding to the ought-self guide aspect of the self-guide definition did appear to be orthogonal to the theoretically unrelated construct of approach and avoidance; however, they were also orthogonal to all other regulatory focus items. (In fact, the RFQ prevention subscale was actually negatively correlated with the GRFM prevention subscale.) In contrast, the GRFM and the BIS/BAS loaded equally well on the first two factors, suggesting that the two scales may tap into the same underlying construct. The present study leaves open the possibility that this result is an artifact of the lack of emphasis on non-gains and non-losses in the GRFM, as the principal components analysis only reveals that two scales emphasizing gains and losses both load on factors corresponding to gains and losses. In order to conflict with the reference point definition, we would have to find that items related to non-gains and non-losses were respectively correlated with losses and gains, rather than showing the predicted pattern of a relationship between gains and non-gains and between losses and non-losses. We tested precisely this issue in Study 2, and also examined how these regulatory elements related to affect.

Study 2

Study 1 indicated an overlap between the predominately gain and loss focused subscales of the GRFM and the BIS/BAS. In Study 2, we created a subset of non-gain and non-loss items. The reference-point definition of regulatory focus would predict that items assessing attention to non-gains and gains, representing promotion failure and success, would be highly correlated, as would items focused on non-losses and losses (prevention failure and success).

Method

Sixty-nine University of Illinois undergraduates enrolled in an introductory psychology course participated in computerized testing sessions in exchange for course credit.

Using the factor analysis results from Study 1, we assembled a subset of reference point items (5 promotion, 5 prevention), then rewrote them to manipulate their framing (success vs. failure). Thus, 5 promotion items were framed in terms of success (gain), 5 promotion items were framed in terms of failure (non-gain), 5 prevention items were framed in terms of success (non-loss), and 5 prevention items were framed in terms of failure (loss). See Table 1 for these four item sets. Items were presented in random order. Following these reference point items, participants completed the PANAS (Watson, Clark, & Tellegen, 1988), a well-established measure of positive and negative affectivity; these items were also ordered randomly.

Table 1.

Reference-point Subscales (Study 2)

Gain (α = .82)	Non-Gain (α = .67)
Right now, I am focused on achieving positive outcomes.	Right now, I am concerned about missing out on positive outcomes
I typically focus on the success I hope to achieve in the future.	I frequently worry that my future will be less successful than I hope.
I frequently imagine how I will achieve my hopes and aspirations.	I frequently imagine how I might fall short of my hopes and aspirations.
When good things happen to me, it affects me strongly.	When good things fail to materialize, it affects me strongly.
I see myself as someone who is primarily striving to reach my "ideal self"—to fulfill my hopes, wishes, and aspirations.	I see myself as someone who has trouble reaching my "ideal self" (fulfilling my hopes, wishes, and aspirations).
Non-Loss (α = .68)	Loss (α = .66)
I am generally sure that I can bypass negative events in my life.	I frequently think about how I can prevent failures in my life.
I am confident that I can meet my responsibilities and obligations	I am anxious that I will fall short of my responsibilities and obligations.
Right now, I am focused on protecting myself against negative outcomes.	Right now, I am focused on avoiding negative outcomes.
I am generally good at avoiding careless mistakes.	I worry about making mistakes.
I often imagine myself successfully preventing bad things from happening to me.	I often imagine myself experiencing bad things that I fear might happen to me.

Results

The promotion scale framed as a success (i.e., a gain) did not correlate with the promotion scale framed as a failure (i.e., non-gain), r = −.09, n.s.. Similarly, the prevention scale framed in terms of failure (i.e., loss) did not correlate with the prevention scale that was framed in terms of success (non-loss), r = −.01, n.s.. In contrast, the promotion-success (gain) and prevention-success (non-loss) subscales correlated substantially, r = .41, p < .001, as did the promotion-failure (loss) and prevention-failure (non-gain) subscales, r = .53, p < .001.

This pattern continued into the relation of these four scales with positive and negative affectivity. Whereas promotion-success (gain) was positively correlated with positive affect (PA), r = .48, p < .001, reframing it into promotion-failure (non-gain) reversed the correlation with PA (r = −.31, p < .001). Similarly, the correlation with negative affect (NA) shifted from −.16 (n.s.) for promotion-success to .52 for promotion-failure, p < .001. When prevention-failure (loss) was reframed into prevention-success (non-loss), the correlation with NA shifted from .50 (p < .001) to .20 (n.s.), whereas the correlation with PA shifted from .01 (n.s.) to .28 (p = .02).

Discussion

Study 2 indicates that the overlap of the GRFM and the BIS/BAS subscales in Study 1 was not a mere artifact of the GRFM, but may instead represent a deeper issue with the reference-point definition of regulatory focus. Not only is attention to losses not related to the use of an ought-self guide (Study 1), but it is unrelated to non-losses and correlated with attention to non-gains, in contrast to the defining reference-point structure. Moreover, far from being unrelated to affective valence, we found that the four reference-point subscales bore significant relations to affectivity. In Study 3, we tested whether the established measures, the RFQ and the GRFM, showed this same relationship to affectivity.

Study 3

In Study 1, we found that the GRFM and RFQ did not relate as would have been expected if the reference-point and self-guide definitions represented a unitary construct. Given the unexpected associations of our four subscales with affectivity in Study 2, we examined in Study 3 whether either scale violated the principle that regulatory focus is independent of valence.

Method

Forty-one University of Illinois undergraduates enrolled in an introductory psychology course participated in computerized testing sessions in exchange for course credit. Items from the RFQ and GRFM scales were interspersed and presented in random order; participants then completed the PANAS, items of which were also presented in random order.

Results and Discussion

The GRFM promotion subscale and the PANAS positive affect (PA) subscale were significantly correlated, r = .31, p < .05, as were the prevention and negative affect (NA) subscales, r = .39, p = .01. Neither promotion and PA nor prevention and NA were significantly related for the RFQ (rs = .27, −.26, ps = .09, .10). As in Study 1, then, the GRFM was significantly related to constructs which are theoretically orthogonal to regulatory focus.

General Discussion

The GRFM and the RFQ are both billed as measures of individual differences in regulatory focus, yet participants’ responses on the two measures are largely unrelated. We suggest that this stems from the existence of two potentially independent definitions of regulatory focus, which are each assessed by one of these respective scales. The GRFM focuses on the reference-point definition, whereas the RFQ focuses on the selfguide definition. On a conceptual level, this dissociation of these measures suggests that the two definitions of regulatory focus may represent two unique constructs, rather than a single phenomenon. In particular, our results raise questions about the validity of the reference point definition. We found in two studies that both an existing scale, the GRFM, and a newly constructed scale that included non-losses and non-gains both showed an unexpected correlation with affect (as well as between losses and non-gains, and non-losses with gains, in the case of the new measure).

A true assessment of the validity of the reference-point definition is well beyond the scope of this brief note, though we strongly encourage future programmatic research to investigate it. However, we do feel safe in strongly cautioning researchers interested in regulatory focus to exercise caution in selecting a scale and interpreting the results of both present and prior research. In particular, researchers should be aware that the reference-point-based GRFM functions much more like a measure of approach and avoidance (the BIS/BAS) than like the RFQ, which is closer to the self-guide definition. Moreover, given the lack of association between the two measures, results obtained with the RFQ may not extend to the GRFM (and vice-versa). For example, although we replicated the findings of Idson, Liberman, and Higgins (2000) that the RFQ was not related to affect valence, we found that the GRFM was significantly associated with positive and negative affectivity.

In fact, we would suggest that the difference between the GRFM and the RFQ pivots on affect. When defined in terms of reference points, as in the GRFM, promotion focus is associated with positive affectivity, whereas prevention focus is associated with negative affectivity. Moreover, this connection was not a simple artifact of the GRFM’s structure. When we reframed a subset of items to balance across success vs. failure within both promotion and prevention, the pattern of relations to affectivity remained but simply reversed in direction. Although this may be a function of linguistic limitations of both writing and responding to items, it appears difficult to disentangle the referencepoint definition of regulatory focus from affectivity using self-report measures. Indeed, if the connection stems from outcome valence and affect being processed by the same brain systems, as proposed by Gray (1990), disentanglement may be impossible (cf. Cacioppo, Gardner, & Berntson, 1999). Thus, whenever researchers draw conclusions on the basis of reference-point measures, they must be mindful of the fact that their findings will contain significant variation in affectivity. A particularly cautious reading of these data would say that any individual difference measurement of reference-point focus will necessarily involve measurement of emotional tendencies, in violation of the original conceptualization of regulatory focus theory. Researchers concerned by this association would be wise to focus instead on the self-guide definition.