Invariance, or Noninvariance: That Is the Question

Abstract

Nesselroade et al. developed a new, interesting way to enforce invariance at the second-order level in P-technique models, while letting first-order structure to stray from invariance. We discuss our concerns with this approach under the headings of falsifiability, the nature of manifest variables included in models, and differential emphasis on generators of behavior versus output behavior. We supply a modified idiographic model for P-technique analyses and discuss the utility of this new approach in comparison to the Nesselroade et al. procedure.

In their target article, Nesselroade, Gerstorf, Hardy, and Ram (in press) offered an interesting and potentially useful approach to identifying invariant structure in data, using a modified method of P-technique factor analysis with atypical constraints. When using confirmatory factor analysis (CFA) with multiple groups, comparisons are made by enforcing certain invariance constraints across groups to ensure that comparisons are made among parameters that are on a comparable scale. With particular invariance constraints in place, interest is directed to the parameters that vary across groups. Using a hydraulic metaphor, one “pushes down” with invariance constraints on one part of the model, watching for the differences that “well up” in other parts of the model. Typically, invariance constraints are placed on the measurement model (e.g., factor loadings, unique factor variances), and differences between groups are revealed in the structural relations among latent variables. Nesselroade et al. took the unusual approach of “pushing down” with invariance constraints on the structural relations – forcing the factor covariance matrices to be invariant matrices of correlations among latent variables – and allowing differences in factor patterns to “well up” across groups.

Falsifiability

One issue with the Nesselroade et al. (in press) approach that deserves greater consideration is the issue of falsifiability, the criterion serving to separate science from non-science proposed by Popper (1934/1959). Popper argued that, for an investigation to be considered scientific, one must state – prior to initiating a study – the empirical outcomes of an investigation that would require the rejection of the theory that motivated the study. That is, an investigator must say something like: “If my theory were true (or at least reasonable), then my results should conform to a particular pattern. But, if the results come out very differently, my theory must be wrong, and I must reject it.” Nesselroade et al. (in press) mentioned the issue of falsifiability, but did not pursue the topic in any detail.

Invariance, like beauty, may lie in the eye of the beholder. Certain of the Nesselroade et al. factors appeared to have reasonably similar factor patterns across groups (i.e., Factors 1, 2, and 3). But, even with these factors, the variance across groups in which indicators loaded on which factors was unsettling. The remaining two factors showed even less similarity. Thus, only 2 indicators consistently loaded on Factor 4 across 4 of the 5 groups, and no single indicator loaded on Factor 5 across 4 of the 5 groups. With this lack of stability in the pattern of indicators loading on Factors 4 and 5, interpreting either the correlations of Factors 4 and 5 with the remaining factors or the second-order factor loadings for these factors (see Tables 1 and 7, respectively, of Nesselroade et al.) is challenging at best. Moreover, even if one accepts the current solutions in Nesselroade et al. as adequate representations of these data, one wonders how great the lack of invariance in loading patterns would have to be to force one to reject the modeling, concluding instead that the results falsify the theory driving the research.

Manifest indicators in dynamic models

A second issue of importance is the nature of the manifest indicators included in a statistical model, particularly P-technique or dynamic factor models. The manifest variables utilized by Nesselroade et al. (in press) were daily ratings, presumably on 1-to-5 or 1-to-7 scales, on 75 affect items completed by 5 women over more than 100 days. Clearly, problems arose in manipulating the data, as many items had to be discarded due to low variance. Substantial work has been published on the use of item parcels (i.e., simple sums of items) in models, with different rationales for parceling being offered (e.g., Kishton & Widaman, 1994) and summary reviews of pros and cons of parceling available (e.g., Little, Cunningham, Shahar, & Widaman, 2002). Parcels typically have substantially better psychometric properties than do the individual items they comprise. The use of item parcels, rather than individual items, might circumvent at least some of the problems confronted by Nesselroade et al. in their analyses. Further, use of parcels might have led to greater consistency in loadings patterns across groups than occurred in the reported item-based analyses.

Emphasizing generators versus outputs

A third issue concerns a central point to the entire P-technique enterprise – whether the focus of the investigation is on the underlying processes that generate the data or on the outputs of those processes. In factor analytic models, the latent variables are assumed to represent, in some fashion, the processes that generate the manifest variables. In turn, the manifest variables are the outputs of those processes.

The typical approach to factorial invariance is to place constraints on factor loadings, so we know that we are using the same measurement units across groups, and then investigate the differing relations among latent variables across groups. In the present application, the groups correspond to individuals who were measured across time, and the differing relations might signify different degrees of dependence across individuals among particular latent variables and/or different variability across time on given latent variables. This seems to conform to our everyday impressions of other persons: (a) some individuals behave very similarly from day to day, whereas other persons exhibit much greater variability in behavior across days; (b) some individuals seem to have high levels of happiness and well-being only if they have high levels of social interaction with others, whereas the happiness and well-being of other persons is driven down by high levels of “required” social interaction with others; and so on. Thus, the underlying generators of behavior are differently structured across persons under this approach, and this differential structuring reflects the idiographic composition of each individual's personality.

The Nesselroade et al. (in press) approach is very different. The approach forces the underlying generators of behavior to be invariantly structured across individuals, but allows the generators to play out in differential effects on different manifest variables. Thus, a given manifest variable can “means something different” – or reflect different generators – across individuals, and the focus of the analysis seems to shift from the apparent differential functioning across persons of the generators of behavior to the differential impacts of the generators on manifest behavioral variables. Admittedly, Nesselroade et al. were interested in the generators, given their invariance constraints on the relations among the latent generators. But, the differing patterns of effects of the generators on the manifest variables make interpretation of the invariant correlations difficult. The present approach by Nesselroade et al. (in press) seems a bit like pressing down on the heads of 5 persons standing in quicksand until they are all the same “height” – meaning that the top of each person's head is the same height above the top of the quicksand, proclaiming that the heights of the individuals are invariant, and then noting that the observation of one person's legs “stops” at the ankles, observation of another person's legs “stops” at mid-thigh, and so forth. Is satisfying invariance identified with such an approach?

Alternate approaches to specifying idiographic filters

As a final issue, we agree that idiographic filters might be just the answer for many questions, but argue that other alternatives to idiographic filters might be considered. Given the impetus by Nesselroade et al. (in press) to pursue altered notions of factorial invariance, at least one alternative approach to specifying idiographic filters in P-technique and dynamic factor models deserves consideration. Let us assume the presence of p manifest variables (i = 1, …, p), r latent variables (j = 1, …, r), and g groups (k = 1, …, g) where groups can constitute individuals or dyads or another level of abstraction, and assume that each group is assessed across a large number of times of measurement. If covariances among the p manifest variables are calculated across times of measurement, we end up with g (p × p) matrices of covariances among manifest variables. Consider the following equations for representing the latent structure of the data:

$${\sum }_g=D_{M_g}{\Lambda }_g{D}_{L_g}{\Psi }_g{D}_{L_g}^{\prime }{\Lambda }_g^{\prime }{D}_{M_g}^{\prime }+{\Theta }_g$$ (1)

and a slightly modified version as:

$${\sum }_g=D_{M_g}\left[{\Lambda }_g{D}_{L_g}{\Psi }_g{D}_{L_g}^{\prime }{\Lambda }_g^{\prime }+{\Theta }_g\right]D_{M_g}^{\prime }$$ (2)

where Σ_g is a (p × p) matrix of covariances among manifest variables in group g, D_Mg is a (p × p) diagonal matrix of scaling coefficients for the manifest variables (hence the M in the subscript) in group g, Λ_g is a (p × r) matrix of factor loadings, D_Lg is an (r × r) diagonal matrix of scaling coefficients for latent variables (hence the L in the subscript) for group g, Ψ_g (where diag[Ψ_g] = I) is an (r × r) matrix of correlations among latent variables in group g, and Θ_g is a (p × p) diagonal matrix of unique factor variances for group g. The g subscript on all matrices indicates that the parameters in the matrix can freely vary across groups. Equations 1 and 2 differ in one small way – specifically whether the D_Mg matrix rescales only the common factor part of the representation (Equation 1) or rescales the entire factor representation including unique factor variances (Equation 2).

Traditional views of factor invariance impose constraints on Λ_g, allowing for inspection of interindividual differences in D_Lg and Ψ_g. Nesselroade et al. imposed invariance constraints on D_Lg and Ψ_g, allowing for interindividual differences inΛ_g. The constraints imposed by Nesselroade et al. suggest that each individual has the same amount of variance at the latent level (D_Lg) and the same latent variable correlation matrix (Ψ_g). The first of these statements may be seen as problematic considering interindividual differences in the amount of within-person variation has been shown to be a useful individual difference variable (see Nesselroade & Ram, 2004). A possible alternate form of the model could be specified such that Ψ_g is invariant and aspects of Λ_g are invariant. Therefore, we retain the invariance of the factor correlations, but allow for interindividual differences in factor variation. Constraining minimal aspects of Λ_g to be invariant would allow for individual measurement, and partial factorial invariance at the measurement level would further strengthen inferences made at the latent level.

A second alternative is to specify Λ_g to be invariant across groups (Λ_g=Λ). The initial assumption may be that the loadings in Λ conform to a congeneric pattern (i.e., that each manifest variable has a non-zero loading on only one latent variable) and that all latent variable loadings are fixed at either 0 or 1. With 9 manifest variables and 3 latent variables, Λ might have the following pattern:

$$\Lambda =\left[\begin{array}{ccc}1& 0& 0\\ 1& 0& 0\\ 1& 0& 0\\ 0& 1& 0\\ 0& 1& 0\\ 0& 1& 0\\ 0& 0& 1\\ 0& 0& 1\\ 0& 0& 1\end{array}\right]$$

Identification conditions for this model would have to be developed and are clearly beyond the scope of the current paper. Minimal identification restrictions appear to be that r elements of D_Mg, one per latent variable, must be fixed to a non-zero value and this constraint must be invoked in each of the g groups. For example, with r = 3 latent variables, 3 elements of D_Mg – one element per latent variable – must be fixed to a given non-zero value, and the same value must be used in each group to ensure that the latent variables would be on a comparable scale across groups. Which indicator for a given latent variable has the fixed element in D_Mg and the value of the fixed element are choices that deserve further attention. Additionally, whether certain loadings in Λ could be estimated, rather than fixed at zero or unity, is a topic for future investigation, as is the possibility of allowing cross-loadings for indicators.

The models in Equations 1 and 2 are more general than the typical CFA model. If one constrained the D_Mg matrices to invariance across groups and this constraint were not rejected, the resulting model would be essentially equivalent to one having the usual constraint of loading invariance. That is, we could drop the g subscript from the D_Mg matrix and have an invariant factor loading matrixΛ*, whereΛ* = D_MΛ. However, if one or more manifest variables showed little variance within certain groups, this could be accommodated by allowing the corresponding elements of the D_Mg matrices to approach or reach zero, resulting in a form of partial measurement invariance. Similar comments apply to the covariances among the latent variables. If we define the covariance matrix Ψ_g* in the following way, ${\Psi }_g\ast =D_{L_g}{\Psi }_g{D}_{L_g}^{\prime }$, then we can consider invariance constraints across groups in either the correlations among the latent variable (in Ψ_g), the variability on the factors (in D_Lg), or both. Thus, the D_Mg and D_Lg are idiographic filters governing represented variance on manifest variables and latent variables, respectively, and the correlations among latent variables in Ψ_g allow for the differential, idiographic structuring of latent variables across individuals. Contrasting with the Nesselroade et al. approach, we are “pushing down” with invariance constraints on the measurement model, allowing differences to “well up” on the latent variables, but our model loosens considerably and flexibly the traditional constraints in CFA models for P-technique factor analysis.

In closing, Nesselroade et al. (in press) cannot be faulted for having advanced an uninteresting idea; their approach is bold indeed. If we fail to embrace fully their particular approach, we do so at our peril: future developments may well prove that Nesselroade et al. have truly got it right. Regardless, we applaud the idiographic focus of Nesselroade et al., and we look forward to working on approaches that capture human individuality in more adequate ways.