Measuring Loneliness in Different Age Groups: The Measurement Invariance of the UCLA Loneliness Scale

Abstract

Age differences in the prevalence of loneliness have been a key focus among researchers, practitioners, and policy makers. However, the degree to which those reflect genuine differences in the experience of loneliness or the way individuals understand and respond to loneliness measures is yet to be examined. The current study explored the age measurement invariance of the 20-item Revised University of California Los Angeles, Loneliness Scale (UCLA-LSR) and its shorter forms in a U.K. sample of adults aged 18 to 99 years (M = 50.6, SD = 19.7). The fit of different structures/versions was explored through multigroup confirmatory factor analysis (CFA; N = 4,375) and local structural equation modeling (N = 19,521). Results indicated a poor and/or inconsistent structure for the 20-item UCLA-LSR and many of its shorter forms. Of the structures considered, 12 showed acceptable model fit and received age measurement invariance testing through multigroup CFA and alignment; 10 of these achieved full, partial, or approximate measurement invariance. Our findings suggest that the age measurement invariance of loneliness measures should not be assumed, and crucially, this must be explored before accurate and meaningful age comparisons can be made. Implications for measurement research, and clinical and community practice, are discussed.

Loneliness is described as the discrepancy between desired and perceived social relations (Peplau & Perlman, 1982). It is considered to be a public health priority (Gerst-Emerson & Jayawardhana, 2015; Holt-Lunstad, 2017) because of its detrimental impact on mental and physical health and educational and workplace performance (Lim et al., 2020). Post the onset of the COVID-19 pandemic, researchers, practitioners, and policy makers have highlighted the importance of understanding how to better address loneliness (Bu et al., 2020; Department for Digital, Culture, Media & Sport et al., 2020) and measure it accurately across groups, including those differentiated by age.

Evidence on the age differences in loneliness has been mixed (see Hawkley et al., 2020 for a recent review of the literature). Some work suggests heightened levels in younger groups and a decrease in loneliness with age (Barreto et al., 2021; Ending Loneliness Together, 2020; Office for National Statistics [ONS], 2018a), while other studies find support for a nonlinear relationship between age and loneliness, with late adolescents and older adults experiencing heightened levels (Luhmann & Hawkley, 2016; Victor & Yang, 2012; Yang & Victor, 2011). In some cases, peaks in middle adulthood were also reported (Hawkley et al., 2020). The variation in sampling techniques and inclusion of key correlates as well as the inconsistent definition and measurement of loneliness might explain some of these inconsistencies (Hawkley et al., 2020). Despite the strong focus on age differences, there are no studies that have investigated such potential differences across common tools of loneliness. In other words, there is insufficient evidence of measurement invariance by age―the degree to which a scale measures the same thing across different age groups of people. Hence, it is difficult to ascertain whether differences in loneliness levels are due to true differences in the experience of loneliness or due to differences in the way age groups understand and respond to loneliness questionnaires. In the current study, we fill that gap in our understanding by exploring the age invariance of the University of California Los Angeles, Loneliness Scale (UCLA-LS) and its short forms.

The UCLA-LS and its associated short forms are the focus of the current study because they are the most widely used and recommended measures of loneliness by local and national initiatives tackling loneliness. Despite its popularity, the history of the UCLA-LS speaks to the measurement issues observed in the loneliness literature (Maes et al., 2022). Both the 20-item Revised UCLA-LS (UCLA-LSR; Russell et al., 1980) and its updated form (UCLA-LS3; Russell, 1996) receive substantial modifications and abbreviations to this day (e.g., Wongpakaran et al., 2020), resulting in multiple different versions, with limited and or inconsistent evidence on their structural validity (Maes et al., 2022). Support has been found for a single, unified construct of loneliness (e.g., Dodeen, 2014; Hartshorne, 1993; Lasgaard, 2007); however, a number of studies have failed to find evidence for the unidimensionality of the 20-item UCLA-LS (Durak & Senol-Durak, 2010; Dussault et al., 2009; Elphinstone, 2018; Hawkley et al., 2005; Knight et al., 1988). A two-factor structure has been proposed by some, with negative and positive valenced items representing distinct domains, consistent with evidence for reverse-wording effects (Dodeen, 2014; Knight et al., 1988). Three-factor solutions have also been proposed, tapping constructs of intimate others, social others, and affiliative environment (McWhirter, 1990) and constructs of intimate connectedness, relational connectedness, and collective connectedness (Badcock et al., 2016; Hawkley et al., 2005). While a few studies suggest a three-factor bifactor model as the best-fitting solution (Auné et al., 2022; Dodeen, 2014; Elphinstone, 2018), such proposals should be interpreted with caution, given the tendency of bifactor models to achieve superior fit even with random or implausible solutions (Bonifay et al., 2017; Reise et al., 2016). The structural validity of the UCLA-LS, specifically within U.K. population, is currently lacking, with the exception of Shevlin et al. (2015) who found support for Hawkley and colleagues’ (2005) three-factor structure for the 20-item measure, in a sample of adolescents.

Regardless of which is the best structure for the UCLA-LS, it is critical that it can be used to accurately measure loneliness in people of different ages―there needs to be measurement invariance. Measurement invariance refers to different groups reporting the same scores on the survey items when they have the same level of the underlying trait (Reise et al., 1993). This means that the UCLA-LS should function in the same way for individuals with the same levels of loneliness, irrespective of the group they belong to (in the same way a weight scale should be functioning the same way for people with the same weight; Millsap, 2012). Therefore, for meaningful comparisons between groups of people, measurement invariance across those groups is a prerequisite (van de Schoot et al., 2013). If there is weak or no support for invariance, then it follows that the groups in question cannot be justifiably compared because the same observed score for the groups does not reflect the same level of the underlying trait.

Currently, when researchers examine age differences in loneliness using the UCLA-LS, they make the implicit assumption that the measure is indeed invariant across different age groups. However, such an assertion is untenable given that the age measurement invariance for the UCLA-LS and its short forms is yet to be confirmed. Russell (1996) was the first to speak of possible invariance problems with the UCLA-LS3, although this was not formally explored in that study. Since then, evidence has been scarce, despite the prominence of the measure in establishing age differences or stability in loneliness levels (Luhmann & Hawkley, 2016; Mund et al., 2020; Schultz & Moore, 1988). However, the degree to which these are true mean differences or differences in the measurement properties of the scale is still unclear. An attempt by Penning et al. (2014) to explore this issue in the UCLA-LSR was unsuccessful: a proposed four-factor structure failed to meet minimum criteria (i.e., acceptable configural model) for a measurement invariance test in a Canadian sample. The low sample size (≤ 70 per group) of that study may partially explain their findings (Meade & Bauer, 2007), although the method effects associated with the UCLA-LSR item wording and the lack of an adequate structure (Penning et al., 2014) suggest caution in its use and a clear need for further exploration.

While it is unclear what led to such a crucial gap in the UCLA-LS psychometric validation research, the inconsistent structure of the measure and the varied conceptualization of age groups are likely among the most important challenges. Indeed, existing literature focusing on age differences has inconsistently defined and assessed age groups. From a theoretical perspective, definitions of aging and developmental periods vary based on sociocultural environments (Zittoun & Baucal, 2021) and the disease/condition under study (Geifman et al., 2013), and are constantly changing due to increases in life expectancy, and changes in the socioeconomic status of individuals (e.g., Sawyer et al., 2018). The ambiguity in the definition of age groups also poses statistical challenges because invariance may vary from study to study. In addition, distinct age groups are also often based on unbalanced sample sizes within personality and loneliness research (Brandt et al., 2020; Danneel et al., 2018; Maes et al., 2014), thus threatening the validity of the results (Yoon & Lai, 2018).

Given that an agreed categorization of adulthood into distinct periods is currently lacking (Lea, 2017), the current study focused on the following periods consistent with developmental theories and evidence (Eriksson et al., 2020; Qualter et al., 2015; Sawyer et al., 2018; Zarrett & Eccles, 2006): late adolescence, early adulthood, middle adulthood, and older age. These groups are characterized by distinct developments and challenges. For example, late adolescents experience many changes in relation to their social and personal identity, future goals, and relationships (e.g., independence from parents, intimate partners; Sawyer et al., 2018; Zarrett & Eccles, 2006). Relationship issues have been noted as an important source of loneliness for not only late adolescence but also early and middle adulthood (Qualter et al., 2015). On the other end, older age is characterized by increased frailty, ill-health, and decreased mobility accompanied by the loss of loved ones, all of which were shown to be important factors in the experience of loneliness (Hawkley et al., 2019; Qualter et al., 2015).

The Current Study

The current study examined the structure of the UCLA-LSR and its measurement invariance across age groups in a U.K. sample, which has been notably absent from the literature. First, we explored the factor structure of the UCLA-LSR, focusing on the full, 20-item measure and its performance as a unidimensional, two-factor, and three-factor structure. Second, we explored any short forms proposed in the literature. Third, using balanced age groups, the age measurement invariance of the UCLA-LSR was explored for those factor structures deemed statistically acceptable. Due to the inconsistent structure and fit of the 20-item measure in the literature, the invariance of an Exploratory Structural Equation Modeling (ESEM) model was also explored. Finally, acknowledging that age is inherently a continuous variable, the age measurement invariance of the measures was also explored using local structural equation modeling, which allowed us to treat age as a continuous moderator.

Method

Touch Test Project

Data were drawn from the Touch Test project commissioned by the Wellcome Collection. The Touch Test was an online self-report survey exploring attitudes and experiences related to touch. The survey comprised several measurements, including the UCLA-LSR, and was part of a wider public engagement project (for the full list see Bowling et al., 2020). Participants were recruited through broadcasts on a British national radio station (BBC Radio 4) and social media. All participants were required to have internet access on a computer, smartphone, or tablet to complete the survey. Data collection spanned January 20 to March 31, 2020. All responses are reflective of attitudes prior to Covid-19 lockdown restrictions that were enforced in the United Kingdom from March 26, 2020. Each participant only answered the survey once. After providing consent, participants were able to complete the survey at any point during the following 7 days. Participation was voluntary and those taking part did not receive any monetary reward. Ethics approval was granted by the Goldsmiths University Research Ethics and Integrity Sub-Committee (project reference 1521).

The original sample comprised 24,024 participants aged 18–99 years (M = 56.7, SD = 14.3). Unlike national estimates (females = 51%; ONS, 2022), a higher number self-described as female (n = 17,677; 73.6%), with 24.8% self-describing 24.8% as male (n = 5,946), 0.5% as non-binary (n = 117), and 0.9% indicated that they prefer to self-describe/not to say (n = 226); 58 participants had missing data on the gender item (0.2%). The majority of the sample were White (n = 22,865; 95.2%), 1.5% (n = 368) Mixed or from multiple ethnic groups, 1.2% (n = 282) Asian or Asian British, 0.5% (n = 123) Black, Black British, Caribbean or African, 0.04% (n = 10) Arab, and 0.4% (n = 99) any other ethnic group. The remaining 277 participants had missing data (n = 82; 0.3%) or preferred not to say (n = 195; 0.8%). The ethnic composition of the current sample was not representative of the U.K. population (White = 86%; Asian = 7.5%; Black = 3.3% Mixed = 2.2%; ONS, 2020).

Measure

The 20-item UCLA-LSR contains 10 positively (e.g., “I feel in tune with the people around me”) and 10 negatively (e.g., “I lack companionship”) worded items. Participants respond to each item using a 4-point scale (never, rarely, sometimes, often). This version differs slightly from the more recent UCLA-LS3 (Russell, 1996; see Appendix A for more information).

Analytical Considerations

Several challenges relating to the UCLA-LS and measurement invariance testing were considered and addressed. These are summarized in Table 1. Analyses were carried out in Mplus 8.5 and R, the code of which can be found here: https://osf.io/cqxrd/

Table 1.

Methodological Considerations.

Challenge	Alternative method used
UCLA-LS structure
1. The structure of the UCLA measure is unclear	1. Several structures were considered alongside a three-factor ESEM model
Sample
2. Unbalanced age groups may lead to biased measurement invariance findings	2. Analyses were based on 100 random samples of balanced groups
Measurement invariance testing
3. The chi-square difference test used to compare nested models is sensitive to sample size	3. A CFI difference of .01 was used to compare the nested models
4. The accuracy of CFI difference with polychoric data and WLSMV is not well established	4. Items were treated as continuous and MLR was used to account for skewed data
5. Model modification in the search for partial measurement invariance based on modification indices can be biased	5. Where full measurement invariance was not achieved, alignment was considered
6. Alignment is not implemented yet for ESEM models and modification indices are not available for multiple imputation	6. Model modifications for the ESEM model were based on post-hoc pairwise comparisons using Wald tests
7. RMSEA can be overestimated in models with small samples and degrees of freedom	7. RMSEA was not considered in baseline models with degrees of freedom < 20
8. Breaking a continuous variable such as age into discrete groups has several theoretical and methodological limitations	8. Local structural equation modeling was employed where the age invariance of the measures was considered through a continuous moderator

Note. UCLA-LS = University of California Los Angeles, Loneliness Scale; ESEM = Exploratory Structural Equation Modeling; CFI = comparative fit index; WLSMV = weighted least squares mean and variance adjusted; MLR = robust maximum likelihood; RMSEA = root mean square error of approximation.

Sample

Of the 24,024 participants, 4,503 (18.7%) were removed from the current analyses due to missing data on age and all 20 UCLA items. Data from the remaining 19,521 participants were used in the local structural equation modeling (LSEM) models. To explore the invariance of distinct age groups, this sample was then grouped into seven age groups, which allowed for a more nuanced analysis of age differences. Each age group represents a distinct developmental period, consistent with theory and evidence (discussed above): 18–25 (late adolescence; n = 625), 26–35 (early young adulthood; n = 1,300), 36–45 (late young adulthood; n = 1,846), 46–55 (early middle adulthood; n = 3,800), 56–65 (late middle adulthood; n = 5,919), 66–75 (early old age; n = 4,991), and 76+ (middle-oldest old age; n = 1,040). To ensure that seven balanced groups were used in the current analyses, we followed the analytical technique proposed by Yoon and Lai (2018). This is achieved using Monte Carlo simulation, which draws 100 random datasets from the original data that match the size of the smallest group (in our case 18–25; n = 625). Subsequent analyses are based on those 100 random datasets, and fit indices and parameters represent the average values across the random samples. Our analyses were, therefore, based on seven age groups, each group with a sample size of 625 (total N = 4,375).

Factor Structures

The versions and structures of the UCLA-LS examined in the current study are shown in Table 2. A unidimensional, two-factor (Factor 1: positive items, Factor 2: negative items), and three-factor structures (Hawkley et al., 2005; McWhirter, 1990) were explored for the 20-item measure. Exploratory Structural Equation Modeling (ESEM) was also considered in the scenario that none of the suggested structures achieved an acceptable fit. In such a model, every item is permitted to load onto every specified factor. ESEM provides a unique opportunity to explore the invariance of a measure when said measure fails to meet minimum fit criteria in the more restrictive confirmatory factor analysis (CFA) model. ESEM, therefore, provides a more realistic framework for measurement invariance, with better fit and much more differentiated latent factors (Marsh et al., 2013). The number of factors for the ESEM model in the current study was determined through parallel analysis using principal component eigenvalues. This technique compares the observed eigenvalues to those of randomly drawn data (1000 used here), and only factors with eigenvalues higher than the random ones are extracted (O’Connor, 2000). Three factors had eigenvalues (F1: 9.28, F2: 1.22, F3: 1.19) higher than those derived by chance. Thus, a three-factor ESEM model was explored. The fit of alternative structures and versions suggested in the literature was also explored (see Table 2).

Table 2.

UCLA Loneliness Scale Structures Explored.

Version	Number of items	Baseline model df (RMSEA considered?)	Items	Parent version
UCLA-LS-4
Russell et al. (1980)	4	2 (N)	1, 13, 15, 18	UCLA-LSR, UCLA-LS3
Roberts et al. (1993)	4	2 (N)	2, 7, 11, 14	UCLA-LSR, UCLA-LS3
UCLA-LS-7
Oshagan & Allen (1992)	7	14 (N)	2, 3, 11, 12, 13, 14, 18	UCLA-LSR
UCLA-LS-8
Roberts et al. (1993)	8	20 (Y)	1, 2, 4, 5, 7, 11, 14, 15	UCLA-LSR
Hays & DiMatteo (1987)	8	20 (Y)	2, 3, 9, 11, 14, 15, 17, 18	UCLA-LSR
UCLA-LS-9
Hawkley et al. (2005)	9	24 (Y)	F1: 2, 11, 14 F2: 19, 20, 10 F3: 1, 6, 5	UCLA-LSR, UCLA-LS3
UCLA-LS-10
Russell (1996)	10	35 (Y)	2, 6, 10, 11, 13, 14, 16, 18, 19, 20	UCLA-LSR, UCLA-LS3
Knight et al. (1988) version1 – one-factor	10	35 (Y)	1, 2, 4, 6, 7, 8, 11, 16, 18, 19	UCLA-LSR
Knight et al. (1988) version1 – two-factor	10	34 (Y)	F1: 1, 4, 6, 16, 19 F2: 2, 7, 8, 11, 18
Knight et al. (1988) version2 – one-factor	10	35 (Y)	3, 5, 9, 10, 12, 13, 14, 15, 17, 20	UCLA-LSR
Knight et al. (1988) version2 – two-factor		34 (Y)	F1: 3, 12, 13, 14, 17 F2: 5, 9, 10, 15, 20
UCLA-LS-20
Russell et al. (1980) – one-factor	20	170 (Y)	1–20
Russell et al. (1980) – two-factor	20	169 (Y)	F1: 2, 3, 7, 8, 11, 12, 13, 14, 17, 18 F2: 1, 4, 5, 6, 9, 10, 15, 16, 19, 20	UCLA-LSR, UCLA-LS3
Hawkley et al. (2005) – three-factor	20	167 (Y)	F1: 2, 3, 4, 7, 8, 11, 12, 13, 14, 17, 18 F2: 10, 15, 16, 19, 20 F3: 1, 5, 6, 9	UCLA-LSR
McWhirter (1990) – three-factor	20	167 (Y)	F1: 2, 3, 7, 8, 11, 12, 13, 14, 17, 18 F2: 10, 16, 19, 20 F3: 1, 4, 5, 6, 9, 15	UCLA-LSR

Note. UCLA = University of California Los Angeles; RMSEA = root mean square error of approximation; UCLA-LS-4 = University of California Los Angeles, Loneliness Scale-4-item measures; UCLA-LSR = 20-item Revised University of California Los Angeles, Loneliness Scale; UCLA-LS3 = University of California Los Angeles, Loneliness Scale Version 3; UCLA-LS-7 = University of California Los Angeles, Loneliness Scale-7-item measures; UCLA-LS-8 = University of California Los Angeles, Loneliness Scale-8-item measures; UCLA-LS-9 = University of California Los Angeles, Loneliness Scale-9-item measures; UCLA-LS-10 = University of California Los Angeles, Loneliness Scale-10-item measures; UCLA-LS-20 = University of California Los Angeles, Loneliness Scale-20-item measures.

Measurement Invariance Testing

Treating items as ordinal with weighted least squares mean and variance adjusted (WLSMV) estimation posed several analytical challenges. First, the use of imputed datasets precludes the examination of the chi-square difference test, which is typically used for comparing nested models using WLSMV. In addition, the chi-square difference test can be sensitive to sample size, erroneously supporting a conclusion of non-invariance (Chen, 2007). That would be especially problematic in models with very few parameters, such as the 4-item versions of the loneliness scale. For those reasons, it was considered more appropriate to compare the nested models using alternative fit indices, such as the comparative fit index (CFI) difference. However, the performance of those indices has been largely explored within continuous items using maximum likelihood estimators. Given that the chi-square of WLSMV is not comparable to that of maximum likelihood (Sass et al., 2014), the degree to which the CFI difference is appropriate in models using polychoric matrices and WLSMV is yet to be confirmed. Items were, therefore, treated as continuous with robust maximum likelihood (MLR) to account for the skewed distribution of Item 10 (skewness = 1.43), Item 19 (skewness = 1.30), and Item 20 (skewness = 1.35) and for violations with the multivariate normal distribution assumption (Li, 2016).

The age measurement invariance of the measures was assessed through two methods: (a) multigroup means and covariance structure analyses and (b) LSEM. The former collapses the continuous variable age into discrete age groups and the variance–covariance matrix of the indicators is equal within the members of each group (Hildebrandt et al., 2016). While categorization of age can be arbitrary, this is a common practice within the loneliness literature during comparisons of loneliness levels but also in the development of loneliness measures (e.g., the same items are developed for adolescents and a period that could span a decade). This analysis, therefore, allowed for the exploration of measurement invariance in commonly defined age groups. Still, breaking age into discrete groups has a few theoretical and methodological limitations (e.g., see Hildebrandt et al., 2009; Preacher et al., 2005). Beyond the fact that these groups are arbitrarily defined, this practice can lead to loss of information, increasing the risk of overlooking nonlinear relationships and treats participant responses within each group as equal (they are assumed to be homogeneous). Therefore, what multigroup models fail to consider is the possibility that an individual is more similar to individuals at the edge of the group next to them than individuals in their own group (Basarkod et al., 2022). LSEM is a good alternative that was developed to account for these limitations by exploring the performance of measures on a continuous moderator using weights instead of categorization (Hildebrandt et al., 2016; Hülür et al., 2011). Specifically, instead of grouping participants, the moderator variable, in this case age, is divided into several focal points (i.e., specific values of age). Participants (i.e., observations) are then weighted around those focal points using a Gaussian kernel function (i.e., normally distributed weights) under the assumption that participants that are close to each other on the continuous variable are more similar to participants who are more distant. Therefore, for every focal point, in our case age in years, participants with that focal point receive the highest weight (w = 1), and those farther away from this point receive lower weights (Hildebrandt et al., 2016). For example, the estimated model at the focal point 50 will be strongly influenced not only by observations with that age but also by observations near that age and will be less influenced by younger observations. This is repeated across all the focal points of the moderator, thus allowing testing of how key model parameters (model fit, intercepts, loadings) change across age.

Method 1: Multigroup Measurement Invariance

Measurement invariance within multigroup models followed a four-step procedure (baselines, configural, metric, and scalar/alignment): (a) the baseline models of each group were considered; (b) following acceptable model fit for the baseline models, the configural model in which loadings and intercepts are free to vary across the seven age groups was explored; (c) the configural model was then compared with the metric model in which loadings are held equal; and (d) the metric model was compared with the scalar model, which holds both loadings and intercepts equal across groups. Models in Steps (c) and (d) were compared using the CFI difference, where a CFI increase lower than .01 in the more restrictive model was considered as indicating full measurement invariance (Cheung & Rensvold, 2002).

Generally, an acceptable model fit is considered with a CFI of ≥.95 and standardized root mean squared residual (SRMR) ≤.08. Root mean square error of approximation (RMSEA) values below .05 to .06 are considered good (Fabrigar et al., 1999; Hu & Bentler, 1999), values between .05 and .08 are acceptable, between .08 and .10 are marginal, and values greater than .10 are considered poor (Fabrigar et al., 1999). To ensure that acceptable structures were not missed, a more lenient threshold was used for baseline models, with RMSEA values below .10 being considered adequate for proceeding to invariance testing (Step b). A stricter RMSEA threshold of .08 was used for the configural invariance models. Given the evidence that RMSEA can be overestimated for models with small samples and degrees of freedom, the RMSEA was not considered in baseline models with less than 20 degrees of freedom, following the simulation findings by Kenny and colleagues (2015).

Alignment

Upon failing to find full metric or scalar measurement invariance, researchers typically rely on the multigroup CFA method for exploring partial measurement invariance. That involves multiple post hoc model adjustments guided by the modification indices. This method, however, is less practical with many groups, is more likely to lead to the rejection of full measurement invariance, and suffers from important criticisms inherent to such post hoc stepwise procedures (Marsh et al., 2018). Alignment is a more robust alternative that can be used to estimate factor means and variances without requiring full measurement invariance and can be used to explore the most non-invariant items, and specifically which groups contribute to the observed non-invariance (Asparouhov & Muthén, 2014). Thus, in the absence of full metric or scalar invariance, invariance testing with alignment was considered. However, alignment is not yet available for ESEM models, and relying on traditional multigroup CFA testing was not possible given that modification indices are not available in multiple imputation. Therefore, where full metric or scalar measurement invariance was not achieved for the ESEM model, partial measurement invariance was explored through pairwise statistical comparisons of parameters using Wald tests.

Alignment was carried out in Mplus 8.5 using the FIXED ¹ alignment optimization, which assumes that the intercept of the first group is zero. Given that the average results across multiple imputed datasets are not yet available for alignment, results were extracted in R for each of the 100 datasets using the MplusAutomation package (Version 0.8; Hallquist & Wiley, 2018) and MIE.package (Version 0.5-3; Rudnev, 2020). Results were then averaged using a function developed by the authors. Three pieces of information were extracted: (a) the average R² for loadings and intercepts, which indicates the degree of non-invariance, with values close to 0 indicating high non-invariance, and a value of 1 indicating full invariance (Asparouhov & Muthén, 2014), (b) the weighted average loading and intercept value for each item across the invariant groups, and (c) the approximate non-invariance for each item, based on pairwise comparisons between the groups (Asparouhov & Muthén, 2014). The latter was summarized across the 100 datasets by presenting the number of times the item was found to be non-invariant in a group.

The percentage of intercept and loading non-invariance was also calculated. This is the ratio of non-invariant parameters to the total parameters. For instance, for the 10-item UCLA-LS, a total of 5 non-invariant parameters would equate to 3.6% non-invariance (i.e., 5 non-invariant parameters/140 free parameters [70 for loadings and 70 for intercepts] × 100). The percentage of non-invariance in our study represents the average non-invariance across the 100 random samples (total number of non-invariant parameters/14,000 free parameters × 100). A threshold of 25% non-invariance across both loadings and intercepts was considered to indicate trustworthy alignment results (Muthén & Asparouhov, 2014). Furthermore, a Monte Carlo simulation using the starting values of the aligned models with 1,000 repetitions was carried out to explore the performance of the alignment. Near-perfect correlations between the real data and generated factor means would indicate that the latent means were accurately estimated (Muthén & Asparouhov, 2014).

Method 2: Local Structural Equation Modeling

LSEM models were estimated in R using lavaan 0.6-11 (Rosseel, 2012) and sirt 3.11-21 (Robitzsch, 2022). Focal points on the moderator age were set to every year ranging between 18 and 83 years. Although the maximum age in our sample was 94, the upper limit was set to 83 due to the small samples observed in ages over 83 (n < 25). Therefore, sample weights for 66 focal points were estimated along the moderator. Following Hildebrandt et al. (2016), we set the bandwidth factor at h = 2, which was shown to smooth out noise, while also being accurate to change. A bandwidth factor of h = 2 means that observations farther than 2 times the bandwidth from the focal point receive very small weights. For the 16 different models, we plotted, using ggplot2 3.3.5 (Wickham, 2016), key fit indices (CFI, Tucker–Lewis index [TLI], RMSEA, SRMR) and the loadings and intercepts. In all plots, the x-axis represents the moderator age and the y-axis represents the relevant parameter estimates. The age focal points are where solid lines connect (e.g., with a circle). Due to the volume of different measures/structures, the plots for the loadings can be found in the Supplementary Material (Figures S1–S13).

LSEM measurement invariance was explored only for the models that achieved acceptable configural (free factor loadings and intercepts) model fit. Consistent with the multigroup analysis, configural models with inconsistent fit were not considered for further measurement invariance testing. For measures that received invariance testing, the configural, metric, and scalar measurement invariance models were compared using the CFI difference, with a threshold of .01 (Cheung & Rensvold, 2002). Where full metric and/or scalar invariance were not achieved, pointwise comparisons were modeled to provide information about which ages specifically were shown to have non-invariant parameters. This was achieved through permutation tests. Permutation tests are used in LSEM to statistically test parameter variations (Hildebrandt et al., 2016). These test the null hypothesis that a parameter, in our case the loadings and/or intercepts, is constant across the age values. Due to the volume of models, focal points, and sample size, we used 100 permutations for each model, consistent with previous work (e.g., Basarkod et al., 2022). To account for multiple testing and reduce the likelihood of Type I error, we set the α level to .01.

Results

Multigroup Measurement Invariance

Table 3 summarizes the fit for the baseline models across the seven age groups and the findings from measurement invariance testing. Detailed results for each structure can be found in the Supplementary Material (Tables S1–S20). Generally, a nonviable structure was found for the 20-item UCLA-LSR measure (UCLA-LSR-20). Of those tested, the unidimensional model was shown to have a poor fit across all age groups, while inconsistent fit was found for the two-factor and three-factor structures, rendering these models inappropriate for further measurement invariance testing. As expected, the three-factor ESEM model had a good baseline fit in all age groups but did not achieve full scalar invariance. Pairwise group comparisons were carried out for each of the 20 intercepts, and the alpha level was adjusted to .0025 to control for multiple testing error. As seen in Table 4, 10 items were found to differ in their means between the seven age groups (p < .0025). To explore for partial measurement invariance, the intercepts with the largest chi-square value were freed sequentially. Partial measurement invariance was achieved when the intercepts of Items 17 (“I am unhappy being so withdrawn”), 1 (“I feel in tune with the people around me”), 11 (“I feel left out”) and 14 (“I feel isolated from others”) were allowed to vary across the age groups.

Table 3.

Measurement Invariance Findings and Internal Consistency Coefficients.

Models	ω [95% CI]	Baseline model by age group							Measurement invariance
		18–25	26–35	36–45	46–55	56–65	66–75	76+	Configural	Full metric	Full scalar	Alignment considered	Finding
UCLA-LS-4
Russell et al. (1980)	.76 [.74, .77]	+	+	+	+	+	+	+	+	+	−	YES	Approx.
Roberts et al. (1993)		+	+	+	+	+	+	+	−/+	NA	NA	NO
UCLA-LS-7
Oshagan & Allen (1992)		−/+	+	−	+	−/+	+	+	−/+	NA	NA	NO
UCLA-LS-8
Roberts et al. (1993)	.88 [.87, .88]	−	+	+	+	+	+	+	+	+	−	YES	Approx.
Hays & DiMatteo (1987)	.84 [.83, .85]	−/+	+	+	+	+	+	+	+	−	NA	YES	Approx.
UCLA-LS-9 a
Hawkley et al. (2005)	F1:.81 [.80, .82] F2:.89 [.88, .90] F3:.79 [.77, .80]	+	+	+	+	+	+	+	+	+	+		Full
UCLA-LS-10
Russell (1996) – one-factor		−	−	−	−	−	−	−	NA	NA	NA
Russell (1996) – two-factor		−	−	−	−/+	−/+	−/+	+	NA	NA	NA
Knight et al. (1988) version1 – one-factor		+	+	−/+	−/+	−/+	−/+	−/+	NA	NA	NA
Knight et al. (1988) version1 – two-factor	F1: .74 [.72, .75] F2: .80 [.79, .81]	+	+	+	+	+	+	+	+	+	−	YES	Approx.
Knight et al. (1988) version2 – one-factor		−/+	−/+	−/+	−/+	−/+	−/+	−/+	NA	NA	NA
Knight et al. (1988) version2 – two-factor	F1: .82 [.81, .83] F2: .79 [.78, .80]	−/+	−/+	+	+	+	+	+	+	+	−	YES	Approx.
UCLA-LS-20
Russell et al. (1980) – one-factor		−	−	−	−	−	−	−/+	NA	NA	NA
Russell et al. (1980) – two-factor		−/+	−/+	−/+	−/+	−/+	−/+	−/+	NA	NA	NA
Hawkley et al. (2005) - 3factor		−/+	−/+	−/+	−/+	−/+	−/+	−/+	NA	NA	NA
McWhirter (1990) – three-factor		−/+	−/+	−/+	−/+	−/+	−/+	−/+	NA	NA	NA
Three-factor ESEM		+	+	+	+	+	+	+	+	+	−	NO	Partial

Note. ω = omega internal consistency; CI = confidence interval; “+” = acceptable fit (acceptable root mean square error of approximation, standardized root mean square, and comparative fit index); “−” = poor fit; UCLA-LS-7 = University of California Los Angeles, Loneliness Scale-7-item measures; “−/+” = inconsistent fit; NA = not applicable; UCLA-LS-8 = University of California Los Angeles, Loneliness Scale-8-item measures; UCLA-LS-9 = University of California Los Angeles, Loneliness Scale-9-item measures; UCLA-LS-10 = University of California Los Angeles, Loneliness Scale-10-item measures; UCLA-LS-20 = University of California Los Angeles, Loneliness Scale-20-item measures; ESEM = Exploratory Structural Equation Modeling.

^aThe UCLA-9 tested here is a three-factor model, with the three factors correlated.

Table 4.

Three-Factor Exploratory Structural Equation Modeling Pairwise Comparisons of Item Intercepts Between the Seven Age Groups.

Items	Chi-square	p-value	Age group intercepts
			18–25	26–35	36–45	46–55	56–65	66–75	76+
1. I feel in tune with the people around me (R)	132.548	0.0000*	1.998	1.853	1.781	1.704	1.658	1.629	1.566
2. I lack companionship	14.928	0.0208	2.632	2.449	2.469	2.498	2.505	2.530	2.591
3. There is no one I can turn to	3.167	0.7876	2.064	1.969	2.008	1.991	1.991	1.991	2.014
4. I do not feel alone (R)	56.487	0.0000*	2.102	1.928	1.948	2.030	2.114	2.226	2.358
5. I feel part of a group of friends (R)	20.599	0.0022*	2.083	2.002	2.006	1.976	1.934	1.854	1.869
6. I have a lot in common with the people around me (R)	31.376	0.0000*	2.115	2.033	2.002	1.962	1.924	1.878	1.906
7. I am no longer close to anyone	19.410	0.0035	2.050	1.854	1.891	1.863	1.855	1.907	2.039
8. My interests and ideas are not shared by those around me	14.925	0.0208	2.614	2.437	2.475	2.489	2.475	2.487	2.554
9. I am an outgoing person (R)	66.628	0.0000*	2.253	2.204	2.196	2.096	2.027	1.934	1.895
10. There are people I feel close to (R)	6.436	0.3762	1.514	1.451	1.472	1.438	1.429	1.427	1.477
11. I feel left out	88.607	0.0000*	2.726	2.639	2.596	2.510	2.441	2.348	2.330
12. My social relationships are superficial	17.273	0.0083	2.246	2.241	2.344	2.313	2.355	2.372	2.429
13. No one really knows me well	16.959	0.0094	2.542	2.303	2.382	2.397	2.402	2.421	2.456
14. I feel isolated from others	78.867	0.0000*	2.632	2.400	2.375	2.336	2.272	2.203	2.194
15. I can find companionship when I want it (R)	56.812	0.0000*	1.965	1.812	1.762	1.684	1.668	1.653	1.682
16. There are people who really understand me (R)	12.038	0.0611	1.883	1.744	1.763	1.747	1.751	1.786	1.831
17. I am unhappy being so withdrawn	272.853	0.0000*	2.638	2.310	2.160	2.052	1.948	1.861	1.839
18. People are around me but not with me	39.116	0.0000*	2.707	2.493	2.550	2.503	2.451	2.430	2.414
19. There are people I can talk to (R)	7.378	0.2873	1.585	1.490	1.535	1.495	1.493	1.506	1.506
20. There are people I can turn to (R)	4.569	0.6001	1.569	1.482	1.509	1.499	1.498	1.508	1.518

Note. Higher scores on each item indicate higher loneliness. Items in bold are those that can achieve partial measurement invariance when not constrained. (R) = reversed items.

^*p < .0025.

Beyond the 20-item measure, structures that achieved acceptable baseline fit in most groups were those from both 4-item measures (UCLA-LS-4), 7-item (UCLA-LS-7), 8-item (UCLA-LS-8), 9-item (UCLA-LS-9), and Knight and colleagues’ (1988) 10-item (UCLA-LS-10) two-factor measures (see Table 3). Still, not all achieved a good configural invariance fit. Therefore, only UCLA-LS-4 by Russell et al. (1980), UCLA-LS-8 (Hays & DiMatteo, 1987; Roberts et al., 1993), UCLA-LS-9 (Hawkley et al., 2005), and Knight and colleagues’ (1988) two-factor UCLA-LS-10 measures received subsequent metric and scalar measurement invariance testing. Of these, the nine-item version was shown to achieve full scalar invariance (thus, further alignment testing was not necessary). However, given that this was modeled as a three-factor correlated model, we conducted post hoc analyses to explore the measurement invariance of each of the three factors separately. This would help us understand whether these can be used across different age groups as standalone measures. Findings indicated that only Factor 2 (relational connectedness) achieved full scalar measurement invariance (thus, further alignment testing was not necessary).

The remaining measures achieved a full metric, but not scalar, measurement invariance, meaning that alignment measurement invariance was carried out for those measures. The average alignment results across the 100 datasets are shown in Tables 5 and 6 for loadings and intercepts, respectively. Generally, all measures (including Factors 1 and 3 of UCLA-LS-9) were shown to have acceptable levels of non-invariance, well below the suggested threshold of 25% (Muthén & Asparouhov, 2014), suggesting approximate measurement invariance. The UCLA-LS-8 (Roberts et al., 1993) showed the highest average non-invariance across loadings and intercepts (12%), followed by a similar non-invariance for the two UCLA-LS-10 measures by Knight et al. (1988; Version 1: 8.9%; Version 2: 7.9%), Hays and DiMatteo’s (1987) UCLA-LS- 8 (8.8%), and Factor 1 (intimate connectedness) of UCLA-LS- 9 (8.5%; Hawkley et al., 2005). Russell and colleagues’ (1980) UCLA-LS-4 and Factor 3 of UCLA-LS-9 (collective connectedness) were the ones with the smallest average non-invariance, with 7% and 5.6%, respectively. Monte Carlo cross-validation simulations indicated high correlations between the real data and factor-generated means for the UCLA-LS-4 (r = .97) and UCLA-LS-8 (Hays & DiMatteo, 1987: r = .98; Roberts et al., 1993: r = .97) measures, as well as Factors 1 and 3 of the UCLA-LS-9 (F1: r = .97; F3: r = .95), which we consider as further support of approximate invariance within the seven age groups (Muthén & Asparouhov, 2014). The two UCLA-LS-10 versions (Knight et al., 1988) were shown to have worse reliability (Version 1: r = .91, .92; Version 2: r = .87, .93). It is also worth noting the poor discriminant validity was observed for the two factors of these versions (r > .83; see Tables S18 and S20 in Supplementary Material).

Table 5.

Scalar Invariance Alignment Findings for Loadings Averaged Across the 100 Datasets.

Structure and items	Percentage of approximate non-invariance across the 100 datasets							Average % of non-invariance	Weighted average loading across invariant groups		R 2
	18–25	26–35	36–45	46–55	56–65	66–75	76+		M	SD	M	SD
UCLA-LS-4 Russell et al. (1980)								0.07%
1	0	0	0	0	0	1	0		0.41	0.01	0.40	0.22
13	0	0	0	0	0	0	0		0.70	0.01	0.57	0.22
15	0	0	0	0	1	0	0		0.50	0.01	0.03	0.07
18	0	0	0	0	0	0	0		0.67	0.01	0.31	0.28
UCLA-LS-8 Hays & DiMatteo (1987)								0.05%
2	0	0	0	0	0	0	0		0.62	0.01	0.53	0.10
3	0	0	0	0	0	0	0		0.65	0.01	0.58	0.19
9	0	0	0	0	0	2	0		0.33	0.01	0.27	0.17
11	0	0	0	0	0	0	0		0.57	0.01	0.43	0.27
14	0	0	0	0	0	1	0		0.75	0.01	0.32	0.27
15	0	0	0	0	0	0	0		0.48	0.01	0.45	0.23
17	0	0	0	0	0	0	0		0.63	0.01	0.46	0.19
18	0	0	0	0	0	0	0		0.60	0.01	0.02	0.07
15	0	0	0	0	0	0	0		0.53	0.01	0.37	0.25
UCLA-LS-8 Roberts et al. (1993)								5.8%
1	0	0	0	0	0	1	0		0.36	0.01	0.07	0.09
2	1	0	0	0	0	4	0		0.64	0.01	0.26	0.14
4	86	54	19	4	20	35	97		0.30	0.08	0.00	0.01
5	0	0	0	0	0	0	0		0.56	0.01	0.33	0.23
7	0	0	0	1	0	0	1		0.70	0.01	0.35	0.20
11	0	0	0	0	0	0	0		0.57	0.01	0.42	0.26
14	0	0	0	0	0	1	0		0.76	0.01	0.38	0.23
15	0	0	0	0	0	0	0		0.53	0.01	0.37	0.25
UCLA-LS-9 Hawkley et al. (2005)
F1: 2	0	0	0	0	0	0	0	0%	0.57	0.01	0.62	0.09
11	0	0	0	0	0	0	0		0.57	0.00	0.80	0.15
14	0	0	0	0	0	0	0		0.78	0.01	0.02	0.07
F3: 1	0	0	0	0	0	0	0	0%	0.42	0.01	0.39	0.19
5	0	0	0	0	0	0	0		0.66	0.01	0.26	0.26
6	0	0	0	0	0	0	0		0.69	0.01	0.33	0.30
UCLA-LS-10 Knight et al. (1988) Version 1 – two-factor								4% a
F1: 1	0	0	0	0	0	0	0		0.43	0.01	0.55	0.21
4	98	34	14	6	6	27	93		0.29	0.07	0.08	0.03
6	0	0	0	0	0	0	0		0.56	0.01	0.26	0.23
16	0	0	0	0	0	0	0		0.62	0.01	0.40	0.33
19	0	0	0	0	0	0	0		0.55	0.01	0.50	0.27
F2: 2	0	0	0	0	0	0	0		0.58	0.01	0.52	0.11
7	0	0	0	0	0	0	0		0.71	0.01	0.51	0.24
8	0	4	0	0	0	0	0		0.46	0.01	0.06	0.13
11	0	0	0	0	0	0	1		0.51	0.01	0.44	0.22
18	0	0	0	0	0	0	0		0.61	0.01	0.09	0.17
UCLA-LS-10 Knight et al. (1988) Version 2 – two-factor								0.16%
F1: 3	1	0	0	0	0	0	0		0.73	0.27	0.52	0.28
12	1	0	0	0	0	0	0		0.52	0.19	0.43	0.27
13	1	0	0	0	0	0	0		0.70	0.25	0.05	0.11
14	1	0	0	0	0	0	0		0.76	0.28	0.51	0.24
17	1	0	0	0	0	0	0		0.60	0.23	0.31	0.14
F2: 5	1	0	0	0	0	0	0		0.63	0.22	0.36	0.26
9	1	0	0	0	0	0	0		0.41	0.14	0.34	0.22
10	1	0	0	0	0	0	0		0.49	0.17	0.42	0.24
15	1	0	0	0	0	0	1		0.58	0.20	0.37	0.25
20	1	0	0	0	0	0	0		0.59	0.20	0.18	0.20

Note. Versions that achieved full measurement invariance (UCLA-LS9 and its factor 2) are not presented here, as they did not receive alignment testing. UCLA-LS-4 = University of California Los Angeles, Loneliness Scale-4-item measures; UCLA-LS-8 = University of California Los Angeles, Loneliness Scale-8-item measures; UCLA-LS-9 = University of California Los Angeles, Loneliness Scale-9-item measures; UCLA-LS-10 = University of California Los Angeles, Loneliness Scale-10-item measures.

^aFormula used to calculate the average % of non-invariance: 283 non-invariant parameters across the 100 datasets/7,000 calculated parameters*100.

Table 6.

Scalar Invariance Alignment Findings for Intercepts Averaged Across the 100 Datasets.

Structure and items	Percentage of approximate non-invariance across the 100 datasets							Average % of non-invariance	Weighted average intercept across invariant groups		R 2
	18–25	26–35	36–45	46–55	56–65	66–75	76+		M	SD	M	SD
UCLA-LS-4 Russell et al. (1980)								13.9%
1	81	67	11	1	3	8	22		1.85	0.04	0.66	0.05
13	62	59	14	1	7	26	27		2.68	0.06	0.01	0.03
15	0	0	0	1	0	0	0		1.92	0.01	0.87	0.05
18	0	0	0	0	0	0	0		2.74	0.01	0.93	0.03
UCLA-LS-8 Hays & DiMatteo (1987)								17.6%
2	16	16	16	17	74	84	89		2.69	0.07	0.00	0.00
3	92	76	47	7	7	12	24		2.29	0.06	0.00	0.00
9	1	11	7	0	1	40	59		2.23	0.03	0.63	0.06
11	0	30	7	0	0	5	3		2.71	0.01	0.87	0.04
14	0	0	0	0	0	0	0		2.62	0.01	0.98	0.01
15	0	0	0	1	0	0	0		1.92	0.01	0.90	0.04
17	100	99	32	3	1	3	4		2.25	0.03	0.67	0.03
18	0	2	0	0	0	1	0		2.72	0.01	0.85	0.06
UCLA-LS-8 Roberts et al. (1993)								18.2%
1	100	77	19	1	3	1	38		1.83	0.03	0.66	0.03
2	17	18	16	5	47	83	87		2.71	0.07	0.00	0.00
4	27	28	27	4	16	62	69		2.16	0.07	0.00	0.00
5	0	0	0	0	0	0	0		2.16	0.01	0.76	0.09
7	2	4	0	0	2	57	100		2.11	0.02	0.00	0.00
11	0	59	36	2	1	5	6		2.70	0.02	0.80	0.04
14	0	0	0	0	0	0	0		2.62	0.01	0.98	0.01
15	0	0	0	0	0	0	0		1.93	0.01	0.90	0.04
UCLA-LS-9 Hawkley et al. (2005)
F1: 2	4	8	4	2	63	93	97	17%	2.66	0.04	0.00	0.00
11	0	48	31	0	1	2	2		2.70	0.02	0.86	0.04
14	0	0	0	0	0	2	0		2.61	0.01	0.99	0.01
F3: 1	100	69	19	0	2	2	42	11.1%	1.79	0.03	0.57	0.03
5	0	0	0	0	0	0	0		2.10	0.01	0.95	0.03
6	0	0	0	0	0	0	0		2.12	0.00	0.96	0.02
UCLA-LS-10 Knight et al. (1988) Version 1 – two-factor								13.7% a
F1: 1	80	75	32	11	21	24	64		1.83	0.07	0.44	0.06
4	3	6	7	2	15	73	96		2.11	0.04	0.00	0.01
6	0	10	0	0	0	6	0		2.11	0.01	0.68	0.09
16	0	0	0	0	0	2	22		1.93	0.01	0.29	0.22
19	0	0	0	0	0	1	0		1.65	0.01	0.33	0.24
F2: 2	0	0	0	0	0	1	0		2.66	0.01	0.74	0.14
7	0	0	0	0	0	0	1		2.08	0.01	0.79	0.10
8	0	0	0	0	0	0	0		2.61	0.01	0.88	0.05
11	14	27	14	22	55	85	90		2.69	0.07	0.14	0.10
18	0	0	0	0	0	9	92		2.67	0.01	0.49	0.14
UCLA-LS-10 Knight et al. (1988) Version 2 – two-factor								15.7%
F1: 3	0	6	0	0	0	0	0		2.13	0.01	0.35	0.26
12	99	7	0	1	0	2	5		2.45	0.02	0.00	0.00
13	0	0	0	0	0	1	0		2.53	0.01	0.76	0.10
14	51	39	9	8	37	59	68		2.49	0.07	0.43	0.09
17	100	100	86	65	4	15	25		2.04	0.07	0.19	0.04
F2: 5	0	0	0	0	0	0	0		2.08	0.01	0.86	0.08
9	5	5	2	1	20	70	95		2.21	0.03	0.47	0.07
10	0	0	0	0	0	0	1		1.55	0.01	0.18	0.20
15	59	1	0	0	0	0	0		1.85	0.02	0.75	0.05
20	0	6	1	0	2	37	8		1.62	0.01	0.01	0.04

Note. Versions that achieved full measurement invariance (UCLA-LS-9 and its factor 2) are not presented here, as they did not receive alignment testing. UCLA-LS-4 = University of California Los Angeles, Loneliness Scale-4-item measures; UCLA-LS-8 = University of California Los Angeles, Loneliness Scale-8-item measures; UCLA-LS-9 = University of California Los Angeles, Loneliness Scale-9-item measures; UCLA-LS-10 = University of California Los Angeles, Loneliness Scale-10-item measures.

^aFormula used to calculate the average % of non-invariance: 960 non-invariant parameters across the 100 datasets/7,000 calculated parameters*100.

Item-Level Testing

Table 5 summarizes the alignment findings for factor loadings. These are generally consistent with the measurement invariance findings and point to high measurement invariance (with the exception of item 4 of Knight and colleague’s Version 1 measure, and Roberts and colleagues’ UCLA-LS-8 measure). Indeed, the majority of non-invariance in the current study was due to the intercepts, which is where we turn our focus next. Closer inspection of the intercept findings detailed in Table 6 shows that the majority of items in all measures showed small intercept non-invariance. The first section of Table 6 indicates the percentage of approximate non-invariance across the 100 datasets. For instance, the intercept of Item 1 of Knight et al. (1988) UCLA-LS-10 was shown to be non-invariant in 18- to 25-year-olds in 80% of the 100 random samples.

With the exception of the UCLA-LS-4 (Russell et al., 1980), and Factors 1 and 3 of the UCLA-LS-9 (Hawkley et al., 2005), all measures were shown to have multiple items with substantial intercept non-invariance that were mostly observed in the two lower (ages 18–35) and two higher (ages 66–94) age groups. Consistent with the ESEM findings, Item 17 (“I am unhappy being so withdrawn”; also part of UCLA-LS-8 by Hays and DiMatteo (1987) and Version 2 of Knight et al. (1988) UCLA-LS-10) was shown to be non-invariant in younger age groups in all 100 simulated datasets, with younger groups scoring significantly higher on this item, compared with the average invariant scores. Other items were also shown to have substantial intercept non-invariance in multiple groups. For instance, as seen in Table 6, in the 10-item UCLA-LS-10 Version 1 of Knight et al. (1988), our findings showed that older individuals score significantly higher on the reverse-worded Item 4 (“I do not feel alone”; also part of UCLA-LS-8 by Roberts et al. (1993)) than younger groups, with higher scores indicating higher loneliness. Notably, results were inconsistent for some of the items across different measures. For example, Item 18 (“People are around me but not with me”) was shown to have a high degree of intercept invariance (R² = .85–.93) in the Russell et al. (1980) UCLA-LS-4 and both UCLA-LS-8 measures (Hays & DiMatteo, 1987; Roberts et al., 1993), but that was not the case for Knight et al. (1988) UCLA-LS-10 Version 1 (R² = .49), which indicated that older individuals score significantly lower than younger groups (see Table 6). Such discrepancies might be due to the alignment occurring for one factor at a time in multidimensional structures (Asparouhov & Muthén, 2014), as was the case for Version 1 and 2 of Knight et al. (1988) UCLA-LS-10.

The internal consistency coefficient of the measures that achieved full or approximate measurement invariance was also calculated. Results are summarized in Table 3 and indicate high internal consistency for all measures.

LSEM Measurement Invariance

All LSEM parameters for each model can be found at https://osf.io/cqxrd/. The findings from measurement invariance testing are summarized in Table 7 and the detailed information can be found in Tables S21–S23 of the Supplementary Material. Findings were consistent with the multigroup invariance testing. Specifically, the same models failed to achieve acceptable configural invariance and therefore did not receive further measurement invariance testing. Of the nine models that had good configural model fit, only the correlated UCLA-LS-9 (Hawkley et al., 2005) and Factor 2 of the same measure achieved full scalar measurement invariance. The remaining seven achieved full metric but not scalar invariance. Permutation testing of those models indicated that most ages had non-invariant intercepts on all 20 items. Invariant ages are summarized in Table 8, although these should be cautiously interpreted, as despite the adjustment of the alpha level, the repetition of testing on the same items likely increased the Type I error further. Thus, it is likely that more ages have invariant intercepts.

Table 7.

Measurement Invariance of LSEM Models.

Models	Measurement invariance (ΔCFI)
	Configural	Metric	Scalar
UCLA-LS-4
Russell et al. (1980)	+	(.000)	− (.023)
Roberts et al. (1993)	−/+	NA	NA
UCLA-LS-7
Oshagan & Allen (1992)	−/+	NA	NA
UCLA-LS-8
Hays & DiMatteo (1987)	+	+ (.001)	− (.018)
Roberts et al. (1993)	+	+ (.004)	− (.020)
UCLA-LS-9
Hawkley et al. (2005) three-factor	+	+(.001)	− (.008)
Factor 1	Saturated model	− (.001)	− (.022)
Factor 2	Saturated model	− (.001)	(.000)
Factor 3	Saturated model	− (.001)	− (.014)
UCLA-LS-10
Russell (1996) – one-factor	−	NA	NA
Russell (1996) – two-factor	−/+	NA	NA
Knight et al. (1988) version1 – one-factor	−	NA	NA
Two-factor	+	− (.001)	− (.017)
Knight et al. (1988) version2 – one-factor	−	NA	NA
Two-factor	+	+(.001)	− (.015)
UCLA-LS-20
Russell et al. (1980) – one-factor	−	NA	NA
Russell et al. (1980) – two-factor	−	NA	NA
Hawkley et al. (2005) – three-factor	−/+	NA	NA
McWhirter (1990) – three-factor	−/+	NA	NA

Note. LESM = local structural equation modeling; ΔCFI = comparative fit index difference calculated as metric—configural; scalar—metric (negative values indicate worse fit); UCLA-LS-4 = University of California Los Angeles, Loneliness Scale-4-item measures; “+” = acceptable fit; “−” = poor fit; “−/+” = inconsistent fit; NA = not applicable; UCLA-LS-7 = University of California Los Angeles, Loneliness Scale-7-item measures; UCLA-LS-8 = University of California Los Angeles, Loneliness Scale-8-item measures; UCLA-LS-9 = University of California Los Angeles, Loneliness Scale-9-item measures; UCLA-LS-10 = University of California Los Angeles, Loneliness Scale-10-item measures; UCLA-LS-20 = University of California Los Angeles, Loneliness Scale-20-item measures.

Table 8.

Ages With Invariant Intercepts for the UCLA-LS Measures That Did Not Achieve Full Scalar Invariance.

UCLA-LS items	UCLA-LS-4 (Russell et al., 1980)	UCLA-LS-8 Hays & DiMatteo (1987)	UCLA-LS-8 Roberts et al. (1993)	Knight et al. (1988) Version 1 – two-factor	Knight et al. (1988) Version 2 – two-factor
i1	54		54	54
i2		25,60	25,60	25, 60
i3		22–23, 36–38, 49–52, 63–75			22–23, 36–38, 49–51, 61, 69, 73–76
i4			18	18–19
i5			60		60
i6				58
i7			25–26, 38–42, 64	25–26, 38–42, 64
i8				26, 42–52, 64–66
i9		58			58
i10					48–50, 75–76
i11		57	57	57
i12					57–58
i13	23, 60				60
i14		56	56		56
i15	49–50, 82–83	49–50, 82–83	49–50, 82–83		49–50, 82–83
i16				26, 39, 64
i17		—			—
i18	55	55		55
i19				25–26, 33–35, 48, 62–64, 68–69, 78–81
i20					23, 37–47, 59–61, 70–71, 76–79

Note. UCLA-LS = University of California Los Angeles, Loneliness Scale; UCLA-LS-4 = University of California Los Angeles, Loneliness Scale-4-item measures; UCLA-LS-8 = University of California Los Angeles, Loneliness Scale-8-item measures.

Fit Indices

The fit indices for all models are graphically summarized in Figure 1. Findings are generally consistent with the multigroup analyses and indicate that the fit of the UCLA-LS-20 is poor across all ages, irrespective of the structure, although the fit seems to be better for older ages. This indicates that the 20-item model is specifically bad for younger individuals. As seen in Figure 1, the worst performing structure among those tested, was the unidimensional UCLA-LS-10 (Russell, 1996). While there is clear variation between ages, the fit is consistently poor. Notably, the structures with the best fit in most ages were the two UCLA-LS-4 (Russell et al., 1980) and the correlated UCLA-LS-9 (Hawkley et al., 2005) measures. Still, some notable findings in the LSEM were not evident in the multigroup analyses. For example, the same figure shows a large dip in model fit for the UCLA-LS-4 by Roberts et al. (1993) at age 50 years which is worse than the neighboring ages 45 and 55 years. This piece of information is masked in the multigroup analysis due to the grouping of these ages.

Figure 1.

Fit indices of the UCLA-LS LSEMs Across Age Values

Note. UCLA-LS = University of California Los Angeles, Loneliness Scale; LSEM = local structural equation modeling; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square of error approximation; SRMR = standardized root mean square residual.

Intercepts and Factor Loadings

The intercepts for the 20 items are graphically summarized in Figure 2. As with the model fit, results are generally consistent with the multigroup analyses. For example, Items 3 (“no one I can turn to”), 10 (“people I feel close to”), 16 (“people who really understand you”) 19 (“people I can talk to”), and 20 (“people I can turn to”) have stable intercepts across all values of the moderator. On the contrary, Items 11 and 17 (part of UCLA-LS-8 and UCLA-LS-10 by Knight et al., 1988) show a steep negative slope, where younger ages score more highly on this item than older ages (they feel more left out and they are more unhappy being withdrawn). The opposite is observed for Item 4 (in UCLA-LS-8 by Roberts et al., 1993 and UCLA-LS-10 Version 1 Knight et al., 1988), where older individuals indicate feeling more alone than younger people; interestingly individuals near the age of 35 years feel the least alone. The factor loadings are graphically presented in Figures S1 to S13 of the Supplementary Material. When compared with the alignment findings they present a consistent picture. For example, Item 4 (“I do not feel alone”) shows a steep negative slope, indicating a very large difference in the responses between the younger (λ near .57) and older ages (λ < 0). On the contrary, although Item 2 (“I lack companionship”) did not present invariance issues in the multigroup models, Figure S8 indicates inconsistent responses between the younger (λ < .55) and the older ages (λ > .65). Together, these findings indicate the variation in the responses on loneliness questions across different ages.

Figure 2.

Intercepts for the 20 University of California Los Angeles, Loneliness Scale Items Across Age Values

Means for the UCLA-LS-4 and UCLA-LS-9 (Russell et al., 1980; Hawkley et al., 2005)

To provide some practical advice and information on average loneliness scores for the most suitable age-invariant UCLA-LS scales, we explored the means and SD for the 4-item (Russell et al., 1980) and 9-item (Hawkley et al., 2005) versions of the UCLA-LSR across the different age groups (see Table 9). Higher means on the UCLA-LS-4 and UCLA-LS-9 would indicate higher levels of loneliness (but also low connectedness for the UCLA-LS-9). In practical settings, it is often necessary to use a cut-off score to categorize individuals into those who can be deemed “highly lonely” and those who are not. Here, scores that are 2 SDs above the mean were used to define a very high level of loneliness, a standard practice within psychology research. For the UCLA-LS-4, our data show that a score of around 14 would denote very high loneliness for people aged 45 years and younger; a score of around 13 would denote very high loneliness for those older than 45 years of age. In practice, this means that individuals in the “younger” age range would need to score highly (Likert-type-type response 4: “Always”) on at least 2 of the items to be classified as reporting very high loneliness; those in the older group would have to score at least a 4 (“Always”) and a 3 (“Sometimes”) on the Likert-scale response. The means and SD for the three factors of the UCLA-LS-9 can be used and interpreted in the same way.

Table 9.

Means for the 4-Item and 9-Item UCLA Loneliness Scales.

Models	Group	M	SD	95% CI	Very high loneliness threshold
UCLA-LS-4 Russell et al. (1980) a	18–25	9.22	2.61	[9.00, 9.42]	14.4
	26–35	8.44	2.59	[8.30, 8.58]	13.66
	36–45	8.45	2.61	[8.34, 8.57]	13.59
	46–55	8.31	2.57	[8.22, 8.39]	13.43
	56–65	8.19	2.56	[8.12, 8.25]	13.15
	66–75	8.14	2.48	[8.07, 8.21]	13.06
	76+	8.11	2.46	[7.96, 8.26]	8.11
UCLA-LS-9 Hawkley et al. (2005) b
F1: Intimate Connectedness	18–25	7.99	2.17	[7.81, 8.16]	12.33
	26–35	7.49	2.18	[7.36, 7.60]	11.85
	36–45	7.43	2.37	[7.32, 7.54]	12.17
	46–55	7.35	2.29	[7.28, 7.42]	11.93
	56–65	7.24	2.32	[7.18, 7.30]	11.88
	66–75	7.08	2.3	[7.02, 7.15]	11.68
	76+	7.11	2.33	[6.97, 7.26]	11.77
F2: Relational Connectedness	18–25	4.67	1.91	[4.52, 4.81]	8.49
	26–35	4.41	1.84	[4.31, 4.51]	8.09
	36–45	4.51	1.89	[4.42, 4.59]	8.29
	46–55	4.44	1.86	[4.38, 4.50]	8.16
	56–65	4.44	1.87	[4.39, 4.48]	8.18
	66–75	4.44	1.83	[4.39, 4.49]	8.1
	76+	4.49	1.76	[4.38, 4.60]	8.01
F3: Collective Connectedness	18–25	6.19	2.05	[6.03, 6.36]	10.29
	26–35	5.88	2.02	[5.77, 5.99]	9.92
	36–45	5.78	2.04	[5.69, 5.88]	9.86
	46–55	5.66	2.01	[5.59, 5.72]	9.68
	56–65	5.51	1.98	[5.46, 5.56]	9.47
	66–75	5.36	1.9	[5.30, 5.41]	9.16
	76+	5.34	1.95	[5.22, 5.45]	9.24

Note. CI = confidence interval; UCLA-LS-4 = University of California Los Angeles, Loneliness Scale-4-item measures; UCLA-LS-9 = University of California Los Angeles, Loneliness Scale-9-item measures.

^aHigher scores here indicate higher levels of loneliness. ^b Higher scores on the UCLA-LS-9 indicate higher levels of loneliness/low connectedness.

Discussion

In this study, we examined the factor structure of the UCLA-LSR and its shorter versions. We also assessed measurement equivalence across age groups to determine whether scores between those groups could be compared and interpreted with confidence. This was explored through multigroup measurement invariance (comparison of seven age groups) and LSEM (comparison of 66 ages). Findings were consistent across both methods. We did not find support for any of the current factor structures for the UCLA-LSR-20. This is consistent with existing work and might reflect some of the issues inherent to measures with reverse-worded items (e.g., creation of method factors, an increase in the risk of inattention and confusion; Lindwall et al., 2012; van Sonderen et al., 2013). A good model fit was found for the three-factor ESEM model, although this is expected in such unrestricted models. This structure achieved partial measurement invariance (20% non-invariance), with four items (17, 1, 11, 14) indicating substantial differences between age groups.

Of the 15 shorter versions of the UCLA-LS explored, only nine showed a good baseline fit and configural invariance. This begs the question of how the construct of loneliness is conceptualized and whether a new scale altogether would be advantageous. Both the multigroup and the LSEM models showed that of those versions, only the three-factor correlated UCLA-LS-9 (Hawkley et al., 2005), and Factor 2 of that measure (relational connectedness), were shown to achieve full scalar invariance, endorsing them as robust measures assessing the same trait irrespective of age. The remaining seven measures (Russell et al. (1980) UCLA-LS-4, both UCLA-LS-8, both Knight and colleagues’ (1988) two-factor UCLA-LS-10, and Factors 1 and 3 of the UCLA-LS-9) failed to achieve full metric and/or scalar invariance. This indicates that the contribution of some or all items to the underlying latent construct (metric invariance) and the item means (scalar invariance) varies by age group. Although permutation testing of the LSEM models showed that most ages had non-invariant intercepts, we interpret these findings with caution due to a large number of comparisons within each model (66 age) and the repeated use of the same data, which likely increased Type I error and failed to identify invariant ages. Alignment testing is not available in LSEM models but when applied within the multigroup models showed that all measures have in fact acceptable levels of non-invariance (below 25%), synonymous with achieving “approximate” measurement invariance. This fits with existing evidence that finds full scalar invariance unrealistic and untenable, especially in studies with many groups (Marsh et al., 2018). Further replication analyses indicated that the UCLA-LS-4 by Russell et al. (1980), Factors 1 and 3 of the UCLA-LS-9, and both UCLA-LS-8 achieved higher alignment reliability than the UCLA-LS-10 (Knight et al., 1988), inspiring more confidence in their appropriateness for age group comparisons. The measure with the least non-invariance and highest reliability was the UCLA-LS-4 (Russell et al., 1980) and Factor 1 of the UCLA-LS-9 (Hawkley et al., 2005).

The findings of age differences for specific items of the different UCLA loneliness measures are also intriguing. For example, the item “I am unhappy being so withdrawn” (i17), which appears in the UCLA-LSR loneliness scale and several adaptations, was shown to be non-invariant, suggesting that this statement is interpreted differently based on age, or carries particular meaning and salience for some age groups compared with others. In fact, this item was part of the original UCLA-LSR measure but was removed during the development of the more recent 20-item UCLA-LS3. Similarly, older adults scored significantly higher than younger age groups on the reverse-worded item “I do not feel alone” (i4), indicating a different interpretation or experience for this statement, although the potential response bias arising from such a negatively worded statement must be noted. This fits with the poor factor loading observed for this item, especially for older age groups (see Supplementary Material). This finding could also represent changing attitudes to being alone as one ages, with older adults challenging conventional definitions of social well-being and highlighting personal agency in decision-making about being alone (Finlay & Kobayashi, 2018). LSEM provided additional information through a more nuanced analysis across ages. For example, in our upper age group (76+) all participants aged 76 to 94 were assumed to have the same response to the measures. However, as shown through the LSEM this is not always the case. For example, the model fit of the UCLA-LS-4 (Russell et al., 1980) for those aged 75 was noticeably worse than for those aged 80. This is also the case for some of the item intercepts. For example, Item 12 (“my social relationships are superficial”) was not stable across ages more than 75. The application of both alignment and LSEM to testing measurement invariance in the current study thus provides significant and complementary information about the item-level age non-invariance of the UCLA-LS that can be useful in the understanding of loneliness across ages as well as in the refinement of items in future applications (Marsh et al., 2018).

Generally, our findings suggest that the nine UCLA-LS measures that achieved full or approximate measurement invariance can be confidently used by adults of different ages, as their structure, item contribution, and item mean scores do not substantially vary by age. In other words, different age groups appear to understand and respond to loneliness items approximately the same way. We should, thus, reasonably expect that similar scores on those measures would be achieved by individuals with similar loneliness levels, irrespective of their age, and that any differences between age groups will be due to true differences in the experience of loneliness and not due to differences in the measure used. Such work has important implications for researchers, practitioners, and policy makers working with people who report loneliness. Indeed, the current findings present a significant step forward in the studying of loneliness. In the literature, there is consensus that age influences loneliness severity (Qualter et al., 2015). In fact, some evidence suggests that age accounts for the majority of its variance (Shovestul et al., 2020). Still, researchers also note that the nature of loneliness experienced by young adults differs qualitatively from that of older adults (Matthews et al., 2022), supporting the claim that more qualitative work is needed to understand the experience of loneliness for people of different ages (Verity et al., 2022). That need is highlighted by the findings reported herein and the fact that different age groups interpreted and responded to many of the UCLA-LS items, with the exception of the correlated UCLA-LS-9 (Hawkley et al., 2005), with significant variability.

Together, our findings suggest that the UCLA-LS-9 (correlated or as standalone measures), Russell’s UCLA-LS-4 (Russell et al., 1980), and both UCLA-LS-8 (Hays & DiMatteo, 1987; Roberts et al., 1993) are all brief measures of loneliness (or related facets) that work reliably well for adults of different ages. However, the choice between these measures will depend on the study aims, the study design, and the available data, especially within secondary data analysis. The latter is an important consideration, as both UCLA-LS-8 were derived from the UCLA-LSR-20 (as described in Table 1 and Appendix A) and as such cannot be used in studies using the updated 20-item UCLA-LS3. Of the measures that received good results, the UCLA-LS-4 (Russell et al., 1980) and UCLA-LS-9 (Hawkley et al., 2005) provide greater flexibility because they can be drawn from both the UCLA-LSR and the UCLA-LS3. These measures are where we focus our discussion next.

Four-Item and 9-Item UCLA-LS Measures

We argue that the UCLA-LS-4 is a robust measure for reliably measuring loneliness in all adult age groups, but also capturing how core components of loneliness might differ between age groups. This is particularly important when there are resource-, financial-, or time-related constraints to data collection. Still, one must consider the issues inherent to short unidimensional and age-invariant measures: They are not able to capture specific thematic experiences for those at different life stages during adulthood. In such situations, the researcher will need to decide whether a longer measure of loneliness specific to the age group of interest (e.g., different measures have been developed for children and adolescents; Cole et al., 2021) offers a better coverage of the construct for that population and can, thus, assess differential experiences of loneliness. For example, if a researcher is interested in the behavioral correlates of loneliness in emerging adulthood, they would need to ensure that their measure includes the important components of loneliness as they relate to that developmental stage and group.

As evidenced by the number of versions, the UCLA-LS measure is susceptible to inconsistent factor structure. This means that depending on the version used, each UCLA-LS might be tapping on overall loneliness or distinct domains. For example, researchers interested in a multidimensional assessment of loneliness can also consider using the UCLA-LS-9. This version is the shorter form of the three-factor UCLA-LSR-20 suggested by Hawkley and colleagues (2005). It is considered to be assessing three distinct dimensions of loneliness related to intimate, relational, and collective connectedness, which correspond to the structure of human personal social networks identified by Weiss (1973) and Dunbar (2014). Based on these theories, intimate connectedness (Factor 1) measures feelings of aloneness, rejection, and the perceived absence of a significant someone. Relational connectedness (Factor 2) refers to the perceived presence/absence of quality friendships or family connections and includes feelings of familiarity, closeness, and support. Finally, collective connectedness (Factor 3) measures feelings of group identification and cohesion and generally refers to individuals’ valued social identities or active networks (e.g., group, school, team, or national identity) wherein they can connect to similar others (see Cacioppo et al., 2015; Hawkley et al., 2005 for more information). It is important to note, however, the lack of psychometric evidence for this measure in the literature. Similarly, there are methodological challenges with using any of its 3-item domains, as these are saturated models. This means that their model fit cannot be estimated and, therefore, judged.

Currently, in the United Kingdom, a three-item adaptation of the UCLA-LS, developed by Hughes et al. (2004), is recommended for population surveys by the ONS (2018a). Given that this is the same measure as Factor 1 of the UCLA-LS-9 (Hawkley et al., 2005), which achieved approximate invariance in the current study, there might be a temptation to agree with the ONS recommendation, as an appropriate measure of loneliness. Such recommendation, however, is not possible based on the current findings because the UCLA-LS-3 uses a revised three-category response option (never/hardly ever, some of the time, and often), rather than the four response categories evaluated here. Therefore, the age measurement invariance of the three-item measure suggested by the ONS is unclear, and future work is needed to explore that. This is particularly urgent given that Item 2 of that measure (“I lack companionship”) was found to be non-invariant in our study.

Full scalar invariance is particularly relevant for clinical and community practice and when the means of loneliness need to be compared. Thus, the use of the correlated UCLA-LS-9, as well as Factor 2 (relational connectedness) from that measure, can offer more benefits as they are the only ones for which this was achieved. Importantly, this finding was consistent irrespective of whether 7 or 66 ages were compared. The degree to which partial or approximate invariance is sufficient for mean comparisons is highly debated (Pokropek et al., 2019), and the majority of this work has focused on the comparison of latent means. While lack of full scalar invariance might be of little practical significance when comparing latent means, some work suggests that this can lead to erroneous conclusions if comparisons are based on composite (total scores) mean differences (Steinmetz, 2013) as is typically the case in the study of loneliness. Steinmetz (2013) found that this was especially true when the non-invariance was in the item intercepts as is the case in the current study. However, this simulation was based on a small number of groups (2–3) and sample size per group (100–300). Thus, the degree to which these findings can generalize to studies with larger groups and samples, as the ones here (7 groups x 625 participants), is unclear. Under the alignment method and approximate invariance, the latent means of measures with small non-invariance (less than 25%) and large samples were shown to be accurately estimated in a simulation study (Asparouhov & Muthén, 2014). Alignment has also been the recommended method for recovering latent means in cases with only a few non-invariant parameters across groups (Pokropek et al., 2019). Although the accuracy of alignment within composite scores is unclear, the promising results found for latent means in measures with <25% non-invariance and large samples (Asparouhov & Muthén, 2014) lends confidence about the use of the UCLA-LS-4 (which had 7% non-invariance) in comparing composite mean scores. Caution however is needed when using this in studies with small samples and with few age groups.

Strengths and Limitations

This is the first study to explore age-invariance across the different versions of the UCLA Loneliness scale, offering important new data about whether their use across adulthood is appropriate. Our findings clearly indicate that UCLA’s measurement invariance should not be assumed and must be confirmed before meaningful and accurate comparisons can be made. The current study used a large dataset with a wide range of age groups. The use of the UCLA-LSR-20 is a unique strength because it allowed for the exploration of 10 different versions of the UCLA-LS and 15 different structures. Also, the methodological advancements used in the current study, including the use of LSEM, as well as using balanced age groups and alignment for the multigroup invariance testing, provided a thorough and robust analysis of the UCLA-LSR, and provide confidence in the accuracy of findings reported herein. The general recommendation from our findings is that adoption of any one of the age-invariant UCLA loneliness measures should be selected with a particular regard to the research question, study design, and data. This recommendation should, however, be considered in light of the following limitations.

First, while the use of UCLA-LSR allowed the examination of different versions, it is worth noting that the shorter ones examined here were embedded in the 20-item version. This is not the same as exploring their measurement invariance as isolated measures. Thus, the degree to which these findings would hold had the participants completed, for example, only the four items of the UCLA-LS-4, is unclear. It is possible that the psychometric properties of the shorter versions have been influenced by the order of items or the fact that the 20 items facilitated a better understanding of the construct. The former has been explored in the literature, although only in the context of multidimensional measures, with suggestions to avoid presenting items from the same dimension together (Şahin, 2021). The effect of the latter is yet to be examined. There is no evidence in the current study to suggest that the model fit of shorter versions was positively biased; indeed not all had a good fit (e.g., the UCLA-LS-4 by Roberts et al. (1993)). Still, it would be crucial that the current findings are replicated in samples that use isolated short versions of the UCLA-LS.

Second, while our recruitment strategy enabled us to reach a large number of participants, this was a non-probability sample (e.g., more women and White ethnic groups were represented). The study involved self-selection and online participation, likely missing specific populations (e.g., individuals with existing or severe mental illness, those lacking digital access etc.; Pierce et al., 2020) and biasing the sample (i.e., participants taking part may have had a particular interest in touch). Importantly, the degree to which gender, ethnicity, or other characteristics had an influence on the measurement invariance findings was not explored in the current study and should thus be considered in future work. The current findings are also based on a single study and as such must be cross-validated. For example, while our choice of age groups was guided by theory and evidence, it should be noted that such theories typically consider the age and timing of key life transitions (e.g., completing an education, getting married, starting a family, retiring from work; Hawkley et al., 2020). It is, therefore, unclear, to what extent the current findings would hold under different age groups. Third, to ensure a homogeneous sample, we used only participants from the United Kingdom who completed the survey; thus, the generalization of our findings in other countries and cultures cannot be assumed. Fourth, the use of the UCLA-LSR-20, although allowing for an extensive examination of structures, meant we were unable to explore the age-invariance of the more recent 20-item UCLA-LS3. Given the inconsistent evidence for its structure, as discussed earlier (Maes et al., 2022), future work focusing on UCLA-LS3 is clearly necessary. Fifth, from a methodological perspective, while ESEM provided a unique perspective into the invariance of the 20-item measure, it is not yet possible to implement ESEM with alignment and multiple imputation. Thus, we necessarily had to use a post hoc stepwise method for exploring partial measuring invariance, which has been heavily criticized (Asparouhov & Muthén, 2014; Marsh et al., 2018). Similarly, alignment with LSEM is not currently available, which meant approximate LSEM measurement invariant testing was not possible.

Finally, it is important to note that, while we discuss age differences in loneliness as if they reflect aging effects, with genuine changes in the experience of loneliness, such age differences might reflect true generational (birth cohort) differences in the willingness to endorse items measuring certain aspects of loneliness. The use of the prospective design in studies is required to address that issue (e.g., see Hawkley et al., 2019).

Implications for Research

Before drawing substantive conclusions about age differences based on national survey data, researchers need to assess whether loneliness is measured in the same way across age groups. While the UCLA-LS and its short forms are used widely to compare groups, we found that the majority of these measures are largely non-invariant across ages. This work, as previously mentioned, must be replicated in samples that use the UCLA-LS versions as isolated measures. Still, our findings are an important early contribution and have implications for researchers who want to explore age trends in loneliness and how loneliness is understood at different stages in life. There are, of course, other reasons to explore measurement invariance—for example, to explore cross-national data on loneliness. If cross-national data on loneliness are not tested for comparability, we risk making claims about differences across countries when we simply have methodological artifacts (Davidov et al., 2014). Given that comparisons are currently being made about how the COVID-19 pandemic and associated social restrictions, have impacted people in different countries, most often using one of the UCLA loneliness scales, there is a need to extend recent work (Hudiyana et al., 2021) and explore which versions of the UCLA loneliness scale are invariant across cultures (Demkowicz et al., 2021).

As previously noted, we were unable to explore the 3-item UCLA measure (Hughes et al., 2004) recommended as the national indicators of loneliness by the ONS (2018b). Given its current use in a number of national surveys in England, it will be possible, in the future, to explore its age invariance across child, adolescent, and adult samples.

Implications for Community and Clinical Practice

As the number of national organizations tackling loneliness increases (e.g., Campaign to End Loneliness, 2021; Ending Loneliness Together, 2020), there is a growing need for robust, yet realistic, tools for measuring loneliness in (often under-resourced) community settings, that is, that are valid, reliable, and brief for community organizations working on this issue. The current findings indicate that the UCLA-LS-4 and UCLA-LS-9 fulfill these criteria and may therefore be a suitable choice for this purpose. The cut-off scores provided for identifying who is highly lonely (and who is not) also offer a useful “rule of thumb” to help community organizations and clinicians to evaluate whether their programs have been effective or not and to make decisions accordingly. Still, it is important to note that the diagnostic accuracy (sensitivity, specificity) of such cut-offs are yet to be formally tested and as such should be used with caution.

In clinical practice settings, along with strong psychometric properties, brevity of scales and ease of administration and scoring are also valuable (such as reducing testing burden on both client and practitioner). However, these features must be weighed against the need to gain a comprehensive assessment of the experience of loneliness for each client. Consequently, both the UCLA-LS-4 and the UCLA-LS-9 recommended here may better serve as screeners for loneliness in clinical practice, identifying those who may benefit from a more detailed assessment. While these measures offer potential value for immediate use in community and clinical settings, there is also a rationale for developing a new practical scale that involves consumer and community service engagement together with the current evidence base. Such a scale may be more time-consuming to develop, validate, and implement but may lead to a more versatile measure of loneliness in the longer term (e.g., more relevant to multiculturally diverse communities).

Conclusion and Recommendations

The age-related measurement invariance testing for the UCLA-LS is crucially important for researchers, practitioners, and policy makers, and we have provided a start here that can be replicated in other countries. As this is the first study to explore the age measurement invariance of UCLA-LSR, more work is warranted to replicate these findings.

Generally, our findings point to a poor and, or inconsistent structure across age groups for the UCLA-LSR-20 loneliness measure as well as for many of its shorter forms. Full or approximate measurement invariance was reliably achieved for the following four UCLA-LS measures: both UCLA-LS-8 (Hays & DiMatteo, 1987; Roberts et al., 1993), the UCLA-LS-9 (Hawkley et al., 2005), and UCLA-LS-4 (Russell et al., 1980). This indicates that different age groups appear to understand and respond to loneliness items approximately the same way, and confirms their use for comparing loneliness levels between different age groups. Given the non-probability sample of the current study, the UCLA-LS measurement invariance found here should not be automatically assumed in other samples. Researchers are, therefore, urged to explore this in their own sample before meaningful comparisons can be made.

Until such time, we offer the following recommendations for researchers and practitioners looking to explore loneliness and how they can choose the most appropriate measure. Researchers and practitioners who are

1. collecting new data can choose to use either of the four measures that reliably achieved full or approximate scalar measurement invariance (see Appendix B);

2. using existing data can only use the two UCLA-LS-8 measures if the UCLA-LSR is used. The UCLA-LS-4 and UCLA-LS-9 versions can be drawn from both the UCLA-LSR and UCLA-LS3;

3. interested in using a multidimensional construct of loneliness should use the UCLA-LS-9;

4. interested in comparing latent means across age groups (e.g., within structural equation modeling) can use either of the four measures through alignment (the code is provided at https://osf.io/cqxrd/); and

5. interested in comparing composite means across age groups (i.e., total scores) should do so using the correlated UCLA-LS-9, and Factor 2 from that measure. The UCLA-LS-4, and Factors 1 and 3 from the UCLA-LS-9 can be used with caution, assuming large samples and small non-invariance.

Appendix A

Description of Items for the UCLA-LS3 and UCLA-LSR Measures

Item	UCLA-LS3	UCLA-LSR
1	How often do you feel that you are “in tune” with the people around you?	I feel in tune with the people around me.
2	How often do you feel that you lack companionship?	I lack companionship.
3	How often do you feel that there is no one you can turn to?	There is no one I can turn to.
4	How often do you feel alone?	I do not feel alone.
5	How often do you feel part of a group of friends?	I feel part of a group of friends.
6	How often do you feel that you have a lot in common with the people around you?	I have a lot in common with the people around me.
7	How often do you feel that you are no longer close to anyone?	I am no longer close to anyone.
8	How often do you feel that your interests and ideas are not shared by those around you?	My interests and ideas are not shared by those around me.
9	How often do you feel outgoing and friendly?	I am an outgoing person.
10	How often do you feel close to people?	There are people I feel close to.
11	How often do you feel left out?	I feel left out.
12	How often do you feel that your relationships with others are not meaningful?	My social relationships are superficial.
13	How often do you feel that no one really knows you well?	No one really knows me well.
14	How often do you feel isolated from others?	I feel isolated from others.
15	How often do you feel you can find companionship when you want it?	I can find companionship when I want it.
16	How often do you feel that there are people who really understand you?	There are people who really understand me.
17	How often do you feel shy?	I am unhappy being so withdrawn.
18	How often do you feel that people are around you but not with you?	People are around me but not with me.
19	How often do you feel that there are people you can talk to?	There are people I can talk to.
20	How often do you feel that there are people you can turn to?	There are people I can turn to.

Note. UCLA-LS3 = University of California Los Angeles, Loneliness Scale Version 3; UCLA-LSR = Revised University of California Los Angeles, Loneliness Scale.

Appendix B

The Four UCLA-LS Measures That Reliably Achieved Full or Approximate Scalar Measurement Invariance

UCLA-LS-4 by Russell et al. (1980).

Indicate how often you feel the way described in each of the following statements. Circle one number for each.
	Never	Rarely	Sometimes	Often
1. I feel in tune with the people around me. a	1	2	3	4
2. No one really knows me well.	1	2	3	4
3. I can find companionship when I want it. a	1	2	3	4
4. People are around me but not with me.	1	2	3	4

Note. UCLA-LS-4 = University of California Los Angeles, Loneliness Scale-4-item measures.

^aReverse-scored items. Scores are summed, with higher scores reflecting higher levels of loneliness.

UCLA-LS-8 by Roberts et al. (1993).

Indicate how often you feel the way described in each of the following statements. Circle one number for each.
	Never	Rarely	Sometimes	Often
1. I feel in tune with the people around me. a	0	1	2	3
2. I lack companionship.	0	1	2	3
3. I do not feel alone. a	0	1	2	3
4. I feel part of a group of friends. a	0	1	2	3
5. I am no longer close to anyone.	0	1	2	3
6. I feel left out.	0	1	2	3
7. I feel isolated from others.	0	1	2	3
8. I can find companionship when I want it. a	0	1	2	3

Note. UCLA-LS-8 = University of California Los Angeles, Loneliness Scale-8-item measures.

^aReverse-scored items. Scores are summed, with higher scores reflecting higher levels of loneliness.

UCLA-LS-8 by Hays & DiMatteo (1987).

Indicate how often you feel the way described in each of the following statements. Circle one number for each.
	Never	Rarely	Sometimes	Often
1. I lack companionship.	1	2	3	4
2. There is no one I can turn to.	1	2	3	4
3. I am an outgoing person. a	1	2	3	4
4. I feel left out.	1	2	3	4
5. I feel isolated from others.	1	2	3	4
6. I can find companionship when I want it. a	1	2	3	4
7. I am unhappy being so withdrawn.	1	2	3	4
8. People are around me but not with me.	1	2	3	4

Note. UCLA-LS-8 = University of California Los Angeles, Loneliness Scale-8-item measures.

^aReverse-scored items. Scores are summed, with higher scores reflecting higher levels of loneliness.

UCLA-LS-9 by Hawkley et al. (2005).

Indicate how often you feel the way described in each of the following statements. Circle one number for each.
	Never	Rarely	Sometimes	Often
1. How often do you feel that you lack companionship? a	1	2	3	4
2. How often do you feel that you are ‘in tune’ with the people around you?	1	2	3	4
3. How often do you feel that there are people you can talk to?	1	2	3	4
4. How often do you feel left out? a	1	2	3	4
5. How often do you feel that there are people you can turn to?	1	2	3	4
6. How often do you feel that you have a lot in common with the people around you?	1	2	3	4
7. How often do you feel close to people?	1	2	3	4
8. How often do you feel isolated from others? a	1	2	3	4
9. How often do you feel part of a group of friends?	1	2	3	4

Note. UCLA-LS-9 = University of California Los Angeles, Loneliness Scale-9-item measures.

^aReverse-scored items. Intimate Connectedness: 1, 4, 8; Relational Connectedness: 3, 5, 7; Collective Connectedness: 2, 6, 9.