Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Jan 1.
Published in final edited form as: Meas Eval Couns Dev. 2021 Oct 6;55(2):84–97. doi: 10.1080/07481756.2021.1955214

Validation of Scores on The Lifestyle Practices and Health Consciousness Inventory with Black and Latinx Adults in the United States: A Three-Dimensional Model

Michael T Kalkbrenner 1
PMCID: PMC9004479  NIHMSID: NIHMS1761167  PMID: 35422573

Abstract

The author tested the psychometric properties of the Lifestyle Practices and Health Consciousness Inventory (LPHCI), an interdisciplinary healthcare screening tool, with a stratified random sample (N = 4,009) of Black and Latinx adults in all 50 states. Results of EFA, CFA, higher-order CFA, and multiple-group CFA all supported a 3-dimensional LPHCI model.

Keywords: Lifestyle Practices and Health Consciousness Inventory, interdisciplinary health care, internal structure, psychometrics

Promoting diversity and inclusiveness through equitable access to mental and physical healthcare services is a core component in the identity of professional counselors (American Counseling Association, 2014). Minority ethnic populations, for example Black and Latinx adults living in the United States tend to face significant medical and mental healthcare disparities (Trinh et al., 2017). There is a growing need to address these healthcare disparities, as the Latinx and Black populations living in the United States reached record numbers in 2019 including 60,724,311 and 43,984,096, respectively (United States Census Bureau, 2020). Moreover, interdisciplinary healthcare is a key element in the practice of professional counselors (Mellin et al., 2011). Interdisciplinary healthcare involves professional counselors working on interprofessional teams with a variety of medical and other mental health professionals to simultaneously provide clients with high-quality mental and physical healthcare services (Johnson et al., 2017 ; Vogel et al., 2014). Professional counselors’ social justice orientation coupled with their increased engagement in interdisciplinary healthcare has created an important need for counseling research centered on promoting the mental health and wellness of clients who identify as Latinx or Black.

Healthcare screening or gathering data about a client’s mental and physical health before, during, and after treatment is an essential component in interdisciplinary healthcare (Kalkbrenner & Gormley, 2020). A key benefit of interdisciplinary health care is the concurrent focus on providing holistic (mental and physical) healthcare (Colorafi et al., 2017; Ritchie et al., 2016). The measurement literature, however, is only starting to catch up with this emerging interdisciplinary healthcare philosophy. A brief and publically available screening tool with strong psychometric support for capturing a single composite score of mental and physical wellness was not available until recently. Kalkbrenner and Gormley (2020) developed and validated scores on the Lifestyle Practices and Health Consciousness Inventory (LPHCI), an interdisciplinary healthcare screening tool for appraising integrated aspects of physical and mental health. Specifically, the LPHCI is comprised of four factors of holistic wellness that collectively comprise a higher-order factor (Global Wellness), which simultaneously appraises “(a) lifestyle practices, or activities, routines, and dietary habits, which are related to physical and/or mental wellness, and (b) health consciousness, or the extent to which people are aware of their own mental and physical health” (Kalkbrenner, in press). The psychometric properties of the LPHCI (see instrumentation section below) were established with a national sample of primarily White adults living in the United States. However, the measurement literature is lacking an interdisciplinary healthcare screening tool for appraising mental and physical health that has been normed with Latinx and Black populations. To this end, the primary purpose of the present study was to test the psychometric properties of scores on the LPHCI with a large, national sample of adults living in the United States who identify as Black or Latinx, populations that tend to face healthcare disparities. Counselors need screening tools for measuring overall wellness with clients who self-identify as Latinx or Black, considering the significant healthcare disparities facing these populations.

Healthcare Disparities Facing Latinx and Black Populations

Underrepresented populations, for example adults living in the United States who identify as Black or Latinx tend to face disparities in medical and mental health care, which places them at risk for suffering from non-communicable diseases and mental health disorders (Bouye et al., 2016; Kim & Richardson, 2012; Schneiderman et al., 2014; Trinh et al., 2017), which are associated with millions of premature deaths each year (World Health Organization, 2018). Moreover, Latinx and Black populations face a number of additional mental and physical health care disparities including, unequal access to health care (Bouye et al., 2016; Lee et al., 2015), discrimination in clinical encounters (Findling et al., 2019), implicit bias from clinicians (Blair et al., 2013), and lower healthcare retention rates (Sheehan et al., 2017). Furthermore, a number of demographic variables, including gender and help-seeking history (past attendance in counseling) can intersect with ethnic identity and further contribute to disparities in mental and medical healthcare (Kim et al., 2015; Neukrug et al., 2013; Trinh et al., 2017). Thus, researchers should test the factorial invariance (psychometric equivalence) of instrumentation across a number of demographic variables (i.e., factorial invariance testing) when testing construct validity.

Research Questions and Hypotheses

One of the first steps in supporting holistic wellness (mental and physical) is establishing an interdisciplinary healthcare screening tool yielding valid and reliable scores to measure this latent variable (Kalkbrenner & Gormley, 2020). However, the literature is lacking such a screening tool normed with Black and Latinx adults in the United States. This study sought to validate scores on an interdisciplinary healthcare screening tool (the LPHCI) with a large national sample of Black and Latinx adults, stratified by gender, age, and geographic region of the United States. The following research questions (RQs): RQ 1: What is the internal structure of the LPHCI with a large national, diverse sample of Black and Latinx adults in the United States?. RQ1a: Is a higher-order factor present in the data? RQ 2: Does the LPHCI maintain factorial invariance across key demographic variables that are associated with physical and mental health among Black and Latinx adults in the United States? RQ 3: What is the convergence of scores on the LPHCI’s higher-order factor (Global Wellness) and relevant theoretical constructs among adults living in the U.S. are Latinx or Black?

Methods

Participants and Procedures

The Researcher received an Institutional Development Award (IDeA) grant from the National Institute of General Medical Science to fund data collection for this project. Grant funding was used to hire a data collection contractor, Qualtrics Sample Services (2020). In recent years, a growing number of counseling researchers (Kalkbrenner & Gormley, 2020; Watson et al., 2020) are using Qualtrics Sample Services to collect data. The lead investigator entered the instrumentation (see section below) into the Qualtrics online survey platform and sent the electronic distribution link to a project manager from Qualtrics Sample Services. The project manager recruited a national random sample, stratified by the United States Census estimates for gender, age, and geographic location of adults living in the United States who identified as Black or Latinx. A team of analysts from Qualtrics Sample Services conducted a quality check on the data set to identify and remove potential poor quality responses, including speeders (less than 1/3 median time), random response patterns, and unrealistic answers (e.g., 18 years old with Ph.D.) A total of 4,009 quality responses were collected. A missing values analysis revealed less than 5% missing data for all cases and Little’s MCAR test indicated that the data were missing at random (p = .995). Accordingly, expectation maximization was used to impute missing values.

Participants were recruited from all 50 states in the United States and ranged in age from 18 to 90 years (M = 42.6; SD = 17.50). The demographic profile of the sample (N = 4,009) was as follows. Ethnicity was 51.5% (n = 2,063) Black and 48.5% (n = 1,946) Latinx. Gender identity was 50.1% (n = 2,008) identified as male, 49.0% (n = 1,963) female, 0.6% (n = 25) transgender, 0.2% (n = 8) gender non-binary, and 0.1% (n = 4) did not specify their gender identity. The demographic profile of the present sample for age and gender were consistent with the 2019 United States census estimates (United States Census Bureau, 2020). For help-seeking history, 39.0% (n = 1,562) had sought at least one session of personal counseling, 60.9% (n = 2,443) had never sought personal counseling, and 0.1% (n = 4) did not report their help-seeking history.

Methods for Estimating Construct Validity and Reliability Evidence

Two of the primary methods of estimating construct validity are as follows: tests of internal structure (typically factor analysis) and relations with other established theoretical constructs (Kane & Bridgeman, 2017; Swank & Mullen, 2017). Accordingly, the present author applied factor analysis to test the internal structure of scores on the LPHCI with a large sample of adults in the United States who self-identify as Latinx or Black. Pearson product-moment correlations are a commonplace method for testing the relationship between scores on newer tests with scores on established tests that measure similar constructs (Swank & Mullen, 2017). To this end, the present investigator tested the convergent validity of LPHCI scores by computing correlations with established tests. Although a myriad of psychometric support already existed for the convergent-related validity instruments (see the Instrumentation section), the present investigator tested internal structure validity of these measures with the present sample before conducting convergent validity testing, as the dimensionality of established instrumentation can vary between different samples (Byrne, 2016; Mvududu & Sink, 2013; Weston & Gore, 2006). To this end, confirmatory factor analyses (CFAs) were computed using IBM SPSS AMOS version 26 to test the internal structure validity of the instrumentation with the present sample. The combined recommendations of Dimitrov (2012) and Schreiber et al. (2006) for goodness-of-fit (GOF) indexes and thresholds for interpreting an acceptable model fit for CFA were investigated: The normative fit index (NFI, .90 to .95 = acceptable fit and > 0.95 = close fit), comparative fit index (CFI, .90 to .95 = acceptable fit and > 0.95 = close fit), root mean square error of approximation (RMSEA < 0.08), and standardized root mean square residual (SRMR < 0.08). The Chi square absolute fit index (CMIN, non-significant p-value) is also reported for each CFA, however, results are interpreted tentatively as this index is sensitive to large samples (Dimitrov, 2012).

Cronbach’s coefficient alpha (α) is the most frequently reported internal consistency reliability estimate in social sciences research (Kalkbrenner, 2021; McNeish, 2018). Despite α’s popularity it tends to misrepresent internal consistency reliability unless the data meet a number of specific statistical assumptions. For example, α requires that the assumption of tau-equivalence is met, however, tau-equivalence is rarely met in psychometric research on attitudinal variables (McNeish, 2018). Tau-equivalence is not required for composite reliability (CR) estimates (e.g., McDonald’s omega [ω]), which is a more stable and robust internal consistency reliability estimate than α, especially in psychometric research (McNeish, 2018), as α is just a special case of ω (McDonald, 1999). In other words, α = ω when the data meet all of α’s necessary statistical assumptions (McDonald, 1999; McNeish, 2018). To this end, the present investigator utilized ω to estimate internal consistency reliability of scores on the LPHCI. Based on the findings of a Monte Carlo simulation study conducted by Nájera Catalán (2019), ω > .65 is the minimum acceptable threshold for acceptable internal consistency reliability evidence.

Instrumentation

Respondents first confirmed that they met the eligibility criteria for participation in the study including, (a) at least 18 years old and (b) self-identified as Latinx or Black. Participants then completed a battery of tests including, a demographic questionnaire, the Lifestyle Practices and Health Consciousness Inventory (LPHCI), the General Anxiety Disorder-7 (GAD-7), Patient Health Questionnaire-9 (PHQ-9), and the Health-Related Quality of Life (HRQOL).

General Anxiety Disorder-7

The General Anxiety Disorder-7 (GAD-7) is a screening tool for measuring anxiety severity or the extent to which one has experienced symptoms of an Anxiety Disorder within the last 30 days (Spitzer et al., 2006). With over 8,000 citations (Google Scholar, 2020), the GAD-7 is a commonly used and rigorously tested screening tools for appraising anxiety severity. Dozens of past investigators (e.g., Omani-Samani et al., 2018; Seo & Park, 2015) demonstrated moderate-to-strong reliability evidence and validity evidence for scores on the GAD-7. The present investigator found strong reliability evidence (α = .92, ω = .93) for scores on the GAD-7 with the sample of Black and Latinx participants in the present study. The GAD-7 items were entered into a CFA to test for construct validity and a moderate-to-strong model fit emerged: CMIN, χ2 (14) = 213.04, p < 0.001; CFI = 0.99; NFI = 0.99; RMSEA = 0.061, 90% CI (0.054, 0.068); and SRMR = 0.02.

Patient Health Questionnaire-9

The Patient Health Questionnaire-9 (PHQ-9) is an internationally used screening tool for measuring depression severity or the degree to which one experienced symptoms of a Depressive Disorder in the past 30 days (Kroenke et al., 2001). Myriad psychometric researchers (e.g., Keum et al., 2018; Kocalevent et al., 2013) demonstrated strong internal structure validity (factor analysis) and reliability evidence (α = .86 to .93) for scores on the PHQ-9 across a number of different populations. Strong internal consistency reliability evidence (α = .93, ω = .93) emerged for scores on the PHQ-9 with the sample of Black and Latinx participants in the present study. The PHQ-9 items were entered into a CFA to test for internal structure validity and collectively, an acceptable model fit emerged: CMIN, χ2 (27) = 844.83, p < 0.001; CFI = 0.97; NFI = 0.96; RMSEA = 0.088, 90% CI (0.083, 0.094); and SRMR = 0.03.

The Centers for Disease Control Health-Related Quality of Life

The Centers for Disease Control Health-Related Quality of Life (CDC HRQOL-4) is a screening tool for measuring unhealthy days or “an estimate of the overall number of days during the previous 30 days when the respondent felt that either his or her physical or mental health was not good” (United States Department of Health and Human Services, 2000, p. 8). Yin et al. (2016) found satisfactory internal consistency reliability evidence (α = .76) and structural validity evidence (factor analysis) for the CDC HRQOL-4 scores. The present investigator entered the HRQOL-4 scores into a CFA to test for internal structure validity and inconclusive validity evidence emerged. The iteration limit was reached, indicating that that the fit statistics were unreliable as the model failed to reach minimization (Arbuckle, 2016). It was not possible to test internal consistency using ω as factor loadings were not available due to the inconclusive CFA results. However, Cronbach’s coefficient alpha revealed very poor internal consistency reliability evidence (α = .45) for CDC HRQOL-4 scores with the sample of Black and Latinx participants in the present study. Thus, the CDC HRQOL-4 was not included in the present study due to poor reliability, inconclusive validity, and since it was not essential for answering the third research question. The CDC HRQOL-4 was just one of three screening tools for testing the convergent-related validity evidence of scores on the LPHCI’s Global Wellness scale.

Lifestyle Practices and Health Consciousness Inventory

The LPHCI is a 20-item screening tool for appraising holistic (mental and physical) wellness (Kalkbrenner & Gormley, 2020). The LPHCI is comprised of four subscales including Consciousness of Stress (self-awareness of one’s own physical and mental health), Self-Care (engagement in fulfilling and relaxing activities), Aerobic Exercise (participation in physical activities), and Food Choices (dietary habits and nutrition). Although the subscales can be administered and scored separately, the LPHCI’s novel construct of measurement is the Global Wellness scale (composite score across the four subscales), which simultaneously appraises “(a) lifestyle practices, or activities, routines, and dietary habits, which are related to physical and/or mental wellness, and (b) health consciousness, or the extent to which people are aware of their own mental and physical health” (Kalkbrenner, in press, p. 6). Participants respond to a Likert-type scale with the following anchor definitions: “in the past 30 days, how often have you…,” on the following scale, 0 = never, 1 = one to five times, 2 = six to ten times, 3 = eleven to fifteen times, 4 = sixteen to twenty times, or 5 = twenty one or more times.” Kalkbrenner and Gormley (2020) found adequate internal consistency reliability evidence (α ≥ .83) for the Consciousness of Stress, Self-Care, Aerobic Exercise, Food Choices, and Global Wellness scales of the LPHCI. Kalkbrenner and Gormley (2020) also demonstrated structural validity evidence for the LPHCI with a large stratified random sample of adults living in the United States through a series of major psychometric analyses (i.e., EFA, CFA, & higher-order CFA). In the present study, satisfactory internal consistency reliability evidence emerged for scores on the Consciousness of Stress (ω = .86), Aerobic Exercise (ω = .70), and Food Choices (ω = .68), which all surpassed the minimum threshold for acceptable internal consistency reliability (ω > .65) based on the findings of Nájera Catalán (2019). Poor reliability evidence (ω = .52), however, emerged for the Self-Care subscale with the present sample, which indicated a need for additional psychometric testing.

Results

The LPHCI: Psychometric Testing

The reliability and validity of scores on latent traits (subscales) of an established measure can vary substantially between different populations (Byrne, 2016; Mvududu & Sink, 2013; Weston & Gore, 2006), as these latent traits are theoretical constructs, which are not inherently valid across different samples and populations. The weak internal consistency reliability estimate that emerged for the Self-Care subscale of the LPHCI in the present study suggests that this subscale was not reliably capturing the latent trait of Self-Care among Latinx and Black participants in the present sample. Based on the recommendations of these leading psychometric researchers, the present investigator re-tested the dimensionality of the LPHCI model via EFA CFA to answer the first research question.

Exploratory Factor Analysis

The data were randomly divided into two samples for computing EFA (N = 2,005) and CFA (N = 2,004). A review of skewness and kurtosis values for both data sets revealed no extreme deviations of item scores from normality (skewness > ±2 and Kurtosis > ±7, Dimitrov, 2012). The Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO = 0.89), Bartlett’s Test of Sphericity (B[120] = 9,443.01, p < 0.001), and an inter-item correlation matrix indicated that the data were factorable. An EFA with a maximum likelihood (ML) extraction method and an oblique rotation (direct oblimin, Δ = 0) was employed as latent traits of mental and physical wellness tend to inter-correlate (Kalkbrenner & Gormley, 2020). The investigator used the following factor retention criteria based on the recommendations of Beavers et al. (2013) and Mvududu and Sink (2013): factor loadings > 0.40, communalities (h2) > 0.30, and cross-loadings < 0.30. The commonality values of the four items that comprise the Self-Care subscale were all < 0.30. Thus these four items were removed one at a time from the inter-item correlation matrix until all of the h2 values were > 0.30.

There exist a number of methods for determining the appropriate number of factors to retain in EFA and researchers are tasked with retaining the most parsimonious factor solution that makes both statistical and practical sense based on previous research and theory (Bandalos & Finney, 2019). The EFA revealed a clear 3-factor solution based on the following factor retention criteria: Kaiser Criterion (KV, Eigenvalues ≥ 1), meaningful variance accounted for by a factor (≥ 5%), and scree test. A parallel analysis (PA) revealed a 7-factor solution, however, PA can overestimate the number of extracted factors with large sample sizes. Moreover, there were only 16 items in the inter-item correlation matrix, thus a 7-factor solution did not make practical sense, as each scale would be comprised of approximately two items. Based on the KV, meaningful variance, Scree plot, parsimony, and previous theory (Kalkbrenner & Gormley, 2020) a final 3-factor solution was retained (see Table 1).

Table 1.

Exploratory Factor Analysis Results: Pattern Matrix with an Oblique Rotation (N = 2,005)

Consciousness of Stress Aerobic Exercise Food Choices
Item Content Loadings
*14. Had stomachache severe enough to interfere with your daily routine 0.74 −0.11
*4. Felt too nervous to eat 0.72 −0.13
*9. Had headache severe enough to interfere with your daily routine 0.72
*20. Were unable to go about your daily routine (e.g., work or school) due to feeling anxious or scared 0.72
*16. Skipped a meal despite feeling hungry 0.63
*17. Had no one to talk to 0.63
*6. Had any kind of sleep problem 0.57 0.11
*11.Had muscle ache (e.g., back, shoulder, foot) severe enough to interfere with your daily routine 0.50 0.20
*19. Eaten a desert containing refined sugar (e.g., cookies, donuts, pastries, ice cream, cake) −0.10 0.64
*3. Drank 12oz or more of soda (or pop) 0.58
*8. Eaten fast food (McDonalds, Burger King, Taco Bell) 0.10 0.57
*13. Eaten one serving of (10 – 15) chips 0.56 −0.12
12. Participated in outdoor exercise (e.g., biking, hiking, jogging, rock-climbing) 0.11 −0.10 0.71
18. Used exercise equipment (e.g., stationary bike, treadmill, weight lifting) 0.59
1. Played an active sport 0.57
5. Engaged in ascetic exercise (e.g., yoga, martial arts, Qigong, Tai Chi) 0.49
Eigen Value 4.99 1.96 1.54
% of variance 31.1% 12.24% 9.61%
Open in a new tab

Note: Factor loadings over 0.40 appear in bold and mark the particular factor. Blank cells indicate factor loadings < 0.10.

The 16 items that comprised the 3-factor solution (see Table 1) clustered identically to the structure of Kalkbrenner & Gormley’s Consciousness of Stress, Aerobic Exercise, and Food Choices subscales. Thus, the present investigator kept the names of the three factors consistent: Consciousness of Stress, Aerobic Exercise, and Food Choices, respectively. CR (McDonald’s omega [ω]) estimates were computed to test the internal consistency reliability of scores on the 3-dimensional LPHCI. The CR coefficients for the Consciousness of Stress (ω = .86), Aerobic Exercise (ω = .70), Food Choices (ω = .70), and Global Wellness (ωh = .83) subscales all exceeded the threshold for acceptable internal consistency reliability (ω > .65, ωh > .80) based on the findings of Nájera Catalán (2019).

Content Validity Considerations

Although the results of the EFA supported the structural validity of scores on the 3-Dimensional LPHCI model, the investigator was concerned about the content validity of the Global Wellness subscale (total composite score across all LPHCI items) since the Self-Care subscale was not included in the 3-factor model. To this end, the investigator collaborated with an external research team and compared the content of the 16 items that comprise the 3-factor LPHCI model (see Table 1) to the theoretical constructs that the Global Wellness scale appraises including, “(a) lifestyle practices, or activities, routines, and dietary habits, which are related to physical and/or mental wellness, and (b) health consciousness, or the extent to which people are aware of their own mental and physical health” (Kalkbrenner, in press, p. 6). Research team members concluded that the 16 item 3-dimensional LPHCI model adequately captured the content domains of Global Wellness including, health-related activities, practices, and dietary habits (items: 1, 4, 5, 12, 16, 17, 18, & 20), nutrition (items: 3, 8, 13, & 19), and one’s health consciousness (items: 6, 9, 11, & 14).

Confirmatory Factor Analysis

CFA was performed to confirm the internal structure of the LPHCI data with the second sample (N = 2,004) of Black and Latinx participants. A maximum likelihood (ML) estimation method was used, as a review of skewness and kurtosis values revealed that the data were consistent with a normal distribution. The 3-dimensional LPHCI model (see Figure 1, Model 1) produced acceptable fit statistics: CMIN, χ2 (101) = 1,398.94, p < 0.001; CFI = 0.94; NFI = 0.93; RMSEA = 0.052, 90% CI (0.048, 0.056); and SRMR = 0.044. Except for the CMIN (which is sensitive to large samples), all fit indices demonstrated an acceptable model fit and support for the internal structure of the 3-factor LPHCI. The CFA data did not meet the assumption of tau-equivalence (see Figure 1, Model, 1), thus composite reliability estimates were computed to test for internal consistency reliability. The composite reliability estimates for the Consciousness of Stress (ω = .87), Aerobic Exercise (ω = .69), and Food Choices (ω = .68) subscales all demonstrated acceptable internal consistency reliability evidence based on the findings of Nájera Catalán (2019).

Figure 1.

Figure 1

Open in a new tab

LPHCI Path Models with Standardized Estimates

Higher-Order Confirmatory Factor Analysis

Higher-order CFA is a theory testing method for investigating if the co-variation between single-order factors is explained by a second-order latent trait (Credé & Harms, 2015). A second-order factor might be present in the data if (a) at least moderate associations exist between factors, (b) the unidimensional model demonstrates a poor model fit, and (c) there is a theoretical justification for a second-order factor (Kalkbrenner & Gormley, 2020). An investigation of the path model coefficients (see Figure 1, Model 1) revealed moderate associations between the Consciousness of Stress and Food Choices, as well as Aerobic Exercise and the Consciousness of Stress subscales. The unidimensional LPHCI model demonstrated poor fit with the data (CMIN, χ2 [104] = 4,753.35, p < 0.001; CFI = 0.75; NFI = 0.75; RMSEA = 0.106, 90% CI [0.103, 0.108); and SRMR = 0.09). Finally, the theoretical underpinning of the LPHCI (see Kalkbrenner & Gormley, 2020) is based on a higher-order factor (composite latent trait) of mental and physical wellness. A higher-order CFA was computed (see Figure 1, Model 2) to test for the presence of a higher-order factor (RQ1a). Taken together, the higher-order CFA results supported acceptable model fit: CMIN, χ2 (101) = 1,398.94, p < 0.001; CFI = 0.94; NFI = 0.93; RMSEA = 0.052, 90% CI (0.048, 0.056); and SRMR = 0.044).

Factorial Invariance Testing

A multiple-group CFA was computed (see Table 2) to answer the second research question about the psychometric equivalence of the 3- Dimensional LPHCI with the following subgroups of the total sample (N = 4,009), gender identity (male n = 2,008 and female n = 1,964), ethnic identity (Black n = 2,063 and Latinx n = 1,945), and help-seeking history (previous attendance in counseling n = 1,562 and no previous attendance in counseling n = 2,443). Chen’s (2007) recommendations for fit indexes and invariance thresholds for configural, metric, and scalar invariance were investigated including, a non-significant Satorra and Bentler chi-square difference test, ≤ Δ 0.010 in CFI, and ≤ Δ 0.015 in RMSEA. The chi-square difference tests revealed significant decreases in model fit for all subgroups (see Table 2), however, the chi-square difference test is sensitive to large samples (Chen, 2007; Dimitrov, 2010). Thus, the following GOF indices and invariance thresholds that are more appropriate for large samples were used to test for invariance: < Δ 0.010 in CFI and < Δ 0.015 in RMSEA (see Table 2). Based on the guidelines provided by Dimitrov (2010), the RMSEA fit index (see Table 2) revealed strong measurement invariance (metric and scalar) for all groups (ethnicity, gender, and help-seeking history). The CFI index indicated strong measurement invariance (metric and scalar) for ethnicity and acceptable invariance (metric) for gender and help-seeking history. Collectively, the RMSEA and CFI estimates (see Table 2) revealed moderate-to-strong evidence for the factorial invariance of the 3- dimensional LPHCI based on the recommendations of Dimitrov (2010).

Table 2.

Multiple-Group Confirmatory Factor Analysis: Measurement Invariance of the Mental Distress Response Scale

Invariance Forms χ2 df Δ χ2 Δdf p CFI ΔCFI RMSEA ΔRMSEA Model Comparison
Gender: Male vs. Female
Configural 1579.80 204 <.001 .93 .041
Metric 1587.40 215 7.6 11 <.001 .93 0 .040 .001 Configural
Scalar 1864.16 231 276.76 16 <.001 .91 .02 .042 .002 Metric
Help-Seeking History: Past Attendance in Counseling vs. Nonattendance in Counseling
Configural 1571.37 204 <.001 .92 .041
Metric 1576.40 215 5.03 11 <.001 .92 0 .040 .001 Configural
Scalar 2298.40 231 722 16 <.001 .87 .05 .047 .007 Metric
Ethnicity: Black vs. Latinx
Configural 1535.67 204 <.001 .93 .040
Metric 1547.59 215 2.55 11.92 <.001 .93 0 .001 .003 Configural
Scalar 1663.88 231 26.44 116.29 <.001 .92 .01 .039 .002 Metric
Open in a new tab

Convergent-Related Validity

Bivariate correlations between new and established theoretical constructs are a common method for demonstrating evidence of convergent-related validity in counseling research (Swank & Mullen, 2017). Pearson product moment correlations were computed to answer the third research question about the associations between Global Wellness (higher-order scale of the LPHCI) and depression severity (PHQ-9 score), and anxiety severity (GAD-7 score). A significant, negative association emerged between Global Wellness and GAD-7 anxiety severity, r = −.58, p < .001, r2 = .34. A significant negative correlation also emerged between Global Wellness and PHQ-9 depression severity, r = −.60, p < .001, r2 = .36.

Discussion

Collectively, the results of an EFA, CFA, higher-order CFA, multiple-group CFA, and correlational analyses supported the internal structure and convergent validity of scores on a revised, 3-dimensional version of the LPHCI with a large U.S. sample of Black and Latinx participants. Similar to Kalkbrenner and Gormley (2020), the results of the present investigation indicated that Food Choices, Aerobic Exercise, and Consciousness of Stress are correlated dimensions of mental health and physical wellness, which collectively comprise a higher-order Global Wellness scale. However, inconsistent with Kalkbrenner and Gormley (2020), the results of the present study revealed poor reliability and inconclusive validity evidence for the Self-Care subscale scores on the LPHCI with participants who self-identify as Black or Latinx. The Self-Care subscale of the LPHCI was uncovered (EFA) and confirmed (CFA) among a general sample of primarily White adults living in the United States. It is possible that the construct of self-care among adults living in the United States who identify as Latinx or Black differs from populations in which White cultural worldviews are dominant. To this end, the present investigator removed the Self-Care subscale items due to their poor score reliability and inconclusive validity evidence and then followed the recommendations of leading psychometric researchers (e.g., Byrne, 2016; Mvududu & Sink, 2013) to uncover (EFA) and confirm (CFA) the dimensionality of a re-specified, 3-dimensional LPHCI model (RQ 1). This type of model re-specification is common in psychometric research as the dimensionality of existing instrumentation can vary substantially between different populations (Byrne, 2016; Mvududu & Sink, 2013; Weston & Gore, 2006). As just one example, Lim and Kim (2020) tested the psychometric properties of the California Brief Multicultural Competence Scale, which had an established 4-factor model with a new population and retained a 3-factor model.

Similar to Kalkbrenner and Gormley (2020), the results of a higher-order CFA revealed a second-order factor (Global Wellness) among adults living in the United States who self-identify as Black or Latinx (RQ1a). Consistent with interdisciplinary health care frameworks and theoretical models of integrated physical and mental wellness (e.g., Servan-Schreiber, 2009), these findings suggested that mental and physical health are interconnected constructs. The results of the present study extend this interconnected mental and physical healthcare philosophy to adults living in the United States who identify as Black or Latinx. Specifically, Consciousness of Stress, Aerobic Exercise, and Food Choices appear to be three related dimensions of wellness that collectively comprise a higher-order latent construct of Global Wellness.

Past investigators found a number of mental and physical heath disparities by gender (Neukrug et al., 2013), ethnicity (Trinh et al., 2017), and help-seeking history (Kim et al., 2015). The results of a multiple-group CFA in the present study demonstrated moderate-to-strong evidence (Dimitrov, 2010) for the factorial invariance of the 3-dimensional LPHCI across ethnicity, gender, and help-seeking history among a large sample of adults living in the United States who identify as Black or Latinx (RQ2). Collectively, results suggest that Consciousness of Stress, Aerobic Exercise, Food Choices, and Global Wellness are elements of holistic wellness among important demographic subgroups of adults living in the United States who identify as Latinx or Black.

Based on the recommendations of Swank & Mullen (2017), the present investigator tested the convergent-related validity evidence of scores on the 3-dimensional LPHCI by conducting bivariate correlations between Global Wellness and other well-established measures (RQ3). Correlations demonstrated significant negative associations between Global Wellness and PHQ-9 total scores as well as between Global Wellness and GAD-7 total scores. The effect size of these associations were in the strong range (see Swank & Mullen, 2017), suggesting a noteworthy amount of shared variance between constructs. These findings support the convergent validity of scores on the Global Wellness subscale as higher levels of holistic (physical and mental) health tend to predict lower levels of physical and mental distress (Bertheussen et al., 2011; Velten et al., 2018). Future research is needed to test the directionality of these findings, however, engaging in healthy lifestyle practices (LPs) might lead to lower levels of anxiety severity and depression severity among adults living in the United States who identify as Black or Latinx.

The present investigator also tested the convergent-related validity of scores on the Global Wellness scale with the CDC HRQOL-4; a measure that has been subjected to rigorous psychometric testing (see Yin et al., 2016). However, inconclusive validity and poor reliability evidence emerged for scores on the CDC HRQOL-4 with the present sample. It is possible that the content of the underlying latent trait that the CDC HRQOL-4 is designed to appraise (unhealthy days) differs among the present sample of Latinx and Black populations. It is also possible that differences in the data collection procedures between the present investigator and Yin et al. (2016) contributed to the disparities in validity and reliability estimates. The present investigator collected data through an electronic survey platform while Yin et al. (2016) collected data via a random-digit-dial telephone health survey system. Future research is needed to test these possibilities, as current findings suggested that the CDC HRQOL-4 might not yield valid or reliable scores as a screening tool for use with adults living in the United States who identify as Latinx or Black.

Limitations, Implications, and Future Directions

The large (N = 4,009) national stratified random sample in the present study is among the most comprehensive samples of Latinx and Black participants in the extant literature to date. Although the data were stratified by the United States Census Bureau’s estimates for gender, age, and geographic location of the United States, the data are not necessarily representative of all Latinx and Black participants in the United States, as a number of additional key demographic variables (e.g., socioeconomic status) exist. Relatedly, the present author’s use of a fully online sampling method might have limited the data to responses from participants who had convenient internet accessibility. The findings of the present study were promising and provide initial evidence for the internal structure, convergent validity, and internal consistency reliability of scores on the 3-dimensional LPHCI with a large sample of Latinx and Black participants. Cross-cultural fairness is a vast construct that includes, however, is not limited to internal structure, convergent validity, and internal consistency reliability (Kane, 2010; Neukrug & Fawcett, 2015). Future investigators can further investigate the generalizability of the 3-dimensional LPHCI by conducting criterion-related validity testing. As just one example, future researchers can test the extent to which scores on the 3-dimensional LPHCI predict mental and physical health outcomes. Moreover, Latinx and Black populations are not homogenous groups, thus future researchers can extend this line of inquiry by testing the factorial invariance of the 3-dimensional LPHCI with groups of participants who identity with specific Black and Latinx heritage groups (e.g., Dominicans, Puerto Ricans, see Schneiderman et al., 2014). In addition, future researchers can continue to investigate cross-cultural fairness by translating the 3-dimensional LPHCI into different languages and testing its psychometric properties.

Consistent with RQ3, negative associations emerged between Global Wellness and both PHQ-9 and GAD-7 scores. Future researchers can test the directionality of these findings using a structural equation model. Furthermore, past investigators identified demographic correlates (gender, ethnicity, help-seeking history) with mental and physical health disparities among underrepresented populations (Kim et al., 2015; Neukrug et al., 2013; Trinh et al., 2017). The multiple-group CFA in the present study indicated that the 3-dimensional LPHCI was psychometrically equivalent across ethnicity, gender, and help-seeking history among a large sample of adults living in the United States who self-identify as Black or Latinx. Future researchers should examine differences by gender, help-seeking history, and other relevant demographic variables in Global Wellness among Black and Latinx populations. Results might help counseling practitioners and researchers identify sub-groups of Latinx or Black clients who are vulnerable to lower levels of Global Wellness.

To the best of our knowledge, the present study is the first interdisciplinary healthcare-related investigation of the utility of a holistic healthcare inventory (the LPHCI) among Black and Latinx populations. Considering the health disparities facing Black and Latinx populations, the 3-dimensional LPHCI has a number of implications for counselors who work with Black or Latinx clients. The LPHCI offers counseling researchers and practitioners a screening tool for appraising healthy LPs and health consciousness among these populations. Counselors can administer the LPHCI to clients before, during, and after treatment as one way to set goals, monitor progress, and evaluate the efficacy of treatment. Counselors can use the results to track their clients’ Global Wellness scores throughout treatment and identify instances where the client’s Global Wellness scores are higher. Clients and counselors can then work together to uncover the life-circumstances/daily routine that is associated with the client’s higher engagement in healthy LPs and health consciousness. Results of the LPHCI might also aid in interprofessional treatment meetings in which counselors collaborate with medical and other mental health professionals on interdisciplinary teams. Collectively, the results of this study suggest that the 3-dimensional LPHCI might have utility for enhancing counselors’ mental and physical healthcare screening when working with clients who self-identify as Black or Latinx.

Funding:

This research was supported by an Institutional Development Award (IDeA) from the National Institute of General Medical Sciences of the National Institutes of Health under grant number P20GM103451.

Footnotes

Conflict of Interest:

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Publisher's Disclaimer: The Version of Record of this manuscript has been published and is available online in the journal of Measurement and Evaluation in Counseling and Development. Published online: 06 October 2021, doi:10.1080/07481756.2021.1955214

References

  1. American Counseling Association. (2014). 2014 ACA code of ethics. https://www.counseling.org/docs/default-source/default-document-library/2014-code-of-ethics-finaladdress.pdf
  2. Arbuckle JL (2016). IBM SPSS AMOS user’s guide. (24th ed.). IBM Corp. [Google Scholar]
  3. Bandalos DL, & Finney SJ (2019). Factor analysis: Exploratory and confirmatory. In Hancock GR, Stapleton LM, & Mueller RO (Eds.), The reviewer’s guide to quantitative methods in the social sciences (pp. 98–122). Routledge. [Google Scholar]
  4. Beavers AA, Lounsbury JW, Richards JK, Huck SW, Skolits GJ, & Esquivel SL (2013). Practical considerations for using exploratory factor analysis in educational research. Practical Assessment, Research & Evaluation, 18(5/6), 1–13. 10.7275/qv2q-rk76 [DOI] [Google Scholar]
  5. Bertheussen F, Romundstad R, Landmark L, Kaasa L, Dale L, & Helbostad L (2011). Associations between physical activity and physical and mental health- A HUNT 3 Study. Medicine & Science in Sports & Exercise, 43(7), 1220–1228. 10.1249/MSS.0b013e318206c66 [DOI] [PubMed] [Google Scholar]
  6. Blair I, Steiner J, Fairclough D, Hanratty R, Price D, Hirsh H, Wright L, Bronsert M, Karimkhani E, Magid D, & Havranek E (2013). Clinicians’ implicit ethnic/racial bias and perceptions of care among Black and Latino patients. Annals of Family Medicine, 11(1), 43–52. 10.1370/afm.1442 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bouye K, Mccleary K, Williams K, & Bouye K (2016). Increasing diversity in the health professions: Reflections on student pipeline programs. Journal of Healthcare, Science and the Humanities, 6(1), 67–79. http://search.proquest.com/docview/2024477126/ [PMC free article] [PubMed] [Google Scholar]
  8. Byrne B (2016). Structural equation modeling with AMOS: Basic concepts, applications, and programming (3rd ed.). Routledge. [Google Scholar]
  9. Chen FF (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504. 10.1080/10705510701301834 [DOI] [Google Scholar]
  10. Colorafi K, Vanselow J, & Nelson T (2017). Treating anxiety and depression in primary care: Reducing barriers to access. Family Practice Management, 24(4), 11–16. https://www.aafp.org/fpm/2017/0700/fpm20170700p11.pdf [PubMed] [Google Scholar]
  11. Credé M, & Harms P (2015). 25 years of higher-order confirmatory factor analysis in the organizational sciences: A critical review and development of reporting recommendations. Journal of Organizational Behavior, 36(6), 845–872. 10.1002/job.2008 [DOI] [Google Scholar]
  12. Dimitrov DM (2010). Testing for factorial invariance in the context of construct validation. Measurement and Evaluation in Counseling and Development, 43(2), 121–149. 10.1177/0748175610373459 [DOI] [Google Scholar]
  13. Dimitrov D (2012). Statistical methods for validation of assessment scale data in counseling and related fields. American Counseling Association. [Google Scholar]
  14. Findling M, Bleich S, Casey L, Blendon R, Benson J, Sayde J, & Miller C (2019). Discrimination in the United States: Experiences of Latinos. Health Services Research, 54(S2), 1409–1418. 10.1111/1475-6773.13216 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Google Scholar. (2020, August). A brief measure for assessing Generalized Anxiety Disorder: The GAD-7. https://scholar.google.com/scholar?hl=en&as_sdt=0%2C32&sciodt=0%2C32&cites=3790605624533378432&scipsc=&q=A+brief+measure+for+assessing+Generalized+Anxiety+Disorder%3A+The+GAD-7&btnG= [DOI] [PubMed]
  16. Johnson KF, & Kalkbrenner MT (2017). The utilization of technological innovations to support college student mental health: Mobile Health Communication, Journal of Technology in Human Services, 35(4), 1–26. 10.1080/15228835.2017.1368428 [DOI] [Google Scholar]
  17. Kalkbrenner MT (2021). A practical guide to instrument development and score validation in the social sciences: The MEASURE Approach. Practical Assessment, Research & Evaluation. 26, Article 1. 10.7275/svg4-e671 [DOI] [Google Scholar]
  18. Kalkbrenner MT (in press). Global Wellness: Predicting lower levels of anxiety and depression severity. Journal of Counseling and Development. [Google Scholar]
  19. Kalkbrenner MT, & Gormley B (2020). Development and initial validation of scores on the Lifestyle Practices and Health Consciousness Inventory (LPHCI). Measurement and Evaluation in Counseling and Development. 53(4), 219–237. 10.1080/07481756.2020.1722703 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Kane M (2010). Validity and fairness. Language Testing, 27(2), 177–182. 10.1177/0265532209349467 [DOI] [Google Scholar]
  21. Kane M, & Bridgeman B (2017). Research on validity theory and practice at ETS. In Bennett RE, & von Davier M (Eds.), Methodology of educational measurement and assessment. Advancing human assessment: The methodological, psychological and policy contributions of ETS (p. 489–552). Springer Science + Business Media. 10.1007/978-3-319-58689-2_16 [DOI] [Google Scholar]
  22. Keum B, Miller M, & Inkelas K (2018). Testing the factor structure and measurement invariance of the PHQ-9 across racially diverse U.S. college students. Psychological Assessment, 30(8), 1096–1106. 10.1037/pas0000550 [DOI] [PubMed] [Google Scholar]
  23. Kim J, & Richardson V (2012). The impact of socioeconomic inequalities and lack of health insurance on physical functioning among middle-aged and older adults in the United States. Health & Social Care in the Community, 20(1), 42–51. 10.1111/j.1365-2524.2011.01012.x [DOI] [PubMed] [Google Scholar]
  24. Kim JE, Saw A, & Zane N (2015). The influence of psychological symptoms on mental health literacy of college students. American Journal of Orthopsychiatry, 85(6), 620–630. 10.1037/ort0000074 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Kocalevent RD, Hinz A, & Brähler E (2013). Standardization of the depression screener Patient Health Questionnaire (PHQ-9) in the general population. General Hospital Psychiatry, 35(5), 551–555. 10.1016/j.genhosppsych.2013.04.006 [DOI] [PubMed] [Google Scholar]
  26. Kroenke K, Spitzer R, & Williams J (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613. 10.1046/j.1525-1497.2001.016009606.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Lee S, Matejkowski J, & Han W (2015). Racial–ethnic variation in mental health service utilization among people with a major affective disorder and a criminal history. Community Mental Health Journal, 53(1), 8–14. 10.1007/s10597-015-9899-8 [DOI] [PubMed] [Google Scholar]
  28. Lim E, & Kim S (2020). A validation of a multicultural competency measure among South Korean counselors. Journal of Multicultural Counseling and Development, 48(1), 15–29. 10.1002/jmcd.12161 [DOI] [Google Scholar]
  29. McDonald RP (1999). Test theory: A unified treatment. Lawrence Erlbaum. [Google Scholar]
  30. McNeish D (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412–433. 10.1037/met0000144 [DOI] [PubMed] [Google Scholar]
  31. Mellin E, Hunt B, & Nichols L (2011). Counselor professional identity: Findings and implications for counseling and interprofessional collaboration. Journal of Counseling & Development, 89(2), 140–147. 10.1002/j.1556-6678.2011.tb00071.x [DOI] [Google Scholar]
  32. Mvududu NH, & Sink CA (2013). Factor analysis in counseling research and practice. Counseling Outcome Research and Evaluation, 4(2), 75–98. 10.1177/2150137813494766 [DOI] [Google Scholar]
  33. Nájera Catalán H (2019). Reliability, population classification and weighting in multidimensional poverty measurement: A Monte Carlo study. Social Indicators Research, 142(3), 887–910. 10.1007/s11205-018-1950-z [DOI] [Google Scholar]
  34. Neukrug ES, Britton BS, & Crews RC (2013). Common health-related concerns of men: implications for counselors. Journal of Counseling & Development, 91(4), 390–397. 10.1002/j.1556-6676.2013.00109 [DOI] [Google Scholar]
  35. Neukrug ES, & Fawcett CR (2015). Essentials of testing and assessment: A practical guide for counselors, social workers, and psychologists (3rd ed.).Cengage. [Google Scholar]
  36. Omani-Samani R, Maroufizadeh S, Ghaheri A, & Navid B (2018). Generalized Anxiety Disorder-7 (GAD-7) in people with infertility: A reliability and validity study. Middle East Fertility Society Journal, 23(4), 446–449. 10.1016/j.mefs.2018.01.013 [DOI] [Google Scholar]
  37. Qualtrics Sample Services [Online sampling service service]. (2020). https://www.qualtrics.com/research-services/online-sample/
  38. Ritchie C, Andersen R, Eng J, Garrigues SK, Intinarelli G, Kao H, Kawahara S, Patel K, Sapiro L, Thibault A, Tunick E, & Barnes DE (2016). Implementation of an interdisciplinary, team-based complex care support health care model at an academic medical center: Impact on health care utilization and quality of life. PLoS ONE, 11(2), 1–14. 10.1371/journal.pone.0148096 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Schneiderman N, Llabre M, Cowie C, Barnhart J, Carnethon M, Gallo L, Giachello A, Heiss G, Kaplan R, LaVange L, Teng Y, Villa-Caballero L, & Avilés-Santa M (2014). Prevalence of diabetes among Hispanics/Latinos from diverse backgrounds: The Hispanic Community Health Study/Study of Latinos (HCHS/SOL). Diabetes Care, 37(8), 2233–2239. 10.2337/dc13-2939 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Schreiber JB, Nora A, Stage FK, Barlow EA, & King J (2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. Journal of Educational Research, 99(6), 323–337. 10.3200/JOER.99.6.323-338 [DOI] [Google Scholar]
  41. Seo J-G, & Park S-P (2015). Validation of the Generalized Anxiety Disorder-7 (GAD-7) and GAD-2 in patients with migraine. The Journal of Headache and Pain, 16(1), 97. 10.1186/s10194-015-0583-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Servan-Schreiber D (2009). Anticancer: A new way of life (3rd ed.). Viking Publishing. [Google Scholar]
  43. Sheehan D, Mauck D, Fennie K, Cyrus E, Maddox L, Lieb S, & Trepka M (2017). Black-White and country of birth disparities in retention in HIV care and viral suppression among Latinos with HIV in Florida. International Journal of Environmental Research and Public Health, 14(2), 120–132. 10.3390/ijerph14020120 [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Spitzer RL, Kroenke K, Williams JBW, & Löwe B (2006). A brief measure for assessing Generalized Anxiety Disorder: The GAD-7. Archives of Internal Medicine, 166(10), 1092. 10.1001/archinte.166.10.1092 [DOI] [PubMed] [Google Scholar]
  45. Swank J, & Mullen P (2017). Evaluating evidence for conceptually related constructs using bivariate correlations. Measurement and Evaluation in Counseling and Development, 50(4), 270–274. 10.1080/07481756.2017.1339562 [DOI] [Google Scholar]
  46. Trinh M, Agénor M, Austin S, & Jackson C (2017). Health and healthcare disparities among U.S. women and men at the intersection of sexual orientation and race/ethnicity: A nationally representative cross-sectional study. BMC Public Health, 17(1), 964. 10.1186/s12889-017-4937-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. United States Census Bureau. (2020). Quick facts: United States. https://www.census.gov/quickfacts/fact/table/US/PST045219
  48. United States Department of Health and Human Services: Centers for Disease Prevention and Health Promotion, Division of Adult and Community Health. (2000). Measuring healthy days: population assessment of health-related quality of life. Technical manual. CDC. https://www.cdc.gov/hrqol/pdfs/mhd.pdf [Google Scholar]
  49. Velten J, Bieda A, Scholten S, Wannemüller A, Margraf J, & Velten J (2018). Lifestyle choices and mental health: a longitudinal survey with German and Chinese students. BMC Public Health, 18(1), 632–632. 10.1186/s12889-018-5526-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Vogel M, Malcore S, Illes R, & Kirkpatrick H (2014). Integrated primary care: Why you should care and how to get started. Journal of Mental Health Counseling, 36(2), 130–144. 10.17744/mehc.36.2.5312041n10767k51 [DOI] [Google Scholar]
  51. Watson JC, Prosek EA, & Giordano AL (2020). Investigating psychometric properties of social media addiction measures among adolescents. Journal of Counseling & Development, 98(4), 458–466. 10.1002/jcad.12347 [DOI] [Google Scholar]
  52. Weston R, & Gore PA (2006). A brief guide to structural equation modeling. Counseling Psychologist, 34, 719–751. 10.1177/0011000006286345 [DOI] [Google Scholar]
  53. World Health Organization. (2018). Noncommunicable diseases and mental health: Global action plan for the prevention and control of NCDs 2013–2020. http://www.who.int/nmh/publications/ncd-action-plan/en/
  54. Yin S, Njai R, Barker L, Siegel P, & Liao Y (2016). Summarizing health-related quality of life (HRQOL): Development and testing of a one-factor model. Population Health Metrics, 14(1), 22. 10.1186/s12963-016-0091-3 [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES