Australian validation of the Universal Mental Health Literacy Scale for Adolescents

ABSTRACT

Objective

In order to support mental health help-seeking for Australian adolescents, it is imperative to understand and improve their mental health literacy (MHL). MHL measures are needed to identify MHL needs and to evaluate MHL interventions; however, a standardised MHL measure is yet to be validated for Australian adolescents. The current study aimed to validate the Universal Mental Health Literacy Scale for Adolescents (UMHL-A) with Australian adolescents.

Method

Participants were recruited through a national recruitment company and included 402 Australian youth, 15 to 18 years old. First, the factor structure of the UMHL-A was evaluated through confirmatory factor analysis. Second, reliability was assessed through internal consistency and item-total score reliability. Finally, convergent and divergent validity were appraised.

Results

Confirmatory factor analysis validated the structure of the UMHL-A with good fit indices for the four factors of help-seeking efficacy, stigma, knowledge of mental health, and knowledge of mental illness. Moreover, adequate reliability, convergent and divergent validity were demonstrated.

Conclusion

This evaluation of the UMHL-A enables theoretically grounded and psychometrically validated measurement of MHL with Australian adolescents. The UMHL-A has widespread application in research and practice, for clinicians, schools, mental health services and government policies.

KEY POINTS

What is already known about this topic:

1. The large proportion of mental health concerns emerge during adolescence; hence, it is imperative to promote mental health help-seeking with Australian youth.

2. Understanding mental health literacy (MHL) is central to facilitate mental health help-seeking for Australian adolescents.

3. Standardised, psychometrically sound and multi-dimensional measurement of MHL for adolescents is needed.

What this topic adds:

1. The validation of the Universal Mental Health Literacy Scale for Adolescents (UMHL-A) enables theoretically grounded, multidimensional and psychometrically sound measurement of MHL with Australian adolescents.

2. The UMHL-A promotes inclusive practice through short administration time, developmentally appropriate and non-diagnostically driven language, which is essential to consider within multi-cultural Australia.

3. The UMHL-A enables practitioners and services to accurately understand MHL needs, evaluate MHL interventions and reliably compare MHL across contexts and population groups.

Introduction

The World Health Organisation (2024) assert that health literacy is critical knowledge that empowers consumers to make informed decisions which promote and maintain their wellbeing. The conceptual framework of Mental Health Literacy (MHL) has evolved over recent years. While originally limited to the recognition and beliefs about mental illness (Jorm et al., 1997), MHL has now expanded to encompass four dimensions as described by Kutcher et al. (2016, p. 567):

(1) understanding how to obtain and maintain good mental health; (2) understanding mental disorders and their treatments; (3) decreasing stigma related to mental disorders; (4) enhancing help-seeking efficacy.

(knowing when, where, and how to obtain good mental health care and developing competencies needed for self-care)

It is widely accepted that MHL is a major influence on help-seeking (Jorm, 2012) and improving MHL is recognised as a key priority for Australian adolescents (Department of Health, 2019). From mid to late adolescence (15- to 18-year-olds), health-related behaviours are shaped as they become independent decision makers for their health and wellbeing (Rickwood et al., 2005). Hence, it is important to understand the MHL needs of Australian adolescents within this key developmental phase as they prepare for adulthood. To accurately understand adolescent MHL needs, psychometrically sound measurement of MHL is essential. MHL prevention and intervention programmes (henceforth referred to as MHL interventions) also rely upon accurate measurement of MHL to guide both programme development and standardised evaluation (Bale et al., 2018; Wei et al., 2016). However, there is currently no MHL measure validated for use with Australian adolescents.

Given the expansion of how MHL is defined, Spiker and Hammer (2018) proposed that MHL should be reconceptualised as a multi-construct theory. They asserted that recognising MHL theory enables more accurate measurement of the individual constructs, enriching understanding about the relationship between these dimensions, and their influence on the intended outcome; that is, improved help-seeking and management of mental health. Research has provided initial evidence for this expanded MHL theory by demonstrating associated relationships between MHL dimensions. In a meta-analysis Lien et al. (2024) investigated MHL and found that a decrease in stigma was associated with an increase in maintaining good mental health (positive MHL), and help-seeking efficacy. This quantitative synthesis also found that an increase in positive MHL was associated with improved help-seeking efficacy, and favourable help-seeking attitudes. Self-stigma has been found to mediate the relationship between perceived stigma and help-seeking attitudes (Vogel et al., 2007; Yu et al., 2022). In comparison to the more externalised perceived or public stigma, self-stigma is the internalised perception of mental health–related stigma which negatively impacts self-identity and self-esteem (Corrigan, 2004). Furthermore, help-seeking efficacy has been found to be a key mediator between the dimensions of MHL, for instance, the relationship between help-seeking attitudes and knowledge of mental illness was mediated by help-seeking efficacy (Lu et al., 2021). Moreover, knowledge of mental illness has been found to have a direct positive relationship with help-seeking efficacy and positive MHL (Lu et al., 2021; Nobre et al., 2022). Overall, the associations between the four dimensions of MHL demonstrate the importance of measuring them as individual constructs within MHL theory.

For adolescents, help-seeking intentions have been found to mediate the relationship between life stressors and life satisfaction (Sotardi et al., 2021). Life satisfaction is associated with subjective mental health (Lombardo et al., 2018) and it is an important secondary outcome measure across the mental health sector (Seligson et al., 2003). Moreover, life satisfaction is an integral component of wellbeing for adolescents, both as an outcome and predictor of mental wellness and psychosocial connections (Proctor et al., 2009). For example, improved well-being is significantly associated with positive MHL for adolescents (Bjornsen et al., 2019). MHL is important to consider within this broader perspective of wellbeing, although MHL and wellbeing have fundamental differences in application to prevention programmes (Wei et al., 2013). For instance, many well-being programmes fail to encompass all domains of MHL, such as knowledge about mental illness (Kutcher et al., 2016). Together, this highlights that whilst the experience of mental health in general impacts life satisfaction, MHL theory as a whole is not represented within life satisfaction and requires a specific approach to measurement.

Mental health literacy measurement in adolescents

There is a paucity of comprehensive, standardised MHL measures for adolescents, with current measures ranging in psychometric quality, offering limited breadth across the four dimensions of MHL and lacking cross-cultural validation (Bale et al., 2018; Kučera et al., 2023). Many measures of MHL knowledge are disorder-specific (Wei et al., 2016), assess a single dimension of MHL (Kučera et al., 2023), or assess MHL through the use of mental ill health–related vignettes with associated questions (Bale et al., 2018; O’Connor et al., 2014; Wei et al., 2015). These approaches to measurement limit the scope in which MHL is understood and fail to reflect contemporary MHL theory (Spiker & Hammer, 2018).

Systematic reviews have found that MHL measures vary in psychometric quality (Wei et al., 2016, 2017). For instance, Divin et al. (2018) identified that the psychometric evaluation of measures focused predominately on reliability, with a paucity of research that evaluated the validity of adolescent measures. Furthermore, well-researched MHL measures, such as the Mental Health Literacy Scale (MHLS; Neto et al., 2021; O’Connor & Casey, 2015) and the Mental Health Literacy Questionnaire – Short version for adults (MHLq-SVa; Campos et al., 2022) draw upon participant samples who are largely tertiary educated and young adults, rather than adolescents. MHL measures designed for adults are not developmentally specific or conceptually equivalent to understand adolescent MHL needs (Divin et al., 2018). The choice of outcome measurement tool for adolescents requires specific considerations due to their vulnerability to mental ill health and unique developmental needs (Bale et al., 2018; Wei et al., 2016). Together, this research highlights the need for psychometrically validated, developmentally and theoretically sound MHL measurement tools for use with adolescents.

Universal Mental Health Literacy Scale for Adolescents

To provide a multi-dimensional scale, Kågström et al. (2023) developed and validated the Universal Mental Health Literacy Scale for Adolescents (UMHL-A) with a Western Czech Republic adolescent sample aged 10 to 14 years and they found support for a four-factor structure. Kågström et al. (2023) described the dimensions of help-seeking efficacy and stigma related to mental health disorders as Attitudes within Likert scale UMHL-A. The dimensions of knowledge about how to obtain and maintain good mental health (KMH), and knowledge of mental health disorders and their treatment (KMI) were described as Knowledge within True/False (T/F) scale UMHL-A. This four-factor structure for the UMHL-A has been supported in cross-cultural validation studies with Turkish (Ciydem & Avci, 2024) and Chinese (Wang et al., 2024) adolescents, with good internal consistency and adequate test-retest reliability reported.

The UMHL-A offers multiple benefits as a MHL measure for adolescents. First, the UMHL-A is consistent with Kutcher et al. (2016) definition of MHL. The four-factor structure is grounded in MHL theory, and addresses concerns related to construct proliferation (Spiker & Hammer, 2018) which is of concern when MHL measurement is reduced to a single factor structure. For example, O’Connor and Casey (2015) acknowledge that whilst there is benefit that the MHLS is brief this can also cause some MHL dimensions to be under-represented. Second, a measure needs to have minimal burden on consumers through ease of administration with as few items as possible (Happell, 2008). Many existing MHL measures are lengthy, ranging from 33-items for MHLq (Campos et al., 2016), to 50-items in the Knowledge and Attitudes to Mental Health Scales (KAMHS; Simkiss et al., 2021). In comparison, the UMHL-A is efficient to administer with a total of 17 questions. Third, to promote inclusion, a MHL measure needs to have utility for adolescents varying literacy levels (Bale et al., 2018). As the literacy level of an adolescent cannot be assumed from chronological age, MHL measures need to have appropriate language that considers educational level (Dias et al., 2018). Whilst there are adult scales which report to have a year-7 reading level, such as the MHLS (O’Connor & Casey, 2015), other scales such as the mental health literacy scale for healthcare students (Chao et al., 2020), may not be appropriate for adolescents of a lower literacy ability. The UMHL-A was originally designed for younger adolescents (10–14 years old); hence, validating this scale with older adolescents (15 to 18 years old) promotes inclusive practice for a wide range of literacy levels.

Whilst several MHL include multiple dimensions of MHL, many utilise diagnostically driven language based on the Diagnostic and Statistical Manual of Mental Disorders (DSM-5-TR; American Psychiatric Association, 2022), such as the MHLq (Campos et al., 2016), MHLS (O’Connor & Casey, 2015) and the Adolescent Mental Health Literacy Questionnaire (AMHLQ; Zare et al., 2022). Whilst diagnostic-based questions play a valid role in assessing MHL, the use of this language also narrows our assessment of MHL to DSM-5-TR terms, and diagnosis is not always suitable to encompass the unique social and developmental considerations within adolescent mental health concerns. MHL measurement needs to reflect a holistic understanding of mental health, not just DSM-5-TR mental disorders (Mansfield et al., 2020). Furthermore, Australia is a multicultural country with varying perspectives around mental health; hence, a comprehensive measure, beyond DSM-5-TR disorder-specific criteria is needed. Yet there is a paucity of MHL measures which consider mental health in non-diagnostically driven language (Kučera et al., 2023). The UMHL-A offers a viable tool to address these issues, as the measure is largely based on non-diagnostically driven language which promotes inclusion across culturally diverse populations.

Overall, MHL measurement tools for adolescents need to be considered within the context of the populations needs (Wei et al., 2017), and a validated MHL measure is necessary for the Australian context. The UMHL-A has potential to address this need for standardised, developmentally appropriate and theoretically sound measurement of MHL in adolescents. Hence, the current study aims to validate the UMH-A for use with Australian adolescents. It is hypothesised that:

1. The four-factor structure of UMHL-A, found by Kågström et al. (2023) will be supported by confirmatory factor analysis.

2. The four subscales of the UMHL-A will demonstrate adequate internal consistency reliability.

3. Convergent validity will be demonstrated through the Likert UMHL-A subscales positive association with help-seeking attitudes and negative association with self-stigma. Further, the T/F UMHL-A subscales are expected to have a positive correlation with positive MHL.

4. Divergent validity will be demonstrated through small or non-significant association between UMHL-A and life satisfaction.

Method

Participants

A widespread sample of 402 Australian adolescents aged 15 to-18 years were recruited via Online Research Unit (ORU) during May 2024. All participants completed online surveys unsupervised, with surveys developed and distributed via Qualtrics XM survey software (Provo, UT). Demographic characteristics of participants were collected including gender, age, year level at school, cultural background, geographic location and previous service support. For each demographic question, participants were given the option to “prefer not to say”.

Procedure

This research was approved by the University of New England’s Human Research and Ethics Committee (approval number: HE24–008). The invitation to participate was available to parents/carers via a customised Qualtrics link through the ORU system, and inclusion criteria comprising Australian residence and having a child currently aged 15 to 18 years. Parents/carers who met these inclusion criteria read through the research information sheet and provided informed consent before passing the survey to their child. Each adolescent was also provided with age-appropriate research information and multiple-choice question to ensure comprehension, enabling them to provide informed consent. Mental health support line numbers and website resources were provided to parents/carers and participants on information sheets and at the end of the survey.

Initially, 433 parents/carers provided consent, but 11 young people declined participation. A further 20 survey responses were excluded from analysis due to invalid responses (Curran, 2016). The median survey completion time was nine minutes with response burden further reduced through ensuring that the survey was user-friendly for computer, tablet and mobile phone devices. Demographic questions were presented at the start of the survey, and all participants completed the measures in the same order, with notional credit provided through ORU for completed surveys. A total of 402 completed survey responses were included in the analysis, with no missing item responses.

Measures

Universal Mental Health Literacy Scale for Adolescents (UMHL-A)

The UMHL-A (Kågström et al., 2023) consists of 17 items over four subscale dimensions. The Likert UMHL-A, encompassing help-seeking efficacy and stigma, are measured on a 5-point Likert scale (ranging from 1 = strongly disagree to 5 = strongly agree) with factor loadings ranging from .389 to .611. The T/F UMHL-A, encompassing KMH and KMI, are measured through T/F responses and demonstrated factor loadings ranging from .446 to .648. All items contain an “I don’t know” response option scored as 3 for Likert UMHL-A and “false” for T/F UMHL-A (Kågström et al., 2023). Higher scores on the UMHL-A indicate higher MHL knowledge, lower stigmatized attitudes towards mental health and greater help-seeking efficacy. Scores for the UMHL-A subscale dimensions range from 5 to 25; 3 to 15; 0 to 5; and 0 to 4 for help-seeking efficacy, stigma, KMH and KMI respectively (Kågström et al., 2023) and permission to use the UMHL-A for this study was granted by lead author Kågström. The four-factor structure of the UMHL-A has been supported in cross-cultural samples and good internal consistency was demonstrated, with Cronbach alpha values ranging from 0.82 to 0.93 for Likert UMHL-A and 0.79 to 0.86 for T/F UMHL-A (Ciydem & Avci, 2024; Wang et al., 2024). Further, Ciydem and Avci (2024) showed sufficient item-total score reliability for each item across the UMHL-A (r = 0.48 to 0.86). Adequate and significant test-retest reliability has also been found with r = 0.76 to 0.998 across the four dimensions (Ciydem & Avci, 2024; Wang et al., 2024). Convergent validity of the UMHL-A has also been supported (Kågström et al., 2023; Wang et al., 2024). Analysis of the psychometric properties of the UMHL-A with Australian adolescents is the focus of this study and is presented in the results below.

Mental Health Promoting Knowledge (MHPK-10)

The MHPK-10 (Bjornsen et al., 2017) was utilized to measure convergent validity in recognition of the importance of positive MHL for adolescents. The MHPK-10 measures positive MHL, encompassing competence, autonomy and relatedness (Bjornsen et al., 2017). The MHPK-10 contains 10 items measured on a 5-point Likert scale (ranging from 1 = completely wrong to 5 = completely correct), where high scores indicate greater knowledge of how to take care of your mental health. The MHPK-10 has been validated with Norwegian young people, 15 to 21 years old, and it has demonstrated good internal consistency (ω = .84), test-retest reliability (r = .74) (Bjornsen et al., 2017) and cross-cultural validity (Liu et al., 2023). The internal consistency of the MHPK-10 for the current study was ω = .82.

Attitudes Towards Seeking Professional Psychological Help – Short Form (ATSPPH-SF)

The ATSPPH-SF (Fischer & Farina, 1995) is a widely used measure of attitudes towards mental health help-seeking. Wang et al. (2024) utilised the 29-item ATSPPH to assess convergent validity of the UMHL-A; however, the shortened version was employed in the current study to increase accessibility with the adolescent sample. The ATSPPH-SF contains 10-items measured on a 4-point Likert scale (ranging from 1 = disagree and 4 = agree), where high scores indicate more positive attitudes to help-seeking. The ATSPPH-SF demonstrated adequate internal consistency (α = 0.84), test-retest reliability (r(32) = 0.8, p < .05) (Fischer & Farina, 1995) and cross-cultural validity (Picco et al., 2016; Torres et al., 2021). The internal consistency of the ATSPPH-SF for the current study was α = .76.

Self-Stigma of Seeking Help (SSOSH)

The SSOSH (Vogel et al., 2006) measures self-stigma related to seeking professional mental health support, and it will be used to measure convergent validity in the current study. Stigma is widely accepted as a major barrier to mental health help-seeking (Vogel et al., 2007) and the SSOSH has previously been used to measure UMHL-A convergent validity (Wang et al., 2024). The SSOSH contains 10-items measured on a 5-point Likert scale (ranging from 1 = strongly disagree to 5 = strongly agree), with high scores indicating a greater degree that negative concerns related to professional help-seeking are internalised as personal attributes and self-worth (Vogel et al., 2006). The SSOSH has demonstrated good internal consistency (α = .91), test-retest reliability and evidence for construct, criterion, discriminative and predictive validity (Vogel et al., 2006). The internal consistency of the SSOSH for the current study was α = .84.

Brief Multidimensional Students’ Life Satisfaction Scale (BMSLSS)

Life Satisfaction is a key indicator for wellbeing in youth (Proctor et al., 2009). However, there are distinct differences between the construct of life satisfaction and MHL (Kutcher et al., 2016; Wei et al., 2013); hence, life satisfaction will be utilised to assess divergent validity of the UMHL-A. The BMSLSS (Seligson et al., 2003) investigates satisfaction across five aspects of life, including family, friendships, schools, yourself, homelife, and provides an overall life satisfaction score. The five-items of the BMSLSS are measured on a 5-point Likert scale (ranging from 1 = very dissatisfied and 5 = very satisfied), with higher scores indicating greater satisfaction with life. The BMSLSS has demonstrated adequate internal consistency (α = .75), strong criterion-related validity, construct validity, convergent and discriminant validity (Seligson et al., 2003). Further, the BMSLSS has been validated with adolescents across more than 23 countries with consistent psychometrically sound properties confirmed (Abubakar et al., 2015; Costa et al., 2022; Pittman et al., 2021). The internal consistency of BMSLSS for the current study was α = .89.

Analysis

Descriptive statistics regarding demographic characteristics of participants were reported. Key demographics, including gender and previous contact with a mental health professional, were analysed through a Mann-Whitney U test due to the ordinal scale type of UMHL-A.

The analysis for this research was guided by best practice guidelines for scale evaluation (Boateng et al., 2018), and dimensionality, reliability and validity were assessed. As the UMHL-A has a theoretically driven and empirically tested factor structure (Ciydem & Avci, 2024; Kågström et al., 2023; Wang et al., 2024), a confirmatory factor analysis (CFA) was conducted to test the first hypothesis, using Jamovi (The Jamovi Project, 2023). The sample size exceeded the recommended minimum of 300 participants for factor analysis (Tabachnick & Fidell, 2019), and two separate CFAs were performed due to the difference in response types between the Likert UMHL-A and T/F UMHL-A. Two CFAs also allowed for a more detailed understanding of each factor and effective management of potential model fit concerns (Roos & Bauldry, 2022). The first CFA, for Likert UMHL-A, consisted of help-seeking efficacy and stigma subscale dimensions. The second CFA was for T/F UMHL-A and consisted of KMH and KMI subscale dimensions.

For both CFAs, the Robust Weighted Least Squares (WLSM) estimator with standard errors was employed to provide an unbiased estimate for ordinal response types (Flora & Curran, 2004; Li, 2016). CFA model fit was evaluated using the chi-square test (χ²), comparative fit index (CFI), Tucker-Lewis index (TLI), root-mean-square error of approximation (RMSEA), and standard root-mean-square residual (SRMR). The following fit criteria was used: CFI & TLI > .95 and RMSEA < .60 for good fit, SRMR < .08 for acceptable fit with values closer to zero a better fit (Boateng et al., 2018).

Additional analyses were conducted using the program Statistical Package for the Social Sciences (SPSS; IBM Corp, 2023). The second hypothesis was assessed through internal consistency and item-total score reliability. Internal consistency was estimated for each of the four factors within UMHL-A using the McDonalds omega coefficient (ω), which is a preferred reliability coefficient for ordinal scales (Kalkbrenner, 2021). The third and fourth hypotheses, assessing convergent and divergent validity, were evaluated through a series of bivariate correlations using Spearman Rho, which is appropriate for ordinal data sets (Choi et al., 2010). Given the four-factor structure of the UMHL-A, convergent validity of the subscales was measured against conceptually appropriate scales. Convergent validity for help-seeking efficacy and stigma dimensions were compared with ATSPPH-SF and SSOSH. Convergent validity for KMH and KMI were compared with MHPK-10. Divergent Validity was measured by assessing the correlation between BMSLSS and all four factors of the UMHL-A.

Results

Participants demographic characteristics are presented in Table 1. Participants ranged in age from 15 to 18 years old (M = 16.4, SD = 1.03) with a comparable proportion who identified as male or female. The majority of participants (71%) were in Years 11 and 12 at school. While the majority of participants identified as Non-Indigenous Australians, the sample did include Indigenous Australians and other cultural groups, for example, Indian, Asian, Chinese. Most participants resided in NSW and Victoria, although each state and territory in Australia was represented. Close to 20% of participants were from a regional or remote area with the majority based in a major city as defined by the accessibility/remoteness index of Australia (ARIA+; Australian Bureau of Statistics, 2023).

Table 1.

Participant demographics.

Demographic	n	%
Gender
Males	209	52
Female	184	46
Non-Binarya	8	2
Age (Years)
15	100	25
16	102	25
17	136	34
18	64	16
High School Grade
Year 9	25	6
Year 10	84	21
Year 11	113	28
Year 12a	153	38
Cultural Background
Non-Indigenous Australian	346	86
Indigenous Australianb	7	2
Another Cultural Groupa	38	9
Australian State of Residence
New South Wales	121	30
Queensland	66	16
Victoria	110	27
South Australia	38	9
Western Australia	46	11
Tasmania	11	3
Northern Territory	3	1
Australian Capital Territory	7	2
Ruralityc
Major City	328	81
Regional	71	18
Remote	3	1
Service Support Needs
Previous contact with either school counsellor or psychologist	206	51
Multiple Service Needsd	76	19
Preferred not to say	118	30

Note. a: 1 participant preferred not to disclose gender; 27 participants preferred not to disclose school grade; 11 participants preferred not to disclose cultural background. B: Indigenous Australian is used to respectfully refer to Aboriginal and Torres Strait Islander’s, the First Nations People of Australia. c: rurality is according to Australian Statistical Geographical Standard (ASGS), which is based on Accessibility/Remoteness Index of Australia (ARIA+) defining rurality based on service availability (Australian Bureau of Statistics, 2023) broken down into major city, regional (both inner regional and outer regional combined), and remote (both remote and very remote combined). d: Multiple service needs included service needs beyond mental health concerns, for example, police contact, disability support, employment services or youth programmes.

A more in-depth analysis regarding differences in UMHL-A scores for key demographics was also carried out. First, Mann-Whitney U test indicated no significant difference between male (n = 209) and female (n = 184) genders with p > .05 for each subscale of UMHL-A (Table 2). Second, Mann-Whitney U test indicated no significant difference between participants who had previous contact with a mental health professional (n = 206) and those who had none (n = 78) with p > .05 for each subscale of the UMHL-A (Table 2).

Table 2.

Analysis of gender and previous contact with a mental health professional.

	Males	Females	Previous Contacta	No Contacta
n	209	184	206	78
UMHL-A Help-Seeking Efficacy
U	18299.50		7144.00
z	−.83		−1.45
p	.41		.15
UMHL-A Stigma
U	18087.00		7969.50
z	−1.03		−.11
p	.30		.92
UMHL-A KMH
U	17926.00		8028.50
z	−1.30		−.01
p	.20		.99
UMHL-A KMI
U	18256.50		7547.50
z	−1.10		−1.01
p	.27		.32

Note. a. previous contact = previous contact with a mental health professional; no contact = no previous contact with a mental health professional reported. p 2-tailed significance at .05.

The means and standard deviations of the UMHL-A subscale dimensions are presented in Table 3. Further, examination of descriptive statistics and visual inspection of histograms and Q-Q plots (Tabachnick & Fidell, 2019) revealed sufficient variability across all individual items on the Likert UMHL-A and no significant floor or ceiling effects. The T/F UMHL-A demonstrated a non-normal distribution, with moderate negative skewness, and as such non-parametric tests were used, including the estimator WLSM as it is robust against non-normal latent response distributions (Flora & Curran, 2004).

Table 3.

UMHL-A mean and SD.

Scale	M	SD
Likert UMHL-A
Help-seeking	19.16	3.44
Stigma	12.67	1.92
T/F UMHL-A
Knowledge of Mental Health (KMH)	4.16	1.22
Knowledge of Mental Illness (KMI)	3.56	.83

Note. M = mean of total subscale score and SD = standard deviation.

Hypothesis 1: factor structure

As hypothesised, the four-factor structure of UMHL-A proposed by Kågström et al. (2023) was supported and the results for each CFA are outlined below.

Likert UMHL-A

The two-factor model for Likert UMHL-A was confirmed with good model fit indices as shown in Table 4. To achieve this goodness of fit, the substantial residual covariance between specific questions was controlled for, based on modification indices. This included question 2a and 6a on help-seeking efficacy subscale; and 1a on help-seeking efficacy and 3a on stigma subscale. A visual representation of the factor structure for Likert UMHL-A is shown in Figure 1.

Table 4.

Goodness of fit indices Likert UMHL-A.

	n	X2	df	p	CFI	TLI	RMSEA	SRMR
Likert UMHL-A	402	40.5	17	.005	.992	.986	.059	.048

Figure 1.

CFA for Likert UMHL-A.

Questions and corresponding factor loadings are shown in Table 5 with standardised factor loadings ranged from .508 to .773 for help-seeking efficacy and .476 to .850 for stigma. All standardised factor loadings were statistically significant (p < .001) indicating significant association with the respective latent factors. Further, all factor loadings were above .4 demonstrating adequate loading of each item onto the respective factor (Matsunaga, 2010). The covariance between latent factors help-seeking efficacy and stigma indicated a statistically significant, strong association (r [402] = .76, p < .001).

Table 5.

Questions and factors loadings for Likert UMHL-A.

Question	Factor Loadings
Help-Seeking Efficacy
1a. Talking about my feelings with someone helps to improve mental health.	.669
2a. I am comfortable talking to my peers about my feelings.	.508
4a. If I experienced mental health problems, I would seek help.	.773
6a. I am comfortable talking to adults in my life about my feelings	.695
7a. If I had a mental disorder, I would speak about it with others.	.702
Stigma
3a. How I get along with others affects my mental health.	.476
5a. If someone I care about had mental health problems for a long time, I would encourage them to get professional help.	.850
8a. I would be willing to continue a friendship with someone who developed a mental health problem.	.673

True/False UMHL-A

The two-factor model for T/F UMHL-A was confirmed with good model fit indices as shown in Table 6. A visual representation of factor structure for T/F UMHL-A is shown in Figure 2.

Table 6.

Goodness of fit indices T/F UMHL-A.

	n	X2	df	p	CFI	TLI	RMSEA	SRMR
T/F UMHL-A	402	37.4	26	.069	.993	.991	.033	.079

Figure 2.

CFA for T/F UMHL-A.

Questions and corresponding factor loadings T/F UMHL-A are shown in Table 7. The standardised factor loadings ranged from .614 to .862 for KMH and .654 to .974 for KMI were all statistically significant at p < .001 and all factor loadings exceed the recommended factor loading threshold of .4 (Matsunaga, 2010). The covariance between latent factors KMH and KMI indicated a statistically significant, strong association (r [402] = .77, p < .001).

Table 7.

Questions and factors loadings for T/F UMHL-A.

Question	Factor Loadings
Knowledge of mental health (KMH)
1b. Getting along with others is important for mental health	.614
6b. The way people commonly feel is a sign of their mental health	.773
7b. How people think about things affects their mental health	.846
8b. How people get along with others affects how they feel	.862
9b. How people think about things affects how they feel	.827
Knowledge of mental illness (KMI)
2b. Mental illnesses are caused by different things	.808
3b. Mental health impacts people’s behaviour	.821
4b. Mental disorders affect people’s emotions	.974
5b. Depression is one of the most common mental illnesses among young people	.654

Hypothesis 2: reliability

The second hypothesis predicted that the UMHL-A would demonstrate adequate reliability across subscale dimensions, assessed through internal consistency and item-total correlations. The internal consistency for Likert UMHL-A, T/F UMHL-A, and subscale dimensions are shown in Table 8. For attitudinal scales acceptable internal consistency is recommended to be ω ≥ .65 to .80 (Kalkbrenner, 2021). The UMHL-A reliability ranged from .67 to .80 demonstrating adequate internal consistency and provided support for the second hypothesis. A more detailed analysis of reliability showed that removal of an item within UMHL-A help-seeking efficacy, stigma and KMH would not improve the subscale reliability. In relation to KMI, internal consistency increased from .68 to .72 with the removal of item five which stated “depression is one of the most common mental illnesses among young people”.

Table 8.

UMHL-A internal consistency.

	Likert UMHL-A	Help-Seeking Efficacy	Stigma	T/F UMHL-A	KMH	KMI
ω	.80	.78	.67	.78	.72	.68

Item-total score correlations were employed to analyse the item reliability for UMHL-A. Item-total reliability assesses whether each item on a scale is independent and comparable weight contribution to the latent factor (Ferketich, 1991). Ferketich (1991) recommended that item-total correlation coefficients greater than r = .3 indicate adequate reliability. In the current study all items exhibit an item-total correlation coefficient greater than .3 (Table 9). This indicates that all items on the UMHL-A significantly and independently contribute to their respective factor. Together, this demonstrated acceptable reliability for each item on the UMHL-A and supported hypothesis two.

Table 9.

UMHL-A item-total reliability.

Items	r
Likert UMHL-A
1	.448
2	.492
3	.405
4	.594
5	.620
6	.574
7	.476
8	.521
T/F UMHL-A
1	.332
2	.446
3	.417
4	.510
5	.320
6	.437
7	.562
8	.517
9	.496

Hypothesis 3: convergent validity

Convergent validity was assessed through Spearman Rho correlations and interpreted in accordance with Cohen (2013) conventions. First, the third hypothesis proposed that help-seeking efficacy and stigma would be positively correlated with help-seeking attitudes and negatively correlated with self-stigma, as measured by ATSPPH-SF and SSOSH, respectively. Table 10 shows that help-seeking efficacy and stigma demonstrated medium positive correlations with ATSPPH-SF and medium negative correlations with SSOSH. Second, it was proposed that KMH and KMI would be positively correlated with positive MHL, measured by MHPK-10. Table 10 shows that MHPK-10 has a medium positive correlation with KMH and small positive correlation with KMI. Together, these significant correlations support hypothesis three and demonstrate convergent validity of the UMHL-A.

Table 10.

Correlations of UMHL-A with existing measures.

	Help-Seeking Efficacy	Stigma	KMHa	KMIa	MHPK-10a	ATSPPH-SFa	SSOSHa
Stigma	.46**
KMHa	.19**	.21**
KMIa	.10	.15**	.46**
MHPK-10a	.43**	.33**	.37**	.26**
ATSPPH-SFa	.34**	.37**	.20**	.21**	.32**
SSOSHa	−.35**	−.39**	−.15**	−.19**	−.35**	−.61**
BMSLSS	.49**	.25**	.17**	.07	.44**	.16**	−.28**

Note. a KMH = knowledge of mental health; KMI = knowledge of mental illness; MHPK-10 = mental health promoting knowledge scale; ATSPPH-SF = attitudes towards seeking professional psychological help scale – short form; SSOSH = self-stigma of seeking help scale; BMSLSS = Brief Multidimensional Students’ Life Satisfaction Scale. ** Correlation is significant at the 0.01 level (2-tailed).

Hypothesis 4: divergent validity

The fourth hypothesis proposed that divergent validity would be demonstrated through small or non-significant associations between UMHL-A and life satisfaction. First, BMSLSS showed a small positive correlation with stigma and KMH, and no significant correlation KMI (Table 10). Surprisingly help-seeking efficacy and BMSLSS showed a strong positive correlation. Fisher r-to-z transformation was applied to further test convergent and divergent validity. Table 11 shows a significant difference in convergent and divergent correlations for help-seeking efficacy and stigma. This suggests that the Likert UMHL-A subscale dimensions differed significantly from the convergent measures of ATSPPH-SF and SSOSH compared to the divergent measure of BMSLSS. In relation to the T/F UMHL-A Table 11 shows a significant difference in convergent and divergent correlations for KMH and KMI. This suggests that the KMH and KMI differed significantly on the convergent measure of MHPK-10 compared to divergent measure of BMSLSS. Together, these findings support hypothesis four and provide evidence for the divergent validity of the UMHL-A.

Table 11.

Fisher r-to-z transformations.

Comparison of Convergent and Divergent Correlations	z
Likert UMHL-A
Help-Seeking Efficacy/ATSPPH-SF and Help-Seeking Efficacy/BMSLSS	−2.57**
Help-Seeking Efficacy/SSOSH and Help-Seeking Efficacy/BMSLSS	−12.73**
Stigma/ATSPPH-SF and Stigma/BMSLSS	1.88*
Stigma/SSOSH and Stigma/BMSLSS	−9.424**
T/F UMHL-A
KMH/MHPK-10 and KMH/BMSLSS	3.06**
KMI/MHPK-10 and KMI/BMSLSS	2.77**

Note * significant p < 0.05; **significant p < 0.01.

Discussion

This research aimed to validate the UMHL-A for use with Australian adolescents, aged 15 to 18 years old. It was hypothesised that the four-factor structure of the UMHL-A, found by Kågström et al. (2023) would be upheld and that the UMHL-A would demonstrate adequate reliability, convergent and divergent validity. The results supported all four hypotheses.

The UMHL-A demonstrated excellent goodness-of-fit indices, supporting a four-factor structure and supporting hypothesis one. This finding is consistent with cross-cultural validation papers for the UMHL-A (Ciydem & Avci, 2024; Kågström et al., 2023; Wang et al., 2024). Further, the four-factor structure of the UMHL-A is consistent with the theoretical underpinnings of MHL as proposed by Kutcher et al. (2016), conceptualising MHL as a multi-dimensional construct. Consistent with this definition and MHL theory, the dimensions within MHL are best measured independently as opposed to a singular outcome which risks construct proliferation (Spiker & Hammer, 2018). The current study found support for the four MHL dimensions independently, suggesting that the UMHL-A may adequately assess a multi-dimensional representation of MHL for Australian adolescents.

The non-significant difference between male and female genders on the UMHL-A is inconsistent with existing literature, where it is generally accepted that females have higher MHL in comparison to their male counterparts (Furnham & Swami, 2018). However, there are key differences in methodology, including how MHL is measured, which need to be considered when interpreting gender differences in MHL. For instance, research where females have significantly higher levels of MHL in comparison to males have relied largely on vignettes and unstandardised measurement tools (Burns & Rapee, 2006; Chandra & Minkovitz, 2006; Cotton et al., 2006; Reavley et al., 2014). In comparison, research that has found no difference in MHL in relation to gender has used standardised assessment tools (Goodfellow et al., 2023; Ozbicakci & Salkim, 2024; Ratnayake & Hyde, 2019). This highlights that whilst gender differences are likely to exist it is important to understand MHL in respect to context (Clark et al., 2025), and through the utility of standardised assessment tools which allow for reliable comparison between population groups.

In support of the second hypothesis, the results found the UMHL-A to have adequate reliability. More specifically item five on KMI showed that if removed internal consistency of this subscale would improve; however, item five also demonstrated adequate item-total score correlation and factor loading, and removal would reduce the number of items in KMI to the minimum of three items. Taking these analyses together, item five within KMI was retained. Overall, reliability analyses provide evidence that each item across the UMHL-A independently contributes to that subscale dimension and that the UMHL-A reliability measures each dimension of MHL.

Convergent and divergent validity of the UMHL-A was demonstrated, supporting the third and fourth hypotheses. First, help-seeking efficacy, stigma, KMH and KMI showed significant expected associations with established measures of similar constructs. Whilst these significant correlations were small to medium in magnitude, this is consistent with existing literature and reflect the theoretical foundation of both direct and indirect associations between the dimensions of MHL theory (Lien et al., 2024; Lu et al., 2021; Yu et al., 2022). Second, it is notable that help-seeking efficacy showed a strong positive correlation with life satisfaction. For adolescents, increased life satisfaction is consistently associated with enhanced self-efficacy (Proctor et al., 2009), which can be likened to the MHL domain help-seeking efficacy. Overall, the differences between convergent and divergent correlations across UMHL-A were statistically significant. These similarities and unique differences suggest that the UMHL-A is able to discriminate between similar and opposing constructs, establishing convergent and divergent validity.

Clinical implications

This validation of UMHL-A has important implications for clinicians, schools and mental health services. First, the UMHL-A provides a valuable and standardised tool that can be used by clinicians to better understand their clients MHL. Understanding and assessing the MHL needs of individual adolescent clients can foster dialogue that enables help-seeking barriers to be addressed and ongoing engagement with professional mental health services enhanced. For example, if KMI on the UMHL-A is relatively low in comparison to other MHL dimensions then further relevant psychoeducation can be incorporated into the client’s treatment plan. Additionally, the UMHL-A is a practical tool that can be used to understand MHL for adolescents who are hesitant to engage with mental health services, enabling clinicians to tailor their engagement approach to better meet consumer needs. For example, if stigmatising attitudes on the UMHL-A was a considerable area of concern, then conversation around relevant support can be offered to engage that young person and foster help-seeking. Being flexible in our approach to understand and respond to nuanced MHL needs is important to foster help-seeking in adolescents who are hesitant to engage with mental health services, like young men who have disengaged from education (Clark et al., 2024). Second, this validation of the UMHL-A offers the ability to evaluate and modify MHL interventions for Australian adolescents. For example, MHL interventions can be appropriately modified when the UMHL-A shows that one dimension, like stigma, is of higher concern in comparison to another dimension, like KMI. Understanding nuanced differences within and between population groups is important to be able to tailor MHL interventions to population needs and improve programme effectiveness (Bennett et al., 2023; Clark et al., 2025). Hence, the validation of UMHL-A in this study empowers clinicians, mental health services and schools to tailor their approach according to the four dimensions of MHL.

Unstandardised questions specifically designed for individual MHL interventions or research projects dominate the literature (Wei et al., 2013, 2015). The results of this study support the application of UMHL-A to enable MHL to be reliably compared between Australian adolescent population groups and across time points. This measure provides utility in the evaluation of MHL interventions and clarity of adolescent MHL needs in schools and mental health services. Additionally, UMHL-A enables researchers to utilize a standardized approach to measurement that creates a consistent dialogue for understanding MHL in adolescents across research projects and multiple timepoints. Finally, the UMHL-A provides a valid assessment tool that delineates between MHL and wellbeing outcomes, accurately informing government and service policies of Australian adolescent MHL needs.

Overall, this research provides important evidence for the UMHL-A as a psychometrically sound measurement tool with Australian adolescents. The UMHL-A promotes inclusive practice through its concise number of items with developmentally, non-diagnostically driven language, which is imperative for multicultural Australia. Further, the UMHL-A meets COSMIN criteria for quality psychometric measurement (Prinsen et al., 2018) suggesting that the UMHL-A can be recommended as a suitable measure of MHL with Australian adolescents in National MHL Surveys, and to evaluate National mental health prevention campaigns.

Limitations and future research

There are several limitations from the current study to consider in future research. The first limitation to acknowledge is that the T/F UMHL-A showed a negatively skewed distribution, with the average score for KMH and KMI subscale dimensions towards the higher end of MHL knowledge. The skewness in the current study was not evident in previous UMHL-A validation research (Ciydem & Avci, 2024; Kågström et al., 2023; Wang et al., 2024). Additionally, there was no significant difference found between participants who had previous contact with a mental health professional and those who reported no previous contact. In comparison, previous studies have shown that past positive counselling experience is associated with less stigma and greater willingness to disclose personal information to seek professional mental health support (Ciarrochi et al., 2002; Coates et al., 2019; Mills et al., 2012). The findings from the current study could be explained by the influence of the recruitment strategy through ORU as participants had pre-existing interest in completing research surveys and general improvement in MHL in recent years (Jorm et al., 2006). For example, the number of mental health prevention campaigns in Australia have increased exponentially and MHL interventions, such as Youth Mental Health First Aid and Teen Mental Health First Aid in Australia, have demonstrated effectiveness in improving adolescent knowledge of mental illness and confidence to seek help (Ng et al., 2021). Together, this suggests that these results may reflect a trend towards overall good mental health knowledge within the participant sample. Future research should investigate whether these results are replicated in other population groups with a varying amount of experience with mental health professionals, differences in geographic locations and socio-cultural backgrounds.

The UMHL-A may be advantageous across cultural groups as it does not rely upon a diagnostic interpretation of mental health. However, different cultural groups have varying perspectives in relation to MHL (Clark et al., 2025; Gonzalez et al., 2005) and these nuanced perspectives between and within cultural groups need to be respected and central to understanding MHL (Clark et al., 2025; Vance et al., 2017). For example, for Indigenous Australians the Social and Emotional Wellbeing (SEWB) framework (Gee et al., 2014) is a culturally relevant perspective of mental health. Future research should consider how the SEWB framework may be incorporated into understandings of MHL for Indigenous Australian youth.

The assessment of psychometrically sound measurement is a recurrent process where validity and reliability are tested over different contexts and population groups (Wei et al., 2016). Future research should consider exploratory factor analysis for UMHL-A with diverse population groups to explore the cross-cultural validity of this four-factor structure. Second, testing for measurement invariance of UMHL-A through multi-group CFA which includes diverse groups of adolescents is important for the validity of this measure. Finally, longitudinal assessment related to the predictive validity and test-retest reliability of the UHML-A is needed.

Conclusion

The validation of the UMHL-A in this study facilitates psychometrically sound measurement of MHL with Australian adolescents. The UMHL-A has important application across research, mental health services, schools and government policies. First, accurate and theoretically ground guidance to tailor MHL intervention development and service delivery based on the four dimensions of MHL. Second, reliable and valid evaluation of adolescent MHL needs and MHL interventions to accurately compare MHL across context and population groups. Overall, this research enables the wide, practical application of the UMHL-A with Australian adolescents to understand MHL needs.

Funding Statement

This work was supported by the Australian Government Research Training Program Stipend and Fee Offset Scholarship.

Disclosure statement

The authors have no conflict of interests to declare. The data supporting the findings of this research are openly available in https://rune.une.edu.au/web/index.jsp.

Author contributions

• Julie C. Clark conceived and designed this quantitative research, supported data collection through ORU, completed analysis, authored the manuscript including preparation of tables and figures, approved the final draft.

• Warren Bartik conceived and designed the quantitative research, supported analysis, authored and reviewed the manuscript including tables and figures, approved the final draft.

• Kylie Rice conceived and designed the quantitative research, supported analysis, authored and reviewed the manuscript including tables and figures, approved the final draft.