The validity of the Alcohol Use Disorders Identification Test (AUDIT) among Australian nurses

Abstract

Introduction

The Alcohol Use Disorders Identification Test (AUDIT) has been used in various global settings as a rapid, reliable screening instrument to detect risky and hazardous alcohol consumption. However, there remain populations where the AUDIT has not been validated. Nurses make up a substantial proportion of healthcare workers globally, and their experiences during the recent pandemic response have indicated that risky and hazardous alcohol consumption has occurred among this occupational group. The objective of this study was to validate the AUDIT amongst a cohort of nurses.

Methods

This paper uses a dataset of Australian nurses (n = 1,159) who completed the AUDIT as part of a nationwide survey on alcohol consumption conducted between July and October 2021. A three-step factor analysis method was used to determine the validity and reliability of the AUDIT as a screen for risky and hazardous alcohol consumption among Australian nurses.

Results

Initial confirmatory factor analysis found poor performance on items 5, 6 and 9 of the AUDIT. Further exploratory factor analysis confirmed these results, additionally finding other items (1 and 10) that contributed marginally to the AUDIT performance among nurses. A final Goodness-of-Fit test on the remaining five items in the AUDIT was significant (= 59.871(5), p < 0.001), suggesting that a five-item AUDIT scale is reliable amongst nurses.

Conclusion

The factor analysis process confirmed that that original 10-item AUDIT was not valid and reliable to screen for risky and hazardous alcohol consumption among nurses in Australian settings. A modified, five item version of the AUDIT tool (the AUDIT-N) was a substantially better fit for use in a cohort of Australian nurses, although further testing for construct validity is required prior to deployment. Our findings have applicability for the use of the AUDIT in future workforce surveys of alcohol consumption among not only nurses, but the wider population of healthcare workers.

Highlights

The AUDIT has been validated in several settings, and with various populations.

However, the AUDIT has not been tested for validity and reliability among nurses.

Exploratory and confirmatory factor analysis conducted with this sample found the 3-factor 10-item AUDIT was not valid and reliable among nurses.

We recommend further testing of a specific modification (the AUDIT-N) for this population.

Introduction

Alcohol remains the most widely used drug worldwide, and risky or hazardous consumption of alcohol represents an extensive burden: both financial and societal [1–3]. The hazardous use of alcohol is responsible for several deleterious effects to health, including cancers and cardiovascular disease [4–7], and is attributed to several motor vehicle accidents and violent assaults worldwide [8–12]. In Australia alone, alcohol accounted for 80,400 hospitalisations and 4.1% of the total burden of disease in the 2022-23 reporting period, remains a leading risk factor for death and disease burden in males aged 15–44, and is a substantial risk factor for injury and chronic health conditions [13]. As an example of the direct and indirect costs of hazardous alcohol use, it is estimated that the cost to the Australian economy is AUD15 billion per annum [14], and according to the latest available data, the cost to the United States economy was USD249 billion per annum in 2010 [15].

To assist in screening of risky and hazardous alcohol use, the Alcohol Use Disorders Identification Test (AUDIT) was developed as a World Health Organisation collaborative project spanning six countries. It was initially validated among 1,888 individuals attending primary health care facilities [16] and thus, the initial intended use of the AUDIT tool was to detect adult patients with risky or hazardous alcohol consumption patterns. Since its development, the AUDIT has been validated in several populations and languages, for example among medical settings [17], in primary care [18, 19], and among individuals with serious mental illness [20]. The AUDIT tool’s psychometric properties have been explored in populations other than healthcare consumers, including college and university students [21, 22], adolescents [23, 24], twins and their families [25], carers of children [26], and pregnant women [27]. These studies reported widely varied validity and reliability, indicating the need for re-validation when using this tool outside of its intended population.

Versions of the AUDIT have also been used among cohorts of healthcare workers to examine risky and hazardous alcohol consumption. For example, Albano et al. [28] used the AUDIT-C to explore alcohol consumption among healthcare workers (physicians, nurses and ‘others’) in Italy (n = 639). Cedrone et al. (2022) repeated a similar study design among 1,745 healthcare and office workers during the COVID-19 pandemic [29]. Recently, the AUDIT tool has also been used to report alcohol consumption among Australian nurses including those working closely with healthcare consumers using alcohol and other drug (AOD) services [30].

Despite these studies, the validity of the AUDIT tool for use among healthcare professionals is lacking, and the interpretation of the AUDIT score based on such use may be limited. More specifically, according to the American Educational Research Association, the American Psychological Association and the National Council on Measurement in Education (2014), the use of the tool in a different population or condition for the purpose that differs from the validated population or condition may affect the validity and interpretation of the tool, with the suggestion that users are responsible for reviewing the validity of the tool in these circumstances, and if necessary, to conduct local validation. As outlined previously, the AUDIT has been validated among the general public, particularly in primary and tertiary healthcare settings [31, 32], but has not undergone validation among a population comprising only nurses. Therefore, this study has been conducted to validate the use of the AUDIT tool as an alcohol screening tool among Australian nurses. This is an important step to enable our future investigation of alcohol assumptions by nurses to assist workforce strategy and policy design, and future exploration and potential intervention in cases where nurses are consuming alcohol at risky or hazardous levels.

Aim

To validate the use of the AUDIT tool for alcohol screening among nurses practising in Australia.

Methods

Participants and data

This study used the existing dataset collected by Searby and colleagues [30]. The original study aimed to determine the prevalence of high-risk alcohol consumption in Australian nurses, and to explore potential correlations between workplace stress and alcohol consumption. To achieve this aim, nurses across Australia were surveyed from a wide variety of settings and clinical specialties. The surveyed nurses self-rated their alcohol consumption across the full, ten-item AUDIT tool, returning a range of scores that indicated a wide variety of drinking patterns, from abstinence to risky or dependent alcohol consumption (AUDIT score range 0–34). The dataset included responses from 1,159 Australian nurses working in 35 acute and primary health care settings [for further information about the participants, see Searby et al. [30]). After removing three cases due to substantial missing data related to the AUDIT items, responses from 1,156 participants were analysed.

Survey invitations were disseminated through Australian professional nursing associations, nursing unions, and social media, and the survey containing the AUDIT was administered to participants as part of an online, cross-sectional survey conducted from June to October 2021. At the time of survey dissemination, Australian nursing registration data indicated a potential pool of 349,589 nurses [33]. The sample size required was calculated at 1,065 responses (95% confidence level, 3% margin of error). The survey was reviewed by the relevant health service human research ethics committee, and participants were provided with a Participant Information Sheet and gave informed consent before completing the survey.

Data analysis

Factor analysis, a direct extension of regression and partial correlation theories [34], was applied. Both exploratory (EFA) and confirmatory factor analysis (CFA) were conducted in this study. The former is an approach “to analyse interrelationships among a large number of variables and to explain these variables in terms of their common underlying dimensions,” [35, p. 25]] while the latter is “a way of testing how well a prespecified measurement theory composed of measured variables and factors fits reality as captured by data,” [35, p. 660]]. We conducted a 3-step approach (CFA-EFA-CFA) supported by Knekta et al. [36] using the existing data collected in a recent study [30] with permission of the authors.

In step one, using the whole dataset (N = 1156), we conducted an initial CFA in IBM SPSS AMOS version 27.0. to examine the validity of the AUDIT tool specified by Babor et al. [37]. This model included three domains: hazardous alcohol use (item 1–3), dependence symptoms (item 4–6) and harmful alcohol use (items 7–10). Criteria for construct validity were assessed in accordance with Hair et al.’s (2019) recommendations. These criteria include standardised loadings ≥ 0.5 (ideally 0.7) and Average Variance Extraction ≥ 0.5 to indicate convergent validity; Construct Reliability ≥ 0.7 to indicate reliability; and Maximum Shared Variance lower than Average Variance Extraction to indicate discriminant validity [35]. Of the 1159 participants, 58 participants worked in alcohol and other drugs settings and were likely more familiar with the AUDIT tool and its purpose as compared to nurses working in other settings; to determine whether this enhanced familiarity affected their own interpretation of the tool, we also examined the validity of the 3-domain AUDIT tool in this subgroup in step one.

In step two, two rounds of EFA were conducted in IBM SPSS version 27.0 using two third of the whole dataset (n₁ = 800) which was then split again into two subsets. The first round of EFA was to explore the underlying patterns of the AUDIT tool (n_1a = 420) when validity could not be established in the initial CFA and to additionally re-specify the factor model if required. The second EFA was to assess the stability of the model identified in the first round (n_1b = 380). The suitability and adequacy of the data was examined using Bartlett’s Test of Sphericity (p < 0.05) prior to deriving factors. Following Hair Jr et al.’s (2019) recommendations, Eigenvalue (> 1.0) scree plot, and percentage of variance (≥ 0.6) were used as guidelines to select the optimum number of factors. Communalities (≥ 0.4) were assessed to understand the amount of an item’s variance explained by its own loadings. Factor loadings (≥ 0.3 to be acceptable based on the sample size, or higher to be practically significant) were examined to shed light on the correlation between an item and the loaded factor, and correlation between an item and all other items in the EFA [35]. Cronbach’s alpha (> 0.7) was used to assess the reliability of the derived factor model [35].

In step three, a subsequent CFA was used to assess validity of the model specified in step two using the remainder of the data (n₂ = 356). The assessment was based on significant standardised factor loadings (≥ 0.5) and multiple fit indices. These indices included Chi-square value, degree of freedom, absolute fit indices (Goodness-of-Fit Index [GFI > 0.9], Root Mean Square Error of Approximation [RMSEA < 0.08], and standardised residuals [absolute values less than 4.0]); and incremental fit indices (Comparative Fit Index [CFI ≥ 0.96]) [35].

Results

Step one: initial confirmatory factor analysis

We examined item loadings and validity and reliability indices of the specified 3-domain measurement model. Visual diagram of this model is presented in Fig. 1a. When using the whole dataset including all participants (N = 1156), CFA outputs indicated that items 6 and 9 were not well captured by the data (loadings of 0.25 and 0.39 respectively); items 2, 3, 4, 7 and 8 were excellently captured by the data (loadings > 0.7); and items 1, 5 and 10 require further considerations (loadings between 0.5 and 0.7) (see Fig. 1b). The criteria for Construct Reliability, Maximum Shared Variance and Average Variance Extracted were not met (see Table 1). In the subset of data by participants working in alcohol and other drug settings, these result patterns were similar although items 5, 6, and 9 performed more poorly (see Fig. 1c; Table 1). These results demonstrated that both reliability and validity of the specified 3-domain model AUDIT tool was not supported by the existing data, warranting EFA to explore potentially different underlying patterns.

Fig. 1.

The specified, estimated and modified measurement models. a. The measurement model’s visual path. b. The measurement model estimated by all nurses (N = 1156). model’s visual path. c. The measurement model estimated by nurses working in alcohol and other drugs settings (n = 58). d. The modified measurement model (n = 356)

Table 1.

Results of initial confirmatory factor analysis (step one)

Model	Model fitness indices	Domain	CR	AVE	MSV	Domain*
						Hazardous Alcohol Use	Dependence Symptoms	Harmful Alcohol Use
Model 1b (N = 1156)	CMIN = 294.182 Df = 32 CMIN/df = 9.193 CFI = 0.941 GFI = 0.950 RMSEA = 0.084	Hazardous Alcohol Use	0.797	0.577	0.692	0.760
		Dependence Symptoms	0.615	0.384	0.990	0.788	0.619
		Harmful Alcohol Use	0.710	0.395	0.990	0.832	0.995	0.629
Model 1c (n = 58)	CMIN = 43.496 Df = 32 CMIN/df = 1.359 CFI = 0.947 GFI = 0.875 RMSEA = 0.079	Hazardous Alcohol Use	0.807	0.596	0.590	0.772
		Dependence Symptoms	0.472	0.332	0.876	0.746	0.576
		Harmful Alcohol Use	0.738	0.447	0.876	0.768	0.936	0.669

Note: AVE: Average variance extraction. CR: construct reliability. MSV: maximum shared variance. ASV: average shared squared variance. *: Inter-domain correlations

Step two: exploratory factor analysis

In the first round of EFA (n_1a = 420), the measure of sampling adequacy (0.8) and the significant Bartlett’s Test of Sphericity ( = 1306.9, p < 0.001) indicated that the subset of the data was sufficient and appropriate for the analysis. The EFA with maximum likelihood extraction and varimax rotation method derived at three factors that explained 61.4% of the variance in the data. Of the 10 AUDIT items, there were seven significant loaders (item 1, 2, 3, 4, 5, 7, and 10), two low loaders (item 6 and 9), and one significant cross-loader (item 8) (see Table 2). Communality extractions indicated three items 5, 6, and 9 as potentially problematic (0.082, 0.095 and 0.336 respectively) (see Table 2). Next, when removing item 6, two factors were derived explaining just under 60% of variance of the data; one low factor loading (item 9), one significant cross loader (item 1); and communalities of items 1, 5, 9, and 10 less than 0.4 (0.270, 0.334, 0.041, and 0.204). Subsequently, item 9 thus was removed from the model which now derived only one factor that explained 47.9% of variance in the data. Communalities of item 1, 5 and 10 remained suboptimal (0.263, 0.255 and 0.180 respectively) although all loadings were above 0.5 except item 10’s loading (0.424). These three low communalities could be acceptable at this stage given the relatively large sample size. We did not take action to remove these three items but continued to examine them through the next round of EFA and reliability coefficients.

Table 2.

Rotated factor loadings and communalities resulting from the first round of exploratory factor analysis (step two)

Item	First round of EFA*				Second round of EFA (End of Step two)
	Factor 1	Factor 2	Factor 3	Communalities	Factor 1**	Communalities
Item 1	0.205	0.139	0.968	0.999	-	-
Item 2	0.991	0.109	0.077	0.999	0.762	0.581
Item 3	0.624	0.347	0.325	0.615	0.774	0.599
Item 4	0.368	0.640	0.154	0.569	0.793	0.628
Item 5	0.192	0.531	0.131	0.336	-	-
Item 6	0.016	0.280	0.054	0.082	-	-
Item 7	0.390	0.642	0.283	0.644	0.787	0.620
Item 8	0.455	0.467	0.079	0.431	-	-
Item 9	0.040	0.284	− 0.113	0.095	0.821	0.674
Item 10	0.161	0.415	0.159	0.223	-	-

Note: EFA: Exploratory Factor Analysis. Extraction Method: Maximum Likelihood. Rotation Method: Varimax with Kaiser Normalization. Rotation converged in 4 iterations. *Rotated factor loadings. **Unrotated factor loadings as only one factor was extracted, the solution could not be rotated

Replicating the EFA in the second subset of the data (n_1b = 380) yielded similar results including the number of factors extracted (one factor), factor loadings (unrotated, ranging 0.499–0.798), total variance (54.160%) and communalities. Communalities results demonstrated consistently poor performance of items 1, 5 and 10 (0.249, 0.285, and 0.250 respectively) while the remaining items (2,3,4,7 and 8) performed excellently (ranging 0.545–0.661). Cronbach’s Alpha on deleted items in both rounds of EFA also suggested that these three items (1, 5, and 10) contributed marginally to the tool performance if not actually worsening the overall internal consistency of the tool (see Table 3).

Table 3.

Examination of reliability coefficients through EFA

First round of EFA					Second round of EFA				Modified model at the end of Step two
Item-Total Statistics					Item-Total Statistics				Item-Total Statistics
Item	Scale Mean if Item Deleted	Scale Variance if Item Deleted	Corrected Item-Total Correlation	Cronbach’s Alpha if Item Deleted	Scale Mean if Item Deleted	Scale Variance if Item Deleted	Corrected Item-Total Correlation	Cronbach’s Alpha if Item Deleted	Scale Mean if Item Deleted	Scale Variance if Item Deleted	Corrected Item-Total Correlation	Cronbach’s Alpha if Item Deleted
Item 1	3.23	17.048	0.473	0.814	3.43	20.125	0.480	0.862	-	-	-	-
Item 2	5.05	18.160	0.590	0.788	5.26	20.557	0.673	0.835	2.27	8.813	0.726	0.854
Item 3	4.69	16.159	0.707	0.768	4.82	18.246	0.726	0.826	1.82	7.445	0.732	0.860
Item 4	5.25	18.090	0.631	0.783	5.42	19.853	0.734	0.827	2.42	8.593	0.729	0.852
Item 5	5.47	21.295	0.450	0.813	5.65	23.675	0.479	0.857	-	-	-	-
Item 7	5.06	16.972	0.717	0.769	5.30	19.578	0.757	0.824	2.31	8.572	0.715	0.856
Item 8	5.44	20.514	0.554	0.803	5.52	21.548	0.719	0.836	2.53	9.611	0.754	0.857
Item 10	5.17	18.357	0.391	0.822	5.42	20.682	0.460	0.862	-	-	-	-
Cronbach’s alpha (total scale)				0.817				0.859				0.881

Note: EFA: Exploratory Factor Analysis

Discussion among the team and with experts in nursing workforce research led to the removal of these items. This step was taken due to consistently poor performance (item 1 and 10), and potentially confusing language among Australian nurses (item 5). The results of EFA now with five items (2, 3, 4, 7, and 8) were reassuring of the removal decision: high communalities (0.581–0.628), 69.610% of total variance in the sub-dataset explained, and high unrotated factor loadings (0.762–0.821) (see Table 2). A Goodness-of-Fit test was significant (= 59.871(5), p < 0.001). Reliability coefficients of the total scale and of the scale if item deleted were also meritorious (see Table 3). These results demonstrated that reliability of 5-item AUDIT tool was supported in the current dataset.

Step three: confirmatory factor analysis

The five-item unidimensional AUDIT tool that was specified at the end of step two was proceeded to CFA in the remainder of data (n₂ = 356). All loadings were above 0.7 and all standardised residual covariances between − 1.487 and 1.691; however, the fit indices were suboptimal (CMIN/DF = 10.204, p < 0.001, CFI = 0.945, GFI = 0.942 and RMSEA = 0.161). Based on modification fit indices, covariances between error terms for items 2 and 3, and items 4 and 7 were made to improve model fitness. This led to significantly improved fitness with excellent values of CFI (0.991), GFI (0.989) and significant chi-square test. The RMSEA were slightly above the recommended threshold; however, it did not raise concerns due to its sensitivity to relatively sample size. We thus did not pursue further modification to achieve a perfect fit. The modified model specified in step two was confirmed and illustrated in Fig. 1d and now referred to as AUDIT-N (N = Nursing).

Among the five items, item 3 (How often do you have six or more standard drinks on one occasion?) had highest loading (0.82) which indicated good potential of the AUDIT-N to screen the participants’ frequency of heavy alcohol consumption. The Average Variance Extraction of this modified tool (0.008) was well below the recommended threshold, demonstrating its poor convergent validity. Meanwhile, the construct reliability of this model was 0.723, indicating satisfactory internal consistency of this modified tool.

Discussion

This paper validated the use of the AUDIT among a cohort of Australian nurses, both as a reliable measure of risky and hazardous alcohol consumption, and for validity of the AUDIT questions to a professional audience. Following a rigorous and iterative approach of data analysis, we confirm that (1) the original 3-factor 10-item AUDIT tool was not valid and reliable to screen for alcohol consumption among nurses in Australian settings, (2) a modified version of the AUDIT tool – AUDIT-N (unidimensional 5-item model) was a substantially better fit for use in this population and setting although, (3), this version was only established with reliability but not construct validity.

Five items (1, 5, 6, 9 and 10) were removed from the measurement model during the analysis process. Item 1 (How often do you have a drink containing alcohol), despite being removed from the tool, could still be used as a stand-alone screening question before implementing the modified 5-item AUDIT tool. Meanwhile, the consistent statistical evidence in both CFA and EFA demonstrated that item 6 (How often during the last year have you needed a first drink in the morning to get yourself going after a heavy drinking session) and item 9 (Have you or someone else been injured because of your drinking) were particularly not supported by the data and should not be used among nurses. Item 5 (How often during the last year have you failed to do what was normally expected of you because of drinking) and item 10 (Has a relative, friend, doctor, or other health care worker been concerned about your drinking or suggested you cut down) were also not well perceived by the participants, although their performance was somewhat better than that of item 6 and 9. The poor performance of these items might be related to their wording and its potential implication on professional image as well as professional consequences among registered health professionals such as participants in this study.

Previous studies exploring alcohol consumption among nurses have noted that wording of testing and assessment items need to consider issues specific to the nursing profession, such as knowledge, professional language and the need for confidentiality when disclosing alcohol consumption patterns that may be harmful. For example, in a study conducted with nurses exploring the use of technology-based interventions for alcohol use, privacy and confidentiality were factors in the uptake of interventions [38], and this consideration may also relate to concerns of how results of these questions are stored. Among the nursing workforce, there are examples of negative attitudes or stigma toward risky and/or dependent alcohol consumption [39–43], which may influence nurses’ perceptions of questions related to harmful outcomes of alcohol consumption.

The AUDIT has been validated in several contexts and languages and has been used extensively among English and non-English speaking countries (Horváth et al., 2023). Our findings indicate that largely, the full 10-item AUDIT is appropriate to be used among a population of nurses, however there are questions within the tool that were not supported by the analysis, requiring removal or rewording. Previous validation of the AUDIT questions in specific populations has found similar results, with the modification of questions to suit the context of testing; an example is the modification to create the “USAUDIT,” a version of the AUDIT-C created for use across the United States to “… provide a more precise measurement of drinking frequency,” due to differences in standard alcoholic drink sizes and alcohol consumption guidelines (Higgins-Biddle & Babor, 2018, p. 7).

Although the AUDIT has been validated extensively in the general population and displayed good validity as a screening test (Humeniuk et al., 2008), to our knowledge this is the first time that the AUDIT has been validated amongst a population of nurses. Both the AUDIT and AUDIT-C have been used in studies exploring alcohol consumption among workers, however validation remains absent, with the vast majority of evaluation and validation work examining the general public, or specialty healthcare populations such as primary care patients, those hospitalised with trauma, or those with mental illness (Bradley et al., 2007; Humeniuk et al., 2008; Maisto et al., 2000; Vitesnikova et al., 2014). Despite this lack of validation, it should be noted that the AUDIT performs best against other screens in specific populations in evaluations against established alcohol abuse or dependence criteria (Chung et al., 2000), however our validation shows that adaptation is required to ensure optimal performance of the AUDIT in the nursing population. The AUDIT has performed well under these situations; for example, modification of the AUDIT in Australia to reflect local alcohol consumption guidelines found good internal consistency with an ability to detect 85% of alcohol use disorders in a population of 370, however with the caveat of limited specifity (J. Degenhardt, 2001).

Recommendations

The strength (reliability established) and weakness (absence of validity) of this modified 5-item AUDIT tool warrant the need for future investigations. We strongly recommend the development and validation of a new tool with careful consideration of language use to ensure its intended use for screening of alcohol consumption in registered health professionals such as nurses. Given the resource-intensive nature of tool development research, the modified 5-item AUDIT tool (AUDIT-N) can still be used to screen the frequency of heavy drinking among nurses if its validity is explored in comparison with similar existing tools.

Conclusion

Although the AUDIT has been validated extensively in the general populations, and displayed relatively acceptable validity as a screening test [44], to our knowledge this is the first time that the AUDIT has been validated amongst a population of practising nurses. The modification involving item removal has occurred previously with the AUDIT questions, with the modification of questions to suit the context of testing; an example is the modification to create the “USAUDIT” for use across the United States to “… provide a more precise measurement of drinking frequency,” due to differences in standard alcoholic drink sizes and alcohol consumption guidelines [45, p. 7]. The modification of the 10-item AUDIT tool to inform AUDIT-N is necessary to reflect the experience of our participants – practising nurses who might be aware of the application of this tool through their education/training and in daily practice, including those who worked in Alcohol and drug services. The modification is also necessary to better reflect Australian local alcohol consumption guidelines albeit with the caveat of limited psychometric properties which was also the case in another study (Degenhardt, 2001).

Author contributions

A.S. completed intial data collection for this manuscript and curated the data. V.N. performed data analysis. V.N. and A.S. wrote and edited the main manuscript text, while V. N. prepared the figures. Both authors reviewed the manuscript.

Funding

No external funding was received by any authors for this study.

Data availability

The datasets generated and/or analysed during the current study are not publicly available due to ethical requirements but are available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate

The survey data this study was based on was reviewed and approved by the Monash Health Human Research Ethics Committee (HREC), and all participants were provided with a written Participant Information Sheet, and all participants were required to provide informed consent before participating. This study adheres to the Declaration of Helsinki.

Consent for publication

No individual data is presented in this manuscript.

Competing interests

The authors declare no competing interests.