Performance of the alcohol use disorders identification test among tuberculosis patients in Russia

Abstract

The Alcohol Use Disorders Identification Test (AUDIT), a screening instrument to identify individuals at risk of alcohol use problems, has not been validated in a Russian primary care population. We assessed the reliability, factor structure, sensitivity and specificity of AUDIT scores among 254 subjects initiating tuberculosis treatment from 2005 – 2007 in Tomsk City. Our findings support the use of the AUDIT as a screening instrument among Russian individuals seeking primary care. We discuss implications, limitations and future research.

BACKGROUND

Tuberculosis (TB) rates increased dramatically after the disintegration of Soviet Union and continues to be a critical public health issue in present day Russia (Shilova and Dye 2001). The mortality rate among TB patients in Russia remains high despite expanding implementation of short-course Directly Observed Therapy (DOTS) and use of second-line drugs to treat multidrug-resistant TB (MDR-TB) (Dewan, Arguin et al. 2004; WHO 2008). Among Russian TB patients the prevalence of alcohol dependence and abuse is estimated to range from 24% to 55% (Gelmanova, Taran et al. 2007). Patients with concomitant alcohol use during TB treatment are known to have poor treatment outcomes (Shin, Pasechnikov et al. 2006; Jakubowiak, Bogorodskaya et al. 2007). Thus, addressing alcohol use disorders (AUDs) is an important aspect of TB treatment in Russia (Krupitsky, Zvartau et al. 2006).

One barrier to effective management of AUDs among TB patients in Russia is the lack of a valid instrument to screen for hazardous and harmful alcohol consumption. The Alcohol Use Disorders Identification Test (AUDIT) was developed by the World Health Organization as an effective screening instrument for alcohol use problems among patients seeking primary care for other medical problems (Saunders, Aasland et al. 1993; Babor, Higgins-Biddle et al. 2001). Among the many strengths of the AUDIT is that it was developed cross-nationally among primary care patients in 6 countries. While the AUDIT has been used to assess for alcohol use problems in Russian primary care patients (Averina, Nilssen et al. 2005; Nilssen, Averina et al. 2005), validation of AUDIT performance in such populations has yet to be published. Given that standardized screening for AUDs is rarely performed for patients seeking primary care (Fleming, Meyroyan et al. 1994; Fleming 1996) and alcohol use may account for up to 30-40% of all premature mortality in Russia (Leon, Chenet et al. 1997; Leon, Saburova et al. 2007), validated screening instruments for Russian populations are sorely needed (Reinert and Allen 2007). Here, we describe the psychometric properties of the AUDIT when administered to TB patients in Tomsk, Western Siberia.

METHODS

Study setting

This study was conducted in Tomsk Oblast in Western Siberia, Russia with a population of 1,200,000. In 2003, TB incidence in Tomsk was 93.4 per 100,000. TB mortality was 17.7 per 100,000 persons that year, compared with a national rate of 21.9 deaths per 100,000 inhabitants. In that year, there were 800 new and 100 retreatment cases (Golubkov 2008). Civilian TB care and treatment is provided by the Tomsk Oblast TB Services (TOTBS) in the TB Oblast hospital, TB day hospital and polyclinic. Institutional review boards of the Siberian State Medical University in Tomsk and Brigham and Women’s Hospital in Boston approved this study.

Study population and sample size considerations

All adult patients initiating treatment for TB in one of the three TB treatment centers between November 2005 and March 2007 were referred for this study. 13 individuals (4.9% of those approached) refused study participation. Individuals provided written informed consent. We enrolled a total of 255 individuals to provide a margin of error smaller than 5% for estimating sensitivity and specificity, based on an assumed sensitivity of > 90%.

Assessments

AUDIT

The Alcohol Use Disorders Identification Test (AUDIT) is a self-report screening tool designed to assess current drinking patterns. Each of the 10 questions is scored on a 0 to 4 scale, creating a range of 0 to 40 (higher scores are indicative of drinking problems). Based on responses from nearly 2,000 participants from six countries, AUDIT items were originally selected to reflect three factors thought to be closely associated with harmful and hazardous drinking patterns. These factors include alcohol consumption, alcohol dependence symptoms, and adverse consequences of alcohol use (Saunders, Aasland et al. 1993).

In November 2005, the TOTBS instituted the AUDIT as part of routine TB care. All patients initiating TB treatment provided written responses to the AUDIT questionnaire as part of their initial intake. Next, as part of the pre-treatment evaluation, TB physicians totaled and reviewed the AUDIT score with patients to open discussion about alcohol consumption during TB treatment. Physicians also referred patients with AUDIT scores ≥ 8 to usual care for alcohol treatment.

CIDI

Based on both Diagnostic and Statistical Manual of Mental Disorders-IV (DSM IV) and International Composite Diagnoses (ICD-10) criteria, the Composite International Diagnostic Interview (CIDI) was derived by the World Health Organization to evaluate a spectrum of mental health disorders. The CIDI has been used in Russian populations for diagnosing alcohol abuse and dependence (Zanis, McLellan et al. 1995; Pakriev, Vasar et al. 1998). Eight Russian TB physicians, who were trained and certified in the CIDI, were responsible for administering the substance abuse module to participants within three weeks of TB treatment initiation. CIDI interviewers were blinded to AUDIT scores. Quality control included onsite observation and audiotaped review of random CIDI interviews. Data were double-entered into a database using Blaise software (Westat, Rockville, MD).

Clinical and demographic data

TB physicians completed a standardized form that included patient baseline demographic and clinical characteristics, as well as AUDIT scores.

Analysis

Data were entered or imported into an Access 2000 database (Microsoft Corporation, Seattle, WA). We analyzed data using SAS 9.0 (SAS Institute, Inc., Cary, NC). We defined any alcohol use disorder as either alcohol abuse or alcohol dependence based on DSM-IV criteria using the CIDI software. We assessed the psychometric performance of AUDIT scores in several different ways.

Internal consistency reliability

While the AUDIT is generally capable of producing scores of adequate reliability for most research purposes, the circumstances under which it may not produce scores of adequate reliability is unclear (Shields and Caruso 2003). To better understand how reliably the AUDIT performs among TB patients in Russia, we subjected AUDIT scores to an internal consistency reliability analysis. We report Cronbach’s alpha (α) for the total AUDIT score as well as for each of its three subscales.

Factor analyses

Although the AUDIT was developed to cover three dimensions of alcohol use (consumption, dependence, and consequences of heavy drinking), several studies in diverse populations suggest that the 10 AUDIT items have a correlation structure that is more consistent with only two factors, representing alcohol consumption and alcohol-related consequences (Doyle, Donovan et al. 2007). We performed an exploratory factor analysis (EFA) using maximum likelihood extraction methods to determine the number of underlying dimensions supported by the instrument in this population; to aid interpretation, results of the factor analysis were obliquely rotated using the oblimin criterion. Oblique rotation, which allows the underlying factors to be potentially correlated, was performed because there was no a priori reason to expect the factors to be unrelated to each other.

Sensitivity and specificity

To determine the performance of the AUDIT when compared to a gold standard, we also measured the sensitivity and specificity and reported the receiver operating characteristic (ROC) of the AUDIT compared to CIDI endorsement of any alcohol abuse or dependence based on DSM-IV criteria. For the ROC curve, we plotted sensitivity against “1-Specificity” and estimated the area under the curve (AUC).

RESULTS

Of 255 individuals enrolled, one was excluded because the AUDIT score was missing. The remaining 254 individuals comprise the cohort analyzed in this study. As shown in Table 1, the cohort was predominantly male (73.4%), unemployed (53.0%) and middle-aged (mean 40.2 years old). For this cohort, the mean AUDIT score was 11.50 (standard error 0.58) and 148 individuals (58.3%) had an AUDIT score of ≥ 8. The internal consistency reliability was assessed for the total AUDIT score and for its three subscales. The Cronbach’s alpha for the total AUDIT score was 0.91. The internal consistency reliability for the original AUDIT subscales were α = 0.85 for consumption, α = 0.83 for dependence, and α = 0.78 for consequences of alcohol use. Our factor analysis of the 10 AUDIT items (see below) yielded a two-factor structure and the internal consistency reliability for these two factors were α = 0.85 for the “Alcohol Consumption” scale (items 1 to 3) and α = 0.87 for the “Alcohol Dependence and Consequences” scale (items 4 to 9). Overall, the total AUDIT score demonstrates excellent reliability in this population, while the 3 subscales have adequate to good reliability.

Table 1. Clinical and demographic characteristics of cohort (N=254).

Variable (N, if not complete cohort)	Total cohort N (%) unless specified
Age (Mean ± STD), N=253	40.2 ± 11.4
Male gender	187 (73.6)
Civil status  Married  Separated  Divorced  Widowed  Single  Not known	82 (32.3) 23 (9.1) 59 (23.2) 13 (5.1) 76 (29.9) 1 (0.4)
Occupation, N=253  Employed  Retired  Student  Unemployed  Other	91 (36.0) 16 (6.3) 6 (2.4) 134 (53.0) 6 (2.4)
Receiving disability pension, N=253	21 (8.3)
Prior incarceration, N=253	56 (22.1)
Smoker  Current  Prior  Never	192 (75.6) 29 (11.4) 33 (13.0)
TB treatment history  No prior TB treatment  Prior TB treatment	208 (81.9) 46 (18.1)

As mentioned previously, factor analysis of the 10 AUDIT items (Table 2) supported a two-factor model, as determined by the proportion criterion. To aid interpretation, the two factors were obliquely rotated. From the rotated factor loadings presented in Table 2, it is transparent that questions 4-9 load heavily on the first factor only; the first factor can be interpreted as a measure of “Alcohol Dependence and Consequences”. Questions 1-3 load heavily on the second factor only and this factor can be interpreted as a measure of “Alcohol Consumption”. Finally, question 10 has a relatively modest loading of similar magnitude on both factors; question 10 relates to others being concerned about drinking. The two derived factors, “Alcohol Dependence and Consequences” and Alcohol Consumption”, are positively correlated, with an estimated correlation of 0.83. This indicates that these two factors share approximately 70% common variability, which likely reflects the strong, positive relationship between “Alcohol Consumption” and the extent/severity of negative “Alcohol Consequences and Dependence”.

Table 2. Rotated factor analysis of AUDIT questions, N=254.

Number	Question	Factor1	Factor2
1	How often do you have a drink containing alcohol?	0.03	0.72
2	How many drinks containing alcohol do you have on a typical day when you are drinking?	−0.01	0.76
3	How often do you have six or more drinks on one occasion?	0.01	0.90
4	How often during the last year have you found that you were not able to stop drinking once you had started?	0.73	0.01
5	How often during the last year have you failed to do what was normally expected from you because of drinking?	0.73	0.03
6	How often during the last year have you needed a first drink in the morning to get yourself going after a heavy drinking session?	0.74	0.08
7	How often during the last year have you had a feeling of guilt or remorse after drinking?	0.63	0.13
8	How often during the last year have you been unable to remember what happened the night before because you had been drinking?	0.81	−0.09
9	Have you or someone else been injured as a result of your drinking?	0.61	−0.01
10	Has a relative or friend or a doctor or other health worker been concerned about your drinking or suggested you cut down?	0.33	0.34

Factor 1: “Alcohol Consequences and Dependence”

Factor 2: “Alcohol Consumption”

As shown in Table 3, according to the gold standard CIDI, 50% of all individuals had a lifetime diagnosis of an AUD, 28.3% had an AUD in the 12 months prior to assessment, and 8.7% had an AUD in the past month. Recall that 148 individuals (58.3%) had an AUDIT score of ≥ 8. Table 4 demonstrates the performance of the AUDIT compared to the CIDI. Based on a cut-off score of ≥ 8, the AUDIT had a sensitivity of 91.7% compared with a CIDI endorsement in the past year; when compared to a lifetime diagnosis, the sensitivity and specificity at the same cut-off was 83.5% and 71.6%, respectively. The area under the curve (AUC) of the AUDIT compared with a CIDI diagnosis in the past year was 0.80 (with 95% confidence interval: 0.75 – 0.86), as shown in Figure 1. Based on our data, a threshold of ≥ 8 provides an acceptable cut-off score for the AUDIT to identify those at risk of an AUD in the past year. When compared with a lifetime diagnosis, the AUC was 0.82, with 95% confidence interval: 0.77 – 0.87 (figure not shown).

Table 3. Summary of DSM AUD diagnosis by CIDI, N=254.

Diagnosis	30 days	12 months	Lifetime
No endorsement	232 (91.3)	182 (71.7)	127 (50.0)
Alcohol abuse	5 (2.0)	26 (10.2)	63 (24.8)
Alcohol dependence	17 (6.7)	46 (18.1)	64 (25.2)

Table 4. Sensitivity and specificity of AUDIT cut-off scores compared with CIDI endorsement of DSM-IV diagnosis of AUD in the past year, N=254.

AUDIT cut-off	DSM-IV past-year AUD		DSM-IV lifetime AUD
	Sensitivity	Specificity	Sensitivity	Specificity
≥20	45.8	60.0	38.6	89.1
≥19	55.6	60.6	46.5	89.4
≥18	59.7	62.3	48.8	89.9
≥17	59.7	59.7	50.4	88.9
≥16	62.5	56.3	53.5	85.0
≥15	68.1	54.4	59.1	83.3
≥14	73.6	54.1	63.0	81.6
≥13	73.6	50.5	63.8	77.1
≥12	80.6	49.2	70.9	76.3
≥11	80.6	47.9	71.7	75.2
≥10	84.7	47.7	76.4	75.8
≥9	88.9	45.4	81.1	73.1
≥8	91.7	44.6	83.5	71.6
≥7	93.1	41.6	85.8	67.7
≥6	94.4	40.5	88.2	66.7
≥5	94.4	39.3	89.0	65.3
≥4	94.4	36.4	92.9	63.1
≥3	97.2	35.7	94.5	61.2

Figure 1.

ROC of AUDIT compared with DSM-IV dependence or abuse, past year, N=254

DISCUSSION

In a cohort of Russian patients with TB, we measured a high prevalence of alcohol use disorders. Based on our “gold standard” of the CIDI, more than half of individuals had a lifetime diagnosis of alcohol abuse or dependence and 28.3% had a diagnosis within the past year. These figures are higher than previous reports (24% – 45%) among Russian tuberculosis patients using non-standardized measures of AUDs (Gelmanova, Taran et al. 2007), and comparable to the only other published report using DSM IV criteria (Fleming, Krupitsky et al. 2006).

To our knowledge, this is the first published report of AUDIT validation in a Russian-speaking cohort. The performance of the AUDIT in this cohort was acceptable and comparable to other experiences, where the internal reliability has ranged from 0.75 to 0.97 (Reinert and Allen 2007), sensitivity and specificity range from 0.31 – 0.89 and 0.83 – 0.96, respectively among primary care populations (Berner, Kriston et al. 2007), and the AUC when compared to various gold standard instruments has been reported between 0.72 – 0.96 (Gache, Michaud et al. 2005; Kills Small, Simons et al. 2007; Cassidy, Schmitz et al. 2008; De Silva, Jayawardana et al. 2008). Furthermore, results from our factor analysis are consistent with recent data supporting the use of the AUDIT as either a single-factor or two-factor tool (Shields and Caruso 2003; Shevlin and Smith 2007). While limited by moderate sample size and a low proportion of women (27%), our data support the use of the AUDIT as a screening instrument in Russia and the FSU. Another limitation is that our cohort represents a convenience sample of TB patients; as such, our findings may not be generalizable to the broader Russian population. Further, we could not assess differences between those who refused study participation and the enrolled cohort.

We have demonstrated that the AUDIT produces reliable and valid scores when administered among Russian patients seeking primary care for TB. The internal reliability, as well as the sensitivity, specificity and area under the curve are comparable to validation results of the AUDIT in other populations, thus substantiating the use of the AUDIT, using a standard cut-off score of ≥ 8, as a tool for detecting alcohol use problems in Russia and the Former Soviet Union. However, further research is needed to assess whether AUDIT performance varies by gender in such populations.

The validation of a brief instrument to screen for hazardous and harmful alcohol consumption addresses a crucial gap in the management of alcohol use disorders among Russians seeking primary care. The AUDIT offers a simple and effective tool to screen for alcohol use disorders for individuals in Russia and the Former Soviet Union, among the world’s heaviest and most harmful consumers of alcohol.