The Reliability and Validity of the Self-Reported Drinking Measures in the Army’s Health Risk Appraisal Survey

Abstract

Background

The reliability and validity of self-reported drinking behaviors from the Army Health Risk Appraisal (HRA) survey are unknown.

Methods

We compared demographics and health experiences of those who completed the HRA with those who did not (1991–1998). We also evaluated the reliability and validity of eight HRA alcohol-related items, including the CAGE, weekly drinking quantity, and drinking and driving measures. We used Cohen’s κ and Pearson’s r to assess reliability and convergent validity. To assess criterion (predictive) validity, we used proportional hazards and logistical regression models predicting alcohol-related hospitalizations and alcohol-related separations from the Army, respectively.

Results

A total of 404,966 soldiers completed an HRA. No particular demographic group seems to be over- or underrepresented. Although few respondents skipped alcohol items, those who did tended to be older and of minority race. The alcohol items demonstrate a reasonable degree of reliability, with Cronbach’s α = 0.69 and test-retest reliability associations in the 0.75–0.80 range for most items over 2- to 30-day interims between surveys. The alcohol measures showed good criterion-related validity: those consuming more than 21 drinks per week were at 6 times the risk for subsequent alcohol-related hospitalization versus those who abstained from drinking (hazard ratio, 6.36; 95% confidence interval=5.79, 6.99). Those who said their friends worried about their drinking were almost 5 times more likely to be discharged due to alcoholism (risk ratio, 4.9; 95% confidence interval=4.00, 6.04) and 6 times more likely to experience an alcohol-related hospitalization (hazard ratio, 6.24; 95% confidence interval=5.74, 6.77).

Conclusions

The Army’s HRA alcohol items seem to elicit reliable and valid responses. Because HRAs contain identifiers, alcohol use can be linked with subsequent health and occupational outcomes, making the HRA a useful epidemiological research tool. Associations between perceived peer opinions of drinking and subsequent problems deserve further exploration.

The US army offered soldiers a health-behavior screening tool, the Health Risk Appraisal (HRA), for more than a decade. Although its original purpose was to facilitate a workforce wellness plan, this program has been underused for one of its greatest potential benefits: research that links self-reported behaviors, such as alcohol abuse, with adverse health and occupational outcomes. Although the potential uses of the Army’s HRA survey data are numerous, the reliability and validity of the items contained in the HRA have not yet been evaluated thoroughly among military personnel. This article assesses the generalizability of the HRA to the rest of the Army population and evaluates the reliability and validity of the HRA’s alcohol use measures.

The Army’s HRA was implemented in 1988 and was ultimately discontinued in 1998, although it is still in use at a few active-duty Army installations (Bell et al., 2002). The method of HRA administration was nonrandom, but it was routinely offered to soldiers as part of in-processing to new posts or duty assignments. Although we do not know the precise reason that soldiers may have responded to an HRA, there is an item on the HRA that asks why the individual is taking the survey (i.e., in-processing, periodic physical examination, prephysical fitness test, occupational health program, walk-in clinic, or other).

The Army’s HRA questionnaire (DA Form 5675) comprises 75 items that include basic demographic, administrative, and anthropometric information; clinical data; and self-reported information about health behaviors (e.g., alcohol and tobacco use). The eight HRA alcohol items document the amount of alcohol consumed weekly (in drinks per week); whether friends are worried about the respondent’s drinking; whether he or she has ever had a drinking problem; how much exposure the respondent has had in the past month to drinking and driving (either as a driver or as a passenger riding with someone who had consumed too much alcohol); and the CAGE. The CAGE comprises four questions that follow the acronym of its name: “Have you ever felt you should Cut down on your drinking?” “Have people ever Annoyed you by criticizing your drinking?” “Have you ever felt bad or Guilty about your drinking?” and “Have you ever had a drink first thing in the morning to steady your nerves or get rid of a hangover (Eye opener)?” (Ewing, 1984).

Although the CAGE, in use since the 1970s, has been evaluated extensively in civilian populations (Bush et al., 1987; Mayfield et al., 1974), only a few studies have examined its utility among military populations. It is conceivable that the reliability and validity of responses to the CAGE may differ in a military population, where the responses are not confidential. The military exerts considerable control over behavior both during and after work hours; it is also a culture that has traditionally supported cultural norms that condone heavy drinking (Ames et al., 2002; Bray et al., 1991, 1995).

Several studies have evaluated the utility of the HRA alcohol items for research. None of these studies, however, specifically examined its reliability and validity (Fertig and Allen, 1996; Fertig et al., 1993; Fitzpatrick and Shannon, 1992). A cross-sectional analysis of six HRA drinking items in a group of active-duty Army soldiers demonstrated that the CAGE was a good predictor of hazardous drinking (i.e., 21 drinks per week for men and 14 drinks per week for women) and that the combination of the CAGE with two other alcohol-related items from the HRA (exposure to drinking and driving and ever had a drinking problem) was an even better predictor than the CAGE alone (Fertig et al., 1993).

Despite this previous work, the criterion validity of the CAGE, that is, its ability to predict subsequent alcohol-related outcomes, has not been evaluated in military populations. Less has been documented about the predictive capacity of the other four alcohol items on the Army’s HRA. Numerous civilian studies have demonstrated that self-reports of excessive alcohol use are related to alcohol-related problems (Chipman, 1995; McIntosh et al., 1994; O’Hare, 1993; Smith et al., 1995; Thompson et al., 1993; Vingilis et al., 1994). Few studies, however, have quantified the link between self-reported alcohol use and subsequent adverse events among military personnel. This is especially important to evaluate in this population because the HRA is not administered anonymously and soldiers may fear reprisals if they provide responses indicative of high-risk or dependent drinking. An anonymous, comprehensive, independent survey of military personnel from all services worldwide (the so-called Worldwide Survey) suggests that military personnel are more likely to report drinking in unhealthy ways than their civilian counterparts (Bray et al., 1995). This survey also identified a sizable group of individuals whose self-reported drinking might place them at risk for serious adverse outcomes, productivity loss, or dependence symptoms (Bray et al., 1995). However, because the Worldwide Survey was offered anonymously, data cannot be linked to subsequent health outcomes.

Neither the Worldwide Survey nor the HRA’s ability to measure actual drinking behaviors is known. It is, however, possible to evaluate the quality and utility of the HRA alcohol use measures by linking HRA survey data to other sources of information on demographics, health, and occupational outcomes.

METHODS

The Data

Data for this study come from the Total Army Injury and Health Outcomes Database (TAIHOD; Amoroso et al., 1997, 1999). The TAIHOD uses encrypted Social Security Numbers to link individual records from a number of Department of Defense administrative and health data sources. Once data are linked, unique identifiers are removed, and each subject is assigned a randomly selected subject identification number. Portions of the TAIHOD used in this article include HRAs, personnel records (i.e., demographics and discharge information), and hospitalizations.

The Alcohol Use Measures

Thirteen alcohol use variables were evaluated: eight survey items and five constructed variables derived from these survey items. Table 1 details the questions, response options, frequencies, and percentages of missing data for each variable. The Drinking Quantity item captures the number of drinks consumed in a typical week. Responses to this item were grouped into five categories to approximate the drinking values often used to help define appropriate or “safe” drinking habits (Campbell et al., 1999; Gordis, 1992; Sanchez-Craig et al., 1995). We also constructed a Heavy Drinking variable by grouping weekly drinking quantity into quartiles and defining the top 25th percentile as a potentially high-risk group. Because women consume less alcohol than men on average (Stinson et al., 1998) and are at greater risk for adverse outcomes at lower levels of alcohol use (Coates et al., 1986; Gross and Billingham, 1998; Tuyns and Pequignot, 1984; Urbano-Marquez et al., 1995; Wechsler et al., 1994, 1995; Zador, 1991), we developed these percentiles for men and women separately and then combined the gender groups in each drinking risk category. The Drinking and Driving item was dichotomized to reflect any exposure (i.e., one or more times in the past month) versus no exposure. This grouping was necessary given the small proportion of respondents reporting any exposure to drinking and driving.

Table 1.

Descriptions of Alcohol-Related Survey Items and Constructed Variables on the Army’s Health Risk Appraisal Survey: Response Categories and Number of Active-Duty Army Soldiers Responding in Each Category, 1991–1998

Item or constructed variable name	Definition	Response codes	Frequency (%)
Drinking quantity (survey item)	How many drinks of alcoholic beverages do you have in a typical week?	0 to 99	399,746 (98.7)
		Missing	5,220 (1.3)
Drinking category (constructed variable)	Categorical variable based on Drinking Quantity	1 = 0 drinks	177,468 (43.8)
		2 = 1—7 drinks	167,543 (41.4)
		3 = 8—14 drinks	34,557 (8.5)
		4 = 15—21 drinks	10,237 (2.5)
		5 = >21 drinks	9,941 (2.5)
		Missing	5,220 (1.3)
Heavy Drinking (constructed variable)	Using gender-based frequency distributions for the variable Drinking Quantity, respondents falling in the top 25th percentile were coded as Heavy Drinking	0 = not in top 25th percentile	290,075 (71.6)
		1 = in top 25th percentile	108,923 (26.9)
		Missing	5,698 (1.5)
Cut Down (survey item)	“Have you ever felt you should cut down on your drinking?”	0 = No	272,462 (67.3)
		1 = Yes	63,097 (15.6)
		Missing	69,407 (17.1)
Annoy (survey item)	“Have people ever annoyed you by criticizing your drinking?”	0 = No	314,844 (77.8)
		1 = Yes	20,187 (5.0)
		Missing	69,935 (17.3)
Guilty (survey item)	“Have you ever felt bad or guilty about your drinking?”	0 = No	305,507 (75.4)
		1 = Yes	28,635 (7.1)
		Missing	70,824 (17.5)
Eye Opener (survey item)	“Have you ever had a drink first thing in the morning to steady your nerves or get rid of a hangover (eye opener)?”	0 = No	317,330 (78.4)
		1 = Yes	16,685 (4.1)
		Missing	70,951 (17.5)
CAGE (constructed variable)	Two or more “yes” responses to any of the following items: Cut Down, Annoy, Guilty, Eye Opener	0 = No	300,688 (74.3)
		1 = Yes	33,366 (8.2)
		Missing	70,912 (17.5)
Friends Worry (survey item)	“Do your friends ever worry about your drinking?”	0 = No	326,021 (80.5)
		1 = Yes	8,311 (2.1)
		Missing	70,634 (17.4)
Drinking Problem (survey item)	“Have you ever had a drinking problem?”	0 = No	321,746 (79.5)
		1 = Yes	13,959 (3.5)
		Missing	69,261 (17.1)
CAGE2 (constructed variable)	Two or more “yes” responses to any of the following items: Cut Down, Annoy, Guilty, Eye Opener, Friends Worry, Drinking Problem	0 = No	296,375 (73.2)
		1 = Yes	37,349 (9.2)
		Missing	71,242 (17.6)
Drink and Drive (survey item/categorized)	“How many times in the last month did you drive or ride when the driver had perhaps too much alcohol to drink?”	0 = Never	358,380 (88.5)
		1 = 1 + times	41,920 (10.4)
		Missing	4,666 (1.2)

The four items comprising the CAGE were evaluated separately and as a group. Respondents answering yes to two or more of these questions were classified as at high risk for dependent drinking. We also constructed a composite variable for an expanded measure of alcohol dependence (referred to hereafter as CAGE2) based on responses to the four CAGE items and the two other dichotomous items, the Friends Worry and the Drinking Problem items. These last two items measure potential for alcohol dependence, and we therefore expected them to correlate closely with the four CAGE items. A respondent reporting a positive response to two or more items in the CAGE2 was classified as at high risk for dependent drinking. Respondents missing responses to any of the items used to create the CAGE or two or more of the items used to create CAGE2 were excluded from analyses involving these constructed variables.

These analyses were complicated by a skip instruction on the 1990 version of the HRA. It followed the Drinking Quantity item and instructed abstainers to skip the items on alcohol-related problems. This skip instruction was deleted in the 1992 version of the form. Unfortunately, there is no variable in the HRA database that indicates which version of the survey was used. Our prior work suggests that both versions were probably in use concurrently for a long period of time (Bell et al., 2002). To assess the relative proportions of respondents who skipped alcohol items because they were following the skip instructions versus those who may have skipped them inappropriately, we examined the proportion of respondents missing data on the items about alcohol-related problems among drinkers only, because drinkers should have answered these items regardless of which version of the survey they took.

The Study Population

The primary study population included all active-duty Army soldiers who completed at least one HRA between January 1, 1991, and December 31, 1998 (n = 404,966). HRAs taken before October 1, 1990, are not currently available in electronic form for analysis; we thus initiated our analyses at the start of the first calendar year for which HRAs were available.

Analysis

Individual variables were examined by using frequency distributions and standard descriptive techniques. To evaluate the generalizability of our findings to Army personnel who did not complete an HRA, we made year-by-year comparisons of the demographic compositions of HRA takers and nontakers. We compared the demographic profiles of respondents missing responses to some or all of the alcohol items with the profile of the entire study population.

To evaluate whether or not HRA takers were more likely to have been sick than nontakers before completing an HRA (and thus also more likely to have been offered an HRA at a healthcare facility), we selected soldiers who had completed at least one year of active duty and then compared hospitalizations among HRA takers and nontakers in the year before they took the HRA. We also compared the proportions of HRA takers and nontakers who ultimately experienced an alcohol-related hospitalization or were discharged from the Army due to alcoholism.

We evaluated test-retest reliability by using responses from soldiers who completed two or more HRA surveys during their Army career. Caution is warranted in assessing the reliability of self-reported alcohol use because alcohol consumption habits typically change as people age (Brody, 1982; Williams and Debakey, 1992). Still, if the alcohol items were reliable, it would be surprising to see drastic changes between self-reported drinking habits at time 1 and time 2 if there were short intervals of time between measurements of alcohol consumption. Although 17% of soldiers who took the HRA more than once actually took it three or more times, for comparability purposes we focused on only the first and second surveys and limited the study population to repeat surveys with at least 2 days between the first and second survey and to only those who reported consuming any alcohol (nonabstainers). Measures of association between the alcohol items were assessed with Cohen’s κ (Agresti, 1990) and Pearson’s r. We measured the internal consistency of the eight original HRA alcohol items with Cronbach’s α (Bohrnstedt, 1983). We also conducted a factor analysis by using the SAS (SAS Institute, Cary, NC) orthogonal factor analysis procedure with varimax rotation to explore the relationships between the alcohol measures.

We assessed convergent validity, or the extent to which different HRA alcohol questions measure the same construct, by examining the strength and direction of the associations between alcohol-use measures. We dichotomized the Drinking Quantity item (with low-risk drinkers consuming 0–14 drinks per week and high-risk drinkers consuming 15 drinks per week or more) and the Drinking and Driving item (no exposure versus one or more times per month; Campbell et al., 1999). We then calculated Cohen’s κ statistics (Agresti, 1990) between each of the alcohol-related items.

The ability of the HRA alcohol items to correctly identify those who are alcohol abusers cannot be directly measured. Another option is to evaluate the association between these drinking items and the prevalence of alcohol-related problems; this is also known as criterion validity. To assess criterion validity, or the predictive capability of the items, we conducted two separate analyses. The first model compared HRA respondents in high- and low-risk alcohol groups in terms of their risk of 1 or more subsequent hospitalizations for any of 43 alcohol-related conditions (list of ICD-9-CM conditions available on request). An alcohol diagnosis in any of the eight possible diagnostic fields was considered an alcohol-related hospitalization. We examined Kaplan-Meier survival curves and log-rank tests were constructed for each alcohol item, followed by univariate Cox proportional hazards regression models. The study cohort was followed from the date of their HRA through December 31, 1998, until they experienced an alcohol-related hospitalization, or until they left the Army (were censored).

The second model compared the risk for discharge from the Army because of alcoholism versus other reasons (including honorable discharge) among HRA respondents in high- and low-risk alcohol-use groups. We used univariate logistical regression models to evaluate the relationship between self-reported drinking and risk for alcohol-related discharge as compared with separation from the military for other reasons among soldiers who had both completed an HRA and were subsequently discharged by 1998.

All analyses were conducted in SAS version 8.1 and adhere to the policies for the protection of human subjects as prescribed in Army Regulation 70-25 and with the provisions of 45 CFR 46.

RESULTS

Frequency Distributions of the HRA Alcohol Items

The distributions of the eight HRA alcohol items are skewed toward low-risk drinking (Table 1). Most soldiers (>85%) reported consuming zero to seven drinks per week and no exposure to drinking and driving, and most responded no to all of the items about alcohol-related problems. Only 1.3% of the population was missing responses to the Drinking Quantity item, and 1.2% did not answer the Drinking and Driving item.

Between 17 and 18% of the population were missing responses to the six items about alcohol-related problems. This may, however, be attributable to the skip instruction on the 1990 version of the form. Limiting this analysis to nonabstainers revealed that approximately 1% of respondents skipped the items about alcohol-related problems (results not shown).

Generalizability to the Rest of the Army

Most HRA respondents (59%) took the survey during in-processing to a new base or job assignment. Only a small percentage of respondents completed an HRA as part of a walk-in clinic visit (3%) or an occupational health program (2%).

There were only small gender and racial differences in reasons for taking the HRA over time. However, there were wide variations across age and rank. Younger and lower-ranking soldiers were more likely to complete an HRA during in-processing; in contrast, older and higher-ranking soldiers were more likely to complete one during a periodic physical examination (data not shown).

HRA takers were almost identical to HRA nontakers in each year with respect to age, gender, rank, education, marital status, and time in service and were no more likely to have experienced a hospitalization in the year before taking an HRA than nontakers. HRA takers were slightly more likely than nontakers to experience an alcohol-related hospitalization after taking the HRA (2.4 vs. 2.1%, respectively; p < 0.05). However, among soldiers who left the military, those who took an HRA were slightly less likely to have been discharged for alcoholism (0.41 vs. 0.49%, respectively; p < 0.05; data not shown).

There were some notable demographic differences between the total population of HRA respondents and those who skipped the alcohol-related problem items (Table 2). Compared with the demographic profile of the total population, a greater proportion of respondents who skipped the Drinking Quantity item and one or more of the alcohol-related problem items was older (i.e., 36 years or older), African American, and higher-ranking enlisted (E5–E9). Nonabstainers who skipped one or more of the CAGE items were more likely to be African American and enlisted than were nonabstainers who answered all the alcohol items.

Table 2.

Demographic Characteristics of Active-Duty Army Soldiers Who Took the Health Risk Appraisal, Those Who Responded to All Alcohol-Related Items, and Those Missing Responses to Items About Alcohol-Related Problems^a

				Missing 1+ yes/no items
Variable	All respondents (n = 404,966) (%)	Answered all alcohol items (n = 326,011) (%)	Nonabstainers answering all alcohol items (n = 217,813) (%)	Missing drinks/week (n = 2,684) (%)	Abstainers (n = 68,962) (%)	Nonabstainers (n = 3,558) (%)
Gender
Male	86.3	87.3	90.0	85.7	81.2	88.5
Female	13.6	12.5	9.8	14.3	18.6	11.4
Race
White	62.4	64.6	67.2	51.3	53.2	53.4
African American	27.5	25.5	23.7	35.8	36.0	35.6
Hispanic	5.1	5.0	4.6	6.2	5.3	5.6
Other	4.9	4.7	4.4	6.6	5.5	5.4
Age (years)
<21	15.8	14.7	10.6	14.2	21.4	13.7
21–25	32.4	33.7	36.5	21.7	26.5	33.7
26–30	19.7	20.3	21.4	14.3	17.4	19.9
31–35	13.9	13.9	14.1	10.8	13.8	12.8
36–40	11.1	10.7	10.6	19.4	12.5	12.2
>40	7.0	6.5	6.7	19.4	8.3	7.5
Rank
E1–E4	52.3	52.4	50.4	41.5	52.6	53.5
E5–E9	33.3	32.4	32.9	42.3	37.2	34.2
WO	1.9	1.9	2.0	2.8	1.7	1.5
O1–O3	7.8	8.5	9.3	4.8	5.0	6.0
O4–O5	4.0	4.1	4.6	6.7	3.2	4.0
O6–O11	0.6	0.7	0.8	1.9	0.4	0.8

WO, warrant officer.

^aPercentages may not add to 100% due to a small portion of missing data. Abstainer was defined as respondents reporting drinking zero drinks per week. Nonabstainers are those reporting drinking one or more drinks per week on average.

Reliability

We assessed the test-retest reliability of the eight HRA alcohol items among the 40,870 nonabstaining soldiers who took the HRA more than once with at least a day between administrations of the HRA. Table 3 displays the relationship between responses at time 1 (first HRA) and time 2 (second HRA) stratified by six time periods (number of days between first and second HRA). Cohen’s κ and Pearson’s r values for these items indicate that, as expected, agreement in responses decreased over time in an almost linear fashion, with correlations being highest for surveys with the shortest time interval between them (2 to 30 days) and lowest for surveys with the longest time interval (>2 years). In general, the largest κ and Pearson’s r values observed across all time periods were for the items Cut Down, Drinking Problem, and Drinking Quantity.

Table 3.

Test-Retest Reliability of Responses to the Alcohol-Related Items on the Army’s Health Risk Appraisal Among Active-Duty Army Soldiers, 1991–1998, Stratified by Time Between Surveys^a

	Time between first and second HRA
Alcohol item	2–30 days (n = 1,124)	>1–2 months (n = 884)	>2–6 months (n = 4,179)	>6–12 months (n = 6,839)	>1–2 years (n = 10,813)	>2 years (n = 17,031)
Drinking Quantityb	.8207	.5480	.4644	.4224	.3903	.3025
Cut Downc	.8309	.5463	.4593	.4210	.3784	.3452
Annoyc	.7885	.4704	.3889	.3037	.2758	.2367
Guiltyc	.7349	.4678	.3784	.3440	.2929	.2639
Eye Openerc	.8028	.4119	.3986	.3926	.2993	.2293
Friends Worryc	.6573	.4132	.2962	.2718	.1732	.1879
Drinking Problemc	.8115	.5414	.4838	.4905	.3559	.3382
Drink and Drivec	.7512	.3843	.3034	.2511	.2574	.2032

^aAnalysis excludes abstainers. Among soldiers who took more than two HRAs, only the first and second were included in this analysis.

^bPearson’s r was used to assess the correlation between responses on the first and second HRAs.

^cCohen’s κ was used to assess associations between responses on the first and second HRAs.

The standardized Cronbach’s coefficient α was 0.69. The coefficient could be increased minimally (by approximately 0.002) if the Drinking and Driving item were removed. Otherwise, the deletion of any other item would cause a decrease in α.

Construct Validity

Table 4 details the relationships between each of the eight HRA alcohol items. Associations ranged from 0.05 (for drinking and driving and reporting ever having had a drinking problem) to 0.47 (for the CAGE items Guilty and Cut Down).

Table 4.

Convergent Validity of Alcohol-Related Items on the Army’s Health Risk Appraisal in a Population of Active-Duty Army Soldiers Taking the HRA Between 1991 and 1998 (φ Coefficient)

Variable	Drink and Drive	Drinking Quantity	Cut Down	Annoy	Guilty	Eye Opener	Friends Worry	Drinking Problem
Drink and Drivea	1.0	0.2119	0.1874	0.1441	0.1365	0.1696	0.1235	0.0543
Drinking Quantityb	0.2119	1.0	0.1842	0.1769	0.0981	0.2534	0.1425	0.0758
Cut Down	0.1874	0.1842	1.0	0.3139	0.4715	0.1962	0.2417	0.2278
Annoy	0.1441	0.1769	0.3139	1.0	0.3293	0.2440	0.3496	0.2207
Guilty	0.1365	0.0981	0.4715	0.3293	1.0	0.1712	0.2733	0.2555
Eye Opener	0.1696	0.2534	0.1962	0.2440	0.1712	1.0	0.2193	0.1713
Friends Worry	0.1235	0.1425	0.2417	0.3496	0.2733	0.2193	1.0	0.2195
Drinking Problem	0.0543	0.0758	0.2278	0.2207	0.2555	0.1713	0.2195	1.0

^aDrink and Drive variable dichotomized as no exposure (i.e., zero times per month) versus one or more times per month.

^bDrinking Quantity variable dichotomized as 0–14 drinks per week versus 15 drinks per week or more.

Factor analysis results suggest that the alcohol items are closely related or exhibit good construct validity. However, two factors with eigenvalues greater than 1 were identified in the rotated factor analyses, suggesting the possibility that the alcohol items may measure two unique constructs. The first explained 39% of the variation and the second an additional 15%. Items loading heaviest on the first factor were those assessing dependent drinking: the CAGE items and the two similar items (friends worry about your drinking and ever had a drinking problem). The remaining measures loaded heaviest on the second factor. All variable factor loadings were greater than 0.40, with the exception of the drink and drive variable, which demonstrated a loading of 0.38 on the second factor (data not shown).

Criterion Validity

Table 5 displays results from Cox proportional hazards models showing risk for alcohol-related hospitalization and logistical regression models showing risk for discharge due to alcoholism. The median time to alcohol-related hospitalization after the first HRA was 988 days. There were 5337 hospitalizations in total. All of the alcohol items were significant predictors of future alcohol-related hospitalizations. The most common alcohol-related hospitalizations were related to alcohol dependence (ICD-9-CM code 303); half of the hospital diagnoses (49.2%) came from this group. There is a strong linear relationship between self-reported weekly drinking quantity and subsequent risk for an alcohol-related hospitalization. At greatest risk were those who reported consuming more than 21 drinks per week. Also at greater risk were those who indicated that their friends were worried about their drinking and those who admitted having had a drinking problem.

Table 5.

Associations Between Self-Reported Alcohol Use and Subsequent Adverse Health and Occupational Outcomes Among Active-Duty Army Soldiers Taking the HRA, January 1, 1991 to December 31, 1998

	Alcohol-related hospitalizationa (n = 404,966)		Discharge from the military for alcoholismb (n = 222,843)
Alcohol Variable	Hazard ratio	95% Confidence interval	Relative risk	95% Confidence interval
Drinking Quantity	1.04	1.04–1.04	1.04	1.03–1.04
Drinking Group
1–7 drinks/week	1.19	1.12–1.27	1.38	1.17–1.63
8–14 drinks/week	2.16	1.98–2.35	2.44	1.98–3.01
15–21 drinks/week	3.23	2.86–3.65	3.14	2.34–4.21
>21 drinks/week	6.36	5.79–6.99	6.04	4.83–7.56
Heavy Drinking	2.27	2.15–2.40	2.37	2.07–2.70
Cut Down	2.94	2.78–3.12	2.56	2.22–2.94
Annoy	4.27	3.99–4.57	4.36	3.71–5.13
Guilty	3.67	3.44–3.91	3.07	2.61–3.61
Eye Opener	3.79	3.51–4.09	3.74	3.14–4.46
CAGE	3.94	3.71–4.19	3.57	3.07–4.15
Friends Worry	6.24	5.74–6.77	4.92	4.00–6.04
Drinking Problem	5.92	5.52–6.34	4.94	4.16–5.88
CAGE2	4.00	3.76–4.25	3.65	3.15–4.22
Drink and Drive	2.11	1.97–2.26	2.21	1.88–2.58

^aSingle variable Cox proportional hazards models. Diagnoses include any of 31 alcohol-related conditions found in any of the 8 possible diagnostic fields (primary, secondary, and so on). The list of conditions is not shown but is available on request from the corresponding author. The study population includes all first-time HRA survey takers who completed at least one HRA between 1991 and 1998.

^bSingle-variable logistical regression models. Discharge for other reasons includes honorable discharge. Study population includes those who both completed an HRA sometime between 1991 and 1998 and were discharged from the Army by 1998.

Approximately half (55.6%; n = 222,843) of the population who completed an HRA had left the military by December 31, 1998. All measures of self-reported drinking were strongly associated with alcohol-related discharge, and there seems to be a linear increase in risk with successively greater amounts of reported weekly alcohol use, which mirrors the risk for alcohol-related hospitalization. Soldiers reporting they ever had a drinking problem were at approximately 5 times greater risk for experiencing a subsequent alcohol-related discharge. Believing that friends worry about one’s drinking is associated with a five-fold increased risk (risk ratio, 4.92; 95% confidence interval, 4.00–6.04) of discharge due to alcoholism, and reporting feelings of annoyance when others criticize one’s drinking is related to a four-fold increased risk (risk ratio, 4.36; 95% confidence interval, 3.71–5.13).

DISCUSSION

Most Army soldiers report light to moderate drinking. However, a sizable minority reports high-risk drinking behaviors. The spread of responses on the Drinking Quantity and Drink and Drive items suggests that even if respondents fear reprisal, some of them are reporting very risky drinking behaviors. The distribution of responses suggests that the HRA may have reasonable discriminatory power for identifying problem drinking. However, it is worth noting that the HRA does not include information that allows one to distinguish respondents who are lifetime abstainers from former drinkers.

Although the HRA was never randomly administered, the process by which it was offered does not seem to have oversampled from any particular demographic group. Completing an HRA during in-processing was more common among younger, lower-ranking soldiers. These differences followed expected patterns in that younger soldiers are more likely to in-process to new assignments as they complete training and enter specific jobs. Although the HRA was offered in health clinics, as well as other settings, there does not seem to be overrepresentation of sicker soldiers among HRA takers, at least with respect to hospitalizations in the prior year.

Nonabstainers who skipped the remaining HRA alcohol items were more likely to be older, African American, and higher-ranking enlisted soldiers. Soldiers in these demographic groups may have greater fears of reprisal for reporting high-risk alcohol use and thus skip these items if they are high-risk drinkers. This sort of information bias would cause an underestimation of the true influence of alcohol on the health of these subgroups. Alternatively, it may simply be that older, higher-ranking enlisted soldiers and racial/ethnic minorities are less likely to be in the high-risk group and thus skip these questions because they do not seem to apply.

HRA takers had a slightly increased risk for alcohol-related hospitalizations compared with nontakers. However, among those soldiers who left the military during the period of follow-up, those who completed HRAs were slightly less likely to have been discharged for alcoholism. This may be due to a healthy-worker bias such that those who have an alcohol problem are more likely to leave the service sooner or be forced to leave as a result of behavior problems (not identified as alcohol-related in the discharge data) and thus are less likely to have an opportunity to complete an HRA. Recent research confirms that soldiers diagnosed with mental health disorders (which include alcohol diagnoses) are discharged more quickly from the Army after their diagnosis than are soldiers diagnosed with any other health condition (Hoge et al., 2002). It may also be that soldiers who indicated heavy alcohol use on the HRA received appropriate treatment.

Test-retest reliability of the HRA alcohol items is quite strong over short time periods (i.e., 2 to 30 days) but gradually declines over time. The decrease in consistency of responses when repeat surveys are taken over long intervals could be due to an actual change in drinking behavior rather than to poor reliability of items (Heather and Robertson, 1997; Stinson et al., 1998). Moreover, our results show that respondents who seem to be at higher risk for alcohol-related problems (i.e., those reporting very heavy weekly drinking, ever having an alcohol problem, or trying to cut down on their drinking) are most consistent in their reported drinking habits across time.

The associations between alcohol items are lower than expected in some cases. For example, it is surprising that drinking and driving exposure is not strongly associated with ever having had a drinking problem. This may be because the drinking and driving item measures not only drinking and driving, but also riding as a passenger when the driver has been drinking. It also could be that problem or dependent drinking measures (e.g., CAGE, Drinking Problem, and Friends Worry items) are measuring a different construct than typical drinking (Drinking Quantity) and drinking and driving behaviors. This was also suggested in the factor analysis.

The data presented demonstrate that the HRA alcohol items are indeed predictive of alcohol-related health problems and adverse occupational outcomes such as discharge for alcoholism. One intriguing and unexpected finding was the apparent link between subjective norms and risk for future alcohol problems. Respondents who reported that their friends were worried about their drinking were at more than 6 times greater risk than other respondents for a subsequent alcohol-related hospitalization and approximately 5 times greater risk for an alcohol-related discharge from the Army. Respondents annoyed by their friends’ concern over their drinking habits were also at substantially greater risk for serious adverse health and occupational consequences. These items were more strongly associated with long-range problems than was the CAGE.

There are some limitations to this study that are important to consider. First, there is no corroborating information available, such as blood alcohol concentration measures, by which to directly evaluate the validity of the alcohol-use items. We have had to rely on more indirect measures of validity, such as the variables’ predictive ability. Strong associations between the alcohol-use measures and subsequent alcohol-related events (hospitalizations and alcohol-related discharge) suggest good validity.

Second, we identified high-risk drinkers in the CAGE and CAGE2 constructed variables as those respondents with positive responses to two or more of the individual items. Although this cutpoint has been used in many other studies and has performed well in identifying problem drinkers in various populations (Bush et al., 1987; Ewing, 1984; Mayfield et al., 1974; Smart et al., 1991; Soderstrom et al., 1997), some studies have suggested that at a traditional cutpoint of two, the CAGE performs less well in identifying problem drinking among women and among racial or ethnic minorities (Cherpitel, 1995; Cherpitel and Clark, 1995). Among the few studies evaluating the predictive utility of the CAGE in the context of the Army’s HRA is a study by Fertig et al. (1993), who discovered that a single affirmative response to any of the four CAGE items was associated with high-risk self-reported drinking on the same survey. Because many researchers and clinicians use a traditional cutpoint of two or more positive responses and because many prior validation studies of the CAGE examined its performance at this traditional cutpoint, we also chose to use the traditional cutpoint of two or more affirmative responses.

The HRA itself has several limitations. It does not include a measure of binge drinking, nor does it assess the quantity and frequency of drinking separately. Responses to the Drinking Quantity item are truncated at 99 drinks per week, making this a closed-ended item. Studies suggest that drinkers are more likely to underreport their drinking behaviors when given closed-ended response options as opposed to open-ended options (Converse and Presser, 1986). It is possible that some soldiers consume more than 99 drinks per week but are unable to honestly report such on the form. Finally, Steinweg and Worth (1993) repoted that the sensitivity of the CAGE in detecting alcoholism was attenuated when it was preceded by items about drinking quantity and frequency, as is true on the Army’s HRA.

Despite these acknowledged limitations, this work makes important contributions to our understanding of drinking patterns and problem drinking in the military and enhances our understanding of some alcohol survey items that are commonly used in both civilian and military contexts. Because the HRA elicits identifying information, such as name and Social Security Number, these survey responses can be linked to demographic, occupational, and health outcomes data. This enables us to develop predictive models identifying which soldiers may be most likely to engage in risky drinking and which ones may be at particularly great risks for adverse outcomes as a consequence of their drinking. Also, because the HRA has a recorded date field, we can examine temporal associations between these self-reported drinking patterns and subsequent contacts with the healthcare system. These results support the use of the HRA alcohol items in epidemiological research.