Factor structure of the SOCRATES questionnaire in hospitalized medical patients

Abstract

The Stages of Change Readiness and Treatment Eagerness Scale (SOCRATES), a 19-item instrument developed to assess readiness to change alcohol use among individuals presenting for specialized alcohol treatment, has been used in various populations and settings. Its factor structure and concurrent validity has been described for specialized alcohol treatment settings and primary care. The purpose of this study was to determine the factor structure and concurrent validity of the SOCRATES among medical inpatients with unhealthy alcohol use not seeking help for specialized alcohol treatment. The subjects were 337 medical inpatients with unhealthy alcohol use, identified during their hospital stay. Most of them had alcohol dependence (76%). We performed an Alpha Factor Analysis (AFA) and Principal Component Analysis (PCA) of the 19 SOCRATES items, and forced 3 factors and 2 components, in order to replicate findings from Miller & Tonigan (1996) and Maisto et al (1999). Our analysis supported the view that the 2 component solution proposed by Maisto et al (1999) is more appropriate for our data than the 3 factor solution proposed by Miller & Tonigan (1996). The first component measured Perception of Problems and was more strongly correlated with severity of alcohol related consequences, presence of alcohol dependence, and alcohol consumption levels (average number of drinks per day and total number of binge drinking days over the past 30 days)compared to the second component measuring Taking Action. Our findings support the view that the SOCRATES is comprised of two important readiness constructs in general medical patients identified by screening

INTRODUCTION

Brief motivational counseling interventions have efficacy for people with nondependent unhealthy alcohol use (Bertholet, Daeppen, Wietlisbach, Fleming, & Burnand, 2005; Bien, Miller, & Tonigan, 1993; Dunn, Deroo, & Rivara, 2001; Saitz, 2005). Motivational interviewing is an extended intervention that has efficacy for a number of health behaviors, including alcohol dependence (Carbonari & DiClemente, 2000; Project MATCH Research Group1997). Readiness-to-change and motivation are frequently viewed as intermediate outcomes and have been seen as mediators and potential predictors of change (Demmel, Beck, Richter, & Reker, 2004; Heather, Rollnick, & Bell, 1993; Maisto et al., 1999; Williams, Horton, Samet, & Saitz, 2007). As a result, the assessment of motivation to change has been of great interest to researchers and clinicians alike (Fiellin, Reid, & O'Connor, 2000; Miller & Rollnick, 1991).

The Stages of Change Readiness and Treatment Eagerness Scale (SOCRATES) was developed and designed to measure stages of readiness to change alcohol use (W. R. Miller & J. S. Tonigan, 1996). It was first intended as a self-administered questionnaire to categorize individuals into one of four stages of change (pre-contemplation, contemplation, determination, and action (Prochaska & DiClemente, 1984). After several iterations of the SOCRATES, Miller and Tonigan validated a shorter (19-item) version of the SOCRATES in 1996 and reported on the factor structure in a population of participants with alcohol dependence in a multi-site clinical treatment trial [9]. These participants were in specialty settings: either in outpatient treatment programs or seen for aftercare following residential or day treatment. The authors identified 3 independent factors (using exploratory alpha factor analysis with a varimax rotation): Ambivalence, Recognition, and Taking Steps. The 19-item version of the SOCRATES is widely used and non-English versions of the scale have been validated among treatment seeking populations (Demmel et al., 2004; Figlie, Dunn, & Laranjeira, 2005). Each item response is based on a 5-point Likert scale (i.e., 1=strongly disagree, 2=disagree, 3=undecided or unsure, 4=agree, 5=strongly agree). The 19-item SOCRATES is included in Table 2; more information is available online at http://casaa.unm.edu/inst/SOCRATESv8.pdf.

Table 2.

Determination of the most appropriate component structure solution in the study sample (exploratory PCA):

Item	Question	Component 1	Component 2
1	I really want to make changes in my drinking	0.49	0.46
2	Sometimes I wonder if I am an alcoholic	0.73	0.00
3	If I don't change my drinking soon, my problems are going to get worse	0.78	0.24
4	I have already started to make some changes in my drinking	0.14	0.80
5	I was drinking too much at one time, but I've managed to change my drinking	0.02	0.58
6	Sometimes I wonder if my drinking is hurting other people	0.69	0.22
7	I am a problem drinker	0.87	0.09
8	I am not just thinking about changing my drinking, I am already doing something about it	0.26	0.80
9	I have already changed my drinking, and I am looking for ways to keep from slipping back into my old pattern	0.16	0.80
10	I have a serious problem with drinking	0.85	0.19
11	Sometimes I wonder if I am in control of my drinking	0.39	0.26
12	My drinking is causing a lot of harm	0.79	0.22
13	I am actively doing things now to cut down or stop drinking	0.15	0.82
14	I want help to keep from going back to the drinking problems that I had before	0.72	0.38
15	I know that I have a drinking problem	0.88	0.22
16	There are times when I wonder if I drink too much	0.64	0.21
17	I am an alcoholic	0.85	0.12
18	I am working hard to change my drinking	0.33	0.79
19	I have made changes in my drinking and I want some help to keep from going back to the way I used to drink	0.51	0.55
Percentage of Variance Explained		37%	24%
Cronbach's Alpha (Standardized)		0.94	0.88

Because most of the research on the psychometric properties of the SOCRATES has been done in specialized settings in patients with alcohol dependence, there has been concern about whether the questionnaire's factor structure is applicable to non-treatment seeking patients in other settings.

To address the question of the applicability of the factor structure in other populations, Maisto et al. investigated the factor structure of the SOCRATES in a population of opportunistically screened primary care patients (Maisto et al., 1999). Using exploratory component analysis and confirmatory factor analysis, they concluded that a two factor solution (retaining 15 items) was more parsimonious than the three factor solution. Specifically, the first factor contained 9 of the Ambivalence and Recognition items (named AMREC) and the second factor comprised 6 of the Taking Steps items (and was named Taking Steps).

Several publications have reported either a 2 or 3- factor solution for the SOCRATES (Burrow-Sanchez & Lundberg, 2007; Demmel et al., 2004; Figlie et al., 2005). Given inconsistencies in the literature among different populations, we investigated the most appropriate factor structure solution based on a sample hospitalized for medical illness in a general hospital (i.e. not seeking or receiving specialty alcohol treatment) who were identified opportunistically by screening for the spectrum of unhealthy alcohol use (i.e. from use of risky amounts through dependence). This is to our knowledge the first study to investigate the appropriate factor structure solution of the SOCRATES in this population. We performed both an exploratory analysis replicating techniques utilized by and compared our results to those of Miller & Tonigan (1996), and Maisto et al. (1999).

METHODS

Participants

The participants were enrolled in a randomized trial of a brief motivational intervention for unhealthy alcohol use (Saitz et al., 2007). They were recruited while on the inpatient internal medicine service of a large, urban, academic medical center hospital. Eligibility criteria included: 18 or more years old, fluent in English or Spanish, currently (past month) drinking risky amounts (defined as more than 14 standard drinks per week or 5 or more drinks per occasion for men 18 to 65 years of age, and more than 11 standard drinks per week or 4 or more drinks per occasion for women and people over age 65), availability of 2 contacts to assist with follow-up, no plans to relocate in the next 2 years, and a Mini-Mental State Examination score of at least 21. Subjects provided written informed consent and completed the SOCRATES at the time of enrollment.

Study assessments (collected prior to randomization) were administered by trained research associates. Questions regarding alcohol consumption and consequences, medical and mental health, and health care utilization as well as other domains were obtained. Subjects completed the Short Inventory of Problems, a questionnaire assessing alcohol-related consequences (Miller, Tonigan, & Longabaugh, 1995), and the Alcohol Use Disorders Identification Test (Saunders, Aasland, Babor, de la Fuente, & Grant, 1993). The presence of an alcohol use disorder diagnosis was determined with the Composite International Diagnostic Interview (CIDI) Alcohol Module (WHO, 1996). Alcohol consumption was assessed with a validated calendar method (30-day Timeline Followback) (Sobell & Sobell, 1995). The study was approved by the Boston University Medical Center Institutional Review Board and all subjects provided written informed consent. Details of the methodology of the randomized controlled trial are reported elsewhere (Saitz et al., 2007).

Analyses

The first step of the analysis was to ensure that the underlying assumptions of the factor analytic model were appropriate and to investigate the distributional properties of each of the 19 items comprising the SOCRATES. In order to replicate findings from Miller & Tonigan and Maisto et al. (Maisto et al., 1999; W.R. Miller & J.S. Tonigan, 1996), we performed an Alpha Factor Analysis (AFA) and Principal Component Analysis (PCA) of the 19 SOCRATES items, using orthogonal (varimax) rotation and forced the 3 factors and 2 components, respectively. When evaluating the factor and component structures, we retained items with component or factor loadings ≥ 0.4 and with factorial complexity of one (Nunally & Bernstein, 1994). In order to assess the potential impact of the inadvertent omission of item 16 of the SOCRATES by Maisto et al, we also performed analyses removing this item to allow comparison with their findings. As a confirmatory FA technique we computed the coefficient of congruence to compare matrix structures between datasets or studies (Cureton & D'Agostino, 1993). Internal consistency using Cronbach's alpha coefficients was assessed utilizing items that loaded on our factors or components. Cluster-based scores were created for each factor or component and concurrent validity was assessed for demographic, drinking and drug use measures using Pearson and Spearman correlation coefficients. All computations were performed using SAS Version 9.1. Unless otherwise specified, all statistical tests and/or confidence intervals were performed at α=0.05 (2-sided).

RESULTS

Participants

Of 986 medical inpatients who reported at-risk drinking amounts during screening, 341 enrolled. Subjects enrolled were more likely to be African American (45% vs 31%) and drank larger quantities of alcohol (median 24 vs 18 drinks per week) compared to eligible subjects who refused participation, but did not significantly differ on readiness to change measured with a 1 to 10 visual analog scale. Of enrolled subjects, 337 completed the SOCRATES and comprise our analytic sample. Baseline characteristics are presented in Table 1.

Table 1.

Subjects' baseline characteristics^* (n=337)

Demographics
	Women		99 (29.4%)
	Age, mean (SD) median		44.4 (10.7) 45.0
	Race/Ethnicity
		Black	153 (45.4%)
		White	132 (39.2%)
		Hispanic	29 ( 8.6%)
		Other	23 ( 6.8%)
Alcohol diagnosis (past year) †
	Alcohol abuse		15 (4.5%)
	Alcohol dependence		257 (76.3%)
	No diagnosis		65 (19.3%)
Alcohol consumption (past 30 days) mean (SD) median
	Drinks per day		6.8 ( 9.0) 3.7
	Days with heavy episodic drinking		12.8 (10.7) 9.0
Drug use (last 30 days)
	Heroin or cocaine use		88 (25.8%)

^*Number and percentage are presented for categorical variables, Mean (SD) and median are presented for continuous variables.

^†Determined with the Composite International Diagnostic Interview (CIDI) Alcohol Module. Alcohol consumption was assessed over the past 30 days with the Timeline Followback Heavy episodic drinking was defined as drinking more than 5 drinks per occasion for men and more than 4 drinks per occasion for women and persons >65 years.

Determination of the most appropriate factor structure solution in the study sample

Principal Component Analysis (PCA)

The PCA yielded 2 components based upon Kaiser's Rule (i.e., eigenvalue > 1). The first three eigenvalues were 8.97, 2.65, 0.91. The orthogonal component structure and associated loadings from the PCA with a varimax rotation are presented in Table 2. Sixteen of the 19 items comprising the SOCRATES were retained based on the criteria of having a component loading ≥ 0.4 and a factorial complexity of one. The first component, accounting for 37% of the item response variance, was comprised of 10 items: 3 that were originally described as Ambivalence, 6 as Recognition, and 1 as Taking Steps. Six items originally described as Taking Steps composed the second component, accounting for an additional 24% of the item response variation. The remaining items split a load between two components (items 1 and 19) or did not load on either component (item 11).

Comparison with previously published factor structure solutions

Maisto et al.

To compare our data and findings with those of Maisto et al., we repeated the PCA excluding item 16 (Maisto et al., 1999) (Table 3). In these analyses, the PCA using Kaiser's Rule yielded 2 factors (first three eigenvalues: 8.57, 2.62, 0.87). The first component is comprised of 11 items (using scale names from Miller & Tonigan): 2 Ambivalence, 7 Recognition and 2 Taking Steps, while the second component contains 8 items: 1 Recognition and 7 Taking Steps. Fifteen of the 18 SOCRATES items included in this procedure had a component loading ≥ 0.4 and a factorial complexity of one, and as with the prior analysis, two of the items (1 and 19) split a load between two components and item 11 did not load on either component. Analyses performed with a non-orthogonal rotation without item 16 yielded similar results.

Table 3.

Comparison with previously published component and factor structure solutions

	Present study		Maisto et al.		Present study			Miller & Tonigan
Item	C1	C2	C1 (AMREC)	C2 (TS)	F1	F2	F3	F1 (R)	F2 (TS)	F3 (A)
1. I really want to make changes in my drinking	0.49	0.46	*	*	0.48	0.42	−0.05	0.38	0.16	0.04
2. Sometimes I wonder if I am an alcoholic	0.72	0.00	0.58	0.26	0.67	0.05	−0.07	−0.07	−0.07	0.58
3. If I don't change my drinking soon, my problems are going to get worse	0.78	0.24	0.82	0.14	0.77	0.25	−0.02	0.6	0.12	0.05
4. I have already started to make some changes in my drinking	0.14	0.8	0.15	0.81	0.15	0.79	−0.13	0.15	0.73	0
5. I was drinking too much at one time, but I've managed to change my drinking	0.01	0.58	0.02	0.65	0.06	0.48	−0.1	−0.24	0.4	0.16
6. Sometimes I wonder if my drinking is hurting other people	0.68	0.23	0.58	0.29	0.67	0.25	−0.24	0.3	0.07	0.31
7. I am a problem drinker	0.88	0.09	0.84	0.12	0.85	0.1	0.05	0.61	0.03	0.04
8. I am not just thinking about changing my drinking, I am already doing something about it	0.26	0.8	0.18	0.85	0.25	0.77	0.1	0.25	0.69	−0.06
9. I have already changed my drinking, and I am looking for ways to keep from slipping back into my old pattern	0.16	0.8	0.26	0.8	0.15	0.77	0.1	0.09	0.81	−0.02
10. I have a serious problem with drinking	0.86	0.19	0.82	0.15	0.83	0.2	0.23	0.8	0.09	−0.02
11. Sometimes I wonder if I am in control of my drinking	0.38	0.27	0.63	0.22	0.37	0.25	−0.14	−0.06	0	0.55
12. My drinking is causing a lot of harm	0.79	0.23	0.79	0.11	0.76	0.24	−0.03	0.62	0.15	−0.01
13. I am actively doing things now to cut down or stop drinking	0.15	0.82	0.23	0.8	0.16	0.78	0.03	0.22	0.76	−0.04
14. I want help to keep from going back to the drinking problems that I had before	0.72	0.38	*	*	0.69	0.38	0.28	0.45	0.46	0
15. I know that I have a drinking problem	0.87	0.22	0.8	0.32	0.86	0.23	0.13	0.76	0.15	−0.03
16. There are times when I wonder if I drink too much	**	**	**	**	0.61	0.24	−0.24	0.06	0.04	0.66
17. I am an alcoholic	0.86	0.12	0.77	0.13	0.82	0.14	0.15	0.68	0.22	−0.18
18. I am working hard to change my drinking	0.32	0.79	0.37	0.78	0.32	0.78	0.05	0.28	0.76	−0.05
19. I have made changes in my drinking and I want some help to keep from going back to the way I used to drink	0.52	0.55	*	*	0.49	0.53	0.27	0.16	0.68	0.06
Variance Explained by each Component/Factor	37%	25%	48%	13%	35%	22%	2%	11%	27%	7%
Cronbach Standardized Coeffficient Alpha	0.92	0.88	0.91	0.89	0.93	0.89	0.72	0.85	0.83	0.60

C: component

F: factor

A: Ambivalence, R: Recognition, TS: Taking Steps, AMREC: Ambivalence and Recognition (see text)

^*item not retained in Maisto (did not load on one and only one factor at >0.40)

^**because of clerical error in original Maisto et al. study, item was not included in analyses

Miller and Tonigan

The hypothesized underlying three factor structure, based on Miller & Tonigan's published work, is displayed in Table 3 (W.R. Miller & J.S. Tonigan, 1996). This structure is comprised of 4 Ambivalence items (items 2, 6, 11 and 16), 7 Recognition items (items 1, 3, 7, 10, 12, 15 and 17), and 8 Taking Steps items (items 4, 5, 8, 9, 13, 14, 18 and 19). Utilizing Miller's approach with our data, the AFA yielded 2 factors using Kaiser's rule. However, for comparison, we assessed a forced 3 factor solution. The first three eigenvalues were 14.09, 3.81 and 0.80. The orthogonal factor structure and associated loadings from the AFA with a varimax rotation are presented in Table 3. Three of the 4 hypothesized Ambivalence items, all 7 of the hypothesized Recognition items, and 2 of the 8 hypothesized Taking Steps items loaded on the first factor. Seven of the 8 hypothesized Taking Steps items and 1 of the hypothesized Recognition items loaded on the second factor. No items loaded on the third factor. Analyses performed utilizing a non-orthogonal solution yielded similar results (not shown).

Confirmatory Factor Analytic Techniques

To compare our component and factor analytic structure with that described by Miller & Tonigan and Maisto et al., we estimated the coefficients of congruence (Maisto et al., 1999; W.R. Miller & J.S. Tonigan, 1996). Coefficients of congruence (CC) range from −1 to 1, with greater absolute coefficients indicating increased concordance between structures (Cureton & D'Agostino, 1993). Comparing the 3 factor structure solution from our data to the structure reported by Miller & Tonigan (W.R. Miller & J.S. Tonigan, 1996), the coefficients of congruence implied strong concordance between both of the Recognition factors and the Taking Steps factors (CC=.885 and .963, respectively) and a weak dissimiliarity between the Ambivalence factors (CC=−0.527, due to no items from our data loading on the third factor). For the 2 component structure compared to the solution published by Maisto et al., the coefficients of congruence were 0.988 (for comparison with Maisto et al.'s AMREC factor) and 0.985 (for comparison with their Taking Steps factor), for the first and second components from our data, implying excellent concordance. The 2 component solution proposed by Maisto et al is therefore more appropriate for our data in comparison to the 3 factor solution. A confirmatory factor analysis (also omitting item 16) for the 2 and 3 factor models also provided better fit for the 2 factor solution (not shown).

Concurrent validity

To assess concurrent validity we developed cluster-based scores based on the two component PCA solution and assessed the associations between these scores and important measures. Results are reported in Table 4. Both of the cluster-based scores are correlated with the presence of alcohol dependence, the presence of alcohol related problems, and Alcohol Use Disorder Identification Test (AUDIT) score. Component 1 consistently has stronger correlations with each of these alcohol related measures. In addition, component 1 is significantly correlated with alcohol consumption levels (average number of drinks per day and total number of heavy drinking days over the past 30 days). Neither component is significantly correlated with drug use.

Table 4.

Concurrent validity - Correlation^* of components with various clinical variables

Variable	First component	Second component
Demographics
Age	0.07	0.09
Gender	0.04	0.07
Alcohol Measures
Alcohol dependence diagnosis (DSM-IV)†	0.55**	0.30**
Alcohol related problems (SIP score)	0.73**	0.24**
AUDIT score	0.68**	0.21**
Alcohol consumption (average drinks per day)	0.39**	−0.02
Number of binge drinking days (past 30 days)	0.47**	0.01
Drug Use
Heroin or cocaine use (past 30 days)	0.10	0.07
Marijuana use (past 30 days)	−0.02	−0.06

^*Spearman correlation coefficients are presented for dichotomous variables, and Pearson correlation coefficients are presented for continuous variables.

^**p<0.05

^†Determined with the Composite International Diagnostic Interview (CIDI) Alcohol Module.

SIP: Short Inventory of Problems

AUDIT: Alcohol Use Disorder Identification Test

Alcohol consumption was assessed with the Timeline Followback

DISCUSSION

We examined the factor structure of the SOCRATES questionnaire in a population of adult men and women with unhealthy alcohol use identified by opportunistic screening, who were hospitalized in a general hospital and not attending specialized alcohol treatment.

We found a 2 component structure. We propose that the first component be named “Perception of Problems” (PP)(consisting of 10 items) and the second component be named “Taking Action” (TA)(consisting of 6 items). PP includes 3 items originally classified by Miller & Tonigan as Ambivalence, 6 as Recognition and one as Taking Steps. This component reflects the cognitive dimension of acceptance and recognition of alcohol problems. Item 14, “I want help to keep from going back to the drinking problems that I had before,” originally classified as Taking Steps, is part of PP. This could reflect that the acceptance of needing help is more a recognition of an underlying problem than an action statement. PP appears to reflect both the perception of problems related to alcohol drinking and a need for help. TA consists of 6 items originally described as Taking Steps, and appears to report actions that individuals are already doing in order to address their drinking problem. The desire to get help appears to be separate from taking actions to change drinking behavior.

In the PCA, 3 of the 19 items had component loadings ≥0.4 with a factorial complexity of one. Item 19 (“I have made changes in my drinking and I want some help to keep from going back to the way I used to drink”), originally classified as Taking Steps, is a composite question made of two statements, one on changes already made in drinking and the other on the desire to get help. It loaded ≥0.4 on both components, consistent with what the 2 factors appear to capture. Item 1 loaded ≥0.4 on both components, and item 11 did not load ≥0.4 on any component.

The assessment of concurrent validity indicates that PP (the first component) is correlated with alcohol consumption level (drinks per day and heavy drinking episodes). PP has stronger associations with the presence of alcohol-related consequences than TA (the second component). This is consistent with the interpretation that PP reflects perception of alcohol problems, but suggests also that PP could reflect the severity of the problems related to alcohol use.

The exploratory analysis results were reinforced by the comparative analysis. In comparative and confirmatory analyses, the most appropriate structure in our data was similar to that found by Maisto et al. and less similar to that found by Miller & Tonigan.

Our data support the evidence that the factor structure of the SOCRATES questionnaire may be dependent upon the population and the therapeutic setting in which the questionnaire is administered. The use of a 3 factor solution seems to be appropriate in specialized addiction and psychiatric settings, especially with alcohol dependent patients, even if this remains questionable since Figlie and colleagues demonstrated a 2 factor solution in a mixed population of patients from specialized setting and from a gastroenterology clinic (Demmel et al., 2004; Figlie et al., 2005; W.R. Miller & J.S. Tonigan, 1996). On the other hand, the use of a 2 factor solution seems more appropriate for patients screened opportunistically in general health settings such as primary care clinics, community samples or hospitals (Maisto et al., 1999) (Burrow-Sanchez & Lundberg, 2007). Among adolescents and young adults, published data are inconsistent in favor of one or the other structures (Maisto, Chung, Cornelius, & Martin, 2003; Vik, Culbertson, & Sellers, 2000). Contrary to the ambivalence and recognition constructs that the originally described Ambivalence and Recognition factors intended to capture, the concept captured in the factor called “Taking Steps” (measure of actions taken towards change, or change-related actions) in both the Maisto et al. and Miller and Tonigan studies is consistent across populations and settings.

The main strength of our study was the examination of the SOCRATES in a large sample of medical inpatients, adding to the literature on readiness to change in patients who are identified opportunistically, and are not seeking treatment (in contrast to studies in specialty addiction treatment settings). This is the first study to our knowledge investigating the factor structure in this population. Nevertheless, the generalizability of these results should be limited to hospitalized medical patients. The results are particularly applicable to those who agreed to participate in a clinical trial where they could receive alcohol counseling. It is possible that the subjects included in our sample were more motivated to change than were those who refused to participate. However, individuals who agreed to participate had similar readiness scores on a 1 to 10 visual analog scale compared to eligible subjects who refused participation.

In conclusion, our findings support the likelihood that the SOCRATES can assess and measure two important motivational constructs in patients identified by screening, who are not necessarily seeking nor receiving specialty alcohol treatment. One of these constructs, change-related actions, was consistently found across settings and populations. The first component identified in our sample (PP) reflects perception of problems and need for help, and the second taking action or change-related actions (TA). Nevertheless, identification of these two readiness-to-change constructs is of interest primarily as potential predictors of change or determinants of behavior change. The predictive validity of the 2 components and their relationship with behavior change need to be further explored. Since only about 5% of individuals with alcohol dependence seek and receive treatment, having tools that help researchers to better study the 95% who do not seek help is important and relevant, particularly when it is assumed that seeking treatment is related to motivation and problem recognition.