Skip to main content
Translational Behavioral Medicine logoLink to Translational Behavioral Medicine
. 2019 Feb 4;10(2):457–468. doi: 10.1093/tbm/ibz001

Cross-validation of transtheoretical model smoking cessation measures in Chicago WIHS women smokers with and at risk for HIV

Colleen A Redding 1,, David Goldberg 2, Kathleen M Weber 3, Hui-Qing Yin 1, Andrea L Paiva 1, Jane Burke-Miller 3, Mardge H Cohen 2,3, Joseph S Rossi 1
PMCID: PMC7237546  PMID: 30715533

Abstract

People with and at risk for HIV have high rates of smoking, increasing their morbidity and mortality. Effective cessation interventions are needed for this group. Transtheoretical model (TTM)-tailored interventions have demonstrated efficacy, but measures need cross-validation in this population. TTM cessation measures were evaluated in women smokers with and at risk for HIV (N = 111) from Chicago Women’s Interagency HIV Study (WIHS). Confirmatory factor analyses evaluated measurement models. MANOVAs examined relationships between constructs and stage subgroups. For decisional balance, the two-factor uncorrelated model was best (χ2(20) = 13.96; comparative fit index [CFI], 1.0; root mean square error of approximation [RMSEA] = .00), with good (pros α = .78) and fair (cons α = .55) four-item alphas. The one-factor temptations model (α = .90) showed reasonable fit (χ2(18) = 80.22; CFI = .89; RMSEA = .177). Processes of change subscales had fair to good two-item alphas (α = .49–.77) and fit a 10-factor fully correlated model (χ2(125) = 222.72; CFI = .88; RMSEA = .084). MANOVAs by stage of change replicated expected patterns for the pros, overall temptations, and two process subscales with medium-sized effects (η2 = .06–.18). Contrary to expectations, no differences by stage were found for cons or temptation negative affect subscales. The structures of these TTM measures replicated with good internal and external validity, except for the cons, which needs refinement. Negative affect temptations was structurally sound, but did not vary by stage group potentially reflecting this sample’s moderate depression levels and/or their reliance on smoking to deal with negative affect. Results support the use of most TTM measures in research and tailored interventions to increase smoking cessation among women smokers with and at risk for HIV and highlight the importance of managing negative affect in cessation materials targeting this group.

Keywords: Transtheoretical model, Smoking cessation, Measurement validation, Adult women, HIV+, TTM


Chicago women smokers with and at risk for HIV and high rates of stress/depression used theory-based smoking cessation measures similarly to other smokers, supporting future use of these measures.


Implications

Practice: Transtheoretical model (TTM) smoking cessation measures, including stages of change, pros of smoking, temptations to smoke, and processes of change, can be used to assess and provide feedback on readiness to quit smoking in samples of adult women smokers with and at risk for human immunodeficiency virus (HIV).

Policy: Policymakers interested in enhancing cessation in high-risk groups such as those at risk for and living with HIV can consider TTM-tailored interventions.

Research: Future research should examine a range of TTM-tailored cessation interventions in samples at risk for and living with HIV to enhance quitting in this high-risk group.

INTRODUCTION

Smoking remains a major public health problem in the United States, with implications for cardiac, pulmonary, cancer-related, and other chronic diseases. In the United States, poverty, limited education, and minority group status are associated with higher rates of both tobacco smoking [1] and human immunodeficiency virus (HIV) infection [2, 3], resulting in disproportionate disease burden, among other disparities. Some of the highest rates of smoking in the United States have been reported among individuals with or at risk for HIV [4–6]. Between 2009 and 2014, the general U.S. adult smoking rate declined from 20.6% (confidence interval [CI]: 19.9–21.3) to 16.8% (CI: 16.2–17.4), whereas the smoking rate in individuals living with HIV declined from 37.6% (CI: 34.7–40.6) to 33.6% (CI: 29.8–37.8) [5, 6]. Within the Women’s Interagency HIV Study (WIHS) cohort of women with and at risk for HIV, smoking rates also declined over time from a high of 57% in 1995 to 39% in 2011 [5]. Rates of cessation also vary widely by ethnicity, poverty, education, and insurance status [5–8]. Women with and at risk for HIV have high smoking rates, low rates of cessation [5–9], and disproportionate health and socioeconomic challenges and together, these underscore the urgent need for effective interventions to increase cessation in this cohort.

One important theoretical framework that has been widely used in smoking cessation research is the transtheoretical model (TTM) of behavior change, a model of intentional behavior change that integrates constructs across other health behavior models [10, 11]. Key TTM constructs include stage of change (readiness), decisional balance (pros and cons of smoking), temptations, and the processes of change. These TTM constructs and their interrelationships have been previously validated, specifically for smoking cessation [12–14]. In addition, TTM-tailored cessation interventions [15] using these measures have demonstrated efficacy in various samples and populations [16–18]. Despite a good amount of TTM cessation research, these smoking cessation measures have minimal validation in minority populations [19–23] and none in cohorts with and at risk for HIV. Cross-validating existing TTM cessation measures in diverse, comorbid, low-socioeconomic status, and underserved samples, such as women smokers with and at risk for HIV, is needed.

This study addressed this gap by cross-validating all key TTM construct measures in a representative sample of women smokers with and at risk for HIV. The structure or the relationships between TTM constructs were not expected to be different in this population. However, examining the validity of these measures remains important, especially since these measures can be used for empirical decision making and tailored intervention recommendations (i.e., based on individual’s scores). Investigating hypothesized measurement model fit and the functional relationships between these constructs and stage of change provides a necessary empirical foundation for tailored interventions. If the structure or relationships differ in this sample, it might be necessary to further refine these measures for this population and/or alter decision-making rules [21, 24]. It was hypothesized, based on expected factor structure and direction of relationships from previous TTM studies, that in this sample: (a) the TTM constructs would reveal adequate fit to the theorized measurement models for decisional balance (two-factor uncorrelated model for the pros and cons of smoking), temptations (one-factor model), and processes of change (10-factor, fully correlated model), and (b) relationships between all constructs and stage of change would replicate previously found relationships, although with reduced effect sizes because this sample only included women smokers in the first three stages of cessation (Precontemplation, Contemplation, and Preparation).

METHODS

Procedures

Eligible participants for this study were Chicago WIHS CORE Center cohort members who spoke English, were at least 18 years old, were in the first three stages for smoking cessation (i.e., current smokers), and agreed to participate after providing written informed consent. Baseline TTM measures were assessed from September 2005 to January 2007 directly from participants using a computer expert system [21, 24]. All study procedures were approved by the WIHS Executive Committee and both the Cook County Health and Hospital System and the University of Rhode Island Institutional Review Boards. This study used the baseline visit of a 2-year pilot smoking cessation intervention study conducted within the Chicago WIHS, funded by the National Cancer Institute. WIHS is a prospective cohort study of HIV infected and demographically similar uninfected women in five US regions with enrollment between 1994 and 1995 and again in 2001–2002. WIHS study methods have been described [25, 26], but, in brief, women with and at risk for HIV were seen every 6 months by trained study personnel who conducted an extensive interview and brief physical exam and collected blood and gynecologic specimens.

Measures

At WIHS enrollment, participant date of birth, self-reported race and ethnicity, and highest education level attained were assessed. HIV status was determined by ELISA and an approved confirmatory test if positive. All women who tested HIV negative were retested at each follow-up visit. The semiannual WIHS core study visit corresponding to this study’s baseline included self-report measures of sociodemographics (income, employment, health insurance, marital, and housing status) health status and behaviors (licit and illicit substance use), depressive symptoms (CES-D) [27], health-related quality of life (MOS-HIV) [28], perceived stress (PSS) [29], nicotine dependence (FTND) [30], recent hospitalization, and use of highly active antiretroviral therapy to treat HIV. Height and weight were measured and used to calculate body mass index. Hypertension was assessed by current measures of systolic and diastolic blood pressure greater than or equal to 140/90, current use of antihypertensive medications, and/or confirmed history of hypertension diagnosis. Diabetes was assessed by current fasting blood glucose tests greater than or equal to 126, current use of hypoglycemic medications, and/or confirmed history of diabetes diagnosis. HIV-specific measures included CD4+ cells/mm3 and HIV-1 RNA viral load measured from blood samples using immunofluorescence flow cytometry and anisothermal nucleic acid sequence-based amplification method performed in AIDS Clinical Trials Group (ACTG) certified and National Institutes of Health Viral Quality Assurance Program participating laboratories, respectively.

Stage of change

Stage of change was measured using an algorithm assessing readiness to quit smoking, with response options reflecting Precontemplation (PC) (not intending to quit in the next 6 months), Contemplation (C) (intending to quit in the next 6 months), Preparation (PR) (intending to quit in the next 30 days), Action (A) (quit for <6 months), and Maintenance (M) (quit for 6 months or more). The reliability, utility, and predictive validity of this algorithm have been demonstrated in other samples [10, 31, 32].

Decisional balance

An eight-item decisional balance measure assessed the relative importance of various perceived advantages (pros) and disadvantages (cons) of smoking. Participants rated the importance of each item in their decision to smoke on a five-point scale, ranging from 1 = not at all important to 5 = extremely important [12]. Both four-item subscales, pros and cons, have been found internally consistent and significantly related to stage in primarily white populations [12, 20].

Temptations

A nine-item temptations measure assessed individuals’ temptations to smoke in three types of challenging situations: negative affect, habit strength, and positive social [14]. Participants rated their temptation to smoke on a five-point scale ranging from 1 = not at all tempted to 5 = very tempted. The overall temptations to smoke measure and each three-item subscale have been found to be internally consistent and demonstrated known groups validity by stages of change, with the last two stages (A, M) differing significantly from the first three stages (PC, C, PR).

Processes of change

A 20-item measure assessed the 10 common cognitive, affective, experiential, and behavioral strategies used by individuals to progress through the stages of change. Individuals rated the frequency of use of each item over the past 30 days on a five-point scale ranging from 1 = never to 5 = repeatedly. This measure demonstrated internal consistency and known group validity by stage of change in primarily white samples [13] with coefficient alphas for the 10 subscales ranging from .64 to .86. Because this study only included women smokers in the first three stages of cessation, more clear differentiation (statistically significant differences and/or moderate effect sizes) between the experiential processes (Consciousness Raising [CR], Dramatic Relief [DR], Environmental Reevaluation [ER], Self-reevaluation [SR], Social Liberation [SO]) was expected than between the behavioral processes (Counterconditioning [CC], Helping Relationships [HR], Reinforcement Management [RM], Stimulus Control [SC], Self-liberation [SL]), which generally differentiate more in later stages [13].

Analysis

There were two sets of analyses, testing the best-fitting confirmatory structural measurement models for the decisional balance, temptations, and processes of change measures [33–35] using EQS version 6.1 [36, 37]. The second set of multivariate analysis of variance (MANOVA) determined if the hypothesized functional relationships between each construct and stages of change would replicate, supporting the known group and external validity of the scales in this sample using SPSS, version 22 [35, 38].

Confirmatory factor analyses

To establish the best-fitting model for each of the confirmatory factor analysis (CFA) procedures, several different macro fit indices were compared. These included the (a) comparative fit index (CFI), (b) average absolute standardized residual (AASR), (c) root mean square error of approximation (RMSEA), and (d) likelihood ratio χ2 test statistic [34, 36]. For the goodness of fit (GFI) and CFI, values of .80 to .89 on the GFI and CFI indicate adequate fit, whereas values of .90 and greater indicate good or excellent fit [39]. For the AASR and RMSEA, values less than .06 indicate excellent fit [33]. In addition, individual item factor loadings were examined and expected to be greater than .40.

For the decisional balance measure, four confirmatory structural models were compared, including the null model, a two-factor uncorrelated model, a two-factor correlated model, and a general one-factor decisional balance model. Two models were compared for the temptations measure, which included the null model and a single-factor model. Last, for the processes of change measure, three models were compared. These included the null model, a two-factor correlated model representing only the combined experiential and behavioral processes, and a 10-factor fully correlated model.

Known group validation

Functional relationships between decisional balance, temptations, and processes of change were examined across the three stage of change groups (PC, C, PR), using a MANOVA with ANOVA and Tukey follow-up tests. Eta square was used to estimate effect sizes, ranges from zero to one, and can be interpreted as the proportion of the dependent variable that is attributable to the independent variable [40, 41]. Cohen categorized eta-square values descriptively as small (.01), medium (.06), or large (.14), respectively [41].

RESULTS

Sample

Of 118 English-speaking smokers who were recruited to participate in this study during a WIHS core study visit between August 2005 and September 2007, 115 (97%) were enrolled. English-speaking smokers over 18 years old were offered study participation at their WIHS study visit (see Procedures for recruitment details). Subsequently, four women were excluded because 1 had quit smoking already and the other 3 did not complete baseline assessments. These analyses include the remaining 111 participants; a participation rate of 95% (111/117). Demographic, behavioral, and HIV-related characteristics of study participants are shown in Table 1. Three quarters of participants (83/111, 75%) were HIV seropositive. Most were African American (89%), with 5% White, 5% Hispanic/Latina, and <1% other racial/ethnic group. Almost half (48%) had less than a high school education, 28% had a high school diploma or GED and 24% had more than a high school education. Age at study entry ranged from 22 to 61 years, with a mean age of 41 years (SD = 8 years). About a quarter (26%) were married or living as married with a partner, and most (73%) lived in their own homes or apartments. Only 24% were employed full time or part-time, and the majority (75%) reported an annual household income of $12,000 or less per year. Over three quarters (78%) had some form of health insurance.

Table 1.

Descriptive and demographic characteristics (N = 111)

Characteristic % (n) Mean (SD)
HIV positive 75 (83)
Age, years 41 (8)
Race/ethnicity 89 (99)
 Black (non-Hispanic) 89 (99)
 Hispanic 5 (5)
 White (non-Hispanic) 5 (6)
 Other racial/ethnic group <1 (1)
Education level
 Less than high school 48 (53)
 High school graduate or equivalent 28 (31)
 More than high school 24 (27)
Married/living as married 26 (28)
Stable housing (own home/apartment) 73 (81)
Employed (full time or part-time) 24 (27)
Household income ≤ $12,000/year 75 (83)
Health insurance 78 (86)
Substance use
 Alcohol (number of drinks per week)
  None 48 (53)
  Light (<7/week) 39 (44) 3 (6)
  Hazardous (>6/week) 13 (14)
 Marijuana use
  Never 14 (16)
  Former 59 (65)
  Current 27 (30)
 Crack, cocaine, and heroin use
  Never 16 (18)
  Former 57 (63)
  Current 27 (30)
 Intravenous drug use
  Never 60 (67)
  Former 38 (42)
  Current 2 (2)
Psychosocial measures
 Probable depression (CES-D score ≥ 16) 56 (62)
 Health-related quality of life 66 (20)
 Perceived stress scale* 15 (7)
Physical health
 Body mass index
  Obese (≥30) 25 (28)
  Overweight (25–29.9) 32 (36)
  Normal (18.5–24.9) 38 (42)
  Underweight (<18.5) 5 (5)
 Hypertension 29 (32)
 Diabetes 15 (17)
 Hospitalized in prior year 22 (24)
Smoking related
 Fagerstrom score 4.2 (2.3)
 Number cigarettes smoked per day 8 (7)
 Any quit attempt in prior 3 years 37 (41)
 Stage of cessation
  Precontemplation 34 (38)
  Contemplation 47 (52)
  Preparation 19 (21)
HIV-related characteristics (n = 83)
 CD4 cells/µL
  <200 22 (18) 430 (294)
  200–349 22 (18)
  350–499 30 (25)
  >500 26 (22)
 HIV RNA copies/mL
  Detectable 69 (57) 42,342 (175,664)
  Undetectable 31 (26)
 Highly active antiretroviral therapy 64 (53)

*Perceived stress scores were available on n = 88.

Almost half of participants reported no alcohol use (48%), 39% reported light or moderate drinking, and 13% were potentially hazardous drinkers, defined for women by the National Institute of Alcohol Abuse and Alcoholism as more than six drinks per week; average alcohol use was three drinks per week (SD = 6); 27% were current marijuana users and another 59% reported former use; 27% were current crack, cocaine, or heroin users, and another 57% reported former use; although 38% reported histories of intravenous drug use, only 2% reported current use.

Just over half the sample (56%) screened positive for probable depression, with a CES-D score greater than 16 [25]. Health-related quality-of-life scores (MOS-HIV) were relatively low with a mean of 66 (SD = 20) on a scale that ranged from 0 to 100 [26]. Perceived stress scores were moderate with a mean of 15 (SD = 7) on a scale ranging from 0 to 36 [27].

More than half of participants were either overweight or obese. Almost a third (29%) had at least one clinical indicator for hypertension including systolic blood pressure ≥ 140, diastolic blood pressure ≥ 90, use of antihypertension medications, or self-report of clinical diagnosis; and 15% had clinical indications for diabetes including fasting blood glucose ≥ 126, hemoglobin A1c test ≥ 6.5%, use of diabetes medications, or self-report of clinical diagnosis. Almost a quarter (22%) had been hospitalized within the past year.

Among HIV+ participants, the mean CD4 count was 430 cells/µL (SD = 294); 74% were less than 500 cells/µL. Mean HIV viral load was 42,342 copies/mL (SD = 175,664) and 69% of participants had detectable levels. Almost two thirds (64%) reported use of highly active antiretroviral therapy.

Participants reported smoking an average of eight cigarettes per day (SD = 7), and 37% reported quit attempts in the past 3 years. The mean Fagerstrom Nicotine Dependency Score [28] was 4.2 (SD = 2.3) with a range from 0 to 9. The distribution of stages of change for smoking cessation was Precontemplation, 34% (n = 38); Contemplation, 47% (n = 52); and Preparation, 19% (n = 21). Stages of change did not vary significantly by HIV serostatus, χ2(2) = 3.62, p = .164.

Transtheoretical model measures

The scale statistics for the TTM measures of decisional balance (pros and cons of smoking), temptations, and processes of change are reported in Table 2. Internal consistencies ranged from acceptable to very good, based on Cronbach’s alphas between .49 and .90. One item on the cons of smoking scale, “Smoking cigarettes is hazardous to my health” had a relatively high importance rating, with a mean of 4.50 on a scale ranging from 1 to 5. Also, another item on the cons scale “I’m embarrassed to have to smoke” had a relatively low importance rating, with a mean of 2.23.

Table 2.

TTM construct scale and subscale means ± SD and Cronbach’s alphas

Scale Number of items Scale/subscale mean (SD) Cronbachs α
Decisional balance
  Pros of smoking 4 2.98 (0.96) .78
  Cons of smoking 4 3.47 (0.76) .55
Temptations 9 3.72 (0.92) .90
  Negative affect 3 4.04 (0.95) .87
  Positive social 3 3.68 (0.98) .70
  Habit strength 3 3.44 (1.06) .71
Processes of change
 Experiential processes 10 3.00 (0.71) .83
  Consciousness raising 2 2.71 (0.93) .59
  Dramatic relief 2 2.79 (0.91) .53
  Environmental reevaluation 2 3.11 (1.02) .76
  Social liberation 2 3.58 (0.99) .66
  Self-reevaluation 2 2.80 (1.07) .70
 Behavioral processes 10 2.48 (0.72) .82
  Counterconditioning 2 2.58 (0.84) .49
  Helping relationships 2 2.67 (1.11) .74
  Reinforcement management 2 2.34 (1.15) .77
  Stimulus control 2 1.91 (0.90) .65
  Self-liberation 2 2.88 (1.06) .68

Confirmatory factor analyses

CFAs were conducted on the decisional balance, self-efficacy, and processes of change measures. For decisional balance, three alternative models besides the null model were assessed. The one-factor model showed a reasonable fit to the data (χ220 = 49.48; p < .001; CFI = .80; GFI = .98; AASR = .06; RMSEA = .12). The best-fitting models proved to be both the two-factor correlated model (χ219 = 13.42; p = .81; CFI = 1.0; GFI = .97; AASR = .027; RMSEA = .00), and the two-factor uncorrelated model (χ220 = 13.96; p = .83; CFI = 1.0; GFI = .97; AASR = .030; RMSEA = .00). A χ2 difference test comparing the correlated and uncorrelated models was not significant (χ21 = 0.54; p > .05). The fit indices of the correlated and uncorrelated models were nearly identical; however, the uncorrelated model requires fewer parameters to be estimated. The correlation of 0.10 estimated between the pros and cons scales was very low and not significant. Thus, based on parsimony, the uncorrelated model was selected as best fitting. This model, including items and factor loadings is shown in Figure 1. Most factor loadings were adequate to good ranging from .56 to .75. Two cons items had poor loadings (.18 and .42). Removing the poorest loading item (.18) increased the internal consistency of the cons scale to α = .59.

Fig. 1.

Fig. 1

Two-factor uncorrelated decisional balance confirmatory model with standardized factor loadings (N = 111).

Three measurement models were tested for the temptations measure: (a) the null model, (b) a one-factor model with uncoupled residuals, and (c) a one-factor model with residuals correlated within subscales. The one-factor model with uncoupled residuals provided an adequate fit to the data (χ227 = 130.23; p < .001; CFI = .82; GFI = .80; AASR = .055; RMSEA = .186). The one-factor model with coupled residuals showed the best fit to the data, χ218 = 80.22; p < .001; CFI = .89; GFI = .87; AASR = .040; RMSEA = .177. A χ2 difference test comparing both models was significant (χ29 = 50.01; p < .001), confirming the three theoretically implied subscales. Factor loadings were good, ranging from .60 to .88. Figure 2 displays this model, including items and factor loadings.

Fig. 2.

Fig. 2

One-factor temptations confirmatory model with standardized factor loadings (N = 111).

Three measurement models were compared for the processes of change measure: (a) the null model, (b) the correlated two-factor (experiential and behavioral) model with coupled residuals, and (c) a 10-factor fully correlated model. The correlated two-factor model did not fit the data well (χ2159 = 342.14; p < .001; CFI = .78; GFI = .76; AASR = .065; RMSEA = .102). The 10-factor fully correlated model showed the best fit to the data (χ2125 = 222.72; p < .001; CFI = .88; GFI = .84; AASR = .050; RMSEA = .084). Factor loadings ranged from .46 to .99, and two-item coefficient alphas ranged from .49 to .77. Figure 3 includes all items, factor loadings, and correlations between processes.

Fig. 3.

Fig. 3

Ten-factor correlated processes of change model with standardized factor loadings (N = 111).

External validation

MANOVA was conducted to determine whether TTM constructs differed by the three stage subgroups. A MANOVA assessing decisional balance (pros and cons of smoking), temptation subscales (negative affect, positive social, habit strength), and 10 processes of change revealed a substantial and significant main effect for stage (Wilks’ Λ = .59; F30,188 = 1.88; p < .01; multivariate η2 = .408).

Scale means for the pros and cons are shown in Table 3. Follow-up ANOVAs found that the pros significantly differed by stage (F2,108 = 4.91; p < .01; η2 = .083). Participants in Precontemplation reported significantly higher pros of smoking than those in Contemplation and Preparation. The cons of smoking did not vary significantly (F2,108 = 0.35; p > .05) by stage.

Table 3.

Standardized T-scores ± SD for decisional balance and temptation by stage

TTM construct n Mean (SD) 95% confidence interval η2 Post hoc Tukey HSD
Pros of smoking* .083 PC > C, PR
 Precontemplation 38 53.98 (8.96) [50.88, 57.01]
 Contemplation 52 48.01 (9.48) [45.35, 50.66]
 Preparation 21 47.73 (11.25) [43.55, 51.91]
Cons of smoking .006
 Precontemplation 38 49.18 (9.09) [45.94, 52.41]
 Contemplation 52 50.01 (10.63) [47.24, 52.77]
 Preparation 21 51.48 (10.27) [47.12, 55.83]
Temptations total* .079 PC > PR
 Precontemplation 38 52.96 (8.73) [49.85, 56.08]
 Contemplation 52 49.88 (10.27) [47.22, 52.54]
 Preparation 21 44.94 (9.82) [40.75, 49.12]
Negative affect .028
 Precontemplation 38 51.83 (9.24) [48.63, 55.02]
 Contemplation 52 49.86 (10.08) [47.13, 52.60]
 Preparation 21 47.03 (10.83) [42.73, 51.34]
Positive/social* .061 PC > PR
 Precontemplation 38 52.70 (9.13) [49.56, 55.84]
 Contemplation 52 49.78 (10.09) [47.10, 52.47]
 Preparation 21 45.65 (10.10) [41.42, 49.88]
Habit strength** .123 PC, C > PR
 Precontemplation 38 53.55 (8.44) [50.51, 56.59]
 Contemplation 52 50.01 (10.17) [47.41, 52.61]
 Preparation 21 43.55 (9.32) [39.46, 47.63]

Eta-square values can be categorized as small (.01), medium (.06), or large (.14) [40, 41].

F-test significant at *p < .05; ** p < .01.

Table 3 also shows the ANOVA and Tukey test results for overall temptations, which were significantly different by stage (F2,108 = 4.66; p < .05; η2 = .079), with participants in Preparation reporting significantly lower temptations than those in Precontemplation. Interestingly, Table 3 shows that all three subscales (negative affect, positive social, habit strength) showed similar patterns of decrease across stage subgroups; however, the negative affect subscale was generally more highly endorsed at each stage of change and differences between stage groups were not significant. Significant stage differences were found for both the positive social (F2,108 = 3.54; p < .05; η2 = .061) and habit strength (F2,108 = 7.58; p < .01; η2 = .123) subscales, with participants in Preparation reporting significantly lower temptations than those in the earlier stages.

Processes of change subscale means, and follow-up ANOVA and Tukey results are presented in Table 4. Follow-up ANOVAs on subscales found significant stage group differences on 2 out of 10 subscales, Self-liberation (F2,108 = 11.57; p < .001; η2 = .176) and Self-reevaluation (F2,108 = 3.70; p < .05; η2 = .064), with participants in the Preparation stage reporting more frequent process use than those in earlier stage of change groups. Table 4 shows that although not statistically significant, small-to-medium effects sizes were observed across stage groups for an additional five processes of change (CR, DR, SO, CC, SC). In contrast, for three processes (ER, HR, RM), effect sizes across stage groups were smaller than expected and near zero.

Table 4.

Processes of change subscale standardized T-scores ± SD by stage

Process subscale n Mean (SD) 95% confidence interval η2 Post hoc Tukey HSD
Consciousness raising .040
 Precontemplation 38 48.27 (7.86) [45.09, 51.45]
 Contemplation 52 49.68 (10.65) [46.97, 52.40]
 Preparation 21 53.91 (11.16) [49.63, 58.18]
Dramatic relief .042
 Precontemplation 38 47.22 (8.79) [44.04, 50.39]
 Contemplation 52 51.11 (9.92) [48.40, 53.83]
 Preparation 21 52.28 (11.52) [48.00, 56.55]
Environmental reevaluation .001
 Precontemplation 38 49.80 (10.10) [46.55, 53.04]
 Contemplation 52 50.31 (10.06) [47.54, 53.08]
 Preparation 21 49.59 (10.13) [45.23, 53.96]
Social liberation .025
 Precontemplation 38 47.99 (9.90) [44.79, 51.19]
 Contemplation 52 50.54 (9.94) [47.80, 53.28]
 Preparation 21 52.30 (10.14) [47.99, 56.61]
Self-reevaluation* .064 PR > PC
 Precontemplation 38 47.35 (8.85) [44.21, 50.49]
 Contemplation 52 50.10 (9.75) [47.41, 52.78]
 Preparation 21 54.56 (11.30) [50.34, 58.78]
Counterconditioning .048
 Precontemplation 38 48.62 (8.87) [45.46, 51.79]
 Contemplation 52 49.20 (10.07) [46.50, 51.91]
 Preparation 21 54.46 (10.93) [50.20, 58.72]
Helping relationships .001
 Precontemplation 38 49.64 (10.49) [46.40, 52.89]
 Contemplation 52 50.10 (9.55) [47.33, 52.88]
 Preparation 21 50.39 (10.65) [46.03, 54.75]
Reinforcement management .003
 Precontemplation 38 50.68 (10.22) [47.44, 53.92]
 Contemplation 52 49.87 (9.83) [47.10, 52.64]
 Preparation 21 49.10 (10.42) [44.74, 53.45]
Stimulus control .049
 Precontemplation 38 47.28 (9.01) [44.12, 50.45]
 Contemplation 52 50.63 (10.20) [47.93, 53.34]
 Preparation 21 53.34 (10.37) [49.09, 57.60]
Self-liberation*** .176 PR > C > PC
 Precontemplation 38 44.90 (8.99) [41.95, 47.84]
 Contemplation 52 51.10 (9.01) [48.59, 53.62]
 Preparation 21 56.0 (9.82) [52.54, 60.46]

Eta-square values can be categorized as small (.01), medium (.06), or large (.14) [40, 41].

F-test significant at *p < .05; ***p < .001.

DISCUSSION

This study cross-validated most existing TTM cessation measures in this important population of women smokers with and at risk for HIV infection with multiple barriers to cessation, providing an empirical foundation for future tailored interventions. The WIHS cohort, a representative sample of US women with and at risk for HIV, provides an important opportunity to refine our understanding of high risk, comorbid smokers. These women smokers in the Chicago WIHS sample were mostly African American, poor, with limited educational attainment, and either uninsured or on Medicaid, each associated with poorer smoking cessation outcomes [4–9]. Many also had a history of and/or current substance use and current symptoms of stress and depression, even when compared with other mixed gender HIV+ samples of smokers [42]. Moreover, this was a group of relatively light smokers (fewer than 10 cigarettes per day). Behavioral interventions would be the mainstay of their smoking cessation treatments because they smoke below the threshold for routine prescription of cessation medications [43]. This study cross-validated three well-established TTM cessation measures: decision balance, temptations, and processes of change in this high-risk cohort of female smokers. Confirmatory analyses for the decisional balance, temptations, and processes of change measures demonstrated factor structures consistent with those found in other samples and indicated generally good model fit. Results from this comparison and evaluation of alternative structural models for each construct confirmed the structures and internal validity of these three measures in this sample. In addition, most measures showed good known groups external validity by three stages of change. In an independent randomized trial of nicotine replacement therapy with male and female HIV+ smokers, two of these measures (decisional balance and temptations) functioned as mediators of 6-month cessation outcomes [42].

Decisional balance

This study replicated a two-factor (pros and cons) uncorrelated measurement structure for the decisional balance instrument in this sample, consistent with prior results showing that the pros and cons were orthogonal, and the scales showed adequate to good internal consistency. This measurement structure replicated the structure found for this scale previously in other samples [12, 20, 22]. These results suggested that, like other samples, these women discriminated between the positive and negative aspects of deciding to quit smoking. The pros of smoking varied by these three stage of change, replicating previous findings and TTM predictions, supporting the external validity of this subscale. However, this four-item cons of smoking scale had both a low alpha level (.53) and contrary to TTM predictions, did not vary significantly across these three early stages. Two cons items (“I’m embarrassed to have to smoke.” and “Smoking is hazardous to my health.”) had poor item loadings, suggesting that in the context of HIV, these items did not function well. One item was endorsed so highly (… hazardous to my health) that it created a ceiling effect, limiting its ability to assess scale variability. This high mean score may reflect unique attributes of the women in this study who were participants for many years in ongoing health research or it may reflect the increased importance of their health, partially in response to HIV. The remaining item (… embarrassing) was endorsed at a very low level and loaded very poorly, suggesting that in the context of these women’s experiences of HIV and associated stigma, smoking may not actually be so embarrassing. Consideration of additional HIV specific (e.g., viral load) and other (e.g., costs) items may improve the functioning of this scale in samples like this. Improving the cons of smoking measure would be worthwhile because decisional balance has shown robust relationships to stages of change both cross-sectionally [44–47] and longitudinally [32, 47] and can provide an important contribution to tailored feedback messages [15, 24]. Better measurement of the cons of smoking may show results more comparable to previous studies; however, this remains to be demonstrated.

Temptations

This study confirmed a one-factor model for smoking temptations in this sample. These results replicated the underlying structure found in previous studies of smoking temptations in other samples [14]. In addition, temptations varied across the first three stage of change consistent with TTM predictions [14, 48] and replicated previous studies in adults and adolescents [14, 22]. As expected, participants’ temptations to smoke were highest in Precontemplation and lower among those in Preparation. Interestingly, the only subscale that did not vary significantly here by stages of change was negative affect, which was more highly endorsed than the other subscales. Temptations to smoke in response to stress, anxiety, and/or depression may be especially important for these women who endorsed moderate levels of depression and faced many health and social stressors. Additional cessation interventions targeting these stressors and/or providing effective mood management strategies may prove useful in improving and sustaining cessation efforts in these smokers. TTM-tailored smoking cessation programs have replicated their efficacy in two other samples of smokers receiving outpatient depression treatment and even, inpatient psychiatric care [49, 50]. Given levels of depression in this sample, effective treatments for depression [51] may also prove useful to address both depression and potentially augment cessation. These results support the use of this measure for both assessing temptations to smoke and tailored intervention purposes in this female sample of smokers with and at risk for HIV.

Processes of change

This study confirmed the structure of a 10-factor correlated model for the processes of change measure for smoking in this sample, replicating the structure found in previous studies in other samples [13]. This sample included women in only the first three stages of cessation, and only two process subscales (SR, SL) varied significantly between these three stages, with moderate- to large-sized effects. More variability in process subscales may be expected across the full range of stage groups and over time in larger samples, followed longitudinally. As expected, participants in Precontemplation used both process subscales significantly less often than those in Preparation. The large effect sizes of both subscales, Self-reevaluation and Self-liberation, suggest that they may be especially useful targets for tailored interventions for early-stage women smokers with or at risk for HIV. These results suggest that additional tailoring based on Self-reevaluation and Self-liberation, addressing both self-image and choice, may be useful enhancements to cessation programs for these women. Although many subscale analyses were not able to find even medium-sized effects significant due to small cell sample sizes, the effect sizes of most remaining subscales were generally consistent with previous findings and TTM predictions, supporting the known group external validity of this measure. The exception to this finding occurred for three process subscales, environmental reevaluation, helping relationships, and reinforcement management, each of which demonstrated little variability across these three stage subgroups. This finding may reflect a lack of social and environmental supports for quitting experienced by these women, and it suggests that providing additional intervention components aimed at improving health-related and social supports for quitting may also be indicated.

Limitations

There are several limitations to the present study, with the foremost being the sample size, especially in stage subgroups, and the cross-sectional nature of these data. These scales would benefit from more longitudinal analysis and should be validated in additional samples of smokers with multiple social and behavioral challenges, including HIV, poverty, low levels of education, substance use, and depression. Another limitation of this study is that only final items from the original scales were used. Thus, the original item content did not include more specific factors that may be especially salient for women smokers with or at risk for HIV. Additional constructs of interest that were not included in this study but may prove useful in the future include stigma, social support, and social determinants of health. More in-depth qualitative research may yield more valuable information about smoking and the role it may play in these women’s lives, suggesting measurement and intervention enhancement opportunities. This may be particularly true for the cons scale, which did not replicate the results of previous studies and should be improved. The final limitation was that this sample was recruited from one site in Chicago about a decade ago; future research would benefit from including updated, more demographically and geographically representative samples of male and female smokers with and at risk for HIV.

CONCLUSION

The results of this study support the internal and external validity of the decisional balance, temptations, and processes of change measures for smoking cessation in female smokers with and at risk for HIV, including predominantly poor African American women with lower levels of education and moderate rates of depression. All of the relationships demonstrated in previous samples were replicated, with the exception of the relationships between the cons of smoking and stage of change and between one temptation subscale, negative affect, and stage. The lack of variability by stage of change for the negative affect subscale was probably related to the depression levels endorsed by this sample. Further research is necessary to examine both (a) how to improve the cons scale by adding specific items that may better reflect the experiences and values of HIV+ smokers and (b) how to address both depression and smoking cessation effectively in these smokers. These scales can be used to deliver and evaluate TTM-tailored smoking cessation programs that have demonstrated effectiveness in other samples.

Acknowledgments

The contents of this publication are solely the responsibility of the authors and do not necessarily represent the official views of the National Institutes of Health. We would also like to acknowledge data collection and management support from Chicago WIHS staff, in particular Ms. Sally Urwin.

Compliance with Ethical Standards

Funding: Data in this manuscript were collected by the Women’s Interagency HIV Study (WIHS) Chicago Consortium (PI, M.H.C.). The WIHS is funded by the National Institute of Allergy and Infectious Diseases (UO1-AI-35004, UO1-AI-31834, UO1-AI-34994, UO1-AI-34989, UO1-AI-34993, and UO1-AI-42590) and the Eunice Kennedy Shriver National Institute of Child Health and Human Development (UO1-HD-32632). The study is cofunded by the National Cancer Institute, the National Institute on Drug Abuse, and the National Institute on Deafness and Other Communication Disorders. Funding is also provided by the National Center for Research Resources (UCSF-CTSI Grant Number UL1 RR024131). The Chicago WIHS is funded by the National Institute of Allergy and Infectious Diseases (UO1-AI-34993). This project, including collaborators at the Cancer Prevention Research Center, University of Rhode Island led by C.A.R., was funded through an administrative supplement to the Chicago WIHS from the National Cancer Institute. K.M.W. is also funded by P30 AI082151.

Conflict of Interest: Colleen A. Redding; David Goldberg, Kathleen M. Weber, Hui-Qing Yin, Andrea L. Paiva, Jane Burke-Miller, Mardge H. Cohen, and Joseph S. Rossi declare that they have no conflicts of interest.

Ethical Approval: All procedures performed in this study involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. All study procedures were approved by the Women’s Interagency HIV Study (WIHS) Executive Committee, and both the Cook County Health and Hospital System and the University of Rhode Island Institutional Review Boards.

Informed Consent: Written informed consent was obtained from all individual participants included in this study.

Welfare of Animals: This article does not contain any studies with animals by any of the authors.

References

  • 1. Centers for Disease Control and Prevention. Vital signs: Current cigarette smoking among adults aged ≥18 years – United States, 2009. MMWR. 2010;59(35):1135–1140. [PubMed] [Google Scholar]
  • 2. Kraut-Becher J, Eisenberg M, Voytek C, Brown T, Metzger DS, Aral S. Examining racial disparities in HIV: Lessons from sexually transmitted infections research. J Acquir Immune Defic Syndr. 2008;47(suppl 1):S20–S27. [DOI] [PubMed] [Google Scholar]
  • 3. Adimora AA, Schoenbach VJ, Floris-Moore MA. Ending the epidemic of heterosexual HIV transmission among African Americans. Am J Prev Med. 2009;37(5):468–471. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4. Vidrine DJ. Cigarette smoking and HIV/AIDS: Health implications, smoker characteristics and cessation strategies. AIDS Educ Prev. 2009;21(3 suppl):3–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5. Hessol NA, Weber KM, D’Souza G, et al. . Smoking cessation and recidivism in the Women’s interagency human immunodeficiency virus study. Am J Prev Med. 2014;47(1):53–69. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6. Frazier EL, Sutton MY, Brooks JT, Shouse RL, Weiser J. Trends in cigarette smoking among adults with HIV compared with the general adult population, United States – 2009–2014. Prev Med. 2018;111:231–234. [DOI] [PubMed] [Google Scholar]
  • 7. Centers for Disease Control and Prevention. CDC Health disparities and inequalities report – United States, 2011. MMWR. 2011;60(suppl):109–113. [PubMed] [Google Scholar]
  • 8. Centers for Disease Control and Prevention. Quitting smoking among adults – United States, 2001–2010. MMWR. 2011;60(44):1513–1519. [PubMed] [Google Scholar]
  • 9. Goldberg D, Weber KM, Orsi J, et al. . Smoking cessation among women with and at risk for HIV: Are they quitting? J Gen Intern Med. 2010;25(1):39–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10. Prochaska JO, DiClemente CC. Stages and processes of self-change of smoking: Toward an integrative model of change. J Consult Clin Psychol. 1983;51(3):390–395. [DOI] [PubMed] [Google Scholar]
  • 11. Prochaska JO, Redding CA, Evers K. The transtheoretical model and stages of change. In: Glanz K, Rimer BK, Viswanath KV, eds. Health Behavior and Health Education: Theory, Research and Practice. 5th ed. San Francisco, CA: Jossey-Bass Inc; 2015:125–148. [Google Scholar]
  • 12. Velicer WF, DiClemente CC, Prochaska JO, Brandenburg N. Decisional balance measure for assessing and predicting smoking status. J Pers Soc Psychol. 1985;48(5):1279–1289. [DOI] [PubMed] [Google Scholar]
  • 13. Prochaska JO, Velicer WF, DiClemente CC, Fava J. Measuring processes of change: Applications to the cessation of smoking. J Consult Clin Psychol. 1988;56(4):520–528. [DOI] [PubMed] [Google Scholar]
  • 14. Velicer WF, Diclemente CC, Rossi JS, Prochaska JO. Relapse situations and self-efficacy: An integrative model. Addict Behav. 1990;15(3):271–283. [DOI] [PubMed] [Google Scholar]
  • 15. Velicer WF, Prochaska JO, Bellis JM, et al. . An expert system intervention for smoking cessation. Addict Behav. 1993;18(3):269–290. [DOI] [PubMed] [Google Scholar]
  • 16. Prochaska JO, Velicer WF, Fava JL, Rossi JS, Tsoh JY. Evaluating a population-based recruitment approach and a stage-based expert system intervention for smoking cessation. Addict Behav. 2001;26(4):583–602. [DOI] [PubMed] [Google Scholar]
  • 17. Prochaska JO, Velicer WF, Fava JL, et al. . Counselor and stimulus control enhancements of a stage-matched expert system intervention for smokers in a managed care setting. Prev Med. 2001;32(1):23–32. [DOI] [PubMed] [Google Scholar]
  • 18. Velicer WF, Prochaska JO, Redding CA. Tailored communications for smoking cessation: Past successes and future directions. Drug Alcohol Rev. 2006;25(1):49–57. [DOI] [PubMed] [Google Scholar]
  • 19. Johnson JL, Fava JL, Velicer WF, Monroe AD, Emmons K. Testing stage effects in an ethnically diverse sample. Addict Behav. 2002;27(4):605–617. [DOI] [PubMed] [Google Scholar]
  • 20. Ward RM, Velicer WF, Rossi JS, Fava JL, Prochaska JO. Factorial invariance and internal consistency for the decisional balance inventory – short form. Addict Behav. 2004;29(5):953–958. [DOI] [PubMed] [Google Scholar]
  • 21. Hoffman AM, Redding CA, Goldberg D, et al. . Computer expert systems for African-American smokers in physicians offices: A feasibility study. Prev Med. 2006;43(3):204–211. [DOI] [PubMed] [Google Scholar]
  • 22. Hoeppner BB, Redding CA, Rossi JS, Pallonen UE, Prochaska JO, Velicer WF. Factor structure of decisional balance and temptations scales for smoking: Cross-validation in urban female African-American adolescents. Int J Behav Med. 2012;19(2):217–227. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23. Kumanyika S. Minority populations. In: Burke L, Ockene I, eds. Compliance in Healthcare and Research. Armonk, NY: Futura Publishing Company Inc; 2001:195–218. [Google Scholar]
  • 24. Redding CA, Prochaska JO, Pallonen UE, et al. . Transtheoretical individualized multimedia expert systems targeting adolescents’ health behaviors. Cogn Behav Pract. 1999;6(2):144–153. [Google Scholar]
  • 25. Barkan SE, Melnick SL, Preston-Martin S, et al. . The Women’s Interagency HIV Study. WIHS Collaborative Study Group. Epidemiology. 1998;9(2):117–125. [PubMed] [Google Scholar]
  • 26. Bacon MC, von Wyl V, Alden C, et al. . The Women’s Interagency HIV study: An observational cohort brings clinical sciences to the bench. Clin Diagn Lab Immunol. 2005;12(9):1013–1019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27. Radloff L. The CES-D Scale: A self-report depression scale for research in the general population. Appl Psychol Meas. 1977;1(3):385–401. [Google Scholar]
  • 28. Bozzette SA, Hays RD, Berry SH, Kanouse DE, Wu AW. Derivation and properties of a brief health status assessment instrument for use in HIV disease. J Acquir Immune Defic Syndr Hum Retrovirol. 1995;8(3):253–265. [DOI] [PubMed] [Google Scholar]
  • 29. Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983;24(4):385–396. [PubMed] [Google Scholar]
  • 30. Heatherton TF, Kozlowski LT, Frecker RC, Fagerström KO. The Fagerström Test for Nicotine Dependence: A revision of the Fagerström Tolerance Questionnaire. Br J Addict. 1991;86(9):1119–1127. [DOI] [PubMed] [Google Scholar]
  • 31. DiClemente CC, Prochaska JO, Fairhurst SK, Velicer WF, Velasquez MM, Rossi JS. The process of smoking cessation: An analysis of precontemplation, contemplation, and preparation stages of change. J Consult Clin Psychol. 1991;59(2):295–304. [DOI] [PubMed] [Google Scholar]
  • 32. Velicer WF, Redding CA, Sun X, Prochaska JO. Demographic variables, smoking variables, and outcome across five studies. Health Psychol. 2007;26(3):278–287. [DOI] [PubMed] [Google Scholar]
  • 33. Kline RB. In: Kenny DA, ed. Principles and Practice of Structural Equation Modeling. New York, NY: Guilford Press; 2005. [Google Scholar]
  • 34. Noar SM. The role of structural equation modeling in scale development. Struct Equ Modeling. 2003;10(4):622–647. [Google Scholar]
  • 35. Redding CA, Maddock JE, Rossi JS. The sequential approach to measurement of health behavior constructs: Issues in selecting and developing measures. Cali J Health Promot. 2006;4(1):83–101. [Google Scholar]
  • 36. Bentler PM. EQS 6 Structural Equations Program Manual. Encino, CA: Multivariate Software; 2006. [Google Scholar]
  • 37. Bentler PM. EQS (for Windows) [Computer Program]. Version 6.1. Encino, CA: Multivariate Software; 1995. [Google Scholar]
  • 38. IBM Corp. IBM SPSS Statistics for Windows [Computer Program], Version 22.0. Armonk, NY: IBM Corp; 2016. [Google Scholar]
  • 39. Hu L, Bentler P. Evaluating model fit. In: Hoyle R, ed. Structural Equation Modeling. Thousand Oaks, CA: Sage; 1995:76–99. [Google Scholar]
  • 40. Grissom RJ, Kim JJ.. Effect Sizes for Research: Univariate and Multivariate Applications. 2nd ed. New York, NY: Routledge; 2012. [Google Scholar]
  • 41. Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988. [Google Scholar]
  • 42. Stanton CA, Lloyd-Richardson EE, Papandonatos GD, de Dios MA, Niaura R. Mediators of the relationship between nicotine replacement therapy and smoking abstinence among people living with HIV/AIDS. AIDS Educ Prev. 2009;21(3 suppl):65–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43. Fiore MC, Jaen CR, Baker TB, et al. . Clinical practice guideline. Treating Tobacco Use and Dependence: 2008 Update. Rockville, MD: US Department of Health and Human Services. Public Health Service; 2008. [Google Scholar]
  • 44. Hall KL, Rossi JS. Meta-analytic examination of the strong and weak principles across 48 health behaviors. Prev Med. 2008;46(3):266–274. [DOI] [PubMed] [Google Scholar]
  • 45. Prochaska JO. Strong and weak principles for progressing from precontemplation to action on the basis of twelve problem behaviors. Health Psychol. 1994;13(1):47–51. [DOI] [PubMed] [Google Scholar]
  • 46. Prochaska JO, Velicer WF, Rossi JS, et al. . Stages of change and decisional balance for 12 problem behaviors. Health Psychol. 1994;13(1):39–46. [DOI] [PubMed] [Google Scholar]
  • 47. Redding CA, Prochaska JO, Paiva A, et al. . Baseline stage, severity, and effort effects differentiate stable smokers from maintainers and relapsers. Subst Use Misuse. 2011;46(13):1664–1674. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48. Rossi JS, Redding CA. Structure and function of self-efficacy across the stages of change for 10 health behaviors [abstract]. Ann Behav Med. 2001;23(suppl):S094. [Google Scholar]
  • 49. Hall SM, Tsoh JY, Prochaska JJ, et al. . Treatment for cigarette smoking among depressed mental health outpatients: A randomized clinical trial. Am J Public Health. 2006;96(10):1808–1814. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50. Prochaska JJ, Hall SE, Delucchi K, Hall SM. Efficacy of initiating tobacco dependence treatment in inpatient psychiatry: A randomized controlled trial. Am J Public Health. 2014;104(8):1557–1565. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51. Levesque DA, Van Marter DF, Schneider RJ, et al. . Randomized trial of a computer-tailored intervention for patients with depression. Am J Health Promot. 2011;26(2):77–89. [DOI] [PubMed] [Google Scholar]

Articles from Translational Behavioral Medicine are provided here courtesy of Oxford University Press

RESOURCES