Abstract
While there are a number of general measures that assess interpersonal and psychological distress experienced by individuals who are in a close relationship with a substance abusing adult, until recently the field has lacked a psychometrically sound, self-administered multidimensional measure explicitly designed to measure the problems of non-substance abusing adults who are concerned about a substance using loved one. This study examined the psychometric properties of a 54-item, self-administered (SA) version of the Significant Other Survey (SOS), a measure designed to address this gap. The SOS-SA assesses problems across seven problem domains (emotional, relationship, family, financial, physical violence, legal, health). Coefficient alpha estimates (N=168) were good to excellent for five of the domains, the test-retest reliability (N=83) across a 7-day time frame was fair to excellent for all seven domains. Similar reliability coefficients were identified regardless of whether the item queried about the problem frequency or perceived severity. There was preliminary support for the construct and discriminant validity of the SOS-SA. The SOS-SA appears to be a promising instrument given that it is brief, requires no specialized training to administer, and has good psychometric properties.
Keywords: Significant Other Survey, Assessment, Substance Abuse, Family
Individuals who are in a relationship with a substance abusing loved one experience a range of life problems either as a direct or indirect consequence of their interactions with the substance using person. These negative effects include psychological difficulties (Lipscomb, Dement, & Leiming, 2003; Oreo & Ozgul, 2007; Orford, Natera, Davies et al., 1998), interpersonal conflict (Haber & Jacob, 1997), relationship dissatisfaction and family conflict (Hornish, Leonard, & Cornelius, 2008; Kahler, McCrady, & Epstein, 2003), financial difficulties (Benishek, Kirby, Dugosh, 2011; Kirby, Dugosh, Benishek, & Harrington, 2005), a heightened risk for domestic violence (Murphy, O’Farrell, Fals-Stewart, & Feehan, 2001; Wekerle & Wall, 2002), legal problems (Benishek et al., 2011; Kirby et al., 2005), and illness and/or increased use of medical services (Lennox, Scott-Lennox, & Holder, 1992; Ray, Mertens, & Weisner, 2008).
Although much attention has been paid to identifying the problems that substance using individuals (SUIs) experience in their lives, significantly less attention has been given to assessing related problems experienced by their family members and concerned significant others (CSOs; Butler & Bauld, 2005; Copello, Velleman, & Templeton, 2005; Csiernik, 2002). This is unwarranted given the evidence that substance abusers’ behaviors negatively impact the lives of these individuals (Schneider Institute for Health Policy, 2001) and that family members often maintain the relationship even though it is conflicted and troublesome (Hudson, Kirby, Firely, Festinger, & Marlowe, 2002; Landau et al., 2000).
We began examining the paucity of measures that assess the objective problems of CSOs over a decade ago (Kirby, Marlowe, Festinger, Garvey, & LaMonaca, 1999) and have repeatedly completed literature searches in an effort to identify existing instruments that have been used to assess the difficulties experienced by adults who were concerned about a substance abusing individual. We reviewed the literature published between 1995 and 2011 using two electronic bibliographic data bases: PsycINFO and Medline and searched two instrument-related electronic databases: Health and Psychosocial Instruments (HAPI) and a National Institute on Drug Abuse (NIDA)-funded web-based drug abuse instrument repository (i.e., www.instrument.wizard.com). As suggested by the Cochrane Collaborative (Higgins & Green, 2008), a hand-searching procedure was completed on tables of contents of family-oriented journals and measurement-related periodicals (e.g., Measures for Clinical Practice).
This search process identified several instruments that assess difficulties experienced by significant others. However, the majority of these instruments were fairly constricted in utility. Some of the instruments focused on a single problem domain (e.g., emotional/psychological difficulties; Profile of Mood States/McNair, Loor, & Droppleman, 2003) or assessed problems experienced by CSOs of individuals who abuse alcohol but not other drugs (e.g., Spouse Enabling Inventory/Thomas, Yoshioka, & Ager, 1996). Some of the measures were not applicable to different types of CSOs (i.e., contained items pertaining to sexual intimacy or family responsibilities that would be relevant to spouses/partners but not to parents or siblings of an SUI; e.g., Behavioral Enabling Scale/Rotunda, West, & O’Farrell, 2004), assessed the presence and frequency of the difficulty but did not attend to the perceived impact of the problem on the CSO (e.g., Family Member Impact/Orford, Templeton, Velleman, & Copello, 2005) or had a limited amount of published psychometric properties (e.g., Drinker’s Partner Distress Scale/Crisp & Barber, 1995). As a whole, these features restrict the instrument’s utility for collecting reliable and valid information about the various types of problems experienced by a wide range of CSOs of individuals abusing a host of different substances.
Although many of the instruments did, in fact, have their own unique strengths and make contributions to clinical practice and research, the majority of them did not explicitly assess multiple problem domains, address both the frequency and the extensiveness of the problem in the CSO’s life, and have reasonable and/or known psychometric properties. Only two measures that we identified as a result of our search processes, the Drinker’s Partner Distress Scale (DPDS; Crisp & Barber, 1995) and the Significant Other Survey interview (SOS; Benishek, Dugosh, Faranda-Diedrich, & Kirby, 2006), met these three criteria.
The 12-item DPDS addresses three life areas (i.e., emotional, relationship, financial difficulties), assesses both the frequency and perceived distress associated with each problem, and has acceptable psychometric properties (i.e., internal consistency reliability; construct and discriminant validity). The Significant Other Survey (SOS) interview also assesses the frequency and perceived distress associated with each problem and has reasonable psychometric properties. However, it evaluates problems across a broader range of seven life areas (emotional, relationship, family, financial, physical violence, legal, health), focuses on problems related to alcohol and other licit and illicit substance use, and is applicable to a wider range of CSOs. For example, items pertaining to sexual behaviors were in an optional section for spouse/partners and not administered to CSOs who were not in a romantic relationship with the SUI (e.g., parents).
The earliest version of the SOS was a 70-item Significant Other Checklist (Kirby et al., 1999; 2005). The problem domains selected for the Checklist paralleled those found in the Addiction Severity Index (McLellan, Luborsky, Cacciola, & Griffith, 1985). Interviews conducted with 29 CSOs (i.e., family members of drug users) were used to identify the initial set of specific problems that were associated with each of the domains. This list of problems was then confirmed and expanded upon via interviews with a second unique group of CSOs (n=25). Feedback from addiction treatment experts and data analyses conducted on the Checklist were used to develop the SOS interview (Benishek et al., 2006). An interview format was introduced because items were added to address the CSOs’ interactions with the SUI and to estimate costs of substance use to the family and society. These items were complex and required trained interviewers. The items pertaining to problems in seven life domains were similar to the items from the earlier checklist. Based on data derived from the SOS interview sample, the seven domains were found to be relatively independent of one another, and there was reasonable internal consistency reliability among six of the domains (i.e., coefficient alphas ranged from .61 to .83 with the lowest alpha associated with the legal domain).
Despite these emerging content- and psychometrically-oriented strengths, a practical limitation was that the interview format was less convenient to use and we suspected that this format was not necessary to assess CSO problems. In response, we developed a 54-item self-administered version (i.e., SOS-SA) to increase its real-world utility for practitioners and researchers. Based on data derived on the SOS interview sample, items contained in the SOS interview were included in the SOS-SA if they had a test-retest coefficient greater than .40 (Cicchetti & Sparrow, 1981) and an item-total correlation greater than .20 (Nunnally, 1967).
This study continues our systematic efforts to develop a brief self-administered, psychometrically sound measure of problems experienced by CSOs who are concerned about a substance abusing loved one that can be easily implemented in both treatment and research settings (see Benishek et al., 2006; Benishek et al., 2011; Kirby et al., 2005). The purpose of this study was to examine the internal consistency reliability, test-retest reliability, problem domain scores, construct validity, and discriminant validity of a self-administered SOS (SOS-SA) among a diverse sample of CSOs whose substance using loved one was not in treatment.
Methods
Participants and Procedures
Study participants were recruited as a part of a larger clinical trial designed to examine the efficacy of three interventions at engaging treatment-resistant substance abusing adult family members or close friends in treatment. These individuals, who self-identified as being concerned about the drug use or drinking of a loved one (i.e., concerned significant others/CSOs), were recruited through various venues including radio, newspapers, television, internet, and public transportation in the greater Philadelphia area. Individuals who contacted the clinic completed a preliminary telephone screening to verify that they were CSOs and to determine if they met basic study eligibility criteria for the clinical trial: (a) both the CSO and the substance using individual (SUI) were at least 18 years of age; (b) the CSO did not have a substance use problem or had been in recovery for a minimum of two years; (c) the CSO and SUI had face-to-face contact at least 12 of the past 30 days (i.e., This amount of contact was considered necessary for the CSO to be able to implement the clinical interventions for engaging the SUI and has been used in similar clinical trials of treatment-resistant SUIs; Meyers, Miller, Smith, & Tonigan, 2002); (d) the CSO reported that the SUI used alcohol, stimulants, or opiates in the past 30 days; and (e) the SUI was presently resistant to engaging in substance abuse treatment and had not received treatment within the past six weeks. CSOs who were not eliminated during the telephone screening process were then scheduled for an in-person interview that allowed verification of the above criteria and assessment of the following additional eligibility criteria: (f) the CSO reported that the SUI’s pattern of using alcohol, stimulants, or opiates met DSM-IV criteria for abuse (≥1 of 4 possible criteria) or dependence (≥3 of 7 possible criteria); and (g) there was no indication of serious physical violence in the CSO-SUI relationship (e.g., conflict involving guns, knives). During the initial telephone screening, 303 individuals were determined ineligible for the clinical trial, 116 were determined ineligible during the in-person screening, and 197 were eligible and consented to participate in the clinical trial. CSOs who were not eligible to participate in the clinical trial were provided with appropriate referral information. All clinical trial participants who provided baseline SOS-SA data were included in the SOS-SA psychometric study.
Institutional Review Board approval was obtained before recruiting participants for this study. All eligible CSOs provided informed consent to participate in the study before completing the SOS-SA in-person at the host study clinic with research staff. The data associated with the internal consistency reliability estimates (n=168) were collected as a part of the clinical trial baseline assessment, whereas the retest data (n=83) were collected one to seven days (Mn=4.85; SD=2.14) after completing the baseline assessment and prior to completing the first counseling session associated with the clinical trial.
Significant Other Survey - Self-Administered (SOS-SA)
The 54-item SOS-SA is a self-report questionnaire that is written at a 7th grade Flesch-Kincaid reading level and can be completed in 10–15 minutes. The SOS-SA assesses the frequency and perceived severity of current problems (i.e., the past 30 days) that different types of CSOs experience in seven life domains. The Emotional domain assesses emotions that CSO’s identify as problematic (10 items, e.g., “You felt hopeless”). The Relationship domain assesses problems the CSO perceives in their relationship with the SUI including arguing, lack of closeness, enmeshment, and lack of balance (7 items, e.g., “You did things for your loved one that you think s/he should’ve done for him/herself”). The Family domain addresses disruptions in close relationships and the home environment (12 items, e.g., “Family members had arguments with your loved one”). The Financial domain addresses financial problems related to the SUI (8 items, e.g., “You paid fines or bills for your loved one”). The Physical Violence domain assesses the receipt and perpetration of physical violence by the SUI and/or other loved ones and/or the CSO (14 items, e.g., “Your loved one physically attacked a family member other than you”). The Legal domain assesses legal problems related to the SUI (1 item, i.e., “You dealt with legal problems related to your loved one”), and the Health domain assesses the medical problems of the CSO (2 items; e.g., “You experienced medical problems”). Participants indicate (a) the frequency or how often they experienced the problem (0=never; 1=rarely; 2=occasionally; 3=frequently; 4=almost always) and (b) the severity or how bothered they were by each endorsed problem (0=not at all; 1=a little; 2=somewhat; 3=a lot; 4=a great deal). The SOS-SA is available at http://www.tresearch.org (search for “Significant Other Survey”) or by sending an email request to K. C. Kirby.
Data Analyses
Four types of analyses were used to examine the psychometric properties of the SOS-SA. First, the internal consistency of the SOS-SA items was evaluated for each problem domain using item-total correlations and Cronbach’s coefficient alpha (Cronbach, 1951). Items reflecting the frequency with which a problem occurred and items reflecting the severity of the problem were evaluated separately. Internal consistency was evaluated only for the five problem domains (Emotional, Relationship, Family, Financial, Physical Violence) that retained seven or more items through previous psychometric testing. Based on Nunally’s (1967) guidelines related to early exploratory scale development, coefficient alpha estimates below .60 were considered to be indicative of a problematic scale and item-total correlations below .20 were considered to be indicative of problematic items. It should be noted that we examined internal consistency in this manner because the response scale uses a Likert-type format and that the values obtained in these analyses may actually be underestimates of their true values (Zumbo, Gadermann, & Zeisser, 2007).
Second, weighted kappa statistics were calculated on six of the seven problem domains to examine the test-retest reliability of items since the items used an ordinal response scale. Kappa values below 0.40, 0.40 – 0.59, 0.60–0.74, and 0.75–1.00 are considered poor, fair, good, and excellent, respectively (Cicchetti, 1994; Cicchetti & Sparrow, 1981). A Pearson r was used to calculate the test-retest coefficient for the single item legal problem domain. An item was identified as being a poor performer if it had both (a) a test-retest reliability coefficient in the poor range (i.e., below 0.40, Cicchetti & Sparrow, 1981) and (b) an item-total correlation below 0.20 (Nunnally, 1967). Next, scores were calculated separately for frequency and severity. We then averaged the retained item responses within each problem domain; thus, domain scores could range from 0 to 4. Inter-item correlations within each scale were examined to provide a preliminary evaluation of the SOS-SA’s construct validity with higher inter-item correlations reflecting greater levels of construct validity. Finally, the correlations among the problem domain scores were examined to provide an indication of the instrument’s discriminant validity with lower inter-scale correlations being indicative of greater levels of discriminant validity.
Results
Participant Characteristics
Characteristics of the CSOs and SUIs are reported in Table 1. The majority of the CSOs were employed females who had completed three years of post high school education; they averaged 49 years of age. The SUIs tended to be employed males with slightly more than a high school education who were using alcohol, stimulants, or opiates; they were slightly over 40 years of age. Chi-square analyses and t-tests revealed no statistically significant differences among the demographic and substance use characteristics associated with the 85 participants who completed only the baseline assessment and the 83 who additionally provided test-retest data (p-values ranged from 0.11 to 0.96).
Table 1.
Descriptive Characteristics of CSOs and SUIs for Test-Retest Reliability and Internal Consistency Reliability Samples
| Internal Consistency Reliability Sample (N=168) | Test-Retest Reliability Sample (N=83) | |||
|---|---|---|---|---|
|
| ||||
| Variable | Concerned Significant Other | Substance Using Individual | Concerned Significant Other | Substance Using Individual |
| Gender % (N) | ||||
| Female | 74.4 (125) | 36.3 (61) | 73.5 (61) | 36.2 (30) |
| Male | 25.6 (43) | 63.7 (107) | 26.5 (22) | 63.8 (53) |
| Race % (N) | ||||
| White/Caucasian | 52.9 (89) | 52.4 (88) | 49.4 (41) | 41.1 (41) |
| African American/Black | 42.4 (71) | 41.0 (69) | 44.6 (37) | 43.4 (36) |
| Other | 4.5 (8) | 6.5 (11) | 6.0 (5) | 7.2 (6) |
| Employment % (N) | ||||
| Employed | 81.5 (137) | 64.8 (109) | 77 (64) | 63.8 (53) |
| Unemployed | 4.8 (8) | 22.6 (38) | 4.8 (4) | 22.9 (19) |
| Other | 13.7 (23) | 12.6 (21) | 18.2 (15) | 13.3 (11) |
| Marital Status % (N) | ||||
| Married/Living as Married | 53 (89) | 50 (84) | 44.6 (37) | 47 (39) |
| Never Married | 21.4 (36) | 35.1 (59) | 25.3 (21) | 38.5 (32) |
| Other | 25.6 (43) | 14.9 (25) | 30.1 (25) | 14.5 (12) |
| Age M (SD) | 49.1 (11.09) | 43.25 (12.63) | 49.2 (10.2) | 41.6 (11.3) |
| Years of Education M (SD) | 15.3 (3.42) | 13.29 (2.69) | 15.1 (3.09) | 13.5 (2.67) |
| Primary Drug of Abuse % (N) | ||||
| Alcohol | 54.2 (91) | 50.6 (42) | ||
| Stimulants | 30.4 (52) | 35 (29) | ||
| Opiates | 10.7 (18) | 10.8 (9) | ||
| Other* | 4.7 (7) | 3.6 (3) | ||
| Relationship Type % (N) | ||||
| Parent | 27.4 (46) | 31.3 (26) | ||
| Spouse/Partner | 48.2 (81) | 43.4 (36) | ||
| Other | 24.4 (41) | 25.3 (21) | ||
| Living with SUI % (N) | 64.3 (108) | 61.5 (51) | ||
| Days Contact with SUI M (SD) | 78.1 (19.9) | 77.24 (17.7) | ||
Note.
indicates that the SUI also had a diagnosis of abuse or dependence for alcohol, stimulants, or opiates.
Internal Consistency Reliability and Test-Retest Reliability
Problem frequency
Acceptable internal consistency was demonstrated for the frequency items within five problem domains. As seen in Table 2, the coefficient alpha values ranged from .76 (Relationship) to .91 (Emotional), and the average item-total correlation for these five domains ranged from .41 (Family) to .68 (Emotional).
Table 2.
Internal Consistency Reliability and Test-Retest Reliability
| How often have you experienced the problem? | Internal Consistency Reliability | Test-Retest Reliability | ||||
|---|---|---|---|---|---|---|
|
| ||||||
| Number of Items | Alpha | Item-total r | Weighted kappa | |||
|
| ||||||
| M (SD) | Range | M (SD) | Range | |||
| Emotional | 10 | .91 | .68 (.08) | .57–.80 | .56 (.06) | .48–.68 |
| Relationship | 7 | .76 | .48 (.08) | .41–.64 | .48 (.08) | .37–.61 |
| Family | 12 | .77 | .41 (.09) | .25–.54 | .55 (.09) | .44–.75 |
| Financial | 8 | .83 | .56 (.13) | .31–.69 | .57 (.09) | .49–.68 |
| Physical Violence | 14 | .87 | .54 (.12) | .35–.74 | .55 (.17) | .25–.85 |
| Legal | 1 | n/a | n/a | n/a | n/a | n/a |
| Health | 2 | n/a | n/a | n/a | .62 (.02) | .61–.64 |
|
| ||||||
| How bothered were you by the problem? | ||||||
|
| ||||||
| Emotional | 10 | .90 | .66 (.09) | .46–.78 | .51 (.08) | .39–.63 |
| Relationship | 7 | .78 | .51 (.06) | .44–.62 | .44 (.14) | .25–.64 |
| Family | 12 | .79 | .44 (.08) | .30–.57 | .52 (.06) | .43–.62 |
| Financial | 8 | .87 | .62 (.12) | .37–.76 | .59 (.08) | .47–.72 |
| Physical Violence | 14 | .86 | .51 (.12) | .20–.67 | .51 (.15) | .32–.75 |
| Legal | 1 | n/a | n/a | n/a | n/a | n/a |
| Health | 2 | n/a | n/a | n/a | .55 (.03) | .53–.56 |
Note. Nα= 168; Ntest-retest = 83.
With regard to each of the six domains, the mean test-retest coefficients ranged from .48 (Relationship) to .62 (Health) with 93% of items falling into the fair to excellent range (i.e., coefficients ≥ .40; see Table 2). Only four items in two of the domains had test-retest coefficients < .40 (i.e., Physical Violence [3/14 items] and Relationship [1/7 items]). All items in the remaining four domains had at least fair reliability. The test-retest coefficient for the single-item legal domain was .58.
Problem severity
As seen in Table 2, acceptable internal consistency reliability was found for the severity items within all five problem domains with coefficient alpha estimates ranging from .78 (Relationship) to .90 (Emotional). The average item-total correlation for the five domains ranged from .44 (Family) to .66 (Emotional).
With regard to each of the six domains, the mean test-retest coefficients ranged from .44 (Relationship) to .59 (Financial) with 85% of the items in the fair to excellent range (see Table 2). Nine items in three domains had poor test-retest coefficients (i.e., Emotional [1/10 items], Physical [6/14 items], and Relationship [2/7 items]). The other three domains had all items with at least fair reliability. The test-retest coefficient for the single-item legal domain was .61.
Problem Domain Scores, Construct Validity, and Discriminant Validity
No items met the a priori criteria necessary to define them as a “poor performer” (i.e., an item-total correlation below .20 and a test-retest reliability coefficient below .40). For this reason, scale scores were calculated for the frequency and severity sub-sets of items for each problem domain using all items. Mean frequency and severity scores for each domain are presented in Table 3. The mean frequency and severity scores ranged from .21 to 2.05 and from .35 to 2.05, respectively. Participants endorsed values ranging from 0 to 4.00 for the frequency and severity items.
Table 3.
SOS-SA Problem Domain Scores
| How Often? | N | M | SD | Min | Max |
|---|---|---|---|---|---|
| Emotional | 166 | 1.79 | 0.90 | 0 | 3.60 |
| Relationship | 168 | 2.05 | 0.75 | 0 | 3.71 |
| Family | 168 | 1.14 | 0.68 | 0 | 3.25 |
| Financial | 168 | 1.17 | 0.94 | 0 | 3.75 |
| Physical Violence | 167 | 0.21 | 0.36 | 0 | 2.57 |
| Legal | 166 | 0.61 | 1.00 | 0 | 4.00 |
| Health | 166 | 1.61 | 1.37 | 0 | 4.00 |
|
| |||||
| How Bothered? | N | M | SD | Min | Max |
|
| |||||
| Emotional | 166 | 1.70 | 1.00 | 0 | 4.00 |
| Relationship | 168 | 2.05 | 0.89 | 0 | 4.00 |
| Family | 168 | 1.25 | 0.83 | 0 | 3.58 |
| Financial | 168 | 0.99 | 1.02 | 0 | 4.00 |
| Physical Violence | 167 | 0.35 | 0.60 | 0 | 2.93 |
| Legal | 166 | 0.86 | 1.45 | 0 | 4.00 |
| Health | 166 | 1.16 | 1.35 | 0 | 4.00 |
Note. Response options for How Often: 0=never, 1=rarely, 2=occasionally, 3=frequently, 4=almost always; Response options for How Bothered: 0=not at all, 1=a little, 2=somewhat, 3=a lot, 4=a great deal).
Inter-item correlations were calculated to assess the construct validity of the SOS-SA problem domains. With regard to problem frequency, mean inter-item correlations were within an acceptable range: .51/emotional, .31/relationship, .23/family, .38/financial, .34 physical violence, and .54/health. Mean values associated with problem severity were also within the acceptable range: .49/emotional, .35/relationship, .25/family, .44/financial, .31 physical violence, and .64/health.
Inter-scale correlations among the problem domains are presented in Table 4 and provide support for the discriminant validity of the SOS-SA. For the frequency items, correlations ranged from .12 (Legal/Health) to .62 (Relationship/Financial), and three of these inter-domain correlations were not statistically significant (i.e., Family/Health, Physical/Health, Legal/Health). These values indicate that there was either no relationship between the domains or, at most, 38% overlap. For the severity items, all inter-domain correlations were statistically significant. Correlations ranged from .22 (Legal/Health) to .64 (Emotional/Relationship), indicating that there was 5% to 41% overlap between the domains.
Table 4.
SOS-SA Inter-Domain Correlations
| Emotional | Relationship | Family | Financial | Physical | Legal | Health | ||
|---|---|---|---|---|---|---|---|---|
|
|
||||||||
| Often | Emotional | 1.00 | 0.59*** | 0.41*** | 0.33*** | 0.26*** | 0.17* | 0.29*** |
| Relationship | 1.00 | 0.59*** | 0.62*** | 0.50*** | 0.37*** | 0.17* | ||
| Family | 1.00 | 0.49*** | 0.50*** | 0.29*** | 0.15 | |||
| Financial | 1.00 | 0.48*** | 0.43*** | 0.25** | ||||
| Physical | 1.00 | 0.31*** | 0.13 | |||||
| Legal | 1.00 | 0.12 | ||||||
| Health | 1.00 | |||||||
|
| ||||||||
| Bothered | Emotional | 1.00 | 0.64*** | 0.43*** | 0.38*** | 0.28*** | 0.26*** | 0.48*** |
| Relationship | 1.00 | 0.57*** | 0.61*** | 0.46*** | 0.35*** | 0.32*** | ||
| Family | 1.00 | 0.56*** | 0.52*** | 0.29*** | 0.30*** | |||
| Financial | 1.00 | 0.52*** | 0.47*** | 0.35*** | ||||
| Physical | 1.00 | 0.26*** | 0.26*** | |||||
| Legal | 1.00 | 0.22** | ||||||
| Health | 1.00 | |||||||
Note.
= p < 0.05,
= p < 0.01,
= p < 0.001.
Discussion
The results associated with this initial evaluation of the SOS-SA are promising. The SOS-SA is a brief, easily administered self-report measure with good to excellent internal consistency reliability for five of the seven problem domains and fair to excellent item test-retest reliability over a 7-day period of time. Overall, the magnitude of the internal consistency reliability values are greater than those that we found for the SOS Checklist and the SOS interview (see Kirby et al., 2005 and Benishek et al., 2006). There appears to be preliminary support for the construct validity of the SOS-SA with at least moderate correlations, on average, between the items that comprise each domain and acceptable levels of coefficient alpha. Similarly, we found some degree of support for the discriminant validity of the SOS-SA. As desired, there were relatively low levels of overlap among the seven problem domains, and the higher degrees of overlap were expected and made sense conceptually. For example, CSOs who reported more relationship problems would also be expected to report more emotional problems and, similarly, those CSOs who were more bothered by those relationship problems would be likely report that they were also more emotionally troubled. Alternatively, we would not expect that CSOs who report concerns about their physical wellness to report legal difficulties.
The mean values associated with the problem domains were substantially lower than those of a previous study with CSOs who participated in a clinical trial because of a treatment-resistant loved one (i.e., Kirby et al., 2005). This may be related to the differences in the items between this and the earlier version of the instrument or due to differences in the samples. Compared to the Kirby et al. (2005) study, this sample had fewer females, had more employed and racial minority participants and was, on average, older. There were also procedural differences in the time of administration of the instrument. In the Kirby et al. (2005) study, participants completed the instrument at the time of their first visit, before they had met with a counselor. In the present study, participants were given an opportunity to be briefly introduced to their counselor prior to completing their initial SOS-SA. It is possible that levels of distress decline quickly after CSOs contact a counselor even if treatment has not formally begun. A rapid decline in distress has been noted among SUIs during the first week after entering treatment (Husband et al., 1996).
The SOS-SA appears to have benefits over both the initial SOS interview and the small number of existing measures that have been used to assess problems experienced by family members with a substance abusing loved one. First, in contrast to the SOS interview, intensive training to administer and interpret the SOS-SA is not needed given its straight-forward format and item content. Second, it assesses a broader range of CSO problem areas than do other existing measures of family problems. Finally, an additional appealing feature of the SOS-SA is its ability to assess the frequency of problems as well as the degree of distress experienced by those CSOs.
The psychometric properties of the SOS-SA coupled with its simple format and broad-based content give credibility to its utility in both treatment and research settings. Counselors may find the SOS-SA appealing given that it can be used to provide clinically relevant information at two levels: the problem domain level and the specific problem item level. In addition to determining the broad life areas that are/are not problematic for a CSO, the items endorsed as occurring frequently and/or as being particularly distressing can be targeted and addressed in a timely manner. Those specific problems can also be incorporated into a treatment plan and then monitored over time to identify changes in their frequency and bothersomeness during the clinical intervention Similarly, substance abuse researchers may use the SOS-SA responses to look for changes in seven problem domains and specific items over time in pre-test/post-test clinical studies.
There are also limitations associated with this study. First, the 7-day maximum test-retest time period was established for practical reasons (clinic visits tended to be weekly) and may have allowed more intervening events than was optimal and increased the variability of the findings. A shorter test – retest interval (2 – 3 days) would have been sufficient to reduce the likelihood of participants remembering their previous responses, but reduced the opportunity for intervening events to change participants’ answers. The longer average interval that occurred (Mn=4.85 days) provides a conservative estimate of the SOS-SA coefficients, producing reliability estimates that may have been lower than would have been found had a more restrictive time window been used. Prior research indicates that a 7-day time window has proven acceptable for measures that assess multiple life domains similar to those found in the SOS-SA (e.g., physical health, mental health and emotional vitality, social functioning; Marx, Menezes, Horovitz, Jones, & Warren, 2003). Marx and his colleagues found no statistically significant differences in test-retest coefficients associated with the SF-36 based on both a 2-day and a 2-week time interval.
Another limitation is associated with the small number of items contained in the legal and health domains. Given that the number of scale items and internal consistency reliability are positively correlated, the small number of items associated with the health domain was likely to attenuate its reliability estimate (Hinkin, 1995). An improved measure would contain a larger number of items for the legal and health domains. Examples of legal and health items that were included in the interview but not the SOS-SA because of their low rates of endorsement include: paying SUI legal fines or lawyer fees; visits to a primary care physician or an emergency room; being informed by a physician of having a stress-related condition. These items may have been endorsed more frequently if an effort had been made to recruit CSOs of individuals involved in the criminal justice system and will be considered for inclusion in a revised SOS-SA.
Third, although a relationship between the problems reported by these CSOs and the substance abuse of their loved one is likely, these results cannot be used to indicate that the problems are a result of the SUI’s behavior. Since it is not possible to conduct the best test of a causal relationship by randomly assigning participants to have or not have a relationship with an SUI, future research might compare SOS-SA responses of CSO’s with those of a sample of demographically similar family members who do not have a loved one abusing substances.
Finally, the generalizability of these findings is somewhat limited based on two sets of demographic characteristics. First, the average educational level of the CSOs and their SUIs are relatively high when compared to many other addiction treatment studies. Many of these studies take place in public treatment programs or provide free treatment in a research setting. As a result they may disproportionally sample individuals who come from lower income levels and educational backgrounds. It is important to note, however, that substance abuse also occurs in middle and high income families. Therefore, our not-in-treatment sample may, in fact, be more representative of the variety of concerned family members in the US population and may not generalize to family members of individuals who are involved in treatment. Nonetheless, an effort should be made to include CSOs from a range of educational levels. In addition, CSOs of individuals thought to be abusing or dependent on alcohol, stimulants, or opiates were targeted for this study, whereas CSOs whose SUIs were dependent only on other classes of druges were not included. As such, the extent to which similar psychometric properties would result from SOS-SA data collected from a broader sample is not clear.
Although the SOS-SA is clearly a promising measure, further instrument development must capitalize on the measure’s existing strengths, begin to address its limitations, and also be guided by measurement development theory. With regard to the latter issue, further delineation of the interplay between the conceptual and measurement aspects of the SOS-SA is warranted. For instance, it is possible that the SOS-SA might best be conceptualized as a mixed measurement model that contains both causal (i.e., how often the event occurs) and effect (how bothersome the event is perceived to be) indicators that are associated with each of the seven problem domains (Bollen & Lennox, 1991). Models could be developed to evaluate the validity of this type of conceptualization and those findings, in turn, could be used for instrument revision purposes (e.g., as part of exploratory and confirmatory factor analyses).
The SOS-SA appears to have good initial psychometric properties and is likely to be useful to both clinicians and researchers. Given the paucity of existing broad-based psychometrically-validated measures designed to assess the problems experienced by CSOs, the psychometric properties of the SOS-SA provides support for continued efforts to refine the instrument.
Acknowledgments
This research was funded by NIH grants R01 DA12720, R01 DA018696 and P50 DA027841 from the National Institute of Drug Abuse. The authors would like to acknowledge Alicia Padovano and Stephanie Shealy for their data collection efforts.
References
- Benishek LA, Kirby KC, Dugosh KL. Prevalence and frequency of problems of concerned family members with a substance abusing loved one. American Journal of Drug and Alcohol Abuse. 2011;37(2):82–88. doi: 10.3109/00952990.2010.540276. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Benishek LA, Dugosh KL, Faranda-Diedrich TM, Kirby KC. Development of the Significant Other Survey: An interview for family members of drug users. American Journal of Family Therapy. 2006;34:1–14. [Google Scholar]
- Bollen K, Lennox R. Conventional wisdom on measurement: A structural equation perspective. Psychological Bulletin. 1991;110(2):305–314. [Google Scholar]
- Butler R, Bauld L. The parents’ experience: Coping with drug use in the family. Drugs: Education, Prevention, and Policy. 2005;12(1):35–45. [Google Scholar]
- Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized instruments in psychology. Psychological Assessment. 1994;4:284–290. [Google Scholar]
- Cicchetti DV, Sparrow SS. Developing criteria for establishing interrater reliability of specific items. Applications of assessment of adaptive behavior. American Journal of Mental Deficiency. 1981;86:127–137. [PubMed] [Google Scholar]
- Copello AG, Velleman RDB, Templeton LJ. Family interventions in the treatment of alcohol and drug problems. Drug and Alcohol Review. 2005;24:369–385. doi: 10.1080/09595230500302356. [DOI] [PubMed] [Google Scholar]
- Crisp BR, Barber JG. The Drinker’s Partner Distress Scale: An instrument for measuring the distress caused by drinkers to their partners. International Journal of Addictions. 1995;30(8):1009–1017. doi: 10.3109/10826089509055824. [DOI] [PubMed] [Google Scholar]
- Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297–334. [Google Scholar]
- Csiernik R. Counseling for the family: The neglected aspect of addiction treatment in Canada. Journal of Social Work Practice in the Addictions. 2002;2(1):79–92. [Google Scholar]
- Haber JR, Jacob T. Marital interactions of male versus female alcoholics. Family Process. 1997;36(4):385–402. doi: 10.1111/j.1545-5300.1997.00385.x. [DOI] [PubMed] [Google Scholar]
- Higgins J, Green S, editors. Cochrane handbook for systematic reviews of interventions. Chichester, West Sussex, England: Wiley-Blackwell; 2008. [Google Scholar]
- Hinkin TR. A review of scale development practices in the study of organizations. Journal of Management. 1995;21(5):967–988. [Google Scholar]
- Hornish GG, Leonard KE, Cornelius JR. Illicit drug use and marital satisfaction. Addictive Behaviors. 2008;33:279–291. doi: 10.1016/j.addbeh.2007.09.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hudson CR, Kirby KC, Firely ML, Festinger DS, Marlowe DB. Social adjustment of family members and significant others (FSOs) of drug users. Journal of Substance Abuse Treatment. 2002;23(3):171–181. doi: 10.1016/s0740-5472(02)00245-3. [DOI] [PubMed] [Google Scholar]
- Husband SD, Marlowe DB, Lamb RJ, Iguchi MY, Bux DA, Kirby KC, Platt JJ. Decline in self-reported dysphoria following treatment entry in inner-city cocaine addicts. Journal of Consulting and Clinical Psychology. 1996;64:221–224. doi: 10.1037//0022-006x.64.1.221. [DOI] [PubMed] [Google Scholar]
- Kahler CW, McCrady BS, Epstein EE. Sources of distress among women in treatment with their alcoholic partners. Journal of Substance Abuse Treatment. 2003;24(3):257–265. doi: 10.1016/s0740-5472(03)00033-3. [DOI] [PubMed] [Google Scholar]
- Kirby KC, Dugosh KL, Benishek LA, Harrington VM. The Significant Other Checklist: Measuring the problems experienced by family members of drug users. Addictive Behaviors. 2005;30(1):29–47. doi: 10.1016/j.addbeh.2004.04.010. [DOI] [PubMed] [Google Scholar]
- Kirby KC, Marlowe DM, Festinger DS, Garvey KA, LaMonaca V. Community reinforcement training for family and significant others of drug abusers: A unilateral intervention to increase treatment entry of drug users. Drug and Alcohol Dependence. 1999;56:85–96. doi: 10.1016/s0376-8716(99)00022-8. [DOI] [PubMed] [Google Scholar]
- Landau J, Garrett J, Shea RR, Stanton MD, Brinkman-Sull D, Baciewicz G. Strength in numbers: The ARISE method for mobilizing family and network to engage substance abusers in treatment. American Journal of Drug & Alcohol Abuse. 2000;26(3):379–298. doi: 10.1081/ada-100100251. [DOI] [PubMed] [Google Scholar]
- Lennox RD, Scott-Lennox JA, Holder HD. Substance abuse and family illness: Evidence from health care utilization and cost-offset research. Journal of Mental Health Administration. 1992;19(1):83–95. doi: 10.1007/BF02521310. [DOI] [PubMed] [Google Scholar]
- Lipscomb HJ, Dement JM, Leiming L. Health care utilization of families of carpenters with alcohol or substance abuse-related diagnoses. American Journal of Industrial Medicine. 2003;43:361–368. doi: 10.1002/ajim.10214. [DOI] [PubMed] [Google Scholar]
- Marx RG, Menezes A, Horovitz L, Jones EC, Warren RF. A comparison of two time intervals for test-retest reliability of health status instruments. Journal of Clinical Epidemiology. 2003;56:730–735. doi: 10.1016/s0895-4356(03)00084-2. [DOI] [PubMed] [Google Scholar]
- McLellan AT, Luborsky L, Cacciola J, Griffith J. New data from the Addiction Severity Index: Reliability and validity in three centers. Journal of Nervous and Mental Disease. 1985;173(7):412–423. doi: 10.1097/00005053-198507000-00005. [DOI] [PubMed] [Google Scholar]
- McNair DM, Loor M, Droppleman LF. Profile of Mood States Manual. New York: Multi-Health Systems; 2003. [Google Scholar]
- Meyers RJ, Miller WR, Smith JE, Tonigan JS. A randomized trial of two methods for engaging treatment-refusing drug users through concerned significant others. Journal of Consulting and Clinical Psychology. 2002;70(5):1182–1185. [PubMed] [Google Scholar]
- Murphy CM, O’Farrell TJ, Fals-Stewart W, Feehan M. Correlates of intimate partner violence among male alcoholic patients. Journal of Consulting & Clinical Psychology. 2001;69(3):528–540. doi: 10.1037/0022-006X.69.3.528. [DOI] [PubMed] [Google Scholar]
- Nunnally JC. Psychometric theory. New York: McGraw Hill; 1967. [Google Scholar]
- Oreo A, Ozgul S. Grief experiences of parents coping with an adult child with problem substance use. Addiction Research and Theory. 2007;15(1):71–83. [Google Scholar]
- Orford J, Templeton L, Velleman R, Copello A. Family members of relatives with alcohol, drug and gambling problems: A set of standardized questionnaires for assessing stress, coping, and strain. Addiction. 2005;100:1611–1624. doi: 10.1111/j.1360-0443.2005.01178.x. [DOI] [PubMed] [Google Scholar]
- Orford J, Natera G, Davies J, Nava A, Mora J, Rigby K, Bradbury C, Velleman R. Tolerate, engage or withdraw: A study of the structure of families coping with alcohol and drug problems in South West England and Mexico City. Addiction. 1998;93(12):1799–1813. doi: 10.1046/j.1360-0443.1998.931217996.x. [DOI] [PubMed] [Google Scholar]
- Ray GT, Mertens JR, Weisner C. Family members of people who alcohol or drug dependence: Health problems and medical cost compared to family members of people with diabetes and asthma. Addiction. 2008;104:203–214. doi: 10.1111/j.1360-0443.2008.02447.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rotunda RJ, West L, O’Farrell TJ. Enabling behavior in a clinical sample of alcohol-dependent clients and their partners. Journal of Substance Abuse Treatment. 2004;26:269–276. doi: 10.1016/j.jsat.2004.01.007. [DOI] [PubMed] [Google Scholar]
- Schneider Institute for Health Policy. Substance abuse: The nation’s number one health problem. Princeton, NJ: Robert Wood Johnson Foundation; 2001. [Google Scholar]
- Thomas EJ, Yoshioka M, Ager RD. Spouse enabling and alcohol abuse: Conception, assessment, and modification. Journal of Substance Abuse. 1996;8(1):61–80. doi: 10.1016/s0899-3289(96)90091-7. [DOI] [PubMed] [Google Scholar]
- Wekerle C, Wall A, editors. The violence and addiction equation: Theoretical and clinical issues in substance abuse and relationship violence. New York: Brunner-Routledge; 2002. [Google Scholar]
- Zumbo BD, Gadermann AM, Zeisser C. Ordinal versions of coefficients alpha and theta for Likert rating scales. Journal of Modern Applied Statistical Methods. 2007;6:21–29. [Google Scholar]