Psychometric Evaluation of the Parent Situation Inventory: A Role-Play Measure of Coping in Parents of Substance-Using Adolescents

Abstract

This article reports on the generalizability, reliability, and construct validity of the Parent Situation Inventory (PSI), a role-play measure of coping skills in parents experiencing problems from an adolescent’s drug and alcohol use. Generalizability was robust (.80) and alternate form and test–retest reliability were satisfactory. PSI skillfulness was negatively related to the parent’s own substance use and to the adolescent’s alcohol use. The PSI shows promise as a reliable and potentially valid measure of coping in this population and has direct implications for developing and evaluating skill-based parent training programs.

Family and parent factors (e.g., high conflict, poor communication) account, at least in part, for adolescent drug and alcohol use and abuse (e.g., Baumrind, 1991). However, this association can be bidirectional and reciprocal (e.g., Stice & Barrera, 1995). In other words, adolescent substance use and its consequences can also produce much family and parental stress and can result in ineffective parental coping. Assessing parents on coping skills related to adolescent substance use and training parents to cope more effectively with the sequelae of adolescent substance use may reduce parental stress, improve parenting skills, and reduce adolescent substance use.

Unfortunately, measures of parental skillfulness in coping with adolescent substance use are limited. To date, the literature in this area has relied on ratings of parent behavior in situations not related to substance use (e.g., Szapocznik, Kurtines, Foote, Perez-Vidal, & Hervis, 1986) or on measures of general coping and family functioning (e.g., Joanning, Quinn, Thomas, & Mullen, 1992). These measures, however, appear to have limited clinical utility for assessing how parents cope with the real-life issues of raising an adolescent engaging in drug and alcohol use. They also provide little direction for teaching more effective skills.

To address the measurement limitations in this area, we developed the Parent Situation Inventory (PSI; McGillicuddy, Rychtarik, Duquette, & Morsheimer, 2001). The PSI is a situation-specific role-play measure assessing drug-and-alcohol-related coping skills in parents of substance-using adolescents. Two alternate forms of the inventory (Forms X and Y) were constructed using the behavior-analytic model of scale development (Goldfried & D’Zurilla, 1969). A detailed description of PSI development is reported in McGillicuddy et al.’s (2001) study.

The PSI consists of 28 representative problem situation vignettes encountered by parents of substance-using adolescents. Table 1 provides a sample of representative PSI topic areas and problem situation descriptions. During PSI assessment, the parent reads and imagines being in a situation presented by an administrator. The parent is asked to imagine that the administrator is the adolescent and to say exactly what would be said and/or what would be done in the situation. The response is subsequently rated on a 6-point effectiveness scale (1 = worst; 6 = best) according to situation-specific scoring guidelines. In addition, the parent indicates on a 5-point scale each situation’s similarity (1= not at all; 5 = extremely) to situations encountered during the past year. In a recent pilot investigation (McGillicuddy et al., 2001), we found the PSI to be sensitive to skill-level changes following participation in a skill training program for parents experiencing problems from an adolescent’s substance use. Also, relative to a control condition, the skill training program resulted in moderate to large reductions in parental depression, anxiety, and anger, and in a reduction of adolescent marijuana use.

Table 1.

Representative Topic Areas and Description of Representative Situations on Parent Situation Inventory

Topic areas	Representative situation
Legal problems arise from the adolescent’s substance use	Parent receives telephone call indicating that adolescent has been arrested for drug possession
Parent believes that changes in adolescent are due to substance use	Adolescent has lost interest in sports and old friends; new and suspicious friends are hanging around
Parent is confronted by intoxicated/high adolescent	Intoxicated adolescent returns home three hours past curfew
Issues that result from the adolescent’s participation in substance abuse treatment	Adolescent refuses to speak with substance abuse counselor at initial treatment session
Adolescent substance use affects marital/family functioning	Parents disagree about discipline for substance-using adolescent

In the current study, we assessed the psychometric properties of the PSI from within generalizability theory (e.g., Brennan, 2001) and supplemented these analyses with alternate form and test–retest analyses. Generalizability theory was used because we conceptualized PSI skillfulness as representing a sampling of coping from a wide range of potential problem situations. A generalizability analysis allows for examination of how well the PSI score generalizes to all possible similarly sized and administered inventories of problematic situations that parents of a substance-using adolescent may confront. This approach is also useful for evaluating role-play tests because it can simultaneously apportion variance into multiple sources (e.g., person, rater, situation) and their interactions, and if needed, can provide direction for further inventory development (e.g., by varying the number of raters and situations). For our purposes, desirable variance on the PSI is that accounted for by skill differences between persons (i.e., parents) and the Person × Situation interaction.

We also assessed PSI construct validity by testing assumptions about the relationship between PSI skillfulness and parent and adolescent functioning. On the basis of prior research, we hypothesized that frequency of parental substance use would be associated positively with poor parenting (see Mayes & Truman, 2002) and, hence, with lower PSI scores. In addition, we assumed that PSI scores would be associated negatively with parental depression. The latter prediction was based on the assumption that not only would depression be associated with poor parenting (e.g., Goodman & Gotlib, 1999) but poor parenting would lead to increased adolescent problem behavior and higher parental depression. Also, on the basis of our work with a similar measure of coping in women with alcohol-dependent partners (Rychtarik & McGillicuddy, 1997), we hypothesized that PSI similarity (a presumed measure of burden) would be associated with higher parental depression and increased adolescent drug and alcohol use.

Method

Participants and Design

Participants were 200 parents recruited, via media advertisements and via fliers distributed at treatment centers, for a study of stress and coping in parents experiencing problems from an adolescent’s substance use. In addition, 75 adolescents accompanied a parent for a separate interview. To be eligible, the parent had to live with an adolescent (age 12–22) (a) whose substance use was reportedly causing family problems or (b) who was currently engaged in formal treatment or self-help for substance abuse.

The study used a 2 (adolescent treatment group: treatment vs. nontreatment) × 2 (PSI form: Form X vs. Form Y) factorial design. Parent participants were randomly assigned to PSI form with the stipulation that within each adolescent treatment group, approximately equal numbers of participants would be assigned to each form. All parents were scheduled to return for a second assessment 7–14 days following the initial assessment. Assignment to second assessment PSI form also was random, with the provision that within each treatment group, approximately 33% of the sample receive the same PSI form as administered initially and 67% receive the alternate form.

Measures

PSI

PSI responses were videotaped and rated for skill by trained research staff. A primary rater scored all tapes. A secondary rater scored 68% of the tapes, and an additional secondary rater scored the remainder. Secondary rater data were collapsed and treated as those of one rater. Individual rater scores were used for all generalizability analyses. For other analyses, the mean rater score was computed for each PSI situation per participant, and the mean situation score across the 28 situations was used as the PSI skill measure. The mean situation PSI score was 2.82 (SD = 0.34) and 2.76 (SD = 0.36) for Forms X and Y, respectively. The mean similarity rating across the 28 situations served as the PSI similarity measure; average situation similarity was 2.42 (SD = 0.86) and 2.30 (SD = 0.69) for the two respective forms.¹

Background and demographic variables

We assessed adolescent and parent age and gender (1 = male; 2 = female), parent years of education, and parent race. The majority of participants were biological parents (94%) and mothers (77%).² The sample averaged 39.50 years of age (SD = 5.88) with 12.77 years of education (SD = 2.77). With regard to race, 31% of the parents were White, 63.5% were Black, and 5.5% were either Native American, Hispanic American, or Asian American. Because descriptive data of the latter three groups was similar to that of the Black parents, race was transformed into a dichotomous variable (1 = White; 2 = non-White). Adolescent treatment group also was coded (1 = treatment; 2 = nontreatment). Parent reports indicated that their adolescent averaged 16.54 years of age (SD = 2.23), that 62% were male, and that 20% were receiving treatment or self-help. The 75 adolescents who participated were comparable to the full sample; they averaged 16.11 years of age (SD = 2.01), 60% were male, and 24% were receiving treatment or self-help. Regarding race, 35% of the participating adolescents were White, 64% were Black, and 1% were Native American.

Adolescent substance use

Parents were administered the Personal Experience Screening Questionnaire (PESQ; Winters, 1991) to assess problematic adolescent substance use. Parent PESQ scores suggested that 77% of the adolescents scored above the threshold indicating the potential need for a comprehensive drug use assessment.

Parent functioning variables

Parent substance use and depression were the parent functioning measures. The substance use measure was the proportion of days the parent used alcohol or illicit drugs over the previous 6 months. This variable was extracted from a timeline-followback procedure in which the parent provided a daily account of their alcohol and drug use (e.g., Fals-Stewart, O’Farrell, Freitas, McFarlin, & Rutigliano, 2000). On average, parents used substances on 20% of the days, with most of the use limited to alcohol. An arcsine transformation reduced positive skew.

The Beck Depression Inventory (Beck & Steer, 1993) was used to measure depression. It is comprised of 21 items scored on a 0–3 rating scale; higher scores indicate increased depressive symptoms. Average scores were 12.38 (SD = 9.30), suggesting that in general the sample was mildly depressed. A square-root transformation reduced positive skew.

Adolescent reports

The subset of 75 adolescents were interviewed using a timeline-followback procedure measuring their substance use over the previous 6 months. Reports of illicit drug use other than marijuana were low, so we focused only on reports of alcohol and marijuana use. On average, adolescents reported use of alcohol and marijuana on 19% and 42% of the days, respectively. An arcsine transformation of each variable reduced positive skew. Adolescents also were administered the PESQ; 87% of the participating adolescents scored above the threshold indicating the potential need for a further comprehensive drug use assessment.

Results

The distribution of PSI scores was normal and did not differ between forms. The forms also did not differ in mean PSI skillfulness or situation similarity rating. Also, initial PSI form assignment conditions did not differ significantly on any variable.

PSI Generalizability

Separate Person × Rater × Situation generalizability analyses of variance (Brennan, 2001) were conducted on each PSI form using a random-effects model. Resulting variance components are presented in Table 2. Within each form, 90%–91% of the variance was accounted for by differences between persons and by the Person × Situation interaction. Sources of error variance were low. The generalizability coefficient using the mean rater score of the 28-situation inventory was .80 for each form. This coefficient is equivalent to the expected value of the correlation between pairs of randomly parallel PSI inventories of similar length. Using variance component findings, we also computed generalizability coefficients for shorter PSI forms and for the scores of a single rater.³ The generalizability coefficient on the 28-item form was reduced only slightly with one rater (.76 for each form). Further, results suggested that a 20-item inventory with one rater would still achieve good generalizability (.70 for each form).

Table 2.

Estimated Variance Components and Proportion of Variance Accounted for in Generalizability Analysis of Parent Situation Inventory Forms X and Y

	Form X (n= 102)		Form Y (n= 98)
Source of variation	Estimated variance component	Proportion accounted for	Estimated variance component	Proportion accounted fora
Persons	.09691	.77	.10266	.75
Raters	.00214	.02	.00367	.03
Situations	.00229	.02	.00533	.04
Persons × Raters	.00149	.01	.00158	.01
Persons × Situations	.01818	.14	.02002	.15
Raters × Situations	.00002	.00	.00010	.00
Residual (Persons × Raters × Situations)	.00457	.04	.00438	.03

Note. Analyses were based on 2 raters and 28 situations.

^aProportion of variance accounted for is greater than 1.00 because of rounding.

Alternate Form and Test–Retest Reliability

Initial analyses revealed no significant effect for alternate form administration order (i.e., X–Y vs. Y–X), for time, or for the Order × Time interaction. As a result, the alternate form administration order groups were combined in an overall Person × Form × Rater × Situation within form generalizability analysis. Again, the majority of PSI variance was accounted for by differences between persons (77%) and the Person × Situation interaction within forms (10%). It is important to note that negligible proportions of variance were found for form (<1%), Rater × Form (<1%), and Person × Rater × Form (<1.5%). The Pearson correlation between Form X and Form Y was .76.

Test–retest correlations were .78 and .62 for Forms X and Y, respectively. The apparent stability difference between the forms seemed to result from less error variance from the Person × Testing Occasion interaction in Form X (18%) than in Form Y (31%). These findings must be viewed with caution given the small sample sizes (Ns = 33 and 27 in Forms X and Y, respectively). Nevertheless, exposure to PSI situations, in particular, those in Form Y, may influence future responses to the same situations. This influence is not systematic; mean scores were similar between the first and second testing occasions for both Form X, Ms = 2.83 (SD = .34) and 2.87 (SD = .38), respectively, and Form Y, Ms = 2.81 (SD = .35) and 2.81 (SD = .26), respectively.

Construct Validity

In preliminary analyses, bivariate relationships between the two PSI measures (i.e., PSI skillfulness and PSI similarity) and 13 variables were examined; significance levels were adjusted for the number of tests conducted on each variable (.05/13). PSI skillfulness was correlated significantly with parent race (r = −.31, p < .05), adolescent gender (r = −.27, p < .05), and parent years of education (r = .23, p < .05). When PSI skillfulness was simultaneously regressed on these variables, all contributed significantly; PSI skillfulness was higher among White parents, parents of a female, and parents with more years of education. Consequently, these variables served as covariates in the remaining regression models. Adolescent treatment status was correlated significantly with PSI similarity (r = −.25, p < .05); parents of adolescents receiving treatment or self-help viewed the PSI situations as more similar to situations in their own life. Consequently, treatment status was also included in all regression models. The two PSI measures were not significantly correlated with one another (r = −.11, ns).

Parent functioning

Hierarchical multiple regression analyses were conducted separately on parent substance use and parent depression (see Table 3). PSI skillfulness was inversely related to parental substance use. In addition, higher PSI similarity, but not PSI skillfulness, predicted elevated levels of parent depression.

Table 3.

Hierarchical Regression Analyses of Parent Substance Use Days and Depression on Demographic, Treatment Status, and Parent Situation Inventory (PSI) Measures

	Substance use daysa		Depressionb
Variable	R2	β	R2	β
Background	.01		.01
Adolescent gender		.03		.02
Parent years of education		.11		−.03
Parent race		.01		.00
Treatment group	.01	−.02	.01	.11
PSI	.08***e		.14****f
Similarityc		−.01		.38****
Skillfulnessd		−.29****		.05

Note. Significance levels in R² columns reflect significance of R² change; beta values are for the final, full regression model. Adolescent gender was coded 1 (male) or 2 (female). Parent race was coded 1 (White) or 2 (non-White). Treatment group was coded 1 (adolescent in treatment) or 2 (adolescent not in treatment).

^aProportion of previous 180 days that parent reported using alcohol or an illicit drug.

^bParent Beck Depression Inventory score.

^cMean situation similarity 1 (not similar at all) to 5 (extremely similar) of PSI situations.

^dMean situation skill level (potential range 1 [worst] to 6 [best]).

^eOverall F(6, 187) = 2.53, p < .05.

^fOverall F(6, 187) = 4.89, p < .001.

^***p < .01.

^****p < .001.

Adolescent functioning

Separate hierarchical regression models were conducted on the adolescent’s report of alcohol use and marijuana use. Table 4 shows that upon entry into the alcohol-use model, the block of PSI variables approached significance. Although the full model was not significant, results indicated that PSI skillfulness contributed negatively and significantly to adolescent alcohol use. Neither PSI variable accounted for variation in the adolescent’s report of marijuana use.

Table 4.

Hierarchical Regression Analyses on Adolescent Reports of Own Alcohol Use Days and Marijuana Use Days on Demographic, Treatment Status, and Parent Situation Inventory (PSI) Measures

	Alcohol use daysa		Marijuana use daysb
Variable	R2	β	R2	β
Background	.05		.05
Adolescent gender		.24		.12
Parent years of education		.14		−.01
Parent race		.13		.18
Treatment group	.06	−.01	.06	.07
Parent substance use daysc	.06	−.09	.09	.13
PSI	.14*f		.14g
Similarityd		−.20		.17
Skillfulnesse		−.32**		−.15

Note. Significance levels in R² columns reflect significance of R² change; beta values are for the final, full regression model. Adolescent gender was coded as 1 (male) or 2 (female). Parent race was coded as 1 (White) or 2 (non-White). Treatment group was coded as 1 (adolescent in treatment) or 2 (adolescent not in treatment).

^aProportion of previous 180 days that adolescent reported using alcohol.

^bProportion of previous 180 days that adolescent reported using marijuana.

^cProportion of previous 180 days that parent reported using alcohol or an illicit drug.

^dMean situation similarity 1 (not similar at all) to 5 (extremely similar) of PSI situations.

^eMean situation skill level (potential range 1 [worst] to 6 [best]) on PSI situations.

^fOverall F(7, 64) = 1.53, ns.

^gOverall F(7, 65) = 1.45, ns.

^*p < .10.

^**p < .05.

Discussion

The PSI appears to be a reliable and generalizable measure of parental skills in coping with adolescent substance use and its sequelae. We found partial support for hypothesized relationships between PSI score and parent and adolescent functioning. Contrary to our predictions, however, higher skill was not associated with lower levels of parental depression. Also, although PSI skillfulness was inversely related to adolescent alcohol use, it was unrelated to adolescent marijuana use. The limited range of parental skills may have precluded finding stronger relationships between the PSI score and measures of parent and adolescent functioning. Support for this notion is found in our earlier work in which parent skill training resulted in higher PSI score and reduced parent psychological distress and adolescent substance use (McGillicuddy et al., 2001).

It is noteworthy that non-White parents, and parents of male adolescents, scored relatively low on PSI skillfulness. The mechanisms accounting for these findings are unclear. It may be, as some studies suggest, that Black parents, and parents of adolescent males, discipline their children more harshly than other parent groups (e.g., Ferrari, 2002; Giles-Sims, Straus, & Sugarman, 1995); responses incorporating harsh discipline (e.g., yelling at the adolescent) generally receive low PSI scores. Also, PSI scoring rules were based on the consensual judgments of expert raters who were predominantly White and female (see McGillicuddy et al., 2001). The scoring rules may have differed had the raters been balanced along racial and gender lines. Of course, other factors that were not measured may have covaried with parent ethnicity and adolescent gender to account for the findings. Further research is needed to examine the impact that PSI skillfulness has on parent and adolescent functioning within and between different gender, ethnic, and racial groups.

Several study limitations should also be noted. First, study eligibility was based solely on parental reports that adolescent substance use was a problem. Reliance on parental reports may have resulted in some adolescents not meeting diagnostic criteria for substance abuse. Nonetheless, the majority of adolescents scored above the PESQ threshold suggesting the need for further comprehensive substance use assessment. Second, the relatively low number of participating adolescents may have reduced the statistical power of analyses examining the relationship between PSI skillfulness and the adolescent’s substance use. Third, within-form stability across time appeared to vary between forms and needs further study. Until additional work in this area can be done, we suggest that alternate PSI forms be used in any longitudinal analysis of parental skill. Finally, PSI validity findings are limited by the lack of an additional measure of coping skills; future studies need to compare PSI skillfulness to other parenting skill measures.

In sum, the PSI provides a reliable and generalizable measure of clinically relevant coping skills in parents of substance-using adolescents. PSI skillfulness appears to be related to adolescent functioning. Moreover, our recent preliminary work suggests that PSI skillfulness may be amenable to positive change through skill training interventions, which may improve parent and adolescent functioning. Though more evaluation is needed, the PSI shows promise as both a process and outcome measure of the effects of interventions for parents of substance-using adolescents.