Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Jan 1.
Published in final edited form as: Neuropsychol Rehabil. 2015 Jun 8;26(3):446–463. doi: 10.1080/09602011.2015.1044458

Factor Structure and Item Level Psychometrics of the Social Problem Solving Inventory Revised-Short Form in Traumatic Brain Injury

Chih-Ying Li 1, Julia Waid-Ebbs 2, Craig A Velozo 1, Shelley C Heaton 3
PMCID: PMC4965163  NIHMSID: NIHMS803620  PMID: 26052731

Abstract

Primary Objective

Social problem solving deficits characterize individuals with traumatic brain injury (TBI). Poor social problem solving interferes with daily functioning and productive lifestyles. Therefore, it is of vital importance to use the appropriate instrument to identify deficits in social problem solving for individuals with TBI. This study investigates factor structure and item-level psychometrics of the Social Problem Solving Inventory-Revised Short Form (SPSI-R:S), for adults with moderate and severe TBI.

Research Design

Secondary analysis of 90 adults with moderate and severe TBI who completed the SPSI-R:S.

Methods and Procedures

An exploratory factor analysis (EFA), principal components analysis (PCA) and Rasch analysis examined the factor structure and item-level psychometrics of the SPSI-R:S.

Main Outcomes and Results

The EFA showed three dominant factors, with positively worded items represented as the most definite factor. The other two factors are negative problem solving orientation and skills; and negative problem solving emotion. Rasch analyses confirmed the three factors are each unidimensional constructs.

Conclusions

The total score interpretability of the SPSI-R:S may be challenging due to the multidimensional structure of the total measure. Instead, we propose using three separate SPSI-R:S subscores to measure social problem solving for the TBI population.

Keywords: social problem solving, traumatic brain injury, factor structure, item structure, Rasch analysis

Introduction

Social problem solving is a goal-directed cognitive-behavioral process (D’Zurilla, Neuz, & Maydeu-Olivares, 2004), which involves defining problems, generating possible solutions, making decisions, and verifying or using solutions (D’Zurilla, & Goldfried, 1971). D’Zurilla et al. (1971) developed a multidimensional model of social problem solving consisting of two partially independent components: problem orientation and problem solving skills. Through several iterative processes, the current final model is represented with five social problem solving components: two problem-oriented dimensions (positive problem orientation, negative problem orientation) and three problem solving styles dimensions (rational problem solving, impulsive/careless problem solving style, and avoidance problem solving style) :, (D’Zurilla, & Goldfried, 1971; D’Zurilla, Neuz, & Maydeu-Olivares, 2002; D’Zurilla et al., 2004; Maydeu-Olivares, & D'Zurilla, 1996).

Social problem solving presents more challenges for individuals with traumatic brain injury (TBI; Hanten et al., 2008; Janusz, Kirkwood, Yeates, & Taylor, 2002; Krpan, Stuss, & Anderson, 2011, a &b; McDonald, Flashman, & Saykin, 2002; Rath, Hennessy, & Diller, 2003; Rath, Simon, Langenbahn, Sherr, & Diller, 2003; Robertson & Knight, 2008; Von Cramon, Matthes-von Cramon, & Mai, 1991). These clients struggle in a large part because a lack of self-awareness, including the incapacity to evaluate self-performance accurately and the inability to process information efficiently (Ashley, Ashley, & Kreber, 2012; Milders, Fuchs, & Crawford, 2003; Robertson & Knight, 2008). Not only do individuals with TBI demonstrate poor social problem solving in their everyday lives but they also demonstrate poor social problem solving in role-playing situations at the post-acute stage (Robertson & Knight, 2008). The devastating consequence of deficits in social problem solving result in the inability for this population to engage in productive activities (Green et al., 2008; Rath et al., 2003) and in a need for higher levels of supervision in order to remain safe in the community (Hart et al., 2003).

While impaired social problem solving can be a disabling deficit in adults with TBI, few studies have measured social problem solving for this population. Studies are especially lacking for a self-reported instrument. Two studies demonstrated the importance and benefits from using reliable self-reported measures assessing problem solving ability for the TBI or related populations. Cantor and colleagues (2014) used the composite executive function measure generated from four self-report tests (the Problem Solving Inventory, Frontal Systems Behavior Scale, Behavioral Assessment of the Dysexecutive Syndrome, and Self-Awareness of Deficits Interview) to evaluate the effectiveness of the Short-Term Executive Plus (STEP) cognitive rehabilitation program. The authors found improvement in self-reported post-TBI executive function but not from neuropsychological measures for the patients with TBI (Cantor, et al., 2014). In addition, Rath, Hradil, Litke, and Diller (2011) suggested that for the outpatients with acquired brain injury, use of objective measures addressing cognitive deficits are necessary, but not sufficient to provide practical and optimal information unless accompanied with patient’s subjective experiences of deficits. Thus, it is crucial to examine psychometric properties of self-reported problem solving instrument for the TBI population.

The Social Problem Solving Inventory-Revised (SPSI-R™) is a well-developed self-report instrument measuring individual’s social problem solving based on D’Zurilla and Nezu’s five-component Social Problem Solving Model. A shorter version is Social Problem Solving Inventory-Revised Short Form (SPSI-R: S), has the same five components as the longer version (D’Zurilla et al., 2002). The SPSI-R:S has efficient administration procedures, standardized norms (D’Zurilla et al., 2002) and has been used to measure social problem solving in a wide range of populations, including, but not limited to, people with low vision (Dreer et al., 2009; Dreer , Elliott, Fletcher, & Swanson, 2005), individuals with a recent suicidal attempt (Ghahramanlou-Holloway, Bhar, Brown, Olsen, & Beck, 2012), college students (Chang, 2002; Hawkins, Sofronoff, & Sheffield, 2009; Belzer, D’Zurilla, & Maydeu-Olivares, 2002) and family caregivers of persons with advanced cancer (Cameron, Shin, Williams, & Stewart, 2004). The SPSI-R: S has also been used internationally (Cameron et al., 2004; Hawkins et al., 2009).

Additionally, the SPSI-R:S has good psychometric properties, with Cronbach’s alpha ranging from .67 to .92 (Cameron et al., 2004; D’Zurilla et al., 2002; Hawkins et al., 2009; Wang et al., 2013) and test-retest reliability ranging from .72 to .87 (D’Zurilla et al., 2002; Hawkins et al., 2009). The five-components of the SPSI-R: S had been examined by confirmatory factor analysis (CFA), suggesting good model fit for college students (D’Zurilla et al., 2002; Hawkins et al., 2009). However, this result was recently challenged by Wang et al. (2013), who found the hypothesized five-factor structure proposed by D’Zurilla et al. (1971) did not fit the data well in a sample of adults who were overweight and obese (Wang et al., 2013).

In addition to the issue of inconsistent CFA results, there is a disagreement about how to interpret the scores of the SPSI-R: S. While the SPSI-R: S has five components, D’Zurilla et al. (2002) recommended using the total score to evaluate the general/overall problem solving functioning (Dreer et al., 2009; D’Zurilla et al., 2002). In a literature review, a variety of score interpretations were reported in the SPSI-R: S. Some studies only report a single total score (Chang, 2002) while others simply report five subscores (Cameron et al., 2004; Dreer et al., 2005); and still others report both (Hawkins et al., 2009; Wang et al., 2013).

Rasch analysis is one of the item response theory (IRT) methods that can examine item-level psychometrics and factor structures such as the unidimensionality of an instrument (Bond, & Fox, 2007, a & b; Drasgow, & Hulin, 1990; Velozo, Forsyth, & Kielhofner, 2006). However, studies using IRT methods to examine psychometric properties of the SPSI-R: S are lacking; only one study used Rasch analysis to create a 10-item short form of the SPSI-R: S (Dreer et. al, 2009). Studies using IRT to examine the dimensionality psychometrics of the entire SPSI-R: S is needed to determine the utility of the SPSI-R: S in individuals with TBI.

Given the social problem solving deficits experienced by individuals with TBI, it is imperative to measure social problem solving accurately using a valid and reliable self-reported assessment tool with appropriate score interpretation for this population. While the SPSI-R: S has been developed and tested in other populations, to date it has not been testing in individuals with TBI. The purpose of this exploratory study is to determine, whether the SPSI-R: S demonstrates acceptable dimensionality and IRT psychometrics to measure social problem solving for adults with TBI.

Methods

Participants

The responses of 90 participants with TBI were obtained from a larger study reviewed and approved by the Institute Review Board at the University of Florida. Participants were recruited from Shands Hospital and Brooks Health Systems in Florida, and the Shepherd Center in Georgia. Inclusion criteria consisted of: (a) diagnosis of a moderate/severe traumatic brain injury, defined as an injury to the head that resulted in loss of consciousness and required hospitalization; (b) ages from 18 to 85; (c) no previous diagnosis of schizophrenia or psychotic disorder; (d) no prior diagnosis of mental retardation; and (e) reported English as their first language.

Instrument

The Social Problem Solving Inventory Revised- Short Form (SPSI-R: S) is a 25-item, self-report instrument that evaluates characteristics of social problem solving, including problem solving orientation and problem solving performance (D’Zurilla & Nezu, 1990). The SPSI-R: S consists of five subscores: positive problem orientation (PPO), negative problem orientation (NPO), rational problem solving style (RPS), impulsivity/carelessness style (ICS), and avoidance style (AS). Each sub-score contains five items that are scored on a five-point Likert-type rating scale, ranging from 0 (not at all true) to 4 (extremely true). Each subscale scores range from 0 to 20, and the total scores of the SPSI-R: S range from 0 to 100. Higher subscores on PPO and RPS, and lower subscores of NPO, ICS, and AS indicate good social problem solving. For this study, the scores for 15 negatively worded items (NPO, ICS and AS) were reversed for all analyses to allow higher total scores to represent higher levels of social problem solving.

Data Analysis

Descriptive statistics were conducted with SPSS 20.0. Exploratory Factor Analysis (EFA) for ordinal data was conducted with Mplus 7.3. The Rasch residual principle component analysis (PCA) and item analysis were conducted with Winstep 3.75. The EFA and the Rasch residual PCA were used to examine factor structure. Rasch analysis was used to examine item-level psychometrics of the SPSI-R: S.

The EFA explores the number and nature of the underlying latent factors with no prior assumptions. The weighted least squares means and variance adjusted (WLSMV) was used as the factor extraction method in this study without assuming variables need to have a normal distribution and thus provides the best option for modelling categorical or ordered data (Brown, 2006; Proitsi et al., 2009). An oblique rotation with Geomin method was used in this study to allow factors to be correlated based on the assumption that the latent factors may be relevant to each other (Brown, 2009). Initial eigenvalues provided information from the initial solution with all possible factors, while the extraction sums of squared loadings were used to determine the final retained factors. Items with factor loadings less than 0.30 were not assigned to that factor. Factors were determined based on multiple extraction rules, including eigenvalues and eigenvalue plot, and cumulative percent of variance explained. Items with significant loading on more than one factor were assigned to the factor with higher loading value or consistent conceptual meanings with other items loaded to the same factor. Eigenvalues represent the amount of variance accounted for by each factor, while an eigenvalue plot was used to provide a cut-off point when additional factors failed to add significant cumulative explained variances. Factors were extracted when eigenvalues >1 (Kaiser, 1960) and the inflexion point occurred in the eigenvalue plot (Cattell, 1966). The Rasch residual PCA was used to assess if there were meaningful structures of residuals after extracting the primary Rasch dimension. First contrast in the Rasch residual PCA represents the first PCA component in the correlation matrix of the residuals after extracting the Rasch dimension (Linacre, 2004, 2010 & 2012). Linacre (2004, 2010 & 2012) suggests that unidimensionality of an instrument is supported when the Rasch dimension explains more than 40% variance of the data, the first contrast of the Rasch residual explains less than 5% variance of the data, and the eigenvalue of the first contrast is less than or equal to 2.0 (Linacre, 2004, 2010 & 2012).

Item-level analyses involved examining the rating scale structure; items fit statistics; person reliability/strata; person/item map and ceiling/floor effects. The rating scale was examined based on three criteria: a minimum of ten responses in each rating category, a monotonous pattern of category logit measure, and the outfit mean square value for each rating scale ±2.0 (Linacre, 1999; Linacre, 2002). Fit statistics is an index to measure the difference between the estimated scores of the Rasch model and the observed scores (Wu & Adams, 2013). Item fit analyses include an Infit mean square (Infit Mnsq) and Outfit mean square (Outfit Mnsq). A chi-square ratio of Infit and Outfit Mnsq within the range of 0.7 to 1.3 are indicators of good model fit along a standardized fit statistics (ZSTD) value in the range of ±2 based on Wu and Adams (2013)’s formula to calculate fit mean square range considering sample size: 1±√2/n (n=sample size). Point measure correlation is an index with a range of ±1 to show the correlation between the item observations and the corresponding person measures (Linacre, 1998). A value larger than the absolute value of 0.3 was considered acceptable. Person reliability less than 0.75 indicates low reliability; 0.75 to 0.95 indicates moderate reliability; and more than 0.95 indicates good reliability (Bond, & Fox, 2001). Person separation Index was used to determine the number of person ability strata/clinical group differences (Andrich, 1982). An item-person map was used to verify ceiling/floor effects. Ceiling effects are determined by the lack of items at the “difficult” end of the item-person map, while floor effect is determined by the lack of items at the “easy” end of the item-person map; in other words, items are not difficult or easy enough to separate individuals with different abilities (Velozo, et al., 2008).

Results

Demographics

Participants had a mean age of 39 years (SD=16.1), with a range from 18 to 84 years old. There were 61 males (67.78%) and 29 females (32.22%) in this study. The three major educational levels were “no-degree college” (27.8%), “12th grade” (21.1%) and “degree college” (15.6%). The majority of the participants were White (80%) and had the average time post-TBI more than 1 year (56.8%) (Table1). Participants were mainly outpatients (52.2%) with the average time post-TBI of 2.7 years (SD= 5.2), ranging from 26 days to 30 years (Table1). The mean score of the Disability Rating Scale (DRS) was 6.4 (SD= 4.5), while the mean score of the Glasgow Outcome Scale Extended (GOSE) was 32.0 (SD= 3.1). The majority of participants were not currently working (80%) nor driving (65.6%), could walk (74.4%) and transfer (86.7%), but did not use the cane (85.6%). The mean of Digital Symbol Coding test was 47.3 (SD=20.4) and the mean of the Galveston Orientation and Amnesia Test was 89.4 (SD= 20.7) (Table1).

Table 1.

Demographic characteristics of individuals with Traumatic Brain Injuries

Variables People with Traumatic Brain Injuries (n = 90)
No. %
Age (range: 18-84) Mean= 38.7 (SD= 16.07)
Average Time post-TBI (Month) Mean= 32.86 (SD= 62.24)
Min= 0.87 Max=360.77
Gender
  Male 61 67.78
  Female 29 32.22
Ethnicity
  White 72 80.0
  African American 10 11.1
  Hispanic 4 4.4
  Other 3 3.3
  Missing 1 1.1
Education
  8th grade 2 2.2
  10th grade 5 5.6
  11th grade 6 6.7
  12th grade 19 21.1
  GED 5 5.6
  No degree college 25 27.8
  Degree college 14 15.6
  4 year college 6 6.7
  Graduate/professional degree 8 8.9
Chronicity Status
  Outpatient 47 52.2
  More than 1-year post injury 43 47.8
Functional Status
  Currently Working (Yes) 17 (Yes) 18.9%
(No) 72 (No) 80%
  Currently Driving (Yes) 27 (Yes) 30%
(No) 59 (No) 65.6%
  Able to Transfer (Yes) 78 (Yes) 86.7%
(No) 12 (No) 13.3%
  Able to Walk (Yes) 23 (Yes) 74.4%
(No) 67 (No) 25.6%
Glasgow Outcome Scale Extended
(GOSE)		 Mean= 32.0 (SD= 3.1)
Disability Rating Scale (DRS) Mean= 6.4 (SD= 4.5)
Digital Symbol Coding test Mean= 47.3 (SD= 20.4)
Galveston Orientation and Amnesia
Test		 Mean= 89.4 (SD= 20.7)
Open in a new tab

Psychometric Properties

Six factors had an eigenvalue >1 (Table2). The eigenvalue plot showed three main latent factors in the EFA model while the fourth latent factor failed to add significant cumulative explained variances (Figure1). Ten positively worded items from the positive problem orientation (PPO) and the rational problem solving style (RPS) represented the third factor, the most definite latent factor. The 15 negatively worded items from the negative problem orientation (NPO), the impulsivity/carelessness style (ICS), and the avoidance style (AS) were unevenly distributed among the first (n=12) and the second (n=3) latent factors (Table2). Based on this EFA result, we concluded that the positive problem solving orientation and skills items compose one factor while the negative problem solving emotion items and the negative problem solving orientation and skills items are composed of the second and third factors. The PCA revealed that the Rasch dimension explained a total 23.7% of the variance in the data, which is below the unidimensional criteria of 40%. The variance explained by the items alone was 17.4%. After primary Rasch dimension was extracted, the first contrast explained 15.3% variance, nearly 3 times larger than the expected 5% variance (Table 3), indicating multidimensionality. While the PCA approach sets out to represent all of the variance in a set of variables across components, the EFA method attempts to understand the shared variance in a set of variables across latent factors (Brown, 2009; Matasunaga, 2010). In this study, PCA demonstrates multidimensionality of the scale and the EFA further reveals the multiple factor structure of the scale. Since the results of EFA and PCA both do not support the unidimensionality of the SPSI-R: S, we consequently conducted Rasch analysis for each of the three factors generated from the EFA results.

Table 2.

Variance Explained and Factor Loadings in the EFA model
Factor Eigenvalue
1 6.201
2 5.009
3 2.176
4 1.325
5 1.213
6 1.050
   
Domain Item Factor Loadings (Omit Loadings < 0.3)
Factor1 Factor2 Factor3
NPO 1 −0.521
ICS 2 0.507
NPO 3 −0.589
PPO 4 0.502
PPO 5 0.620
AS 6 0.458
NPO 7 0.675
NPO 8 0.743
PPO 9 0.591
AS 10 0.406
NPO 11 −0.368
RPS 12 0.578
PPO 13 0.756
ICS 14 0.434
PPO 15 0.592
RPS 16 0.790
AS 17 0.723
AS 18 0.813
RPS 19 0.745
ICS 20 0.524
RPS 21 0.804
AS 22 0.658
RPS 23 0.660
ICS 24 0.495
ICS 25 0.616
Open in a new tab

Note: PPO: positive problem orientation; NPO: negative problem orientation; RPS: rational problem solving style; ICS: impulsivity/carelessness style; AS: avoidance style

Figure 1. Eigenvalues Plot for Exploratory Factor Analysis.

Figure 1

Open in a new tab

This plot shows eigenvalues of each factor and cumulative explained variances added by each factor

Table 3.

Principal Component Analysis of Rasch Residuals Values

Standardized Residual Variance
Eigenvalue Percent
Total Raw Variance in Observations 32.7 100.0%
Raw Variance Explained by Measures 7.7 23.7%
Raw Variance Explained by Persons 2.1 6.3%
Raw Variance Explained by Items 5.6 17.4%
Raw Unexplained Variance (Total) 25.0 76.3%
Unexplained Variance in First Contrast 5.0 15.3%
Open in a new tab

Both the subscales of negative problem solving emotion items and the negative problem solving orientation and skills met the three rating scale diagnostic criteria, meaning each rating category had more than ten responses; the outfit statistics were within a range of ± 2.0 and showed a monotonous pattern (Table 4). The positive problem solving orientation and skills subscale did not meet the rating scale diagnostic of monotonicity, requiring the collapse of rating scale categories 0 and 1. Fit statistics demonstrated all items of all three subscales met the Infit Mnsq criteria within the range from 0.7 to 1.3, with ZSTD ranging within ±2.0 (Table 5). All items in the three subscales had point measure correlations larger than 0.3. We collapsed the rating scale categories of 0 and 1 of the positive problem solving orientation and skills subscale because these two categories violated monotonicity.

Table 4.

Rating Scale Diagnostics of Three Subscales

Category
Level		 Observed
Count		 % Observed
Average		 Sample
Expect		 INFIT
MMSQ		 OUTFIT
MMSQ		 Andrich
Threshold		 Original
Score
Negative Problem Solving Orientation and Skills (12 Items)
0 53 5 −.22 −.36 1.18 1.17 NONE 4
1 99 9 .12 .10 1.02 1.08 −.75 3
2 168 16 .48 .49 1.00 .91 −.23 2
3 305 29 .84 .93 1.17 .95 .10 1
4 428 41 1.59 1.54 .93 .96 .87 0
Negative Problem Solving Emotions (3Items)
0 10 5 −1.11 −1.29 1.15 1.07 NONE 4
1 23 11 −.61 −.39 .73 .65 −1.64 3
2 33 15 .49 .41 1.00 .89 −.35 2
3 78 36 1.41 1.38 .94 .96 .02 1
4 72 33 2.35 2.37 1.21 1.11 1.97 0
Positive Problem Solving Orientation and Skills (10 Items)
1 148 17 −.61 −.71 1.19 1.26 NONE 0,1
2 173 20 −.24 −.13 .85 .92 −.57 2
3 286 33 .44 .44 .84 .77 −.35 3
4 252 29 1.23 1.20 .98 .98 .92 4
Open in a new tab

Table 5.

Item Measure Table of the SPSI-R:S of Three Subscales

Item No. Score
 Infit
 Pt. Measure
Correlation
Raw Measure Mnsq ZSTD
Negative Problem Solving Orientation and Skills (12Items)
10 216 .59 1.19 1.3 .51
6 220 .59 1.27 1.9 .50
14 222 .57 1.18 1.3 .50
24 234 .43 .99 .0 .52
7 233 .41 .97 −.1 .55
2 261 .11 .70 −2.2 .45
25 264 .07 .98 .0 .55
20 284 −.21 1.24 1.3 .47
8 295 −.46 .84 −.9 .53
17 301 −.51 .99 .0 .51
18 306 −.61 .84 −.8 .57
22 322 −.99 1.13 .7 .44
Negative Problem Solving Emotions (3Items)
11 250 .54 1.25 1.5 .77
3 264 .23 .73 −1.7 .83
1 301 −.77 1.03 .2 .73
Positive Problem Solving Orientation and Skills (10 Items)
19 215 .54 .83 −1.3 .65
23 222 .42 1.09 .7 .58
21 224 .39 .73 −2.2 .70
5 234 .22 1.01 .1 .64
12 241 .05 1.21 1.5 .57
15 250 −.05 1.21 1.5 .58
13 259 −.21 .97 −.2 .64
16 259 −.21 .70 −2.3 .69
4 273 −.47 1.29 1.8 .55
9 283 −.68 1.10 .7 .59
Open in a new tab

Both first (negative problem solving orientation and skills) and second (negative problem solving emotion) subscales demonstrated ceiling effects with the mean person ability of 0.97 and 1.25 logits compared to mean item difficulty (zero). The item difficulty ranged from −.99 to .59 logits (range = 1.44) for the first subscale, and from −.77 to .54 logits (range = 1.31) for the second subscale (Table 5). For the third subscale (positive problem solving orientation and skills), there was no evidence of floor or ceiling effects. The spread of item difficulty matched with person ability levels; with mean person ability only 0.34 logits higher than the mean item difficulty (Table5).

Both the first and second subscales had low person separation reliability of .73 and .49, respectively, separating the sample into 2.49 and 1.64 person strata, respectively. The third subscale had a moderate person separation reliability of .80, separating the sample into 2.96 person strata. In sum, the first and the second subscales distinguished the sample into two hierarchical ability levels while the third subscale distinguished the sample into three hierarchical ability levels.

Discussion

Social problem solving skills are important for community functioning and quality of life and are a common problem area for individuals who have sustained moderate/severe traumatic brain injury (TBI). Though measures assessing this important skill area exist, research is limited regarding their psychometric properties and utility in assessing this domain in TBI. The current study examined these issues using the SPSI-R: S and identified a multidimensional structure of social problem solving skills in adults with moderate to severe TBI. This study highlights the utility of the SPSI-R: S as an assessment tool in TBI, but suggests that the generated total score appears to be multidimensional and may be difficult to interpret. Current study results also highlight important considerations regarding item wording (positive vs negatively worded items) when developing or refining self-report measures of social problem solving for use in the TBI population.

The PCA of Rasch residuals approach determines whether the residuals are random noise and extracts the primary component explaining the largest potential amount of variance in the residuals (Linacre, 2014). While, the EFA method attempts to optimize/maximize commonalities among variables using rotation methods to obtain the strongest possible factor structure across a set of variables (Brown, 2009). In this study, PCA and EFA yielded consistent results about the factor structure of the SPSI-R: S. The PCA demonstrated a potential secondary dimension beyond the primary Rasch dimension. The EFA further revealed the details of the factor structure of the scale, by identifying the scale had three main factors. Both the results of the EFA and the Rasch residual PCA demonstrated multidimensionality of the SPSI-R: S.

The EFA yielded a six latent factor model, of which three-factor were extracted. Ten positively worded items (positive problem solving orientation and skills) constituted the most distinct latent factor, and negative problem solving emotion, and negative problem solving orientation and skills constituted the other two factors. The items of Impulsivity/Carelessness Style scale and Avoidance Style scale may not be representative of the constructs originally proposed (D’Zurilla et al., 2002; D’Zurilla et al., 2004) because the items were distributed among a variety of latent factors divergently. One possible reason is the emotional nature of these negatively worded items; for example, “feel threatened” and “make me upset,” could represent a construct other than negative problem solving skills. Previous studies also reported that the SPSI-R: S moderately correlated with a variety of psychological constructs, including psychological distress, well-being, and depressive symptoms (Dreer et al., 2005; Hawkins et al., 2009), suggesting that the SPSI-R:S may represent an emotional aspect.

The three-factor model of the SPSI-R: S demonstrated adequate item-level psychometrics based on the results of each Rasch analysis. The rating scale of 0 and 1 were collapsed to improve rating scale clarity and avoid monotonicity violations for the positively-worded items (third factor). The third factor, positive problem solving orientation and skills of the SPSI-R: S, distinguishes three levels of social problem solving ability in this sample of individuals with TBI, reflecting an average level of measurement precision. However, the remaining two factors, negative problem solving orientation and skills, and negative problems solving emotion, could only distinguish people into two different functional levels. In other words, the positive worded items of the SPSI-R: S separated this sample of adults with TBI into low, medium and high levels of social problem solving abilities; representing a better measurement precision compared to the negative worded items.

While the SPSI-R: S is an efficient self-report measure of social problem solving, caution may be required when interpreting scores in the TBI population due to its fractionated factor structure. In general, the results of this study did not support to use the total score of the SPSI-R: S to interpret an individuals’ social problem solving ability. Additionally, we suggest that in our sample, the SPSI-R: S represents three domains instead of the currently published five domains. The three domains were positive problem solving orientation and skills; negative problem solving orientation and skills and negative problem solving emotion. However, one must consider the significant limitation of our small sample size in this study. Based on the level of communality of the variables and the level of over-determination of the factors, Hogarty et al. (2005) recommended the appropriate sample size as 500 subjects in order to achieve accurate estimates when conducting exploratory factor analysis. Thus, small sample size in this study is a significant limitation regarding the EFA results, and may impede EFA explanations in this study and subsequent EFA replications. Since Rasch analyses, which do not require a large sample size (Smith, Rush, Fallowfield, Velikova, & Sharpe, 2008; Wang, & Chen, 2005), supported the unidimensionality of each of the three domains, we suggest using three subscores instead of the total score of the SPSI-R: S for the adults with TBI. Since the items of the scale do not represent a unitary concept, the total score may obscure information about each of the three unitary concepts when combined.

Since positively and negatively worded items load on different factors, consideration should be given to using one or the other, but not both. Use of items with reverse-worded or negatively and positively worded items does not appear to prevent or reduce response bias (van Sonderen, Sanderman, & Coyne, 2013). Thus, we suggest that in future studies, all items should be worded in one direction, either positively or negatively worded, since changing between positively and negatively worded items could further add cognitive burden for the individuals with TBI, as well as cause additional measurement error (van Sonderen, Sanderman, & Coyne, 2013). Furthermore, although an acceptable match was found between item difficulty and social problem solving ability of our study sample using the SPSI-R: S, results demonstrated that the average person ability level is higher than the average item difficulty level of the SPSI-R: S. Thus, we suggest that the SPSI-R: S is more appropriately used for the patients with more severe social problem solving problems.

Future studies should examine whether brain injury severity and/or cognitive impairment severity is associated with severity of social problem solving deficits. This would aid identification of those patient subgroups best suited for assessment of social problem solving. Future studies are also needed to identify appropriate assessment methods for patients with more subtle social problem solving difficulties.

Conclusion

This study represents the first examination of the SPSI-R: S in a sample of adults who had sustained moderate to severe TBI. Because of the multidimensionality of the SPSI-R: S, and majority variance of the scale not being explained by the measurement components of the SPSI-R:S, we recommend using the three subscores over the total score of the SPSI-R:S to represent the social problem solving ability of the patients. We also suggest potential future revisions of the SPSI-R: S, such as using three subscales based on its factor structure and using one-direction worded items.

Acknowledgements

This study is funded by National Institute of Health (NIH) R21 (Project #HD045869) “Developing a Computer Adaptive Traumatic Brain Injury Cognitive Measure.” and from a Rehabilitation Research and Development Service, Department of Veterans Affairs’ Pre-Doctoral Fellowship Award. The authors are also grateful for the editing assistance of the Center for Academic Excellence & Writing Center at the Medical University of South Carolina.

References

  1. Andrich D. An index of person separation in latent trait theory, the traditional KR20 index, and the Guttman scale response pattern. Educational Psychology Research. 1982;9:95–104. [Google Scholar]
  2. Ashley MJ, Ashley J, Kreber L. Remediation of information processing following traumatic brain injury: A community-based rehabilitation approach. NeuroRehabilitation. 2012;31(1):31–9. doi: 10.3233/NRE-2012-0772. doi: 10.3233/NRE-2012-0772. [DOI] [PubMed] [Google Scholar]
  3. Belzer KD, D’Zurilla TJ, Maydeu-Olivares A. Social problem solving and trait anxiety as predictors of worry in a college student population. Personality and Individual Differences. 2002;33:573–585. doi:10.1016/S0191-8869(01)00173-8. [Google Scholar]
  4. Bond TG, Fox CM. Applying the Rasch model : fundamental measurement in the human sciences. Lawrence Erlbaum Associates Publishers; Mahwah, N.J.: 2001. Applying the Rasch Model. [Google Scholar]
  5. Brown JD. Choosing the right type of rotation in PCA and EFA. Shiken: JALT Testing & Evaluation SIG Newsletter. 2009;13(3):20–25. ISSN 1881-5537. [Google Scholar]
  6. Brown T. Confirmatory factor analysis for applied research. The Guildford Press; New York, NY: 2006. [Google Scholar]
  7. Cameron JI, Shin JL, Williams D, Stewart DE. A brief problem-solving intervention for family caregivers to individuals with advanced cancer. Journal of Psychosomatic Research. 2004;57(2):137–43. doi: 10.1016/S0022-3999(03)00609-3. doi:10.1016/S0022-3999(03)00609-3. [DOI] [PubMed] [Google Scholar]
  8. Cantor J, Ashman T, Dams-O'Connor K, Dijkers MP, Gordon W, Spielman L, Tsaousides T, Allen H, Nguyen M, Oswald J. Evaluation of the short-term executive plus intervention for executive dysfunction after traumatic brain injury: a randomized controlled trial with minimization. Archives of Physical Medicine and Rehabilitation. 2014;95(1):1–9.e3. doi: 10.1016/j.apmr.2013.08.005. doi: 10.1016/j.apmr.2013.08.005. [DOI] [PubMed] [Google Scholar]
  9. Cattell RB. The scree test for the number of factors. Multivariate Behavioral Research. 1966;1(2):245–76. doi: 10.1207/s15327906mbr0102_10. [DOI] [PubMed] [Google Scholar]
  10. Chang EC. Examining the link between perfectionism and psychological maladjustment: Social problem solving as a buffer. Cognitive Therapy and Research. 2002;26(5):581–95. [Google Scholar]
  11. Costello AB, Osborne JW. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation. 2005;10(7):1–9. [Google Scholar]
  12. Drasgow F, Hulin CL. Item response theory. In: Dunnette MD, Hough LM, editors. Handbook of industrial and organizational psychology. 2nd I. Consulting Psychologist Press; Pa10 Alto, CA: 1990. pp. 577–636. [Google Scholar]
  13. Dreer LE, Berry J, Rivera P, Snow M, Elliott TR, Miller D, Little TD. Efficient assessment of social problem-solving abilities in medical and rehabilitation settings: a Rasch analysis of the Social Problem-Solving Inventory-Revised. Journal of Clinical Psychology. 2009;65(7):653–69. doi: 10.1002/jclp.20573. doi: 10.1002/jclp.20573. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Dreer LE, Elliott TR, Fletcher DC, Swanson M. Social problem-solving abilities and psychological adjustment of persons in low vision rehabilitation. Rehabilitation Psychology. 2005;50(3):232–8. [Google Scholar]
  15. D’Zurilla TJ, Goldfried MR. Problem solving and behavior modification. Journal of Abnormal Psychology. 1971;78:107–126. doi: 10.1037/h0031360. [DOI] [PubMed] [Google Scholar]
  16. D’Zurilla TJ, Nezu AM, Maydeu-Olivares A. Social problem-solving inventory-revised (SPSI-R): Technical manual. Multi-Health Systems, Inc.; North Tonawanda, NY: 2002. [Google Scholar]
  17. D’Zurilla TJ, Neuz AM, Maydeu-Olivares A, Chang EC, D'Zurilla TJ, Sanna LJ. Social problem solving: Theory, research, and training. American Psychological Association; Washington, DC: 2004. Social problem solving: theory and assessment. [Google Scholar]
  18. D’Zurilla TL, Nezu AM. Development and Preliminary Evaluation of the Social Problem-Solving Inventory. A Journal of Consulting and Clinical Psychology. 1990;2(2):156–163. [Google Scholar]
  19. Ghahramanlou-Holloway M, Bhar SS, Brown GK, Olsen C, Beck AT. Changes in problem-solving appraisal after cognitive therapy for the prevention of suicide. Psychological Medicine. 2012;42(6):1185–93. doi: 10.1017/S0033291711002169. doi: 10.1017/S0033291711002169. [DOI] [PubMed] [Google Scholar]
  20. Green RE, Colella B, Hebert DA, Bayley M, Kang HS, Till C, Monette G. Prediction of return to productivity after severe traumatic brain injury: investigations of optimal neuropsychological tests and timing of assessment. Archives of Physical Medicine and Rehabilitation. 2008;89(12 Suppl):S51–60. doi: 10.1016/j.apmr.2008.09.552. doi: 10.1016/j.apmr.2008.09.552. [DOI] [PubMed] [Google Scholar]
  21. Hambleton RK, Jones RW. Comparison of classical test theory and item response theory and their applications to test development. Educational Measurement: Issues and Practice. 1993;12(3):38–47. doi: 10.1111/j.1745-3992.1993.tb00543.x. [Google Scholar]
  22. Hanten G, Wilde EA, Menefee DS, Li X, Lane S, Vasquez C, Chu Z, Ramos MA, Yallampalli R, Swank P, Chapman SB, Gamino J, Hunter JV, Levin HS. Correlates of social problem solving during the first year after traumatic brain injury in children. Neuropsychology. 2008;22(3):357–70. doi: 10.1037/0894-4105.22.3.357. doi: 10.1037/0894-4105.22.3.357. [DOI] [PubMed] [Google Scholar]
  23. Hart T, Millis S, Novack T, Englander J, Fidler-Sheppard R, Bell KR. The relationship between neuropsychologic function and level of caregiver supervision at 1 year after traumatic brain injury. Archives of Physical Medicine and Rehabilitation. 2003;84(2):221–30. doi: 10.1053/apmr.2003.50023. doi: http://dx.doi.org/10.1053/apmr.2003.50023. [DOI] [PubMed] [Google Scholar]
  24. Hawkins D, Sofronoff K, Sheffield J. Psychometric properties of the Social Problem Solving Inventory-Revised Short-Form: Is the short form a valid and reliable measure for young adults? Cognitive Therapy & Research. 2009;33(5):462–70. [Google Scholar]
  25. Janusz JA, Kirkwood MW, Yeates KO, Taylor HG. Social problem-solving skills in children with traumatic brain injury: long-term outcomes and prediction of social competence. Child Neuropsychology. 2002;8(3):179–94. doi: 10.1076/chin.8.3.179.13499. doi: 10.1076/chin.8.3.179.13499. [DOI] [PubMed] [Google Scholar]
  26. Kaiser HF. The application of electronic computers to factor analysis. Educational and Psychological Measurement. 1960;20:141–51. doi: http://dx.doi.org/10.1177/001316446002000116. [Google Scholar]
  27. Krpan KM, Stuss DT, Anderson ND. Planful versus avoidant coping: behavior of individuals with moderate-to-severe traumatic brain injury during a psychosocial stress test. Journal of the International Neuropsychological Society. 2011a;7(2):248–55. doi: 10.1017/S1355617710001499. doi: 10.1017/S1355617710001499. [DOI] [PubMed] [Google Scholar]
  28. Krpan KM, Stuss DT, Anderson ND. Coping behaviour following traumatic brain injury: what makes a planner plan and an avoider avoid? Brain Injury. 2011b;25(10):989–96. doi: 10.3109/02699052.2011.597045. doi: 10.3109/02699052.2011.597045. [DOI] [PubMed] [Google Scholar]
  29. Linacre JM. Table 13.1 Item statistics in measure order. Rasch Measurement Forum. 1998 Retrieved on 12/12/2013 from http://www.winsteps.com/winman/index.htm? correlations.htm.
  30. Linacre JM. Investigating rating scale category utility. Journal of Outcome Measurement. 1999;3(2):103–122. [PubMed] [Google Scholar]
  31. Linacre JM. Optimizing rating scale category effectiveness. Journal of Applied Measurement. 2002;3(1):85–106. [PubMed] [Google Scholar]
  32. Linacre JM. Rasch model estimation: Further topics. Journal of Applied Measurement. 2004;5:95–110. [PubMed] [Google Scholar]
  33. Linacre JM. Predicting responses from Rasch measures. Journal of Applied Measurement. 2010;11:1–10. [PubMed] [Google Scholar]
  34. Linacre JM. A user’s guide to Winsteps ministep 3.70.0: Rasch model computer programs. Winsteps; Chicago, IL: 2012. [Google Scholar]
  35. Linacre M. Dimensionality: contrasts & variances. 2014 Retrieved on 1/26/2015 from Rasch Measurement Forum at http://www.winsteps.com/winman/principalcomponents.htm.
  36. Matasunaga M. How to factor-analyze your data right: Do's, don'ts, and how-to's. International Journal of Psychological Research. 2010;3(1):97–110. [Google Scholar]
  37. Maydeu-Olivares A, D'Zurilla TJ. A Factor-Analytic Study of the Social Problem-Solving Inventory: An Integration Theory and Data. Cognitive Therapy and Research. 1996;20(2):115–133. [Google Scholar]
  38. McDonald BC, Flashman LA, Saykin AJ. Executive dysfunction following traumatic brain injury: neural substrates and treatment strategies. Journal of Neurorehabilitation and Neural Repair. 2002;17(4):333–44. [PubMed] [Google Scholar]
  39. Milders M, Fuchs S, Crawford JR. Neuropsychological impairments and changes in emotional and social behaviour following severe traumatic brain injury. Journal of Clinical Experimental Neuropsychology. 2003;25(2):157–72. doi: 10.1076/jcen.25.2.157.13642. [DOI] [PubMed] [Google Scholar]
  40. Proitsi P, Hamilton G, Tsolaki M, Lupton M, Daniilidou M, Hollingworth P, Archer N, Foy C, Stylios F, McGuinness B, Todd S, Lawlor B, Gill M, Brayne C, Rubinsztein DC, Owen M, Williams J, Craig D, Passmore P, Lovestone S, Powell JF. A multiple indicators multiple causes (MIMIC) model of behavioural and psychological symptoms in dementia (BPSD) Neurobiology Aging, 2009 doi: 10.1016/j.neurobiolaging.2009.03.005. doi:10.1016/j.neurobiolaging.2009.03.005. [DOI] [PubMed] [Google Scholar]
  41. Rath JF, Hennessy JJ, Diller L. Social problem solving and community integration in post-acute rehabilitation outpatients with traumatic brain injury. Rehabilitation Psychology. 2003;48(3):137–44. [Google Scholar]
  42. Rath JF, Simon D, Langenbahn DM, Sherr RL, Diller L. Group treatment of problem-solving deficits in outpatients with traumatic brain injury: A randomised outcome study. Neuropsychological Rehabilitation: An International Journal. 2003;13(4):461–488. [Google Scholar]
  43. Rath JF, Hradil AL, Litke DR, Diller L. Clinical applications of problem-solving research in neuropsychological rehabilitation: addressing the subjective experience of cognitive deficits in outpatients with acquired brain injury. Rehabilitation Psychology. 2011;56(4):320–8. doi: 10.1037/a0025817. doi: 10.1037/a0025817. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Robertson RH, Knight RG. Evaluation of social problem solving after traumatic brain injury. Neuropsychology Rehabilitation. 2008;18(2):236–50. doi: 10.1080/09602010701734438. [DOI] [PubMed] [Google Scholar]
  45. Smith AB, Rush R, Fallowfield LJ, Velikova G, Sharpe M. Rasch fit statistics and sample size considerations for polytomous data. BMC Med Res Methodology. 2008;8(33):1–11. doi: 10.1186/1471-2288-8-33. doi: 10.1186/1471-2288-8-33. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Velozo CA, Forsyth K, Kielhofner G. Objective measurement: The influence of item response theory on research and practice. In: Kielhofner G, editor. Research in occupational therapy. F.A. Davis; Philadelphia: 2006. pp. 177–200. [Google Scholar]
  47. Velozo CA, Wang Y, Lehman L, Wang JH. Utilizing Rasch measurement models to develop a computer adaptive self-report of walking, climbing, and running. Disability Rehabilitation. 2008;30(6):458–467. doi: 10.1080/09638280701617317. [DOI] [PubMed] [Google Scholar]
  48. Von Cramon DY, Matthes-von Cramon G, Mai N. Problem-solving deficits in brain injured patients: A therapeutic approach. Neuropsychological Rehabilitation. 1991;1(1):45–64. [Google Scholar]
  49. Wang WC, Chen CT. Item parameter recovery, standard error estimates, and fit statistics of the Winsteps program for the family of rasch models. Educational and Psychological Measurement. 2005;65(3):376–404. doi: 10.1177/0013164404268673. [Google Scholar]
  50. Wang J, Matthews J, Sereika S, Chasens E, Ewing L, Burke L. Psychometric evaluation of the Social Problem-Solving Inventory−Revised among overweight or obese adults. Journal of Psychoeducational Assessment. 2013;31(6):585–590. doi: 10.1177/0734282913480470. doi: 10.1177/0734282913480470. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Wright B, Linacre J. Observations are always ordinal; measurements, however, must be interval. Archives of Physical Medicine and Rehabilitation. 1989;71:857–860. [PubMed] [Google Scholar]
  52. Wu M, Adams RJ. Properties of Rasch residual fit statistics. Journal of Applied Measurement. 2013;14(4):339–55. [PubMed] [Google Scholar]

RESOURCES