Comment on “Shoulder Pain and Disability Index: Italian cross-cultural validation in patients with non-specific shoulder pain”

Dear Editor,

We read with interest the paper by Brindisino et al.,¹ performing the Italian cross-cultural validation of the Shoulder Pain and Disability Index (SPADI) in individuals with non-specific shoulder pain. While we recognize the uttermost importance of using psychometrically sound Patient-Reported Outcome Measures (PROMs) in research and clinical settings, we are concerned about some methodological aspects which may lead to an overestimation of study findings.

It is our opinion that the instrument has not been appropriately validated. According to the more recent guidelines for psychometric assessment of PROMs (i.e., COnsensus-based Standards for the selection of health Measurement INstruments [COSMIN]^2,3), three measurement domains should be investigated, i.e., reliability, validity, and responsiveness. The domain reliability refers to three measurement properties: internal consistency, reliability (i.e., test-retest, inter/intra-rater), and measurement error. Validity includes content validity, construct validity (i.e., including structural validity, hypotheses testing, and cross-cultural validity), and criterion validity.

Structural validity

COSMIN suggests a minimum of 100 subjects for studying the factorial structure.⁴ In this study, the structural validity was investigated through Principal Component Analysis (PCA) and Confirmatory factor analysis (CFA) using a sample size of 59 individuals that may have impaired the power of CFA. This problem may explain why indices such as the Comparative Fit Index (CFI) and the Root Mean Square Error of Approximation (RMSEA) are not acceptable (i.e., lower than 0.95 and higher than 0.06, respectively, as suggested by the literature⁵) indicating a poor fit-to-the-model of the two-factor structure. Furthermore, the sentence of the discussion section “factor analysis in our data revealed that SPADI-I has two factors, with six items (four pain/two disability) weighting the first factor by >0.5, and seven items (six disability/one pain) weighting the second factor” depicts a different scenario from the one derived from the CFA path diagram. In fact, it is unclear the reason for which the items and the corresponding loadings of PCA reported on the right side of Figure 2 do not correspond to the items and corresponding loadings reported in the path diagram of CFA on the left part of Figure 2. Indeed, in the PCA, the principal components are identified upon two principal components, the first composed of 6 items (1, 2, 3, 5, 8, and 11) and the second composed of 7 items (4, 6, 7, 9, 10, 12, and 13). Unfortunately, the diagram of the CFA confirmed a different structure, with the first domain composed of 7 items (1, 2, 3, 5, 7, 8, and 11), 4 from pain and 3 from disability original subscales, and the second composed by 6 items (4, 6, 9, 10, 12, and 13), 5 from disability and 1 from pain original subscales.

Construct validity

COSMIN guidelines² recommend using a priori hypotheses testing method. According to this perspective, when dealing with construct validity using correlation coefficients, we should be interested in the strength and direction of the association rather than on p-values that depend highly on the sample size. The hypothesis testing method recommends generating a priori list of hypotheses on the strength and direction of association with the other administered measurement instruments considered. For example, the pain subscale showed a moderate correlation (r = 0.53) with the Visual Analogue Scale which is intended to measure the same construct, thus expecting to be r ≥ 0.60. Furthermore, it is not clear which subscales have been used for this estimation, as two different subscales emerged according to structural validity.

Internal consistency

Regarding the study of internal consistency, we found that result section did not follow what described in the methods section. In fact, in the methods the Authors “analysed internal consistency by performing a principal component analysis (PCA) followed by a confirmatory factor analysis (CFA)” and then “Internal consistency was tested using Cronbach’s alpha value of correlations between all the items of the SPADI-I scale and for each of the subscales.” Therefore, it seems that internal consistency was assessed on the factors extracted from CFA. However, as detailed in Table 7, the internal consistency was calculated on the original subscales of the questionnaire, which are called pain (5 items) and disability (8 items) in contrast to the reported structural validity of the scale, as mentioned above. Therefore, we struggle to understand the reason that leads the Authors to calculate the internal consistency on the original subscales of the questionnaire instead of on those extracted.

Measurement error

Standard Error of Measurement (SEM) and Minimally Detectable Change (MDC) are usually calculated after the calculation of a proper test–retest reliability coefficient (e.g., Intraclass Correlation Coefficient), using the formula $\mathit{SEM}=\text{SD}\sqrt{1-\mathit{reliabilitycoefficient}}$, where SD is the standard deviation of the observed scores.⁶ The assessment of the test–retest reliability requires repeated (usually two) measurements which were not reported in the study we are commenting on.

To this end, we believe that the SPADI questionnaire should not be recommended for its use in clinical practice and research because of the several concerns we have raised in this letter.

ORCID iDs

Daniele Piscitelli https://orcid.org/0000-0002-5240-3798

Tommaso Geri https://orcid.org/0000-0001-9198-900X