Phone Administration of the Western Ontario Shoulder Instability Index Is More Reliable Than Administration via Email

Abstract

Background

The Western Ontario Shoulder Instability (WOSI) questionnaire is a 21-item questionnaire to evaluate quality of life in patients with shoulder instability. Completing the questionnaire is time-consuming because each item is evaluated on a visual analog scale. Telephone or email versions of the score are appealing alternatives to administering it during the standard in-person patient visit; however, their validity and reliability remain unknown.

Questions/purposes

(1) Does the numerical scale (NS) version of the WOSI correlate with the original WOSI and Quick-DASH? (2) Do telephone and email administration of the NS-WOSI have good reliability and consistency? (3) Compared with the original WOSI form, does the NS form lead to faster completion for patients and quicker data extraction for researchers?

Methods

Between 2014 and 2019, 50 patients with a documented history of shoulder dislocation with persistent symptomatic shoulder instability, whether anterior, posterior, or multidirectional; patients scheduled for surgery; and patients with traumatic or nontraumatic injuries were prospectively recruited from the outpatient clinic of two university hospitals acting as Level 1 trauma centers and sports traumatology tertiary referral centers. The median (IQR) age was 28 years (24 to 36), and 80% (40 of 50) were men. Most (52% [26 of 50]) patients had two to five lifetime shoulder dislocations. Validity of the NS-WOSI was assessed using the Pearson correlation coefficient during an in-person visit; the original WOSI questionnaire (or its previously validated French-language version), NS-WOSI, and Quick-DASH questionnaires were administered in a random order. After a minimum 7-day interval, 78% (39 of 50) of patients completed the phone interview, and 74% (37 of 50) of patients completed the email version of the NS-WOSI score to evaluate NS-WOSI’s reliability using the intraclass correlation coefficient (ICC), which was interpreted as poor (< 0.5), moderate (0.50-0.75), strong (0.75-0.90), and very strong (> 0.90). The standard error of measurement (SEM) was used to evaluate variability around the true score, with a low value indicating a high reliability. The 95% minimal detectable change (MDC_95%) was calculated to evaluate the minimal change in score that was not related to measurement errors. Lastly, the Cronbach alpha was used to assess internal consistency (intercorrelation strength), where a value > 0.70 was considered good. The time needed for the patient to complete the various versions and for researchers to extract data was recorded.

Results

The NS-WOSI score was very strongly correlated with the original WOSI score (r = 0.96 [95% confidence interval (CI) 0.93 to 0.98]; p < 0.001). Although telephone-acquired and email-acquired data for the NS-WOSI questionnaires were correlated with the NS-WOSI (telephone r = 0.91 [95% CI 0.83 to 0.95]; p < 0.001; email r = 0.84 [95% CI 0.71 to 0.91]; p < 0.001), the ICC was higher for telephone interviews (0.92 [95% CI 0.86 to 0.96] versus email 0.80 [95% CI 0.64 to 0.89]), indicating that although both had good reliability, the phone interview was more suitable. The phone interview was also preferable to email regarding SEM (3% [52 of 2100 points] versus 6% [132 of 2100 points]) and the MDC_95% (7% [144 of 2100 points] versus 17% [366 of 2100 points]). The 95% CI of the MDC acquired by email was superior to the reported minimum clinically important difference for the original WOSI (7% [152 of 2100 points]), meaning that an error of measurement could wrongly be interpreted as a clinically significant change in score. Internal consistency was deemed good, with a Cronbach alpha of 0.96 (95% CI 0.92 to 98) and 0.89 (95% CI 0.79 to 0.94) for NS-WOSI telephone and email, respectively. The time to complete the NS-WOSI was reduced compared with the original WOSI (221 ± 153 seconds versus 266 ± 146 seconds, mean difference -45 seconds [95% CI -72 to -12]; p = 0.009). Lastly, data extraction was faster (62 ± 15 seconds versus 209 ± 52 seconds, mean difference -147 seconds [95% CI -164 to -130]; p < 0.001) with the NS-WOSI than with the original WOSI.

Conclusion

The NS-WOSI in person, by telephone, or by email is a valid, reliable, and timesaving alternative to the original WOSI questionnaire. However, the reliability of data acquisition by telephone interviews was superior to that of email.

Clinical Relevance

Given that there were no important differences in performance for the NS-WOSI, regardless of whether it was administered in person or by phone, we suggest that physicians use both interchangeably based on patient convenience. However, we do not recommend using the email version, especially for research purposes, since it was not as reliable when compared with in-person administration. The responsiveness of the modified NS-WOSI, as well as factors influencing response rates to phone interview, are questions that remain to be explored.

Introduction

Primary shoulder instability typically affects young male athletes, with a reported incidence of 23.1 per 100,000 person-years [15, 30]. Although assessing the proportion of recurrence is one way to evaluate outcomes [10], self-administered questionnaires provide a broader picture by evaluating pain, physical function, and health-related quality of life [5, 28]. These tools make it possible to keep track of the persistent apprehension, loss of confidence, and limitations in sports and day-to-day activities frequently reported by these patients [6,7]. Systematic reviews focusing on patient-reported outcome measurements in patients with shoulder instability [22, 28] identified the Western Ontario Shoulder Instability (WOSI) Index as the preferred outcome measurement tool. Furthermore, the WOSI score is commonly used in clinical research to monitor patient function and has been used in 24% of studies with patient-reported outcomes [28]. The WOSI questionnaire, which was first published in 1998, has been translated and cross-culturally validated in French [9, 14, 21]. The original WOSI questionnaire consists of 21 questions divided into four categories (physical symptoms; sports, recreation, or work; lifestyle; and emotions). The patient can complete it with few instructions, and it uses a VAS score ranging from 0 to 100 for each question, with a total score ranging between 0 and 2100 points. The minimum clinically important difference (MCID) for the WOSI score is 152 points [20], meaning that the WOSI score would need to improve by more than 7% (152 of 2100 points) for the patient to consider a change as being worthwhile.

The downside of the VAS in the original WOSI questionnaire is the time-consuming data collection and extraction process. Furthermore, as patients with shoulder instability tend to be young and active, it can be challenging to have them return to the office for follow-up appointments. A simpler, numeric scale (NS) version of the WOSI, ranging from 0 to 10 instead of the VAS, will not only be potentially less time consuming but will also be easily acquired by telephone interview. As highlighted by the current coronavirus-19 pandemic, electronic data acquisition in medical practice and research has become an appealing alternative to classic in-person data collection [16, 18, 24]. To the best of our knowledge, reliability and internal consistency of telephone or electronic (email) data acquisition have not been studied. Therefore, it seems crucial to evaluate the validity and reliability of email-acquired and telephone-acquired data for self-administered quality of life questionnaires.

Therefore, we asked: (1) Does the NS version of the WOSI correlate with the original WOSI and Quick-DASH? (2) Do telephone and email administration of the NS-WOSI have good reliability and consistency? (3) Compared with the original WOSI form, does the NS form lead to faster completion for patients and quicker data extraction for researchers?

Patients and Methods

Study Design and Setting

This prospective study was designed in accordance with the questionnaire validation guidelines of the American Academy of Orthopaedic Surgeons [11, 26] to answer the aforementioned questions. Data were collected between July 2014 and April 2019 at two university hospitals (Centre Intégré Universitaire de Santé et de Services Sociaux [CIUSSS] du Nord-de-l’île, in Montreal, Quebec, Canada and CIUSSS de-l’Estrie, Sherbrooke, Quebec, Canada), both acting as Level 1 trauma centers and sports traumatology tertiary referral centers. Patients were included by the senior orthopaedic surgeon (FB, DMR) who specialized in shoulder surgery at each center.

Participants

Inclusion criteria were a documented history of shoulder dislocation with persistent symptomatic shoulder instability, whether anterior, posterior, or multidirectional; patients scheduled for surgery; patients with traumatic or nontraumatic injuries; and native English or French speakers with the ability to read English or French. Exclusion criteria were a major psychiatric or psychologic disorder (including severe bipolar disorder, major depressive disorder, and schizophrenia) and inability or unwillingness to give informed consent for participation in the study.

Fifty patients met the eligibility criteria and were enrolled in our study, with 34% (17 of 50) of patients and 66% (33 of 50) of patients included by Sherbrooke and Montreal, respectively. The patients completed the original WOSI, NS-WOSI, and Quick-DASH questionnaires during an in-person clinic visit. Among these, 78% (39 of 50) completed the NS-WOSI by telephone interview and 74% (37 of 50) completed the NS-WOSI email questionnaire, at a mean interval of 12 ± 15 days and 9 ± 7 days, respectively. Thereafter, 22% (11 of 50) of patients were lost to follow-up for the telephone interview and 26% (13 of 50) for the email questionnaire. Baseline patient parameters including age, gender, number of dislocations, WOSI, NS-WOSI and Quick-DASH were comparable between patients who completed the study and those who did not respond to the phone interview or email questionnaire (Table 1).

Table 1.

Patient demographics and characteristics regarding responsiveness telephone interview and email questionnaire

Parameter	Response to phone interview		p value	Response to email questionnaire		p value
	Yes (n = 39)	No (n = 11)		Yes (n = 37)	No (n = 13)
Age in years (IQR)	29 (24‐40)	26 (24‐34)	0.65	31 (24‐39)	26 (23‐33)	0.30
Gender  Female	21 (8)	18 (2)	0.86	22 (8)	15 (2)	0.63
Occupation  Physical work  Sedentary work  Student  Retired  None	41 (16) 38 (15) 13 (5) 5 (2) 3 (1)	27 (3) 55 (6) 18 (2) 0 (0) 0 (0)	0.74	38 (14) 41 (15) 14 (5) 5 (2) 3 (1)	38 (5) 46 (6) 15 (2) 0 (0) 0 (0)	0.89
Number of lifetime dislocations  1  2-5  >5	5 (2) 59 (23) 36 (14)	9 (1) 27 (3) 64 (7)	0.18	3 (1) 57 (21) 41 (15)	15 (2) 38 (5) 46 (6)	0.20
Quick DASH	28 ± 22	36 ± 19	0.52	27 ± 21	36 ± 21	0.87
WOSI	1102 ± 466	1233 ± 453	0.43	1085 ± 454	1256 ± 479	0.69

Data presented as median (IQR), mean ± SD, or % (n). As a result of rounding, percentages may not always sum to 100%.

Participants’ Baseline Data

Patient demographic characteristics were recorded and classified according to the Frequency, Etiology, Direction, and Severity classification system for shoulder instability [17]. The median (IQR) age of patients in the study was 28 years (24 to 36); 80% (40 of 50) of patients were men (Table 2). Most (52% [26 of 50]) patients had occasional shoulder instability (two to five lifetime shoulder dislocation events), with 96% (48 of 50) presenting with traumatic unidirectional anterior instability. The dominant side was affected in 88% (44 of 50) of patients, and only 10% (5 of 50) of patients had a history of bilateral shoulder instability.

Table 2.

Patient demographics and characteristics (n = 50 patients)

Parameter	Value
Age in years	28 (24 to 36)
Gender  Women	20 (10)
Hand dominance  Right  Left  Ambidextrous	88 (44) 8 (4) 4 (2)
Occupation  Physical work  Sedentary work  Student  Retired  None	38 (19) 42 (21) 14 (7) 4 (2) 2 (1)
Affected shoulder  Right  Left  Bilateral	48 (24) 42 (21) 10 (5)
Dislocation type  Anterior  Posterior  Multidirectional	96 (48) 0 (0) 4 (2)
Number of lifetime dislocations   1   2-5   > 5	6 (3) 52 (26) 42 (21)

Data presented as median (IQR) or % (n).

Study Questionnaires Design

First, the original English-language [9] and a previously validated French-language version of the WOSI questionnaire [14] were converted into a novel NS paper version. Then, the metrological properties of the NS version were evaluated using classic in-person attendance, a telephone interview, and a questionnaire sent by email. A research assistant distributed the questionnaires and collected and extracted the data.

Transposition Process

The WOSI [14] is a disease-specific quality-of-life measurement tool consisting of 21 items, each rated on a VAS from 0 to 100. The questionnaire aims to explore how shoulder disability impacts quality of life. The overall score, between 0 and 2100, consists of four subscores, including physical symptoms (Section A, range 0 to 1000); sports, recreation, and work (Section B, 0 to 400); lifestyle (Section C, 0 to 400); and emotion (Section D, 0 to 300). The score can be read as a numeric value or expressed as a percentage of the maximum value, with a lower score indicating better shoulder function. The distribution-based MCID in the WOSI score has been reported to be 152 points [20], meaning that the WOSI score would need to improve by more than 7% (152 of 2100 points) for a change to be considered worthwhile by the patient.

Numeric scales ranging from 0 to 10 are frequently used to assess pain because they are quick and can also be completed in verbal or written form. In our numeric version of the WOSI questionnaire, “0” represented the absence of symptoms or disability and “10” represented extreme symptoms or disability. The total score and the score for each subcategory were multiplied by 10 to facilitate comparison with the original WOSI score. Therefore, the overall score, also ranging between 0 and 2100, consists of the same four subscores, including physical symptoms (Section A, range 0 to 1000); sports, recreation, and work (Section B, 0 to 400); lifestyle (Section C, 0 to 400); and emotion (Section D, 0 to 300). Score interpretation is equivalent to the original WOSI, with a lower score indicating better shoulder function.

Numeric WOSI Validation

Validity

During an in-person visit at the clinic, patients were asked to complete paper versions of the original WOSI, the NS-WOSI, and the Quick-DASH (including work and sports modules) questionnaires, which were administered in a random order. Validity was assessed through the correlation between the NS-WOSI, the original WOSI, and the Quick-DASH. The Pearson correlation coefficient (r) ranges from 1 (perfect positive linear correlation) to -1 (perfect negative linear correlation), with 0 indicating the absence of correlation. We interpreted r as weak (< 0.39), moderate (0.40-0.69), strong (0.70-0.89), or very strong (> 0.90) [23].

Reliability

After a minimum 7-day interval, patients were contacted by telephone and email to complete the NS-WOSI questionnaire so we could assess its reliability. Reproducibility was evaluated using intraclass correlation coefficient (ICC), where values range from 0 to 1, with 1 meaning a perfect reliability. The benefit of the ICC is to evaluate both correlation and agreement between measurements. ICC was interpreted as poor (< 0.5), moderate (0.50-0.75), strong (0.75-0.90), and very strong (> 0.90) [25]. The standard error of measurement (SEM = SD x √1-reliability) was used to evaluate variability around the “true” score, with a low value indicating a high reliability. The 95% confidence interval of the minimal detectable change (MDC_95%) relates the minimal change in score that is not likely to be related to measurement errors [26]. Lastly, the Cronbach alpha was used to assess internal consistency (intercorrelation strength) and the correlation between the multiple items of the questionnaire. A value > 0.70 was considered good [26].

Floor and Ceiling Effects

Floor and ceiling effects were measured for the different NS-WOSI questionnaires (NS-WOSI, NS-WOSI telephone, and NS-WOSI email) to verify that 15% or less of patients with shoulder instability reached the minimum or maximum score. This ensures the absence of clustering at the lowest and highest possible scores for the questionnaire, which would prevent tracking patient improvement or worsening during follow-up [19].

Time Efficiency

Time efficiency was evaluated by measuring the time patients needed to complete the different questionnaires (NS-WOSI in person, by telephone interview, and email questionnaire) as well as the time necessary for the research assistant to calculate the respective scores. We used the paired Student t-test to compare data with the original WOSI questionnaire.

Study Size

The sample size of 50 was based on the questionnaire validation guidelines of the American Academy of Orthopaedic Surgeons [11, 26]. The a priori power analysis estimated the sample size to be 15 participants when testing the null hypothesis regarding NS-WOSI validation based on an expected correlation coefficient of 0.67 [9] between NS-WOSI and Quick-DASH (α = 0.05, β = 0.2). The sample size increased to 30 patients with a null hypothesis using the NS-WOSI reliability based on an expected ICC of 0.84 [9] (minimal accepted reliability of 0.6, α = 0.05, β = 0.2). Both sample size calculations were below the final sample of 50 patients included to test questionnaire validity and the 39 and 37 patient samples (telephone interview and email questionnaire, respectively) to test NS-WOSI reliability.

Ethical Approval

The study was approved by the ethics board at each institution.

Statistical Analysis

All data analysis was performed using SPSS, version 26 (IBM Corp). Data was assessed for normality and parametric and nonparametric tests used as appropriate. Significance was set as a p value of < 0.05.

Results

Correlation Between the NS-WOSI and the Original WOSI and Quick-DASH

There was a very strong correlation between the NS-WOSI and the traditional WOSI (r = 0.96 [95% confidence interval (CI) 0.93 to 0.98]; p < 0.001) (Table 3), and this was also true for the four subscales (physical symptoms r = 0.94 [95% CI 0.90 to 0.97] sports, recreation, and work r = 0.92 [95% CI 0.86 to 0.95]; lifestyle r = 0.93 [95% CI 0.88 to 0.96]; emotion r = 0.94 [95% CI 0.90 to 0.97]; all p < 0.001) (Fig. 1). Additionally, the mean scores for the original WOSI and NS-WOSI were 1131 ± 462 points and 1147 ± 454 points, respectively (mean difference -17 [95% CI -55 to 22]; p = 0.39) (Table 4). The mean difference is therefore clearly inferior to the reported MCID of 152 points, confirming the validity of NS-WOSI. The correlation between the Quick-DASH and the original WOSI (r = 0.69 [95% CI 0.51 to 0.81]; p < 0.001) and NS-WOSI (r = 0.67 [95% CI 0.48 to 0.80]; p < 0.001) was moderate and comparable.

Table 3.

Pearson correlation coefficients and p values for the NS-WOSI, WOSI, and Quick-DASH

Measure 1	NS-WOSI scales	r (95% CI)	p value
WOSI total	NS-WOSI total	0.96 (0.93-0.98)	< 0.001
WOSI section A (physical symptoms)	NS-WOSI section A	0.94 (0.90-0.97)	< 0.001
WOSI section B (sports/recreation/work)	NS-WOSI section B	0.92 (0.86-0.95)	< 0.001
WOSI Section C (lifestyle)	NS-WOSI section C	0.93 (0.88-0.96)	< 0.001
WOSI Section D (emotion)	NS-WOSI section D	0.94 (0.90-0.97)	< 0.001
Quick-DASH	NS-WOSI	0.67 (0.48-0.80)	< 0.001
Work module	NS-WOSI section B	0.46 (0.21-0.65)	0.002
Sports module	NS-WOSI section B	0.77 (0.63-0.86)	< 0.001

Fig. 1.

A-C These graphs represent the correlation between the original WOSI score and (A) NS-WOSI, (B) NS-WOSI telephone, and (C) NS-WOSI email scores.

Table 4.

Original WOSI, NS-WOSI, and Quick-DASH scores (n = 50 patients)

Index	Scores
Original WOSI	1131 ± 462
Section A (physical symptoms)	460 ± 231
Section B (sports/recreation/work)	283 ± 101
Section C (lifestyle)	201 ± 103
Section D (emotion)	187 ± 87
NS-WOSI	1147 ± 454
Section A (physical symptoms)	481 ± 219
Section B (sports/recreation/work)	278 ± 103
Section C (lifestyle)	208 ± 98
Section D (emotion)	181 ± 92
Quick-DASH	30 ± 21
Work module	30 ± 39
Sports module	59 ± 35

Data presented as mean ± SD.

Reliability and Validity of Telephone and Email Administration of the NS-WOSI

Although telephone-acquired and email-acquired data for the NS-WOSI questionnaires were strongly correlated to the in-person NS-WOSI (telephone r = 0.91 [95% CI 0.83 to 0.95]; p < 0.001, email r = 0.84 [95% CI 0.71 to 0.91]; p < 0.001), ICCs for telephone interviews were superior to those for email: 0.92 (95% CI 0.86 to 0.96) versus 0.80 (95% CI 0.64 to 0.89) (Table 5). The mean scores for the NS-WOSI by telephone and email were 1092 ± 476 points and 1128 ± 488 points, respectively, and they were comparable with the total NS-WOSI score (mean difference for telephone -23 points [95% CI -82 to 37]; p = 0.44, and mean difference for email 20 points [95% CI -79 to 118]; p = 0.69). Variations around the true score as measured by the SEM were 3% (52 of 2100 points) and 6% (132 of 2100 points) for NS-WOSI telephone and NS-WOSI email, respectively, and both were inferior to the reported MCID for WOSI of 7% (152 of 2100 points). The 95% CI of minimal change in score that might be related to measurement errors (MDC_95%) for NS-WOSI telephone of 7% (144 of 2100 points) remained below the MCID, as opposed to the 17% (366 of 2100 points) for MDC_95% NS-WOSI email, which was clearly superior to the MCID (7% [152 of 2100 points]). Therefore, a clinically relevant change in score might not be differentiated from an error of measurement. Particularly Sections B (sports, recreation, or work) and D (emotion) with an MDC_95% of 32% (128 of 400 points) and 29% (88 of 300 points), respectively, bring into question the relevance of comparing an email-acquired score to the classic in-person–acquired score. Lastly, internal consistency was deemed good, with a Cronbach alpha of 0.96 (95% CI 0.92 to 0.98) and 0.89 (95% CI 0.79 to 0.94) for NS-WOSI telephone and email, respectively. No floor or ceiling effect was detected for the NS-WOSI, NS-WOSI telephone, and NS-WOSI email as no patient (0% [0 of 50 patients]) scored 0 or 2100 points at any given timepoint.

Table 5.

Reliability of the NS-WOSI and its sections^a

	Telephone (n = 39) ICC (95% CI)	Email (n = 37) ICC (95% CI)	p value
Total NS-WOSI	0.92 (0.86-0.96)	0.80 (0.64-0.89)	< 0.001
Section A (physical symptoms)	0.89 (0.80-0.94)	0.82 (0.69-0.90)	< 0.001
Section B (sports/recreation/work)	0.87 (0.77-0.93)	0.69 (0.48-0.83)	< 0.001
Section C (lifestyle)	0.94 (0.88-0.97)	0.82 (0.67-0.90)	< 0.001
Section D (emotion)	0.85 (0.73-0.92)	0.75 (0.56-0.86)	< 0.001

^aICC to evaluate both correlation and agreement between measurement. The ICC was interpreted as poor (< 0.5), moderate (0.50-0.75), strong (0.75-0.90), and very strong (> 0.90).

Time for Questionnaire Completion and Scoring

Time to complete the NS-WOSI was shorter compared with the original WOSI (221 ± 153 seconds versus 266 ± 146 seconds, mean difference -45 seconds [95% CI -72 to -12]; p = 0.009) when the data were acquired in person. Data extraction was faster with the NS-WOSI (62 ± 15 seconds versus 209 ± 52 seconds, mean difference -147 seconds [95% CI -164 to -130]; p < 0.001) than with the original WOSI, regardless of the data collection method used (Table 6).

Table 6.

Time to complete questionnaires and calculate scores compared with the original WOSI

Parameter	NS-WOSI (n = 50)	Mean difference (95% CI)	p value	NS-WOSI telephone (n = 39)	Mean difference (95% CI)	p value	NS-WOSI email (n = 37)	Mean difference (95% CI)	p value
Time to complete questionnaire	221 ± 153 seconds vs 266 ± 146 seconds	-45 seconds (-72 to -12)	p = 0.009	224 ± 90 seconds vs 259 ± 144 seconds	-34 seconds (-78 to 9)	p = 0.12	289 ± 184 seconds vs 255 ± 135 seconds	34 seconds (-43 to 110)	p = 0.38
Time to calculate score	62 ± 15 seconds vs 209 seconds ± 52 seconds	-147 seconds (-164 to -130)	p < 0.001	59 ± 9 seconds vs 210 ± 53 seconds	-151 seconds (-171 to -130)	p < 0.001	59 ± 12 seconds vs 214 ± 54 seconds	-155 seconds (-177 to -132)	p < 0.001

Data presented as mean ± SD.

Discussion

The WOSI questionnaire is the single, most complete, disease-specific score because it covers pain, physical function, and health-related quality of life, making it a complete psychometric test [28]. Given the current trend toward electronic data acquisition in medical practice and research, a simpler, NS version of the WOSI will not only be potentially less time consuming but could also easily be acquired by telephone interview [24]. Validity and reliability of telephone-acquired and email-acquired data for self-administered quality-of-life questionnaires cannot simply be assumed to be identical to their classic in-person counterparts. According to our findings, NS-WOSI is a valid and very strongly correlated alternative to the original WOSI questionnaire. The reliability of data acquisition by telephone interview is superior to the email version when compared with in-person data acquisition. Finally, data extraction was faster with NS-WOSI compared with the original WOSI. Based on our findings, we confirm the benefit of NS-WOSI as a valid, reliable, and timesaving alternative to the original WOSI. However, we suggest that telephone interview should be preferred to the email questionnaire to assess patients, especially for research purposes, given its greater reliability.

Limitations

This study has some limitations. First, we did not evaluate the responsiveness of the NS-WOSI because patients did not undergo any intervention during the study period. Despite the need for a formal analysis, we found a very strong correlation between NS-WOSI and WOSI, as well as a very strong reliability for NS-WOSI telephone with an SEM and MDC_95% well below the reported MCID for the original WOSI score. This suggests that a clinically relevant change in score should be detected with the NS-WOSI by telephone interview. Furthermore, as no question was modified and in the absence of a floor or ceiling effect (clustering of scores at minimum or maximal value preventing a change in score to be detected), it appears unlikely that responsiveness would differ from the original WOSI. Second, although validity was assessed in 50 patients, reliability was only assessed in 39 and 37 patients for the NS-WOSI telephone and email versions, respectively. These numbers remain above the threshold determined by our sample size calculation based on ICC. We were, however, surprised by the high (> 20%) and nonnegligible attrition rate. We found comparable baseline characteristics between the patients who completed the entire study and those who did not. Published data regarding mail, telephone interviews, and email health surveys (range 52% to 81%) [3, 4, 8, 11], as well as the 60% traditionally acceptable response rate for participating patients contacted by telephone interview and email, seem insufficient for high-quality research [12]. As our study was not designed to evaluate the response rate, future studies should focus on that specific and relevant topic to avoid creating a selection bias when conducting research studies. Although the sample size was adequate to answer our research questions, a relatively small sample size has some inherent limitations. Even though our study population reflects typical age and gender distribution (predominantly young, active, male patients) for patients with a documented history of shoulder dislocation, subgroups of older or female patients were too small to generalize our findings. Finally, for patient convenience and to maximize study participation, patients were not asked to return to the clinic after 7 days and complete the NS-WOSI in person again. Given the very strong ICC of the NS-WOSI telephone, it appears unlikely that in-person NS-WOSI score reliability would have been any different.

Correlation Between the NS-WOSI and the Original WOSI and Quick-DASH

The NS-WOSI questionnaire is a valid alternative to the original WOSI score. It appears that scores obtained with the NS-WOSI can be used for research purposes and compared with those obtained from the original WOSI questionnaire. The cross-validation between NS-WOSI and the Quick-DASH score revealed comparable correlation to previously published data between the WOSI and Quick-DASH [9, 13, 14]. The cross-validation highlights the utility of a disease-specific outcome measurement tool, as a more global questionnaire like the Quick-DASH tends to underestimate patient relevant and pathology-specific symptoms (such as persistent apprehension) [6].

Reliability and Validity of Telephone and Email Administration of the NS-WOSI

The telephone and email versions have good reliability, with telephone being better. Conducting prospective studies in a typical shoulder instability population (mainly consisting of young patients) is challenging, as their active lifestyle and busy work and personal lives typically interfere with a classic in-person clinical visit. Our findings support the telephone interview as a convenient and reliable alternative, whereas the email questionnaire should be used with caution given its lower reliability to the in-person–acquired score. With specific commercial software becoming available to help physicians with patient follow-up, our study highlights the necessity to properly assess the reliability of the tools used, as the different media used may impact scores (telephone and email in our case). The reproducibility of the NS-WOSI telephone was comparable with that of the original WOSI score and in line with previously reported ICCs for the original WOSI [9, 13, 14]. Although there are no cutoff values [1], SEM and MDC_95% for the NS-WOSI telephone were excellent and comparable with previously reported values in cross-cultural adaptations of the WOSI score (range 3.4% to 8.3% and 9.3% to 17.2% for SEM % and MDC_95% %, respectively) [2, 21, 27, 29]. Conversely, NS-WOSI email values for the MDC_95% showed poorer reproducibility, particularly for Sections B and D (the sports, recreation, or work and emotions sections), which questions the relevance of an email-acquired score compared with an in-person–acquired score because a change in score superior to the MCID could be solely dependent on a measurement error; this could therefore jeopardize the interpretation of results in clinical or research practice. We also want to stress that considering the lower reproducibility between NS-WOSI by email and in person compared with NS-WOSI by telephone and in person, the scores obtained by email should not necessarily be disregarded but should definitely be interpreted with caution.

Time for Questionnaire Completion and Scoring

The in-person NS-WOSI required less time to complete and score than the conventional version. This was not the case regarding time to complete the questionnaire for telephone interview and email. Although one could argue that an email questionnaire gives patients more time to reflect on each question and take more time to fill the questionnaire, our results did not support this assumption. In a busy research clinic, time is always a concern. As shown in our study, although the NS-WOSI questionnaire has only a limited impact on the data acquisition time, the final score was calculated more quickly. The time for score interpretation could be improved further by using commercial software that would also allow patients to complete the questionnaire on an electronic tablet in the waiting room or on their personal device at home. The software could also have an integrated VAS, making it possible to process the results immediately, without someone having to grade it. This would likely be the least time-consuming option.

Conclusion

Given that there were no important differences in performance of the NS-WOSI, regardless of whether it was administered in person or by phone, we suggest that physicians use both interchangeably, based on patient convenience. However, we do not recommend using the email version as an alternative to in-person WOSI, especially for research purposes, since it was not as reliable. The responsiveness of the modified NS-WOSI, as well as factors influencing response rates to phone interviews, are questions that remain to be explored.