Abstract
Aim
To determine the threshold score for Oxford Shoulder Instability Score (OSIS), Constant-Murley score (CMS), and University of California Los Angeles (UCLA) shoulder score that predict treatment success after arthroscopic bankart repairs.
Methods
77 patients were followed up for 12-mmonth. Threshold scores were determined by Receiver Operating Characteristic (ROC) analyses with treatment success defined as improvement in pain, no re-dislocation, expectation and satisfaction met.
Results
The follow threshold scores were identified:6-month: OSIS 35, CMS 68,UCLA 2612-month: OSIS 24, CMS 84, UCLA 33.
Conclusion
OSIS, CMS, and UCLA are good predictors of treatment success after arthroscopic bankart repair.
Level of EvidenceIII.
Keywords: Threshold score, Oxford shoulder instability score, Constant-murley score, University of California los angeles shoulder score, Arthroscopy, bankart
1. Introduction
Bankart lesion is a common injury associated with anterior shoulder dislocation that can result in persistent pain and limitation in function. Bankart repair is an effective solution with high patient reported satisfaction rate and return to previous level of function.1, 2, 3 Compared to open bankart repairs, arthroscopic repair has a shorter operating time and is more cost-effective,4,5 with comparable long-term results and recurrence rates.6,7
In today's world of modern medical advancements, where new treatment modalities and surgical techniques are emerging at an unprecedented rate and the educated patients are taking a more pro-active role in their treatment selection, increasing emphasis is placed on evidence-based medicine and outcome studies. Patient-reported outcome measures (PROMs) are an integral part in any outcome studies that measure the efficacy of a treatment based on patient's perceived outcomes. While various PROMs have been developed to quantify and standardise treatment outcome measurements after bankart repair, there is no cut-off value to determine if treatment has been successful from the patients' perspective.
The concept of threshold scores is an absolute cut-off value in a specific outcome measure that has both good sensitivity and specificity to determine if patients perceived that they had received successful treatment. A score equal to or better than the threshold score equates to treatment success whereas a score poorer than threshold score suggest unsuccessful treatment in the patients’ perceptive.
The Oxford Shoulder Instability Score (OSIS), Constant-Murley score (CMS), and University of California Los Angeles (UCLA) shoulder score are common PROMs used to assess patient outcome after bankart repair. However, there is currently a lack of literature on their valid threshold score to predict treatment success. Therefore, the purpose of this study was to determine the threshold score for OSIS, CMS, and UCLA shoulder score that predicts treatment success after arthroscopic bankart repairs. The authors hypothesize that OSIS, CMS, and UCLA are valid PROMs with good predictive values to determine treatment success after arthroscopic bankart repairs.
2. Methods
This is a retrospective study with prospectively collected data. Between 2010 and 2015, the records of all patients with Magnetic Resonance Imaging (MRI) proven unilateral bankart lesion due to anterior shoulder dislocation who underwent unilateral arthroscopic bankart repair in the authors’ institution were extracted from the institutional joint registry. Patients with bilateral bankart repairs, foreign patients who returned to their home country for further follow up, and those who refused additional follow up assessment in the joint registry were excluded.
The patients were prospectively followed up for 12-months. They were assessed by an independent physiotherapist working in the authors’ institutional joint registry pre-operatively, at 6-months, and at 12-months post-operatively for pain, patient satisfaction, and expectation fulfilment. At each follow up visit, patients were also asked to complete the various PROMs questionnaires unaided and their response recorded.
Patient's perceived pain was measured with the 11-point Visual Analogue Scale (VAS), with 0 being ‘No pain’, and 10 being ‘Worst pain imaginable’. Satisfaction was rated on a 6-point Likert scale with 1 being “Extremely satisfied” to 6 being “Extremely dissatisfied” while expectation was rated on a 7-point Likert scale with 1 being “Greatly exceeding expectation” and 7 being “Much less than expected”. As there is currently no general consensus on the definition of treatment success after arthroscopic bankart repair, the authors defined treatment success as simultaneous fulfilment of 4 criteria: improvement in pain (any decrease in VAS), no re-dislocation, expectation of surgery met, and patient satisfied with surgery.
Functional outcome was measured using OSIS, CMS, and UCLA shoulder score. All three scores have proven to provide reliable, valid, and responsive information in assessing patients’ response in a variety of shoulder pathologies and perception of shoulder instability and were the outcome measures used routinely in the authors institution for monitoring treatment response8, 9, 10, 11, 12, 13, 14, 15, 16, 17
2.1. Statistical analysis
Statistical analyses were carried out in consultation with a statistician, using SPSS 23.0 software (IBM, Armonk, USA) and R 3.4.2 (R Core Team (2017) and R package (R Foundation for Statistical Computing, Vienna, Austria.). Threshold scores were determined by Receiver Operating Characteristic (ROC) analyses, using the PROMs as predictors and the defined treatment success as criterion in the logistic regression. In a ROC curve, sensitivity is plotted against specificity for different cut-off points of a diagnostic test and it summarize the trade-off between true positive rate and false positive rate for a predictive model using different threshold scores.
The Area Under the ROC Curve (AUC) represents the degree or measure of separability (the ability to separate treatment success and treatment failure) and is an effective representation of the inherent validity of a diagnostic test.18 AUC of 0.70–0.80 signifies good validity of a test to discriminate treatment success from failure and an AUC of more than 0.80 signifies excellent discriminative ability. In this study, the AUC with 95% confidence interval was calculated for OSIS, CMS, and UCLA at 6 and 12-month. The optimal cut-off point (c*) for threshold scores were determined by the Youden's Index (J) which is a point on the ROC curve that gives equal weightage to both sensitivity and specificity of a test and is deemed as the point with the highest discriminative value.
3. Results
A total of 113 patients with MRI proven bankart lesion due to anterior shoulder dislocation underwent unilateral arthroscopic bankart repair in the authors’ institution from 2010 to 2015. 36 patients with missing values at any of the follow-up time points were excluded, leaving a total of 77 patients included in the final study. The patient demographics (age, gender, and side of surgery) and pre-operative PROMs of those excluded were comparable to those included in the study. Only patients with complete data at all follow up time points were included in the analysis to ensure that the change in scores were reflective of the change in outcome in the same group of patients. Table 1 shows the demographics and PROMs pre-operatively and at the 2 follow up time points. Overall, vast majority of patients experienced improvement in pain post-operatively (≥82% at 6-month and 12-month follow up). Patient satisfaction with surgery and expectation fulfilment were also high with 92% of patients satisfied with their outcome and 88% of patients reporting expectation fulfilment at 12-month post-operatively. 2 patients sustained re-dislocations, 1 at 8 month and another at 11 month post-operatively.
Table 1.
Clinical outcome scores.
| Age, Year |
30.4 ± 13.4 |
||
|---|---|---|---|
| Gender, Male:Female (%) | 65:12 | ||
| Side, Left:Right (%) | 27:50 | ||
| Preoperative | 6-month | 12-month | |
| OSIS | 35 ± 10 | 26 ± 9 | 21 ± 8 |
| CMS | 65 ± 21 | 75 ± 17 | 82 ± 15 |
| UCLA score | 21 ± 6 | 28 ± 5 | 30 ± 5 |
OSIS, Oxford Shoulder Instability Score; CMS, Constant-Murley Score; UCLA, University of California, Los Angeles.
Overall, with the definition of having to fulfill all 4 criteria (decrease in pain, no re-dislocation, patient satisfied with surgery, and expectation met), 75% of patients had successful arthroscopic bankart repair at 6-month and 79% at 12-month post-operatively (Table 2).
Table 2.
Percentage fulfillment of treatment success criteria.
| 6-Month | 12-Month | |
|---|---|---|
| Improvement in pain (%) | 82 | 88 |
| No re-dislocation (%) | 100 | 97 |
| Expectations Fulfilled (%) | 84 | 88 |
| Satisfaction met (%) | 94 | 92 |
| Fulfillment of all criteria (%) | 75 | 79 |
ROC analyses revealed good prediction value for treatment success using OSIS at all 3 follow-up time points (AUC >0.70) with excellent prediction using UCLA at 6 month (AUC 0.834, CI 0.714–0.954) and OSIS at 12 month (AUC 0.858, CI 0.763–0.953) (Table 3). The ROC curve with the various PROMs at different follow-up time points are displayed in Fig. 1 and Fig. 2.
Table 3.
AUC and 95% CI for receiver operating characteristic analysis.
| AUC (95% CI) | ||
|---|---|---|
| 6-Month | 12-Month | |
| OSIS (95% CI) | 0.757 (0.626–0.889) | 0.858 (0.763–0.953) |
| CMS (95% CI) | 0.783 (0.654–0.912) | 0.741 (0.591–0.890) |
| UCLA score (95% CI) | 0.834 (0.714–0.954) | 0.749 (0.628–0.870) |
AUC, area under receiver operating characteristic curve; CI, confidence interval; CMS, Constant-Murley score; OSIS, Oxford Shoulder Instability Score; UCLA, University of California, Los Angeles.
Fig. 1.
Receiver Operating Characteristic (ROC) Curve using Oxford Shoulder Instability Score (OSIS), Constant-Murley score (CMS), and University of California Los Angeles (UCLA) shoulder score and their changes as predictors and treatment success as the criterion at 6-month. The red dots denoted the optimum cut-off value (c*). (AUC, area under receiver. operating characteristic curve; CI, confidence interval.).
Fig. 2.
Receiver Operating Characteristic (ROC) Curve using Oxford Shoulder Instability Score (OSIS), Constant-Murley score (CMS), and University of California Los Angeles (UCLA) shoulder score and their changes as predictors and treatment success as the criterion at 12-month. The red dots denoted the optimum cut-off value (c*). (AUC, area under receiver.
The various threshold score for treatment success as determined by the Youden's Index are shown in Table 4. A UCLA of 26 or higher and an OSIS of 24 points or lower at 12-month had excellent predictive value that a patient had successful treatment after arthroscopic bankart repair.
Table 4.
Clinical outcome scores and threshold scores.
| Threshold Score (Sensitivity/Specificity) | ||
|---|---|---|
| 6-Month | 12-Month | |
| OSIS | 35 (0.91/0.44) | 24 (0.79/0.87) |
| CMS | 68 (0.86/0.67) | 84 (0.67/0.79) |
| UCLA | 26 (0.86/0.78) | 33 (0.51/1.00) |
OSIS, Oxford Shoulder Instability Score; CMS, Constant-Murley Score; UCLA, University of California, Los Angeles.
4. Discussion
The most significant findings of this study are that OSIS, CMS, and UCLA are good predictors of treatment success at 6 months and 12 months post arthroscopic bankart repair. This is useful in interpretation of the various PROMs at the different follow-up time points. Having a definitive value to quantify treatment success can allow clinicians a better gauge of their patients' outcome during follow up to facilitate patient counselling on the need for further treatment or rehabilitation in those who failed to achieve a satisfactory score. A specific cut off value for threshold score can also facilitate future comparative studies with emphasis on patient's perceived outcome.
There have been increasing interest in meaningful interpretation of PROMs. While a numerical improvement in score can signify some form of change in outcome after treatment, it is equally important to understand if this difference in score is actually clinically significant. Two general concepts have been introduced in the literature: Minimal Clinically Important Difference (MCID) and threshold score. MCID is the smallest change in the outcome of a medical intervention that a patient would identify as important.19,20 However, MCID is limited by the influence of patient's expectation and their baseline score, which may prevent meaningful interpretation of results in patients with high expectation or higher baseline score. This is addressed by the concept of threshold scores which take into consideration patient's expectation as well as provide an absolute cut-off value irrespective of baseline score. Therefore, both change in scores and absolute scores play a crucial role in monitoring patient's progress after surgery.
There is currently no consensus on what constitute post-operative treatment success. The majority of literature focus on pain reduction, functional improvement (either through increase in strength and range of motion or PROMs), and patients' satisfaction rate and expectation fulfilment.21,22 Peterson et al.23 and Giesinger et al.24 calculated threshold score of Oxford Knee Score (OKS) and Knee Society Score (KSS) respectively after Total Knee Arthroplasty (TKA) based on patient satisfaction, pain relief, and functional improvement after surgery. Xu et al.25 defined treatment success as improvement in pain, expectation of surgery met, and patient satisfied with surgery for calculation of threshold score of CMS, UCLA shoulder score, and Oxford Shoulder Score (OSS) after arthroscopic rotator cuff repair. The authors used a similar criterion of simultaneous fulfilment of decrease in pain, patient satisfied with surgery, and expectation met as these give a general indication of patient's perceived treatment outcome. The authors added a fourth criterion of no dislocation as that is often one of the primary end points of bankart repairs.
Another important finding of this study is that useful threshold scores can be calculated from OSIS, CMS, and UCLA. All three scores are valid and reliable tools for assessing response after treatment for shoulder instability.8, 9, 10, 11, 12, 13, 14, 15, 16, 17 The ability to derive statistically significant threshold scores from these outcome measures further validates their ability as a responsive tool in assisting clinicians in monitoring patients’ outcome after arthroscopic bankart repairs.
4.1. Limitations
All patients were recruited from a single institution and different patient demographics might give rise to different threshold scores. The follow-up time is limited to 12 months as majority of patients were discharged from follow-up after their 12-month review.
5. Conclusion
OSIS, CMS, and UCLA are good predictors of treatment success after arthroscopic bankart repair and meaningful threshold scores can be derived. The cut-off scores are time-dependent with a higher score required to constitute treatment success at longer follow-up.
Funding
Nil.
Declaration of competing interestCOI
The authors of this manuscript have nothing to disclose that would bias our work.
Acknowledgments
The conduct of this study was funded by Singhealth DukeNUS Academic Medicine Research Grant, Grant number: AM/TP005/2018 (SRDUKAMR18T5).
References
- 1.Saper M.G., Milchteim C., Zondervan R.L., Andrews J.R., Ostran der R.V., III Outcomes after arthroscopic Bankart repair in adolescent athletes participating in collision and contact sports. Orthop J Sports Med. 2017 Mar 28;5(3) doi: 10.1177/2325967117697950. 2325967117697950. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Sugaya H., Moriishi J., Kanisawa I., Tsuchiya A. Arthroscopic bony Bankart repair for chronic recurrent traumatic anterior glenohumeral instability (SS-30) Arthrosc J Arthrosc Relat Surg. 2004;(20):e13–e14. [Google Scholar]
- 3.Saier T., Plath J.E., Waibel S. How satisfied are patients with arthroscopic Bankart repair? A 2-year follow-up on quality-of-life outcome. Arthrosc J Arthrosc Relat Surg. 2017 Oct 1;33(10):1777–1785. doi: 10.1016/j.arthro.2017.04.017. [DOI] [PubMed] [Google Scholar]
- 4.Wang C., Ghalambor N., Zarins B., Warner J.J. Arthroscopic versus open Bankart repair: analysis of patient subjective outcome and cost. Arthrosc J Arthrosc Relat Surg. 2005 Oct 1;21(10):1219–1222. doi: 10.1016/j.arthro.2005.07.004. [DOI] [PubMed] [Google Scholar]
- 5.Barber F.A., Click S.D., Weideman C.A. Arthroscopic or open Bankart procedures: what are the costs? Arthrosc J Arthrosc Relat Surg. 1998 Oct 1;14(7):671–674. doi: 10.1016/s0749-8063(98)70092-1. [DOI] [PubMed] [Google Scholar]
- 6.Weber S. Arthroscopic Bankart repair: minimum 10-year follow-up with emphasis on survivorship. Arthroscopy. 2014 Jun 1;30(6):e4. [Google Scholar]
- 7.Delgrande D., Werthel J.D., Lonjon G., Ciais G., Klouche S., Hardy P.P. Arthroscopic bankart repair in patients over 30 Years of age. Clinical and radiologic results at more than 10 Years follow-up. Arthroscopy. 2017 Oct 1;33(10):e162. [Google Scholar]
- 8.Dawson J., Fitzpatrick R., Carr A. The assessment of shoulder instability: the development and validation of a questionnaire. The Journal of bone and joint surgery. Br vol. 1999 May;81(3):420–426. doi: 10.1302/0301-620x.81b3.9044. [DOI] [PubMed] [Google Scholar]
- 9.Skare Ø., Liavaag S., Reikerås O., Mowinckel P., Brox J.I. Evaluation of Oxford instability shoulder score, Western Ontario shoulder instability index and Euroqol in patients with SLAP (superior labral anterior posterior) lesions or recurrent anterior dislocations of the shoulder. BMC Res Notes. 2013 Dec;6(1):273. doi: 10.1186/1756-0500-6-273. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.van der Linde J.A., van Kampen D.A., van Beers L.W., van Deurzen D.F., Terwee C.B., Willems W.J. The Oxford Shoulder Instability Score; validation in Dutch and first-time assessment of its smallest detectable change. J Orthop Surg Res. 2015 Dec;10(1):146. doi: 10.1186/s13018-015-0286-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Constant C., Murley A. A clinical method of functional assessment of the shoulder. Clin Orthop Relat Res. 1985;214:160–164. [PubMed] [Google Scholar]
- 12.Kemp K.A.R., Sheps D.M. An evaluation of the responsiveness and discriminant validity of shoulder questionnaires among patients receiving surgical correction of shoulder instability. Sci World J. 2012 doi: 10.1100/2012/410125. Article ID 410125. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Stiller J., Timothy L. Outcomes measurement of upper extremity function. Hum Kinet ATT. 2005;10(3):24–25. 2005. [Google Scholar]
- 14.Hirschmann M.T., Wind B., Amsler F., Gross T. Reliability of shoulder abduction strength measure for the constant-murley score. Clin Orthop Relat Res. 2010;468:1565–1571. doi: 10.1007/s11999-009-1007-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Amstutz H.C., Sew Hoy A.L., Clarke I.C. UCLA anatomic total shoulder arthroplasty. Clin Orthop. 1981;155:7–20. [PubMed] [Google Scholar]
- 16.Romeo A.A., Bach B.R., Jr., O'Halloran K.L. Scoring systems for shoulder conditions. Am J Sports Med. 1996;24:472–476. doi: 10.1177/036354659602400411. [DOI] [PubMed] [Google Scholar]
- 17.Placzek J.D., Lukens S.C., Badalanmenti S. Shoulder outcome measures: a comparison of 6 functional tests. Am J Sports Med. 2004;32:1270–1277. doi: 10.1177/0363546503262193. [DOI] [PubMed] [Google Scholar]
- 18.Kumar Rajeev, Indrayan Abhaya. Receiver operating characteristic (ROC) curve for medical researchers. Indian Pediatr. 2011;48:277–287. doi: 10.1007/s13312-011-0055-4. [DOI] [PubMed] [Google Scholar]
- 19.Kukkonen J., Kauko T., Vahlberg T., Joukainen A., Aärimaa V. Investigating minimal clinically important difference for Constant score in patients undergoing rotator cuff surgery. J Shoulder Elbow Surg. 2013;22:1650–1655. doi: 10.1016/j.jse.2013.05.002. [DOI] [PubMed] [Google Scholar]
- 20.van Kampen DA, Willems WJ, van Beers LW, Castelein RM, Scholtes VA, Terwee CB. Determination and comparison of the smallest detectable change (SDC) and the minimal important change (MIC) of four-shoulder patient-reported outcome measures (PROMs) J Orthop Surg Res. 2013;8(1):40. doi: 10.1186/1749-799X-8-40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Castricini R., Panfoli N., Nittoli R., Spurio S., Pirani O. Transtendon arthroscopic repair of partial-thickness, articular surface tears of the supraspinatus: results at 2 years. Chir Organi Mov. 2009;93(Suppl 1):S49–S54. doi: 10.1007/s12306-009-0002-x. [DOI] [PubMed] [Google Scholar]
- 22.Ide J., Maeda S., Takagi K. Arthroscopic transtendon repair of partial-thickness articular-side tears of the rotator cuff: anatomical and clinical study. Am J Sports Med. 2005;33:1672–1679. doi: 10.1177/0363546505277141. [DOI] [PubMed] [Google Scholar]
- 23.Petersen CL, Kjærsgaard JB, Kjærsgaard N, Jensen MU, Laursen MB. Thresholds for Oxford Knee Score after total knee replacement surgery: a novel approach to post-operative evaluation. J Orthop Surg Res. 2017;12:89. doi: 10.1186/s13018-017-0592-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Giesinger K, Giesinger JM, Hamilton DF, Jost B, Behrend H. WOMAC, EQ-5D and knee society score thresholds for treatment success after total knee arthroplasty. J Arthroplasty. 2015;30(12):2154–2158. doi: 10.1016/j.arth.2015.06.012. [DOI] [PubMed] [Google Scholar]
- 25.Xu S., Chen J.Y., Lie H.M., Hao Y., Lie D.T. Determination of threshold scores for treatment success after arthroscopic rotator cuff repair using Oxford, constant, and university of California, Los Angeles shoulder scores. Arthrosc J Arthrosc Relat Surg. 2019 Feb 1;35(2):304–311. doi: 10.1016/j.arthro.2018.07.047. [DOI] [PubMed] [Google Scholar]