The Nonoperative Instability Severity Index Score (NISIS): A Simple Tool to Guide Operative Versus Nonoperative Treatment of the Unstable Shoulder

Abstract

Background:

The management of the adolescent athlete after initial shoulder instability remains controversial.

Hypothesis:

Individual risk factors in athletes with shoulder instability who are managed nonoperatively can be integrated into a scoring system that can predict successful return to sport.

Study Design:

Retrospective cohort study.

Level of Evidence:

Level 4.

Methods:

A total of 57 scholastic athletes with primary anterior shoulder instability who were managed nonoperatively were reviewed. Success was defined as a return to index sport at the same level and playing at least 1 subsequent season without missed time as a result of the shoulder. Patient-specific risk factors were individually evaluated, and odds ratios were calculated. A 10-point Nonoperative Injury Severity Index Score (NISIS) incorporated the risk factors for failure. This score was then retrospectively applied with regression analysis and a chi-square analysis to determine the overall optimal score that predicted failure of nonoperative management.

Results:

In total, 6 risk factors for failure were included in the NISIS: age (>15 years), bone loss, type of instability, type of sport (contact vs noncontact), male sex, and arm dominance. Overall, 79% of patients treated nonoperatively were able to successfully return to sport. Nearly all (97%) low-risk patients (NISIS <7) successfully returned to sport, while only 59% of high-risk patients returned to sport, a relative risk of 12.2 (P = 0.001). High-risk patients with unipolar bone loss successfully returned (100%), but 67% of high-risk patients with bipolar bone loss failed.

Conclusion:

The NISIS is a simple and effective clinical tool to determine successful nonoperative management following anterior shoulder instability and may be helpful in guiding decision making when presented with the unstable shoulder in the scholastic athlete.

Treatment of the scholastic athlete who sustains an anterior shoulder instability episode remains controversial. Several studies have shown that athletes at both the high school and collegiate levels can successfully return to sport within the same season^3,4 but suffer a high rate of recurrence. Risk for recurrence is a multifactorial process, with age, bone loss, type of sport, and other factors contributing to higher risks of failure of conservative management.^1,2,7,12 To our knowledge, no study has evaluated the combined impact of these risk factors on the success of nonoperative management of the unstable shoulder. The purpose of this study was to evaluate a cohort of scholastic athletes who sustained an in-season instability episode and attempted nonoperative management—specifically, the likelihood these athletes could complete a subsequent season without time loss due to shoulder instability. Second, risk factors that were predictive of failure of conservative management were determined. Finally, these risk factors were combined into an overall predictive scale, the Nonoperative Instability Severity Index Score (NISIS). This score was evaluated for its ability to predict the success of nonoperative management in these athletes.

Methods

Patient Selection

After institutional review board approval, all patients who presented for first-time treatment for anterior shoulder instability to a single sports medicine practice were screened for inclusion into this study. Inclusion criteria required that patients were managed nonoperatively, that they were involved in high school sports with at least 1 season of eligibility remaining, and that complete information was available on their ultimate return to their previous sport. Patients were excluded if they had sustained their instability episode in their senior season (subsequent participation was not applicable) or if they sustained a season-ending injury that did not involve their index shoulder.

All patients were part of the Greenville county school system (Greenville, SC), which has a single network of employed athletic trainers. This network has a standardized injury surveillance system, which requires athletic trainers to document daily a list of athletes who are not fully participating in both practice and games and the reason for any missed participation. All data are contained within a single server, which was the source for return-to-play data and subsequent analysis. The senior author queried the database to determine the eligible cohort. Athletes were then classified as a “success” if they had sustained an anterior instability episode (as diagnosed by a sports medicine physician), were treated nonoperatively, and were able to return to their previous sport and complete an entire subsequent season in that same sport without a single documented time-loss event due to their shoulder. Athletes were defined as “failures” if they were treated nonoperatively and either did not return to their subsequent index sport due to their shoulder, returned to their previous sport but sustained an additional time-loss injury (as defined as missing even 1 practice or game due to their index injury) during the subsequent season, or underwent surgery to correct their instability. Ultimate success was then reported overall and stratified by several published risk factors.

Risk Factors

Several well-published risk factors were evaluated, which have been shown to result in suboptimal results in the setting of shoulder instability.^1,6,9,11 These included age, type of sport, sex, bone loss, arm dominance, and type of instability. These individual risk factors were evaluated to determine their individual recurrence risk for failure. Age was evaluated for each year between 13 and 18 years for each individual risk for failure, and the most significant was used as the cut line. Type of sport was assigned based on the sport in which the athlete sustained the index instability event and was divided into contact versus noncontact sport based on the classification by the American Academy of Pediatrics.¹⁰ Bone loss was classified as any glenoid or humeral bone loss that could be detected on standard radiographic evaluation, which included anterior/posterior view, Grashey view, supraspinatus outlet (scapular Y) view, and an axillary view. Advanced imaging was not employed, as most of the cohort did not receive this imaging. Two blinded orthopaedic surgeons independently evaluated each radiograph at 2 separate time points. The presence of a Hill-Sachs lesion, loss of contour of the anterior glenoid, or an avulsion fragment of the glenoid was considered positive for glenoid bone loss. Intra- and interobserver reliabilities were kappa = 0.92 and kappa = 0.95, respectively, and there were no cases of disagreement. Sex was evaluated by comparing the rates of failure between male and female athletes. Arm dominance was determined by patient query. Relative risk for failure was determined for all individual factors.

Statistical Analysis and Creation of the NISIS

All statistical analyses were performed by a doctoral-level research scientist with advanced training in biostatistics using SPSS Version 25 (IBM Corp). Odds ratios and relative risks for individual factors were calculated using chi-square analysis with statistical significance set at P < 0.05. Risk factors were then assigned a point total based on their individual impact on failure rate and were combined into the 10-point NISIS. A receiver operating characteristic (ROC) curve was constructed to determine the cutoff score with the highest accuracy of predicting failure. This score was then retrospectively applied to the cohort with regression and chi-square analyses to determine the overall risk for failure based on the NISIS.

Results

A total of 57 patients sustained an anterior instability episode, were treated nonoperatively, had minimum eligibility to complete a subsequent season, and were included in this study. The mean age was 15.8 ± 1.5 years, and 82% were male. Overall, 79% of interscholastic athletes with a first-time anterior glenohumeral instability event managed nonoperatively returned to and completed play in the subsequent season without any missed time due to a shoulder injury. Six risk factors significantly affected failure within the NISIS (Figure 1). There were 21 patients aged 15 years or less, and 36 patients aged >15 years. Patients aged >15 years were significantly more likely to fail nonoperative treatment (odds ratio [OR], 6.0; P = 0.02). There were 36 patients without detectable bone loss on radiographic evaluation, and 21 patients with detectable bone loss. The presence of bone loss resulted in a significantly higher risk of failure (OR, 4.5; P = 0.01). A total of 22 patients were noncollision/contact sport athletes, and 35 were collision/contact athletes. These collision/contact sport athletes were at significantly greater risk for failure (OR, 23.4; P = 0.001). The dominant arm was involved in 28 patients compared with the nondominant arm in 29 patients. Patients with dominant arm instability had a higher risk of failure (OR, 2.8; P = 0.09). Type of instability included 26 subluxors and 31 dislocators. Those patients with dislocation had a greater risk for failure compared with those with subluxation (OR, 3.6; P = 0.02). There were 47 males and 10 females in the cohort. Males were at increased risk of failure (OR, 1.5; P = 0.03). Based on these data, we combined these factors into an individual NISIS that was set to equal a score of 10. One factor, type of sport, had a dominant odds ratio and was thus assigned 3 points. The next 2 highest risk factors (bone loss and age >15 years) were assigned 2 points, and the lowest 3 risk factors (male sex, arm dominance, and type of instability) were assigned 1 point each.

Figure 1.

Nonoperative Instability Severity Index Score (NISIS). XR, x-ray.

An ROC curve determined that a cutoff score of 7 (area under the curve = 0.81; P = 0.001) gave the highest accuracy of predicting which athletes would be successful with nonoperative management (Figure 2). Athletes who scored <7 on the NISIS were deemed low risk. These athletes had a successful result 97% of the time. Those athletes who scored greater than or equal to 7 were deemed high risk and had a successful result only 59% of the time. High-risk athletes had 12.2 times relative risk of failure compared with low-risk athletes (P = 0.001). A deeper analysis of the high-risk patients revealed that those with bipolar bone loss were at particular risk for failure. In fact, all high-risk patients with unipolar bone loss were successfully treated with nonoperative management, while 67% of high-risk patients with bipolar bone loss failed nonoperative management (P = 0.03).

Figure 2.

Receiver operating characteristic (ROC) curve for Nonoperative Instability Severity Index Score cutoff.

Discussion

Decision making in scholastic athletes with anterior shoulder instability remains controversial. Strongly held opinions are held between a “fix them all” and a “wait and see” approach. Most studies that have evaluated this question have defined “failure” as recurrence of instability. This may be a misleading outcomes measure. Hovelius et al⁸ reported in their landmark article that half of their young patients who sustained an anterior instability event either did not experience a recurrence or sustained 1 or 2 recurrences and then became stable over time. Furthermore, studies^3,4 have shown that many athletes can return to sport, at least in the index season, in spite of sustaining recurrent instability episodes. Only 1 previous study has evaluated whether nonoperatively treated athletes with anterior shoulder instability could successfully return to play the subsequent season. Dickens et al⁵ followed 10 patients who chose nonoperative management after completing a season in which they sustained an anterior instability episode and determined whether they were able to return the subsequent season. In that study, only 4 of 10 (40%) were successful, which was much lower than the 79% success rate reported in the current study. Dickens et al⁵ studied intercollegiate contact athletes. Our study showed that older age and contact sport participation were significant risk factors for failure and may partially explain the differences in results.

Perhaps the most important finding of this study is that while each risk factor demonstrated an increased risk of failure, the combination of factors into the NISIS further improved the ability to predict success with nonoperative management. Low-risk patients had a 97% success rate while high-risk patients had only a 59% success rate with nonoperative management. Balg and Boileau¹ described the Instability Severity Index Score, where they applied a similar set of criteria to determine who was at risk for failure after arthroscopic management of shoulder instability. This study was similar in that age, contact sports, and bone loss were risk factors for failure. This study is different, however, in that we examined nonoperative management as opposed to surgical management and defined failure differently. The success rate was defined not by recurrence but by an athlete’s ability to return to his or her index sport and complete at least 1 subsequent season with no time loss due to disability of their index shoulder. This definition of success is different from most previous studies that use recurrence as a litmus test for failure. Successful return to play can occur in the setting of recurrence. Buss et al³ reported on 30 mostly high school athletes who were managed for in-season anterior shoulder instability; 26 of 30 (87%) athletes were able to return during the same season in spite of the fact that 37% sustained sport-related recurrences. Similarly, Dickens et al⁴ reported a similar rate of 73% of intercollegiate contact athletes returning to sport in the same season, with 64% sustaining in-season recurrences. Neither of these studies followed patients who continued nonoperative management into the following season. It is possible that the current cohort contained patients who “coped” with recurrent instability events but were not significantly limited to the point of missing practices or games.

This study has several limitations. First, the study is at risk for selection bias in that this cohort excluded those who underwent early operative intervention and may have excluded the highest risk patients from this cohort. Next, recurrence is not reported as a measure of success, and given that recurrence is the traditional measure of outcomes in most studies, the ability to compare the results to other studies is limited in this regard. It is possible that successful patients in this cohort sustained recurrences but coped with these and continued playing. These copers may risk joint damage and future complications by playing with recurrent instability. The definition of success may be more meaningful to the athlete who wishes to return to play and seeks counseling on whether nonoperative management will achieve this goal. Another limitation was that our measure of bone loss was limited to radiographs and we did not include advanced imaging. Generally, patients only obtained magnetic resonance imaging or computed tomography if they underwent surgical treatment. We therefore classified bone loss as significant if it was present either as a Hill-Sachs lesion, anterior glenoid erosion, or a bony Bankart lesion. It is possible that some of the bone loss was missed that may have been detected with specialized views or advanced imaging. Third, we only followed patients for 1 subsequent competitive season. It is possible that longer follow-up would result in higher failure rates. Finally, this is a retrospective study and is subject to the limitations of this study design. Further prospective study would be necessary to validate it as a predictive tool in the clinical setting.

Conclusion

The results of this study suggest that the nonoperative management of high school–aged athletes with shoulder instability is surprisingly high, with 79% of patients able to return to sport and complete a subsequent season without time loss due to their shoulder. The NISIS differentiated between successful return and failure, with a score <7 resulting in successful return in 97% of patients. Thus, the NISIS may be a clinically useful tool in decision making regarding which scholastic athletes may benefit from nonoperative treatment after anterior shoulder instability and which patients should be classified as inappropriately high risk.