Abstract
Background
Successful utilization of the International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI) requires a comprehensive understanding of its rules, terminology, and several complex concepts. There have been no studies investigating classification accuracy since the newest ISNCSCI revision (2019).
Objectives
To evaluate classification accuracy of SCI professionals using the 2019 ISNCSCI edition, identify common mistakes and areas of confusion, and assess associations between experience in ISNCSCI classification and performance.
Methods
Members of the International Spinal Cord Society (ISCoS) and attendees of the ISCoS Annual Scientific Meeting 2021 were invited to complete an online survey that included six ISNCSCI cases to classify.
Results
A total of 107 persons completed the survey, with overall classification accuracy of 74.6%. Accuracy was highest for injury completeness (95.3%) and sensory level (91.1%) and lowest for motor zone of partial preservation (ZPP; 54.7%) and ASIA Impairment Scale (AIS) grade (57.3%). Newer concepts, including the appropriate documentation of non-SCI conditions and classification of ZPP in incomplete injuries, contributed to several common errors. There was a significant association between overall classification accuracy and self-rated experience in the ISNCSCI classification (p = .017). Experience with the ISNCSCI examination, experience in SCI medicine, and occupation were not found to be significantly associated with overall classification accuracy.
Conclusion
Classification accuracy of an international cohort of SCI professionals was modest but greater than previous reports. Knowledge deficits about the 2019 ISNCSCI updates are prevalent and contribute to common classification errors. Further training in the utilization of the ISNCSCI is needed.
Keywords: International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI), classification accuracy, spinal cord injury
Introduction
First published by the American Spinal Injury Association (ASIA) in 1982,1 the International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI) is the internationally adopted standard for the characterization of sensorimotor impairments after spinal cord injury (SCI). The ISNCSCI has important clinical and research applications. It imparts a medical diagnosis, allows for prognostication, and can be used to evaluate neurological changes over time. Results of the ISNCSCI may influence management decisions, functional goal setting, and the development of individualized rehabilitation programs. Additionally, results are often used to determine clinical trial eligibility and the efficacy of treatment interventions. For these reasons, achieving accuracy in both the ISNCSCI examination and classification is essential, and the latter is the focus of this study.
Successful utilization of this nuanced classifi-cation system requires a comprehensive understanding of ISNCSCI rules, terminology/definitions, and several complex concepts. Inherent challenges and common mistakes have been identified,2,3 and high error rates in classification have been reported.4–6 Studies have shown that incomplete injuries are associated with lower classification accuracy than complete lesions.7,8 The most frequent types of errors include the incorrect identification of motor levels, ASIA Impairment Scale (AIS) grade, and the zones of partial preservation (ZPPs).4–7 Performance on the ISNCSCI has been found to be directly related to experience level,4–7 and formal training has been shown to significantly increase classification accuracy.7,9,10
The ISNCSCI continues to evolve, with its most recent edition published in 2019.11 This update introduced two important concepts, including the documentation of non-SCI conditions and a revised definition of the ZPP.11–14 Difficulties and questions regarding the 2019 ISNCSCI have been described,12 but there have been no studies investigating ISNCSCI classification accuracy since the newest revision was introduced.
The primary aims of this study were to (1) evaluate classification accuracy of SCI professionals using the 2019 ISNCSCI edition, (2) identify the most common mistakes and areas of confusion, and (3) assess associations between experience in ISNCSCI classification and accuracy. Additionally, our goal was to offer detailed explanations to enable replication of correct classifications based upon the most updated revisions for further reference (see eAppendix). Two main hypotheses were proposed: (1) Knowledge deficits about the classification rules and concepts introduced in the 2019 revision would result in common errors; and (2) self-rated experience in ISNCSCI classification would be directly associated with classification performance.
Methods
Survey setting and participants
In October 2021, members of the International Spinal Cord Society (ISCoS) and attendees of the ISCoS Annual Scientific Meeting 2021 were invited to complete an anonymous ISNCSCI survey. A description of the study and survey link were sent by ISCoS via an electronic newsletter to a total of 2050 recipients included in the ISCoS electronic mailing list. To increase the survey response rate, a reminder e-mail was sent to all participants after 1 week. The survey remained open for 2 weeks.
Statement of ethics
This study was submitted to the authors’ institutional review board and qualified for exempt status. Participation in this survey was voluntary and anonymous.
Survey design and data handling
The survey was developed and data were collected using REDCap hosted at the authors’ academic institution. Data were stored within an Excel spreadsheet.
The survey included five items modified from the previously published European Multicenter Study about Spinal Cord Injury (EMSCI) Participants’ Questionnaire for the ISNCSCI Instructional Course15 about participant occupation (physician, physical therapist, occupational therapist, nurse, researcher, other rehabilitation professional), experience in SCI medicine (<1 year, 1–5 years, 6–10 years, >10 years), experience in performing the ISNCSCI examination (novice, experienced, highly experienced, expert), experience in performing the ISNCSCI classification (novice, experienced, highly experienced, expert), and frequency of performing the ISNCSCI classification (<1x/month, 1x/month, 1x/week, 2x/week, ≥1x/day). For participants who selected “physician” as their occupation, branching logic was used to further characterize their experience (i.e., trainee vs. attending physician) and medical specialty.
Participants were also asked to classify six ISNCSCI test cases. The cases included complete data of the sensory, motor, and anorectal examinations presented on 2019 ISNCSCI worksheets. There were 11 multiple choice questions for each case that corresponded to the right/left sensory level, right/left motor level, neurological level of injury (NLI), completeness, AIS grade, right/left sensory ZPP, and right/left motor ZPP. There were five answer choices for each question. Correct responses were agreed upon by the authors and confirmed using an electronic computational algorithm (version 2.0)16 maintained and updated by Praxis Spinal Cord Institute. Participants were instructed to classify the cases independently without referring to other individuals or resources.
Test cases
The cases tested the participants’ understanding and application of the rules for the determination of levels, ZPP, AIS grade, and use of non-key muscles, as well as their proficiency with newer concepts from the 2019 revisions. Several cases were either similar (Cases 1 and 39) or identical (Cases 29 and 514) to previously published cases in order to evaluate important ISNCSCI rules and established classification challenges. Cases 4 and 6 were created by the authors to assess the documentation of non-SCI conditions. Cases 1–3 and 5–6 tested the utilization of the revised ZPP definition, particularly for incomplete injuries (see Figures 1–6). All cases included one or more challenging concepts, such as the differentiation of motor incomplete injuries, to avoid ceiling effects in the experts.
Figure 1.
Worksheet for Case 1, representing a cervical motor incomplete (C3 AIS C) lesion without voluntary anal contraction (VAC). Newer concepts tested in this case include the utilization of not applicable (NA) for both sensory zones of partial preservation (ZPPs) due to bilateral sensory sacral sparing and the applicability of motor ZPPs in the absence of VAC, as documentation of ZPPs in incomplete injuries was one of the changes in 2019. AIS = ASIA Impairment Scale; I = incomplete.
Figure 6.
Worksheet for Case 6, representing a thoracic motor incomplete injury (T6* AIS C*) in the setting of a right arm contusion and pain. Newer concepts tested in this case include the use of the “*” tag for documentation of a non-SCI condition as well as the utilization of not applicable (NA) for bilateral sensory and motor zones of partial preservation (ZPPs) in the presence of sensory and motor sacral sparing. AIS = ASIA Impairment Scale; I = incomplete; RUE = right upper extremity; VAC = voluntary anal contraction.
Figure 2.
Worksheet for Case 2, representing a thoracic sensory incomplete injury (T11 AIS B). Newer concepts tested in this case include the utilization of not applicable (NA) for both sensory zones of partial preservation (ZPPs) due to bilateral sensory sacral sparing and the applicability of motor ZPPs in the absence of voluntary anal contraction (VAC). AIS = ASIA Impairment Scale; I = incomplete.
Figure 3.
Worksheet for Case 3, representing a lumbar motor incomplete lesion (L2 AIS D). A newer concept tested in this case is the utilization of not applicable (NA) for bilateral sensory and motor zones of partial preservation (ZPPs) due to sensory and motor sacral sparing. AIS = ASIA Impairment Scale; I = incomplete.
Figure 4.
Worksheet for Case 4, representing a thoracic neurological complete injury (T11* AIS A) in the presence of a left transradial amputation. A newer concept tested in this case is the use of the “*” tag for documentation of a non-SCI condition. AIS = ASIA Impairment Scale; C = complete; LUE = left upper extremity; NT = not testable.
Figure 5.
Worksheet for Case 5, representing a thoracic motor incomplete injury (T6 AIS C) with non-key muscle function. Important concepts tested in this case include the use of non-key muscle function for the determination of an AIS B versus C injury, utilization of not applicable (NA) for each sensory zone of partial preservation (ZPP) due to bilateral sensory sacral sparing, and the applicability of bilateral motor ZPPs in the absence of voluntary anal contraction (VAC). AIS = ASIA Impairment Scale; I = incomplete.
Statistical analysis
Descriptive statistics (i.e., counts, percentages) were used to quantify errors and determine classification accuracy. Types of errors were recorded, and common mistakes were identified. Statistical analysis was conducted using SPSS (version 26.0; IBM, Armonk, NY) as follows: Categorical variables were expressed as percentages and absolute frequencies. Common errors were summed, and mean percentages of correctly determined classification components were calculated. Due to the non-normal distribution of the data, Kruskal-Wallis one-way analysis of variance with Dunn’s test using rank sums and Mann-Whitney U test were used for comparisons within groups. Statistical significance was set a priori at p < .05. The number of subgroups was reduced and pooled for the nonparametric one-way analysis of variance in each group. The subsequent pooled subgroups are listed in eTable 1.
Results
Of the 226 surveys that were opened, 107 were completed and were included in the analysis (47.4%). Most of the participants were physiatrists (49%, n = 52), had >10 years of experience in the field of SCI medicine (59%, n = 63), rated themselves as “highly experienced” in the ISNCSCI examination (36%, n = 38) and classification (42%, n = 45), and performed the ISNCSCI classification 2x/week (31%, n = 33) (see Figure 7). Of the incomplete records, 30% (n = 35) were opened but never initiated, 44% (n = 52) had the questions about occupation and experience fully completed but no cases initiated, and 27% (n = 32) had the questions about occupation and experience answered but the cases only partially completed. Of the respondents who partially completed the cases, 69% (n = 22) were physicians who had >6 years of experience in SCI medicine and described themselves as “experienced” or “highly experienced” in performing the ISNCSCI classification.
Figure 7.
Participant characteristics. ISNCSCI = International Standards for Neurological Classification of Spinal Cord Injury.
Overall classification accuracy was 74.6% ± 15.7%. Only one participant (0.9%) achieved a score of 100% on all six cases. Accuracy was highest for injury completeness (95.3% ± 8.5%) and sensory level (91.1% ± 14.8%), whereas motor ZPP (54.7% ± 23.9%) and AIS grade (57.3% ± 19.7%) were more difficult to correctly determine. Classification accuracy was 75.6% ± 23.5% for motor level, 80.4% ± 21.3% for NLI, and 72.4% ± 33.4% for sensory ZPP (see Figure 8).
Figure 8.
Overall mean accuracy rates for each of the classification components. Values are mean ± SD. AIS = ASIA Impairment Scale; NLI = neurological level of injury; ZPP = zone of partial preservation.
There was a significant association between overall classification accuracy and self-rated experience in the ISNCSCI classification (p = .017). Mean accuracy was significantly higher for participants who classified themselves as either “experienced” or “expert” in ISNCSCI classification than for participants who described themselves as “novice” (75.3% ± 13.0% and 77.8% ± 17.6% vs. 54.8% ± 21.7 %; p = .011 and p = .008, respectively) (see Figure 9). Of the individual classification variables, only motor level had significant associations between accuracy and self-reported level of experience in both the ISNCSCI examination and classification (see eTable 2). For example, motor level accuracy was 40.5% for persons who rated themselves as “novice” in ISNCSCI classification and 79.9% for individuals who self-reported themselves as being “expert” (p = .002). It is important to note that the significant association between experience in ISNCSCI classification and overall classification accuracy can only be attributed to the relationship between motor level accuracy and experience in ISNCSCI classification. Experience with the ISNCSCI examination, experience in SCI medicine, and occupation were not found to be significantly associated with overall classification accuracy (p = .067, p = .529, and p = .494, respectively) (see eTable 3).
Figure 9.
The effect of experience and frequency of International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI) classification on accuracy. Data shown as mean ± SD. * indicates significance (p < .05).
Frequency of performing the ISNCSCI classification was significantly associated with overall classification accuracy (p = .014). Participants who perform the ISNCSCI classification ≥1x/week achieved higher overall classification accuracy than those who perform the classification <1x/month (77.3% + 14.7% vs. 63.2% ± 16.9%, respectively; p = .005) (see Figure 9 and eTable 3). Regarding the individual variables, there were significant associations between the frequency of ISNCSCI classification and accuracy of both AIS grade and motor ZPP. Participants who perform the ISNCSCI classification ≥1x/week achieved a higher percentage of correct responses than those who perform the classification <1x/month (60.2% ± 19.9% vs. 44.9% ± 14.2%, p = .009 for AIS grade and 57.5% ± 24.7% vs. 37.8% ± 19.7%, p = .012 for motor ZPP) (see eTable 2).
Classification accuracy for the individual cases ranged from 66.2% ± 17.9 (Case 4) to 83.3% ± 18.1% (Case 2) (see Table 1). The percentage of participants who achieved a score of 100% was lowest for Case 4 (6.5%) and highest for Case 2 (38.3%). Table 2 demonstrates each case’s challenging classification components and includes the correct response and most common errors for each variable.
Table 1.
Classification accuracy for each case and variable
| Classification variable | No. of correct responses (percentage correct) | |||||
|---|---|---|---|---|---|---|
|
| ||||||
| Case 1 | Case 2 | Case 3 | Case 4 | Case 5 | Case 6 | |
| Right sensory level | 102 (95.3) | 104 (97.2) | 102 (95.3) | 105 (98.1) | 105 (98.1) | 88 (82.2) |
| Left sensory level | 100 (93.5) | 104 (97.2) | 101 (94.4) | 50 (46.7) | 105 (98.1) | 104 (97.2) |
| Right motor level | 67 (62.6) | 90 (84.1) | 72 (67.3) | 95 (88.8) | 97 (90.7) | 54 (50.5) |
| Left motor level | 85 (79.4) | 89 (83.2) | 80 (74.8) | 53 (49.5) | 98 (91.6) | 91 (85.0) |
| NLI | 97 (90.7) | 101 (94.4) | 99 (92.5) | 59 (55.1) | 102 (95.3) | 58 (54.2) |
| Completeness | 84 (78.5) | 105 (98.1) | 107 (100) | 107 (100) | 103 (96.3) | 106 (99.1) |
| AIS grade | 34 (31.8) | 83 (77.6) | 58 (54.2) | 104 (97.2) | 34 (31.8) | 55 (51.4) |
| Right sensory ZPP | 67 (62.6) | 80 (74.8) | 85 (79.4) | 88 (82.2) | 75 (70.1) | 83 (77.6) |
| Left sensory ZPP | 65 (60.8) | 81 (75.7) | 85 (79.4) | 61 (57.0) | 76 (71.0) | 84 (78.5) |
| Right motor ZPP | 74 (69.2) | 72 (67.3) | 76 (71.0) | 30 (28.0) | 27 (25.2) | 76 (71.0) |
| Left motor ZPP | 72 (67.3) | 72 (67.3) | 75 (70.1) | 27 (25.2) | 24 (22.4) | 77 (72.0) |
|
| ||||||
| Total (N = 1177) | 847 (72.0 ± 20.4) | 981 (83.3 ± 18.1) | 940 (79.9 ± 23.4) | 779 (66.2 ± 17.9) | 846 (71.9 ±16.) | 876 (74.4 ± 22.5) |
Note: Shaded scores represent correct responses less than 70%. AIS = ASIA Impairment Scale; NLI = neurological level of injury; ZPP = zone of partial preservation.
Table 2.
Challenging classification components (ordered by level of difficulty), correct responses, and the most common errors for each case
| Case | Classification component | Correct response (% of responses) | Most common incorrect responses (% of responses) |
|---|---|---|---|
| 1 | AIS grade | C (31.8) | B (45.8) |
| Left sensory ZPP | NA (60.8) | S4-5 (24.3) | |
| Right sensory ZPP | NA (62.6) | S4-5 (25.2) | |
| Right motor level | C5 (62.6) | C4 (29.9) | |
| Left motor ZPP | C7 (67.3) | NA (29.0) | |
| Right motor ZPP | C7 (69.2) | NA (25.2) | |
|
| |||
| 2 | Right motor ZPP | L2 (67.3) | NA (29.9) |
| Left motor ZPP | L2 (67.3) | NA (29.2) | |
| Right sensory ZPP | NA (74.8) | S4-5 (18.7) | |
| Left sensory ZPP | NA (75.7) | S4-5 (18.7) | |
| AIS grade | B (76.6) | C (19.6) | |
|
| |||
| 3 | AIS grade | D (54.2) | C (29.4) |
| Right motor level | L3 (67.3) | L2 (23.4) | |
| Left motor ZPP | NA (70.1) | L5 (21.5) | |
| Right motor ZPP | NA (71.0) | L5 (23.4) | |
| Left motor level | L3 (74.8) | L2 (23.4) | |
|
| |||
| 4 | Left motor ZPP | T11* (25.2) | NA (42.1); L1 (24.3) |
| Right motor ZPP | T11 (28.0) | NA (41.1); L1 (24.3) | |
| Left sensory level | T11* (46.7) | T11 (45.8) | |
| Left motor level | T11* (49.5) | T11 (36.5) | |
| NLI | T11* (55.1) | T11 (39.3) | |
| Left sensory ZPP | L1 (57.0) | L1* (33.6) | |
|
| |||
| 5 | Left motor ZPP | L2 (22.4) | NA (59.8); T6 (15.9) |
| Right motor ZPP | L2 (25.2) | NA (57.9); T6 (15.0) | |
| AIS grade | C (31.8) | B (63.6) | |
| Right sensory ZPP | NA (70.1) | S4-5 (21.5) | |
| Left sensory ZPP | NA (71.0) | S4-5 (20.6) | |
|
| |||
| 6 | Right motor level | T6* (50.5) | T6 (33.6); C5* (10.3) |
| AIS grade | C* (51.4) | C (38.3) | |
| NLI | T6* (54.2) | T6 (34.6) | |
| Right motor ZPP | NA (71.0) | L4 (20.6) | |
| Left motor ZPP | NA (72.0) | S1 (19.6) | |
Note: AIS = ASIA Impairment Scale; NLI = neurological level of injury; ZPP = zone of partial preservation.
2019 Updates
The newer concepts, including the appropriate documentation of non-SCI conditions and classification of ZPP in incomplete injuries, contributed to several common errors. In Cases 4 and 6, the side affected by the non-SCI condition had more sensory and motor level errors than the unaffected side. As an example, in Case 4 (left transradial amputation), the left sensory level was incorrectly classified by 53.3% of participants, whereas only 1.9% of right sensory level responses were incorrect; this difference was significant (p = .000). The left and right motor levels were incorrectly classified by 50.5% and 11.2% of participants, respectively (p = .000). In Case 6 (right arm contusion/pain), the right sensory level was classified incorrectly by 17.8% of participants, whereas only 2.8% of left sensory level responses were incorrect; this difference was significant (p = .000). The right and left motor levels were incorrectly classified by 49.5% and 15.0% of participants, respectively (p = .000). Seventy-seven percent of all NLI errors were made on Cases 4 and 6. Of these errors, 81.4% involved the correct level with omission of the “*” tag. Similarly, 78.8% of all AIS errors in Case 6 involved the correct grade (AIS C) but were lacking a “*” tag. The most common mistakes related to the ZPP involved the revised definition for incomplete injuries. In cases with preserved sensory sacral sparing (Cases 1–3 and 5–6), 26% of all sensory ZPP responses were incorrect due to the selection of a ZPP other than not applicable (NA). Similarly, in cases with preserved voluntary anal contraction (VAC; Cases 3 and 6), 29.5% of motor ZPP responses were incorrect due to the selection of a ZPP other than NA. In cases with an incomplete injury and absent VAC (Cases 1, 2, and 5), 38.6% of motor ZPP responses were incorrect due to the selection of NA.
Discussion
A primary aim of this study was to assess ISNCSCI classification accuracy of an international cohort of SCI professionals following the 2019 ISNCSCI updates. Classification performance on these cases, which represent multiple challenges, was relatively modest (overall mean accuracy of 74.6%) but greater than previous reports using earlier editions of the ISNCSCI4–10,17 (see Table 3). It is important to note that a direct comparison of the results of this survey with previous reports is challenging due to a different composition of participants and classification cases. Much of the published data is derived from studies evaluating the effectiveness of trainings of earlier editions of the ISNCSCI4–10,17 and includes accuracy of both pre- and post-training assessments (see Table 3). Within the literature, pretest classification accuracy ranges from 49.6% to 72.5%7–9,17 whereas reported posttest accuracy ranges from 80.4% to 94.3%.7–9,17 Another study by Osunronbi and Sharma4 retrospectively evaluated the classification accuracy of 96 ISNCSCI worksheets and found the overall accuracy of completed components to be 78.1%.
Table 3.
Comparison of ISNCSCI classification accuracy across studies
| Cohen et al., 19988 | Chafetz et al., 20087 | Schuld et al., 20139 | Schuld et al., 20156 | Schuld et al., 201617 | Osunronbi & Sharma, 20194 | Current study | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Pre-test | Post-test | Pre-test | Post-test | Pre-test | Post-test | Chart review | Pre-test | Post-test | Pre-test | Post-test | Chart review | Single survey | |
| Sensory level accuracy | 84.8% | 86.3% | 91% | 97% | 60.6% | 96.8% | 90.4% | 63.2% | 97.9% | 69.1% | 97.9% | 43.5% | 91.1% |
| Motor level accuracy | 66.5% | 68.8% | 58% | 82% | 20.8% | 81.9% | 62.0% | 31.4% | 83.2% | 30.9% | 88.1% | 36.8% | 75.6% |
| NLI accuracy | — | — | 69% | 94% | — | — | — | 50.0% | 86.1% | 49.8% | 90.9% | 71.4% | 80.4% |
| Completeness accuracy | 95.5% | 98.5% | 91% | 100% | 83.6% | 96.2% | — | 82.5% | 98.3% | 79.6% | 98.5% | 83.3% | 95.3% |
| AIS accuracy | 76.0% | 81.5% | 50% | 79% | 45.8% | 88.1% | 83.4% | 49.4% | 86.1% | 47.2% | 86.0% | 75.0% | 57.3% |
| Sensory ZPP accuracy | 55.8% | 78.5% | — | — | 51.4% | 93.3% | 91.6% | 61.0% | 94.4% | 65.7% | 96.8% | — | 72.4% |
| Motor ZPP accuracy | 57.0% | 78.3% | — | — | 50.5% | 93.3% | 80.8% | 61.0% | 93.3% | 65.9% | 96.4% | — | 54.7% |
| Overall accuracy | 70.0% | 80.4% | 72.5% | 89.9% | 49.6% | 91.5% | — | 56.5% | 92.2% | 59.0% | 94.3% | 78.1% | 74.6% |
| Number of participants | 106 | 28 | 106 | — | 72 | 53 | 50 | 107 | |||||
| Number of cases | 2 | 10 | 5 | 420 | 5 | 5 | 96 | 6 | |||||
| ISNCSCI edition | 1992 | 2006 | 2011 | 2003 | 2011 | 2013 | 2006, 2015 | 2019 | |||||
Note: AIS = ASIA Impairment Scale; ISNCSCI = International Standards for Neurological Classification of Spinal Cord Injury; NLI = neurological level of injury; ZPP = zone of partial preservation.
In this survey, motor ZPPs, AIS grade, sensory ZPPs, and motor levels were classification components with the lowest accuracy (54.7%, 57.3%, 72.4%, and 75.6%, respectively). These results are consistent with previously published literature, all using earlier versions of the ISNCSCI (see Table 3).4–9,17 Schuld et al.6 compared 420 manually classified ISNCSCI worksheets to that of a validated computational algorithm and found the lowest agreement for motor levels (right 62.1%; left 61.8%), motor ZPP (right 81.6%; left 80.0%), and AIS grade (83.4%). In a similar study by Armstrong et al.5 comparing 184 manually classified ISNCSCI worksheets to a computerized algorithm, motor levels and ZPPs were found to comprise the greatest percentage of errors (30.1% and 24.0%, respectively). Osunronbi and Sharma4 assessed the accuracy of 96 ISNCSCI worksheets and found that of seven classification components (motor scores, sensory scores, sensory levels, motor levels, NLI, completeness, and AIS grade), motor levels and sensory levels had the lowest accuracy (36.8% and 43.5%, respectively).
Although mean accuracy of AIS grade in this study was only 57.3%, this was higher than several reports in the literature (see Table 3).7,9,17 AIS grade accuracy was highest in Case 4 (97.2%), despite this case having the lowest overall classification accuracy (see Table 1). Of the six cases, Case 4 was the only neurological complete injury (AIS A). These results are similar to other studies that have shown fewer classification challenges for complete versus incomplete lesions.7,8 In this study, classification accuracy of AIS grade was lowest in Cases 1 and 5, both AIS C injuries and each with an accuracy of 31.8%. This is consistent with previous reports that AIS C grade is the most difficult for respondents to classify.6,7 Two important yet challenging concepts in the determination of AIS C from B grade include the consideration of non-key muscle functions (a concept first introduced in the ISNCSCI 2003 edition18 and clarified in the 2011 revision19) and the presence of motor function more than three levels below the motor level on either side in injuries with sensory sacral sparing and no VAC (first presented in the 2000 edition).
Mean motor level accuracy in this study was 75.6%, which is higher than previous reports (see Table 3). Violation of the “motor follows sensory” rule is a common error described in the literature,4,5,9,17,20 but results of this study indicate the rule is now better understood. Motor level errors in this study were primarily due to confusion about the definition (the motor level is the lowest key muscle function with a grade of at least 3, provided that all motor functions are intact above that level) and omission of the “*” tag when indicated due to a non-SCI condition.
This is the first study to assess classification accuracy on challenging cases since publication of the 2019 ISNCSCI revision. This work highlights important classification challenges and knowledge deficits regarding the documentation of non-SCI conditions and revised definition of the ZPP. These newer concepts resulted in a large percentage of errors, validating the first hypothesis of this study. In Cases 4 and 6, difficulty and/or lack of familiarity with the “*” designation is evidenced by lower accuracy of multiple classification components on the side affected by the non-SCI condition. The most notable comparison is between the right and left sensory levels of Case 4, as the classification accuracy of these variables was 98.1% and 46.7%, respectively. For classification components without a side-to-side comparison, knowledge deficits surrounding the “*” concept remained evident. For example, 77.0% of all NLI errors were found on Cases 4 and 6, and 81.4% of these mistakes were due to omission of the “*” tag. Based on the results of this study, it is apparent that the ZPP definitions, especially as they relate to the presence or absence of sacral sparing, are not completely understood, as motor more so than sensory ZPPs were consistently among variables with the lowest classification accuracy. In cases with preserved sensory sacral sparing and in cases with preserved VAC, S4-5 was often selected despite the correct sensory and/or motor ZPP being NA. Additionally, in cases with an incomplete injury and absent VAC, the selection of NA was a common error, violating the rule that a motor ZPP is applicable in all injuries without motor sacral sparing.
Further discussion of each case, including review of important classification rules and clarification of common areas of confusion, can be found in the eAppendix. Table 2 also demonstrates challenging classification components and the most common incorrect responses.
Self-rated experience in ISNCSCI classification was significantly associated with overall classification accuracy, supporting the second hypothesis. Further analysis evaluating the relationship between ISNCSCI classification experience and accuracy of the individual classification components revealed a significant association for motor level accuracy only. Self-rated experience with the ISNCSCI examination was not significantly associated with overall classification accuracy. Skills required for successful classification are distinct from those of the examination. However, it is also important to note that in practice, a valid classification demands an accurate examination. Interestingly, greater experience in SCI medicine was not significantly associated with higher overall classification accuracy. Previous studies have also examined associations between experience and classification accuracy.4,7,9,17 Osunronbi and Sharma4 found that the overall accuracy rate was significantly higher for senior clinicians compared to that of junior clinicians (82.3% vs. 69.8%, respectively). Schuld et al.9 reported that “experience in ISNCSCI” was significantly associated with pretest performance, with self-rated experts receiving the highest scores; it is important to note that this survey question did not differentiate experience in the classification from that of the examination. These authors also found that experience in SCI medicine was not significantly associated with pretest scores. In a study by Chafetz et al.,7 findings demonstrated a significant relationship between both experience with the ISNCSCI and SCI experience. Similar to other studies,7,9,17 occupation was not associated with classification accuracy. Additionally, subgroup analysis failed to identify a significant difference in overall classification accuracy between physicians with and without SCI board certification.
This study found a significant association between overall classification accuracy and frequency of performing the ISNCSCI classification, with participants who perform the ISNCSCI classification ≥1x/week demonstrating the highest scores and those performing the classification <1x/month having the lowest scores. In contrast, Schuld and colleagues9 did not find a significant relationship between the frequency of performing the ISNCSCI examination and pretest accuracy. One possible explanation for the difference in these findings could be that the Schuld et al.9 survey question inquired about examination frequency as opposed to classification frequency.
Previous studies7–9,17 have demonstrated that ISNCSCI training significantly improves classification accuracy (see Table 3 for pre- and posttest comparisons). Evaluating the effectiveness of formal training in enhancing proficiency with the 2019 ISNCSCI edition was beyond the scope of this study but will be a primary focus of future work. The results of this study also show that it can take years for ISNCSCI updates to be taught to and adopted by the global SCI community.21 It is likely that education and widespread utilization of the newest changes were adversely affected by the coronavirus disease 2019 (COVID-19) pandemic as in-person ISNCSCI workshops and courses were limited. More freely available information about the ISNCSCI is needed, including open access publications, webinars, and training sessions.
Limitations
There may have been a nonresponse bias regarding those individuals who failed to complete the survey as it was sent to 2050 individuals. However, this survey response rate is within the range of previously published response rates for comparable surveys.22,23 Additionally, only 47% of the persons who initiated the survey completed it. Although the specific reasons for this are unclear, it is unlikely that this impacted the findings of the classification challenges, and the high percentage of SCI professionals who rated themselves as “highly experienced” or “expert” in ISNCSCI classification helped to disentangle the most difficult rules/concepts.
Another limitation may be that the survey was only available in English, which could have led to some confusion for individuals in an international SCI organization for whom English is not their primary language. Even though the sample size was large compared to other studies (see Table 3), it may not have been large enough to detect significant associations between self-rated experience and accuracy of individual classification components other than motor level. The survey questions related to participant experience also had inherent limitations. First, it is possible there was response bias in terms of the self-reported scales. Second, the self-assessment lacked defined criteria for each level of experience. Third, this study did not collect data on participant experience specifically with the 2019 revision. Lastly, participants were presented with five answer choices when classifying each variable. These options may have improved accuracy because of potential cuing to the correct response, although the true nature of this is not known.
Conclusion
Classification accuracy of an international cohort of SCI professionals is modest. Self-reported experience in ISNCSCI classification is significantly associated with overall classification and motor level performance. Confusion and knowledge deficits regarding the recently introduced documentation of non-SCI conditions and revised definition of ZPP are prevalent and contribute to common classification errors. Further education in the utilization of the 2019 ISNCSCI edition is needed.
Supplementary Material
Footnotes
Disclosures
Brittany A. Snider, DO, is a member of the ASIA Education Committee. Ronald K. Reeves, MD, is a member of the ASIA Education Committee and Vice Chair of the ISCoS Education Committee. Steven C. Kirshblum, MD, serves on ASIA’s Board of Directors and the International Standards Committee. Rüdiger Rupp, PhD, serves on ASIA’s Board of Directors and is Chair of the International Standards Committee.
REFERENCES
- 1.American Spinal Injury Association Standards for Neurological Classifications of Spinal Injured Patients . Chicago, IL: Author; 1982. [Google Scholar]
- 2.Solinsky R, Kirshblum SC. Challenging questions regarding the international standards. J Spinal Cord Med . 2018;41(6):684–690. doi: 10.1080/10790268.2017.1362929. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Kirshblum SC, Biering-Sorensen F, Betz R et al. International Standards for Neurological Classification of Spinal Cord Injury: cases with classification challenges. J Spinal Cord Med . 2014;37(2):120–127. doi: 10.1179/2045772314Y.0000000196. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Osunronbi T, Sharma H. International Standards for Neurological Classification of Spinal Cord Injury: Factors influencing the frequency, completion and accuracy of documentation of neurology for patients with traumatic spinal cord injuries. Eur J Orthop Surg Traumatol . 2019;29(8):1639–1648. doi: 10.1007/s00590-019-02502-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Armstrong AJ, Clark JM, Ho DT et al. Achieving assessor accuracy on the International Standards for Neurological Classification of Spinal Cord Injury. Spinal Cord . 2017;55(11):994–1001. doi: 10.1038/sc.2017.67. [DOI] [PubMed] [Google Scholar]
- 6.Schuld C, Franz S, van Hedel HJA et al. International standards for neurological classification of spinal cord injury: Classification skills of clinicians versus computational algorithms. Spinal Cord . 2015;53(4):324–331. doi: 10.1038/sc.2014.221. [DOI] [PubMed] [Google Scholar]
- 7.Chafetz RS, Vogel LC, Betz RR, Gaughan JP, Mulcahey MJ. International standards for neurological classification of spinal cord injury: Training effect on accurate classification. J Spinal Cord Med . 2008;31(5):538–542. doi: 10.1080/10790268.2008.11753649. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Cohen ME, Ditunno JFJ, Donovan WH, Maynard FMJ. A test of the 1992 International Standards for Neurological and Functional Classification of Spinal Cord Injury. Spinal Cord . 1998;36(8):554–560. doi: 10.1038/sj.sc.3100602. [DOI] [PubMed] [Google Scholar]
- 9.Schuld C, Wiese J, Franz S et al. Effect of formal training in scaling, scoring and classification of the International Standards for Neurological Classification of Spinal Cord Injury. Spinal Cord . 2013;51(4):282–288. doi: 10.1038/sc.2012.149. [DOI] [PubMed] [Google Scholar]
- 10.Mulcahey MJ, Gaughan J, Betz RR, Vogel LC. Rater agreement on the ISCSCI motor and sensory scores obtained before and after formal training in testing technique. J Spinal Cord Med . 2007;30(Suppl 1):S146–9. [PMC free article] [PubMed] [Google Scholar]
- 11.American Spinal Injury Association International Standards for Neurological Classification of Spinal Cord Injury . Richmond, VA: Author; 2019. [Google Scholar]
- 12.Kirshblum S, Schmidt Read M, Rupp R. Classification challenges of the 2019 revised International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI) Spinal Cord . 2022;60(1):11–17. doi: 10.1038/s41393-021-00648-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Rupp R, Biering-Sørensen F, Burns SP et al. International Standards for Neurological Classification of Spinal Cord Injury: Revised 2019. Top Spinal Cord Inj Rehabil . 2021;27(2):1–22. doi: 10.46292/sci2702-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Kirshblum S, Snider B, Rupp R, Read MS. Updates of the International Standards for Neurologic Classification of Spinal Cord Injury: 2015 and 2019. Phys Med Rehabil Clin N Am . 2020;31(3):319–330. doi: 10.1016/j.pmr.2020.03.005. [DOI] [PubMed] [Google Scholar]
- 15.Franz S, Heutehaus L, Weinand S, Weidner N, Rupp R, Schuld C. Theoretical and practical training improves knowledge of the examination guidelines of the International Standards for Neurological Classification of Spinal Cord Injury. Spinal Cord . 2022;60(1):1–10. doi: 10.1038/s41393-020-00578-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Available from: https://www.isncscialgorithm.com.
- 17.Schuld C, Franz S, Brüggemann K et al. International standards for neurological classification of spinal cord injury: impact of the revised worksheet (revision 02/13) on classification performance. J Spinal Cord Med . 2016;39(5):504–512. doi: 10.1080/10790268.2016.1180831. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.American Spinal Injury Association Reference Manual for the International Standards for Neurological Classification of Spinal Cord Injury . Chicago, IL: Author; 2003. [Google Scholar]
- 19.Burns S, Biering-Sørensen F, Donovan W et al. International Standards for Neurological Classification of Spinal Cord Injury, Revised 2011. Top Spinal Cord Inj Rehabil . 2012;18(1):85–99. doi: 10.1310/sci1801-85. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Waring WP, 3rd, Biering-Sorensen F, Burns S et al. 2009 review and revisions of the international standards for the neurological classification of spinal cord injury. J Spinal Cord Med . 2010;33(4):346–352. doi: 10.1080/10790268.2010.11689712. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Schuld C, Franz S, Schweidler J et al. Implementation of multilingual support of the European Multicenter Study about Spinal Cord Injury (EMSCI) ISNCSCI calculator. Spinal Cord . 2022;60(1):37–44. doi: 10.1038/s41393-021-00672-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Yonamine S, Ton L, Rose-Nussbaumer J et al. Survey of the American Glaucoma Society Membership on current glaucoma drainage device placement and postoperative corticosteroid use. Clin Ophthalmol . 2022;16:2305–2310. doi: 10.2147/OPTH.S369673. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Fuchs A, Frodl A, Yilmaz T et al. [Reality of care of patella stabilizing operations : Astatus quo among active members of the German Society for Orthopedic and Trauma Surgery (DGOU)] Orthopadie (Heidelberg Ger . 2022;51(8):652–659. doi: 10.1007/s00132-022-04264-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.