Skip to main content

Some NLM-NCBI services and products are experiencing heavy traffic, which may affect performance and availability. We apologize for the inconvenience and appreciate your patience. For assistance, please contact our Help Desk at info@ncbi.nlm.nih.gov.

The Journal of Spinal Cord Medicine logoLink to The Journal of Spinal Cord Medicine
. 2019 Jun 17;43(5):616–622. doi: 10.1080/10790268.2019.1628496

Reliability of S3 pressure sensation and voluntary hip adduction/toe flexion and agreement with deep anal pressure and voluntary anal contraction in classifying persons with traumatic spinal cord injury

Ralph J Marino 1,, Mary Schmidt-Read 2, Anna Chen 3, Steven C Kirshblum 4,5, Trevor A Dyson-Hudson 5,6, Edelle Field-Fote 7, Ross Zafonte 8,9,10
PMCID: PMC7534209  PMID: 31204908

Abstract

Context/Objective: The sacral examination components of the International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI), namely deep anal pressure (DAP) and voluntary anal sphincter contraction (VAC), are often difficult to perform. We evaluated whether pressure sensation at the S3 dermatome (S3P), and voluntary hip adductor or toe flexor contraction (VHTC) are tenable alternatives. Here we report test–retest reliability and agreement of these components at 1 month after spinal cord injury (SCI), and impact of disagreement on American Spinal Injury Association (ASIA) Impairment Scale (AIS) grades.

Design: Longitudinal cohort. ISNCSCI examination, S3P and VHTC conducted at 1-month post-injury; retest of the sacral exam, S3P and VHTC within 3 days. Follow-up examinations performed at 3, 6, and 12 months.

Setting: Five Spinal Cord Injury Model System Centers.

Participants: Subjects with acute traumatic SCI, neurological levels T12 and above, AIS grades A–C.

Interventions: None.

Outcome Measures: ISNCSCI exam, AIS grades.

Results: Fifty-one subjects had 1-month data, and 39 had at least one follow-up examination. Test–retest reliability indicated perfect agreement (kappa = 1.0) for all data except S3P (kappa = 0.96). The agreement was almost perfect between S3P and DAP (kappa = 0.84) and between VHTC and VAC (kappa = 0.81). VHTC and VAC differed more often with neurologic levels below T10, possibly due to root escape in conus medullaris injuries.

Conclusion: S3P and VHTC show promise as alternatives to DAP and VAC for determining sacral sparing in persons with neurologic levels T10 and above. Reliability and agreement should be evaluated at earlier timepoints and in children with SCI.

Keywords: Spinal cord injuries, Reliability, Sacral sparing, Classification, Neurologic examination

Introduction

The anorectal portion of the International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI) exam can be difficult to perform in the acute period after spinal cord injury (SCI), and patients are often reluctant to undergo a rectal examination. In addition, stimulation of the rectal wall may result in autonomic-mediated sensation resulting in a false determination of a somatic sensory incomplete lesion.1 It would be useful to have alternatives to the digital rectal examination for determining the completeness of injury in SCI. Zariffa et al.2 appraised the ability of light touch or pinprick sensation in the S1, S2 or S3 dermatomes to predict sensory sacral sparing, and motor function in S1 (ankle plantar flexor) to predict voluntary anal contraction (VAC). They found an overall success rate for predicting sensory sacral sparing of 90.5% for S1 to 94.2% for S3 sensation, and 85.4% success rate of S1 motor for predicting VAC.

Clinicians have observed that the hip adductors and the toe flexors may be the earliest muscles in the lower extremities with volitional activity in a person with SCI. The hip adductors and toe flexors are easily tested in the supine position and would be more comfortable and less intrusive than testing VAC. We previously showed that testing for pressure sensibility over the S3 dermatome (S3P) was reliable and had substantial agreement with deep anal pressure (DAP).3 In this study we extend those findings and evaluate the reliability of voluntary hip adductor and toe flexor muscle contraction (VHTC) as an alternative to VAC. We also examine agreement between the classification of completeness using S3P and VHTC compared to DAP and VAC.

Methods

This was a longitudinal study of persons with acute traumatic SCI admitted to one of five participating SCI Model System Centers, with a baseline test–retest reliability study of sacral sparing and alternative sacral sparing components. IRB approval was obtained at all participating centers prior to subject enrollment into the study, and subjects gave written informed consent to participate.

Subjects

Inclusion criteria were: (1) a single traumatic SCI, (2) neurological levels between C1 and T12, American Spinal Injury Association (ASIA) Impairment Scale (AIS) grades A–C at 1-month post-injury (baseline), and (3) consented to participate. Subjects were excluded if they had other causes for neurologic deficits (e.g. stroke, peripheral nerve injury), persistent factors that would limit the quality or completeness of the neurological examination (e.g. extremity fractures, brain injury, dementia) or were unwilling to return for follow-up examinations. We excluded individuals who were AIS D at 1 month because almost all have sensory sacral sparing4 and the differentiation between AIS C and D does not rely on the anorectal assessment but on the motor scores of key muscles below the neurologic level of injury.2

Procedures

At 1-month post-injury, subjects had a neurological examination according to the ISNCSCI standards, including testing for DAP and VAC.5 Subjects also had testing of S3P and VHTC. All examiners underwent training via webinar and completed InSTeP training. For S3P testing, firm pressure was applied to the ischial tuberosity for one second. Subjects were asked to indicate if they felt pressure and to identify the side of pressure. For VHTC testing, subjects were asked to squeeze their knees together to test voluntary hip adduction and to curl their toes down to test voluntary toe flexion. Muscle contraction was recorded as present or absent based on visual observation. Up to three trials were conducted for each muscle group. Ability to contract the muscle in two out of three trials was considered a positive result. Contraction of either the hip adductor or toe flexor on either side was considered a positive VHTC. If there was no contraction of any of these four muscles, then VHTC was negative. The sacral sparing elements (DAP, VAC) and alternative elements (S3P, VHTC) were retested within 3 days of the 1-month exam. If the retest exam was performed on the same day as the full exam, then a different examiner performed the retest. There was no required order for the elements of sacral sparing or alternates. ISNCSCI examinations, S3P and VHTC were repeated at 3, 6 and 12 months post-injury without a retest of any components.

Analyses

Differences in baseline characteristics of subjects who did and did not complete the follow-up testing were examined using the t-test for continuous variables and Chi-square test for categorical variables. Statistical significance was set at 0.05. The kappa statistic was used to determine test–retest reliability of sacral sparing and alternative sacral sparing components, as well as the agreement of DAP with S3P and VAC with VHTC. The prevalence of responses for the agreement was not balanced, especially for VAC/VHTC, which can result in a paradoxically low kappa when the agreement is high.6 We therefore also report the prevalence-adjusted, bias-adjusted kappa (PABAK) which accounts for the imbalance.7 We used the criteria of Landis and Koch8 to interpret the magnitude of agreement based on kappa values: 0, poor; 0.01–0.20, slight; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, substantial; and 0.81–0.99, almost perfect; 1.00, perfect.

For subjects in whom sacral sparing and alternative sacral sparing components differed, we examined the impact on AIS grade if alternative components were used. When there was a difference in the 1-month exam, we looked at the final exam to evaluate conversions in AIS grades. We also assessed the impact of the level of injury (tetraplegia, high paraplegia (T1–9), low paraplegia (T10–12)) on classification using the alternative components.

Results

Subjects included 51 persons with acute traumatic SCI who had data at 1 month. Of these, 39 subjects had at least one follow-up examination. Demographic variables are presented in Table 1. There were not any significant differences in baseline characteristics for those who returned for follow-up and those who did not.

Table 1. Demographic and neurological characteristics of subjects.

Characteristics at baseline 1-month (n = 51) Follow-up (n = 39)
n % n %
Sex
 Men 44 86.27 33 84.62
 Women 7 13.73 6 15.38
Race
 White 24 47.06 19 48.72
 Black 18 35.29 14 35.90
 Hispanic 6 11.76 4 10.26
 Other 3 5.88 2 5.13
Etiology
 Vehicular 23 45.10 17 43.59
 Falls 11 17.65 6 15.38
 Violence 9 15.69 8 20.51
 Sports 8 21.57 8 20.51
Level of injury
 Tetraplegia 25 49.0 20 51.3
 Paraplegia T1–T9 17 33.3 15 38.5
 Paraplegia T10–12 9 17.7 4 10.3
ASIA Impairment Scale
 A 29 56.9 21 53.9
 B 9 17.7 8 20.5
 C 13 25.5 10 25.6
Age, y: mean (sd) 38.47 (18.01) 37.64 (17.99)

Test–retest reliability

Retest examinations were completed on 49 subjects except for VHTC, which was retested in 48 subjects. The retest was completed within 3 days of the first test in all but three cases where it was performed between 9 and 13 days later. The retest was performed by the same examiner in 37 cases and a different examiner in 12 cases. In five cases, the test and retest were performed on the same day; four of the five had a different examiner for the retest. Test–retest reliability was almost perfect for all sacral sparing and alternative sacral sparing components. Reliability was perfect (kappa = 1.0) for DAP, VAC and VHTC and almost perfect (kappa = 0.96) for S3P, where only 1/49 exams differed. In the case that differed, exams were performed by different examiners.

Agreement

Agreement between DAP and S3P at 1 month was almost perfect (Table 2), absolute agreement was 92.2% (kappa = 0.84; PABAK = 0.84); only 4/51 subjects differed – 2 with DAP but no S3P, and 2 with S3P but no DAP. Agreement between VAC and VHTC was similarly high (Table 2), absolute agreement was 94.1% (kappa = 0.81; PABAK = 0.88); only 3/51 subjects differed – all 3 with VHTC present but no VAC. Results were similar when considering all time points combined. Subjects had between one and four observations. There were 51 observations at 1 month, 34 at 3 months, 34 at 6 months and 30 at 12 months (total 149 observations). Absolute agreement for S3P and DAP was 92% (kappa = 0.84; PABAK = 0.84) and for VHTC and VAC was 93% (kappa = 0.82; PABAK = 0.87). The relationship between VAC and VHTC was similar when considering only hip adduction (kappa = 0.82; PABAK = 0.87) or toe flexion (kappa = 0.83; PABAK = 0.88).

Table 2. Agreement between S3P and DAP and between VHTC and VAC at baseline and all exams.

  DAP VAC
n n n n n n
Baseline
S3P Yes No Total VHTC Yes No Total
Yes 20 2 22 Yes 8 3 11
No 2 27 29 No 0 40 40
Total 22 29 51 Total 8 43 51
All exams
S3P Yes No Total VHTC Yes No Total
Yes 63 5 68 Yes 33 10 43
No 7 74 81 No 0 106 106
Total 70 79 149 Total 33 116 149

DAP, deep anal pressure; S3P, pressure over the S3 dermatome; VAC, voluntary anal contraction; VHTC, voluntary hip adductor and toe flexor muscle contraction.

AIS grade determination

Baseline examinations

AIS grade was determined using the ISNCSCI criteria and an alternative AIS grade (alt-AIS) using S3P instead of DAP and VHTC instead of VAC. Note that in our sample all subjects who had any S4-5 sensation also had DAP; therefore DAP was used to determine sensory sacral sparing. Baseline AIS grade and alt-AIS grade agreed in 48 out of 51 cases (kappa = 0.90). For the three subjects in whom VHTC and VAC differed, there was no difference between AIS and alt-AIS grade (Table 3). These three subjects had VHTC but no VAC. However, their AIS grades remained C because they all had DAP and met the criteria for motor incomplete – motor function more than three levels below the motor level on at least one side. When examining their final exam, VAC for two out of those three subjects became positive, agreeing with VHTC. They received a grade of D using both the standard and alternative grading methods.

Table 3. Cases where baseline sacral sparing and alternative components differed.
Case Exam time SNL AIS Alt-AIS S3P DAP VAC VHTC LEMS Notes
1 1 month C2 C C Y Y N Y 8  
1 3 months C4 C C Y Y N Y 16  
2 1 month C5 C C Y Y N Y 4  
2 12 months C6 D D Y Y Y Y 49  
3 1 month T6 C C Y Y N Y 5  
3 6 months T5 D D Y Y Y Y 28  
4 1 month C4 C C N Y Y Y 20  
4 3 months C3 C C N Y Y Y 14  
5 1 month C4 A B Y N N N 0  
5 6 months C2 C A N Y N N 0 Syrinx
6 1 month T11 B A N Y N N 0  
6 12 months T8 C C Y Y Y Y 19  
7 1 month T10 A B Y N N N 0  
7 Lost to follow up  

SNL, single neurologic level; AIS, ASIA Impairment Scale; alt-AIS, alternative AIS; S3P, pressure over the S3 dermatome; DAP, deep anal pressure; VAC, voluntary anal contraction; VHTC, voluntary hip adductor and toe flexor muscle contraction; LEMS, lower extremity motor score.

There was a difference in AIS and alt-AIS grade in three of four subjects whose DAP differed from S3P (Table 3). Two subjects were AIS grade A but had S3P making their alt-AIS grade B. The first was lost to follow-up. The third subject developed a syrinx and the single neurologic level ascended from C4 to C2. At the same time, the subject lost S3P but gained DAP. This subject met the criteria for AIS grade C due to motor function to C7 (five levels below new C2 motor level) but was alt-AIS grade A. One subject was AIS grade B but did not have S3P, making the alt-AIS grade A. This subject converted to AIS/alt-AIS grade C at follow-up. The fourth subject had DAP but not S3P. However, this subject had both VAC and VHTC so that both AIS and alt-AIS grades were C. Both grades remained the same at follow-up.

Post-baseline examinations

There were differences between the sacral sparing and alternative sacral sparing elements on 12 follow-up exams in 9 subjects. In six cases the difference was between S3P and DAP, and in the other six cases between VHTC and VAC. Where the sensory components differed in follow-up, five were classified as grade A by both AIS and alt-AIS methods at baseline. In follow-up, two subjects gained DAP but not S3P and three subjects gained S3P but not DAP, resulting in a difference in grading of A vs. B. The remaining subject was AIS/alt-AIS grade B at baseline but lost S3P at follow-up, resulting in alt-AIS grade A while AIS grade remained B (data not shown in tables).

Of the six subjects with differences in VAC and VHTC after 1 month, two were cervical level and four were thoracic neurologic level at baseline (Table 4). One subject with cervical level injury had a baseline lower extremity motor score (LEMS) of 8 which increased to 16 at 3 months, while VAC remained negative. The other subject with cervical level injury had no VHTC at baseline but recovered toe flexion, while VAC and DAP remained negative. Follow-up AIS grade remained A in this case, while alt-AIS grade improved to C. Interestingly, three out of the four thoracic level subjects who differed between VAC and VHTC had fractures between the levels of T12–L2, while the fourth had a gunshot wound at T10–11. All four had absent VAC but positive VHTC. Three subjects had only hip adduction but not toe flexion. The fourth had both hip adduction and toe flexion, no VAC but did have DAP. The subject’s alt-AIS grade was D. Although LEMS at follow-up was 38 because the single neurologic level was L3, the only muscle more than three levels below motor level was VAC. Since this was absent, the subject’s AIS grade was classified as B.

Table 4. Cases where VAC and VHTC differed after baseline.
Case Exam time SNL AIS alt-AIS S3P DAP VAC VHTC LEMS Notes
1 1 month C2 C C Y Y N Y 8  
1 3 months C4 C C Y Y N Y 16  
8 1 month C4 A A N N N N 0  
8 12 months C4 A C N N N Y 0  
9 1 month T12 B B Y Y N N 0 L2 fracture
9 6 months T12 A C N N N Ya 2  
10 1 month T10 A A N N N N 0 T12–L1 fracture
10 6 months T12 A C N N N Y 2  
11 1 month T11 C C Y Y Y Ya 23 T12 fracture
11 12 months L3 B D Y Y N Y 38  
12 1 month T5 A A N N N N 0 T10–11 gunshot wound
12 12 months T11 C C Y Y N Ya 4  

aOnly hip adductors present, not toe flexors.

SNL, single neurologic level; AIS, ASIA Impairment Scale; alt-AIS, alternative AIS; S3P, pressure over the S3 dermatome; DAP, deep anal pressure; VAC, voluntary anal contraction; VHTC, voluntary hip adductor and toe flexor muscle contraction; LEMS, lower extremity motor score.

Discussion

We found excellent test–retest reliability in this cohort for all sacral sparing and alternative sacral sparing components tested. This is consistent with a prior study on test–retest reliability of S3P, where a kappa of 0.98 was found.3 Test–retest reliability for DAP and VAC was shown to be high in children over age 11 with chronic SCI, with ICC values of 0.94-0.95 for DAP and 0.88-0.93 for VAC.9

Agreement between S3P and DAP and between VHTC and VAC was high at all time points. The agreement between S3P and DAP was similar to that between any S3 light touch or pinprick sensation and DAP observed by Zariffa et al.2 They found that S3 sensation was the same as DAP in 91.1% of examinations, although agreement dropped to 86.9% in persons with lumbar neurologic levels. Agreement of VHTC with VAC (94.1%) was higher than an agreement between S1 motor and VAC (85.8% at baseline exam) found by Zariffa et al.2 This may have been due to our testing toe flexors instead of ankle plantar flexors and including hip adductors.

For the sensory components, the discrepancies were roughly equal between having DAP but not S3P and the opposite. For the most part, the effect of differences on AIS grade classification was minimal, resulting in AIS/alt-AIS classifications of A vs. B. The one exception was a person who developed a cervical cord syrinx and whose sensory level ascended to C2. Because motor function in the upper extremities was present through C7 and DAP was preserved, the AIS grade was C even though LEMS was 0 and both VAC and VHTC were absent.

For the motor sacral sparing components, while there were few discrepancies in the agreement between VHTC and VAC, in all these cases VHTC was present but VAC was not. At the 1-month baseline exam, the presence of VHTC correctly classified all motor incomplete subjects with a LEMS >0, while VAC was absent in three of these. The exceptions were two subjects with sensory sacral sparing, cervical level SCI and motor sparing confined to the upper extremities. Both VHTC and VAC were absent in these two cases. If the intent of designating someone as motor incomplete is to indicate that nerve transmission to muscles is present caudal to the site of the injured spinal cord, then the “>3 motor level” criterion for a motor incomplete may be too lenient. This is particularly true in cases where there is a high cervical sensory level that defines the motor level, and motor function is confined to the upper limbs. An example is case #5 (Table 3) whose sensory level (and therefore motor level) ascended to C2 due to a spinal cord syrinx. This subject converted from AIS grade B to AIS grade C despite neurologic deterioration because motor function extended caudally to C7. Use of VHTC or lower extremity motor function as a criterion for motor incomplete could potentially avoid this problem, at least for individuals with cervical and high thoracic lesions. Additional considerations are required for individuals with low thoracic or lumbar lesions, as discussed below.

At follow-up the difference between VHTC and VAC did impact AIS grade in some cases. This occurred only in subjects with a neurologic level below T10 at the time of the examination. A review of the cases found that these subjects had fractures between T11 and L2, and likely had conus medullaris injuries. At the conus, a highly organized overlapping pattern of nerve roots is present. Specifically, an oblique layering pattern is seen for nerve roots from L2 to L5 which overlaps the terminal spinal cord, with the most rostral roots lying more laterally. Typically lumbar nerve roots adjacent to the spinal cord proper are seen starting caudal to the T10–T11 disc level.10 This may result in the lumbar roots being spared or partially spared in a conus medullaris spinal cord injury, a phenomenon known as “root escape”. As support for this theory of root escape, in three of four cases VHTC was based on the presence of hip adductor (L2–L3 roots) contraction, with absence of toe flexor (S1–S2 roots) contractions. There was one major difference in classification where AIS grade was B but alt-AIS grade was D. This subject had an L3 neurologic level, sensory sacral sparing, no VAC but positive VHTC. Even though LEMS was 38, the AIS grade was B because the anal sphincter is the only muscle more than three levels below the motor level so the criteria for motor incomplete could not be met.

The “sacral sparing” definition of incomplete SCI is based on work by Waters et al.11 These researchers defined motor sparing as motor function in the VAC or the toe flexor (TF) muscles (italics added). Sensory sacral sparing was defined as “presence of sensation in the perineum at the anal mucocutaneous junction, glans penis or clitoris”.11 The article did not describe which component(s) were present in persons classified as incomplete by sacral sparing. The International Standards Committee of ASIA limited testing for sacral sparing to S4-5 light touch and pinprick, DAP and VAC. The reason for the difference from the Waters et al.11 definition was not reported. The results of this study support the use of pressure sensation at S3 for classification as sensory incomplete, and the use of hip adductors and toe flexors for classification of motor incomplete in injuries above the conus. Whether to replace DAP with S3P or add S3P as an indicator of sensory sacral sparing will require further research and discussion. The results are not as clear for replacing VAC with VHTC. While the use of the toe flexors is consistent with the original description of sacral sparing, the inclusion of the hip adductors increases the chances for root escape causing false positive motor incomplete designations in conus injuries.

There may be some types of cases where the use of the S3P and VHTC may be of particular value. For examples, these less intrusive tests may be helpful when a patient refuses the anal exam, and in cases where conveying the concept of DAP is problematic such as a patient with altered mental status, or when language is a barrier and a translator is not available. Moreover, the S3P and VHTC may be valuable alternatives in the acute assessment of persons with polytrauma, especially when the pelvis is involved. Finally, it may be beneficial to include both DAP, S3P, VAC and VHTC in the evaluation of potential conus lesions. Additional work is needed to determine how these variables may contribute to the identification of subtypes in conus, supraconus and cauda equina injuries, and how the information they add may advance our ability to predict future outcomes.

Limitations

The study had a small sample size, with an uneven distribution among AIS grade classifications at baseline, although nearly half the sample was incomplete (AIS grade B or C). Also, at baseline only 13 subjects had motor incomplete injuries, and a larger number of cases where VHTC and VAC differ is needed to confirm that VHTC is a good criterion for motor incomplete status. Examinations were conducted by research personnel who had undergone InSTeP training and participated in a training webinar. Reliability of the examination may not be as high with less well-trained examiners. Even though the retest for reliability was usually performed 1–3 days after the first test, it is possible that the retest examination was biased by results of the first examination. The majority of the re-examinations were conducted by the same examiner, but the examiner did not have original exam results available at the time of the retest. The one case where there was a disagreement in S3P occurred when different examiners performed the test. A longer time interval between exams was considered, but because subjects were only 1-month post-injury, a longer time interval may have resulted in a true change in the examination, which would falsely lower reliability. The study did not include children; it would be useful to know the lower age limit where reliable S3P and VHTC can be obtained, and how this compares to the reliability of DAP and VAC in children.9

Conclusion

We found all sacral examination elements: DAP, S3P, VHTC and VAC to be highly reliable. Agreement between the two sensory sacral elements and the two motor elements was high. S3P seems to be a reasonable alternative to DAP, and VHTC a reasonable alternative to VAC in cases where the neurologic level is above T11. In conus injuries, VHTC may be present due to root escape despite a complete or motor complete injury to the conus.

Disclaimer statements

Contributors None.

Funding This work was supported by the National Institute on Disability, Independent Living, and Rehabilitation Research [grant numbers 90SI5012, 90SI5011, 90SI5007, 90SI5004, 90SI5002]. NIDILRR is a Center within the Administration for Community Living (ACL), Department of Health and Human Services (HHS). The contents of this report do not necessarily represent the policy of NIDILRR, ACL, HHS, and you should not assume endorsement by the Federal Government.

Conflicts of interest The authors have no conflicts of interest to report.

References

  • 1.Wietek BM, Baron CH, Erb M, Hinninghofen H, Badtke A, Kaps HP, et al. Cortical processing of residual ano-rectal sensation in patients with spinal cord injury: an fMRI study. Neurogastroenterol Motil. 2008;20(5):488–97. doi: 10.1111/j.1365-2982.2007.01063.x [DOI] [PubMed] [Google Scholar]
  • 2.Zariffa J, Kramer JLK, Jones LAT, Lammertse DP, Curt A, Steeves JD.. Sacral sparing in SCI: Beyond the S4-S5 and anorectal examination. Spine J. 2012;12(5):389–400. doi: 10.1016/j.spinee.2012.03.028 [DOI] [PubMed] [Google Scholar]
  • 3.Marino RJ, Schmidt-Read M, Kirshblum SC, Dyson-Hudson TA, Tansey K, Morse LR, et al. Reliability and validity of S3 pressure sensation as an alternative to deep anal pressure in neurologic classification of persons with spinal cord injury. Arch Phys Med Rehabil. 2016;97(10):1642–6. doi: 10.1016/j.apmr.2016.02.006 [DOI] [PubMed] [Google Scholar]
  • 4.Kirshblum SC, Botticello AL, Dyson-Hudson TA, Byrne R, Marino RJ, Lammertse DP.. Patterns of sacral sparing components on neurologic recovery in newly injured persons with traumatic spinal cord injury. Arch Phys Med Rehabil. 2016;97(10):1647–55. doi: 10.1016/j.apmr.2016.02.012 [DOI] [PubMed] [Google Scholar]
  • 5.American Spinal Injury Association . International Standards Training E Program. Richmond, VA: ASIA. [cited 2019 January 17]. Available from: https://asia-spinalinjury.org/learning/.
  • 6.Feinstein AR, Cicchetti DV.. High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol. 1990;43(6):543–9. doi: 10.1016/0895-4356(90)90158-L [DOI] [PubMed] [Google Scholar]
  • 7.Byrt T, Bishop J, Carlin JB.. Bias, prevalence and kappa. J Clin Epidemiol. 1993;46(5):423–9. doi: 10.1016/0895-4356(93)90018-V [DOI] [PubMed] [Google Scholar]
  • 8.Landis JR, Koch GG.. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74. doi: 10.2307/2529310 [DOI] [PubMed] [Google Scholar]
  • 9.Mulcahey MJ, Gaughan JP, Chafetz RS, Vogel LC, Samdani AF, Betz RR.. Interrater reliability of the international standards for neurological classification of spinal cord injury in youths with chronic spinal cord injury. Arch Phys Med Rehabil. 2011;92(8):1264–9. doi: 10.1016/j.apmr.2011.03.003 [DOI] [PubMed] [Google Scholar]
  • 10.Wall EJ, S. CM, Abitbol JJ, Garfin SR.. Organization of intrathecal nerve roots at the level of the conus medullaris. J Bone Joint Surg Am. 1990;72-A(10):1495–9. doi: 10.2106/00004623-199072100-00010 [DOI] [PubMed] [Google Scholar]
  • 11.Waters RL, Adkins RH, Yakura JS.. Definition of complete spinal cord injury. Paraplegia. 1991;29(9):573–81. [DOI] [PubMed] [Google Scholar]

Articles from The Journal of Spinal Cord Medicine are provided here courtesy of Taylor & Francis

RESOURCES