Abstract
Background
The manner of measuring the outcome of cervical myelopathy must be patient-oriented and have sufficient reliability and validity. The current Japanese Orthopaedic Association (JOA) scoring system for cervical myelopathy is widely used but has not met this requirement. The first- and second-round surveys established 24 items for inclusion on a new questionnaire for cervical myelopathy. The purpose of this study (the third-round survey A) was to confirm the reproducibility of patient responses to the selected questions.
Methods
A total of 201 patients with cervical myelopathy and with no change of symptoms between the two interviews were included. Each patient was interviewed twice using the same questionnaire at an interval of 4 weeks. The reliability of the questionnaire was evaluated by determining the extension of the weighted kappa coefficients.
Results
The weighted kappa coefficient for each item was >0.4, confirming that the test–retest reliability was acceptable.
Conclusions
The newly developed JOA Cervical Myelopathy Evaluation Questionnaire was proven to have sufficient reliability.
Introduction
As described in Part 11 and Part 2,2 the Japanese Orthopaedic Association (JOA) decided to revise the assessment tools for cervical myelopathy and develop a new JOA Cervical Myelopathy Evaluation Questionnaire (JOACMEQ). The point of the revision was to make it: (1) more scientific from the standpoint of medical statistics, and (2) more patient-oriented. For the first- and second-round surveys, the committee created a new questionnaire of 77 items including SF-36 (the Medical Outcome Study Short-Form 36-Item Health Survey)3 to assess health-related quality of life (HRQOL) and 41 items to assess functioning of the cervical spinal cord and cervical spine. In these surveys, 53 items were excluded.1,2 The purpose of this study (the third-round survey A) was to confirm the reproducibility of the selected 24 questions written in Japanese (see Appendix). The questionnaire was self-administered, so interobserver reliability did not need to be confirmed. To examine the reliability of the questionnaire, only test-retest reliability was evaluated.
Materials and methods
Inclusion criteria were as follows: (1) Patient had to have cervical myelopathy secondary to cervical disc herniation, cervical spondylosis, or ossification of the posterior longitudinal ligament (OPLL); and (2) there was no change of symptoms between the two interviews, which was judged by attending surgeons. Exclusion criteria included (1) patient had other musculoskeletal disease requiring medical treatment; (2) patient had a mental disease and so could potentially furnish inappropriate answers; (3) patient was postoperative; 4) patient had participated in the former surveys.
We asked 460 Japanese board-certified spine surgeons to participate in the survey between January 2004 and June 2004. Patients were recruited from the outpatient clinic of each surgeon during the planned inquiry period. Self-assessment was used in this study. Each patient was interviewed using the same questionnaire twice at an interval of 4 weeks (±3 days). The attending surgeon filled out the patient’s information regarding the diagnosis and the presence or absence of concomitant diseases and judged the severity using a three-step rating scale (mild, moderate, severe). This study was approved by the Ethics Committee of the Japanese Society for Spine Surgery and Related Research, and informed consent was obtained from each patient.
The reliability of the questionnaire was evaluated by determining the extension of the weighted kappa coefficients. According to the most widely accepted criteria proposed by Landis and Koch, a kappa coefficient of ≥0.4 was judged reliable.4 The 95% confidence intervals were calculated for all reliability coefficients.
All statistics were calculated using SPSS software (version 12; SPSS, Chicago, IL, USA).
Results
Of 304 patients who participated in this survey, 103 were excluded owing to a change of symptoms between the two interviews or an inappropriate patient administration interval, leaving 201 available for the study (74 women, 127 men). The mean ± SD patient age was 63.8 ±12.8 years (Table 1). The diagnosis was cervical disc herniation in 29, cervical spondylotic myelopathy in 113, and OPLL/other in 59. Table 2 lists the severity of patient myelopathy evaluated by the current JOA scoring system and shows that the characteristics of the recruited patients were not specific. The distribution of answers for each item during the second interview is shown in Table 3, and neither ceiling nor floor effects were observed. The results for test-retest reliability are shown in Table 4. The weighted kappa coefficient for each item was >0.4, showing that the test-retest reliability of JOACMEQ was acceptable. Among the three groups with severity judged as mild, moderate, or severe, there was some difference in the weighted kappa coefficient for some items, but the difference was statistically not significant. The difference among the three diseases (cervical disc herniation, cervical spondylosis, OPLL) was also statistically not significant (data not shown).
Table 1.
Baseline participant demographics (n = 201)
| Parameter | Male | Female | Total |
|---|---|---|---|
| No. of patients | 127 | 74 | 201 |
| Age (years) | |||
| 30s | 7 | 4 | 11 |
| 40s | 12 | 8 | 20 |
| 50s | 26 | 13 | 39 |
| 60s | 41 | 13 | 54 |
| 70s | 32 | 27 | 59 |
| 80s | 9 | 9 | 18 |
| Average | 63 | 65.2 | 63.8 |
| SD | 12.4 | 13.5 | 12.8 |
| Minimum | 30 | 34 | 30 |
| Maximum | 89 | 84 | 89 |
| Median | 64 | 69 | 66 |
| Diagnosis | |||
| Cervical disc herniation | 17 | 12 | 29 |
| Cervical spondylosis | 68 | 45 | 113 |
| OPLL | 39 | 16 | 55 |
| Other | 3 | 1 | 4 |
OPLL, ossification of the posterior longitudinal ligament
Table 2.
Distribution of the severity evaluated by the current JOA scoring system
| Score | No. |
|---|---|
| Motor function | |
| Upper extremity | |
| 0 | 1 |
| 1 | 10 |
| 2 | 37 |
| 3 | 90 |
| 4 | 63 |
| Lower extremity | |
| 0 | 1 |
| 0.5 | 0 |
| 1 | 21 |
| 1.5 | 22 |
| 2 | 40 |
| 2.5 | 20 |
| 3 | 40 |
| 4 | 57 |
| Sensory function | |
| Upper extremity | |
| 0 | 2 |
| 0.5 | 22 |
| 1 | 88 |
| 1.5 | 61 |
| 2 | 28 |
| Trunk | |
| 0 | 1 |
| 0.5 | 1 |
| 1 | 14 |
| 1.5 | 26 |
| 2 | 159 |
| Lower extremity | |
| 0 | 1 |
| 0.5 | 15 |
| 1 | 55 |
| 1.5 | 44 |
| 2 | 86 |
| Bladder functiona | |
| 0 | 4 |
| 1 | 17 |
| 2 | 52 |
| 3 | 127 |
JOA, Japanese Orthopaedic Association
a Data defect: n = 1
Table 3.
Distribution of answers for each item in the second interview
| Item | 1 | 2 | 3 | 4 | 5 | No answer |
| C01 | 108 | 80 | 13 | |||
| 53.70% | 39.80% | 6.50% | ||||
| C02 | 155 | 41 | 5 | |||
| 77.10% | 20.40% | 2.50% | ||||
| C03 | 124 | 53 | 17 | 6 | 1 | |
| 61.70% | 26.40% | 8.50% | 3.00% | 0.50% | ||
| C04 | 105 | 61 | 23 | 10 | 2 | |
| 52.20% | 30.30% | 11.40% | 5.00% | 1.00% | ||
| C05 | 86 | 74 | 37 | 4 | ||
| 42.80% | 36.80% | 18.40% | 2.00% | |||
| C06 | 132 | 43 | 15 | 10 | 1 | |
| 65.70% | 21.40% | 7.50% | 5.00% | 0.50% | ||
| C07 | 65 | 103 | 33 | |||
| 32.30% | 51.20% | 16.40% | ||||
| C08 | 114 | 70 | 16 | 1 | ||
| 56.70% | 34.80% | 8.00% | 0.50% | |||
| C09 | 129 | 57 | 14 | 1 | ||
| 64.20% | 28.40% | 7.00% | 0.50% | |||
| C10 | 111 | 60 | 27 | 3 | ||
| 55.20% | 29.90% | 13.40% | 1.50% | |||
| C11 | 121 | 68 | 12 | |||
| 60.20% | 33.80% | 6.00% | ||||
| C12 | 138 | 52 | 11 | |||
| 68.70% | 25.90% | 5.50% | ||||
| C13 | 68 | 88 | 44 | 1 | ||
| 33.80% | 43.80% | 21.90% | 0.50% | |||
| QOL01 | 6 | 16 | 77 | 90 | 11 | 1 |
| 3.00% | 8.00% | 38.30% | 44.80% | 5.50% | 0.50% | |
| QOL02 | 39 | 91 | 69 | 2 | ||
| 19.40% | 45.30% | 34.30% | 1.00% | |||
| QOL03 | 36 | 97 | 66 | 2 | ||
| 17.90% | 48.30% | 32.80% | 1.00% | |||
| QOL04 | 33 | 74 | 93 | 1 | ||
| 16.40% | 36.80% | 46.30% | 0.50% | |||
| QOL05 | 16 | 29 | 91 | 44 | 20 | 1 |
| 8.00% | 14.40% | 45.30% | 21.90% | 10.00% | 0.50% | |
| QOL06 | 15 | 49 | 67 | 47 | 22 | 1 |
| 7.50% | 24.40% | 33.30% | 23.40% | 10.90% | 0.50% | |
| QOL07 | 11 | 26 | 95 | 43 | 25 | 1 |
| 5.50% | 12.90% | 47.30% | 21.40% | 12.40% | 0.50% | |
| QOL08 | 12 | 35 | 92 | 45 | 15 | 2 |
| 6.00% | 17.40% | 45.80% | 22.40% | 7.50% | 1.00% | |
| QOL09 | 17 | 46 | 102 | 30 | 5 | 1 |
| 8.50% | 22.90% | 50.70% | 14.90% | 2.50% | 0.50% | |
| QOL10 | 8 | 56 | 65 | 55 | 17 | |
| 4.00% | 27.90% | 32.30% | 27.40% | 8.50% | ||
| QOL11 | 18 | 60 | 74 | 38 | 11 | |
| 9.00% | 29.90% | 36.80% | 18.90% | 5.50% |
Table 4.
Reproducibility of each item
| Item | Item no. | κ | 95% CI |
|---|---|---|---|
| C01 | 3 | 0.76 | 0.69–0.82 |
| C02 | 3 | 0.59 | 0.50–0.67 |
| C03 | 4 | 0.71 | 0.64–0.77 |
| C04 | 5 | 0.74 | 0.69–0.79 |
| C05 | 3 | 0.75 | 0.70–0.80 |
| C06 | 5 | 0.69 | 0.63–0.76 |
| C07 | 3 | 0.80 | 0.75–0.85 |
| C08 | 3 | 0.65 | 0.58–0.71 |
| C09 | 3 | 0.68 | 0.62–0.75 |
| C10 | 3 | 0.63 | 0.57–0.69 |
| C11 | 3 | 0.72 | 0.65–0.77 |
| C12 | 3 | 0.55 | 0.47–0.62 |
| C13 | 3 | 0.71 | 0.66–0.76 |
| QOL01 | 3 | 0.57 | 0.51–0.63 |
| QOL02 | 3 | 0.68 | 0.62–0.74 |
| QOL03 | 3 | 0.64 | 0.58–0.70 |
| QOL04 | 3 | 0.73 | 0.67–0.78 |
| QOL05 | 5 | 0.58 | 0.52–0.63 |
| QOL06 | 5 | 0.55 | 0.49–0.60 |
| QOL07 | 5 | 0.62 | 0.56–0.67 |
| QOL08 | 5 | 0.56 | 0.50–0.61 |
| QOL09 | 5 | 0.56 | 0.50–0.61 |
| QOL10 | 5 | 0.62 | 0.57–0.67 |
| QOL11 | 5 | 0.49 | 0.42–0.54 |
Discussion
Measurements of the outcome of cervical myelopathy must have sufficient reliability and validity and should be proven by means of statistical evaluation. The current JOA scoring system for cervical myelopathy has been widely used but has not been shown to meet such a requirement.5,6 A manner of measuring patient-based outcome has also been regarded as essential for evaluating a patient’s health status. We sought to develop a new questionnaire using a psychometric method. The newly proposed document, which is self-administered and disease-specific, includes function of the cervical spinal cord and cervical spine as well as health-related quality of life. The first- and second-round surveys selected 24 of 77 items after repeated discussions among the subcommittee members.1,2
In this third survey, we succeeded in confirming the reliability of the 24 items. The next steps are to check the validity of the questionnaire, perform a factor analysis to determine the underlying cluster of the question naire items, complete the measurement scale, and confirm the responsiveness of the questionnaire.
Conclusion
The newly developed JOA Cervical Myelopathy Evaluation Questionnaire (JOACMEQ) was proven to be sufficiently reliable.
Footnotes
The authors are members of the Subcommittee on Low Back Pain and Cervical Myelopathy, Evaluation of the Clinical Outcome Committee of the Japanese Orthopaedic Association.
References
- 1.Fukui M, Chiba K, Kawakami M, Kikuchi S, Konno S, Miyamoto M, et al. An outcome measure for patients with cervical myelopathy: Japanese Orthopaedic Association Cervical Myelopathy Evaluation Questionnaire (JOACMEQ). Part 1. J Orthop Sci. 2007;12:227–40. doi: 10.1007/s00776-007-1118-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Fukui M, Chiba K, Kawakami M, Kikuchi S, Konno S, Miyamoto M, et al. Japanese Orthopaedic Association Cervical Myelopathy Evaluation Questionnaire (JOACMEQ). Part 2. Endorsement of the alternative item. J Orthop Sci. 2007;12:241–8. doi: 10.1007/s00776-007-1119-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Ware JE., Jr SF-36 health survey update. Spine. 2000;25:3130–9. doi: 10.1097/00007632-200012150-00008. [DOI] [PubMed] [Google Scholar]
- 4.Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74. doi: 10.2307/2529310. [DOI] [PubMed] [Google Scholar]
- 5.Hirabayashi K, Miyakawa J, Satomi K, Maruyama T, Wakano K. Operative results and postoperative progression of ossification among patients with ossification of cervical posterior longitudinal ligament. Spine. 1981;6:354–64. doi: 10.1097/00007632-198107000-00005. [DOI] [PubMed] [Google Scholar]
- 6.Yonenobu K, Abumi K, Nagata K, Taketomi E, Ueyama K. Interobserver and intraobserver reliability of the Japanese Orthopaedic Association scoring system for evaluation of cervical compression myelopathy. Spine. 2001;26:1890–5. doi: 10.1097/00007632-200109010-00014. [DOI] [PubMed] [Google Scholar]