Abstract
Purpose
to evaluate diagnostic value and image quality of T2-weighted Three-dimensional isotropic turbo spin-echo (SPACE) in comparison with T2-weighted two-dimensional turbo spin-echo (TSE) sequences for comprehensive evaluation of lumbar spine pathologies.
Materials and methods
Thirty-five participants with lumbar discopathy were examined on a 1.5-T MRI system with both 2D TSE and 3D SPACE sequences. Obtained images were analyzed with synedra view personal (V 17.0.0.2) software in terms of calculating image quality factors such as signal to noise ratio (SNR) and contrast to noise ratio (CNR) for selected regions of interest. In addition, images were referred to radiologists to report their pathologic indexes. The visibility of anatomical structures in the 3D and 2D sequences was qualitatively assessed by two radiologists independently. Cohen's kappa (k) and Wilcoxon signed rank test was used for the statistical analysis.
Results
In this study, the 3D SPACE T2-weighted sequence showed significant higher SNR and CNR as well as visibility in all of the regions of interest except vertebrae and intervertebral discs (p-value < 0.05). Inter-observer agreement for visibility of regions of interest was substantial and perfect (k > 0.6). Also, inter-observer and inter-method agreements for pathologic indexes were substantial and perfect for all of the pathologic indexes (k > 0.6). Inter-observer agreement for 3D SPACE sequence was higher (k = 0.793) in comparison with 2D-TSE sequence (k = 0.603). 3D SPACE sequence and its multi-planar reconstructions (MPR) scan time were less (192 s) than 2D TSE in the sagittal, axial and coronal planes (209 s).
Conclusion
3D SPACE sequence for lumbar spine MRI proved to have higher SNR, CNR, and visibility for all regions of lumbar spine except vertebrae and disc. Inter-observer and inter-method agreements for pathologic indexes between 3D SPACE and 2D TSE sequences were substantial and 3D SPACE had a higher inter-observer agreement and less scan time. Therefore, T2 weighted 3D SPACE sequence, and its MPR might be an excellent alternative for 2D TSE in sagittal, axial, and coronal planes, especially for patients with abnormal curvature of the lumbar spine.
Keywords: Isotropic, Magnetic resonance imaging, Sampling perfection with application-optimized contrast using different flip angle evolution (SPACE), Lumbar spine, Diagnosis, Image quality
1. Introduction
Magnetic resonance imaging (MRI) of the lumbar spine has been proven as an effective method in detecting disc and soft tissue pathologies, such as disc bulge, herniation, the spinal canal stenosis, intervertebral foramen, spinal cord and vertebral abnormality [1]. Conventional MRI-protocols consist of two dimensional (2D) turbo spin echo (TSE) sequences repeated in multiple planes [2]. These 2D TSE sequences validity are adversely affected by relatively thick slices and higher inter-slice gaps, especially in axial plane. Moreover, as voxels are not isotropic, and thus multi-planar reconstruction (MPR) cannot be performed without loss of image quality, several measurements are necessary to enable displaying the lumbar spine in multiple planes. Therefore comparatively long examination times that are prone to motion artifacts have to be scheduled [3,4]. For patients with scoliosis, oblique planes can be useful, but 2D TSE sequences have not high-quality oblique reconstruction. Also in patients with lordosis, ordering of slice group in an axial plane might be challenging because of saturation effects. An inter-slice gap in 2D TSE sequence might lead to loss of information among slice groups on axial imaging method [4].
Three-dimensional (3D) sequences minimize these downsides through faster imaging and thin continuous slices acquisitions. Additionally, if isotropic voxels are used, they enable MPR and evaluation in any plane following a single acquisition, thereby eliminating the need to repeat sequences with identical tissue contrast in multiple planes [3].
Many studies have been compared the diagnostic value of these sequences, and concluded that they have similar diagnostic values [1,[3], [4], [5], [6], [7], [8], [9], [10], [11], [12]]. However, comparing image quality factors, like SNR and CNR as well as visibility of anatomical structures, were not investigated in any study before. Higher SNR, and CNR lead to better differentiation of various tissues, and diagnosis of lumbar spine pathologies. The aim of this study is comparing these image quality factors and evaluation of the agreements' results of pathologic indexes between conventional 2D T2W TSE, and 3D T2W isotropic TSE (SPACE, Siemens Healthcare, Erlangen, Germany) pulse sequences on axial and sagittal planes.
2. Materials and methods
2.1. Volunteers
The study has been approved by the review board and written informed consent has been obtained from all patients. From July 2017 to January 2018, a total of Thirty-five patients (including 17 male and 18 female; age range: 28–72; mean age: 49) underwent lumbar spine MRI. Presence of discopathy was the inclusion criteria of the study. At first, patients were imaged by the two-dimensional sequences, following diagnosis of discopathy, a three-dimensional sequence was obtained as well. Exclusion criteria included artifacts and history of surgical operation in the considered region.
2.2. Examination protocol and sequences
The examination was performed on a 1.5 T MRI scanner (MAGNETOM Avanto, Siemens Healthcare, Erlangen, Germany) using the manufacture's CTL coil. All patients were examined with both 2D and 3D protocols with the same field of view. The conventional 2D-MRI protocol (T1W and T2W) was acquired first in sagittal and axial and in some cases, coronal planes. Then additional 3D-SPACE sequences in T2-W were acquired in the sagittal plane and reformatted in transverse plane. The acquisition time was totally 209 s for axial, sagittal and coronal 2D-FSE and 192 s for 3D space sequence (Table 1). If coronal sequence was imaged in addition to axial and sagittal two-dimensional T2w sequences, total two-dimensional sequences would take longer time than three-dimensional sequence to be imaged. However coronal sequence is necessary in specific cases.
Table 1.
Parameters of the imaging protocol. ms: Millisecond, mm: millimeter, s: second, TR: repetition time, TE: echo time, FOV: field of view, TA: acquisition time, ST: slice thickness, ETL: echo train length, R: reduction factor.
| Pulse sequence | TR (ms) | TE (ms) | Number of slices/slice per slab | Matrix size (pixel*pixel) |
FOV (mm) | TA (s) | ST (mm) | ETL | R |
|---|---|---|---|---|---|---|---|---|---|
| Sagittal, T2W 2D TSE | 3000 | 93 | 9 | 240*320 | 300*300 | 46 | 4 | 25 | 1 |
| Axial T2W 2D TSE | 3050 | 108 | 15 | 210*384 | 180*330 | 96 | 4 | 22 | 1 |
| Coronal T2W 2D TSE | 3020 | 96 | 12 | 240*320 | 300*300 | 67 | 4 | 22 | 1 |
| Sagittal T2W 3D SPACE | 1500 | 248 | 60 | 280*280 | 280*280 | 192 | 1 | 129 | 2 |
2.3. Image analysis
2.3.1. SNR and CNR calculation
At first, an average signal intensity of four regions on axial and sagittal 2D FSE T2W, and 3D SPACE T2W was calculated with synedra view personal (V 17.0.0.2) software for each patient, and the data were registered on the tables. The region of interest areas were 20 ± 1 mm2. These four regions include spinal cord on L1 vertebrae level, L3 vertebrae, L3-L4 intervertebral disc, and CSF in level of L3 vertebrae (Fig. 1). Then the standard deviation of background signal intensity at level of L3 was measured as noise. Eventually, SNR and CNR calculated and compared via the related test.
Fig. 1.
regions of interest for SNR calculation on sagittal T2W 2D FSE (a) and 3D SPACE (b) sequence images.
2.3.2. Pathologic indexes and visibility interpretation
Each set of MR images – 2D conventional sequences as well as 3D SPACE sequences – were independently analyzed by two radiologists, one with five years' experience in musculoskeletal radiology and the other with three years' experience in that field. All images were reviewed in a randomized order to prevent a systematic bias. First, images from the conventional 2D TSE sequences were analyzed, two months later the 3D SPACE images were reviewed to prevent a recall bias.
3D-SPACE and 2D TSE T2 weighted MR images were reconstructed and evaluated by two radiologists using synedra view personal (V 17.0.0.2) software. This software has no advantage over Siemens (Syngo) software, the reason for using the software was its accessibility for the radiologists. For each study, the 3D SPACE and 2D TSE images were evaluated using specific criteria for stenosis, herniation and degenerative changes (47 data point per study) for the L1 to L5 level. The pathologic indexes and corresponding severity scores were as follows (Table 2). Visibility of anatomical structures in each sequence was assessed quantitatively using a five-point confidence scale: 1 = not visible; 2 = barely visible; 3 = adequately visible; 4 = good visibility; and 5 = excellent visibility. The structures evaluated include the spinal cord, cerebrospinal fluid (CSF), vertebrae, intervertebral disc, and nerve roots.
Table 2.
Pathologic indexes of the lumbar spine for radiologist interpretation.
| Pathologic index | severity | description |
|---|---|---|
| Sagittal images | ||
| Disc space hydration |
L1-2= L2-3= L3-4= L4-5= L5-S1= |
Normal : 0 Partially reduced: 1 Black disc: 2 |
| Disc space height |
L1= L2= L3= L4= L5= |
Normal: 0 Mild: reduced < 50% : 1 Moderate/severe: reduced>50%: 2 |
| Disc herniation |
L1= L2= L3= L4= L5= |
Absent: 0 Present, but no compression of the canal or diffuse disc bulge: 1 Present with mild compression of the canal (<50%) : 2 Present with marked compression of the canal (>50%) : 3 |
| Transitional vertebrae | No: 0 Yes: 1 |
|
| Endplate changes |
L1= L2= L3= L4= L5= |
Normal : 0 Present : 1 |
| Spondylolisthesis |
L1= L2= L3= L4= L5= |
Normal : 0 Anterolisthesis: 1 Retrolisthesis: 2 |
| Spinal cord signal change | Normal : 0 Present : 1 |
|
| Axial images | ||
| Central stenosis |
L1= L2= L3= L4= L5= |
Normal : 0 Mild: some CSF space loss but CSF still present around cord: 1 Moderate: no or thin layer of CSF visible around chord: 2 Severe: loss of CSF with cord deformation > 25% : 3 |
| Foraminal stenosis |
L1= L2= L3= L4= L5= |
Normal : 0 Mild: reduced <50% : 1 Moderate/severe: reduced > 50% : 2 |
| Disc herniation |
L1= L2= L3= L4= L5= |
Absent : 0 Present, but no compression of the canal or diffuse disc bulge: 1 Present with mild compression of the canal (<50%): 2 Present with marked compression of the canal (>50%): 3 |
| Facet joint (assessment of degeneration) |
L1= L2= L3= L4= L5= |
Absent: 0 Present: 1 |
2.4. Math
The following equations were used for calculation of SNR and CNR [13]:
2.5. Statistical analysis
Differences between SNR and CNR values of each sequence and the visibility of anatomical structures were evaluated in vivo and were tested for statistical significance by the Wilcoxon signed-ranks test. A p-value of less than 0.05 was regarded as statistically significant. Inter-observer agreement of the qualitative in vivo evaluation of pathologic indexes and visibility of anatomical structures was measured using Cohen’s kappa correlation coefficient. Inter-observer agreement was considered less than chance agreement when k was less than 0 (k < 0), slight agreement when 0.01 < k < 0.20, fair agreement when 0.21 < k < 0.40, moderate agreement when 0.41 < k < 0.60, substantial agreement when 0.61 < k < 0.80, and almost perfect agreement when 0.81 < k < 0.99 [14]. All statistical calculations were performed using SPSS version 14.0 (SPSS, Chicago, IL, USA).
3. Results
3.1. SNR and CNR calculation
The higher average of SNR and CNR was obtained in 3D SPACE sequence in all anatomical structures, though the difference between SNR values in CSF and spinal cord, CNR values between CSF and spinal cord and CSF and intervertebral disc was significant. Average SNR and CNR and the p-value can be seen in Tables 3, 4 .
Table 3.
Average SNR and the p-value of anatomical structures for 2D-TSE and 3D SPACE axial and sagittal sequences images.
| CSF | Spinal cord | Vertebrae | Intervertebral disc | |
|---|---|---|---|---|
| Average SNR of axial 2D-TSE | 55.7 | 25.5 | 20 | 6.2 |
| Average SNR of axial 2D-TSE | 219.2 | 39.6 | 65.2 | 14.2 |
| Average SNR of axial 2D-TSE | 103.2 | 34.2 | 39.5 | 8.6 |
| Average SNR of axial 2D-TSE | 262.7 | 45.6 | 61.4 | 16.5 |
| the p-value of axial images | 0.001 | 0.018 | 0.003 | 0.001 |
| the p-value of sagittal images | 0.003 | 0.003 | 0.001 | 0.001 |
Table 4.
Average CNR and p-value of anatomical structures for 2D-TSE and 3D SPACE axial and sagittal sequences images.
| CNR of Axial images | CNR of Sagittal images | the p-value of axial images | the p-value of sagittal images | |
|---|---|---|---|---|
| Disc and vertebrae on 2D-TSE | – | 7.7 | – | 0.064 |
| Disc and vertebrae on 3D SPACE | – | 33.2 | ||
| CSF and spinal cord on 2D-TSE | 4.2 | 14.7 | 0.001 | 0.025 |
| CSF and spinal cord on 3D SPACE | 121.6 | 166.8 | ||
| CSF and disc on 2D-TSE | 6.4 | 21.5 | 0.001 | 0.001 |
| CSF and disc on 3D SPACE | 141.7 | 188.9 |
3.2. Visibility evaluation
A qualitative evaluation revealed significantly improved visibility of intra-spinal nerve roots, CSF and spinal cord (p-value = 0.001) on 3D T2-weighted SPACE sequences compared with 2D T2-weighted TSE sequences. No significant difference was found in the visibility of intervertebral disc and vertebrae between these two sequences (Table 5). Also, the inter-observer agreement was substantial and perfect for 2D-TSE and 3D SPACE in different anatomical structures.
Table 5.
Average visibility, p-value and inter-observer agreement (k) for 2D TSE and 3D SPACE sequences.
| CSF | Spinal cord | vertebrae | disc | Nerve root | |
|---|---|---|---|---|---|
| Average visibility on 2D-TSE | 3.46 | 3.79 | 4.09 | 4.27 | 3.20 |
| Average visibility on 3D SPACE | 4.40 | 4.30 | 4.07 | 4.19 | 4.34 |
| p-value | 0.001 | 0.001 | 0.886 | 0.415 | 0.001 |
| Inter-observer agreement for 2D-TSE | 0.867 | 0.795 | 0.910 | 0.665 | 0.896 |
| Inter-observer agreement for 3D SPACE | 0.680 | 0.853 | 0.870 | 0.955 | 0.630 |
3.3. Pathologic indexes evaluation
Inter-observer agreement in scoring pathologic indexes of the evaluated structures was substantial for 2D TSE and 3D SPACE (k = 0.603 and k = 0.733). Also, Inter-method agreement in scoring pathologic indexes of the evaluated structures was substantial in view of radiologists in 2D-TSE and 3D SPACE (k = 0.679 for first and k = 0.896 for the second radiologist) (Table 6).
Table 6.
k- coefficient showing agreement between 2 sequences and two radiologists.
| k- coefficient | |
|---|---|
| Inter-observer agreement for 2D TSE | 0.603 |
| Inter-observer agreement for 3D SPACE | 0.733 |
| Inter-method agreement for the first radiologist | 0.679 |
| Inter-method agreement for the second radiologist | 0.896 |
4. Discussion
Musculoskeletal MRI using 3D sequences and their multi-planar reconstruction has become clinically feasible [1,4,6,12,15], since they provide three main advantages: first the reduction of partial volume artifacts due to the acquisition of thin continuous slices [16], second, 3D sequences can be used to create MPRs, which allow for the evaluation of the spine in any orientation with only a single data acquisition for every imaging contrast, and third, 3D sequences provide gapless imaging and no data is missed between slices. Therefore principally they can reduce the total examination time in some cases. Although the acquisition time of each 3D sequence is longer than a 2D sequence, due to the ability to reconstruct images in an arbitrary plane for special purposes, time can be saved [4].
The current protocol for lumbar spine MRI includes T2W 2D-TSE sequences that have several limitations: First, in complex anatomies like lordosis and scoliosis conditions, oblique planes in special orientation need to be obtained to see a specific structure that otherwise it's hard to see. In this case, more time has to be spend for imaging different planes. Second, due to limitation of choosing slice thickness in 2D TSE images and a greater inter-slice gaps, some of the small structures such as nerve root compressions that can be the main reason of low back pain or even some lesions may be ignored in these images (Fig. 2).
Fig. 2.
Advantages of gapless imaging with 3D SPACE sequence and MPR. “A” image, shows a mid-sagittal slice that axial T2W images will planned on this image. “B” and “C” images show the location of axial slices of 2D and 3D sequences respectively on the sagittal images. The arrow in the “C” image shows disc herniation fragment that moves lower and places at the back of L5 vertebrae. In 2D FSE this information may be missed due to an inter-slice gap between axial slices (E).
In our study, we evaluated image quality and diagnostic similarity of 2D TSE and 3D SPACE sequences by SNR, CNR, visibility, and agreements among pathologic indexes of the lumbar spine results. If the diagnostic value and image quality of the two sequences are significantly similar, using 3D space for routine lumbar spine MRI protocol will be suggested.
First SNR and CNR of images of two sequences were evaluated in different anatomical structures. SNR and CNR were significantly higher in all anatomical structures in 3D SPACE sequence, however in inter-vertebral disc and vertebrae, SNR in both sequences were similar. CNR between intervertebral disc, CSF and spinal cord was substantial due to a significant difference between the SNR of these structures.
In parallel evaluation, visibility of these structures were assessed qualitatively by two radiologists. Visibility of these structures in 3D SPACE sequence was significantly higher except inter-vertebral disc and vertebrae that may be due to significantly higher SNR, CNR, thinner slices and gapless imaging for the spinal cord, CSF, nerve roots and similar SNR and CNR for inter-vertebral disc and vertebrae (Fig. 3). Inter-observer agreement of visibility was substantial and perfect.
Fig. 3.
Higher visibility of CSF and nerve roots in 3D SPACE image (a) in comparison 2D-TSE sequence image (b). Arrows show nerve roots.
Finally, the inter-method agreement for two sequences for 47 pathologic indexes qualitatively evaluated and this agreement for first and second radiologists was substantial and perfect respectively (k = 0.679 and 0.896). Lack of inter-method agreement may be due to the difference in image quality (SNR, CNR, and visibility) of two sequence and some indexes that placed on the borderline. Furthermore, the inter-observer agreement for 3D SPACE sequence was higher (k = 0.733) in comparison with 2D-TSE (k = 0.603) that may be due to the higher image quality of 3D SPACE and easier diagnosis of pathological indexes. Also, lack of perfect inter-observer agreement may be due to the qualitative assessment of pathologic indexes and dependent on the style of radiologists report.
The Limitation of this study was the lack of a gold standard method for calculation of sensitivity, specificity, and accuracy of both sequences. Due to the higher image quality, it’s possible that 3D SPACE sequence has higher diagnostic value. Although the main purpose of this study was proving of agreement between the diagnostic values of two sequences.
5. Conclusion
-
•
In summary, 3D SPACE sequence has significantly higher SNR, CNR, and visibility in all anatomical structures except inter-vertebral disc and vertebrae in comparison with 2D-TSE. Inter-observer and the inter-method agreement was substantial and perfect for two sequences (k>0.6).
-
•
Considering the results of this study, shorter scan time of 3D SPACE sequence, and its multi-planar reconstructions (192 s) compared to 2D imaging in different planes (209 s), T2 weighted 3D SPACE sequence may be a good alternative and it is recommended in routine MRI study of lumbosacral region, especially for patients with abnormal curvatures of lumbar spine.
Declarations of interest
None.
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Conflict of interest
None.
Contributor Information
Jomleh Hossein, Email: hoseinjomle@sbmu.ac.ir.
Faeghi Fariborz, Email: f_faeghi@sbmu.ac.ir.
Rasteh Mehrnaz, Email: mehrsa_mr@yahoo.com.
Rafiei Babak, Email: babakraf@yahoo.com.
References
- 1.Sung J., Jee W.-H., Jung J.-Y., Jang J., Kim J.-S., Kim Y.-H. Diagnosis of nerve root compromise of the lumbar spine: evaluation of the performance of three-dimensional isotropic T2-weighted turbo spin-echo SPACE sequence at 3T. Korean J. Radiol. 2017;18(1):249–259. doi: 10.3348/kjr.2017.18.1.249. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Elmaoğlu M., Çelik A. Springer Science & Business Media; 2011. MRI Handbook: MR Physics, Patient Positioning, and Protocols. [Google Scholar]
- 3.Kloth J.K., Winterstein M., Akbar M., Meyer E., Paul D., Kauczor H.U. Comparison of 3D turbo spin-echo SPACE sequences with conventional 2D MRI sequences to assess the shoulder joint. Eur. J. Radiol. 2014;83(10):1843–1849. doi: 10.1016/j.ejrad.2014.06.011. [DOI] [PubMed] [Google Scholar]
- 4.Blizzard D.J., Haims A.H., Lischuk A.W., Arunakul R., Hustedt J.W., Grauer J.N. 3D-FSE isotropic MRI of the lumbar spine: novel application of an existing technology. J. Spinal Disord. Tech. 2015;28(4):152–157. doi: 10.1097/BSD.0b013e31827a32ee. [DOI] [PubMed] [Google Scholar]
- 5.Crema M.D., Nogueira-Barbosa M.H., Roemer F.W., Marra M.D., Niu J., Chagas-Neto F.A. Three-dimensional turbo spin-echo magnetic resonance imaging (MRI) and semiquantitative assessment of knee osteoarthritis: comparison with two-dimensional routine MRI. Osteoarthr. Cartil. 2013;21(3):428–433. doi: 10.1016/j.joca.2012.12.011. [DOI] [PubMed] [Google Scholar]
- 6.Fu M.C., Buerba R.A., Neway W.E., 3rd, Brown J.E., Trivedi M., Lischuk A.W. Three-dimensional isotropic MRI of the cervical spine: a diagnostic comparison with conventional MRI. Clin. Spine Surg. 2016;29(2):66–71. doi: 10.1097/BSD.0b013e3182a355e5. [DOI] [PubMed] [Google Scholar]
- 7.Jung J.Y., Yoon Y.C., Jung J.Y., Choe B.K. Qualitative and quantitative assessment of wrist MRI at 3.0T: comparison between isotropic 3D turbo spin echo and isotropic 3D fast field echo and 2D turbo spin echo. Acta Radiologica (Stockholm, Sweden: 1987) 2013;54(3):284–291. doi: 10.1258/ar.2012.120475. [DOI] [PubMed] [Google Scholar]
- 8.Kijowski R., Davis K.W., Woods M.A., Lindstrom M.J., De Smet A.A., Gold G.E. Knee joint: comprehensive assessment with 3D isotropic resolution fast spin-echo MR imaging--diagnostic performance compared with that of conventional MR imaging at 3.0 T. Radiology. 2009;252(2):486–495. doi: 10.1148/radiol.2523090028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Park H.J., Lee S.Y., Park N.H., Rho M.H., Chung E.C., Park J.H. Three-dimensional isotropic T2-weighted fast spin-echo (VISTA) ankle MRI versus two-dimensional fast spin-echo T2-weighted sequences for the evaluation of anterior talofibular ligament injury. Clin. Radiol. 2016;71(4):349–355. doi: 10.1016/j.crad.2015.12.007. [DOI] [PubMed] [Google Scholar]
- 10.Rosse C., Gaddum-Rosse P. The vertebral canal, spinal cord, spinal nerves, and segmental innervation. In: Rosse C., Gaddum Rosse P., editors. Hollinshead’s Textbook of Anatomy. 1997. p. 5. [Google Scholar]
- 11.Stevens K.J., Busse R.F., Han E., Brau A.C., Beatty P.J., Beaulieu C.F. Ankle: isotropic MR imaging with 3D-FSE-cube--initial experience in healthy volunteers. Radiology. 2008;249(3):1026–1033. doi: 10.1148/radiol.2493080227. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Lee S., Jee W.H., Jung J.Y., Lee S.Y., Ryu K.S., Ha K.Y. MRI of the lumbar spine: comparison of 3D isotropic turbo spin-echo SPACE sequence versus conventional 2D sequences at 3.0 T. Acta Radiologica (Stockholm, Sweden: 1987) 2015;56(2):174–181. doi: 10.1177/0284185114524196. [DOI] [PubMed] [Google Scholar]
- 13.Reeder S.B. Springer; 2007. Measurement of Signal-to-noise Ratio and Parallel Imaging. Parallel Imaging in Clinical MR Applications; pp. 49–61. [Google Scholar]
- 14.Viera A.J., Garrett J.M. Understanding interobserver agreement: the kappa statistic. Fam. Med. 2005;37(5):360–363. [PubMed] [Google Scholar]
- 15.Meindl T., Wirth S., Weckbach S., Dietrich O., Reiser M., Schoenberg S.O. Magnetic resonance imaging of the cervical spine: comparison of 2D T2-weighted turbo spin echo, 2D T2*weighted gradient-recalled echo and 3D T2-weighted variable flip-angle turbo spin echo sequences. Eur. Radiol. 2009;19(3):713–721. doi: 10.1007/s00330-008-1175-7. [DOI] [PubMed] [Google Scholar]
- 16.Kijowski R., Gold G.E. Routine 3D magnetic resonance imaging of joints. J. Magn. Reson. Imaging. 2011;33(4):758–771. doi: 10.1002/jmri.22342. [DOI] [PMC free article] [PubMed] [Google Scholar]