Diagnostic accuracy and validity of three manual examination tests to identify alar ligament lesions: results of a blinded case-control study

ABSTRACT

Introduction: Tests to evaluate the integrity of the alar ligaments are important clinical tools for manual therapists, but there is limited research regarding their validity.

Method: A single blinded examiner assessed alar ligament integrity using the lateral shear test (LST), rotation stress test (RST) and side-bending stress test (SBST) on a sample of convenience comprising 7 subjects with MRI confirmed alar ligament lesions and 11 healthy people. Alar ligament lesions were identified using both supine and high-field strength upright MRI.

Results: The RST had a sensitivity of 80% and a specificity of 69.2%. The SBST and the LST both showed a sensitivity of 80% and a specificity of 76.9%. In cases where all three tests were positive, the specificity increased to 84.6%.

Discussion: Tests of manual examination of alar ligament integrity have some diagnostic utility; however, these findings require further corroboration in a larger sample.

1. Introduction

The alar ligaments are important stabilizers of the cervical spine and may become damaged following trauma, hence require careful examination, particularly in patients who present with neck-related problems with a history of injury [1,2]. The gold standard for determining alar ligament damage is MRI [3,4] for which there is a growing body of evidence supporting its use [5–7]. New technical developments in MRI such as high-fidelity 3-T MRI [8] and improved protocols [9] provide better visualization of the alar ligaments than previous MRI methods [8,10].

In the absence of MRI investigations, physiotherapists must perform specific tests to assess the integrity of alar ligaments [1,3,11,12,4] if they suspect ligament injury. The rational for these tests is on the one hand for diagnostic purposes but more so in order to ensure safety during physical examination and treatment [2,13,14,15]. As with many manual examination procedures, the validity and reliability of tests for alar ligament integrity have not been adequately investigated.

A number of tests for evaluating alar ligament integrity have been described (Figure 1) including the side-bending stress test (SBST) [1,12], rotation stress test (RST) [10] and the lateral shear test (LST) [11,12]. Reliability of the RST has been investigated with Kappa scores of 0.69–0.83, with good validity when comparing manual examination with MRI findings [3]. When this test was used as a part of a cluster of tests, sensitivity ranged between 0.69 and 0.84 and specificity between 0.91 and 1.0. The positive predictive value ranged between 0.93 and 1.0, while negative predictive value was 0.8 [3].

Figure 1.

(Continued.)

Figure 1.

(Continued.)

Figure 1.

Tests are performed for the left alar ligament. (a) Side-bending test in sitting. Evaluation of C2 rotation during upper cervical lateral flexion toward the right. The spinous process of C2 is palpated with one hand, while the upper cervical spine is laterally flexed to the right. The spinous process should move to the left. Lack of rotation would indicate a positive test; (b) rotation test in supine. The spinous process of C2 is fixed with one hand while the head is rotated to the right with the other hand. Range of rotation between the occiput and C2 should be approximately 20° to each side. Range greater than 20° indicates a positive test; (c) lateral shear test. The C2 vertebra is fixed on the left posterolateral aspect with one hand while the occiput and C1 are translated to the left with the other hand. Translation is perpendicular to the neck. Any movement would indicate a positive test.

The LST has good intrarater reliability (modified Kappa 0.67). Preliminary support for the validity of this test has been shown in a case report, correlating positive results on the LST and the SBST with X-ray imaging [12]. Further evidence of validity was provided by Osmotherly et al. [1]. In that study, the SBST had excellent inter-tester reliability while the RST inter-tester reliability was rated only as average.

The process of validating diagnostic tests has been subdivided into four phases [16]. Phases 1 and 2 investigate the ability of a test to discriminate and correctly identify subjects with and without a verified diagnosis. Phases 3 and 4 are evaluated in a clinical setting, where the diagnostic properties are evaluated in subjects who are suspected of having the disease and whether implementing the test improves clinical outcomes.

Hence, it would appear that there is preliminary evidence for the validity of the alar ligament stress tests; however, no study has investigated the reliability and validity of a battery of alar ligament stress tests in comparison to MRI. Additionally, it is unclear whether different forms of MRI yield identical results. Therefore, the purpose of this study was (a) to assess the accuracy of detecting lesions of the alar ligaments using three different individual alar ligament tests (Figure 1), (b) to evaluate a cluster of all three tests compared to two different kinds of MRI scans and (c) to evaluate agreement in detecting alar ligament lesions between two different forms of MRI scan.

2. Methods

In this single-blind case-controlled study conducted according to STARD protocols (Appendix 1), specific manual alar ligament tests were carried out on subjects with MRI determined alar ligament lesions as well as asymptomatic healthy controls. Manual examination tests were performed by a blinded, experienced manual therapist on the same day as MRI. High-field 3-T MRI scans were performed in sitting prior to the study commencement and a standard supine MRI carried out as part of the study using a specialized cervical protocol in a radiology clinic in the greater area of Darmstadt, Germany. There is evidence for validity of MRI in determining upper cervical ligament damage in that high-field strength 3-T MRI identifies a higher proportion of alar ligament lesions in people with chronic whiplash-associated disorder (WAD) when compared to a non-injured population. Furthermore, an association was found between high-grade changes in the alar ligaments determined by MRI and disability [17]. In addition, MRI studies evaluating alar ligament integrity show fair-to-moderate agreement in detecting high-signal intensities in healthy subjects [10] and moderate-to-good agreement in detecting lesions in people with chronic WAD [18]. All scans were reviewed by an experienced radiologist, specialized in examination of the craniocervical region, who was blind to the subject group allocation.

2.1. Subjects, inclusion and exclusion criteria

Subjects were recruited from November 2016 to April 2017 in a manner of convenience through advertisements seeking people with high-field strength 3-T MRI confirmed alar ligament lesions diagnosed by a radiologist associated with symptoms of upper cervical instability. Volunteers were recruited from various physiotherapy, orthopedic and radiology clinics. Subjects were required to be able to tolerate manual examination procedures used in this study and were excluded if they were unable to undergo MRI. Additional exclusion criteria included rheumatoid arthritis, Down’s syndrome, Ehlers–Danlos syndrome, Klippel–Feil syndrome and Pierre-Robin syndrome. In total, 15 people volunteered but 6 were subsequently found not to have an alar ligament lesion on MRI, and 1 had extreme dizziness in lying and another fear of manual examination preventing inclusion. Hence, seven subjects (aged 18 and 67 years; five male) were included who demonstrated an alar ligament lesion on high-field strength 3-T MRI with a history of cervical trauma.

Eleven subjects (10 male, 1 female) were included in the control group. These subjects had no complaint of neck pain in the last 3 years, nor had a previous history of WAD, or any form of headache. All subjects were checked for the inclusion and exclusion criteria by the first examiner.

2.2. Study protocol

After recruitment, by the first assessor, subjects were informed about the study protocol and informed consent obtained. The participants were informed that the blinded (second) assessor should not receive any information about their health status so that blinding could be guaranteed. The included manual tests were those most often implemented and mentioned in the literature the SBST, the RST, and the LST (Figure 1). All tests were performed on both the left and right alar ligaments. Determination of a positive test is described in Figure 1. The examiner documented the test results on a form, which was given after each subject to the first assessor. On the day of testing, each subject underwent a supine MRI which was reviewed by a radiologist specialized in imaging the head and neck region. The radiologist was not informed about the results of the clinical test results from the second assessor. The data from the radiologist’s interpretation of the MRI images were collected by the first assessor (Figure 2). The study was conducted in accordance with the Helsinki guidelines and approved by the local ethics committee of the University of Applied Sciences Osnabrück (WiSO MS-MP-WS 1617-08).

Figure 2.

A flowchart of the study.

2.3. Examiner and training of the examiner

The examiner carrying out alar ligament testing was both an experienced manual therapist (IFOMPT standard) with more than 10 years of experience and postgraduate training up to specialist level in the examination of the cervical spine. This examiner received an additional two training sessions to review the alar ligament tests.

2.4. MRI scanning

The standard MRI scan was performed in supine with a cervical coil (GE Discovery 750 3 T with a 16ch HNS coil by General Electric) as described in the literature [2,8]. Images were taken of all 11 healthy subjects as well as the 7 subjects with alar ligament damage. All scans were carried out by a radiology assistant with 9-years clinical experience. The scans were evaluated by the radiologist who was blind to the subjects group allocation. The scan protocol for MRI was specifically matched for the evaluation of the alar ligaments. Proton-weighted, coronal plane images, proton-weighted images in the transverse plane with fat saturation (fatsat), T1-weighted fast spin echo (FSE) images in the transverse plane and T2-weighted FSE images in the transverse plane were taken.

3. Statistical analysis

All collected data were analyzed by R (R core team, 2014). The sample size was based on a comparable study, where significant results were achieved with a group of 16 participants [1]. T-tests and Fisher’s exact tests were used to determine statistical differences between the group with alar ligament damage and the control group. A p-value less than 0.05 was considered as an indication of a statistically significant result. Agreement between the standard supine and high-field strength upright MRI was evaluated by Cohen’s Kappa. Sensitivity, specificity, positive/negative likelihood ratios and the Youden Index of all three tests were compared to both the high-field strength upright and standard supine MRI results through cross-classified tables. It was assumed that all subjects in the control group, who did not have a lesion in the standard supine MRI scans, would also have no lesion in the high-field strength MRI scan. Furthermore, we examined whether a combination of alar ligament manual examination tests would lead to an improvement in sensitivity and specificity. Therefore, the sum of single positive test results was presented through a receiver-operating characteristic curve. The optimal cutoff was assessed by maximizing the Youden Index.

4. Results

4.1. Group characteristics

No significant differences were found between groups regarding gender (alar ligament group: male = 5, female = 2; control group: male: 10, female 1, p = 0.52), neck circumference (alar ligament group: mean 27.4 cm [5.4], control group mean: 39.1 cm [4.3], p = 0.51) and age (alar ligament group: mean 50 [8.5], control group: mean 52.2 [8.4], p = 0.6).

The agreement between standard supine and high-field strength upright MRI is shown in Table 1. Five out of seven subjects with lesions of the alar ligaments detected by standard supine MRI were found to be positive in the high-field strength upright MRI. This corresponds to a Cohen’s Kappa of 0.75, indicating high, yet not perfect agreement. In the control group, no alar ligament lesions were found.

Table 1.

Manual testing versus standard MRI with specialized protocol.

		No alar ligament lesion	Alar ligament lesion	Specificity	Sensitivity	Positive likelihood ratio	Negative likelihood ratio	Youden Index
Lateral shear test	Intact	10	1	76.9%	80%	3.46	0.26	0.57
	Not intact	3	4
Side-bending stress test	Intact	10	1	76.9%	80%	3.46	0.26	0.57
	Not intact	3	4
Rotational stress test	Intact	9	1	69.2%	80%	2.6	0.29	0.49
	Not intact	4	4

Sensitivity, specificity, positive and negative likelihood ratios and the Youden Index of all three tests compared to the high-field strength upright MRI are depicted in Table 2. The accuracy of the LST and the SBST was identical showing a sensitivity of 80% and a specificity of 76.9%. The positive likelihood ratio was 3.46, the negative likelihood ratio was 0.26, while the Youden Index amounted to 0.57. The RST achieved a sensitivity of 80% and a specificity of 69.2%. Positive and negative likelihood ratios were 2.6 and 0.29, respectively, and the Youden Index amounted to 0.49.

Table 2.

Manual testing versus upright MRI.

		No alar ligament lesion	Alar ligament lesion	Specificity	Sensitivity	Negative likelihood ratio	Positive likelihood ratio	Youden Index
Lateral shear test	Intact	10	1	90.9%	85.7%	0.16	9.43	0.77
	Not intact	1	6
Side-bending stress test	Intact	10	1	90.9%	85.7%	0.16	9.43	0.77
	Not intact	1	6
Rotational stress test	Intact	9	1	81.8%	85.7%	0.17	4.71	0.68
	Not intact	2	6
Sum of symptomatic tests	>2	11	1	100%	85.7%	0.15	Inf	0.86
	≤2	0	6

The results of the sensitivity and specificity, positive/negative likelihood ratios and the Youden Index of all three tests compared to the standard supine MRI are depicted in Table 2. Also here, the results of LST and the SBST were identical showing a sensitivity of 85.7% and a specificity of 90.9%. The positive likelihood ratio was 9.43 and the negative 0.16, respectively, and the Youden Index amounted to 0.76. The RST achieved a sensitivity of 85.7% and a specificity of 81.8%. Positive and negative likelihood ratios were 4.71 and 0.17, respectively, and the Youden Index amounted to 0.68.

The results of the cluster of all three tests are presented in Figures 3 and 4. In comparison to standard supine MRI, the area under the curve (AUC) amounts to 93%. The optimal cutoff for detecting an alar ligament lesion was greater than two positive test results. The corresponding specificity and sensitivity were 100% and 85.7%. Positive and negative likelihood ratios were infinity and 0.15, respectively, and the Youden Index amounted to 0.86. In comparison to high-field strength supine MRI, the AUC amounted to 81%. The optimal cutoff for detecting an alar ligament lesion was greater than two positive test results. The corresponding specificity and sensitivity were 80% and 84.5%, respectively. Positive and negative likelihood ratios were 5.19 and 0.24, respectively, and the Youden Index amounted to 0.65.

Figure 3.

Receiver-operating curve of the cluster of all three alar ligament test results in comparison to the standard supine magnetic resonance imaging (with diagonal line for orientation).

Figure 4.

Receiver-operating curve of the cluster of all three alar ligament test results in comparison to high field strength upright magnetic resonance imaging (with diagonal line for orientation).

5. Discussion

In this early phase diagnostic case-control study, 18 subjects (7 with alar ligament lesions) were tested with 3 manual examination tests for alar ligament laxity and 2 different forms of MRI scan to determine alar ligament integrity.

Depending on type of MRI (standard vs. upright), the sensitivity and specificity of the clinical test ranged between 80–85.7% and 69.2–90.9%, respectively. This resulted in positive and negative likelihood ratios ranging from 2.6 to 9.41 and 0.15 to 0.26, respectively, indicating small-to-moderate clinical diagnostic value according to recommended thresholds [19,20]. The values of the Youden Index ranged between 0.49 and 0.77.

However, when using the three clinical tests as a cluster with a threshold of more than two positive test results, the sensitivity and specificity amounted to 85.7% and 100%, respectively (Youden Index of 0.86), with 17 out of 18 diagnoses classified correctly. Only one patient with alar ligament rupture was falsely diagnosed as healthy. Likelihood ratios improved to infinity (positive likelihood ratio) and 0.15 (negative likelihood ratio) indicating moderate-to-excellent clinical diagnostic value. In general, a high positive likelihood ratio increases the probability of having the disease given a positive test and is important for ‘ruling in’. It addresses to what extent a clinician can be confident about the accuracy of a positive test result. In contrast, a low negative likelihood ratio decreases the probability of having the disease given a negative test result and is important for ‘ruling out’. It addresses to what extent a clinician can be confident about the accuracy of a negative test result. In the special case of an infinity positive likelihood ratio, as found in our study, the probability of truly having an alar ligament lesion increases to 100% after being tested positive by the cluster of clinical tests. For a given negative likelihood ratio of 0.15, the probability of truly having an alar ligament lesion after being tested negatively decreases to approximately 8% in our study.

In all healthy subjects, no alar ligament lesions were found. In the seven subjects with a history of trauma who had alar ligament lesions demonstrated on high-field strength 3-T MRI undertaken prior to the study, only five were found to have the lesion on standard MRI in supine even with a specialized craniocervical protocol. Of these seven subjects with alar ligament lesions, six had all three manual examination tests positive. All subjects in this group complained of subjective features associated with cervical instability on initial interview during the recruitment (Table 3). None of the participants in the traumatic group had a diagnosis of alar ligament damage prior to high-field strength MRI. This would suggest that the diagnosis of an alar ligament lesion remains challenging [5,8,21,22], unless highly specialized MRI machines and protocols are followed.

Table 3.

Overview of the demographic details for subjects in the traumatic group.

Subject	Age (years)	Gender	History and type of neck trauma	Upright MRI	Supine MRT	Duration symptoms	Localization and type of the symptoms
1	44	Male	WAD car accident	Alar ligament scarring bilaterally	Right alar ligament attenuated	4 Y, 11 M	Right dorsal headache, impaired cervical ROM, dizziness
2	57	Male	Sports trauma cervical rotation	Left alar ligament rupture	Left alar ligament partial rupture	10 Y, 0 M	Bilateral dorsal headache, tinnitus right ear, impaired cervical ROM
3	41	Female	Sports trauma cervical rotation	Left alar ligament partial rupture	No alar ligament abnormality*	2 Y, 3 M	Left side headache, left ear and eye pain, left upper limb pain, vertigo, impaired cervical ROM, visual disturbance, hearing disturbance
4	64	Male	WAD car accident	Left alar ligament scarring. Right alar ligament elongation	Right alar ligament partial rupture	6 Y, 6 M	Right dorsal headache, impaired cervical ROM, vertigo
5	47	Male	Prolonged cervical extension during surgery	Right alar ligament fibrosis	No alar ligament abnormality*	30 Y, 0 M	Whole body pain, body sweating during activity, impaired concentration ability
6	38	Female	WAD car accident	Alar ligament scarring bilaterally	Bilateral alar ligament fibrosis	2 Y, 6 M	Dorsal and ventral headache, eye pain, impaired cervical ROM, vertigo and eye movement dysfunction
7	55	Male	Motorbike accident	Left alar ligament fibrosis, consolidated dens fracture	Left alar ligament abnormality. Consolidated dense fracture	3 Y, 0 M	Dorsal headache, paresthesia lower and upper extremities, dysarthria, impaired cervical ROM, vertigo, impaired concentration ability

*No alar ligament lesion found.

WAD: Whiplash-associated disorder; ROM: Range of motion; Y: years; M: months.

To the best of the authors’ knowledge, no study has compared different forms of MRI (high-field strength 3-T MRI and standard supine MRI with a specialized craniocervical protocol) to evaluate ligament injury in the upper cervical spine. Hence, it is not possible to say if one form is better than another. High-field strength upright MRI is a relatively new imaging procedure which allows images to be taken in different positions: supine, sitting or standing. Several studies demonstrate that MRI scans in supine show different results than MRIs in a standing position [23,24]. When sitting, cervical spine structures experience three times greater load than lying. Accordingly, this results in more strain to the cervical ligaments which might illuminate ligament injury [24,25]. This might explain why more subjects were found to have alar ligament lesions with upright MRI than with standard MRI.

One participant (subject 5) who was rated negative in each manual examination test did not report a history of WAD; however, his symptoms of upper cervical instability occurred after neck surgery which might have damaged the alar ligaments. This subject had a radiologist confirmed alar ligament lesion determined by high-field strength MRI scans. The manual therapist identified six of the seven symptomatic subjects through the specific alar ligament tests. In these six cases, all three tests were positive.

A combination of three manual examination tests resulted in greater diagnostic accuracy and therefore criterion validity. However, two positive tests provided no additional diagnostic accuracy when compared to one positive test. Specificity, but not sensitivity increased, when all three tests were positive. Improvement in diagnostic accuracy through a combination of test results was also found for manual examination procedures for identifying a symptomatic sacroiliac joint disorder [18].

The specific manual examination tests used in this study seem to be capable of identifying alar ligament lesions. Hence in the clinical setting, three positive tests would be justification to refer to for further imaging using more specialized MRI. In addition, three positive tests indicate caution in the use of certain manual therapy procedures as well as vigorous activity. It might also indicate the need for management aimed at reducing the symptoms associated with upper cervical instability.

It was assumed in this study that an alar ligament lesion seen on MRI would correlate with excessive mobility/instability on alar ligament stress testing. It is possible that an MRI determined alar ligament lesion is not always positive on alar ligament testing, but there is no evidence to confirm or refute this possibility. At what point an MRI lesion becomes clinically relevant in terms of instability requires further investigation. However, we assume that a normal clinical test indicates sufficient ligament integrity to not warrant concern about upper cervical instability.

5.1. Limitations

There are a number of limitations to this study. Even though we demonstrated in this case control study the ability of the tests (when used as a cluster) to clearly discriminate patients with and without alar ligament lesions, we cannot draw any conclusions about diagnostic properties of these tests for their clinical applicability due to the fact that we recruited subjects with extreme status of disease, namely confirmed diseased in comparison to truly not diseased. Overestimation of the accuracy is likely. In future studies, the use of these tests in the clinical setting must be addressed by investigating their diagnostic ability in consecutive recruited subjects who are suspected of having alar ligament injury. This implies that all subjects would receive MRI and manual examination. This would have given a better understanding of the incidence of alar ligament lesions and also the diagnostic accuracy of manual examination and standard MRI. However, this would likely involve a very large number of participants (although the incidence of alar ligament lesions has not been thoroughly investigated). Hence, the costs and time involved would likely not be feasible. Second, although the examiner was blind to subject allocation, all three manual examination tests were performed consecutively. This might have led to confirmation bias if the first test was positive, but there does not appear to be a way around this with such a small sample size. Third, there has been some question as to the veracity of MRI to identify ligament injury in the upper cervical spine [26]. Nonetheless, the ligament injury was confirmed in this study by high-field strength upright MRI with a specialized protocol, which is supported by the increased incidence of alar ligament lesions on high-field strength MRI compared to standard MRI. Fourth, statistical analysis was based on the assumption that upright MRI scanning in the healthy control group would find normal alar ligaments, which would also be the case for standard MRI. Finally, all subjects in this study with MRI confirmed alar ligament lesions were long-term sufferers of neck pain and other symptoms (Table 3). All subjects had a history of trauma or surgery, with the majority suffering symptoms from a WAD which occurred more than 1 year previously. Hence, the evaluated diagnostic property of these tests cannot be transferred to patients with acute or subacute conditions. Despite these limitations, the study provides some preliminary support for the use of clinical examination tests in the diagnosis of alar ligament lesions in clinical practice.

6. Conclusion and clinical implementation

A battery of three alar ligaments test may be helpful in determining the diagnosis of a lesion of the alar ligaments. These tests can be incorporated in the clinical examination of patients with symptoms associated with upper cervical instability. Traditional supine MRI scans have the potential to underdiagnose alar ligament lesions; hence, alar ligament manual examination test should provide some additional information that may be useful in diagnosis. Hence, it is suggested that three positive alar ligament manual examination tests could be a potential reason for referral for upright MRI. Conversely, an alar ligament lesion seen on MRI may not necessarily indicate upper cervical instability. However, the presence of such a lesion with evidence of instability on clinical testing warrants caution and may provide the basis for specific management of the individual patient.

Biography

Piekartz Harry von is Professor for Physical Therapy, course director of the MSc in musculoskeletal Therapy on the University of Applied Science in Osnabrück(Germany), clinician and researcher in head, face and neck pain.

Funding Statement

This research did not receive any specific grant from funding agencies in the public, commercial or not-for-profit sectors.

Disclosure statement

No potential conflict of interest was reported by the authors.

Ethics Approval

The study was conducted in accordance with the Helsinki guidelines and approved by the local ethics committee of the University of Applied Sciences Osnabrück (WiSO MS-MP-WS 1617-08).