Assessing the AIR classification reliability for estimating pronation of the first metatarsal

Abstract

Background

Hallux valgus (HV) is a multiplanar deformity and surgical treatment is often guided by two-dimensional radiographic parameters. This study assessed the reliability and accuracy of the AIR classification(The first metatarsal head’s lateral edge can be delineated as angular (type A), round (type R), or intermediate (type I) through visual inspection or circle measurements on weight-bearing radiographs.)commonly used in clinical settings to categorize the shape of the lateral edge of the first metatarsal head, against measurements from weight-bearing computed tomography (WBCT).

Methods

This retrospective study evaluated 18 patients, including 31 feet affected by HV. Two surgeons independently categorized the first metatarsal head’s lateral edge by both visual inspection and circle measurement. Additionally, two separate surgeons evaluated the α angle relative to the floor in WBCT scans. The reliability of the measurements was assessed using intraclass correlation coefficients (ICC) and weighted kappa statistics.

Results

While the first surgeon demonstrated perfect intra-observer reliability for both visual inspection and circular measurements (kappa values of 1.000 and 0.857, respectively), the second surgeon showed high and perfect reliability (kappa values of 0.759 and 1.000, respectively) for the same assessments. While the interobserver reliability for visual inspection was moderate (kappa values of 0.407 and 0.455, respectively), it was little to low for circular measurements (kappa values of 0.173 and 0.232, respectively). The interobserver reliability for the α angle assessment relative to the floor on WBCT scans was perfect (ICC = 0.968).

Conclusion

The AIR classification may not provide a reliable estimate of first metatarsal pronation, so clinicians should be cautious and recognize these limitations in their diagnostic applications. Between the two AIR classifications, the visual inspection measurement seems to be more reliable according to kappa value. If allowed, it may be better to additionally include a 3D assessment method such as WBCT, in the preoperative evaluation.

Level of evidence

IV, case series.

Introduction

Hallux valgus (HV) is currently recognized as a multiplanar deformity, yet its treatment predominantly adheres to two-dimensional (2D) radiographic parameters [1–4]. The advent of Weight-bearing CT scan (WBCT) has underscored the significance of the first metatarsal rotation, particularly its pronation in the foot’s coronal plane [5–7]. The etiology of first metatarsal pronation remains elusive, although the medial column’s involvement in metatarsal pronation in HV is postulated [8]. Initially, the first metatarsal pronation was inferred from its head’s morphological characteristics [1]. Okuda et al. employed dorsoplantar weight-bearing radiographs to associate the contour of the lateral aspect of the first metatarsal head with HV and observed a link with postoperative recurrence [9]. Yamaguchi et al. determined that the first metatarsal lateral edge configuration was contingent on rotation [2].

While various approaches exist for quantifying the first metatarsal pronation, there is no consensus on a precise measurement technique or definitive method for ascertaining the correction degree required for HV surgical intervention [10]. The first metatarsal head’s lateral edge can be delineated as angular (type A), round (type R), or intermediate (type I) through visual scrutiny or circle measurements [9] on weight-bearing radiographs. This method, known as the AIR classification, has garnered preference in clinical settings for its expedience and compatibility with real-time observational needs and adjustment during surgery.

It would be fair to say that 2D measurements such as met head rotation also suffer from a degree of bias related to having to locate precise surface landmarks which are subject to intrinsic and observer related variability, and from “slice bias” (related to having to perform the measurement in a precisely defined slice: this choice is in itself subject to variability) [11].

Conversely, WBCT provides precise depictions of the extent of first metatarsal pronation, enabling sectioned imaging of 3D entities and 2D evaluations at the designated standardized sections, such as the assessment of the pronation angle of the first metatarsal in relation to the horizontal floor [12, 13]. As WBCT is not yet available everywhere, the authors aim to assess whether the AIR classification is successful in correctly classifying pronation, which would be helpful in centers not yet equipped with cone beam.

This study aimed to appraise the precision, reliability, and reproducibility of the AIR classification using multiple assessments by various surgeons and juxtaposing them with the pronation angles obtained from WBCT. This comparison sought to ascertain the viability of the AIR classification as an intraoperative reference for orthopedic intervention and determine whether weightbearing radiographs are a good approximation of pronation on weightbearing CT scans.

Methods

Inclusion and exclusion criteria

Patients aged 18 years and above, admitted for unilateral or bilateral HV, and with preoperative dorsoplantar weight-bearing radiographs and WBCT scans of the feet were included in the study.

Conversely, patients with (1) any prior foot surgery, (2) concurrent HV with other deformities, such as progressive collapsing foot deformity; and (3) diseases or conditions impacting foot bone and ligaments, such as metabolic bone diseases or prolonged corticosteroid use were excluded.

Patient data

We conducted a retrospective review of consecutive patients presenting with unilateral or bilateral HV who were admitted to our hospital between May 2020 and October 2020. Eighteen female patients, encompassing a total of 31 feet, who satisfied the inclusion criteria were included in the study. The participants’ average age was 51.5 (23–67) years. The hospital’s Ethics Committee approved this study (IRB00006761–M2021250) and waived the requirement for informed consent due to the retrospective nature of the study.

Radiographic measurements

Standard dorsoplantar weight-bearing radiographs and WBCT scans were obtained with subjects standing on both legs and knees fully extended. The x-ray beam, perpendicular to the cassette in the coronal plane, was centered on the mid-point of the third metatarsal.

For measurements by visual inspection [9], the first metatarsal head’s lateral edge was categorized based on its morphology as follows: Type A exhibited an angular contour (Fig. 1A), Type R was identified by a round form (Fig. 1C), and Type I was defined by an intermediate form between angular and round: the lateral contour of the metatarsal head is not as round as a circle, but is rounded without any “stage” (Fig. 1B).

Fig. 1.

Visual inspection measurement. Classification was based on the morphology of the metatarsal head’s lateral edge: Type A was defined by an angular shape (A); Type I was defined by an intermediate form between angular and round: the lateral contour of the metatarsal head is not as round as a circle, but is rounded without any “stage” (B); and Type R was defined by a round shape (C)

For the circle measurements [2], the circle applied to the metatarsal head was defined by three points of contact: the medial edge, distal, and lateral edge of the metatarsal head. The categorization involved the relative positioning of the metatarsal head’s lateral cortical surface and a theoretical circle, along with the measurement of distance D. Type A was characterized when the lateral cortical surface curve was visibly outside the circle, with D ≥ 2 mm (Fig. 2A). Type R was assigned when the curve was on the circle, with D ≥ 0 mm and < 1 mm (Fig. 2C), while Type I was identified when the curve was outside the circle, with D ≥ 1 mm and < 2 mm (Fig. 2B).

Fig. 2.

Circle measurement. The shape was classified as type A if the curve of the lateral cortical surface of the metatarsal head was not apparently located on circle and if distance D(= ON-OM)measured ≥ 2 mm (A), as type I if the curve of the lateral cortical surface of the metatarsal head was not located on circle and if distance D measured ≥ 1 mm and < 2 mm (B), as type R if the curve of the lateral cortical surface of the metatarsal head was located on circle and if distance D measured ≥ 0 mm and < 1 mm (C)

The pronation angle of the first metatarsal was assessed by WBCT as described by Kim et al. [14]. This approach, known as the measurement of the α angle, involves calculating the angle between a line drawn through the midpoints of the inferior and superior aspects of the first metatarsal and a vertical reference line perpendicular to the floor (Fig. 3).

Fig. 3.

Measurement of the α angle. On a standardized coronal slice of a simulated weight-bearing CT, an inferior line was drawn between the lateral edge of the lateral sulcus and the medial edge of the medial sulcus. A superior line was drawn between the points of the medial and lateral corners of the first metatarsal head. Bisections of these two lines intersect with a line drawn perpendicular to the horizontal ground axis, and the α angle was measured between them

Prior to measurement, observers were jointly trained and a standardized protocol was developed by the researchers to minimize variability in the application of AIR classification and α angle measurement. In addition, software-assisted measurement tool was incorporated to reduce subjectivity. Two surgeons, each with significant experience in foot and ankle surgery, independently performed the AIR classification using both visual inspection and the circle measurement technique. To ensure intra-observer reliability, they repeated the classification one week later, blinded to their initial assessments. In a parallel, two other independent surgeons evaluated the α angle in relation to the floor on WBCT scans for the 31 feet with HV included in the study.

Statistical analysis

The reliability of both interobserver and intraobserver measurements for continuous variables was assessed using single-measure intraclass correlation coefficients (ICCs). Reliability categories ranged from little (ICC ≤ 0.25) to low (0.26 to 0.49), moderate (0.50 to 0.69), high (0.70 to 0.89), and perfect (≥ 0.90) [15].

For categorical variables, reliability was calculated using weighted kappa statistics. Agreement levels were delineated according to the Landis and Koch [16] classification system as little (Kappa = 0.0 to 0.20), low (0.21 to 0.40), moderate (0.41 to 0.60), high (0.61 to 0.80), and perfect (0.81 to 1.00).

All statistical analyses were conducted using the Statistical Package for the Social Sciences (SPSS) for Windows, version 29.0 (IBM Corp., Chicago, IL, USA).

Results

We conducted a retrospective review of consecutive patients admitted with unilateral or bilateral HV to our hospital between May 2020 and October 2020. Eighteen female patients, encompassing 31 feet with a mean age of 51.5 (23–67) years, met the inclusion criteria (Table 1).

Table 1.

Preoperative radiographic measurements for the cases with Hallux Valgus

Patients	Age(years)	V 1	V 2	V 3	V 4	Q 1	Q 2		Q 3	Q 4	α1(°)	α2(°)	Average α(°)
1 L	55	A	A	I	I	A	A	A		A	8.90	9.70	9.300
1R		A	A	A	A	A	A	A		A	2.87	4.17	3.520
2R	55	R	R	R	R	A	A	R		R	18.77	18.42	18.595
2 L		R	R	R	I	I	I	R		R	12.59	13.82	13.205
3 L	49	R	R	R	R	R	R	R		R	13.79	13.19	13.490
4 L	59	I	I	I	I	A	A	I		I	27.10	25.40	26.250
4R		I	I	I	I	R	R	A		A	18.43	20.11	19.270
5 L	57	A	A	A	A	A	I	I		I	11.50	10.14	10.820
6R	30	A	A	R	R	R	R	I		I	14.32	17.19	15.755
6 L		A	A	A	A	A	A	I		I	6.24	8.33	7.285
7R	54	R	R	R	R	R	R	R		R	21.64	20.60	21.120
8 L	44	A	A	A	A	R	R	I		I	5.75	5.03	5.390
8R		A	A	A	A	I	I	A		A	12.44	14.28	13.360
9 L	67	I	I	I	A	A	A	I		I	13.11	13.91	13.510
9R		I	I	I	A	I	I	A		A	16.27	15.34	15.805
10R	65	A	A	A	A	A	A	A		A	8.87	10.51	9.690
11 L	67	R	R	R	R	A	I	I		I	10.30	12.10	11.20
11R		I	I	R	I	I	R	R		R	10.56	9.16	9.860
12 L	25	A	A	R	I	A	A	I		I	8.72	11.22	9.970
12R		A	A	R	A	I	I	R		R	2.20	3.80	3.000
13R	62	I	I	A	I	I	I	A		A	1.36	2.63	1.995
14 L	64	I	I	R	I	R	R	R		R	17.53	18.37	17.950
14R		I	I	A	A	A	I	I		I	18.56	17.16	17.860
15R	25	I	I	R	R	A	A	R		R	17.59	18.19	17.890
15 L		R	R	R	R	A	A	R		R	13.47	13.40	13.435
16 L	61	I	I	I	I	R	R	R		R	10.73	8.94	9.835
16R		I	I	I	I	R	R	R		R	13.50	11.66	12.580
17 L	23	A	A	I	I	A	A	I		I	5.92	7.12	6.520
17R		I	I	R	R	I	I	I		I	9.57	11.27	10.420
18 L	65	I	I	A	A	A	A	A		A	5.25	4.75	5.000
18R		A	A	A	A	A	A	A		A	7.81	6.21	7.010

1 L: the No.1 patient with the left foot; 18R: the No.18 patient with the right foot; V1 and V2: first and second visual inspection measurements conducted by Surgeon 1; V3 and V4: first and second visual inspection measurements conducted by Surgeon 2; Q1 and Q2: first and second circle measurements conducted by Surgeon 1; Q3 and Q4: first and second circle measurements conducted by Surgeon 2; α1: pronation angle of the first metatarsal, measured on weight-bearing CT scans by Surgeon 3; α2: pronation angle of the first metatarsal, measured on weight-bearing CT scans by Surgeon 4; Average α: the average α of α1 and α2

The reliability of the intra-observer and inter-observer (Tables 2 and 3) assessments of categorical variables was determined using weighted kappa statistics. The first surgeon demonstrated perfect intraobserver reliability for both visual inspection and circle measurements (kappa values of 1.000 and 0.857, respectively), while the second surgeon showed high-to-perfect reliability (kappa values of 0.759 and 1.000, respectively). Conversely, the inter-observer reliability for visual inspection measurements between the two surgeons was moderate (kappa values of 0.407 and 0.455, respectively), and for circular measurements, it was low (kappa values of 0.173 and 0.232, respectively). According to kappa value, the visual inspection technique seems to be more reliable than the circular technique. The interobserver reliability of the α angle measurements on WBCT scans by the other two surgeons was perfect (ICC = 0.968).

Table 2.

Intraobserver Reliability calculated using Weighted Kappa and ICC

	V1 versus V2	V3 versus V4	Q1 versus Q2	Q3 versus Q4
Kappa	1.000	0.759	0.857	1.000
95% CI	1.000, 1.000	0.577, 0.940	0.723, 0.991	1.000, 1.000
ICC	-	-	-	-
Reliability	perfect	high	perfect	perfect

V1 and V2: first and second visual inspection measurements conducted by Surgeon 1; V3 and V4: first and second visual inspection measurements conducted by Surgeon 2; Q1 and Q2: first and second circle measurements conducted by Surgeon 1; Q3 and Q4: first and second circle measurements conducted by Surgeon 2

Table 3.

Interobserver Reliability calculated using Weighted Kappa and ICC

	V1 versus V3	V2 versus V4	Q1 versus Q3	Q2 versus Q4	α1 versus α2
Kappa	0.407	0.455	0.173	0.232	-
95% CI	0.160, 0.654	0.193, 0.718	−0.082, 0.429	−0.026, 0.491	0.935, 0.984
ICC	-	-	-	-	0.968
Reliability	moderate	moderate	little	low	perfect

V1 and V2: first and second visual inspection measurements conducted by Surgeon 1; V3 and V4: first and second visual inspection measurements conducted by Surgeon 2; Q1 and Q2: first and second circle measurements conducted by Surgeon 1; Q3 and Q4: first and second circle measurements conducted by Surgeon 2; α1&α2: pronation angle of the first metatarsal, measured on weight-bearing CT scans by Surgeon 3&4; ICC: single-measure intraclass correlation coefficients

Two observers, using methods visual inspection and the circle measurement technique and took two measurements of each, the results of all are shown in Table 1.The corresponding α angles for each AIR subtype are detailed in Table 4. For the 93 instances of type A, the α angle ranged from 1.360° to 27.100°, with a mean of 9.711 ± 5.242°. For the 83 instances of type I, the range was consistent, with a mean of 12.548 ± 6.019°. Among the 72 instances of type R, the α angle varied from 2.200° to 21.640°, averaging 14.201 ± 4.555°.

Table 4.

Descriptive analysis of AIR classification and α angle

Type	Sample size	Minimum α(°)	Maximum α(°)	Mean ± SD α(°)
A	93	1.360	27.100	9.711 ± 5.242
I	83	1.360	27.100	12.548 ± 6.019
R	72	2.200	21.640	14.201 ± 4.555

To elucidate the relationship between AIR classification and the α angle, we undertook a Spearman correlation analysis, the results of which are presented in Table 5. A significant negative correlation was identified between type A and the α angle (correlation coefficient − 0.522, p < 0.05), whereas a significant positive correlation was observed between type R and the α angle (correlation coefficient 0.413, p < 0.05), Which indicates that the type A may point to a smaller pronation angle, whereas the type R corresponds to a larger pronation angle. However, the correlation between type I and the α angle was non-significant, with a correlation coefficient of 0.133 and a p-value exceeding 0.05.

Table 5.

Correlation analysis between AIR classification and α angle (Spearman-related)

		Type A	Type I	Type R
Average α	Correlation coefficient	−0.522	0.133	0.413
	p	0.003	0.476	0.021

Discussion

The concept of first metatarsal pronation is well-established in the anatomical literature and has long been acknowledged [17]. Current research reaffirms the presence of metatarsophalangeal pronation in HV through various radiological assessments [4, 14], although orthopedic specialists have often overlooked it. Okuda et al. identified a correlation between the first metatarsal head’s lateral edge and the recurrence rate of HV [9], attributing the variance in these shapes to metatarsal pronation [18]. This has piqued considerable interest in the orthopedic community. There is a burgeoning consensus among researchers advocating correction of the first metatarsal head pronation as an essential consideration in HV management [19, 20].

Preoperative and intraoperative evaluations of the extent of first metatarsal head pronation are crucial for surgical correction. The prevailing gold standard for such assessments is WBCT of the foot [6, 7, 14]. Despite the growing use of WBCT, solutions need to be found when only 2D X-Ray is available. The predominant clinical approach involves analyzing the shape of the first metatarsal head’s lateral edge using a dorsoplantar radiograph [21, 22]. The three lateral edge shapes based on the AIR classification [9, 17] reflect the degree of ascending metatarsal head pronation. The validity of this trend was initially established through cadaveric studies [20].

Despite the routine clinical application of the AIR classification, the specific pronation angles corresponding to types A, I, and R or a distinct angle range that can categorically differentiate them remain ambiguous. Expert consensus currently suggests approximate ranges of 0–10, 10–20, and > 20 degrees for types A, I, and R, respectively. However, these categorizations lack empirical substantiation [10, 23]. Our research found notable discrepancies, with types A, I, and R exhibiting mean α angles of 9.711 ± 5.242°, 12.548 ± 6.019°, and 14.201 ± 4.555°, respectively (Table 4). Hence, a clear distinction between these types remains elusive.

Conti et al. demonstrated a weak correlation between the pronation angle of the first metatarsal, as measured by weight-bearing X-rays and WBCT scans [13, 24]. Patel et al. also found the pronation of the first metatarsal measured on weightbearing AP radiographs had moderate interobserver agreement and was only weakly associated with pronation measured from WBCT scans, which suggest that the first metatarsal pronation measured on weightbearing radiographs is not a substitute for pronation measured on WBCT scans [6]. Similarly, our findings indicated ambiguity in the angles corresponding to the AIR classification, with a significant overlap observed, particularly in defining type I. This inconsistency could be attributed to several factors, including the (1) low ICC for the AIR classification, which remains unimproved even with numerical judgment (α angle), (2) variability in projection angles due to the differing inclinations of the first metatarsal bone, significantly influencing AIR classification, corroborated by prior studies [2, 25], and (3) potential anatomical variations among the metatarsal heads. Nonetheless, our analysis identified a statistically significant correlation (p < 0.05) between types A and R and the α angle in WBCT scans (Fig. 4). Specifically, type A is generally associated with smaller α angles and type R with larger ones (Table 5), suggesting this trend holds some analytical value, at least within the context of our study.

Fig. 4.

Correlation of the α-angle with Types A, I, and R on Weight-Bearing CT Scans. The figure illustrates the distinct relationship between the α-angle and the categorizations of hallux valgus deformity. A notable trend is observed where Type A is associated with smaller α-angles (A), contrasting with Type R that aligns with larger α-angles (C), with the differences being statistically significant as detailed in Table 5. Conversely, Type I exhibits no significant correlation with the α-angles (B), indicating a variable presentation in this group

This study aimed to scrutinize the reliability of the AIR classification based on the notion that a more precise determination of the first metatarsal’s pronation degree on weight-bearing radiographs could significantly enhance the diagnostic and therapeutic approaches to the disease. Such precision is expected to facilitate the intraoperative simulation of weight-bearing radiographs, offering a straightforward and efficient assessment method under fluoroscopic or direct vision [26]. Nonetheless, when juxtaposed with WBCT metrics, the fidelity of the AIR classification in prognosticating pronation angles appears suboptimal and is potentially compromised by various factors. In scenarios requiring preoperative strategizing, reliance on more accurate WBCT measurements seems prudent for surgical decision-making [2, 14, 24, 27]. However, the lack of real-time applicability of WBCT for intraoperative evaluations highlights its limitations and presents a logistical conundrum.

This study has several inherent limitations. First, the constrained sample size may have skewed the interpretative validity of the findings. Second, this analysis amalgamates results from two distinct methodologies-visual inspection and circle measurement, for assessing the morphology of the first metatarsal head’s lateral edge. Although this approach was adopted to enrich the data pool, it introduced a variable that might have nuanced the outcomes, thereby necessitating a cautious interpretation of the results. Third, we chose the α angle measured on WBCT as the “gold standard” because it has been widely used in other literatures and its reliability has been validated to some extent, but the α angle also has some potential biases. Several factors influence the value of the α angle, such as generalized ligament laxity, flat foot, and cavus foot, among others. Therefore, the search for such a “gold standard” for the assessment of first metatarsal pronation is one of the most important aspects of our work in future studies.

Conclusions

In summary, the clinical ubiquity of the AIR classification is due to its simplicity and practicality. However, this study underscores its limited precision in predicting pronation angle, particularly when benchmarking against WBCT measurements. Although the AIR method demonstrates reasonable accuracy for types A and R, it falters the nebulous classification of type I. In daily practice, we still recommend the plain radiographic investigation for assessing AIR classification, but clinicians should exercise caution, recognizing these constraints in their diagnostic applications. Between the two AIR classifications, however, we tend to recommend the visual inspection measurement because, at least in our study, it is slightly more reliable and easier to implement. The sole reliance on plain radiography may result in misconceptions regarding the first metatarsal pronation angle. If conditions allow, we think that it may be better to additionally incorporate a 3D assessment method such as WBCT during the preoperative evaluation. A more rigorous assessment in the preoperative phase may help to refine the precise correction of the anterior rotation angle and improve surgical outcomes.

Authors’ contributions

Ning Sun, MD & Xuewen Wang, MD wrote the main manuscript, they contributed equally to this work; Xiangyu Xu, MD, as the corresponding author, is responsible for the authenticity of the content of the paper, the reliability of the data, the credibility of the conclusions, and the compliance with legal norms, academic norms, and ethical norms; Heng Li, MD, Wenjing Li, Xiaofeng Gong, MD & Hui Du, MD were responsible for the measurement of data; Xiaofeng Gong, MD & Hui Du, MD & Yong Wu, MD were in the same treatment group and completed the surgery together; Yong Wu, MD provided administrative support for the study as administrative director of the department.

Funding

The present study was supported by a hospital level research fund from Beijing Jishuitan Hospital(Grant ID: XKGG202113).

This study was also supported by a fund called “Researches on Intelligent Planning and Precise Key Techniques of Surgical Robot for Total Ankle Arthroplasty” from Beijing Natural Science Foundation(Award number: L232004).

Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate.

The study received approval from the Ethics Committee of the Beijing Jishuitan Hospital, Capital Medical University. All methods were conducted in accordance with relevant guidelines and regulations. The procedures adhered to the principles outlined in the Helsinki Declaration. Because this was a retrospective review, informed consent was waived with agreement of the Ethics Committee.

Consent for publication

Not Applicable.

Competing interests

The authors declare no competing interests.