Comparison of the reliability of ANB Sar and W measurements used in cephalometric diagnostics for the assessment of sagittal discrepancy

Abstract

Accurate cephalometric assessment is used in orthodontic diagnosis and treatment planning. The aim of this study was to confirm the reliability of Sar and W angle measurements with the results obtained for the ANB angle. The reliability of angular measurements for determining sagittal discrepancy was assessed after determining anthropometric points (N, A, B, S, W, M, G). These formed the ANB, Sar, and W angles, respectively. Cephalometric analyses were performed separately, 7 days apart, by 22 orthodontists. To determine inter-rater reliability, a two-way ANOVA without repeated measures was used, using the ICC(3,1) coefficient. To compare the reliability of the ANB, Sar, and W angles, the Blant-Altman plot, Dahlberg formula, intraclass correlation coefficients (ICC), R2 coefficients, and R&R coefficients were used. Pearson’s chi-square test was used to assess the number of individuals classified into each skeletal class. The ICC(3.1) coefficient at a 95% confidence level was 0.941 for the ANB angle, 0.679 for the Sar angle, and 0.688 for the W angle. These results demonstrate excellent measurement reliability for the ANB angle and good reliability for the Sar and W angles. The participating orthodontists measured sagittal discrepancy significantly more accurately using the ANB angle compared to the Sar and W angles. One of the main factors implying poorer reliability of the Sar and W results is operator error. The difficulty in determining the values of the angle tests is related to the determination of new anthropometric points. The obtained results indicate higher reliability of the ANB measurement in assessing sagittal discrepancy in malocclusions. The Sar and W angles can be used to a limited extent, especially in the assessment of extreme cases. Further studies determining the usefulness of the new Sar and W angles in the orthodontic diagnosis of sagittal discrepancy should be correlated with previous measurements, including Wits or Beta angle measurements.

Introduction

Accurate cephalometric analysis is a fundamental element of orthodontic diagnosis^1–3. Its reliable results depend on the reliable determination of anatomical landmarks. Reliability consists of internal repeatability between each examiner and reproducibility, i.e., obtaining similar parameters in the Cartesian coordinate system for each anthropometric point by different orthodontists^1–8. By correctly defining cephalometric points, we obtain reliable angular measurements assessing the sagittal discrepancy between the mandible and maxilla^2,3,9–14. Many cephalometric measurements used in daily clinical practice are not perfect. They rely on reference points, the precision of which, despite the intensive development of specialized skills and technical infrastructure, is subject to numerous limitations^3,15–17. Contemporary research^18–22 using artificial intelligence has not yet led to increased reliability in the automation of the cephalometric analysis process. After Segner and Hasund presented an individual approach to cephalometric analysis in 1994, based, among others, on Steiner’s assumptions, the diagnosis of sagittal discrepancy was based on measurements of the ANB angle²³. Later analyses, the most commonly used of which are the Wits analysis and the Beta angle, assessed independently or in combination with ANB, are still included in the diagnosis of sagittal discrepancy. Acquiring knowledge and skills by physicians in determining anthropometric points does not eliminate doubts in making the correct diagnosis. Doubts result from, among others: - low stability of the ANB angle; instability of the N points, related to the accumulation of bone tissue during growth; and the A point, related to its susceptibility to changes in position during orthodontic treatment³. The Wits parameter is characterized by erroneous assessment, which is directly related to an unstable occlusal plane, which is often difficult to determine correctly at various stages of growth, the eruption of individual teeth or tooth groups, and often due to premature tooth loss due to caries or hypodontia^1–3. Assessment of the Beta angle is also subject to error. Similar to ANB, it is based on an unstable A-point. The second error threshold is the difficulty in assessing the axis of the mandibular condyle³. Diagnostics based on the new Tau^1–3 and Yen^1,3 measurements also require the search for new analyses that would independently allow for a clear diagnosis in the assessment of sagittal discrepancy. Factors influencing the risk of error associated with the use of each parameter include insufficient knowledge of both measurements by dentists. This results in lower reliability of both measurements compared to ANB. At the same time, assessment of the Tau angle is associated with determining the T-point, which, according to Kotuła et al.², is unstable and can influence measurement error.

To minimize the limitations of individual measurements, clinicians combine parameters from selected analyses and compare them to accurately assess sagittal discrepancy. This is especially important in extreme cases.

The limited reliability of individual measurements encourages the search for new, reliable analyses for assessing sagittal discrepancy. A systematic review by Kotuła et al.¹ revealed new possibilities. Some of these have already been evaluated by their authors, Gupta et al.²⁴, Neel et al.²⁵, and Kotuła et al.³. Among the measurements mentioned in the review¹, there are others than those described above, based on new reference points M (center of the anterior maxilla) and G (center of the mandibular symphysis). These measurements are more difficult to plot compared to Tau and Yen. They measure the angle between the MG axis and the perpendicular drawn from point M to the axis connecting the SG (W)²⁷ or WG (Sar)²⁸ axis. The studies by Kotuła et al.^2,3 did not confirm the higher reliability of Tau and Yen measurements. Therefore, it is necessary to assess the reliability of the new Sar and W measurements as parameters based on points M, G, W, and S. The aim of this study was to assess the repeatability and reproducibility of the Sar and W angle measurements compared to the so-called “gold standard,” i.e., the ANB angle. The reliability of measurements (understood as their accuracy) is closely related to both repeatability and reproducibility. A result is reliable when it is both precise (repeatable) and consistent (reproducible).

Materials and methods

Study conditions, inclusion and exclusion criteria

The study participants were patients of the Department of Maxillofacial Orthopedics and Orthodontics of the Medical University of Wrocław, who started the diagnostic process before planning orthodontic treatment. The following inclusion criteria were established:

1. caucasian patients aged 12 to 18 years;

2. not previous orthodontic treatment.

3. with an orthognathic face (79º ≤ SNA ≤ 85º).

4. with various categories of Angle Class I, Class II and Class III.

5. characterized by different base angles (NL/ML).

6. generally healthy patients.

7. without developmental defects.

8. without symptoms of untreated dental caries and periodontal disease.

Each of the patients qualified for the study had a lateral radiograph of the skull in habitual occlusion. The parallelism of the Frankfurt plane to the ground and the perpendicularity of the beam to the sagittal plane were maintained.

After taking the cephalometric image, the radiograph was visually assessed. The following exclusion criteria were applied:

• i.asymmetry interpreted as a divergence of the contours of the right and left side of the mandibular body greater than 8 mm (measured at point Go),
• ii.projection error or abnormal contrast preventing the identification of reference points,

Ethical considerations

The study was conducted in accordance with the Helsinki Declaration (2024)²⁶, as well as national and institutional ethical standards. The study was approved by the Bioethics Committee of the District Medical Chamber in Zielona Góra (decision 01/173/2023 of March 6, 2023).

Before starting the study, written informed consent to participate in the study was obtained from patients or their legal guardians. Participants were informed about the study objectives, protection of personal data in accordance with the GDPR (General Data Protection Regulation - is the EU regulation on the protection of personal data, known in Poland as RODO) and that participation in the study was voluntary and that they could withdraw from the study at any time.

Further declarations of human ethics and consent to participate: not applicable.

Methods

The sample size was selected to be sufficient for analysis. The number of radiographs (patients) was arbitrarily set to approximately 30 patients in each of the three skeletal classes. Assuming a significance level of α = 0.05, an effect size of Ef = 0.6, and a test power of 1 - β = 0.80, the minimum sample size in each of the three skeletal classes is N = 24. Eighty-nine digital lateral cephalograms performed using the 2D technique were qualified for evaluation. Each cephalogram was recorded as the last 6 digits of the patients’ PESEL numbers. The PESEL (Universal Electronic System for Registration of the Population) is an eleven-digit identification number assigned to Polish citizens and foreigners residing in Poland. It uniquely identifies an individual in the Polish administrative system. This number contains an encoded date of birth, a serial number, a gender designation, and a check digit. The radiographs with PESEL numbers recorded in this way were assigned numbers from 1 to 89 in any order. The PESEL numbers together with the corresponding numbers from 1 to 89 were placed in an Exel spreadsheet (Microsoft, Seattle, WA, USA).

The analysis of all radiographs was performed based on reference points entered into the computer in the Ortodoncja V.9.0 program (Ortobajt^®, Wrocław, Poland) using NEC Multisync EA 244 WMI monitors of high laboratory quality certified (NEC, Tokyo, Japan). Monitor specifications: 24-inch diagonal, 1920 × 1200 resolution, 0.270 mm pixel size, 178° vertical and 178° horizontal viewing angle, 1000:1 contrast. The identification of landmarks was performed manually on digital images using a cursor controlled by a computer mouse in the Orthodontics V.9.0 program. Each cephalometric image was entered into the program and initially calibrated (KSz) according to the measuring ruler placed in the calibration window of the program at the level of 3 cm. Each radiograph was initially calibrated by one of the leaders (KSz). The leader (KSz) then sent the calibrated radiographs to the second leader (JK). Images prepared in this way after initial calibration were sent to 22 orthodontists, who then performed cephalometric analyses according to the study schedule. The results were recorded in an Excel spreadsheet (Microsoft, Seattle, WA, USA). Then, statistical analysis was performed. The average Sar and W angles for each of the 89 cephalograms associated with individual skeletal defects in the sagittal plane were correlated with the size of the base angles to assess the stability of the Sar and W angle in the assessment of skeletal defects. The obtained values ​​were then correlated with the parameters from the ANB angle analysis.

Both supervisors (JK and KSz) independently analyzed the database of consecutive cephalograms using the Segner and Hasund method. They assessed the ANB angle and randomly selected 89 of 270 radiographs (patients in skeletal class I, II, and III, qualified for further analysis, to obtain three groups of patients with similar numbers of individuals in skeletal classes I, II, and III. Each of these classes included patients with low, normal, and high NL/ML angles. The selected 89 patients in skeletal classes I, II, and III had a normal ML/NL angle of 20+/−7 degrees, i.e., 13–27 degrees. This selection was made to exclude the influence of vertical discrepancy on the assessment of sagittal discrepancy. This enabled drawing meaningful conclusions and minimizing assessment error. The number of radiographs in each of the three skeletal classes, approximately 30, exceeded the required number of N = 24, which would ensure adequate test power. The agreement between the two orthodontists in classifying the 89 cephalograms into one of the three classes was verified by calculating the Fleiss’ kappa coefficient of agreement (K) = 0.730 and Cramér’s index (V) = 0.751. The Fleiss’ kappa and Cramér’s index values ​​indicate good agreement between the two orthodontists in classifying patients into skeletal classes. The agreement between the measurements obtained by both dentists was also confirmed using the Mann-Whitney U test (Fig. 1). There was no statistically significant difference between dentists JK and KSz in the ANB angle measurements (p > 0.05).

Fig. 1.

Results of ANB angle measurements performed by the leaders (JK and KSz) on 89 cephalograms.

To compare the repeatability of angle measurement results (continuous quantitative variables) on the same radiographs by two leading physicians (JK and KSz), Bland-Altman plots, Cohen’s Kappa coefficient, and, most importantly, the intraclass correlation coefficient (ICC) were used. For the ANB angle, the intraclass correlation coefficient was ICC = 0.970 [0.965–0.975], for the W angle: ICC = 0.959 [0.952–0.965], and for the Sar angle: ICC = 0.966 [0.960–0.972]. ICC values ​​above 0.9 indicate very high (excellent) agreement between the two physicians in angle measurements.

In the group of 89 patients (all three skeletal classes), the ICC values ​​between JK and KSz were: for the ANB angle: 0.98 [0.98–0.99], for the W angle: 0.97 [0.95–0.98], and for the Sar angle: 0.98 [0.97–0.99]. This demonstrates excellent agreement between the leaders.

The ICC(3, 1) for the ANB angle in skeletal class I (29 radiographs) between orthodontists was ICC = 0.62 [0.34–0.80], for the W angle: ICC = 0.64 [0.37–0.81], and for the Sar angle: ICC = 0.78 [0.58–0.89]. In skeletal class II (30 x-rays), for ANB, the ICC coefficient was 0.58 [0.28–0.77], for the W angle: ICC = 0.97 [0.94–0.98], and for Sar: ICC = 0.95 [0.90–0.98].

In skeletal class III (30 x-rays), for ANB, the ICC coefficient was 0.87 [0.75–0.94], for the W angle: ICC = 0.95 [0.91–0.98], and for Sar: ICC = 0.95 [0.90–0.98].

To blind the study, after making a copy of the folder containing the analyzed cephalograms, the PESEL numbers were randomly replaced with numbers from 1 to 89 and then sent to 22 researchers – orthodontic residents trained in marking not only points A, N, and B, but also points used to measure the Sar and W angles (Fig. 2). The analysis of the Sar and W angles was published in the scientific literature only at the end of 2014²⁷ and in 2011²⁸. Orthodontists acquired basic knowledge and skills in correctly determining reference points during a five-hour theoretical and practical training as part of a cephalometric analysis course, culminating in a skills test. Dentists assessed each cephalogram twice. The second examination was conducted seven days after the first, under the same environmental conditions, in rooms with reduced light intensity. The double analysis resulted in 3916 measurements of each variable, for a total of 11,748 measurements.

Fig. 2.

Landmarks A, N, B, T, M and G used in the analysis of ANB, Sar and W angles. Location of new cephalometric points T, M and G; and measurements.

In the first column of the Excel spreadsheet, individual radiographs were randomly assigned numbers from 1 to 89. The second column contained the patients’ PESEL identification numbers, verifying each patient. Before the images were sent to the orthodontists by the treating physicians, the PESEL numbers were blinded to prevent patient identification.

Performing duplicate measurements of each cephalogram allowed the Orthodontics V 9.0 software to generate a consolidated file for each examiner. All files were sent to the treating physicians (JK, KSz).

After unblinding the PESEL numbers assigned to the corresponding radiograph numbers, the obtained results were stored separately in 44 Excel spreadsheets. Each row number corresponded to the exact same patient number assigned by the treating physician JK before sending the images to the orthodontists. This affected the integrity of the results. The ANB, Sar, and W angle values measured during the same examination were collected in a single spreadsheet. Fig. 2 presents the principles for determining the anthropometric points defining the angular measurements ANB, Sar, and W, used to assess sagittal divergence of the maxillary base in cephalometric diagnostics. The Sar angle is measured at point M between the arms of the MG and the perpendicular to the WG segment, derived from point M. The W angle is measured at point M between the arms of the MG and the perpendicular to the SG segment, derived from point M. The locations of points A, N, and B were determined according to the analysis by Segner and Hasund. The locations of the new anthropometric points were defined as:

point M – the construction point representing the center of the largest circle, tangent to the frontal, upper, and palatal surfaces of the maxilla;

point G – the construction point representing the center of the largest circle, tangent to the inner, frontal, posterior, and lower edges of the mandibular symphysis.

Statistical analysis

Statistica V.13.3 (Tibco Software Inc., Palo Alto, CA, USA) was used for statistical analysis. It assessed the reliability of measurements affecting the assessment of the mandibular position in the sagittal dimension, including:

1. Repeatability - differences in results obtained by the same examiner,

2. Reproducibility - differences in results obtained by 22 examiners (reproducibility).

The following statistical tools were used to assess the repeatability of measurements: chi-square test, Bland-Altman plot, Dahlberg formula, repeatability component of the R&R index.

The following statistical tools were used to assess reproducibility: reproducibility component of the R&R index, R2 coefficient of determination, ICC correlation coefficient, and Cohen’s Kappa coefficient with Fleiss correction³⁰.

The Bland-Altman plot was used to graphically assess the agreement between two series of measurements by the same physician. Based on them, we can read the average differences between two measurements and determine the intervals of agreement with a 95% level of certainty.

Dahlberg’s formula was used to estimate the random error between repeated measurements of the same feature.

where:

SE - standard error of repeated measurements,

di - difference between the first and second measurement of the same feature for the i-th object,

n - number of pairs of repeated measurements.

The error value is calculated as a percentage. The lower the error value, the higher the repeatability of the measurement.

The R&R (Repeatability & Reproducibility) module of the STATISTICA program (TIBCO Software Inc., Palo Alto, CA, USA) consists of five components: repeatability, reproducibility, objects (patients), interaction (dentist/patients) and the total assessment of repeatability and reproducibility assessing the cumulative reliability of the measurement. The individual components determine their percentage share in the assessment of the reliability of the parameter. In order to minimize the measurement error resulting from the interaction component, which depends on the skills of the doctors, orthodontists were trained in the location of reference points.

R² – the coefficient of determination determines the percentage of variability in the results of one measurement that is explained by the results of another measurement. Because the distributions of the measurement results deviated from the normal distribution, the Spearman rank correlation coefficient could be calculated. The determination coefficient R² was used to assess the reproducibility of measurements and the strength of the relationship between measurements of the same parameter between orthodontists. The value of the determination coefficient ranges from 0 to 1. A value close to 1 means that the model confirms excellent repeatability of measurements between orthodontists. A value close to 0 suggests that the model does not explain the variability of the data, or the reproducibility of measurements is insufficient.

Two-way analysis of variance was used to estimate the ICC. The ICC index value used to assess reproducibility should be interpreted in accordance with the standard given by Koo and Li²⁹. When considering ICC in the context of cephalometric measurements, it is necessary to assess the consistency of measurements performed by different orthodontists or in different conditions.

Depending on the context and structure of the data, an appropriate ICC analysis model should be adopted:

ICC(1): It is used when measurements are performed by different researchers and each researcher evaluates different units.

ICC(2): This is used when measurements are made by different examiners, but all assessments refer to the same units (e.g. cephalometric measurements in the same patients).

ICC(3): This is used in the context when measurements are made by the same examiners, and all assessments refer to the same units.

Koo and Li attempted to interpret the results obtained during the ICC assessment.

The individual ranges of the values ​​obtained indicate:

• 3.0.0–0.20: Poor agreement.
• 4.0.21–0.40: Moderate agreement.
• 5.0.41–0.60: Good agreement.
• 6.0.61–0.80: Very good agreement.
• 7.0.81–1.00: Excellent agreement.

Since each cephalogram was assessed by each of the 22 orthodontists, and the agreement was assessed between individual measurements, the ICC(3,1) model was adopted as the measure in this study.

Sensitivity and specificity analyses of individual cephalometric analyses were performed using the cumulative sensitivity CSens and cumulative specificity CSpec coefficients.

Cumulative sensitivity is the fraction of cases correctly assigned to a given skeletal class.

Cumulative specificity is the fraction of cases correctly rejected as not belonging to a given skeletal class.

Results

The assessment of the agreement of measurements between the two study leaders qualifying 89 radiographs of patients with an orthognathic face (79º ≤ SNA ≤ 85º) was made using the kappa-Fleiss coefficient (K = 0.730) and the Cramér coefficient (V = 0.751). The obtained results indicate a very good agreement between the leaders qualifying patients to one of the three skeletal classes – I, II, III. Table 2 presents a graphical distribution of radiographs classified by the leaders after performing a double analysis of the ANB angle to one of the three skeletal classes I, II, III.

Table 2.

Values ​​of the ANB, Sar and W parameter ranges corresponding to the individual skeletal classes I, II, III.

		Skeletal class
		I	II	III
Parametr	ANB	≥ 0° ≤4°	> 4°	< 0°
	SAR	≥ 57° ≤60°	< 57°	> 60°
	W	≥ 55° ≤57°	< 55°	> 57°

89 radiographs classified by the leaders into three skeletal classes were sent to 22 orthodontists (Table 1). They were subjected to double evaluation, qualifying them using the ANB angle analysis to one of the three skeletal classes. The Kappa-Cohen concordance coefficient (K = 0.846) assumed a value greater than 0.8, which indicates high consistency of classification of individual radiographs and agreement between doctors.

Table 1.

Criteria for dividing patients into skeletal groups, numbers (N) and fractions (%) of classification results of 89 patients based on double measurements of the ANB angle performed by 22 Doctors and the result of the chi-square test of the observed distribution with a uniform distribution.

Angle (0)	Class I Normocclusion	Class II Mandibular retrognathia	Class III Mandibular prognathism	p
ANB	≥ 0° ≤4° N = 1303 (33.3%)	> 4° N = 1296 (33.1%)	< 0° N = 1317 (33.6%)	0.957

The application of the chi-square test (Table 1) at the significance level of α = 0.05 and the result p = 0.957 indicates excellent agreement between orthodontists when classifying individual radiographs using the ANB angle into the appropriate skeletal classes I, II, III.

The results of the analyses of the ANB, SAR and W angles allowed us to establish the ranges of assessment for the individual skeletal classes. They are presented in Table 2. The ANB angle values ​​for skeletal class I are: ANB ≥ 0 and ANB ≤ 4, II: ANB > 4^o, III: ANB < 0 ^o. The Sar angle values ​​for the skeletal classes are: I: Sar ≥ 57 and Sar ≤ 60 ^o, II: Sar > 60 ^o, III: Sar < 57 ^o. The W angle values ​​for the skeletal classes are: I: W ≥ 55 and W ≤ 57 ^o, II: W > 55 ^o, III: W < 57 ^o.

Assessment of measurement repeatability

Dahlberg’s formula is the first in the parameters that was used to assess the repeatability of measurements of the ANB, Sar and W angles. The interpretation of the calculations of this parameter indicates the size of the standard error between two repetitions of analyses performed by the same orthodontist. The value for the ANB angle is 0.68 ^o, for the Sar angle it is 2.33 ^o and for the W angle it is also 2.33 ^o. The obtained results indicate a much smaller error between two measurements performed by the same orthodontist for the ANB angle. The remaining two angles are characterized by a result that is more than three times greater, which indicates lower measurement repeatability. From a clinical point of view, it can be assumed that the discrepancy in the ANB angle value is so small that it will not have a drastic effect on orthodontic treatment planning. However, in the remaining angles, another analysis should be used to correctly assess the sagittal discrepancy.

Bland–Altman graphs; indicate whether the differences between the measurement series change depending on the size. They are a graphical representation of the agreement between two measurement series (Fig. 3). The average values of the differences between the two measurements, respectively for the ANB angle 0.033°, Sar − 0.09°, W −0.08°, indicate high repeatability. The order of measurement was not observed to have a significant effect on the results. The differences between subsequent measurements are randomly distributed and fall within the limits of agreement, which means that the results of both series are largely consistent. However, the assessment of the 95% interval of agreement is the best for the ANB angle and is: 3.75 ^o (from − 1.84 ^o to 1.91 ^o); for the Sar angle it is 12.9 ^o (from − 6.5 ^o to 6.4 ^o); for the W angle it is 12.93 ^o (from − 6.54 ^o to 6.39 ^o). A discrepancy close to 4 ^o is still acceptable because it constitutes the average range of class (I) However, the use of the remaining parameters, whose confidence interval of repetition is 12 ^o, becomes questionable, which clinically translates into the possibility of assessing the sagittal discrepancy the first time as class III and the next time as class (II) This is associated with completely different orthodontic treatment planning in the same patie/nt. Such large intervals of 95% confidence in the repeated analysis of the same parameter indicate low repeatability of determination.

Fig. 3.

Bland-Altman plot of agreement between duplicate measurements of ANB(a), Sar(b) and W(c) angles by 22 orthodontists in 89 patients.

Evaluation of repeatability and reproducibility of angular measurements in the assessment of sagittal discrepancy using the R&R module

1. The R&R module of STATISTICA (TIBCO Software Inc., Palo Alto, California, USA) was also used to evaluate the repeatability and reproducibility of duplicate ANB, Sar, and W angle measurements performed for each radiograph by 22 physicians (Table 3). The ANB angle had the lowest variability (4.3%), followed by W (15.8%), and the Sar angle had the highest variability (16.4%). The data indicate high variability in determining anthropometric points between two series for the same physician. The W angle had the lowest variability between orthodontists (0.5%), followed by ANB (0.7%), and the Sar angle had the highest variability (0.8%). This indicates high inter-physician agreement in finding reference points and drawing the appropriate angles. All values demonstrate similar levels of repeatability. The total R&R (repeatability and reproducibility) index was 6.4% for ANB, 32.8% for W, and 33.8% for Sar, respectively. General guidelines characterizing the quality of individual measurement systems indicate that results below 10% indicate adequate system design. Values of 10–30% are satisfactory for system quality, while values > 30% require improvement. When assessing the quality of cephalometric analysis of individual angle variables based on this, it should be noted that only the total ANB angle (6.4%) is acceptable. The results obtained for the W angle (32.8%) and Sar angle (33.8%) require improvement and do not allow for the use of these measurements for diagnostic purposes.

Table 3.

Results of the analysis (R&R) of the repeatability and reproducibility of the double measurement of the ANB, Sar, and W angles, performed by 22 dentists on 89 cephalograms.

	ANB	Sar	W
Reproducibility (the same dentist	4.3%	16,4%	15.8%
Repeatability (dentists)	0.7%	0.8%	0,5%
Objects (patients)	93.6%	66,2%	67,2%
Interaction (dentists/patients)	1.4%	16,5%	16,4%
R&R	6.4%	33,8%	32,8%

Further evaluation of the R&R parameter allows for the determination of two levels of measurement error. The first is patient variability. This is a valid conclusion, as the measured sagittal discrepancy will always depend on the severity of the defect. The second factor influencing the level of reliability error (repeatability and reproducibility) is the physician’s knowledge and skills. A thorough assessment of the R&R index allows us to determine which orthodontists in the entire group made the greatest errors (Table 3, row 4). R&R allows us to identify sources of error and raise standards for cephalometric analysis.

Reproducibility analysis

The R² and ICC indices were used to assess the reproducibility of the ANB, Sar and W angle measurements in the assessment of sagittal discrepancies.

1. For the coefficient of determination R² (Table 4), the values ​​were as follows: ANB, W and Sar (91,6; 70,5; 48,4). The R squared (R²) statistical test, also known as the coefficient of determination, was used to assess repeatability. It assesses the strength of the relationship between the results of the first and second measurement of the same parameter under the same conditions.

Table 4.

Comparison of the reproducibility of the angle measurements of ANB, Sar and W angle in two subsequent analyses performed by 22 orthodontists in a group of 89 patients and the value of the coefficient of determination R².

	ANB	Sar	W
R 2	0,916	0,484	0,705
Percentage of repetition in the second study	91,6%	48,4%	70,5%

The R² value ranges from 0 to 1. A value close to 1 means that the model explains the variability of the data well, while a value close to 0 suggests that the model does not explain the variability of the data.

When interpreting the obtained results (Table 4), R² tells us what percentage of the variability of the dependent variable is explained by the independent variables. The data obtained for the ANB angle (0.916) indicate that the adopted model for assessing sagittal inconsistency is excellent. Using the W angle analysis is characterized by good reproducibility (0.705). On the other hand, the value of 0.484 for the Sar angle indicates only sufficient reproducibility. Individual physicians find the reference points in different ways and therefore the parameter indicates high variability.

A limitation of the R² index is the lack of information on the causality of differences in the reproducibility of measurements. It only informs us about the relationship between individual series of measurements and different assessors.

2. ICC (3.1) internal correlation coefficient.

The results of the reproducibility of measurements by individual orthodontists using ICC are included in Table 5. The assessment was made based on ICC (3.1), which is used when the measurements are performed by the same examiners and all assessments refer to the same radiographs each time. The individual angles were characterized by very good reproducibility (Sar and W) and excellent reproducibility (ANB). The analysis of the ANB angle is characterized by the highest reliability based on the repeatability and reproducibility indices. Slightly lower but still satisfactory values ​​were noted for the Sar and W angles.

Table 5.

ICC for individual angles ANB, Sar and W.

	ANB	Sar	W
ICC (3.1)	0,946	0,679	0,688

Sensitivity and specificity assessment

The diagnostic utility of the ANB, Sar, and W angle classification system for classifying patients into one of the three skeletal classes was assessed using cumulative sensitivity and specificity coefficients. Cumulative sensitivity and specificity were calculated by averaging the specificity of each. Given that the individual classes showed nonuniform numbers of observations, a weighted average was calculated to obtain accurate results, where the weight (wi) was the number of observations in each class. Sensitivity is the fraction of cases that were correctly assigned to a given diagnostic group, and specificity is the fraction of cases that were correctly excluded as not belonging to a given diagnostic group (Table 6). The ANB angle showed the highest sensitivity (1), followed by Sar (0.817) and W (0.791). Similarly, the highest specificity was shown by ANB (1.0), W (0.762) and Sar (0.721) (Table 6).

Table 6.

Cumulative values ​​of sensitivity and specificity of ANB, Sar and W angles.

	ANB	Sar	W
Cumspec	1,0	0,721	0,762
Cumsens	1,0	0,817	0,791

Knowledge of the appropriate sensitivity and specificity of measurements has important implications for the diagnostic evaluation of particularly complex cases. Both indicators are particularly important for convincing the orthodontist of a high probability of a correct diagnosis for treatment planning purposes.

The obtained values ​​of the Sar (Table 7) and W (Table 8) angle measurements indicate a change of a certain group of patients to a different class than was agreed upon after the analysis of the ANB angle. These changes were statistically insignificant.

Table 7.

Number (fraction) of respondents in groups differing in skeletal class assessment based on ANB angle and Sar angle values, chi-squared test of independence results and odds ratio values ​​and their 95% confidence intervals.

	Class I 0°≤ ANB ≤4° N = 1303	Class II and III ANB < 0° or ANB > 4° N = 2613	p-value	OR [95% CI]
57°≤ SAR ≤60°	482 (37.0%)	335 (12.8%)	< 0.001	3.99 [3.40; 4.69]
SAR < 57° or SAR > 60°	821 (63.0%)	2278 (87.2%)		1.00 (ref.)
sensitivitySens. = 0.370, specyfity Spec. = 0.872
	Class II ANB > 4° N = 1317	Class I and III ANB ≤4° N = 2599	p -value	OR [95% CI]
SAR < 57°	1150 (87.3%)	612 (23.5%)	< 0.001	22.4 [18.6; 26.9]
SAR ≥57°	167 (12.7%)	1987 (76.5)		1.00 (ref.)
sensitivitySens = 0.873, specyfity Spec = 0.765
	Class III ANB < 0° N = 1296	Class I and II ANB ≥0° N = 2620	p -value	OR [95% CI]
SAR > 60°	952 (73.5%)	385 (14.7%)	< 0.001	16.1 [13.6; 18.9]
SAR ≤60°	344 (26.5%)	2235 (85.3%)		1.00 (ref.)
sensitivitySens = 0.735, specyfity Spec = 0.853

Table 8.

Number (fraction) of respondents in groups differing in the assessment of the skeletal class based on the ANB angle and W angle values, results of the chi-square test of independence and the values ​​of odds ratios and their 95% confidence intervals.

	Class I 0°≤ ANB ≤4° N = 1303	Class II and III ANB < 0° or ANB > 4° N = 2613	p-value	OR [95% CI]
55° ≤ W ≤ 57°	343 (26.3%)	210 (8.0%)	< 0.001	4.09 [3.39; 4.93]
W < 55° or W > 57°	960 (73.7%)	2403 (92.0%)		1.00 (ref.)
Sensitiviti Sens. = 0.263, specyfity Spec. = 0.920
	Class II ANB > 4° N = 1317	Class I and III ANB ≤4° N = 2599	p -value	OR [95% CI]
W < 55°	1181 (89.7%)	699 (26.9%)	< 0.001	23.60 [19.37; 28.76]
W ≥55°	136 (10.3%)	1900 (73.1%)		1.00 (ref.)
Sensitiviti Sens = 0.897, specyfity Spec = 0.731
	Class III ANB < 0° N = 1296	Class I and II ANB ≥0° N = 2620	p -value	OR [95% CI]
W > 57°	1008 (77.8%)	475 (18.1%)	< 0.001	15.81 [13.41; 18.63]
W ≤57°	288 (22.2%)	2145 (81.9%)		1.00 (ref.)
Sensitiviti Sens = 0.778, specyfity Spec = 0.819

Discussion

Correct cephalometric analysis plays a key role in orthodontic diagnosis^{1–3,22–38}. The imperfections of the measurements used so far necessitate a systematic search for new reference points that will be less sensitive to positional changes resulting from growth or orthodontic treatment. The occurrence of new landmarks and angular measurements in cephalometric analysis was noted by Kotuła et al.¹ in a systematic review published in 2022. The authors noted that Bhad et al.²⁷ and Sonahita et al.²⁸ discussed in detail new cephalometric measurements based on the G and M points. These points were included in earlier studies by Neel et al.²⁴ and Gupta et al.²⁵. They substantiated the opinion that the G and M points are more reliable than the previously used A, N, and B points. The authors of the review¹ suggested conducting reliable, randomized comparative studies, including comparable patient groups, to assess the reliability of the new diagnostic parameters. The review’s conclusions indicate the need to assess the repeatability and reproducibility of determining anthropometric points, cephalometric measurements, and their potential use in assessing sagittal discrepancies. They also indicate that studies should be performed by experienced and well-trained orthodontists. For proper comparative evaluation of studies, it is important to use standardized methodological criteria and compare the significance of individual measurements using the same statistical analysis¹. This study followed the suggestions of the review authors¹. The Sar and W angle measurements were based on the new G and M cephalometric points. These were selected based on a comparable method for determining both angles. The need to assess both parameters stems from the need to supplement the knowledge obtained from determining the Tau angles^1–3,25 and Yen angles^1–3,24 based on the G and M points described in the literature. Previous studies by Bhada et al.²⁷ and Sonahit et al.²⁸ demonstrated greater stability of the G and M points. Previous studies have also confirmed that the points that comprise them are less susceptible to changes related to growth or orthodontic treatment^{1–3,27,28,39}. Therefore, these measurements represent an innovation compared to previously standardized measurements for assessing the sagittal relationships of the maxilla and mandibula, such as the ANB angle, the beta angle, Wits analysis, the AF–BF angle, the MM–AB angle, the AH–BH measurement, and the Harvold index^31,33,35,37. The main concerns regarding the reliability of the measurements used include the instability of the S, N, Po, Or, A, and B points during growth, changes in their position during orthodontic treatment, and the unreliability of their correct location on the cephalometric image^31[,3335[,37. Doubts often arise regarding the repeatability and reproducibility of landmarks^{1–3,31,35,37,39}.

Previous publications^{2,22,23,27,28,32,38}, compared the repeatability and reproducibility of individual anthropometric points. In this study, similarly to the study by Kotuła et al.³, and Kotuła et al.³⁹, an attempt was made to assess the reliability of the angular parameters. The studies by Maheen Ahmed et al.³⁶ regarding various analyses of cephalometric examinations, as well as the study by Kotuła et al.^2,3 comparing the analyses of ANB angles with the Tau and Yen angles consistently showed that the ANB angle is the most accurate and reliable indicator of maxillomandibular relations in the sagittal plane. Similar conclusions were also reached by Ahmed et al.³¹, Jedliński et al.³⁵, and Kumar et al.^37[,31[,3537,. The current study concluded a similar conclusion. Although most results allow for a reliable assessment of the sagittal relations of the jaw bases, the ANB angle is still characterized by the highest reliability. ANB measurements, both in terms of repeatability and reproducibility, show fewer errors compared to Sar and W measurements.

These results are mainly related to the narrower coaxial dispersion of angle values determined by the same and different observers. The results of ANB, Sar, and W angle measurements based on R&R analysis, including their repeatability (4.3%; 16.4%; 15.8%), reproducibility (0.7%; 0.8%; 0.5%), interindividual variability (93.6%; 66.2%; 67.2%), and overall repeatability and reproducibility (6.4%; 33.8%; 32.8%), indicate a direct dependence of all angles on individual variability. Kotuła et al.² reached similar conclusions when publishing the results of ANB and Tau measurements, including repeatability (1.61%; 4.3%), reproducibility (0.92%; 3.94%), inter-patient variability (97.47%; 91.76%), and overall repeatability and reproducibility (2.53%; 8.24%), emphasizing the dependence of ANB and Tau angles on individual variability. A recently published study by Kotuła et al.³ also allowed for similar conclusions, comparing ANB, Tau, and Yen angles. The results of the ANB, Tau, and Yen measurements support this finding, including repeatability (4.3%; 15.8%; 14.1%), reproducibility (0.7%; 1.4%; 0.2%), interpatient variability (93.6%; 66.5%; 69.8%), and overall repeatability and reproducibility (6.4%; 33.7%; 30.2%).

The study by Kotuła et al.² assessed the reliability of the ANB and Tau angles not only based on the R&R index. Reliability results were also assessed using the Dahlberg formula (0.265–0.665; 0.891–1.639) and the ICC (0.841–1; 0.147–0.624). All of the aforementioned metrics showed that the participating orthodontists determined ANB with more than three times higher reliability than Tau. The authors also assessed the coaxial dispersion of the coordinates of the points dependent on the OX axis—points A, N, B—or the OY axis—points G, M. The authors found that the discrepancy was influenced by the smaller variability of the horizontal coordinates of the points, which have the greatest impact on the measurement error of the angle on which the sagittal relationship depends. In this study, the assessment of the position of the points in the Cartesian coordinate system was not taken into account when assessing the reliability of the measurements. The obtained values of the ANB, Sar, and W angles based on the Dahlberg formula (0.68%; 2.33%; 2.33%) and the ICC (0.964; 0.679; 0.688) confirm the previous conclusions. The evaluation of the R2 coefficient allowed us to accept the results of the previous studies by Bhad²⁷ and Sonahit²⁸, which determined the value of the Sar angle (0.484) and the W angle (0.705), which indicated a lower value compared to the ANB values (0.916). The value of this study lies in the increased reliability of the results associated with a double-blind study in which observers were unaware of the patients’ clinical parameters and the purpose of the analyses, ensuring an objective assessment. Both studies highlighted the improved effectiveness of the ANB angle as a standard parameter for assessing sagittal relationship.

The validity of this study regarding the comparison of the reliability of the ANB, Sar, and W angles was further enhanced by the use of a double-blind design, in which observers were unaware of the patients’ clinical examination results or the purpose of the analyses. This allowed for an objective assessment despite several limitations that influenced the final assessment value.

A previous study by Kotuła et al.^2,3 found that there was no clear evidence to support the claim that the Tau angle based on the M and G points is resistant to mandibular rotation. It also failed to confirm that it consistently yields stable results for correctly defining sagittal defects. Such claims might suggest similar conclusions for the Sar and W angles, which are based on the same reference points. Unfortunately, this claim could not be confirmed. The authors indicate that errors associated with the identification of anthropometric points are more significant than errors in the reliability of the ANB and Tau angle measurements.

Bhad²⁷ with respect to Sar and Sonahita²⁸ with respect to W attempted to assign the appropriate Sar and W angle values to one of three skeletal classes. This study, conducted on a selected group of patients with orthognathic faces without considering the vertical component assessment, slightly changed the cutoff values for skeletal classes determined using Sar and W angles (Table 2).

Another value of this study is the assessment of the sensitivity and specificity of individual measurements. The analysis allows us to determine the ability of Sar and W measurements to correctly identify cases (sensitivity) and correctly exclude cases that should not be classified as malocclusions (specificity). The sensitivity values for determining Class I, II, or III malocclusions using the Sar/W ratio (0.370/263; 0.873/0.897; 0.735/0.778) indicate that patients were correctly classified into the appropriate class in 37% and 26.3%, respectively. 87.3%/89.7% and 73.5%/77.8% of cases. The higher the measurement sensitivity, the lower the risk of missing individuals with skeletal class II or III during diagnosis. Such high values indicate that both parameters effectively identify malocclusions, although a significant proportion of patients are also assessed incorrectly. The reasons for this phenomenon can be traced to so-called borderline cases between individual malocclusions. We can also expect a similar result when evaluating the previous ANB angle measurement. High sensitivity means that the risk of missing an actual malocclusion is low, which is beneficial in clinical applications in orthodontic diagnosis.

In assessing the measurement sensitivity, Sar Bhad²⁷ and W – Sonahita²⁸ indicated a sensitivity close to 100%, which was not confirmed in this study.

By assessing the specificity of the measurements, we are able to avoid false positive results, i.e., incorrect classification of healthy individuals as having malocclusions. Specificity values for determining Class I, II, or III malocclusions using the Sar/W ratio (0.872/0.920; 0.765/0.731; 0.853/0.819) indicate that each parameter correctly identifies 87.2%/92.0%, 76.5%/73.1%, and 85.3%/81.9% of individuals who do not have these specific sagittal malocclusions. At the same time, between 8% and nearly 27% of healthy individuals may be misdiagnosed as requiring orthodontic intervention due to sagittal discrepancy.

Clinical application of the obtained sensitivity and specificity results for both parameters may necessitate enhanced diagnostics using a different analysis: ANB or Wits. This will be particularly important in diagnosing borderline values between Class I and II or Class I and III. The obtained results indicate that Sar and W angle analyses cannot yet replace ANB and should be used with caution, possibly as a supplement to diagnose sagittal relationships.

Both new parameters are suitable for use in screening tests, minimizing the risk of missing patients with sagittal malocclusions.

Limitations

The study has several limitations that prevent radical generalization of the obtained results.

The fist is the small number of patients. Nevertheless, obtaining nearly 4,000 measurements for each variable allows for some generalizations.

Another limitation is the wide age range of the patients and the lack of gender differentiation, which makes it difficult to generalize the results to a specific patient group.

It is worth noting the small number of 22 orthodontists who conducted the study and the limited training intensity. This is particularly important when comparing the commonly known ANB angle with new measurements based on the introduction of the new M, G, and T points. This may also affect the interpretation of the obtained results.

Another limitation may be the inability to conduct a comparative analysis using a longitudinal study, which prevents broad generalizations from this study.

Limitations of this study include limiting the analyses to the repeatability and reproducibility of cephalometric measurements without considering the reliability of cephalometric points along the x- and y-axis.

The cephalometric analysis software itself also had limitations, although its reliability did not raise concerns for the authors. Another limitation is the performance of cephalometric analyses on a selected group of patients with an orthognathic face.

Future directions of research

Further research is needed, particularly randomized controlled trials (RCTs) comparing alternative methods for assessing sagittal differences. Research should be undertaken comparing analyses based on M and G points not only with previously performed ANB angle analysis, but also with Wits and/or Beta angle assessments and other parameters used in assessing sagittal discrepancy. These should include increased sample size, both in terms of the number of participating orthodontists and the number of patients. Studies should also consider assessing the relationship between new parameters, considering the relationship between sagittal and vertical discrepancies. Studies should be conducted blinded to the radiograph, the patient, and the assessing orthodontist.

The introduction of standardized training for orthodontists, lasting longer than the 5 h offered in the current study, should be considered crucial in the design of future studies. This should reduce interobserver variability and improve the consistency and reliability of results.

Future studies should consider comparing the use of digital tools for cephalometric analysis, which can increase accuracy and reduce human error.

The possibility of using artificial intelligence to perform cephalometric analyses should also be considered.

The authors believe that complementary studies are needed to assess the long-term reliability of these angles in predicting orthodontic treatment outcomes, which would provide valuable clinical information for practicing orthodontists.

Conclusions

The presented study confirms the feasibility of using the new Sar and W cephalometric measurements in assessing sagittal discrepancies, complementing the current ANB angle measurement.

In assessing the reliability of individual angular measurements, the ability to position anthropometric points based on orthodontists’ knowledge and experience is paramount. To improve the reliability of measurements, dentists should be trained in positioning, and computer programs should be developed to assist dentists in visualizing the entered points. Thanks to the development of artificial intelligence, it is possible to begin training artificial intelligence in this area, which could complement the capabilities of mechanical cephalometric analysis, leaving its assessment in the hands of an experienced orthodontist. The mean difference in the repeatability assessment results for all analyses discussed ranged from − 0.09° to 0.03°. Such small mean differences between two measurements performed by the same orthodontist allow for the practical use of each of these parameters in assessing sagittal discrepancy; however, the new measurements are characterized by a larger margin of error, which in extreme cases prevents their use in orthodontic diagnosis. Due to the limitations of this study, the assessment of individual parameters should be approached critically. Particular caution should be exercised in patients of developmental age and when analyzing sagittal discrepancy related to the assessment of the mandibular base angle, taking into account the vertical movement of the G-spot with growth.

The high sensitivity of 80% and specificity of 70% for Sar and W measurements allows for the effective identification of patients with malocclusions caused by sagittal discrepancy. Clinically, it has greater utility in diagnosing Class II and III malocclusions. The presented study confirms the possibility of using, with some caution, the new Sar and W cephalometric measurements in the assessment of sagittal discrepancies as a supplement to the current ANB angle measurement.

Author contributions

JK, AEK and JL conceived the study; JK, KK, ES, and AEK collected the data and were the main co-authors of the manuscript writing. JK, MK, JL, BK, and MS analyzed and interpreted the data. JK, KK, JL, AEK, and MS created the tables and prepared the references. All authors read and approved the published version of the manuscript. All authors have read the manuscript and accept its content.

Funding

The study was funded by the Medical University of Wroclaw SUBZ.B031.25.011, SUBZ.B033.25.014, SUBZ.B190.25.024.

Data availability

The datasets used and/or analysed during the current study are available from the corresponding autor on reasonable request.

Declarations

Competing interests

The authors declare no competing interests.

Ethical approval

The study was conducted in accordance with the Declaration of Helsinki and was approved by the Bioethics Committee of the District Medical Chamber in Zielona Góra (decision 01/173/2023 of March 6, 2023).

Consent to participate

All patients signed informed consent before entering the study.

Informed consent

of the patient to participate in the research.

All those who received information about the results of the research, about their impact on the patient’s health, about the possibility of occurring immediately after the start of the research and signed conscious consent to participate in the research.