Peer Assessment Rating (PAR) Index as an Alternative for Orthodontic Treatment Need Decision in Relation to Angle Classification

Abstract

Objective

This study aimed to determine cut-off points for the Peer Assessment Rating (PAR) index in relation with Angle classification to use as an alternative index for the treatment need assessment.

Methods

This study included 607 orthodontic patients aged between 9 and 18 years. Angle classification, PAR, and dental aesthetic index (DAI) scores were determined. The DAI was used as the gold standard to evaluate the subjects for treatment need. The receiver operating characteristics (ROC) analysis was used to evaluate the PAR index in relation to treatment need assessed by DAI.

Results

The mean PAR scores for Class I, II, and III malocclusions and total sample were 17.54, 14.27, 18.7, and 20.04, respectively. The areas under the ROC of PAR scores in relation to the DAI assessment were found as 68.3% for the total sample, 66.6% Class I, 59.2% Class II, and 71.3% Class III malocclusions. For the total sample, the optimum cut-off PAR score was 14 in relation to DAI assessment. The cut-off scores for Class I, II, and III malocclusions were 13, 11, and 16, respectively, but considering psychosocial aspects, the recommended score is 14 for Class III.

Conclusion

The PAR index can be considered to have an acceptable level of validity for the assessment of orthodontic treatment need regarding Angle classification.

INTRODUCTION

Because of the increasing awareness of acceptable and attractive physical appearance, there is a growing interest to dental aesthetics (1, 2). Hence, the request for orthodontic treatment has increased. In the countries where the cost of orthodontic treatment is being covered by the public dental services for children up to limited ages, an overcrowding in orthodontic clinics and delay in treatment are observed (3). Accordingly, researchers underline the significance of developing treatment priority indices that may allocate limited health resources and decrease the waiting period and overcrowding in orthodontic clinics (3). Based on this premise, several orthodontic indices have been designed to evaluate the orthodontic treatment need by means of malocclusion severity (4–12).

The widely used indices, the index of orthodontic treatment need (IOTN) (5, 6), index of complexity outcome and need (ICON) (8, 9), and dental aesthetic index (DAI) (10–12), were validated to assess an individual’s need of orthodontic treatment that incorporated a measure of the psychosocial impact of malocclusion as well (the aesthetic component). However, some authors claimed that the aesthetic component alone might be insufficient to determine the orthodontic treatment need (13). Additionally, index scores differ in certain cases, and orthodontic treatment need may be identified differently according to each index. Thereby selection of the patients depends on the index used due to the inherent differences in how certain occlusal features are scored (6, 11, 12, 14, 15). In support, a recent study reported 20% of disagreement in treatment need determination for the same individuals in accordance to DAI and ICON (16). DAI may not be sensitive to the specific occlusal problems and treatment requirements of some patients. DAI does not consider the amount of overbite that may strongly influence the determination of treatment need of patients with Class II, Division 2 malocclusion. In addition, DAI scores neglect the edge-to-edge incisor relationships (14, 17). The other index ICON neglects midline diastema, mandibular spacing and crowding, and the amount of overjet.

The Peer Assessment Rating (PAR) index evaluates the dento-occlusal changes in an accurate and rapid way, and it can be used as a treatment need index by both evaluating aesthetic and occlusal features (18–21). The PAR index measures crowding and spacing in both maxillary and mandibular dental arches, buccal segment occlusion (anteroposterior, vertical, and transverse), overjet (including anterior crossbite), overbite, edge-to-edge relationship and openbite, midline discrepancies, and impaction of teeth.

Due to inherent differences in how certain occlusal features are scored between the indices, these factors bring a priority for concise decision of treatment to orthodontists. Considering all these, usage of alternative indices instead of only one index for orthodontic cases in evaluation of the treatment need would be logical and more reliable.

In literature, studies evaluate PAR index as an alternative index to decide orthodontic treatment need (18, 19). The PAR index was developed without recommended cut-off points, and it could not be used in orthodontic treatment need assessment, but as a method of measuring malocclusion and efficacy of treatment. Considering this, in those studies, the cut-off points were calculated and reported between 10 and 22 in maximum accuracy (18, 19). Firestone et al. (18) concluded that the PAR index could be used in the assessment of treatment need by using a PAR score of 17 as the optimal cut-off point in the UK and the US. Recently, Soh et al. (19) reported the PAR cut-off scores as 17 and 20 in relation to dental health component (DHC) and the aesthetic component (EC) of the index of orthodontic need assessment in Asian men.

In a recent study, it was emphasized that the determination of orthodontic treatment need should be performed in conjunction with Angle classification due to the prominent differences in scoring certain occlusal features of the indices (16). Therefore, assessment by the PAR index with Angle classification can be recommended for a more precise and alternative decision system to verify the treatment need. To our knowledge, no study in literature has determined the cut-off points of PAR index according to Angle classification. This study aimed to calculate the cut-off points and to determine the PAR index as an IOTN according to Angle classification.

METHODS

This study included 607 consecutive patients (227 males, 380 females), aged 9–18 years old, randomly selected among those admitted to the Department of Orthodontics. Patients with large restorations/crowns, serial extractions or cleft lip and palate, having previous orthodontic and/or prosthetic treatment were excluded from the study.

The study was approved by the Ethical Committee of the Gazi University (2018-65).

The data on demographic characteristics, Angle classification and DAI, and PAR scores were recorded by two specialists. Two examiners discussed diagnostic criteria of malocclusion. The inter-examiner reliability was assessed. Additionally, dental casts were obtained from the patients after clinical examination to assess any variations in the data of the PAR index originating from cast or clinical scores. To ascertain the reliability of the casts and clinical examination, statistical analysis was performed. The intra-rater correlation coefficient (ICC) for the measurements was found as 0.95 indicating high reliability.

Patients were considered according to Angle classification as Class I, II, and III (22). The DAI and PAR scores were determined for each patient. DAI was used to evaluate the subjects for treatment need. According to treatment need, malocclusions were divided into two groups (23): no treatment need [DAI ≤ 25 (grade 1)] and treatment need [DAI > 25 (grades 2–4)] to compare with PAR scores and find the cut-off points for the PAR index with regard to Angle classification of malocclusions.

Statistical Analysis

Statistical analysis was performed with Statistical Package for Social Sciences version 15.0 (SPSS Inc.; Chicago, IL, USA). Intra-examiner reliability of the data obtained two weeks later was evaluated by the ICC, and the inter-examiner agreement was determined by the Kappa statistics. Descriptive statistics were used to report the PAR scores among the types of Angle malocclusions (Table 1). The receiver operating characteristic (ROC) analysis was used to calculate an optimized cut-off score for the PAR index in relation to treatment need. The specificity, sensitivity, and positive and negative predictive values were calculated. Sensitivity is the percentage of cases with treatment need, and specificity is the percentage of cases without treatment need as identified by the index. The negative and positive predictive values represent the percentages of patients that are correctly determined as not needing (negative) or needing (positive) treatment (24).

Table 1.

The PAR scores in accordance with Angle classification

Angle Classification	Mean	SD	Median (Min–Max)	p
Class I (n=193)	14.27	6.52	14 (2–38)	<0.001
Class II (n=302)	18.70	7.35	18 (2–43)
Class III (n=112)	20.04	8.74	18 (5–52)
Total (n=607)	17.54	7.72	17 (2–52)

RESULTS

For this study, the sample size of 64 subjects per group at α=0.05 yields a statistical power of 80%.

The ICC for repeated measurements for each examiner was close to 1.0, indicating high reliability. Inter-examiner agreement was found to be 0.87, which is within acceptable limits.

The mean PAR scores for Class I, II, and III malocclusions and the total sample were 14.27, 18.7, 20.04, 17.54, respectively (Table 1).

The areas under the ROC of PAR scores in relation to the DAI assessment were 68% for the total sample, 66.6% for Class I, 59.2% for Class II, and 71.3% for Class III malocclusions as shown in Figure 1.

Figure 1.

The ROC curves of PAR scores in relation to DAI assessment for total sample, Class I, Class II, and Class III malocclusions

For the total sample, the optimum cut-off PAR score for DAI was found to be 14 (sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were 69.82, 56.86, 82.8, 38.8, respectively) (Table 2).

Table 2.

The areas under curve, optimum cut-off, positive and negative predictive values of PAR scores in relation to the DAI assessment

DAI
	AUC	p	Cut-off score	Sensitivity (95% CI)	Specificity (95% CI)	PPV (Ratio)	NPV (Ratio)	False negative (n)	False positive (n)	Total correct prediction (%)
Total sample	0.683	<0.001**	14	70 (65–74)	57 (49–64)	0.83	0.39	30	43	67
Class I	0.666	<0.001**	13	61 (51–70)	64 (53–74)	0.70	0.54	39	36	62
Class II	0.592	<0.05*	11	86 (81–90)	32 (20–46)	0.87	0.30	14	68	78
Class III	0.713	<0.001**	16	74 (63–82)	76 (56–88)	0.91	0.45	26	24	74

AUC: Area under curve, PPV: Positive predictive value, NPV: negative predictive value,

^*p<0.05,

^**p<0.001

In accordance to Angle classification, the cut-off PAR scores were found to be 13 for Class I, 11 for Class II, and 16 for Class III malocclusions.

DISCUSSION

The generally used indices’ scores differ in certain cases, and orthodontic treatment need may vary according to each index on which selection of a patient depends. Supporting this, 20% overall disagreement between the DAI and ICON indices was reported in quantifying orthodontic treatment need in a recent study (16). Each index has its own limitations; therefore, the decision of orthodontic treatment depending on the diversity of scores can be conflicting. The usage of alternative indices instead of only one index for orthodontic cases in evaluation of the treatment need to certify the decision would be more reliable. Therefore, in this study, it was aimed to calculate the cut-off points for PAR index in accordance to Angle classification by DAI.

Given that the DAI is recognized by the World Health Organization Oral Health Survey as a cross-cultural international orthodontic index for treatment need assessment (10) and as significant positive correlations were found between PAR and DAI scores for Angle classifications in the previous study (16), DAI was used as the gold standard in this study.

The ROC curve is a graph of sensitivity (y-axis) vs. 1 - specificity (x-axis). The area under an ROC curve defines the capability of the test to distinguish the subjects with and without the disease. In other words, the area is related to the ability of a test to correctly detect normal versus abnormal. An ideal diagnostic test could have a 100% area under the ROC curve (24). The goal of the ROC curve is to determine an optimized cut-off point defining the patient’s treatment need. It is the point nearest to the left upper corner of the ROC curve graph (24).

In previous studies, the PAR index was defined to be a proper method to determine orthodontic treatment need in the US, UK, and Asian populations (18, 19). Soh et al. (19) reported that the areas under the ROC curves for PAR index were 84% and 94% for the DHC and EC assessments, respectively. Firestone et al. (18) defined this as 97%. The results of these studies indicated the high validity of the PAR index. The results of our study can be considered as compatible with those in other studies (18, 19). In this study, the areas under the ROC curves for PAR index were 68.3% for the DAI assessments. These percentages were lower than the Soh et al.’s (19) results due to the preference of different indices as gold standard.

Previously, Firestone et al. (18) had defined a PAR cut-off score of 17. Whereas, in another study, Soh et al. (19) reported cut-off scores as 17 and 20 in relation to DHC and the EC of the IOTN, respectively. Although a direct comparison could not be made between those studies and ours, the optimum cut-off PAR score was found to be 14 in relation to the DAI assessment. Differences in the cut-off points to determine the treatment need among the studies may be the result of methodological variations. The gold standard in the study of Firestone et al. (18) was the expert opinion of 15 orthodontists; and in the study performed by Soh et al. (19), two different indices were considered. In this study, the DAI was used as a gold standard in determining the cut-off points of PAR in this study, whereas orthodontic specialist’s decisions or other indices might have been used as the gold standard to evaluate the treatment need of the patients. This was the limitation of this study.

Hamdan and Rock (20) reported that occlusal features vary in importance in different classes of malocclusion. In association with guidance for subtle verification of features, it would be more valid if each class of malocclusion were assessed separately. In consequence, with this concept, when the cut-off scores were determined regarding the Angle classification in this study, it was seen that the cut-off PAR scores varied. It was 13 for Class I (p<0.001), 11 for Class II (p=0.045) and 16 for Class III (p=0.001) malocclusions in relation to the DAI assessment. In this study, the p-values for each Angle classification were adequate to define that the PAR index can actually diagnose the orthodontic treatment need. If the p-value is small, then it is possible to conclude that the index actually distinguishes the patients who need treatment.

As the cut-off score for the total sample was 14, which is greater than that of the value of Class I and II malocclusions and lower than Class III in this study. Decisions about using either the total cut-off point or a specific cut-off score for each malocclusion are subject to discussion. To determine a particular value as a cut-off point, the costs and benefits should be considered. The important factors in choosing the cut-off point are the economic, risks of missing the treatment need (false negative) that leads to the delay of orthodontic treatment for patients with severe malocclusion and the increase in waiting times in countries where the cost is covered by the public services or incorrectly identifying the treatment need (false positive) that enables allocation of restricted health resources in regard to treatment priority (18, 24).

Considering these, it would be beneficial to use the cut-off score in accordance to each Angle Class I and II malocclusions to precise the actual treatment need. If you choose the lower cut-off score, you would correctly identify the patients who need treatment, but you would also diagnose the treatment need in more cases that do not need treatment (18).

Almost all Class III patients need orthodontic treatment; clinicians may prefer using the lower cut-off score of 14 instead of 16 for Class III malocclusions. If the score 16 is chosen as the cut-off value, there is a risk of missing some of the patients who would need treatment. If the lower threshold is preferred, almost all of the patients with treatment need will be identified. Besides, negative psychosocial effects of having a malocclusion, missing the opportunity to benefit from growth modification and the costs of supplementary treatments such as orthognathic surgery will be minimized.

Each index has its own limitations and restrictions in identifying orthodontic malocclusions. Angle classifications support the international communication worldwide; and as observed from our results in differing cut-off scores, they should be taken into consideration. The PAR index as an alternative to verify the orthodontic treatment need will provide a light on confirming conflicting or varying decisions in clinical evaluations.

CONCLUSION

The results of this study indicated that the PAR index could be used as an alternative index in evaluation of the treatment need in conjunction with Angle classification. For the total sample, the optimum cut-off PAR score was approved to be 14 in relation to DAI assessment. In the decision of treatment need, the usage of specific cut-off scores of 13 and 11 for Angle Class I and II, respectively, and a total score of 14 for Class III malocclusion are recommended.