Comparison of the applicability of four odontological methods for age estimation of the 14 years legal threshold in a sample of Italian adolescents.

Abstract

The 14-years age threshold is especially important in Italy for criminal, civil and administrative laws. Several methods relying on dental calcification of the teeth, up to the second molar, are used for the evaluation of age in childhood. The objective of the research was to compare the inter-rater agreement and accuracy of four common methods for the dental age estimation – Demirjian (D), Willems (W), Cameriere (C) and Haavikko (H) – in a sample of Italian adolescents between 11 and 16 years. The sensitivity and specificity, and the different level of probability, according to the peculiarities of Italian criminal and civil law, were compared for the methods examined, considering the threshold of 14 years. The sample was composed of 501 digital OPGs of Italian children (257 females and 244 males), aged from 11 years and 0 days to 15 years and 364 days. The maturation stage of the teeth was evaluated according to D, W, H and C methods by three independent examiners. Mixed statistical models were applied to compare the accuracy and the errors of each method. The inter-rater agreement was high for the four methods and the intraclass correlation coefficients were all ≥ 0.81. Methods H and C showed a general tendency to underestimate the age in the considered sample while the methods D and W tended to overestimate the child’s age. In females, D and W were more accurate than C, which is more accurate than H. In the males, W is the most accurate method even though it over-estimated age. Considering the 14-years threshold, the sensitivity of D and W methods is quite high (range 0.80; 0.95) and specificity is low (range 0.61; 0.86). The principal findings of the research are: the W and D methods are much more accurate than C and H, but they tend to overestimate the age. The C method largely underestimates the age (by ~1 year) for both genders and for all operators. H is unsuitable for dental age estimation in the Italian population, while W and D yielded high sensitivity but low specificity, thus producing high rates of false positive results. The choice of method to estimate if an Italian child has reached the 14-years legal threshold should mainly be chosen according to the different legal milieu (if civil or criminal) and the gender of the examined individual. The age assessment in criminal case must be prudently managed.

INTRODUCTION

There is now full accordance in the literature that the methods of dental age estimation (DAE) relying on the evaluation of the mineralisation and growth stage of the teeth seem to be scarcely affected by local and systemic factors (1-3) but are dependent on the genetics of the populations as they show an ethnic variability (4, 5). The estimation of age through the study of the calcification of the permanent teeth has been demonstrated to provide reliable and accurate methods and results. The dentition up to the second molar is useful in the procedures in which subjects <16 years are involved, and the remaining third molars are evaluated after this age threshold. Most studies on DAE, and especially those concerning the dentition up to the second molar and the threshold of 14 years, have focused on the methods or the comparison of methods with little consideration of the accuracy of the estimation in cohorts around this age and in classifying individuals in respect to the threshold. In contrast, judges, institutions and agencies generally ask not simply an age estimation but more strictly an age assessment with respect to the legal threshold of age. The age evaluation procedure of the living minor is usually required to determine if the subject is accountable for his actions in criminal law. According to Italian law, the lowest age threshold for criminal accountability is 14 years; the actual accountability between 14 and 18 must be established case by case mainly relying on a psychological maturity assessment basis. In another context - mostly of administrative or civil law significance - the age evaluation of the minor is necessary to determine if the subject shall undergo specific obligations (educational, for instance) or should receive specific aides or other providences from the state administration (immigration, adoption, motorcycle driver license, passport release - to cite only the most common cases). Criminal law requires that the age over the threshold has to be assessed “beyond any reasonable doubt” (according to the Latin aphorism “in dubio pro reo”) and, if doubts persist, the lower age has to be assigned by the judge/court. The reasonable doubt is logically connected to scientific evidence provided by experts, who are aware that only evidence endowed with very high probability, at least > 90%, may turn useful in criminal proceedings and meet the standard of proof. In criminalproceedings the percentage of false positives deserves special consideration, because the overestimation ofthe age threshold is the less desirable error both from the legal and the ethical point of view.

On the contrary, in civil proceedings the general rule is just “more probable than not”, so that a percentage of probability just above the 50% may suffice.

Furthermore, the different kinds of error in assessing the age have to be differently regarded with respect to the criminal or civil law context. The false positive is the worst and the least desirable error for criminal law, given the heavy legal consequences that it implies. In civil proceedings the false positive and false negative tend to have the same meaning: they are both errors, but the first is not necessarily worse than the second. In fact, when the age is over the threshold the subject may be entitled to some civil rights.

To achieve the best accuracy, both in DAE and in assessing if a child has attained the legal threshold, it is therefore important to test the different dental methods of age estimation on a specific ethnic group. Few papers have been published about the comparison of the accuracy, sensitivity and specificity of the different methods for DAE in a sample of Italian children at the 14-years age threshold. Moreover, many studies have reported that the accuracy of the method is dependent on the age of the subject and that the uncertainty of the predictions grows in the older cohorts (6). Therefore, it is meaningful to evaluate the methods comparing the sensitivity, the specificity and the accuracy for age cohorts close to the threshold. To address this issue, we selected ~50 Italian subjects per gender, and an age cohort around 14 years (age span 11-16), and compared the results obtained through four of the most known and adopted methods for DAE: the Demirjian’s, the Willems’, the Cameriere’s and Haavikko’s methods. The comparative analysis evaluated the inter-rater agreement, the mean error of estimations, the sensitivity and specificity of the selected methods as indexes that allow the evaluation of the accuracy of the method to predict the attainment of the 14-yrs threshold (5110 days of age).

MATERIALS AND METHODS

A total of 501 digital orthopantomographs (OPGs) of Italian children of Caucasian origins were taken from three selected clinical radiology offices in northern, central and southern Italy. As shown in table 1, the sample consisted of 244 males and 257 females, almost equally divided in age cohorts from the age of 11 years (4015 days) to 15 years and 364 days (5839 days) (Table 1)

Table 1. Frequency and percentage of the subjects for age and gender.

Age	Females (%)	Males (%)
11	56 (22)	46 (19)
12	51 (20)	46 (19)
13	47 (18)	43 (18)
14	50 (19)	56 (23)
15	53 (21)	53 (22)
All	257 (100)	244 (100)

The chronological age from the OPGs has been recorded as number of days. All the individuals of the sample were healthy and the OPGs had been taken for clinical control purposes. Exclusion criteria were the following: systemic diseases, premature birth, congenital anomalies, tooth agenesia, endodontic treatments, large carious lesions involving the dental pulp, gross mandibular pathologies, poor quality X-rays. 103 subjects (40%) in the female group, and 109 (45%) in the male group, were 14-years old or younger.

The methods utilised to analyse the OPGs and estimate the age were:

• The Demirjian’s method for seven teeth (D). In the conversion table we chose the estimated age (EA) as the age at the 50th centile (6, 7);

• The Willems’ method (W), specifically the conversion score elaborated with the polynomial regression system for the Belgian sample (8-10). The EA is calculated at the 50^th percentile;

• The Cameriere’s (C) method using the European formula (11-16), as it is presented in the AgEstimation Project website (http://agestimation.unimc.it);

• The Haavikko’s (H) method at the 50th centile (17).

Three expert forensic odontologists, all of them blind to the chronological age of the subjects, analysed all the OPGs.

A descriptive analysis of the sample was performed and the chronological age of the 501 subjects was distributed in age and gender.

An inter-rater agreement test was calculated on the estimations of the age provided by the three experts for the whole sample. For this analysis, the two-way intraclass correlation coefficient (R) was used. To evaluate the results of the agreement, the standards of Fleiss were used (18). Values of R below 0.4 may be taken to represent poor agreement, value above 0.75 may be taken to represent excellent agreement and values between 0.4 and 0.75 may be taken to represent fair to good agreement (18).

The difference between estimated and chronological age was calculated to evaluate the accuracy of each method (EA-CA). A positive figure indicates an over-estimation and a negative figure indicates an under-estimation of the age. The mean and standard deviation of each method and gender were calculated for each examiner.

To test the difference in accuracy between the four methods adopted, two linear mixed models were applied. The two mixed models were performed for males and females.

For each mixed model, the subject represented the random effect, while the fixed effects were represented by the examiner, the method and the interaction between the examiner and the method. The outcome variable was the difference between the estimated age and the chronological age. Post hoc differences between examiners and methods were tested with the Tukey Honestly Significant Difference test.

To test the discrimination accuracy, considering the threshold of 14 years, the subjects were dichotomised into two cohorts at the cut-off threshold of 5110 days of age. Sensitivity and specificity were calculated for the Demirjian (D) and Willems (W) methods and for each examiner and gender. Sensitivity in this context is defined as the probability of correctly estimating a subject who is 14-years old or above. Specificity in this context is defined as the probability of correctly estimating a subject who is <14-years old. Sensitivity and specificity for the Haavikko (H) and Cameriere (C) methods have not been analysed because they allow an estimation of 14 years as maximum and both of them resulted to underestimate largely the age of the sample so that we would obtain in every case a sensitivity and a specificity respectively equal to 0 and 1. Comparison of sensitivity and specificity were tested for Demirjian and Willems methods with the McNemar test for each examiner and gender.

MedCalc ® version 12.3.0.0 and JMP ® version 9.0 were used for the statistical analysis. Significance was set at 0.05.

RESULTS

The intraclass correlation coefficients (R) and their confidence intervals for each method are shown in Table 2 (Table2). The estimates of R are quite high and all the lower limits of the confidence intervals

Table 2. Intraclass correlation coefficients (R) for the 3 examiners on 501 radiographs. R: Intraclass correlation coefficient. CI95%: Confidence Interval at 95%.

Method	R	Lower limit CI95%	Upper limit CI95%
Cameriere	0.84	0.80	0.87
Demirjian	0.81	0.76	0.85
Haavikko 50	0.83	0.78	0.86
Willems 50	0.88	0.86	0.89

are above 0.75, representing an excellent agreement among the examiners for all the methods (18).

The mean and standard deviation of the difference between estimated age and chronological age are shown for each method, examiner and gender in Table 3 (Table3).

Table 3. Mean (standard deviation) of the difference between estimated age and chronological age. A positive number indicates an over-estimation of the age and a negative number indicates an under-estimation of the age.

	Cameriere	Demirjian	Haavikko	Willems
Females N=257
Examiner 1	-0.96 (0.99)	-0.04 (1.10)	-1.47 (1.15)	0.51 (1.33)
Examiner 2	-0.81 (1.05)	0.82 (1.21)	-1.33 (1.16)	0.47 (1.32)
Examiner 3	-1.11 (0.98)	0.45 (1.22)	-1.69 (1.09)	0.34 (1.19)
Males N=244
Examiner 1	-1.05 (0.94)	0.65 (1.30)	-1.12 (1.03)	0.20 (1.32)
Examiner 2	-0.90 (1.03)	0.77 (1.27)	-0.86 (1.01)	0.40 (1.28)
Examiner 3	-1.26 (0.98)	0.62 (1.15)	-1.09 (0.97)	0.23 (1.10)

It emerges that C and H tend to under-estimate remarkably the age, while D and W are prone to an overestimation.

In the mixed models, the factors represented by the examiner, the method and the interaction between the examiner and the method are all significant. The p-

values were <0.0001 for all factors in the model for females.

The estimated difference between the estimated age and the chronological age and the results of the Tukey HSD test are reported for females in Table 4 (Table4). In females, it emerges that D and W over-estimate the age, but they are more accurate than C and H; C is more accurate than H.

Table 4. Tukey HSD for females. The outcome variable is the difference between estimated age and chronological age. Levels not connected by the same letter are significantly different.

Method	Examiner								Least Sq Mean
Demirijan	2	A							0.82
Willems	1		B						0.51
Willems	2		B						0.47
Demirijan	3		B						0.45
Willems	3		B						0.34
Demirijan	1			C					-0.04
Cameriere	2				D				-0.81
Cameriere	1				D	E			-0.96
Cameriere	3					E			-1.11
Haavikko	2						F		-1.33
Haavikko	1						F		-1.47
Haavikko	3							G	-1.69

The p-values were <0.0001 for examiner and method and 0.0161 for the interaction term in the model for males. The estimated difference between the estimated age and the chronological age and the results of the Tukey HSD test are shown for males in Table 5 (Table5). The THSD test basically shows that W is the most accurate method for males even if it leads to an over-estimation.

Table 5. Tukey HSD for males. The outcome variable is the difference between estimated age and chronological age.

Method	Examiner								Least Sq Mean
Demirijan	2	A							0.77
Demirijan	1	A							0.65
Demirijan	3	A							0.62
Willems	2		B						0.40
Willems	3		B	C					0.23
Willems	1			C					0.20
Haavikko	2				D				-0.86
Cameriere	2				D	E			-0.90
Cameriere	1					E	F		-1.05
Haavikko	3						F	G	-1.09
Haavikko	1						F	G	-1.12
Cameriere	3							G	-1.26

Sensitivity, specificity and the McNemar test values for D and W methods among the three examiners are shown in Table 6 (Table 6).

Table 6. Sensitivity, specificity and McNemar test for Demirijan and Willems method among the three examiners. Sensitivity: proportion of subject classified ≥ 14 years old when they are ≥ 14 years old.Specificity: proportion of subject classified < 14 years old when they are < 14 years old. * McNemar test; F: Females; M: Males.

Threshold 14 years	Examiner	Gender	Demirijan	Willems	p-value*
Sensitivity	1	F	0.80	0.95	<0.0001
Sensitivity	2	F	0.94	0.94	1.0000
Sensitivity	3	F	0.81	0.89	0.0027
Sensitivity	1	M	0.85	0.84	0.3173
Sensitivity	2	M	0.94	0.92	0.1573
Sensitivity	3	M	0.89	0.89	1.0000
Specificity	1	F	0.86	0.62	<0.0001
Specificity	2	F	0.62	0.64	0.1573
Specificity	3	F	0.80	0.72	0.0073
Specificity	1	M	0.61	0.65	0.0143
Specificity	2	M	0.61	0.61	0.3173
Specificity	3	M	0.73	0.73	1.0000

The sensitivity values of the methods are quite high, but the specificity values are low, especially for males. For females, examiners #1 and #3 yielded the higher sensitivity with W and the best specificity with D. For the male sample, the three examiners gained similar sensitivity and specificity values for W and D, with the only exception of a slightly better specificity value obtained by examiner #1 with the W method.

DISCUSSION

Many studies have demonstrated that dental calcification evaluated on OPG provides reliable evidence to estimate the age of children and youths. The scientific literature is committed to providing ever more accurate methods for the age assessment and to classify individuals with respect to their age given a legal threshold of interest (i.e. 14, 16, 18 years). Many variables potentially can affect the estimation of age and explain some differences in findings among studies even if applying the same method of DAE. The appropriateness of the statistical approach, the influence of the operator (19-21) and the real role of the ethnicity or environmental factors assume a crucial importance. On the other hand, especially criminal law requires that the age assessment or classification are provided with very high probability and a special attention to false positive rates.

In this study, we compared four well-known methods for DAE in an Italian sample distributed around the threshold of 14 years (11-16 years).

The inter-operator agreement was tested for the three experts and on the whole sample and our high values are consistent with those reported by other studies (22) and demonstrates the high repeatability of all the applied methods.

The Demirjian’s and Willems’s methods turn out to be the most accurate but they were demonstrated to be gender-sensitive and partially influenced by the operator performances. Consistently with numerous studies for the D method (23-27) and with a few for the W method (28, 29) as the principal finding, it emerges that the Willems’ and Demirjian’s methods lead to an age overestimation in both genders. Recently Butti et al. (30) concluded that dental maturation standards as described by Haavikko do not appear suitable for Italian children. In our research, the large underestimation of age for both female and male individuals in every cohort of age supports the aforementioned conclusion.

In 2011 Galic et al. (22) reported that the C method is the most accurate for both genders followed by Haavikko and Willems, which was the least accurate. El-Bakary (28) reported an average underestimation by 0.26 years for girls and 0.49 years for boys in the Egyptian population and similar results were observed by Cameriere in a European population (12-16). Our findings contrast with these results given the consistent age underestimation (~1 year) that the C method produces for both genders and in every cohort of age. Few papers have been published about the comparison of the accuracy, specificity and sensitivity of the DAE methods for the 14-years age threshold. Given the different relevance of false attribution in age assessment in civil and criminal proceedings, we have focused our attention on sensitivity (false negative) and specificity (false positive). The specific features of H and C methods and the large underestimation rates they have shown, renders the evaluation of their sensitivity and specificity values meaningless, hence the use of the Cameriere and Haavikko methods cannot be considered reliable for the assessment of the attainment of the 14-years threshold. The Haavikko method is not suitable for the evaluation above the 12-years cohort because it leads to a substantial and progressive inaccuracy (increasing with age of the subjects); the Cameriere European formula, on the other hand, does not allow estimations above the age of 14,06 years in the male individuals and even above the age of 13,68 years (13, if we use a completed year figure) in the female individuals.

The higher sensitivity values obtained with the use of the W and D methods, and therefore the higher probability of identifying the subjects who really are above 14-years old, allow us to say that in a civil law context, in which the evaluation methods that lead to an overestimation are essentially acceptable, the W and D are the methods of choice (and among them the W method when applied to female individuals). In contrast, the age overestimation and the low specificity values obtained by these methods impose prudent application in criminal case, possibly being advisable to apply two or more methods of DAE and/or to compare conclusions with those obtained through other methods of age estimation (e.g. maturation of wrist-hand bones). The forensic application of W or D in criminal cases always requires an accurate and prudent review of the results before the final conclusion is expressed and the odontologists should, at least, provide to the judge or the institution the percentages of false positive results expected with the adoption of these methods.

CONCLUSION

The authors, examining a sample of Italian children aged between 11 and 16 years, verified that:

• The Willems and Demirjian methods are more accurate than Cameriere and Haavikko, but they tend to overestimate the age.

• The Cameriere method largely underestimates the age (~1 year) for both genders and with all the operators.

• The Haavikko method is not suitable for dental age estimation in the Italian population.

• The Willems and Demirjian methods yielded high sensitivity but low specificity, thus producing consistent rates of false positive cases.

• As for the specificity rate, the Demirjian method is more suitable for females, while, for the sensitivity rate, the Willems’s method is the most indicated.

The results from our research make it difficult to identify a method that can be considered universally valid and the best to estimate the age of an Italian sample of children at the 14-years threshold, either for civil or criminal proceedings. Given the different legal requirements imposed by civil and criminal law, and especially for the latter, the examiner should therefore apply at least two methods of DAE for a sound comparative examination and provide estimations that report the false positive rates registered for the applied methods.