Comparison of simple screening criteria with the International Caries Detection and Assessment System classification in determining restorative treatment need

Abstract

Introduction: The Finnish Defence Forces’ unique oral health-screening protocol (FDFsp) has been in use for decades. In FDFsp, restorative treatment need is determined based on the World Health Organization (WHO) criteria. The aim of this study was to compare the outcome of screening restorative treatment need with the outcome of using the International Caries Detection and Assessment System (ICDAS) classification at both individual and tooth levels. Our hypothesis was that the outcome of screening with FDFsp agrees with the outcome of using the ICDAS classification. Methods: In this study, a trained, calibrated examiner estimated, in a visual–tactile manner the restorative treatment need of 337 young healthy adults using the FDFsp. During the screening, 74 conscripts were selected for a thorough inspection. The inclusion criteria for those selected were: having no, having one to five, or having six or more caries lesions needing restorative treatment. In the thorough inspection, the participants were inspected in a visual–tactile manner using the ICDAS classification. The association of the outcomes achieved using the two different methods was analysed at individual and tooth levels. Sensitivity, specificity, and kappa values were calculated. Wisdom teeth were excluded. Results: At the individual level, the agreement between the outcomes of using FDFsp and ICDAS ≥4 was excellent: sensitivity, 94.1%; specificity, 97.5%; and kappa = 0.92. When ICDAS ≥3 was used, the values were 72.7%, 96.7%, and 0.66%, respectively. Conclusion: Screening performed by a trained examiner using specific criteria is a reliable method for detecting individuals with restorative treatment need. The outcome of screening agrees strongly with results using the ICDAS classification.

INTRODUCTION

Oral health of the young (age under 20) has significantly improved in all industrialised countries over the past decades¹. Recently, however, oral health has shown signs of plateauing, or even of deterioration2., 3.. Indeed, our recent studies have shown that caries has not been eliminated among young adults born in the early 1990s, at least not in Finland4., 5.. Although screening is routinely performed to detect the risk of diseases such as breast or cervical cancer⁶, there are very few studies on the application of screening to estimate restorative treatment need, despite the fact that dental caries fulfils the indication for screening: it affects a large proportion among populations and can be prevented7., 8.. There are two phases in making restorative treatment decisions: the first concerns deciding whether or not to treat; and the second chooses the treatment path for those with restorative treatment need (i.e. is non-invasive treatment sufficient or is invasive treatment also needed)⁹. In both cases, the interval for the next appointment to see a dental professional should be determined, as well as the professional who the patient should see. A reliable and accurate screening method for detecting individuals with treatment need should be cost-effective. Such a screening method would also save time that could be used for treating people who actually need treatment. Most of the dental clinical practice in Finland, and also in other industrialised countries, involves restorative treatment procedures¹⁰.

Finnish conscripts have been succesfully screened using the Finnish Defence Forces’ unique oral health-screening protocol (FDFsp) and treated accordingly thereafter, since implementation of the protocol in the 1970s¹¹. The FDFsp is only used for screening conscripts, and its criteria are based directly on the World Health Organization (WHO) criteria for estimating restorative treatment need in epidemiological studies. The modern International Caries Detection and Assessment System (ICDAS) classification protocol has also been used for a screening-like purpose¹². In the present study, the authors wanted to investigate how the outcome of using FDFsp would compare with the outcome achieved using another known protocol (ICDAS), even though it was not used for screening here.

In Finland, military service is mandatory for men and voluntary for women under 28 years of age, unless they have a physical or mental disability preventing them from entering the military service. Around 25,000 young adults enter the service annually (aged 18 to 28 years). In recent years, 79% of the male subjects in each age cohort have completed their 6–12-month service¹³. The military service attendance rate is thus far higher compared with any other country worldwide, and representative of the each age cohort. In 2011, restorative treatment need of 13,819 conscripts (of whom 255 were women), born in the early 1990s, was determined by screening according to the FDFsp. Screening of individual conscripts took approximately 2.48 minutes¹⁴. The mean decayed teeth (DT) and decayed, missing and filled teeth (DMFT) values for the male population born in the early 1990s was obtained. In the national study group, the mean ± standard deviation (SD) DMFT index was 4.1 ± 4.2, the mean ± SD number of DT was 1.4 ± 2.5 and the proportion of subjects with a DT value of ≥ 1 was 45.0%⁵.

The aim of this study was to compare the restorative treatment-need decisions made on the basis of screening (FDFsp) and the ICDAS criteria, using the cut-off points ICDAS ≥3 and ICDAS ≥4. Our hypothesis was that there are no significant statistical disparities between these two protocols and that the outcome of screening is in accordance with that achieved by examination using ICDAS, both at the individual level and at the tooth level.

METHODS

Subjects

Of all draftees entering the Kainuu Brigade for their military service (n = 1654) in July 2011, every fifth conscript was selected, in alphabetical order according to their surnames, for dental screening (n = 337). Seventy-four (71 men and three women) of the 337 conscripts screened were selected to have a thorough dental inspection 1 month after the screening. The dentist performing the screening was instructed to select, during screening, a sample of individuals with no, only some (one to five) or several (six or more) caries lesions needing restorative treatment, for a more thorough dental examination. Because of the strict and inflexible military training schedule, the number of conscripts chosen was limited to 74. This sample may be considered a convenience sample.

After 1 month, this study group of 74 conscripts was invited to attend a more thorough inspection in which the ICDAS criteria were used. The dentist performing the inspections (V.A. – one of the authors and a specialist in restorative dentistry) was blinded to the screening outcome. Both inspections were obligatory for the participants and there were no drop-outs (Figure 1).

Figure 1.

The study protocol, illustrated as a flow chart.

Screening, training and calibration

Oral health screening at the Kainuu Brigade is part of an obligatory health examination. The screening was performed for a total of 13,819 conscripts and was carried out in January and July 2011 at Finnish garrisons, at the same time, by 15 military dentists and two PhD students. In the screening, all dentists used the FDFsp criteria to register treatment need (scores 3, 4, and 5) (Table 1). The flowchart in Figure 1 illustrates the study protocol. Screening was performed under illumination from the light of a dental unit, using a probe and an oral mirror; teeth were not air-dried. To ensure that all examiners involved in screening had similar knowledge, training sessions and lectures were given on FDFsp to remind the examiners about the WHO criteria and the effect of caries activity and histological depth of the lesions on their clinical appearance and subsequent treatment decision. These lessons were available to the examiners throughout the field study on the website of the Institute of Dentistry, University of Oulu, designed for this purpose⁴. All examiners involved in screening, including Veikko Rimpiläinen, were very familiar with the screening protocol used in the Defence Forces. If the examiner was unsure about the severity of any lesion, he/she was advised to choose a more severe option.

Table 1.

Scores according to the Finnish Defence Forces screening protocol (FDFsp) and the International Caries Detection and Assessment System (ICDAS) classification of clinical findings

Clinical findings	Score
	FDFsp	ICDAS
Healthy tooth	1	0
Filled tooth, healthy restored tooth	2	0
Decay, active decay needing restoration	3	3(active)/4
Secondary decay, needing restoration	4	3(active)/4
Decay reaching pulp – tooth needs root canal treatment or extraction	5	5/6
Tooth has been extracted because of dental caries	6	–
Tooth missing or is unerupted	7	–
Prosthetic crown	8	–
Prosthetic tooth (removable denture)	9	–

The calibration process has been described in detail in an earlier publication⁴. The inter-examiner agreement of the dentists performing the oral examination, calculated using the intraclass correlation coefficient (ICC), was 0.73 (minimum, 0.32; maximum, 0.85) before the January survey and 0.71 (minimum, 0.58, maximum, 0.86) before the July survey, when restorative treatment need (DT > 0) was used as the cut-off point. The intra-examiner agreement kappa value was 0.604 (minimum, 0.384; maximum, 0.866)⁴.

The dentist (V.R.) who screened the 337 draftees at the Kainuu Brigade was a senior dentist and had been trained and calibrated to FDFsp at two sessions in November 2010 and June 2011⁴, together with other dentists working in the Defence Forces. The inter- and intra-examiner agreement of the treatment decisions by V.R. was in agreement with the values of all other military dentists (interexaminer agreement ICC = 0.764 and intra-examiner agreement kappa value = 0.666)⁵.

Thorough inspection and ICDAS

The senior dentist, V.A., performed a thorough inspection on 74 conscripts (30 minutes per conscript), at Kainuu Brigade’s Dental Unit, under illumination from the light of a dental unit, using a probe, an oral mirror and fibre-optic transillumination (FOTI). Teeth were air-dried and when needed, visible plaque was removed using a probe. During the thorough inspection, caries lesions were classified according to the ICDAS scores¹⁵ and the highest score for each tooth was recorded by a dental nurse. Table 1 presents the scores of the screening (Defence Forces protocol) and the respective ICDAS scores. The dentist in charge of the thorough examinations was also in charge of educating and calibrating the military dentists before the field studies of the original project⁴. Calibration in a similar set up was performed on V.A. to measure her accuracy. The interexaminer agreement of the treatment decisions by V.A. and other examining dentists was ICC = 0.81 and the intra-examiner agreement kappa value for V.A. was ICC = 0.86.

Statistical analyses

At the tooth level, the scores representing restorative treatment need were 3–5 in the screening and ICDAS active 3 or 4–6. At the individual level, a person was considered to have restorative treatment need if he/she had at least one caries lesion needing restorative treatment in any tooth according to the screening or an ICDAS score of ≥3 (active lesion, localised enamel breakdown) or ≥4 (underlying greyish dentine shadow). To analyse the effect of the overall dental caries prevalence or treatment need, the conscripts were divided into three groups according to the number of caries lesions detected in the screening (DT), or DT = 0, DT = 1–5 and DT ≥ 6. Wisdom teeth were excluded from all analyses.

To investigate the outcome of the screening and of the thorough inspection performed using the ICDAS criteria, sensitivity and specificity values were calculated, and to investigate the agreement between the methods, also kappa values were determined. ICDAS scores of ≥3 and ≥4 were used as the gold standard for screening separately at tooth and individual levels. To analyse the results, cross-tabulation was used. A bar chart was drawn to illustrate the outcome of the screening and of using the ICDAS scores (ICDAS scores ≥3 or ≥4). The analyses were executed using SPSS software (versions 16.0 and 18.0; SPSS, Inc., Chicago, IL, USA) and Excel software.

Ethical issues

The data were collected from the archives of the records of the Finnish Defence Forces with their permission. For identification, all conscripts were assigned identification (ID) numbers. The key for the ID numbers and patient records is kept in the archives of the Finnish Defence Forces. The main research plan was evaluated ethically by the Ethical Committee of the Northern Ostrobothnia Hospital District, and positive consent was given on 30 March 2010. According to the research plan, a favourable review was presented by the research committee and permission was granted to carry on with the field phase. The Centre for the Military Medicine of the Finnish Defence Forces gave permission for the study in June 2010 (AG14218/23.6.2010). The participants in the study group were born in 1990–1992; in 2011, none of the subjects was under 18 years of age. In Finland, patient record data can be used for research purposes with the permission of the patient record holder, in this case, the Finnish Defence Forces. The conscripts agreed to the use of their data by attending the given dental appointments. This research was conducted in full compliance with the World Medical Association Declaration of Helsinki.

RESULTS

Based on the FDFsp criteria, the mean DMFT value in the group of 74 conscripts selected for a more thorough dental examination was 5.0 (minimum, 0; maximum 19; SD = 6.33) and the mean DT value was 2.8 (minimum, 0; maximum, 16; SD = 4.10). The proportion of those individuals with a DT value of ≥ 1 was 44.6%. The number of teeth affected by secondary caries (FDFsp score 4) was 73, and, of those, 68.5% were categorised as needing restorative treatment according to the ICDAS categories 3–6 (Table 2).

Table 2.

Distribution of different tooth types according to the Finnish Defence Forces screening protocol (FDFsp) criteria (A) and International Caries Detection and Assessment System (ICDAS) scores (B)

(A) Tooth	FDFsp score
	1	2	3	4	5	6	7	8	9	Total
x7, x6	389	85	58	51	5	1	3	0	0	592
x5, x4	498	26	50	7	0	2	9	0	0	592
x3	276	8	8	2	0	0	2	0	0	296
x2, xl	526	39	13	13	0	1	0	0	0	592
Total	1689	158	129	73	5	4	14	0	0	2072

(B) Tooth	ICDAS score
	0	1	2	3	4	5	6	Missing	Total
x7,x6	125	29	292	50	27	56	10	3	592
x5, x4	314	18	193	10	31	13	1	12	592
x3	238	4	38	4	4	7	0	1	296
x2, xl	500	3	57	6	19	6	0	1	592
Total	1177	54	580	70	81	82	11	17	2072

Tooth level

At the tooth level, the agreement between the outcome of the treatment need estimation using FDFsp and ICDAS ≥4 was kappa = 0.73. With this cut-off point (ICDAS ≥4), were sensitivity 82.2% and specificity 96.6%. When using the lower ICDAS cut-off point (ICDAS ≥3), the agreement between the two methods was lower (kappa = 0.67); sensitivity was 65.2% and specificity was 97.4%.

The outcomes of the oral health examinations of those individuals with restorative treatment need (45/74), determined by all methods, are presented in Figure 2. The number of participants without treatment need, on the basis of any of the methods, was 29. At the individual level, compared with FDFsp, a few more individuals needing treatment were found when using ICDAS, especially ICDAS ≥3 (12 individuals; four individuals with ICDAS ≥4) (Figure 2).

Figure 2.

Outcome of the screening and of using the International Caries Detection and Assessment System (ICDAS) scores ≥3 and ≥4 for detecting individuals with restorative treatment need according to any method; 29 individuals without any treatment need were excluded from the illustration.

Tooth level according to subgroups of lesions

When the group of 74 individuals selected for a more thorough dental examination was categorised according to the number of lesions needing restorative treatment, the sensitivity increased with increasing number of lesions. On the other hand, the specificity figures were very good, despite the number of lesions (Table 3). This pattern was similar for comparisons of FDFsp with both ICDAS ≥4 and ICDAS ≥3.

Table 3.

Sensitivities and specificities of Finnish Defence Forces screening protocol (FDFsp) and International Caries Detection and Assessment System (ICDAS) classification at tooth level considering the individual’s amount of restorative treatment need

ICDAS criteria	No. of lesions per individual according to FDFsp
	0	1–5	>6	Total	95% CI for total
ICDAS >4
Sensitivity	0.00	0.71	0.88	0.82	0.76–0.88
Specificity	1.00	0.96	0.85	0.97	0.96–0.97
ICDAS >3
Sensitivity	0.00	0.53	0.81	0.65	0.59–0.71
Specificity	1.00	0.97	0.88	0.97	0.97–0.98
Number of individuals	41	17	16	74

Individual level

With the cut-off point ICDAS ≥4 at the individual level, the sensitivity was 94.1% and specificity 97.5%, and the agreement between FDFsp and ICDAS ≥4 was kappa = 0.92. When using the cut-off point ICDAS ≥3, the respective values were 72.7%, 96.7%, and 0.66%.

DISCUSSION

This study revealed that in detecting restorative treatment need, the outcome of dental screening performed based on FDFsp was at least substantially¹⁶ congruent with the outcome achieved using the ICDAS criteria at both tooth and individual levels. Thus, the treatment decisions achieved using FDFsp were well in accordance with the treatment decisions made based on the thorough inspections (ICDAS ≥4) and in accordance with ICDAS ≥3. Therefore, a simple and rapid screening tool¹⁴ (FDFsp) can be regarded as an alternative to ICDAS¹² and applicable for determining restorative treatment need in young adults (19–21 years of age).

Caries is a worldwide multifactorial disease¹⁷. It is the reason for approximately 40% of all dental procedures in Finland¹⁰. In Finland, several medical health screenings are available according to law¹⁸. However, none concerns oral health. An oral health-screening method, FDFsp, has been used internally for decades in the Finnish Defence Forces and is considered a good protocol for assessing the oral health of Finnish conscripts. Similar protocols are also used by the North Atlantic Treaty Organization (NATO) for assessing field capacity¹⁹. In addition to restorative treatment need, FDFsp also includes other assessments, such as evaluations of periodontal and mucousal diseases. The outcome of this study shows that oral health screening could be a valid method for determining restorative treatment need, although prospective studies with larger populations would be valuable.

Knowledge of signs of activity and depth of caries lesions is essential for reliable caries detection and the basis for the whole schema of caries control²⁰. The ICDAS criteria do not include secondary lesions, but they do include initial caries lesions of different depths, which is its strength. When using the ICDAS criteria with the cut-off point 3, one must be aware of the possibility of over-estimating the restorative treatment need. With this cut-off point, activity estimation is essential in terms of the treatment decision. The possibility of non-invasive treatment procedures (sealants and resin infiltration) must always be considered.

The ICDAS protocol can be used for screening, but the examiner must be familiar with the protocol and the diagnostic criteria. In fact, the method has been used in a screening-like situation¹² ending up with similar time spans needed for screening, as presented in this study¹⁴. FDFsp score 4 registers secondary caries lesions and simultaneously the treatment history according to the definition of the score; differentiation between primary and secondary lesions can be considered a strength of the protocol. On the other hand, FDFsp score 5 classifies lesions needing restorative, or even endodontic treatment or extraction. This can be considered as a source of bias in the present study because teeth with an FDFsp score of 5 were considered as needing only restorative treatment. In the present study, when using the cut-off point 3, the sensitivity was lower compared with screening performed using the cut-off point 4. In this study, the dentists performing the screening were advised to choose the more severe option if they were uncertain; this may have caused some over-estimation even compared with ICDAS ≥3. Regardless of the screening protocol chosen, the final treatment decision should always be with a thorough inspection based on an appropriate classification and using additional diagnostic tools, such as bitewing radiography²¹.

In this study, the senior dentist who performed the screening (V.R.) was also advised to select the worse option in situations of uncertainty about the severity of the lesion. This, and the protocol itself, can be assumed to have minimised the underestimation of treatment need. Indeed, it may have even increased the number of lesions recorded as needing restorative treatment during the screening. The better values of specificity than of sensitivity, as well as the improved sensitivity with the increasing number of lesions, may be explained by the ease of detecting open cavities and awareness of clustering of lesions to certain individuals. These findings are also in accordance with the findings of Lussi²², stating that it is easier for a dentist to determine teeth as being sound than to detect teeth in need of restorative treatment.

The participants (n = 74) in this study were selected for the thorough inspection during the screening session, not retrospectively. This can be considered as a possible source of bias in the heterogeneity of the group of 74 conscripts. Yet, the group of 74 conscripts seems to be representative of the group of 13,819 conscripts regarding the proportion of individuals needing restorative treatment, even if the number of lesions per individual was larger in the group of 74 conscripts than in the original study population. Of those selected for a thorough examination, restorative treatment need was recorded in 44.6%; the corresponding value for all the conscripts screened in 2011 (group of 13,819 conscripts) was 45.0%⁵. On average, the DT value among the study population was 2.8, which is higher than among all the conscripts (for whom the DT value was 1.4). Overall, the DT and DMFT values were higher in conscripts of Kainuu Brigade than elsewhere⁴.

The number of participants in the subgroup was limited (n = 74) because in this study the dominant limiting factor was the inflexible military training schedule. Therefore, the number of participants was as large as possible under the circumstances. Some time is reserved for examinations, but after that the focus is on training. Thorough inspections were performed on as many conscripts as possible. As these limitations of the resources were known before starting the study, no power or deviation calculations were indicated or calculated. For randomisation, alphabetical selection of the screened conscripts was used. In Finland, alphabetical randomisation can be considered valid, especially when as many as every fifth person is selected in the study group. In these cases, the initial letters of the individuals’ surname are good representation of the population in the area. Randomisation of this kind was also simple to carry out during the health examinations because of the practices of the military forces in which surnames are listed in alphabetical order.

The activity of the lesions was not recorded here, but the examiners were aware of the signs of activity and remineralisation and were advised to consider them when making treatment decisions. It should be noted that, here, the accuracy of screening can be explained by the fact that the examiners were familiar with the protocol and diagnostic criteria⁴. The role of the thorough teaching of caries detection, including the calibration sessions, arranged before the data collection, must also be emphasised²³. The examiners (including V.R., the senior dentist responsible for screening in this study) underwent two intensive educational and calibration sessions (given by V.A.)⁴. Many of the dentists involved in screening in the present study, including V.R., had also been examiners in the studies of Ankkuriniemi¹¹ and Läärä²⁴. In the present study, the examiner (V.R.) possessed screening skills that represented the average skills of the entire group of examiners in the earlier conscript study (n = 15)⁴. Both the screening and the thorough examination period lasted only for 1 week, which is beneficial for the outcome because the criteria remained comparable when examining all participants. In any case, the outcome of dental examinations performed by public health dentists has been found to be as reliable as those performed by calibrated examiners in epidemiological studies²⁵. To maintain a high standard of diagnostic skills, dentists should have review sessions, for example, on new diagnostic methods and criteria.

CONCLUSIONS

In conclusion, screening performed by following the WHO criteria is readily applicable and valid, despite being a rather rough method for estimating restorative treatment need among young adults, especially at the individual level. For a reliable outcome, those responsible for the screening must be familiar with the criteria. Screening (in the present study using FDFsp) can be used as a tool for assessing the need for further thorough inspection and both non-invasive and invasive caries treatment. In the patient file, there must be an option for registering all these findings. Screening can be considered as time- and cost-effective. However, prospective follow-up studies are needed to investigate this further.

Acknowledgements

We gratefully acknowledge the dental staff of the Finnish Defence Forces, particularly oral hygienist Merja Komulainen and DDS Veikko Rimpiläinen, for their enthusiasm and work in conducting the field survey, as well as the medical conscripts for their assistance. We also want to express our thanks to Mrs Mari Saario, Finnish Defence Forces, for her cooperation in collecting the data. The study was conceived and designed by A.K., T.T., P.P., and V.A. The field study was performed by A.K., T.T., P.P., and V.A. The data were analysed by A.K. and J.P. The paper was written by A.K., T.T., V.A., P.P., and L.T. A.K. received the Finnish Dental Society Apollonia’s grant for this study. None of the funders had any role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Conflict of interests

None of the researchers have any interests which might affect the objectivity of this study in any way.