A reject analysis of cone-beam CTs in under-aged patients

Abstract

Objectives:

The main objective of this study was to perform a retrospective reject analysis (or audit) of 79 cone-beam CTs (CBCTs) taken in under-aged patients at the Ghent University hospital over a 2-year timespan.

Methods:

Observer agreement between two oral radiologists and two senior year Master students in Paediatric Dentistry was performed for quality, diagnostic and therapeutic value. The senior year Master Students followed appropriate modules of an online course. Descriptive and comparative statistics were performed.

Results:

For the oral radiologists, all intra rater reliabilities were moderate to good (Gwet’s AC1 = 0.41–0.75). For the senior students in Paediatric dentistry, these varied highly from fair to very good (Gwet’s AC1 = 0.28–0.95). There was a high level of disagreement between oral radiologists and students (Gwet’s AC1 = 0.16–0.45) and in-between students concerning observed quality (Gwet’s AC1 = 0.29). A total of 16 CBCTs (20%) was rejected, 24 images (30%) were acceptable and 39 images (50%) had an excellent quality. 50 CBCTs were perceived to have a diagnostic advantage. 13 of the images would have no influence on the therapy, according to the oral radiologists. A significant correlation was found between unacceptable quality, absence of perceived diagnostic advantage (p = 0.004, RR = 2.4) and influence on therapy (p < 0.0005, RR = 1.8). A small field of view (FOV) was positively correlated to an excellent quality of the image (p = 0.011, RR = 2.8).

Conclusions:

Image quality did not reach the proposed boundary of 10% according to the European Guidelines on Radiation Protection in Dental Radiology. This is the first published audit on an overall database of under-age children for CBCT.

Introduction

Cone-beam CT (CBCT) was proposed in the late 90s as a three-dimensional imaging modality for hard tissue in dentistry.^1,2 CBCT technology has been refined since then and it is expected that this will continue in the near future.³

This imaging technique has been used with increasing interest and extent since its introduction. In the period between 2005 and 2012, 20 manufacturers offered a total of 47 devices for sale in Europe.⁴ Still, impact on therapeutic planning trough CBCT is only reported for implant planning and some endodontic indications.^5,6 True scientifically supported justification and referral guidelines are often inadequately reported, lacking evidence of adequate methodology.⁷

The main concern in regard to the correct use of CBCT is of course cost-effectiveness, but also radiation dose.⁸ A meta-analysis by Ludlow et al⁹ showed adult effective doses ranging from 5 to 1073 µSv and child effective doses ranging from 7 to 769 µSv. Even doses as low as 1 µSv contribute to the overall background radiation and presumably to the probability to develop a fatal cancer in the patient as well.¹⁰ This is especially important in children which are more sensitive to ionising irradiation.¹¹ Hence, there is especially a need for evidence-based guidelines in the under-age population.^12,13

Naturally, all practitioners should estimate the need for CBCT according to current guidelines with an individualised case-by-case approach. The latter is in accordance with the As Low As Reasonably Achievable (ALARA), As Low As Diagnostically Acceptable or As Low As Diagnostically Acceptable being Indication-oriented and Patient-specific principles.^13–15 Yet, every health center or general practice is free to implement their own protocols compliant with these guidelines. This could lead to a heterogeneity in settings, perceived indications and image quality for every practitioner and/or device. The best imaging procedures for CBCT per indication in agreement with the ALADA statement are still not well researched or standardised. After all, CBCT is still a relatively new and heterogeneous generation of imaging modalities.

The Sedentexct-guidelines recommend in agreement with the Seventh framework Programme of Euratom, that a quality Assurance Plan is necessary.⁸ It should address image quality assessment, practical imaging technique, patient dose and equipment checks along with image viewing checks. Regarding imaging quality assessment, regular reject analysis is proposed. Image quality can be monitored subjectively, looking for features such as image sharpness, adequacy of contrast, adequate coverage of the region of interest, the obtrusive presence of artefacts etc.¹⁶ Practitioners reported in a nationwide survey in Norway that there are indications that quality of current CBCTs is questionable.¹⁷ To the authors knowledge, no reports of audits or reject analyses of CBCTs in an under-age population have been published.

Therefore, the principle aim of the present study was to assess the quality, perceived diagnostic thinking efficacy and perceived therapeutic efficacy of all CBCTs taken in children over a 2-year time frame at the Ghent University hospital. In this respect the null hypothesis was that at least in this institution, quality would be adequate.

In Belgium, no true educational program for maxillofacial radiologist exists. Therefore, the secondary aim of the study was to compare the reliability of the interpretation of image quality, diagnostic and therapeutic value between maxillofacial radiologists and senior year Master students in Paediatric Dentistry.

Methods and materials

This study was revised and approved by the Ethics Committee of the University Hospital Ghent, Belgian registration number B670201214496.

The images used in this database belong to a dataset that was originally collected for a retrospective observational study: material and methods for image selection and some selected results will be shown here.¹⁸

One main investigator (JVA) made a hand search in the Planmeca^® database (Romexis^®) for all patients under the age of 18 years who underwent a CBCT scan over a period of 2 years from the installation of the CBCT unit in the dental out-patient clinic of the University Hospital Ghent in Belgium. In this period, the CBCT-unit was a Planmeca Promax^® 3D Max. All CBCTs were stored in the Romexis database in the hospital during this time and there was no possibility for missing/lost data. From the available CBCT database, 135 patients could be retrieved. Informative letters for parents of patients younger than 12 or for patients 12 years and older and informed consents for the gathering of patient data and the use of radiographic material according to the WHO informed consent templates were sent by mail. These letters contained the following information: aim of the study, what happens when the patient gives no informed consent, obligations of the researchers to report changes in study setup, what was required of the subjects, method of data processing, contact information in case of questions, costs for participation, a request to sign the informed consent. There were two different types of informed consents, the former for parents of patients younger than 12 years and the latter for patients between 12 and 18 years old. When no answer was received, the main investigator tried to contact the patients by telephone for a confirmation of the patient’s address. A duplicate copy of the informed consent was then sent to those patients who did not reply the first time. In case still no response was received, no further attempts were made to make contact. Patients residing outside Belgium at the time of the research were excluded from the study.

For those patients who provided their informed consent, the following data were collected by the main investigator from the Planmeca database and the electronic patient database: age (y), gender, reason for referral, external or internal referral, field of view (FOV; w × h in mm), resolution (µm). All data were collected by the main investigator (JVA) and checked twice. In case of multiple CBCTs per patient, only the first CBCT was included and in case of a retake, the retake was considered valid for evaluation only. Images with artefact reduction were preferred over the originals.

The images were anonymised using DicomCleanerTM (PixelMed PublishingTM, open source initiativeTM) and randomised using a random integer generator (random.org, Randomness and Integrity Services Ltd.). The randomness originates from atmospheric noise, which for many purposes is better than the pseudo random number algorithms typically used in computer programs. The randomisation key was unknown to the observers.

The final study sample consisted of 79 CBCTs. All of these were taken by the same operator: an experienced dentomaxillofacial radiologist. No images had to be retaken; there were three images with artefact reduction. Descriptive statistics concerning patient gender, age group, distribution of FOV, distribution of resolution, internal/external referral and reason for referral as well as their corresponding correlations can be found in the article by Van Acker et al.¹⁸

There were eight main reasons for referral. For the 79 CBCTs, 107 reasons for referral were found, since some CBCTs demonstrated to be referred for multiple reasons at the same time. Distribution of main reasons for referral is shown in Figure 1.

Figure 1.

Distribution of main reasons for referral taken from Van Acker et al¹⁸

The observers consisted of two maxillofacial radiologists and two senior year Master students in Paediatric Dentistry. The senior students in Paediatric dentistry followed the online CBCT Sedentexct training (http://www.Sedentexct.eu/training/index.html). Following training modules were obligatory: justification-principles; justification-referral criteria; dose optimisation-quality assurance. They were not bound to follow the other modules, but this was recommended.

Following data were available to the observers: age, gender, indication according to Van Acker et al¹⁸ FOV and resolution. Additionally, they had a digital version of the Sedentexct guidelines available at all times.⁸ The observers were free to scroll through and manipulate all the images using a DICOM viewer (Planmeca Romexis Viewer). 20 images were scored 2 weeks later in a different randomised order to calculate intra rater reliability. All images were observed in a dark room. The observers had to score the quality according to the following criteria based on the Sedentexct guidelines⁸:

1. JUSTIFICATION: which implies doing more good than harm to the patient taking into account the radiation detriment to staff and other individuals. For the individual being exposed, there must be a net benefit, i.e. more good than harm. Table 1 illustrates the justification criteria linked to each reason for referral.

2. The possibility to make a RADIOGRAPHIC DIAGNOSIS with the CBCT.

   1. Adequate patient preparation, positioning and instruction

      1. No removable metallic foreign bodies which might produce scan artefacts (e.g. earrings, spectacles, dentures).

      2. No motion artefacts.

      3. No evidence of incorrect positioning of imaging guides/stents (e.g. air gap due to incorrect seating of the stent).

      4. Where fixed, metallic, restorations are in the teeth, no artefacts overlying the area of primary interest.

   2. Correct anatomical coverage

      1. Evidence that the smallest FOV available on the equipment has been used, consistent with the clinical application.

      2. The primary area of interest at or near the centre of the FOV.

      3. All of the area of interest included in the scan volume.

   3. Adequate exposure factors used

      1. Absence of significant image noise, low density and contrast.

      2. Correct resolution for the given indication.

3. OPTIMISATION of the practice, also known as the ALARA principle. The radiation exposure should be low, to minimise the risk of cancer and tissue effects. An optimised medical exposure is not always the one with the lowest dose but the one which carefully balances the detriment from the exposure and the resources available for the protection of individuals to get the required information.¹⁹ This is covered in the radiographic diagnosis criteria. Table 2 illustrates the diagnostic quality criteria for resolution and FOV linked to each reason for referral.

4. For medical exposures, the LIMITATION of the dose to the patient is not recommended because it may, by reducing the effectiveness of treatment or diagnosis, do more harm than good. Therefore, for patients the emphasis is on the justification and optimisation of exposures.¹⁹

Table 1.

JUSTIFICATION criteria linked to each reason for referral

Reason for referral: main category	Reason for referral: subcategory	Justification a
Dento-alveolar trauma	Suspected root-fracture	Where conventional intra oral radiographs provide inadequate information for treatment planning.
	Post-trauma complication	Not as a standard method for identification of periapical pathosis or demonstration of root canal anatomy. It may be indicated, in selected cases: 1. For periapical assessment when conventional radiographs give a negative finding when there are contradictory positive clinical signs and symptoms. 2. Where conventional intra oral radiographs provide information on root canal anatomy which is equivocal or inadequate for planning treatment, most probably in multirooted teeth. 3. When planning surgical endodontic procedures. The decision should be based upon potential complicating factors, such as the proximity of important anatomical structures. 4 Suspected, or established, inflammatory root resorption or internal resorption, where three-dimensional information is likely to alter the management or prognosis of the tooth. 5. Where endodontic treatment is complicated by concurrent factors, such as resorption lesions, combined periodontal/endodontic lesions, perforations and atypical pulp anatomy.
Dento-alveolar	Supernumerary teeth	May be indicated for pre-surgical assessment of an unerupted tooth in selected cases where conventional radiographs fail to provide the information required.
	Atypical tooth morphology	May be indicated for pre-surgical assessment of an unerupted tooth in selected cases where conventional radiographs fail to provide the information required.
Developing dentition-generalised	Syndrome	For complex cases of skeletal abnormality, particularly those requiring combined orthodontic/surgical management, particularly where MSCT is the current imaging method of choice.
	Tooth position and localisation	For complex cases of skeletal abnormality, particularly those requiring combined orthodontic/surgical management, particularly where MSCT is the current imaging method of choice.
Developing dentition-localised	Cleft palate assessment	Where the current imaging method of choice for the assessment of cleft palate is MSCT, CBCT may be preferred if radiation dose is lower.
	Tooth impaction	May be indicated (including consideration of resorption of an adjacent tooth) where the current imaging method of choice is conventional dental radiography and when the information cannot be obtained adequately by lower dose conventional (traditional) radiography.
	Tooth position and localisation	May be indicated for pre-surgical assessment of an unerupted tooth in selected cases where conventional radiographs fail to provide the information required.
Endodontics		Not as a standard method for identification of periapical pathosis or demonstration of root canal anatomy. It may be indicated, in selected cases: 1. For periapical assessment when conventional radiographs give a negative finding when there are contradictory positive clinical signs and symptoms. 2. Where conventional intra oral radiographs provide information on root canal anatomy which is equivocal or inadequate for planning treatment, most probably in multirooted teeth. 3. When planning surgical endodontic procedures. The decision should be based upon potential complicating factors, such as the proximity of important anatomical structures. 4 Suspected, or established, inflammatory root resorption or internal resorption, where three-dimensional information is likely to alter the management or prognosis of the tooth. 5. Where endodontic treatment is complicated by concurrent factors, such as resorption lesions, combined periodontal/endodontic lesions, perforations and atypical pulp anatomy.
Periodontics		CBCT is not indicated as a routine method of imaging periodontal bone support. It may be indicated in selected cases of infra bony defects and furcation lesions, where clinical and conventional radiographic examinations do not provide the information needed for management.
Surgical application	Bony pathosis	May be indicated for evaluation of bony invasion of the jaws by oral carcinoma when the initial imaging modality used for diagnosis and staging (MR or MSCT) does not provide satisfactory information.
	Exodontia	Where conventional radiographs suggest a direct inter relationship between a mandibular third molar and the mandibular canal, and when a decision to perform surgical removal has been made, CBCT may be indicated. CBCT may be indicated for pre-surgical assessment of an unerupted tooth in selected cases where conventional radiographs fail to provide the information required.
	Autotransplantation	Possibly indicated: for pre-surgical assessment of an unerupted tooth in selected cases where conventional radiographs fail to provide the information required.
	Implant planning	CBCT is indicated for cross-sectional imaging prior to implant placement as an alternative to existing cross-sectional techniques where the radiation dose of CBCT is shown to be lower.
	Orthognatic surgery	CBCT is indicated where bone information is required, in orthognatic surgery planning, for obtaining three-dimensional datasets of the craniofacial skeleton.
TMJ		Where the existing imaging modality for examination of the TMJ is MSCT, CBCT is indicated as an alternative where radiation dose is shown to be lower.
Other		CBCT is not normally indicated for planning the placement of temporary anchorage devices in orthodontics.
		CBCT is not indicated as a method of caries detection and diagnosis.
		For maxillofacial fracture assessment, where cross-sectional imaging is judged to be necessary, CBCT may be indicated as an alternative imaging modality to MSCT where radiation dose is shown to be lower and soft tissue detail is not required.
		Where it is likely that evaluation of soft tissues will be required as part of the patient’s radiological assessment, the appropriate initial imaging should be MSCT or MR, rather than CBCT.

CBCT, cone-beam CT;MR, magnetic resonance imaging; MSCT, multislice CT; TMJ, temporo mandibular joint.

^aThis implies doing more good than harm to the patient taking into account the radiation detriment to staff and other individuals. For the individual being exposed, there must be a net benefit, i.e. more good than harm.

Table 2.

The diagnostic quality criteria for resolution and FOV linked to each reason for referral

Reason for referral: main category	Reason for referral: subcategory	Resolution	FOV
Dento-alveolar trauma	Suspected root-fracture	ALARA but high resolution	Limited volume
	Post-trauma complication	ALARA but high resolution	Limited volume
Dento-alveolar	Supernumerary teeth	ALARA	Smallest volume size compatible with the situation
	Atypical tooth morphology	ALARA	Smallest volume size compatible with the situation
Developing dentition-generalised	Syndrome	ALARA	Large volume CBCT may be justified
	Tooth position and localisation	ALARA	Large volume CBCT may be justified
Developing dentition-localised	Cleft palate assessment	ALARA	Smallest volume size compatible with the situation
	Tooth impaction	ALARA	Smallest volume size compatible with the situation
	Tooth position and localisation	ALARA	Smallest volume size compatible with the situation
Endodontics		ALARA but high resolution	Limited volume
Periodontics		ALARA but high resolution	Limited volume
Surgical application	Bony pathosis	ALARA but High resolution	Limited volume
	Exodontia	ALARA	Smallest volume size compatible with the situation
	Autotransplantation	ALARA	The smallest volume compatible with the situation
	Implant planning	ALARA	The smallest volume compatible with the situation
	Orthognatic surgery	ALARA	Large volume CBCT may be justified in planning the definitive procedure
TMJ		ALARA	The smallest volume compatible with the situation
Other		ALARA	The smallest volume compatible with the situation

ALARA, as low as reasonably achievable; CBCT, cone-beam CT; FOV, field of view; TMJ, temporo mandibular joint.

Image quality was scored as excellent (no faults), acceptable (some faults but not affecting image interpretation) and unacceptable (or reject) according to the European Guidelines on Radiation Protection in Dental Radiology and The Guidance Notes for Dental Practitioners on the Safe use of X-ray Equipment.^16,20 Perceived diagnostic advantage and influence on therapy were each scored as no, yes or unknown. Diagnostic advantage was defined as: the image is judged “helpful” to making the diagnosis.²¹ Therapeutic efficacy was defined as: the image is judged “helpful” in planning management of the patient.²¹

Analysis was performed using IBM^® SPSS 24 and the trial version of AgreeStat2013 (^© 2010 Advanced analytics). Gwet’s AC1 inter and intra observer agreements were calculated.²² Values were graded as poor (<0.20), fair (0.21–0.40), moderate (0.41–0.60), good (0.61–0.80) or very good (0.81–1.00). Descriptive and comparative statistics were performed. Comparative statistics on reasons for referral were only performed for CBCTs of patients who had “dentoalveolar”, “developing dentition-localised” and “surgical application” as a single reason for referral. Other groups were too small for further analysis and CBCTs with multiple reasons for referral would lead to biased statistical results. After all, the observations were performed on image level. The level of significance was chosen at α = 0.05. Unpaired comparing statistics were done by Pearson chi-square test. Fisher’s exact tests were performed when more than 20% of cells had less than five counts. Binary logistic regression was performed when appropriate.

Results

Observer agreement

For the oral radiologists, all intra rater reliabilities were good (Gwet’s AC1 = 0.62–0.75), except in case of radiologist 2 who had a moderate intra rater reliability for the interpretation of diagnostic advantage (Gwet’s AC1 = 0.41). Concerning the senior students in Paediatric dentistry, intra rater reliability varied highly from only fair to very good (Gwet’s AC1 = 0.28–0.95). Inter rater reliability for diagnostic advantage and influence on therapy was moderate to good for all observers (Gwet’s AC1 = 0.43–0.82). There was a high level of disagreement between oral radiologists and students (Gwet’s AC1 = 0.16–0.45) and in-between students on observed quality (Gwet’s AC1 = 0.29). In-between oral radiologists there was moderate agreement (Gwet’s AC1 = 0.44). These results can be appreciated in Table 3.

Table 3.

Intra and Inter rater reliability for the observation of the CBCT images Gwet's AC1 ±SE (95% CI)

	Quality				Diagnostic advantage				Influence on therapy
	R1	R2	S1	S2	R1	R2	S1	S2	R1	R2	S1	S2
R1	0.62 ± 0.13 (0.34–0.90)	0.44 ± 0.08 (0.27–0.61)	0.21 ± 0.09 (0.04–0.38)	0.45 ± 0.09 (0.28–0.62)	0.75 ± 0.11 (0.52–0.97)	0.59 ± 0.07 (0.45–0.73)	0.68 ± 0.07 (0.54–0.809)	0.73 ± 0.06 (0.61–0.86)	0.75 ± 0.11 (0.52–0.97)	0.80 ± 0.05 (0.7–0.90)	0.43 ± 0.08 (0.26–0.60)	0.72 ± 0.06 (0.59–0.84)
R2		0.66 ± 0.13 (0.38–0.94)	0.16 ± 0,08 (0–0.33)	0.31 ± 0.09 (0.13–0.48)		0.41 ± 0.15 (0.09–0.73)	0.44 ± 0.08 (0.28–0.61)	0.60 ± 0.07 (0.46–0.75)		0.63 ± 0.14 (0.35–0.91)	0.46 ± 0.08 (0.3–0.63)	0.82 ± 0.05 (0.72–0.92)
S1			0.28 ± 0.17 (0–0.63)	0.29 ± 0.09 (0.13–0.46)			0.69 ± 0.13 (0.41–0.98)	0.72 ± 0.06 (0.59–0.84)			0.31 ± 0.16 (0–0.65)	0.45 ± 0.08 (0.29–0.61)
S2				0.39 ± 0.17 (0.03–0.74)				0.95 ± 0.05 (0.84–1)				0.95 ± 0.05 (0.83–1)

CBCT, cone-beam CT; CI, confidence interval; R1, radiologist 1; R2, radiologist 2; S1, student1; S2, student 2; SE, standard error.

Descriptive and comparative statistics

Both oral radiologists (who showed sufficient intra and inter rater reliability) were chosen as a reference for further analysis. For all observations, the lowest of both scores was taken as a reference point. For example, if radiologist 1 scored the image as a reject and radiologist 2 scored it as acceptable, then the final score would be a reject. In case of perceived diagnostic advantage and influence on therapy, the combination of “yes” and “no” would result in a “no”. The combination of “yes” and “unknown” or “no” and “unknown” would result in a “yes” or a “no” respectively.

A total of 16 CBCTs (20%) was rejected, 24 images (30%) were scored as acceptable and 39 images (50%) were perceived to have an excellent quality. 50 CBCTs (63%) were perceived to have a diagnostic advantage. 13 (17%) of the images would have no influence on the therapy, according to the oral radiologists. A significant correlation was found between quality and perceived diagnostic advantage (χ2 (1, N = 79) = 8.866, p = 0.004)). A higher number of images than the expected count had no diagnostic advantage, when the image was rejected (RR = 2.4).

A significant correlation was found between quality and influence on therapy (p < 0.0005, Fisher’s exact test). A higher number of images than the expected count was perceived to have no influence on therapy, when the image was rejected (RR = 22.6). Diagnostic advantage and influence on therapy were positively correlated (p < 0.0005, Fisher’s exact test). There was a higher chance that the image had influence on therapy, when diagnostic advantage was perceived by the observers (RR = 1.8).

Figure 2 shows the absolute counts of accepted versus rejected images and the percentage of rejected images for age group, gender, main reason for referral FOV and resolution. For comparative analysis, acceptable as well as excellent quality were taken as a cut-off point. Fisher’s exact tests showed that by age group (till 10 years old, from 10 till 12 years of age or 12 years and older) the quality of the image did not differ. Reject percentages per age group were 18, 15 and 18% respectively. Also, according to gender, the quality of the image did not differ significantly. Still, 26% of the images in male patients and 16% of the images in female patients were rejected. FOV was divided into localised (50 × 55 µm, n = 64) and non-localised (100 × 55 µm or bigger, n = 15). According to the oral radiologists, FOV was significantly associated with the quality of the image (p = 0.011). A higher amount than expected of the localised CBCTs was perceived as excellent, assuming independence (RR = 2.8). The percentage of rejected images was identical in both groups (20%). Initially, a significant correlation was found between resolution and observed quality of the image (p = 0.043). There was a slightly higher tendency in the 150 µm and 200 µm and a slightly lower tendency in the 400 µm group to be scored as excellent. Some groups however had very low numbers. Consequently, the authors decided to regroup these data. Comparing high (100–150 µm), normal (200 µm) and low (400 µm) resolution images with quality, no statistical correlations could be found. The percentage of rejected images was 40, 18 and 33% for high, normal and low resolution respectively but there were only five images with high and six images with low resolution in total. Only three of these groups were large enough for comparative analysis. No statistical correlation could be found between the reason for referral and quality of the image. 50% of the dento-alveolar traumas were rejected while 13, 33, 15, 0, 36 and 21% of the images were rejected for “dento-alveolar”, “developing dentition-generalised”, “developing dentition-localised”, “endodontics”, “surgical application” and “other” respectively.

Figure 2. .

Absolute counts of accepted versus rejected images and the percentage of rejected images for age group, gender, main reason for referral, field of view (FOV) and resolution.

For perceived diagnostic advantage as well as influence on therapy, Fisher’s exact tests showed no significant relation with age group, gender, FOV or resolution. Fisher’s exact test showed no significant correlation between reason for referral and observed diagnostic advantage or influence on therapy.

Discussion

Until this date, no literature known to the authors compares the judgement or diagnosis of radiologists and dental professionals for CBCT. This research indicates that observers on CBCT image quality and justification should have the adequate educational level. Analysis showed substantial disagreement between both observing groups. Both oral radiologists showed acceptable intra rater reliability. Image quality was the topic with the lowest intra rater agreement score for the senior year Master students in Paediatric Dentistry. This may reflect the lack of experience of these students. To counteract this, these students followed an online course. One online course has shown to improve the knowledge of oral health specialist significantly in anatomical assessment on CBCT (but not for under-graduate students).²³ Even so, an online course cannot substitute for the comprehensive educational program that the oral radiologists followed. Literature shows no clear evidence on the effectiveness of e-Learning in improving performance of students at practice and in enhancing patient health outcomes.²⁴

As a part of a quality assurance plan, clinical image quality assessment can be approached through a systematic audit of CBCT examinations against established clinical image quality criteria. This includes a retrospectively performed reject analysis. The current study is a preliminary example of this.⁸ 16 out of 79 images (20%) were scored as unacceptable by the oral radiologists. Consequently, image quality did not reach the proposed boundary of 10% according to the European Guidelines on Radiation Protection in Dental Radiology.¹⁶ It has to be mentioned that quality assessment was strict, since two different scores always resulted in the lowest of both. Unsurprisingly, these low-quality images are correlated with absence of diagnostic advantage and influence on therapy according to current observers.

Surprisingly, the image quality did not differ significantly according to age or gender, although the percentage of rejected images in the male group was 1.5 times higher compared to the female group. One might suspect that in younger patients, there would be a higher chance for (especially movement) artefacts. According to the justification rule for radioprotection, when there is no cooperation from the patient, one should not take a radiograph. Perhaps these younger less cooperative patients received no CBCT by default.

A correlation between small FOV and higher image quality was perceived by the observers. This is not to be unexpected, since a smaller FOV gives a higher accuracy and a lower radiation dose.^9,25,26 As long as the image contains the ROI, a smaller FOV is often advisable.⁸

Initially, a significant correlation was found between resolution and observed quality of the image. Statistical adjustment for small sample size no longer gave this result. A higher resolution gives a higher accuracy. Yet for a lot of indications a lower resolution is adequate and one must take in mind not to violate the ALARA principles.^8,14 No statistical correlation could be found between the reason for referral and quality of the image. The authors found no literature to compare.

Quality criteria for reject analyses or audits for two-dimensional dental imaging have been reported.¹⁶ Reject rates for these dental imaging techniques are often disappointing.^27–29

There are no evidence-based image quality criteria for dental CBCT. It has been proposed that objective device parameters can be translated to clinical image quality.³⁰ This is proposed because acceptability for clinical purposes is highly dependable on the observer. A human observer introduces a difficult to predict factor of subjectivity.^31–33 These studies could perhaps lead to criteria for specific indications. Two studies especially proposed specific quality criteria for implant planning and periapical diagnosis and for paranasal sinus imaging.^34,35

These studies do not provide usable criteria for an overall reject analysis or audit. This would be more practical from a clinical perspective. The authors could find one proposal in the SedentexCT guidelines.⁸ This study adapted these as can be appreciated in the material and methods. There was only one very basic overall score per image. Further detailed reasons for rejection were not noted in this study. One more project currently aims to provide more evidence-based criteria and is currently in the stage of dose quantification.^13,36

Some limitations of the study need to be considered. The current database only comprises images taken in the first two years after installation of the device. It is well possible that the maxillofacial radiologist faced a learning curve. This can result in lower quality images compared to later on, when the operator became more experienced with the device. Authorisation bias, which is selection bias caused by the obligation to acquire an informed consent, can be a limitation. This was found in a systematic review, although there is a lack in consistency of the direction and the magnitude of effect.³⁷ All images were taken in under-age patients over a smaller time span; this has some implications. In children, a higher reject rate is to be suspected based on movement artefact only. Also, a sample size could lead to higher power. Thus, correlations between quality and device settings or quality and the type of pathology can be found more easily. The present study only shows results for a single device (Planmeca Promax 3D Max). This device is highly adaptable to the indication. In exchange, the high variability in device settings can result in mistakes leading to high exposure doses or insufficient diagnostic quality.^30,36 Different clinics use different devices. The latter can produce different audit results caused by a high variability in design between those devices.⁷ Also at the time of imaging, movement artefact reduction and ultra-low dose scan were not yet implemented and European guidelines were only just published.⁸ Only three images had artefact reduction for shadows and streaks in this sample. The results of this study were also highly depending on the quality of the observers. Therefore, good intra and inter rater agreement are indispensable. The results in this study need to be interpreted cautiously since the observer groups are small. Perceived diagnostic advantage and therapeutic value also need to be interpreted cautiously since this is highly depending on the opinion and the experience of the observers.

Conclusions

• Audit observations need to be performed by a group of well-educated oral radiologists.

• Image quality did not reach the proposed boundary of 10% according to the European Guidelines on Radiation Protection in Dental Radiology; the null hypothesis was rejected.

• Lower image quality led to perceived loss of diagnostic and therapeutic value.

• A small FOV was positively correlated to an excellent quality of the image.

• This is the first published audit on an overall database of under-age children for CBCT.

• It would be interesting to perform more reject analyses with specific quality criteria for specific subgroups on reason for referral. These criteria could be subtracted from epidemiological databases and could improve future protocols. Ideally, the aim should be to reduce the proportion of unacceptable radiographs by 50% at each successive audit cycle.