A meta-review of effective doses in dental and maxillofacial cone beam CT using the ROBIS tool

Abstract

Objectives:

To apply the ROBIS tool for assessment of risk of bias (RoB) in systematic reviews (SRs) in a meta-review on effective doses (EDs) in dental and maxillofacial cone beam CT.

Methods:

Three electronic databases and reference lists of included SRs were searched. Eligible SRs were classified as having low, high or unclear RoB. Findings of SRs were synthesised and data from primary studies combined to relate ED to field of view (FOV) and operating potential (kV).

Results:

Seven SRs were included: three displayed low RoB, three high and one had unclear RoB. Only one SR related ED to image quality. Deficiencies in reporting of eligibility criteria, study selection and synthesis of results in SRs were identified. FOV height had a significant relationship with ED, explaining 27.2% of its variability. Median ED for three FOV categories differed significantly. Operating potential had a weak relationship with ED, with no significant difference in median ED between three operating potential groups.

Conclusion:

The ROBIS tool should have a role for meta-reviews of different aspects of radiology. The disappointing results for RoB might be remedied by developing standards to improve the quality of reporting of primary dosimetry studies and of SRs. Future dosimetry studies should always relate ED to image quality or diagnostic accuracy.

Advances in knowledge:

This meta-review is the first to implement ROBIS for SRs of ED and identified that trustworthiness of some SRs is questionable. The percentage change in average ED per cm increase in FOV height could be calculated, emphasizing the importance of FOV as a determinant of ED in CBCT.

Introduction

Since first being described at the end of the 1990s, cone beam CT (CBCT) has become an established imaging modality for the dental and maxillofacial region¹ with many different manufacturers and models of equipment.² Although initially aimed at use when planning dental implant treatment in adults, its use has extended widely in dentistry, including for paediatric applications, despite there being incomplete evidence for diagnostic efficacy and economic costs. The radiation doses associated with CBCT, however, are higher than those typically seen with the conventional radiographic techniques that it either replaces or supplements. This has raised concerns that have led to numerous publications presenting data on effective dose (ED) and efforts to reduce it.

In addition to primary studies measuring ED of CBCT, systematic reviews (SRs) have addressed this topic. SRs are generally seen as a reliable source of evidence, but there can be bias due to systematic flaws or limitations in the design, conduct or analysis; these might distort the results and lead to erroneous conclusions.³ Thus, there is value in conducting a “SR of SRs”, hereafter labelled a meta-review, to determine their risk of bias (RoB). Guidelines (notably AMSTAR,⁴ AMSTAR 2,⁵ ROBIS⁶ have been developed to assess the methodological quality and RoB of SRs, but no tool has been developed specifically in the context of SRs of radiation dosimetry. While AMSTAR⁴ and AMSTAR 2⁵ are tools for critical appraisal of SRs of randomised and non-randomised clinical trials, ROBIS⁶ was developed for assessment of RoB of SRs for a much wider range of topics, including interventions, diagnosis, prognosis, and aetiology. There are issues of study design and measures of effects relating to SRs of ED, which differ from those in such reviews, but the broad remit of ROBIS offers scope for application to this field. The aim of this meta-review, therefore, was to apply the ROBIS tool⁶ for assessment of RoB in SRs of ED in dental and maxillofacial CBCT.

Methods and materials

Protocol and review questions

The protocol was registered with PROSPERO on 29 October 2020 (registration number CRD42020214530). The reporting adheres to the PRISMA statement.⁷

The review questions regarding ED and dental and maxillofacial CBCT were:

• What is the overlap of included studies across SRs?

• What is the risk of bias in published SRs?

• What are the EDs reported in the SRs?

• Is the operation of CBCT equipment, and thereby ED, related to image quality or diagnostic accuracy?

Information sources and search strategy

The electronic databases searched were: MEDLINE via PubMed, Web of Science and The Cochrane Library. The publication years was restricted to range from 1 January 2000 as the first studies of CBCT were published in the late 1990s^8,9 to 31 October 2020. Details of the literature search strategy can be found in Supplementary Material 1. Reference lists of included SRs were screened for potentially useful additional SRs and the PROSPERO database was searched on 15 October 2020 to allow identification of any forthcoming SRs.

Selection of SRs

In Phase 1, the titles and abstracts of retrieved records were screened for eligibility by two reviewers independently. Included SRs had to meet a minimum set of criteria concerning:

1. study design: SR or meta-analysis

2. imaging modality: dental CBCT

3. reported outcomes: EDs

4. anatomical areas: dental tissues and maxillofacial skeleton

5. language: SRs with English abstracts and written in English or German.

SRs of CBCT used as part of radiotherapy treatment planning were excluded, as were narrative reviews, editorials, and case reports. Records selected by at least one reviewer were retrieved in full-text. In Phase 2, two reviewers independently applied the eligibility criteria to the full-text of the SRs. Any disagreement was resolved by consensus among the reviewers.

Overlap of primary studies across SRs

All primary studies were listed for each SR and a citation matrix that cross-linked individual SRs with studies was generated by one reviewer, but checked by a second reviewer. The degree of overlap was calculated and interpreted as the corrected covered area for all SRs and pairs of SRs as N - r/(r x c) - r where N is the total number of primary studies in all SRs, r is the number of primary studies in a single SR, and c is the number of SRs.¹⁰

Implementation of ROBIS

The ROBIS tool⁶ contains three phases: Phase 1 assesses the relevance of the “target question”. Phase 2 identifies concerns about bias in the review process within four domains: (i) study eligibility criteria, (ii) identification and selection of studies, (iii) data collection and study appraisal and (iv) synthesis and findings, while Phase 3 considers whether the SR as a whole has RoB. Each domain of Phase 2 has three sections: signalling questions, rating guidance to answer each signalling question as “Yes” (indicating low concerns), “Probably Yes”, “Probably No”, “No” and “No information” and judgement of level of concern about RoB in the domain (Supplementary Material 1). Two signalling questions in the domain “Data collection and study appraisal” were a priori decided as being “Not applicable,” as there is no guidance for quality appraisal or assessment of RoB of primary studies of radiation dosimetry. Responses to all signalling questions for a domain are summarised and the level of concern about RoB judged as “low”, “high” or “unclear” using the guidance. The overall judgement of RoB of a SR in Phase 3 uses the same structure as the separate domains in Phase 2 (Supplementary Material 1). The tool was piloted among three reviewers independently and discrepancies resolved through consensus.

SRs were assessed independently by at least two reviewers. Two reviewers were authors of one included SR and a third reviewer of another SR. To avoid potential conflict of interest and reduce bias, no reviewers assessed their own SR, with an external reviewer being brought in as required to substitute. The reviewers compared the answers to each signalling question and domain. Disagreements were resolved by consensus.

Data extraction and analyses of outcomes of ED

Data extraction followed a predefined protocol (available from authors upon request). For each SR, the principal author, publication year, review question(s), eligibility criteria, search strategy, number of included primary studies, details of data extraction and collection including the manufacturer/model of CBCT units, and the presented ED data were extracted. The results and conclusions and any review limitations presented by the authors were also recorded. One reviewer collected and tabulated these data. Then, a second reviewer read the SRs to verify the accuracy of collected data.

In order to conduct an analysis of the relationships between FOV height and ED, and between operating potential and ED, data were extracted from primary studies that had been included in the SRs. Where a primary study was cited in more than one SR, it was included once in the analysis. All measurements were exported to SPSS (IBM SPSS^® Statistics v. 22.0; IBM Corp., New York, NY; formerly SPSS Inc., Chicago, IL). Analysis was undertaken to determine the relationship between ED and FOV height and between ED and operating potential by calculation of Spearman rank correlation coefficient. Regression analysis was undertaken to determine how much FOV height or operating potential (kV), as independent variables, predicted ED as the dependent variable. Analysis was also undertaken of the relationships between ED and three different categories of FOV (>10 cm; 5.1 to 10 cm; ≤5 cm) and between different categories of operating potential. The median, range and quartile values of ED within these categories were calculated and violin plots were produced. Statistical comparisons between ED of different categories of FOV height and operating potential were made using the Kruskal–Wallis test, with the significance level set at 0.05 and post-hoc analysis using the Dunn method as required.

Results

Review selection and overlap of primary studies across SRs

Figure 1 shows the flow of articles identified through the searches. Seven full-text SRs^11–17 were included and each of them contained between 8 and 38 primary studies. In total, 147 primary studies were included, which are listed in Supplementary Material 1, from which 80 were only presented in a single SR. The primary studies were published in 11 different journals and ranged in publication date from 1999 to 2019,, as shown in Supplementary Material 1. The overlap expressed as “Corrected Covered Area” for the 7 SRs was 14%, which is “high” according to Pieper et al¹⁰ and the overlap between pairs of SRs ranged between 0 and 54% (Table 1).

Figure 1.

Flow chart according to the PRISMA Statement 7 presenting study selection process with number of systematic reviews identified, excluded and included for the meta-review of effective doses of dental and maxillofacial CBCT. CBCT, cone beam CT.

Table 1.

Overlap for seven systematic reviews included in the meta-review and overlap between pairs of systematic reviews

				CCA
	Total number of primary studies (N)	Number of primary studies (r)	Number of reviews (c)	Percentage (%)	Interpretation
Overlap for seven reviews	147	80	7	13.9	High
de Vos et al11 vs Bornstein et al.12	36	35	2	3	Slight
de Vos et al11 vs Ludlow et al.16	34	34	2	0	Slight
de Vos et al11 vs Al-Okshi et al.13	52	47	2	11	High
de Vos et al11 vs Goulston et al.14	38	38	2	0	Slight
de Vos et al11 vs da Silva Moura et al.15	37	37	2	0	Slight
de Vos et al11 vs van Acker et al.17	20	20	2	0	Slight
Bornstein et al12 vs Ludlow et al.16	42	32	2	31	Very high
Bornstein et al12 vs Al-Okshi et al.13	60	39	2	54	Very high
Bornstein et al12 vs Goulston et al.14	46	45	2	2	Slight
Bornstein et al12 vs da Silva Moura et al.15	45	33	2	36	Very high
Bornstein et al12 vs van Acker et al.17	28	27	2	3.7	Slight
Ludlow et al16 vs Al-Okshi et al.13	58	40	2	45	Very high
Ludlow et al16 vs Goulston et al.14	44	43	2	2	Slight
Ludlow et al16 vs da Silva Moura et al.15	43	28	2	54	Very high
Ludlow et al16 vs van Acker et al.17	26	24	2	8.3	Moderate
Al-Okshi et al13 vs Goulston et al.14	62	60	2	3	Slight
Al-Okshi et al13 vs da Silva Moura et al.15	61	43	2	42	Very high
Al-Okshi et al13 vs van Acker et al.17	44	42	2	4.7	Slight
Goulston et al14 vs da Silva Moura et al.15	47	44	2	7	Moderate
Goulston et al14 vs van Acker et al.17	30	29	2	3.4	Slight
da Silva Moura et al15 vs van Acker et al.17	29	27	2	7.4	Moderate

CCA, corrected covered area.

Method for calculations of CCA = N r / (r x c) - r, where N is the total number of primary studies in all reviews or pairs of reviews, r is the number of primary studies, and c is the number of reviews. Interpretation of CCA by Pieper et al.10 Slight overlap = 0–5%, Moderate = 6–10%; High = 11–15%; Very high = ˃15%.

Assessment of RoB with ROBIS

Three SRs^13,14,17 had a low overall RoB, being assessed positively across all four domains. Of the other SRs, one¹⁶ was judged as having unclear RoB and three^11,12,15 as having high RoB (Table 2). Ratings for each signalling question are presented in Table 3. In Domain 1 (Study eligibility criteria), many signalling questions were assessed as “No” or “Probably No” due to the review questions not being clearly defined and the eligibility criteria being ambiguous. For Domain 2 (Identification and selection of studies), “Were efforts made to minimise errors in selection of studies?” (signalling question 2.5) was assessed as “No” or “Probably No” because the selection of titles/abstracts or full-text studies did not involve at least two reviewers. Three SRs^11,12,16 were judged as having high RoB for Domain 3 (Data collection and study appraisal), the most common source of bias being that data collection and data extraction were not described as having involved at least two reviewers. In Domain 4 (Synthesis and findings) the synthesis of the study results was insufficiently described in some cases. One SR¹⁶ was unusual in that it included previously unpublished data in the meta-analysis.

Table 2.

Overview of assessment of concerns with review process and of risk of bias in the seven systematic reviews included in the meta-review

	Phase 2 Concerns with review process				Phase 3 Risk of bias in review
Systematic review	Domain 1 Study eligibility criteria	Domain 2 Identification and selection of studies	Domain 3 Data collection and study appraisal	Domain 4 Synthesis and findings
de Vos et al.11
Bornstein et al.12
Ludlow et al.16
Al-Okshi et al.13
Goulston et al.14
da Silva Moura et al.15
van Acker et al.17

low risk high risk unclear

Assessment performed with modified ROBIS tool⁶.

Table 3.

Assessment of risk of bias with ROBIS tool⁶ in the systematic reviews included in the meta-review

Phase, Domain, Signalling question	Systematic review
Phase 2: Identifying concerns with the review process	de Vos et al.11	Bornstein et al.12	Ludlow et al.16	Al-Okshi et al.13	Goulston et al.14	da Silva Moura et al.15	van Acker et al.17
Domain 1: Study eligibility criteria
1.1 Did the review adhere to pre-defined objectives and eligibility criteria?	N	PN	PN	Y	Y	PN	PY
1.2 Were the eligibility criteria appropriate for the review question?	N	PN	PY	PY	Y	N	Y
1.3 Were eligibility criteria unambiguous?	N	PN	PN	Y	Y	N	PY
1.4 Were all restrictions in eligibility criteria based on study characteristics appropriate?	PN	PN	PN	Y	Y	PY	PY
1.5 Were any restrictions in eligibility criteria based on sources of information appropriate?	PN	PY	PN	PY	Y	PN	PY
Level of concern regarding specification of study eligibility criteria (Low, High or Unclear)	High	Unclear	Unclear	Low	Low	High	Low
Domain 2: Identification and selection of studies
2.1 Did the search include an appropriate range of databases/electronic sources for published and unpublished reports?	N	N	PN	Y	Y	Y	Y
2.2 Were methods additional to database searching used to identify relevant reports?	PY	Y	Y	Y	Y	Y	Y
2.3 Were the terms and structure of the search strategy likely to retrieve as many eligible studies as possible?	N	PY	PY	PY	Y	N	Y
2.4 Were restrictions based on date, publication format, or language appropriate?	PN	PY	Y	Y	Y	PN	PY
2.5 Were efforts made to minimize errors in selection of studies?	N	N	N	Y	PN	PN	Y
Level of concern regarding methods used to identify and select studies (Low, High or Unclear)	High	Unclear	Unclear	Low	Low	Unclear	Low
Domain 3: Data collection and study appraisal
3.1 Were efforts made to minimise error in data collection?	N	N	N	PY	Y	PN	Y
3.2 Were sufficient study characteristics available for both review authors and readers to be able to interpret the results?	PN	PN	PY	Y	Y	PY	Y
3.3 Were all relevant study results collected for use in the synthesis?	N	N	PN	Y	PY	PN	Y
3.4 Was risk of bias (or methodological quality) formally assessed using appropriate criteria?	NA	NA	NA	NA	NA	NA	NA
3.5 Were efforts made to minimise error in risk of bias assessment?	NA	NA	NA	NA	NA	NA	NA
Level of concern regarding methods used to collect data and appraise studies (Low, High or Unclear)	High	High	High	Low	Low	Unclear	Low
Domain 4: Synthesis and findings
4.1 Did the synthesis include all studies that it should?	N	NI	PN	Y	PY	N	Y
4.2 Were all predefined analyses followed or departures explained?	N	NI	PY	Y	Y	N	PY
4.3 Was the synthesis appropriate given the nature and similarity in the research questions, study designs and outcomes across included studies?	N	NI	PN	Y	Y	PN	Y
4.4 Was between-studies variation (heterogeneity) minimal or addressed in the synthesis?	PN	PN	PY	Y	Y	N	Y
4.5 Were the findings robust, e.g. as demonstrated through funnel plot or sensitivity analyses?	NA	NA	NA	NA	NA	NA	NA
4.6 Were biases in primary studies minimal or addressed in the synthesis?	NA	NA	NA	NA	NA	NA	NA
Level of concern regarding methods used to synthesize results(Low, High or Unclear)	High	Unclear	Unclear	Low	Low	High	Low
Phase 3: Judging risk of bias
A: Did the interpretation of findings address all of the concerns identified the Phase two assessment?	N	N	PN	Y	Y	N	Y
B. Was the relevance of identified studies to the review's research question appropriately considered?	N	PN	PN	Y	Y	PN	PY
C. Did the reviewers present a balanced account of analyses on the basis of the results of included primary studies?	N	PN	PY	Y	Y	N	Y
Risk of bias introduced by methods used to identify and/or select studies (Low, High or Unclear)	High	High	Unclear	Low	Low	High	Low

For guidance of how the tool was used see Supplementary Material 1.Signalling question were rated Y = Yes, PY = Probably Yes, PN = Probably No, N = No, NI = No Information or NA = Not Applicable. Level of concern in domains of Phase two and risk of bias of Phase three was judged as Low, High or Unclear.

RoB of primary studies was assessed in three SRs. Goulston et al¹⁴ used GRADE.¹⁸ Van Acker et al¹⁷ employed both MINORS¹⁹ and GRADE,¹⁸ the former for intrastudy RoB and the latter for interstudy RoB assessment. Da Silva Moura et al¹⁵ used a points-based system, a method that had been through a series of unvalidated adaptations, but derived ultimately from original publications by Antczak et al²⁰ and Jadad et al.²¹

Description of included SRs

Four SRs^13–15,17 conformed to the PRISMA statement and included pre-defined review questions. All SRs reported EDs but the review focus was different among the SRs as presented in Table 4. The earliest SR also included data on clinical applications and technical aspects of CBCT reported in the included primary studies.¹¹ Two SRs focused purposefully on ED, although both made some analysis of how technical factors influence ED^13,16 while another SR described factors in general that influence ED.¹⁵ One SR¹² limited the SR to available evidence on the use of CBCT in implant dentistry and reported also on “radiation dose risk”. Another SR¹⁷ presented EDs for paediatric examinations as well as radiation protection measures for CBCT and other dental radiographic examinations. The review questions of the SR by Goulston et al¹⁴ differed from those of the other SRs in its focus on optimisation of ED and by only including studies incorporating a measurement of image quality and/or diagnostic accuracy. Supplementary Material 1 presents a detailed description of the characteristics of included SRs.

Table 4.

The main focus of included systematic reviews of effective doses (ED) of cone beam CT (CBCT), the summary of reported EDs and the relation between ED and image quality/diagnostic accuracy

Systematic review (SR)	Main focus of SR	What are the EDs reported in the SRs?	Is the operation of CBCT equipment, and thereby ED, related to image quality or diagnostic accuracy?
De Vos et al.11	Evaluation of data on clinical applications, technical parameters and radiation doses	ED (no categorisation into FOVs) range 52-1025µSv	No information presented
Bornstein et al.12	Assessment of “radiation dose risks” associated with cross-sectional radiography, specifically CBCT for assessment of dental implant sites	ED range small FOV (<40 cm2) 11–252 μSv medium FOV (40 to 100 cm2) 28–652 μSv large FOV (>100 cm2) 52–1073 μSv CBCT units from different manufacturers vary in dose by as much as 10-fold for an equivalent FOV	No quantitative data of ED presented, but result indicate that depending on equipment type, alteration of various exposure factors, image quality and FOV can markedly affect ED
Ludlow et al.16	Analysis of dose measurement and ED estimation of dental CBCT examinations and review of the literature on dosimetry of maxillofacial CBCT imaging	ED with standard protocol mean (range) Adults: small FOV (height ≤10 cm) 84 μSv ❖maxilla 53 (5-140) μSv ❖mandible 102 (18-488) μSv medium FOV (height 10–15 cm) 177 (47-560) μSv large FOV (height ˃15 cm) 212 (46-916) μSv Child: small FOV (height ≤10 cm) 103 (5-582) μSv ❖maxilla 67 (16-177) μSv ❖mandible 128 (24-331) μSv large and medium 175 (13-769) μSv	No quantitative data of ED presented, but the importance of ALADA emphasised in discussion
Al-Okshi et al.13	Estimation of ED of CBCT of the facial skeleton with focus on measurement methods and scanning protocols	ED median (range) small FOV (height ≤5 cm) 29 (10-197) μSv medium FOV (height 5.1–10 cm) 70 (4-674) μSv large FOV (height ˃10 cm) 114 (9–1073) μSv There was inconsistency regarding reported ED of studies of the same CBCT unit with the same FOV dimensions	No quantitative data of ED presented, but result indicate that depending on equipment type, models, version, alteration of various exposure factors, image quality and FOV can markedly affect ED
Goulston et al.14	Evaluation of “Can altering operating potential and tube current exposure time product on CBCT machines reduce ED without a detrimental impact on image quality and therefore diagnostic accuracy?”	ED range One study reported 27–674 μSv for four scanner types Another study reported EDs for different scan protocols using one scanner: ❖“Standard” protocols: 32–85 μSv (adult) and 39–120 μSv (paediatric) ❖“Quick scan” protocols 20–54 μSv (adult) and 23–70 μSv (paediatric) ❖“Quick scan+” protocols: 4–11 μSv (adult) and 5–18 μSv (paediatric) ❖“High resolution protocols”: 65–171 μSv (adult)	ED can be reduced without a detrimental impact on image quality, and therefore diagnostic accuracy, by altering exposure factors Results could not be generalised beyond models of machines and/or diagnostic task(s) investigated in each study
da Silva Moura et al.15	Evaluation of factors that influence ED and their respective effects, and comparison of ED of different CBCT with “similar exposure parameters”	ED range small FOV 19–58 μSv medium FOV 54–603 μSv large FOV 65–266 μSv	No quantitative data of ED presented, but the importance of ALADA emphasised in discussion
Van Acker et al.17	Evaluation of the effectiveness of radioprotective measures in underage patients (<18 years) who undergo a dental radiodiagnostic examination, CBCT included	EDs range 5–769 μSv The measurements include data from Monte Carlo simulations and from TLD-phantom studies Lowest EDs related to dose optimisation studies Limited studies in paediatric context and limited data for specific clinical indications	Evidence that lowering mAs, operating potential, and thereby ED, can be performed while maintaining adequate image quality

CBCT, cone beam CT; ED, effective dose; FOV, field of view.

All ED values reported in included SRs have been rounded up/down to the nearest µSv.

Main findings of EDs reported in the SRs

The main findings of ED reported in the included SRs are presented in Table 4. Two SRs^11,14 included studies that presented ED data using either the tissue weighting factors in ICRP 60 or those from ICRP 103; only the latter data are included in Table 4. There was variation in reported doses for the same CBCT equipment with the same FOV category, attributable to differences in equipment specifications (e.g. filtration) or exposure settings. ED for children was presented separately in four SRs.^12,15–17 Technical specifications were insufficiently described and heterogeneity in measurement methods and scanning protocols made comparisons of ED of different units difficult. In most studies, a commercially available anthropomorphic phantom (ART^®, RANDO^® and ATOM^®) including an adult male skull was used. A phantom that included a female skull and a paediatric phantom (corresponding to a person 10 years of age) were used in a few studies. In two studies, the phantom was one developed at the institution (University of Göttingen, Göttingen, Germany) where the study was performed. The number of phantom slices used ranged between 7 and 10.

The most recent SR,¹⁷ which focused on paediatric CBCT, identified three studies^16,22,23 which were sufficiently homogenous to perform a regression analysis with ED as the dependent variable. This found that 91.1% of the variance in ED was dependent on FOV, mAs and operating potential. A direct correlation was seen between mAs and ED and a significant positive estimate effect was seen for mAs for small FOV scans within the range 86–120 kV operating potential, with ED rising by 1.63 μSv per unit mAs. Non-significant estimate effects were seen when moving between three bands of operating potential (<86 kV; ≥86 kV to ≤120 kV; >120 kV).

While SRs discussed the importance of the ALADA principle (As Low as Diagnostically Acceptable), there was almost no direct evidence of how ED related to image quality or diagnostic accuracy. Although Goulston et al¹⁴ found two studies in which ED had been reported in parallel with objective image quality assessments, only one of these had presented data showing how changes to exposure protocols related to ED.²³ Increasing kV, tube current or exposure time increased contrast-to-noise ratio (CNR), all other factors being kept the same. Increasing FOV diameter led to some reduction in CNR while increasing voxel size was associated with an increase in CNR. Altering kV, current or exposure time had little impact on Modulation Transfer Function, but increasing FOV or voxel size reduced it.

Synthesis of outcomes of ED

There was a significant relationship between ED and FOV height (Spearman’s coefficient of rank correlation = 0.54; CI = 0.47–0.61; p < 0.001). After a log₁₀ transformation of the variables, linear regression established that FOV height could statistically significantly predict ED (F [1,404]=151.1, p < 0.0001) and that FOV height accounted for 27.2% of the explained variability in ED (Supplementary Material 1). The regression equation was: log₁₀(ED) = 0.9142+1.0151 x log₁₀(FOV height). Based on this equation, the theoretical impact of changing FOV height on “average” ED could be calculated; this showed that an increase in FOV height of 1 cm would lead to a rise in ED of, on average, 8.6µSv. The residual standard deviation (0.41) of the regression was, however, high because of the wide distribution of data around the regression line. Figure 2 presents the ED for different height categories of FOV from the primary studies. Median values, confidence interval (CI), ranges and interquartile ranges of ED all increased with the height category of FOV. Kruskal–Wallis H-test with post hoc analysis using the Dunn method found that the EDs for each FOV category were significantly different to each of the other two FOV categories (p < 0.001).

Figure 2.

Violin plots of effective doses (μSv) of dental and maxillofacial cone beam CT with three heights of FOV based on reported data of primary studies included in seven systematic reviews of effective doses. Each dot represents an individual measurement of effective dose. Median ± 95% CI marked on the plots. FOV, field of view; ICRP = International Commission on Radiation Protection.

There was a weak but statistically significant relationship between ED and operating potential (Spearman’s coefficient of rank correlation = 0.102; CI = 0.002–0.200; p = 0.045). Linear regression demonstrated that operating potential (kV) could statistically significantly predict ED (F [1, 381]=10.5, p = 0.001) but that it accounted for only 2.7% of the explained variability in ED (Supplementary Material 1). The regression equation was: log₁₀ ED = −0.4110+(1.1524 x log₁₀ kV). Figure 3 presents ED data according to the kV of the CBCT units included in primary studies (range 68–120 kV). The median ED at 100 kV was substantially higher than at other operating potentials. Analysis of the data into three groups (<90 kV; ≥90–≤100 kV; >100 kV) resulted in median EDs of 81, 91 and 74 µSv, respectively; statistical comparison of these found no significant differences (p = 0.38).

Figure 3.

Violin plots of effective doses (μSv) of dental and maxillofacial cone beam CT with different operating kilovoltage (kV) based on reported data of primary studies included in seven systematic reviews of effective doses. Data for 75kV are not presented as a violin plot because of the few measurements. Each dot represents an individual measurement of effective dose. Median ± 95% CI marked on the plots. ICRP = International Commission on Radiation Protection.

Discussion

The ROBIS tool

To our knowledge, this study is among the first meta-reviews in the field of radiology that has applied ROBIS as a tool to assess RoB in SRs. For a formalised assessment of quality or RoB of SRs, there are over 20 tools available. One strength of the ROBIS tool is that it follows a domains-based approach, with the domains being fundamental to the stages in development of a SR. Appraisal tools have moved from “point-scoring” to a domain-based approach focused on RoB, using a series of “signalling questions” for each domain.²⁴ An additional strength of ROBIS is that it entails descriptive reporting of the rationale used to assess each signalling question, enabling reviewers and readers to understand the reasons for the assessment decisions.

For reasons outlined in the introduction, ROBIS was chosen for the current meta-review because it is said to be applicable to all forms of SRs.⁶ An alternative tool used is AMSTAR 2,⁵ which is designed for assessment of SRs that include randomised and/or non-randomised clinical trials. The two tools differ in their focus; ROBIS focuses on how flaws affect study results, while AMSTAR two emphasises the rigour of the methods. Within the wider medical literature, there are currently about nine meta-reviews applying AMSTAR two and ROBIS in parallel but few have reported results of a comparison between them. The first comparative studies on the tools reported that the confidence rating using AMSTAR two strongly correlated with the overall domains rating in ROBIS²⁵ and that the validity measurements indicated the tools to be closely related.²⁶

Since SRs of ED include primary studies with different designs, analyses and results than those of interventional studies, a few signalling questions of ROBIS were modified or were considered to be “Not applicable” in the absence of any recognised tools for assessment of quality or RoB in primary dosimetry studies. Nevertheless, RoB was assessed for primary studies in three SRs,^14,15,17 using various methods, none of which is well-suited for dosimetry research, but it could be argued that credit for attempting some form of RoB assessment could have been given to these SRs over those that did not. One important finding of the current meta-review is that there is a need for guidelines on reporting of primary studies of radiation dosimetry. Such guidelines might be built on existing standards for performing dosimetry, e.g. from the IAEA²⁷ and could, in turn, allow a tool for assessment of RoB to be developed for SRs of dosimetry studies. A valuable recent step forward has been the “RECORDS” guidance on reporting of Monte Carlo simulation studies.²⁸ To our knowledge, however, no equivalent has been developed for the TLD and phantom-based dosimetry that is predominant in CBCT research. A tentative list of recommended items that could be used as a basis for developing such guidelines is presented in Supplementary Material 1.

Three^11,12,15 of the included SRs presented with high RoB and one meta-analysis¹⁶ was judged as having an unclear RoB. What does this mean in terms of the value and trustworthiness of these SRs compared with those that had low RoB? One SR¹¹ with high RoB is over 10 years old and might be dismissed as out of date, but it should be noted that it continues to be cited regularly. High RoB in SRs resulted from a range of weaknesses, including ambiguous review questions, no dual identification and selection of studies, searches using inadequate number of databases and incomplete description of synthesis methods. Any of these reduces the trustworthiness of the findings. The principles of conducting SRs are now well established and such failings are disappointing.

Analysis of included SRs

Overlap of primary studies across all SRs was very high as interpreted by Pieper et al.¹⁰ The degree of overlap, however, ranged from slight to very high. Those SRs with only slight overlap stand out from the others on the basis of date of publication or on the focus of the SRs. The two SRs which produced very high overlap were those performed at a similar point in time. The main difference between those two SRs, in addition to the number of included primary studies, was their focus. Al-Okshi et al¹³ looked specifically at the methodology of dosimetry as well as collating the ED of CBCT, concluding that comparability of EDs from different research groups is hampered by using different phantoms, dosemeter types and dosemeter positions. Ludlow et al¹⁶ focused on new concepts for dosimetry and reported some unpublished data for ED in both adult and child phantoms.

Outcomes of effective doses

As summarised in Table 4, a wide range of ED values was reported in the SRs, spanning from tens to many hundreds of µSv. These ranges were typically skewed towards the lower end of the ED range. Variations in ED seen between CBCT equipment for an equivalent FOV might be explicable by differences in image receptor technology, filtration, exposure setting used and the resolution chosen.¹⁶ The description of technical parameters of the CBCT units used in studies was often deficient.¹³ Differing reported EDs for what is ostensibly the same scanner, operated in the same way, can be due to manufacturers’ adjustments to technical specifications and changes in recommended “standard” exposure settings that would be used in research studies.¹⁶

The SRs have publication dates ranging from 2009 to 2020 and ought to give an indication of any trends in ED over time. It should be noted, however, that the majority of included primary studies were from 2015 or earlier. Thus, none of the SRs provided solely contemporary data, although it should be remembered that dentists do not replace expensive X-ray equipment very often; old equipment will continue to be used to maximise financial benefit from the expenditure and older ED data might still be relevant.

Five SRs presented data on ED in relation to FOV.^{12,13,15–17} One challenge in comparing findings from these SRs is the absence of a consistent definition of FOV. In the current meta-review, we followed the categories of Al-Okshi et al,¹³ in which small FOV is ≤5 cm height; this seemed more logical when a small FOV is commonly available as an option on modern CBCT equipment in the context of endodontics and other single tooth imaging situations. Gaêta-Araujo et al,² who used yet another definition of FOV based on irradiated surface (height x diameter), reported that 58.5% of available equipment offered a “small” FOV with an irradiated surface ≤40 cm², which might typically be seen for FOV heights no greater than about 6 cm. One recommendation from this meta-review is that we would benefit for a consensus on classification of FOV dimensions, as “small”, “medium” and “large” in SRs, or in primary studies, could be misinterpreted. Of course, height of the reconstructed image is only a representation of the irradiated field. Alternatives could have been used, including area on the detector, reconstructed volume width, X-ray cone angle and also approaches to account for FOV position in the patient, which will affect ED. Unfortunately, cone angle and the X-ray source/ patient/ detector distances were not factors that are presented in primary dosimetry studies, or in the SRs included in our meta-review. Even a recent comprehensive review of CBCT equipment² did not present these technical data, so we are constrained in a meta-review to what was available in the SRs, just as the SRs are themselves constrained by what is reported in the primary studies.

The regression analysis showed that FOV height accounted for over one-quarter of the explained variability in ED, with an increase in FOV height of 1 cm increasing ED by, on average, by 8.6 µSv. As presented in Figure 4, the impact of this on ED would become proportionally less important with larger FOV heights. Thus, changing from a 4 cm FOV height (average ED = 33.5 µSv) to a 6 cm FOV height (average ED = 50.6 µSv) gives an increase in average ED of 51%, while changing from a 10 cm FOV height (average ED = 85 µSv) to a 12 cm FOV height (average ED = 102.3 µSv) would only increase average ED by 20.3%. It is, however, important not to overstate these findings. The wide distribution of data around the regression line and high residual standard deviation indicate that “average” ED is not the normal situation. The R² value, although statistically significant, was low, although with such heterogeneity of CBCT equipment, modes of operation and dosimetry variations, a high value would have been surprising. Nonetheless, a low R² value does not negate a statistically significant predictor (in this case FOV); there is no threshold value of R² which divides a meaningful from a meaningless result. The many variations in other equipment parameters mean that such a neat “µSv/cm FOV height” relationship might only have practical use when considering individual CBCT machines. The violin plots clearly showed this wide range of doses for each FOV category and the large overlap between them. There was no evidence of a bimodal or multimodal distribution. These findings, with the regression analysis, reinforce the previous findings of the SRs that ED tends to increase with FOV height but that it is not the only factor of importance. As shown by van Acker et al,¹⁷ the combination of FOV, mAs and kV explained most of the variation in ED in three studies that were sufficiently homogenous to permit more detailed analysis. They were able to quantify a value for ED per unit increase in mAs. It would be of interest to perform additional analysis, e.g. to plot ED/mAs against FOV as this might highlight the effect of FOV more clearly. Performing this, however, would be challenging because of incomplete reporting of exposure parameters in many primary studies included in the SRs.

Figure 4.

The impact on average Effective Dose (ED) of increasing Field of View (FOV) height. This is derived from the regression equation calculated by the dosimetry data analysis performed in this meta-review. Each data point represents the percentage increase in ED that would result from extending FOV height by 1 cm more than the preceding FOV height. Thus, average ED for a 5 cm FOV height CBCT would be 25.5% greater than at a 4 cm FOV height, while average ED at a 14 cm FOV height CBCT would be 7.8% greater than at 13 cm FOV height.ED, effective dose; FOV, field of view.

It could be postulated that the relationship between ED and FOV might have a smaller distribution of data around the regression line and lower residual standard deviation if only SRs with low RoB had been analysed. There are, however, good reasons why this would not be demonstrable in the current metareview. The three SRs with low RoB had different aims and different search strategies, and so reviewed different subsections of the literature. This heterogeneity of the SRs means that any impact of RoB on findings would be obscured. A similar hypothesis could be proposed by only including data from SRs in which image quality or diagnostic accuracy had been taken into account in the eligibility criteria. Unfortunately, only one SR did this; although they included 22 primary studies in their SR, only 4 made any kind of dose measurement and these were disparate in the clinical context and their quality/diagnostic accuracy measures.¹⁴

Although the relationship between FOV size and ED has been widely recognised, an analogous analysis of operating potential and ED has received little attention, with only Van Acker et al¹⁷ having performed some analysis in their SR. They found no significant trend for ED to be affected by operating potential, but this was based on analysis of only three studies. This stimulated us also to undertake an assessment of whether operating potential had any impact on ED with CBCT, but using data from all the primary studies within the SRs. While a significant correlation between operating potential and ED was demonstrated, it was weak, and the regression analysis found that operating potential explained only a small fraction of the variability in ED. There was also no significant difference in ED between three operating potential groups, in accord with Van Acker et al,¹⁷ presumably because of the relative impact of multiple other factors that affect doses, not least of which is tube current–exposure time product (mAs), which was sometimes poorly reported in primary dosimetry studies. Other factors include filtration, continuous or pulsed X-ray production, number of “basis images”, FOV (which in itself is determined by several variable factors, as described above) along with the anatomical location of the scan FOV in relation to particular organs (notably salivary glands). Ideally, a more complex statistical analysis, taking all these factors into consideration simultaneously, would be required to unpick the contribution of operating potential to ED. Furthermore, this should also take into account image quality needs of particular clinical uses of CBCT.

An important additional confounding factor in the results for ED in relation to operating potential, however, was the predominance of particular brands of CBCT equipment in operating potential categories. For example, the median ED at 100 kV was much higher than at other operating potentials but this was because all the studies in this group reported on the same type of scanner, which had a particularly high ED and which is no longer available.^29–32 Similarly, the studies using a 120 kV setting were dominated by various models of the iCAT scanner (Imaging Sciences, Hatfield, PA). Other specific characteristics of a CBCT machine might overwhelm any impact of kV, presumably contributing to the multimodal distributions seen in the violin plots in Figure 3. Previous research studies on dose optimisation, reviewed in three SRs^14,15,17 have included variation in operating potential for some CBCT machines and shown scope for achieving dose reductions while maintaining adequate image quality at kV settings lower than manufacturers propose.

Altogether, the ED of 57 CBCT units were examined in the SRs and included in the meta-review (Supplementary Material 1). This is only 20% of the 279 CBCT models from 47 manufacturers which are presently available on the market, indicating the lack of knowledge that exists about most commercially available CBCT scanners.² Two factors must be taken into account when CBCT is considered: the radiation burden to the patient and society and the achievable image quality. Only one SR included in the meta-review related ED to image quality and concluded that for some diagnostic tasks ED could be reduced while maintaining diagnostic image quality.¹⁴ As image quality varies immensely amongst diverse CBCT units,³³ outcomes and literature statements regarding radiation doses, image quality and diagnostic applicability of dental CBCT cannot be transferred among different CBCT models. Compounding this is that an image which is ideal or acceptable for one purpose might not be so for another. In the end what counts is whether the use of any new imaging method improves patient outcomes and, at the same time, has an acceptable cost. The choice is easy when the new method is shown to be both less expensive and more effective than the current conventional method. The more common situation is that CBCT might be less cost-effective than current options, but may be cost-effective in some patients. Hence, the method can defend its position in the imaging arsenal, but it should be restricted to evidence-based indications. What need to be elucidated are inappropriate indications rather than inappropriate methods.³⁴

Limitations and strengths

We acknowledge that there are limitations and strengths to this meta-review. To the best of our knowledge, this is one of the first meta-reviews within radiology using the ROBIS. As recommended,²⁴ we performed calibration exercises and assessed the SRs independently by two reviewers. Comparison of inter-rater agreement may further have strengthened this process. Our analysis and implementation of ROBIS is based on a small sample in a particular area and needs further development in other fields of radiology.

The violin plots used to present ED across the FOV and kV have the advantage of visualising the distribution of data and its probability density. The relationship between FOV-size and ED has been widely presented, whilst an analysis of operating potential and ED performed in this meta-review may further contribute to the understanding of how exposure parameters can affect ED and image quality. Our search was limited to studies in English and German, which may create a degree of publication bias.

What are the clinical implications of this meta-review? A common question asked by clinicians is “how much dose will my patient get?” Clinicians find that systematic reviews are the most efficient source of evidence to answer such questions. The first implication of this meta-review for clinicians, therefore, is that not all SRs are equally trustworthy and that using ROBIS, either in meta-reviews or when reviewing a single SR, provides a method of “double-checking” the evidence that is presented for risk of bias. Clinicians should not take any research findings at face value but look at the quality and robustness of the evidence presented. The clinical value of knowing about ED is that it has a direct relationship with radiation risk and can be considered during the justification process. It can also help in communicating risk to patients. The second clinical implication is that, while a clinician might expect all CBCT scans taken for the same purpose to have the same ED, the analysis of data in the meta-review illustrates clearly the broad range in ED. This includes communicating the relationship between ED and FOV and the impact on increasing or decreasing ED by small changes in FOV height. All clinicians are aware that having diagnostically acceptable images is the essential end point of every radiological examination but might not relate this to dose; this meta-review has highlighted the paucity of evidence for ED in the context of image quality requirements.

Conclusions

The ROBIS tool should have a valuable role for meta-reviews of different aspects of radiology, although signalling questions need to be modified to suit review questions. Four of seven SRs had high or unclear RoB with deficiencies in all four ROBIS domains. The absence of accepted guidelines for reporting studies of dosimetry probably contributed to the disappointing levels of RoB. Guidelines and checklists for the design, conduct and analysis in primary studies, analogous to STARD and QUADAS-2 for diagnostic research are urgently needed. The EQUATOR network provides a valuable source of support in developing guidelines. While the meta-review confirmed the relationship between FOV and ED, it provided new information about the impact of changing FOV height. Reducing FOV height has a proportionally greater beneficial impact on ED when working with small FOVs. Conversely, choosing a slightly larger small FOV (e.g., 5 cm rather than 4 cm for single tooth CBCT imaging), perhaps as a strategy to “play safe” so as to avoid missing the region of interest, would have a proportionally greater impact on ED than when using the same strategy with large FOVs. Equipment operating potential (kV) did not have a significant impact on ED. As the number of available CBCT models continues to grow, there is a continuing need to perform dosimetry studies and conduct SRs, although relating ED to some aspect of image quality or diagnostic accuracy would be essential.