Image quality of cone beam computed tomography for evaluation of extremity fractures in the presence of metal hardware: visual grading characteristics analysis

Abstract

Objective:

To evaluate image quality and interobserver reliability of a novel cone-beam CT (CBCT) scanner in comparison with plain radiography for assessment of fracture healing in the presence of metal hardware.

Methods:

In this prospective institutional review board-approved Health Insurance Portability and Accountability Act of 1996-complaint study, written informed consent was obtained from 27 patients (10 females and 17 males; mean age 44 years, age range 21–83 years) with either upper or lower extremity fractures, and with metal hardware, who underwent CBCT scans and had a clinical radiograph of the affected part. Images were assessed by two independent observers for quality and interobserver reliability for seven visualization tasks. Visual grading characteristic (VGC) curve analysis determined the differences in image quality between CBCT and plain radiography. Interobserver agreement was calculated using Pearson's correlation coefficient.

Results:

VGC results displayed preference of CBCT images to plain radiographs in terms of visualizing (1) cortical and (2) trabecular bones; (3) fracture line; (4) callus formation; (5) bridging ossification; and (6) screw thread–bone interface and its inferiority to plain radiograph in the visualization of (7) large metallic side plate contour with strong interobserver correlation (p-value < 0.05), except for visualizing large metallic side plate contour.

Conclusion:

For evaluation of fracture healing in the presence of metal hardware, CBCT image quality is preferable to plain radiograph for all visualization tasks, except for large metallic side plate contours.

Advances in knowledge:

CBCT has the potential to be a good diagnostic alternative to plain radiographs in evaluation of fracture healing in the presence of metal hardware.

INTRODUCTION

Radiography and multidetector CT (MDCT) have an established role in orthopaedic evaluation of the extremities,^1–3 especially in the trauma setting.⁴ While MRI scans are considered a good alternative to radiographs,⁵ frequently post-operative assessments of fracture healing need to be performed in the presence of metal hardware,⁶ which produces significant artefacts on MRI and could prove challenging to interpret. In such cases, MDCT is regarded as an appropriate diagnostic modality in the assessment of fracture healing following initial assessment by plain radiograph (modality of choice),⁷ where volume rendering mitigates the deleterious effects of streak artefacts⁸ that are associated with the presence of metal hardware.^4,9

Recently, we designed, developed and tested the feasibility of a dedicated high-resolution three-dimensional (3D) extremity cone-beam CT (CBCT) scanner for musculoskeletal imaging that has the ability to obtain weight-bearing scan acquisition. Compared with MDCT, it is associated with reduced radiation exposure and can be installed within the orthopaedic clinic with potential improvement in patient workflow.¹⁰ A previous study using cadaver extremity samples has shown that images obtained by the dedicated extremity CBCT scanner had excellent quality for “bone” visualization tasks and at least adequate quality for “soft tissue” visualization tasks; CBCT images were either comparable with or superior to MDCT for “bone” visualization tasks in the side-by-side comparisons.¹¹ In addition, CBCT assessments showed an acceptable interobserver agreement.¹¹ The number of recent animal^12,13 and cadaver^11,14 studies have elucidated the diagnostic performance of CBCT scanners in the imaging of the extremities. Nevertheless, only a limited number of studies have investigated the image quality and interobserver performance of dedicated extremity CBCT scanners in clinical studies using human subjects.^15,16

Recent studies have highlighted the role of CBCT scanners in the evaluation of fracture healing in the dental and maxillofacial bones^17–19 and extremities.^20,21 The performance of dedicated CBCT scanners for the evaluation of fracture healing in the extremities needs to be assessed in the presence of metal hardware,¹⁵ as the cone-beam geometry and the typically reduced X-ray tube output along with lower mean X-ray beam energy of the CBCT scanners may affect the appearance and severity of the artefacts.^19,22,23 In this study, our objective was to evaluate the image quality and interobserver reliability of a novel dedicated extremity CBCT scanner when compared with radiography, for the assessment of fracture healing in the presence of metal hardware.

METHODS AND MATERIALS

Study population

This is an institutional review board-approved, Health Insurance Portability and Accountability Act of 1996-compliant study. Informed consent was obtained from all participants. In this prospective study, 50 patients with known fractures were recruited, out of which 23 patients did not have any hardware implants. The final study population consisted of 27 patients (10 females and 17 males; mean age 44 years, age range 21–83 years) with a history of extremity fracture (lower extremity fracture n = 26 and upper extremity fracture n = 1) and metal hardware placement at the site of the fracture (for fracture stabilization), who underwent a CBCT scan as part of the research protocol. All 27 study participants also had a lateral and an anteroposterior radiograph as part of their clinical care.

Cone-beam CT and radiography acquisition protocols

The nominal CBCT protocols are described in a previous technical assessment,¹⁰ with the X-ray tube energy and total scan tube current set to 80 kVp and 108 mAs, respectively, for all patients. The CBCT image data were reconstructed using a “bone” algorithm with iterative reconstruction. All patients also had a radiograph of the affected extremity as a part of their routine clinical care using standard clinical protocols at our institution. The imaging protocol for radiographs was different based on the body part as follows: knee (70 kVp and 12–16 mAs), lower leg (64 kVp and 4 mAs) and wrist (52 kVp and 2 mAs).

Data analysis

The CBCT and radiographic images were reviewed by a board-certified musculoskeletal radiologist with 7 years' experience and a musculoskeletal radiology fellow with 4 years' experience in the interpretation of musculoskeletal CT images. Readings were conducted using diagnostic quality monitors calibrated to digital imaging and communications in medicine standards. Seven visualization tasks were selected as essential and relevant features for determination and characterization of fracture healing (Table 1): (1) cortical bone, (2) trabecular bone, (3) large metallic side plate contour, (4) screw thread–bone interface, (5) bridging ossification, (6) fracture line and (7) callus formation. Visualization tasks were selected by considering previously defined qualitative measures of fracture healing.²¹ Each visualization task was scored by the two independent observers according to a previously utilized five-point diagnostic satisfaction scale (based on image visibility and ability to assess).¹¹ Observers were blinded to each other's scoring.

Table 1.

Details of the visualization tasks which were used for the evaluation of fracture healing

Task number	Target	Task assessment criteria
1	Cortical bone	Assessment of cortical bones' integrity and density
2	Trabecular bone	Assessment of trabecular bones' architecture and resolution of their trabecular patterns
3	Large metallic side plate contour	Assessment of the presence, clarity and contour of the large metallic side plate(s)
4	Screw thread–bone interface	Assessment of clarity of the interface
5	Fracture line	Assessment of the presence, clarity and number of fracture lines
6	Callus	Assessment of presence, clarity and number of calluses
7	Bridging ossification	Assessment of the presence and clarity of bridging ossifications

Statistical analysis using visual grading characteristics

The means of the two observer ratings were analyzed using the visual grading characteristic (VGC) analysis.^24–26 There are two primary methods for evaluating visual grading—image criteria study and visual grading analysis. Image criteria studies characterize the observer confidence about the fulfilment of specific image quality criterion, while visual grading analysis characterizes the observer opinion about the reproduction of a certain anatomical structure. In contrast, VGC analysis is a stepwise process which involves first creating a frequency table summarizing the results of both modalities separately, followed by the calculation of VGC points. These points are then used as coordinates for the VGC curve. VGC handles the visual grading data in similar fashion to receiver-operating characteristic data, as the origin of a VGC curve is “0” and the last point includes all decisions and is therefore “1”. For the purpose of our study, the previously mentioned diagnostic satisfaction scale was further stratified based on the average scores of two observers, according to the method described in an introductory VGC analysis study (Table 2).²⁴ The observational data were converted into a frequency table for calculating subsequent image criteria scores (ICS) describing the percentage of observations, which satisfy a certain criterion in the image. For example, an ICS score of x% for “good” image quality on the diagnostic satisfaction scale means that x% of all observations on that modality had an image quality of “fair” to “good”, while the remaining (100−x%) observations had an image quality of “good” to “excellent”. ICS were used for plotting the VGC data points. A software, ROCKIT (C E Metz, University of Chicago, Chicago, IL), was then used to obtain a continuous VGC curve using the corresponding VGC points and for calculating the area under the curve (AUC). This AUC is a relative measure of image quality as per the diagnostic satisfaction scale.

Table 2.

Details of the diagnostic satisfaction scale for the assessment of the visualization tasks

Score	Visibility	Ability to assess	Interpretation
5	Excellent	Excellent	Diagnostic quality, without any artefacts
4.75	Indecisive between good and excellent—somewhat confident in excellent
4.50	Indecisive between good and excellent
4.25	Indecisive between good and excellent—somewhat confident in good
4	Good	Good	Diagnostic quality, with minor artefacts
3.75	Indecisive between fair and good—somewhat confident in good
3.5	Indecisive between Fair and Good
3.25	Indecisive between fair and good—somewhat confident in fair
3	Fair	Adequate	Diagnostic quality, with moderate artefacts identified
2a	Poor	Challenging	Non-diagnostic quality, the visualization task can be identified
1a	Very Poor	Very Challenging	Non-diagnostic quality, the visualization task cannot be identified

^aNo image was rated as “poor” or “very poor”, and all the evaluations were rated at least as having a “fair” quality.

Interobserver correlation was assessed using Pearson's correlation coefficient. 95% confidence intervals were extracted, and a linear approximation line was fitted to the data and analyzed using the JMP Pro (v.11, SAS Institute Inc., Cary, NC, IL). A p-value of <5% was considered as the significance threshold in this study.

RESULTS

The visualization tasks were rated by both observers as per the diagnostic satisfaction scale ranging from 1 to 5. No observation was rated as “very poor” or “poor”, which is 1 and 2 on the satisfaction scale. All the evaluations were rated at least as 3 or having a “fair” quality. The interval between the observer scores was further divided, as shown in Table 2, based on the observer level of confidence.²⁴

The interobserver agreement was strong for visualization tasks between the two observers. All valuations had a p-value <0.05, except for the visualization of the large metallic side plate contour. Details of the interobserver correlation analysis for each visualization task are presented in Table 3 with its graphical representation in Figure 1.

Table 3.

Interobserver correlation for the assessment of the visualization tasks

Coefficients	Modality	Cortical bone	Trabecular bone	Large metallic side plate contour	Screw thread–bone interface	Fracture line	Callus	Bridging ossification
Correlation		0.7886	0.8174	0.5422	0.8022	0.7806	0.6873	0.7071
95% CI lower bound	Radiograph	0.5717	0.6237	0.1872	0.5961	0.5574	0.4011	0.4329
95% CI upper bound		0.9025	0.9165	0.7720	0.9092	0.8986	0.8513	0.8615
Correlation		0.9094	0.6786	0.3885	0.9382	0.6414	0.4082	0.8111
95% CI lower bound	CBCT	0.8027	0.3873	−0.0078	0.8630	0.3299	0.0156	0.6122
95% CI upper bound		0.9597	0.8467	0.6794	0.9727	0.8269	0.6918	0.9135

CBCT, cone-beam CT; CI, confidence interval.

All evaluations had a p-value <0.05, except for the visualization of the large metallic side plate contour in CBCT (which was marginally significant).

Figure 1.

Graphical representation of the interobserver correlation data from Table 3: the upper set of images are for radiographs and the lower set of images are for cone-beam CT (CBCT). The central line represents the best linear approximation of the plotted data between observers and the shaded area represents the 95% confidence interval for the linear approximation. The points represent observer ratings. r is the correlation coefficient; the higher the value, the better the correlation.

The first step in VGC analysis involves creating a frequency table, which is shown in Table 4; it depicts the counts of average observer ratings for the visualization tasks, while Table 5 provides the VGC data points from ICS describing the percentage of observations which satisfy a certain criterion in the image. For example, in case of the cortical bone, 24% of observations were scored as having fair to good (3–4 in diagnostic satisfaction scale) image quality on radiographs, as opposed to 8% of observations on CBCT images; in other words, 76% of observations on radiographs and 92% of observations on CBCT images scored good to excellent (4–5 on the diagnostic satisfaction scale). Analysis on these data points provided the results as the VGC curve depicted in Table 6, which shows the favourability of the CBCT images when compared with radiographs for the visualization tasks. With the exception of the large metallic side plate contour (AUC of 0.012), all other visualizations were significantly in favour of CBCT (AUC >0.84 for all remaining tasks) (Figure 2). No images were rated below 3, which corresponds to fair visibility and adequate image quality. A comparative example of CBCT and radiograph images is shown in Figures 3 and 4, depicting difference in visualization of screw thread–bone interface and the large metallic side plate contour, respectively.

Table 4.

Count of average observer rating for the seven visualization tasks

Score	Cortical bone		Trabecular bone		Large metallic side plate contour		Screw thread–bone interface		Fracture line		Callus		Bridging ossification
	Radiography	CBCT	Radiography	CBCT	Radiography	CBCT	Radiography	CBCT	Radiography	CBCT	Radiography	CBCT	Radiography	CBCT
3: fair	0	0	4	0	0	1	11	0	4	0	4	0	6	0
3.25	0	0	2	0	0	0	4	0	1	0	1	0	4	0
3.50	0	0	6	1	0	2	1	2	2	0	4	0	10	0
3.75	0	0	5	1	0	1	4	2	7	2	9	0	2	1
4: good	6	2	8	7	1	20	6	10	13	1	7	2	4	2
4.25	2	2	0	0	0	2	1	0	0	2	0	5	0	0
4.50	2	0	0	4	3	1	0	1	0	21	0	19	1	23
4.75	7	0	0	0	8	0	0	2	0	1	0	1	0	1
5: excellent	10	23	0	14	15	0	0	10	0	0	0	0	0	0

CBCT, cone-beam CT.

Table 5.

The visual grading characteristic data points; pairs of image criteria scores

Score	Cortical bone		Trabecular bone		Large metallic side plate contour		Screw thread–bone interface		Fracture line		Callus		Bridging ossific trabeculation
	Radiography	CBCT	Radiography	CBCT	Radiography	CBCT	Radiography	CBCT	Radiography	CBCT	Radiography	CBCT	Radiography	CBCT
3: fair	0	0	16	0	0	4	40	0	16	0	16	0	20	0
3.25	0	0	24	0	0	4	56	0	20	0	20	0	36	0
3.50	0	0	48	4	0	12	60	8	28	0	36	0	72	0
3.75	0	0	68	8	0	16	76	16	56	8	72	0	80	4
4: good	24	8	100	36	4	92	96	52	100	12	100	8	96	12
4.25	32	16	100	36	4	96	100	52	100	20	100	24	96	12
4.50	36	16	100	52	16	100	100	56	100	96	100	96	100	96
4.75	60	16	100	52	44	100	100	64	100	100	100	100	100	100
5: excellent	100	100	100	100	100	100	100	100	100	100	100	100	100	100

CBCT, cone-beam CT.

All data points are percentage (0–100%).

Table 6.

Area under the curve (AUC) estimates for the visual grading characteristic curves

Score	Cortical bone	Trabecular bone	Large metallic side plate contour	Screw thread–bone interface	Fracture line	Callus	Bridging ossification
AUC	0.8427	0.9416	0.0122a	0.9051	0.9593	0.9888	0.9779
95% CI lower bound	0.6597	0.8279	0.0005	0.7908	0.838	N/A	0.911
95% CI upper bound	0.9452	0.9857	0.1102	0.9651	0.9938	N/A	0.9963

CI, confidence interval; N/A, not applicable.

For callus visualization, comparison between radiography and cone-beam CT (CBCT) revealed a perfect decision performance (in favour of CBCT); thus, no further binormal curve could have been fitted and AUC was calculated using Riemann sums.

^aWith the exception of “large metallic side plate contour”, visualization task which was significantly in favour of radiography, all other visualization tasks were significantly in favour of CBCT.

Figure 2.

Visual grading characteristic analysis with area under the curve for all visualization tasks: radiography is represented in the x-axis and cone-beam CT (CBCT) is represented in the y-axis. The curves show the preference for CBCT images for all tasks, except metallic hardware contour which has preference for radiograph.

Figure 3.

A 56-year-old female with a history of a tibial plateau fracture followed by internal fixation showing the comparison of screw thread–bone interface between cone-beam CT (CBCT) (A) and radiograph (B). The space around the screw threads (arrows) is clearly seen in the CBCT image, while the same space is difficult to assess in the radiograph.

Figure 4.

A 48-year-old male with a history of a right unicondylar tibial plateau fracture followed by open reduction and internal fixation showing the comparison of metallic hardware contour between cone-beam CT (CBCT) (A) and radiograph (B). The metal plate (arrows) on the medial side of the tibia can be seen clearly in the radiograph; however, it is not as clearly seen on the CBCT image owing to artefacts.

DISCUSSION

This study reports the initial results of our clinical experience using dedicated extremity CBCT scanner in assessment of fracture healing in the presence of metal hardware. Cortical bone visualization tasks had excellent visibility and ability to assess in our dedicated extremity CBCT scans, while trabecular bone visualization was also associated with good quality. This further highlights the applicability of a dedicated extremity CBCT scanner in bone visualization tasks, in addition to the previous cadaver study which reported a superior performance of CBCT in comparison with MDCT for bone visualization tasks.¹⁰ However, another study showed that CBCT was inferior to MDCT in both cortical bone and soft tissue visualization.²⁷ This discrepancy could be because of the different CBCT scanners used in the respective studies. Our results in this study demonstrate that CBCT scans can also be superior to plain radiographs for various visualization tasks relevant to fracture healing evaluation in the presence of adjacent metal hardware. Specifically, our results show superior image quality of CBCT which is critical in determination of osseous processes that occur during fracture healing such as callus formation, bridging osseous trabeculation and residual fracture line compared with the plain radiograph. CBCT images are superior to plain radiograph in evaluation of the healing process within the complex 3D configuration of fracture plane as well as demonstration of bone–screw thread interfaces, which helps in the detection of early hardware loosening. This radiologic analysis is important in early detection of non-union or infection. At the same time, the larger metal hardware contours such as side plates are better seen in plain radiographs owing to more prominent artefacts in CBCT images compared with plain radiograph, and therefore standard plain radiograph may be more accurate in detection of subtle hardware fractures. The apparent discrepancy is probably owing to the fact that at bone–screw interface, the higher contrast and spatial resolution of CBCT improves the image quality compared with plain radiograph even in the presence of worse beam hardening artefacts occurring owing to the relatively small size of metallic screws. Conversely, the beam hardening artefact surrounding large side plates in CBCT images compared with plain radiograph dominates the better contrast resolution for imaging these relatively larger metallic objects. It can be noted that the visualization performance was assessed only in “raw” CBCT volumes and that the assessment of hardware contours may be improved when additional rendering (e.g. maximum intensity projection) is applied. In addition, CBCT might produce better yields in terms of depiction of post-instrumentation sequelae to assess for fracture healing, which would potentially reduce the number of follow-ups and radiographs required for the patient.

While artefacts are expected in any X-ray modality involving tomographic reconstruction of projections corrupted by scatter, photon starvation, beam hardening and partial volume effects in the presence of metal hardware, CBCT may, in certain cases, exhibit more severe artefacts than MDCT. Previous studies have highlighted the increase in X-ray scatter as one of the main limitations of obtaining high-quality images using CBCT scanners.¹² Furthermore, since CBCT systems typically operate at lower X-ray tube output than MDCT (which is dictated by the need for compact X-ray sources), they might be more susceptible than MDCT to photon starvation owing to metal hardware. The X-ray spectra employed in CBCT are often shifted towards lower energies (tube potential of 80–100 kVp) than in MDCT, reducing the penetration of the X-rays through the metal and altering the amount of beam hardening. In addition, the cone-beam geometry of CBCT scanners may result in increased artefacts in the presence of metal hardware owing to the oblique X-ray path through the metal components and the cone-beam artefact.²² Cadaver experiments have shown an increase in metal-induced artefacts in CBCT scans (in comparison with MDCT scans), which were obtained following the fixation of scaphoid fractures.¹⁴ On the other hand, no significant difference was found in the magnitude of metal-induced streak artefacts between CBCT and MDCT for a variety of cases with surgical hardware in another study.²⁰ Nevertheless, our results show that with the help of performance optimizations that are specific for extremity scanners,¹⁵ one can obtain images with better than fair visibility and better than adequate ability to assess critical features in fracture healing, despite the presence of surrounding metal hardware.¹⁰ In general, our results confirm previous speculations that CBCT may allow us to image most patients with orthopaedic hardware, in a clinical setting.²²

Recent works have evaluated the performance of CBCT in detection of osseous injuries and reported higher resolution imaging and shorter time acquisition compared with plain radiography.^20,21 However, no prior study has systematically compared the image quality of CBCT for distinct anatomical tasks between the two modalities. To determine the value of CBCT in fracture healing in the presence of metal hardware, it is important to determine the performance of CBCT in each particular task compared with plain radiograph, the current modality of choice. In this regard, VGC facilitates the quantification of subjective assessments of image quality and incorporates grading of the visibility and image quality for specific anatomical tasks on the images. In relative VGC, the visibility of such tasks in CBCT images is compared and graded against the visibility of the same task in plain radiograph (“reference imaging modality”). Our results suggest that for most of the examined tasks relevant to extremity fracture healing assessment in the presence of metal hardware, high-resolution 3D CBCT images are superior to plain radiographs. However, plain radiograph is superior to CBCT for determination of integrity of metal hardware itself and detection of its fracture.

This study has several limitations. It is not known whether or not the quality of images and the observer agreement may be affected by the timing of scan. There was no strict timing for the CBCT scans, relative to the timing of surgery. In other words, while some CBCT scans were obtained early after surgery, others may have been performed within several weeks after surgery. In this study, we did not assess the CBCT image quality for the soft tissue visualization tasks, as in our design plain radiograph was considered the comparative imaging modality, and therefore determination of soft tissue details is not feasible in this comparative modality. Considering the soft tissue visualization tasks, our previous cadaver study reported slight or definite preference for MDCT, in comparison with CBCT.¹⁰ We also did not compare CBCT image quality with MDCT, as the study population did not have clinically indicated MDCT scans. Limited number of patients were evaluated in this study. We used the data from the first series of enrolled patients, and we had a specific inclusion criteria to include only subjects with extremity fractures and metal hardware at (or near) the site of fracture. Future reports using greater number of subjects may provide a more precise evaluation of the diagnostic performance of dedicated extremity CBCT scanners, in post-operative evaluation of fracture healing. Another interesting research study would be to perform a longitudinal study to assess union after instrumentation and follow the fracture healing process in weight-bearing and non-weight-bearing scans.

CONCLUSION

Dedicated extremity CBCT scanners can be reliably used for the assessment of fracture healing, even in the presence of metal hardware, while providing high-resolution 3D imaging with multiplanar reformation along with adequate to excellent image visibility and ability to assess, strong interobserver agreement, low effective radiation and the option of acquiring weight-bearing scans.

CONFLICTS OF INTEREST

Wojciech B Zbijewski receives research grant from Carestream Health. Jeffrey H Siewerdsen received research agreement and licensing agreement from Carestream Health and also serves in the Advisory Board of Carestream Health. Shadpour Demehri receives research support from the General Electric Company and Carestream Health Inc grants from GERRAF and is also a consultant for Toshiba Corporation. John Yorkston is an employee of Carestream Health Inc. Greg Osgood received research support from Carestream Health.

FUNDING

This project was funded by a grant from Carestream Health Inc.