Evaluation of image quality and task performance for a mobile C-arm with a complementary metal–oxide semiconductor detector

Abstract.

We assessed interventional radiologists’ task-based image quality preferences for two- and three-dimensional images obtained with a complementary metal–oxide semiconductor (CMOS) flat-panel detector versus a hydrogenated amorphous silicon (a-Si:H) flat-panel detector. CMOS and a-Si:H detectors were implemented on identical mobile C-arms to acquire radiographic, fluoroscopic, and cone-beam computed tomography (CBCT) images of cadavers undergoing simulated interventional procedures using low- and high-dose settings. Images from both systems were displayed side by side on calibrated, diagnostic-quality displays, and three interventional radiologists evaluated task performance relevant to each image and ranked their preferences based on visibility of pertinent anatomy and interventional devices. Overall, CMOS images were preferred in fluoroscopy ($p=0.002$) and CBCT ($p=0.004$), at low-dose settings ($p=0.001$), and for tasks associated with high levels of spatial resolution [e.g., fine anatomical details ($p=0.006$) and assessment of interventional devices ($p=0.015$)]. No significant difference was found for fluoroscopic imaging tasks emphasizing temporal resolution ($p=0.072$), for radiography tasks ($p=0.825$), when using high-dose settings ($p=0.360$), or tasks involving general anatomy ($p=0.174$). The image quality preferences are consistent with reported technical advantages of CMOS regarding finer pixel size and reduced electronic noise.

1. Introduction

During the past several years, the use of hydrogenated amorphous silicon (a-Si:H) flat-panel detectors (FPDs) has become standard in many imaging modalities, such as radiography, fluoroscopy, and cone-beam computed tomography (CBCT) as alternatives to storage phosphors^1–3 and x-ray image intensifiers. However, limitations in the spatial resolution and low-dose imaging performance of a-Si:H detectors have been reported.^4,5 As the use of imaging in diagnosis and treatment has increased, there is a need for imaging services that expose patients to lower doses of radiation.^6–9 However, electronic noise and spatial resolution limit the quality and diagnostic utility of images produced at lower-dose ranges when using conventional a-Si:H detectors.^4,5,10

Methods to improve performance at lower radiation doses are being researched.^11,12 Complementary metal–oxide semiconductor (CMOS) FPDs have recently been developed with a field of view consistent with clinical application. CMOS detectors provide numerous theoretical advantages over a-Si:H detectors, including reduced noise, higher temporal resolution, higher spatial resolution, and higher frame rate in low-dose and higher spatial frequency settings when using mobile C-arms.^13,14 The field of interventional radiology is associated with constantly evolving technology, and it is often assumed that newer technologies are better than older ones. Although CMOS detectors have been shown to outperform a-Si:H detectors in technical assessments of basic physical performance [e.g., modulation transfer function (MTF) and detective quantum efficiency (DQE)],^13,14 it is unclear how such improvements translate to image quality and task performance in clinical practice. In this study, we assessed interventional radiologists’ preferences for using images obtained with a CMOS detector versus an a-Si:H detector in simulated clinical imaging tasks.

2. Materials and Methods

2.1. Image Acquisition and Processing

Two mobile C-arms (Cios Alpha, Siemens Healthineers, Forchheim, Germany) were used, identical except for the x-ray detector: one incorporated a PaxScan 3030X a-Si:H FPD (Varex Imaging, Salt Lake City, Utah) and the other had a Xineos-3030HS CMOS FPD (Teledyne DALSA, Waterloo, Ontario, Canada). Note that the Cios Alpha platform is not capable of CBCT imaging in its standard commercial/clinical deployment; however, the research prototypes used in this work (one with an a-Si:H FPD and one with a CMOS FPD) were each modified to enable CBCT under computer-controlled, pulsed x-ray exposure, and FPD readout under continuous angular gantry rotation. Geometric calibration, image preprocessing, and three-dimensional (3-D) image reconstruction methods were similar to those of other CBCT systems,¹³ and the focus of this study is on how the choice of detector technology affects two-dimensional (2-D) and 3-D image quality. We, therefore, expect the results reported herein to be generalizable with respect to the choice of detector (a-Si:H or CMOS), and we did not investigate differences associated with the C-arm platform itself (e.g., isocentric versus nonisocentric orbits). Images were acquired using fluoroscopic, radiographic, and CBCT imaging protocols as detailed below.

Each detector has a field of view of $30\times 30{\mathrm{cm}}^2$ with a native pixel pitch of 0.194 mm (a-Si:H FPD) or 0.151 mm (CMOS FPD). Previous work¹³ reported on the technical performance characteristics of each detector in the context of mobile C-arm fluoroscopy and CBCT. Key findings from the technical assessment included an approximate threefold decrease in electronic noise for CMOS ($\sim 1300$ e-rms for the a-Si:H FPD versus $\sim 430$ e-rms for the CMOS FPD), an approximate sevenfold decrease in temporal image lag for CMOS (first-frame lag of $\sim 2.4\%$ for a-Si:H versus $\sim 0.35\%$ for CMOS), and a greater than 20% increase in the DQE at high spatial frequencies for CMOS under low-exposure conditions.

Two fresh adult male cadavers were used in the current work, each of average body habitus, with estimated weights of 65 to 75 kg and approximate body mass index values of 20. One cadaver exhibited a collapsed lung, and the other presented with an automated, implantable cardioverter-defibrillator. Several exemplary medical devices were introduced into the cadavers to emulate a range of image-guided interventional scenarios. Tools used included a biliary stent (E-LUMINEXX, Bard Peripheral Vascular, Tempe, Arizona), a snare and catheter (Medtronic, Minneapolis, Minnesota), a hemodialysis catheter (Bard Peripheral Vascular), a biopsy needle (Bard Peripheral Vascular), a pigtail drain (Boston Scientific, Marlborough, Massachusetts), a Berman wedge catheter (Boston Scientific), a gastrojejunostomy tube (Kimberly-Clark, Irving, Texas), an angioplasty balloon ($5\times 10{\mathrm{mm}}^2$) (OptoPro, Cordis Corporation, Hialeah, Florida), an embolization coil (0.038, Cook Medical, Bloomington, Indiana), a balloon-expandable covered stent (iCast, Atrium Medical Corp, Hudson, New Hampshire), two varieties of microcatheter (Renegade HI-FLO, Boston Scientific, and E-10, Medtronic), a vascular stent (E-LUMINEXX, Bard Peripheral Vascular), a helical coil (Boston Scientific), and an endoprosthesis (GORE EXCLUDER AAA, Gore Medical Products Division, Flagstaff, Arizona).

Acquired images focused on various anatomic sites, including the head, neck, chest, abdomen, and pelvis. Several interventions were simulated using the interventional devices described above and imaged using fluoroscopic, radiographic, and CBCT acquisition in both low-dose and high-dose protocols.

Radiographs were acquired at 100 kV, 0.5 mAs, (low dose) and 100 kV, 1 mAs (high dose) settings. Fluoroscopic images were acquired at 100 kV, 3 mA, 5 ms (low dose), and 100 kV, 6 mA, 5 ms (high dose) at 30 frames per second. The entrance air kerma at the detector was ∼11 nGy in low-dose fluoroscopic settings ($\sim 0.4\mu \mathrm{Gy}$ in low-dose radiographs) and $\sim 20\mathrm{nGy}$ in high-dose fluoroscopic settings ($\sim 0.7\mu \mathrm{Gy}$ in high-dose radiographs). CBCT images were acquired over a 333-deg arc at 100 kV with 2 scan protocols: low-dose protocol (11 mA, 5 ms, 250 projections/scan) and high-dose protocol (30 to 47 mA, 5 ms, 250 to 500 projections/scan), with the default settings for the high-dose protocol being 30 mA, 5 ms, and 250 projections/scan. CBCT dose (weighted central and peripheral air kerma measured in a computed tomography dose index body phantom) was $\sim 0.7\mathrm{mGy}$ for the low-dose protocol and $\sim 1.9\mathrm{mGy}$ for the default high-dose protocol (and up to 5.2 mGy for a scan acquired at 47 mA, 5 ms, and 500 projections/scan). Images were reconstructed using a smooth protocol for images focusing on visualizing soft tissue and a sharp protocol for images focusing on visualizing bone or devices. No corrections were performed for x-ray scatter or beam hardening.

In total, 15 fluoroscopy runs, 12 radiographs, and 18 CBCT image sets were acquired using both CMOS and a-Si:H detectors, totaling 90 images used in the observer studies described below. Each image presented various tasks to the observer, totaling 242 tasks.

2.2. Monitors and Digital Imaging and Communications in Medicine Viewer

The images were evaluated on two identical, monochrome diagnostic-quality displays (MDCG-3210, Barco, Inc., Kortrijk, Belgium) using the RadiANT digital imaging and communications in medicine (DICOM) viewer (Medixant, Poznan, Poland). The two monitors were placed side by side in a reading environment with dimmed lighting as in a typical reading room. Observers had unrestricted access to the image manipulation tools provided by the DICOM viewer, including window and level, zoom, multiplanar views, and measurement tools, among others. Radiographs were displayed as 2-D still images; fluoroscopic images were displayed as time series (with user control of frame rate and pause); and CBCT images were displayed as orthogonal slices (with user control of slice scrolling, slice averaging, and volumetric rendering, if desired).

2.3. Image Evaluation

Images were evaluated by three fellowship-trained interventional radiologists (herein, “observers”) with various levels of experience (19, 17, and 6 years). A custom program developed in MATLAB, version 9.4 (The Mathworks, Inc., Natick, Massachusetts) deidentified the images, loaded the images onto the monitors, and scaled the window and level so that the CMOS and a-Si:H images were displayed at similar levels of contrast (which could be subsequently adjusted by the observer). Images used for particular tasks were shown on the left or right monitor (randomized, with one image from the CMOS detector and one from the a-Si:H detector) in randomized reading order. Approximately one-third of the image pairs were repeated to evaluate intraobserver repeatability.

Observers were asked to perform a series of diagnostic/interventional tasks relevant to each pair of images. Tasks did not query the observers’ overall impressions of image quality. For example, when the object of focus was a stent, observers were asked to rate their preference between the two images in distinguishing the struts of the stent (against a background of soft tissue or bone). For fluoroscopy images, tasks included those affected by temporal resolution (e.g., evaluating the motion of a catheter tip or the rate of expansion of a balloon-deployed stent). Observers had unlimited time to perform each task within a 1-h session. Up to three sessions were allowed, if necessary. A list of example tasks for each modality is provided in Tables 1–3.

Table 1.

List of example tasks: radiographs.

Device or anatomic location	Task
Abdomen	Follow segments of the bowel
	Determine the bowel wall lining integrity
	Distinguish the spinal processes
	Determine the degree of scoliosis
	View and measure the intervertebral disc space
Chest	Distinguish the patency of the airway down to the carina
	Determine the position of the airway relative to the clavicle heads
	Determine the degree of scoliosis
	Distinguish the mediastinal borders from the lung
	Distinguish the lung parenchyma from air in the collapsed lung
	Distinguish the trabeculae of the bone/likelihood of fracture in the ribs
Pelvis	Distinguish the pelvic rim
	Identify the sacroiliac joint spaces
	Identify the arcuate lines
	Measure the pubic symphysis space
Broken stent (PA view)	Distinguish the struts of the stent against soft tissue and against bone
	Distinguish the end markers of the stent against soft tissue and against bone
	Identify the defect in the stent
Broken stent implanted	Distinguish the struts of the stent against soft tissue and against bone
	Distinguish the end markers of the stent against soft tissue
	Identify the defect in the stent
Biliary stent	Distinguish the struts of the stent against soft tissue and against bone
	Distinguish the end markers of the stent against soft tissue
Balloon-expandable stent	Distinguish the struts of the stent against soft tissue

Table 2.

List of example tasks: fluoroscopy.

Device	Task
Berman wedge catheter	Distinguish the balloon edges as it fills with air
	Follow the distal 5-cm tip of the catheter
	See the catheter edge against soft tissue
Biopsy needle	Follow the needle edge against soft tissue and against bone
	Distinguish the direction of the needle’s bevel as it travels through the body
Bolus tube	See the catheter edge against soft tissue and against bone
	Follow the contrast as it moves against soft tissue
Endoprosthesis	Distinguish the end markers of the balloon
	Follow the edge of the deflated balloon
	Accurately track the expansion of the balloon as it fills with air
	Follow the edge of the inflated balloon
Helical coil	Distinguish the coil from the catheter
	Follow the coil edge as it is deployed
	Follow the coil tip as it is deployed
	Distinguish the catheter edge against soft tissue
Balloon-expandable stent	Distinguish the struts of the stent against soft tissue
	Distinguish the rate of expansion of the stent (balloon expanded)
	Distinguish the end markers of the stent against soft tissue
Microcatheter	See the catheter tip/end marker
	Distinguish the catheter edge against soft tissue and against bone
	Differentiate the wire from the catheter as it is deployed
	Follow the wire trajectory as it is deployed
OptoPro balloon2	Distinguish the end markers of balloon against soft tissue
	Distinguish the catheter edge against soft tissue
	Distinguish the catheter tip against soft tissue
	Distinguish the balloon edge (inflated) against soft tissue
	Accurately follow the balloon filling with contrast
Pigtail drain	Accurately and easily follow the side holes as the drain is deployed
	Accurately follow the drain tip trajectory during deployment
Snare	See the catheter tip
	See the shaft edge against soft tissue and against bone
	Accurately and easily follow the snare loop as it rotates

Table 3.

List of example tasks: CBCT.

Device or anatomic location	Task
Head	Identify and evaluate the cochlea and semicircular canals
	Evaluate the inner table contour of the calvarium
	Identify individual mastoid air cells
	Determine the angle of nasal septum deviation
	Differentiate bone from soft tissue in nasal sinuses
Lower abdomen	Follow the bowel
	Identify and measure the bowel wall thickness
	Evaluate the integrity of the aorta wall
	Evaluate the integrity of the vena cava wall
	Differentiate the psoas muscles
	Evaluate the integrity of the rectus abdominis muscles
	Evaluate the vertebral body contour
	Differentiate the erector spinae musculature
	Pinpoint the inferior mesenteric artery origin
	Follow the mesenteric branches
Pelvis	Evaluate the bladder size and wall thickness
	Evaluate the integrity of the internal iliac vessels
	Identify individual parailiac lymph nodes
	Follow the ureters from the renal pelvis to the bladder
Broken stent implant	Distinguish the end markers of the stent
	Detect the broken strut of the stent
	Distinguish the struts of the stent
Upper abdomen	Follow the bowel
	Identify and measure the bowel wall thickness
	Evaluate the integrity of the aorta wall
	Evaluate the integrity of the vena cava wall
	Differentiate the psoas muscles
	Evaluate the vertebral body contour
Neck	Differentiate the muscle fat in the neck area
Chest	Detect a pneumothorax in the apex of the left lung

A seven-point scale was used to score preferences among the image pairs, with positive numbers ($+1$ to $+3$) indicating a preference for performing the task using the image displayed on the right monitor; negative numbers ($-1$ to $-3$) indicating a preference for performing the task using the image displayed on the left monitor; and 0 indicating no preference. Absolute scores of 1, 2, and 3 indicated slight, moderate, and strong preferences, respectively.

2.4. Statistical Analysis

Data were analyzed using Stata, version 14 (StataCorp LLC, College Station, Texas) and MATLAB, version 9.4, software. Boxplots and pie charts showed the distribution of scoring within subsets of data, and Wilcoxon rank sum tests were used to assess the strength of preference. Only data for which a preference was observed (i.e., a rating other than 0) were included in the analysis. Subsets of data were analyzed to investigate particular scenarios affecting task performance, including dose level (low or high), subject of interest (soft tissue, bone, or interventional device), and type of image (e.g., visibility in static images for radiography or CBCT or in dynamic images for fluoroscopy). Data were analyzed for each observer and in aggregate. Statistical significance was determined according to $p$-values from the rank sum test ($\alpha =0.05$). Bonferroni corrections for multiple subgroup analyses were not applied to the resulting $p$-values.

To assess intraobserver consistency, we calculated the proportion of image pairs for which an observer gave the same preference for a particular image on repeated pairs. To assess interobserver consistency, we calculated the proportion of image pairs for which all observers reported the same preference between image pairs. A one-sided binomial test was used to assess intra- and interobserver consistency. These results were compared with proportions expected if observers had selected their preferences at random. For intraobserver consistency, a score of 0.5 corresponded to an observer choosing preferences at random (i.e., the observer would choose the image produced by the same detector 50% of the time on the repeated image pair). For interobserver validity, a score of 0.25 corresponded to all three observers choosing their preferences at random (i.e., all observers would choose the same image for a given pair in one of four trials).

3. Results

Example image pairs are shown in Fig. 1 (radiographs) and Fig. 2 (CBCT slices). When all images were aggregated, a preference was observed in 52% of tasks, as summarized in Table 4. Considering only those tasks for which a preference was observed and aggregating over all modalities, tasks, and observers, images produced by the CMOS system were slightly preferred (median score = 1; interquartile range [IQR] = 1; $p=0.002$).

Fig. 1.

Example radiographs used in the study. Low-dose radiographs of the pelvis taken with (a) a-Si:H and (b) CMOS FPDs. Two of the tasks performed with this image pair were to “distinguish the pelvic rim” and “identify the sacroiliac joint spaces.” High-dose radiographs of the abdomen taken with (c) a-Si:H and (d) CMOS FPDs. Four of the tasks performed with this image pair were to “follow the segments of the bowel,” “determine the bowel wall lining integrity,” “distinguish the spinal processes,” and “measure the intervertebral disc spaces.” Magnification of insert images is $2.4\times$.

Fig. 2.

Example CBCT images used in the study. (a), (b) Cadaver chest with an implanted broken stent. Axial images were acquired with the low-dose protocol and reconstructed using the sharp reconstruction kernel using (a) a-Si:H and (b) CMOS FPDs. Two of the tasks performed with this image pair were to “distinguish the struts of the stent” and “distinguish the end markers of the stent.” Magnification of image insets in (a) and (b) is $3\times$. (c)–(f) The second pair of images are axial CBCT images of the head of the cadaver acquired using the low-dose protocol with (c) a-Si:H and (d) CMOS FPDs and the high-dose protocol with (e) a-Si:H and (f) CMOS FPDs, each reconstructed with a sharp kernel. Two of the tasks performed with this image pair were to “identify individual mastoid air cells” and “identify and evaluate the cochlea and semicircular canals.” Magnification of inset images in (c)–(f) is $1.8\times$.

Table 4.

Observer preferences according to imaging and task variables.

Variable by observer	$N$ (%)			$p$ Value*
	No preference	a-Si:H preferred	CMOS preferred
Imaging modality
Radiography
Observer 1	15 (33)	10 (22)	21 (46)	0.254
Observer 2	12 (26)	17 (37)	17 (37)	0.111
Observer 3	19 (41)	12 (26)	15 (33)	0.870
Total	46 (33)	39 (26)	53 (38)	0.825
Fluoroscopy
Observer 1	55 (63)	3 (3.4)	29 (33)	0.106
Observer 2	46 (53)	9 (10)	32 (37)	0.595
Observer 3	43 (49)	12 (14)	32 (37)	0.007
Total	144 (55)	24 (9.2)	93 (36)	0.002
CBCT
Observer 1	52 (48)	22 (20)	35 (32)	0.033
Observer 2	55 (50)	23 (21)	31 (28)	0.016
Observer 3	58 (53)	4 (3.7)	47 (43)	0.411
Total	165 (50)	49 (15)	113 (35)	0.004
Imaging protocol
Low dose
Observer 1	68 (51)	15 (11)	50 (38)	0.021
Observer 2	59 (44)	24 (18)	50 (38)	0.370
Observer 3	62 (47)	21 (16)	50 (38)	0.046
Total	189 (47)	60 (15)	150 (38)	0.001
High dose
Observer 1	54 (50)	20 (18)	35 (32)	0.056
Observer 2	54 (50)	25 (23)	30 (28)	0.768
Observer 3	58 (53)	7 (6.4)	44 (40)	0.243
Total	166 (51)	52 (16)	109 (33)	0.360
Task subject
Interventional device
Observer 1	70 (55)	8 (6.3)	49 (39)	0.035
Observer 2	56 (44)	21 (17)	50 (39)	0.150
Observer 3	68 (54)	14 (11)	45 (35)	0.016
Total	194 (51)	43 (11)	144 (38)	0.015
Anatomy
Observer 1	52 (45)	27 (23)	36 (31)	0.327
Observer 2	57 (50)	28 (24)	30 (26)	0.016
Observer 3	52 (45)	14 (12)	49 (43)	0.192
Total	161 (47)	69 (20)	115 (33)	0.174
Resolution type
Temporal resolution (fluoroscopy)
Observer 1	21 (91)	0 (0)	2 (8.7)	0.079
Observer 2	15 (65)	4 (17)	4 (17)	0.500
Observer 3	11 (48)	5 (22)	7 (30)	0.106
Total	47 (68)	9 (13)	13 (19)	0.072
Spatial resolution
Observer 1	101 (46)	35 (16)	83 (38)	0.002
Observer 2	98 (45)	45 (21)	76 (35)	0.566
Observer 3	109 (50)	23 (11)	87 (40)	0.718
Total	308 (47)	103 (16)	246 (37)	0.006

Note: CBCT, cone-beam computed tomography; CMOS, complementary metal–oxide semiconductor; a-Si:H, hydrogenated amorphous silicon.

^*From Wilcoxon rank sum test. $p<0.05$ was considered significant.

3.1. Preference by Imaging Modality (Radiography, Fluoroscopy, or CBCT)

In scenarios for which a preference was observed, CMOS was significantly preferred for fluoroscopy ($p=0.002$) and CBCT ($p=0.004$), each with a median value of 1 (slight preference) (fluoroscopy IQR = 1; CBCT IQR = 1), but not for radiography ($p=0.825$) (Fig. 3).

Fig. 3.

Distribution of task performance rankings for CMOS and a-Si:H images by modality (radiography, fluoroscopy, and CBCT). (a) Box plots show the overall distribution in reader preferences, with the x symbol marking the average, the rectangular box showing the IQR, and range bars showing overall range. (b) Pie charts show the distribution of preferences according to detector type, e.g., a fairly even split in radiography and a preponderance in preference of CMOS in fluoroscopy and CBCT.

3.2. Preference by Imaging Protocol (Low- or High-Dose)

The CMOS system was significantly preferred in low-dose settings ($p=0.001$), again with a median value of 1 (slight preference) ($\mathrm{IQR}=1$) (Fig. 4). There was no statistically significant preference observed between detector types at high-dose settings ($p=0.360$). When subanalyzing by the three modalities in low-dose settings, CMOS was significantly preferred for low-dose fluoroscopy and CBCT tasks ($p=0.003$ and 0.008, respectively).

Fig. 4.

Distribution of task performance between CMOS and a-Si:H images by imaging protocol (low-dose or high-dose). As in Fig. 3, (a) box plots show the overall range in preferences and (b) pie charts show the breakdown in preferences by detector type (if any).

3.3. Preference by Subject of Interest (Anatomy or Interventional Device)

Images produced by the CMOS system were slightly preferred (median score 1) when the subject of focus was an interventional device ($p=0.015$; Fig. 5). When anatomy was the subject of focus, observers had no significant preference ($p=0.174$).

Fig. 5.

Distribution of task performance rankings for CMOS and a-Si:H images by subject of focus (viz., interventional device or anatomy). As in Figs. 3 and 4, (a) box plots show the overall distribution in preferences and (b) pie charts show the breakdown of preferences (if any).

3.4. Preference by Resolution (Spatial or Temporal)

The CMOS system was slightly preferred (median score = 1; $\mathrm{IQR}=1$) for tasks requiring higher levels of spatial resolution (i.e., visualization of fine details) ($p=0.006$; Fig. 6). Observers reported no significant preference for tasks focused on temporal resolution (i.e., clarity of motion in fluoroscopy images) ($p=0.072$).

Fig. 6.

Distribution of task performance for CMOS and a-Si:H images by “resolution type” (i.e., spatial or temporal resolution). As in Figs. 3–5, (a) box plots show the overall range of preference and (b) pie charts show the breakdown of preferences (if any) by detector type.

3.5. Strength of Preferences

For several tasks, CMOS was moderately (median score = 2) or strongly (median score = 3) preferred by all three observers. Radiographs centered on the abdomen or pelvis were consistently rated with moderate preference (median score = 2) for CMOS, illustrated by differences in quantum noise and fine details evident in Fig. 1. For fluoroscopy tasks that required observers to resolve the contrast between instruments (moving or stationary) and surrounding tissues, CMOS was moderately preferred (median score = 2). For example, a $+2$ rating (in favor of CMOS) was given consistently when the task was to evaluate the rate of expansion of an iCast balloon expandable covered stent, as well as for the task of tracking the expansion of an OptoPro balloon as it filled with air. With CBCT, observers reported a moderate preference for CMOS (median score = 2) for tasks resolving fine anatomical details (e.g., mastoid air cells and semicircular canal) and the edges of instruments (e.g., struts and end markers of stents), as shown in Fig. 2.

3.5.1. Intraobserver and interobserver consistency

The degree of intraobserver consistency was evident in the rate at which the same choice was made by a given observer for the preferred system (i.e., CMOS or a-Si:H). In these studies, observers were self-consistent in the preferred detector type in 81% of the repeated tasks ($p<0.001$), and they were consistent in both the detector type and the rank score (score = 1, 2, or 3) in 68% of the repeated tasks. In terms of interobserver consistency, the same choice of the preferred detector was made in 13% of the tasks; moreover, in 3.7% of tasks, the same ranking was given ($p<0.001$).

4. Discussion

In a head-to-head comparison, we studied the imaging performance of mobile C-arms incorporating a CMOS or a-Si:H FPD for use in various 2-D and 3-D interventional tasks. As such, our results are specific to the question of x-ray detector (and not the C-arm platform itself). Many of the results of this study corroborate the technical assessment of performance metrics such as MTF and DQE reported by Sheth et al.,¹³ in which the CMOS system demonstrated objective advantages associated with a threefold reduction in electronic noise and slight improvement in spatial resolution. These advantages were most evident in low-dose imaging protocols, consistent with superior noise-equivalent quanta for the CMOS system. The study reported here helps to bridge the relationship between physics-based performance evaluation and observer preference with respect to pertinent clinical imaging tasks.

A preference was observed for approximately half of the tasks presented, and the CMOS system was preferred in 71% of such tasks. The remaining 29% of tasks for which the a-Si:H system was preferred appeared to relate to visualization of soft-tissue anatomy imaged in CBCT. Overall, observers expressed a slight preference (median score = 1) for the CMOS system for general use and moderate (median score = 2) or strong (median score = 3) preference in only a few specific tasks (i.e., those involving resolution of fine details at low-dose settings).

Considering the tasks performed using images acquired with low- or high-dose protocols, the CMOS system was significantly preferred for images acquired with low-dose settings, likely reflecting the effect of electronic noise and photon starvation as reported by Sheth et al.¹³ The advantages of CMOS in low-dose imaging protocols were most evident with fluoroscopy and CBCT.

The CMOS detector has been reported to offer a sevenfold reduction in image lag compared to the a-Si:H detector.¹³ Although the CMOS system was preferred for tasks involving motion, the difference was not as large as might be expected given such a large difference in image lag characteristics. A feature of CMOS detectors that was not investigated in the current work is the capacity for fluoroscopic image acquisition at even higher frame rates ($>30$ frames per second), which might be useful in scenarios such as cardiac interventions, pulmonary interventions, and barium swallow studies.

Preference for the CMOS system was significant for tasks involving an interventional instrument, perhaps because of the slight improvement in spatial resolution. Considering this observation and the preference for CMOS in low-dose conditions, the CMOS system might be preferred in interventional procedures involving long exposure times (e.g., placement of transjugular intrahepatic portosystemic shunts, revascularization of chronic occlusions, and neuroembolization).

Throughout the study, observers commented that it was initially difficult to discern differences between the two images presented, and upon closer inspection, they appreciated subtle differences in noise and resolution at a level that might affect task performance. In most tasks for which a preference was observed, preference was judged to be “slight” (median score = 1). Although our data suggest that the CMOS system might improve performance for certain diagnostic and interventional tasks, for a large proportion of tasks (48%), observers reported no preference between the two detectors. Another limitation of the current work is that the study was conducted in a controlled reading environment, representative of a diagnostic environment in an interventional radiology suite. The controlled study environment was consistent with methods used in observer studies to reduce bias and evaluate potentially subtle differences between imaging technologies. Future studies might consider investigation of the extent to which such differences persist in an actual clinical interventional environment, including variations in the quality of display, viewing distance, and pressure to read large volumes of images in a short time (compared with the unrestricted reading time in this study).

5. Conclusion

This study suggests measurable advantages in imaging performance for pertinent interventional imaging tasks for mobile C-arms incorporating a CMOS detector, particularly in fluoroscopy and CBCT and for low-dose imaging protocols. In most cases, the preference for CMOS detector images was “slight” (median score = 1), and only a few scenarios exhibited moderate (score = 2) or strong (score =3) preference for CMOS. The observations gained from the expert readers study are consistent with the improved MTF and DQE characteristics of CMOS detectors and provide an important mapping of how such objective measures relate to real observer performance. Other factors, such as cost and long-term robustness of the system in routine clinical use, could certainly play a role in selection between the two detector types. Although the studies showed that images acquired with CMOS detector were often preferable in low-dose imaging protocols, careful implementation is required to ensure that the resulting image quality for such protocols is actually sufficient for the imaging task.

Biographies

Godwin O. Abiola received his BA degree in biomedical engineering at Harvard University and his MD degree at the Johns Hopkins School of Medicine. He is currently undergoing residency training at the Beth Israel Deaconess Medical Center.

Niral M. Sheth is a research scientist in the Department of Biomedical Engineering at Johns Hopkins University, working with advanced x-ray detectors. His research includes the characterization of 2-D/3-D imaging performance of CMOS-based flat panel detectors for specific CBCT applications. Niral has his master’s degree in electrical engineering from the University of California, Berkeley, with a background in analog circuit and embedded systems design.

Wojciech Zbijewski is faculty at the Department of Biomedical Engineering at Johns Hopkins University. His main research interests are in development of x-ray imaging systems and in algorithms for image reconstruction and quantitative image analysis. He obtained his PhD from University of Utrecht, the Netherlands, with a thesis on model-based CT reconstruction. Before joining Johns Hopkins, he was a research scientist at Xoran Technologies, working on portable CB CT for head imaging.

Matthew W. Jacobson obtained his MSc degree from the Technion—Israel Institute of Technology and PhD from the University of Michigan. With over 15 years of industry and academic experience, his research interests include dose-efficient image reconstruction methods for PET and CT, patient motion tracking and compensation, novel geometries and calibration methods for tomographic systems, and accelerated 3-D iterative reconstruction. Most recently, his work has focused on CBCT image guidance for radiation oncology.

Christopher Bailey is currently the chief resident (PGY-5) of the interventional radiology residency at Johns Hopkins Hospital in Baltimore, Maryland. He plans to continue his career in academic interventional radiology with a focus on vascular malformation treatment.

John Filtes is an interventional radiology and diagnostic radiology resident in training at Columbia University Medical Center/New York-Presbyterian Hospital. He earned his MD degree from SUNY Downstate. He is interested in biomedical innovation and biodesign, especially in the fields of interventional and diagnostic radiology.

Gerhard Kleinszig is the director for innovation and collaborations at Siemens Healthcare GmbH leveraging academic and clinical partnerships to advance intraoperative imaging for different interventional and surgical disciplines. His research interests support innovations for advanced therapies at Siemens Healthineers and have covered image processing, machine learning for surgical/interventional planning, image guidance, and quality assurance.

Sebastian K. Vogt is the director for research and collaborations at Siemens Medical Solutions USA, Inc., leveraging industry-academic partnerships to advance x-ray imaging research for radiography, women’s health, and surgery applications. His research interests support innovations for diagnostic imaging and advanced therapies at Siemens Healthineers and have covered image processing, machine learning, augmented reality image guidance, cardiovascular computed tomography, mobile C-arm, breast tomosynthesis, and twin-robotic x-ray technologies and applications.

Stefan Söllradl is working as R&D professional and detector expert at Siemens Healthineers in the field of mobile C-arms. He gained his PhD from the University of Bern in the field of prompt-gamma activation analysis and neutron tomography. After that he worked as staff member at the neutron source of the TU München (FRM II) in the field of nondestructive testing with fission neutrons at the instrument NECTAR.

Jens Bialkowski is worked as research assistant at the University of Erlangen-Nuremberg, Germany from 2001 to 2007. He received his degree of Dr.-Ing. in electrical engineering in 2008 at the University of Erlangen-Nuremberg, Germany. The topic was image processing and low-complexity transcoding of video streams. Since 2007, he has been responsible for several HW/SW projects related to x-ray image processing at Siemens Healthineers GmbH.

William S. Anderson received his medical degree from the Johns Hopkins School of Medicine, and subsequently performed a neurosurgical residency there. He completed a functional neurosurgery fellowship in the Department of Neurosurgery from 2005 to 2007. He served as an attending neurosurgeon at Brigham & Women’s Hospital from 2008 to 2010, and additionally holds a PhD in physics from Princeton University. He is currently an associate professor and director of functional neurosurgery at Johns Hopkins.

Jeffrey H. Siewerdsen is a professor of biomedical engineering, computer science, radiology, and neurosurgery at Johns Hopkins University. He received his PhD in physics from the University of Michigan, where he worked on the early development of flat-panel x-ray detectors. He helped to develop the first systems for CBCT guidance of radiotherapy as well as CBCT systems for image-guided surgery and orthopedic imaging. His research focuses on the physics of image quality in digital x-ray imaging, CT, and CBCT, and the development of new image registration techniques for image-guided interventions.

Clifford R. Weiss is an associate professor in the division of vascular and interventional radiology in the Russell H. Morgan Department of Radiology and Radiological Science at the Johns Hopkins University School of Medicine. He also holds appointments in surgery and biomedical engineering. His clinical focus is on the treatment of complex vascular malformations, and his research is focused on the development of new embolic therapies and of interventional devices.

Disclosures

G.K., S.V., S.S., and J.B. are employees of Siemens Healthineers. W.S.A. serves on the advisory boards for Longeviti Neuro Solutions and NeuroLogic Solutions and is a paid consultant for Globus Medical.