On the relevance of modulation transfer function measurements in digital mammography quality control

Abstract.

Purpose: The relevance of presampling modulation transfer function (MTF) measurements in digital mammography (DM) quality control (QC) is examined. Two studies are presented: a case study on the impact of a reduction in MTF on the technical image quality score and analysis of the robustness of routine QC MTF measurements.

Approach: In the first study, two needle computed radiography (CR) plates with identical sensitivities were used with differences in the 50% point of the MTF ($f_{\mathrm{MTF}0.5}$) larger than the limiting value in the European guidelines ($>10\%$ change between successive measurements). Technical image quality was assessed via threshold gold thickness of the CDMAM phantom and threshold microcalcification diameter of the L1 structured phantom. For the second study, presampling MTF results from 595 half-yearly QC tests of 55 DM systems (16 types, six manufacturers) were analyzed for changes from the baseline value and changes in ${\mathrm{f}}_{\mathrm{MTF}0.5}$ between successive tests.

Results: A reduction of 20% in $f_{\mathrm{MTF}0.5}$ of the two CR plates was observed. There was a tendency to a lower score for task-based metrics, but none were significant. Averaging over 55 systems, the absolute relative change in $f_{\mathrm{MTF}0.5}$ between consecutive tests (with 95% confidence interval) was 3% (2.5% to 3.4%). Analysis of the maximum relative change from baseline revealed changes of up to $-10\%$ for one a-Se based system and $-15\%$ for a group of CsI-based systems.

Conclusions: A limit of 10% is a relevant action level for investigation. If exceeded, then the impact on performance has to be verified with extra metrics.

1. Introduction

Sharpness is an important characteristic of medical images and relates to the ability of an imaging system to visualize fine details. In mammography, image sharpness is of fundamental importance when detecting and describing microcalcifications, mass boundaries, and architectural distortions. Different sources of radiographic blurring, including the finite size of the x-ray focal spot and detector characteristics (e.g., detector element size, sampling pitch, and pixel binning), can degrade the sharpness of the image. For the current geometries in two-dimensional (2D) digital mammography (DM), x-ray detector sharpness is usually the dominant factor in determining overall sharpness.^1,2

Quality control (QC) protocols implemented by medical physicists as part of the regulatory framework controlling x-ray system usage include tests to quantify the sharpness.³ One measure used is the presampling modulation transfer function (MTF),^4,5 a spatial frequency domain metric that quantifies how the spatial frequency content of an object is transferred through to the image.⁶ In medical x-ray imaging, the slanted edge method is the most common means of measuring MTF.^4,7 This method has been standardized in the form of the International Electrotechnical Commission (IEC) protocol⁸ for use by equipment manufacturers to benchmark components and has been implemented by medical physicists for QC purposes.^3,9 MTF curves can be summarized in terms of the frequency at which the MTF falls to 0.50 ($f_{\mathrm{MTF}0.5}$) and the MTF value at $5\text{cycles}/\mathrm{mm}$ (${\mathrm{MTF}}_{f5}$) for current mammography detectors. Some protocols define acceptable and remedial levels as means of catching changes in the MTF. For example, in the European Protocol for DM,³ the limiting value for a change in $f_{\mathrm{MTF}0.5}$ between successive (half yearly) QC tests is 10%. Other protocols include a comparison to baseline values. In setting the value of 10%, an early publication⁹ was taken into account that showed typical short-term variations [expressed as a coefficient of variation (COV)] of 1% for a given measurement point and $\sim 3\%$ for measurements on a stable system over a period of 17 months. Given this expected variation, 10% was chosen as signifying a change in MTF for six monthly QC measurements that should lead to further investigation. While this was an initial practical estimate, the relevance of this 10% limit, and hence of the MTF measurements performed for QC purposes, has not been fully explored for a large dataset of QC measurements and is the focus of this study.

This work addresses two different aspects that we hope will convince medical physicists of the value of MTF measurements and encourage the widespread use of these metrics in routine QC practice. These can be framed as follows: (1) If the MTF limiting value is exceeded, is there a problem with the performance of the device? and (2) What magnitude in MTF variation should we expect for a large cohort of systems that are known to be performing well, when tested on a half yearly basis by a medical physics service, and how does this relate to the current 10% limiting value? Answering these related questions forms the two parts of this work.

In part 1, we determine the consequences, in terms of technical image quality, of a reduction in MTF by an amount larger than the limiting value in the European Protocol.³ To do this, we were provided with two needle computed radiography (CR) plates with a difference in $f_{\mathrm{MTF}0.5}$ larger than the limiting value, yet with identical x-ray sensitivity. CR plates were chosen for the investigation because the impact of the sharpness of the detector on technical image quality can then be tested with the same x-ray tube and identical geometry. Technical image quality was assessed via two methods: threshold gold thickness measured with the CDMAM phantom and threshold microcalcification diameter of the L1 structured phantom. These test objects combine individual aspects of system performance (the large area contrast, the sharpness, and noise) into a single technical image quality score. In the CDMAM phantom, the signal consists of small gold disks on a homogenous background and is analysed with computerized reading. This phantom was judged relevant to study the impact of a change in MTF on performance because a correlation had been shown between the threshold gold thicknesses measured with the phantom and calcification detection in a virtual clinical trial.¹⁰ The L1 phantom was scored by human readers. This phantom has a structured background and calcification-like lesions of different sizes. Human reading is known to be associated with a relatively large variability, and the structured background adds an extra dimension to the variability of the results. While even small changes in the MTF are likely to influence small detail-detectability, whether this is manifested in changes to a technical image quality that contains reader and structured background variability is worthy of study.

In part 2, the reproducibility of MTF measurements made routinely on DM systems as part of the QC protocol was analysed to examine whether these results comply with current limiting values. To do this, a retrospective study of MTF measurements from DM half yearly QC data of systems under a strict daily QC program was performed. Measurements were performed by several team members with similar test equipment and after similar training programs, reflecting the real-life situation of medical physics testing. A major part of this study was also dedicated to the generation of typical values for 16 different models of DM systems, from six vendors.

2. Materials and Methods

2.1. Study 1: Relating Changes in MTF to Technical Image Quality Measures

In the first part of this work, technical image quality was assessed for two CR plates with MTF curves, characterized by $f_{\mathrm{MTF}0.5}$, that differed by an amount greater than the acceptable limit in the European Protocol.

2.1.1. Computed radiography systems

Two needle-phosphor CR plates^11,12 were made available by the manufacturer (Agfa HealthCare, Mortsel, Belgium) for this case study. The sharpness was known to be different due to a slightly different phosphor coating and anti-reflective layer on the substrate. The plates, designated CR1 and CR2, were read with the same DX-M flying spot laser digitizer at a $50\text{-}\mu \mathrm{m}$ pixel spacing, and images were acquired on the same Mammomat 3000 Nova mammography system (Siemens Healthineers, Germany). Detector sharpness was characterized by the presampling MTF, while the response function, normalized noise power spectrum (NNPS), and detective quantum efficiency (DQE) were assessed to confirm that detectors had the same x-ray sensitivity. All CR tests were performed in one session on the same day.

Detector response curve

Detector response was measured from flood images with a 2-mm thick 99% pure Al filter at the x-ray tube, a tube voltage of 30 kVp, and a Mo/Rh anode/filter (A/F) setting. The anti-scatter grid was removed. Homogeneously exposed images covering 11 detector air kerma (DAK) values, from 34 to $2443\mu \mathrm{Gy}$, were acquired, with $117\mu \mathrm{Gy}$ chosen as the reference value ($K_{\mathrm{ref}}$). The average pixel value (PV) was measured in a $5\times 5\mathrm{mm}$ region of interest (ROI) at the midpoint of the image (left–right) and 60 mm from the chest wall. The detector response curve was then fitted from the plot of PVs as a function of DAK.

Presampling modulation transfer function

The detector presampling MTF was measured with the slanted edge technique⁴ as implemented in the European Protocol.^3,8 A 60 mm by 120 mm stainless steel edge with a 0.8 mm thickness was used together with a 2 mm Al filter placed at the x-ray tube. The edge test object was angled and positioned at 60 mm from the chest wall; four edge images were then acquired with a 90 deg rotation of the edge test object between each acquisition (Fig. 1). This edge rotation accounted for differences in the signal generated as the laser scans from a low signal region to high signal region, and vice versa.^9,10 The response function was used to linearize the PV data before calculations. MTF curves for the left-right and front-back directions across the detector were calculated and averaged, and the $f_{\mathrm{MTF}0.5}$ and ${\mathrm{MTF}}_{f5}$ of the 2 plates were established.

Fig. 1.

MTF measurement setup.

Normalized noise power spectrum

The NNPS curves were calculated from the detector response flood images at $K_{\mathrm{ref}}$, $0.5\times K_{\mathrm{ref}}$, and $2\times K_{\mathrm{ref}}$, linearized to DAK before calculation. Three images were used for a given DAK level. The NPS was formed by calculating the 2D FFT of each extracted area using software written in IDL, following this equation:⁸

$$NPS(u,v)=\frac{\mathrm{\Delta }x\mathrm{\Delta }y}{M.N^2}\sum _{m=1}^M{|\sum _{i=1}^N\sum _{j=1}^N(I(x_i,y_j)-S(x,y))e^{-2\pi i(u_n{x}_i+v_k{y}_j)}|}^2,$$

in which $M$ is the number of ROIs, $N$ is the dimension of the ROIs, $\mathrm{\Delta }x$ is the pixel spacing in the x direction, $\mathrm{\Delta }y$ is the pixel spacing in the $y$ direction, $I(x_i,y_j)$ are the (linearized) pixel data, and $S(x,y)$ is a 2D polynomial function fitted to the entire extracted region used for NPS analysis. A region of $100\times 100{\mathrm{mm}}^2$ was used, and ROIs of $256\times 256\text{pixels}$ that overlap each other by 128 pixels were extracted from this area. The 2D spectra were averaged from an ensemble of 96 spectra and then radially averaged to form the final one-dimensional NPS curve.

Detective quantum efficiency

The detector DQE was calculated as

$$DQE(u)=\frac{MT{F}^2(u)}{q_0.K.NNPS(u)},$$

in which $q_0$ is the number of photons per unit air kerma $\mathrm{per}{\mathrm{mm}}^2$ for the relevant spectrum taken from the Boone spectral model¹¹ and $K$ is the air kerma at the detector for the given NNPS result.

2.1.2. Technical image quality tests

Two test object analyses were performed using the same CR plates: threshold gold thickness ($T$) measured using the CDMAM phantom^3,13 and the threshold diameter ($d_{\mathrm{tr}}$) of microcalcifications in the L1 phantom.¹⁴

CDMAM phantom

The CDMAM phantom is specified by the European Protocol as a means of measuring the contrast-detail detectability of DM systems (Fig. 2) and to evaluate whether a system conforms to minimum standards.³ To perform the contrast-detail measurement, the phantom was placed between two stacks of 20-mm poly (methyl methacrylate) (PMMA) to give a total PMMA equivalent thickness of 50 mm. A total of 16 for processing images were then acquired for each CR plate at identical acquisition factors of 30 kVp, Mo/Rh A/F, and 63 mAs [tube current-time product (mAs)]. The images were automatically evaluated using in-house software (${\text{Erica}}^2$) that implements the CDCOM module.¹⁵. This produces a matrix from the two CDCOM matrices, which is then processed according to the method of Young et al.¹⁶ For each disk diameter, a psychometric curve fit was applied to the percentage of correctly detected (PC) data and threshold contrast was calculated for a PC of 62.5%, which is halfway between 100% correct and random guessing in a four-alternative forced choice (4-AFC) experiment. The 16 values of threshold gold thickness for the 0.1- and 0.25-mm diameter disks were then input to a nonparametric test (Mann–Whitney test) to compute $p$-values for the significance test, to compare between the CR1 and CR2 plates.

Fig. 2.

(a) The physical³ and mammography images of the CDMAM phantom in (b) CR1 and (c) CR2.

L1 phantom

The L1 phantom is composed of a structured background and lesion-like objects within a semi-circular PMMA container with a thickness of 48 mm and diameter of 200 mm (Fig. 3). The structured background is formed from PMMA spheres with six different diameters (from 15.88 to 1.58 mm), all of them suspended in water. The test object has five microcalcification groups (95 to $237\mu \mathrm{m}$). Under typical mammography exposure settings, this phantom has an attenuation equivalent to a compressed breast thickness of 60 mm. Twelve images of the phantom were acquired for each of the CR plates. As with the CDMAM phantom, the acquisition factors were set at Mo/Rh, 30 kVp, and 63 mAs, with an anti-scatter grid in place.

Fig. 3.

(a) The L1 phantom and 2D images of the microcalcification group of 180 to $200\mu \mathrm{m}$ on (b) CR 1 and (c) CR 2.

The for presentation images, i.e., with clinical image processing applied, were analyzed using the 4-AFC paradigm. Five medical physicists who had been involved in extensive reading studies of the L1 phantom¹⁷ participated as observers. All readers read all images in the study. Twenty regions of interest (ROIs) with dimensions of $20\times 20{\mathrm{mm}}^2$ were segmented from individual images: five ROIs containing a calcification cluster in the center and 15 signal free background ROIs. In each cluster, five microcalcification simulating specs were arranged in a “five” pattern, with a distance of $\sim 5.4\mathrm{mm}$ between adjacent calcifications. The 4-AFC reading studies were performed on a five MP monitor calibrated to the DICOM GSDF standard (Barco, Kortrijk, Belgium) using in-house software for image scoring and statistical evaluation. A viewing distance of $\sim 40\mathrm{cm}$ was used with a constant ambient light level of six lux. The reading software allowed the contrast, brightness, and zoom level for given images to be adjusted. The percentage of correctly detected (PC) targets was obtained for each reader. A psychometric curve fit was applied to the averaged PC reader data as a function of diameter, and the threshold diameter ($d_{\mathrm{tr}}$) was defined as the point at which PC reached 62.5%, i.e., the midpoint between the 25% guessing level and the maximum score of 100%.

Statistical analysis was performed using GraphPad Prism 5, which produced two-sided 95% confidence intervals calculated using the Clopper–Pearson method. A nonparametric Mann–Whitney test was applied to compute $p$-values for the $d_{\mathrm{tr}}$ data and for the $T$ values calculated for the 16 CDMAM images to evaluate whether differences were significant.

2.2. Study 2: Reproducibility of Routine MTF Measurements in Digital Mammography

The second study analyzed MTF curves measured using the slanted edge method^3,4 as part of routine QC by our medical physics service. In our practice, three different tool kits are used by at least three people and in different locations. The tests had a six monthly frequency and were performed on 55 DM systems (16 models from six vendors), over the period 2008–2019. This yielded 595 measurements of $f_{\mathrm{MTF}0.5}$ and ${\mathrm{MTF}}_{f5}$ which were then checked and cleaned for obvious measurement errors and missing data. Incorrect system acquisition settings and image formats were also caught—this typically occurred at the commissioning test of a new device. Data suffering from these problems were excluded from the analysis, and further analysis was performed using a dataset of 573 tests. Next, the change in $f_{\mathrm{MTF}0.5}$ and ${\mathrm{MTF}}_{f5}$ between successive tests and their comparison with the system baseline value were calculated and compared against the limiting value of 10%.³ The number of tests per system was either 20 (i.e., covering 10 years of QC tests) or the number of tests that was available in that study period. The analysis was performed for data of the same detector.

2.2.1. Overview of the mammography systems

All of the systems analyzed in the QC program and some basic detector characteristics are listed in Table 1, and the technical acquisition factors of the MTF measurements are listed in Table 2. In addition to half-yearly tests, QC tests are performed daily by the local personnel, and the results are controlled remotely by the same medical physics team that performs the half-yearly QC tests.¹⁸

Table 1.

Devices involved in the study together with their detector specifications.

No	Mammography system	Acceptance test	Detector specifications
			Type	Technology	Pixel pitch ($\mu \mathrm{m}$)	Pixel matrix
1	Fuji Amulet (1)	January 11	Storage	a-Se/optical switch	50	$4728\times 5928$
2	Fuji Amulet (2)	November 10	Storage	a-Se/optical switch	50	$4728\times 5928$
3	Fuji Amulet (3)	August 10	Storage	a-Se/optical switch	50	$4728\times 5928$
4	Fuji Amulet f	August 16	Storage	a-Se/optical switch	50	$4728\times 5928$
5	Fuji Amulet s (1)	June 13	Storage	a-Se/optical switch	50	$4728\times 5928$
6	Fuji Amulet s (2)	October 13	Storage	a-Se/optical switch	50	$4728\times 5928$
7	Fuji Amulet s (3)	September 13	Storage	a-Se/optical switch	50	$4728\times 5928$
8	Fuji Amulet Innovality (1)	March 14	Storage	a-Se/optical switch	50	$4728\times 5928$
9	Fuji Amulet Innovality (2)	January 15	Storage	a-Se/optical switch	50	$4728\times 5928$
10	Fuji Amulet Innovality (3)	January 16	Storage	a-Se/optical switch	50	$4728\times 5928$
11	Fuji Amulet Innovality (4)	March 16	Storage	a-Se/optical switch	50	$4728\times 5928$
12	Fuji Amulet Innovality (5)	May 14	Storage	a-Se/optical switch	50	$4728\times 5928$
13	GEHC Senographe DS (1)	September 09	Scintillator	CsI/a-Si TFT switch	90	$1914\times 2294$
14	GEHC Senographe DS (2)	August 09	Scintillator	CsI/a-Si TFT switch	90	$1914\times 2294$
15	GEHC Senographe Essential (1)	June 11	Scintillator	CsI/a-Si TFT switch	100	$2394\times 3062$
16	GEHC Senographe Essential (2)	October 12	Scintillator	CsI/a-Si TFT switch	100	$2394\times 3062$
17	GEHC Senographe Essential (3)	September 10	Scintillator	CsI/a-Si TFT switch	100	$2394\times 3062$
18	GEHC Senographe Essential (4)	May 12	Scintillator	CsI/a-Si TFT switch	100	$2394\times 3062$
19	GEHC Senographe Essential (5)	May 12	Scintillator	CsI/a-Si TFT switch	100	$2394\times 3062$
20	GEHC Pristina (1)	Mar 17	Scintillator	CsI/a-Si TFT switch	100	$2394\times 2850$
21	GEHC Pristina (2)	January 19	Scintillator	CsI/a-Si TFT switch	100	$2394\times 2850$
22	GEHC Pristina (3)	October 17	Scintillator	CsI/a-Si TFT switch	100	$2394\times 2850$
23	Hologic Lorad Selenia (1)	January 11	Direct	a-Se/TFT switch	70	$3328\times 4096$
24	Hologic Lorad Selenia (2)	April 11	Direct	a-Se/TFT switch	70	$3328\times 4096$
25	Hologic Lorad Selenia (3)	January 11	Direct	a-Se/TFT switch	70	$3328\times 4096$
26	Hologic Selenia Dimensions (1)	October 11	Direct	a-Se/TFT switch	70	$3328\times 4096$
27	Hologic Selenia Dimensions (2)	June 12	Direct	a-Se/TFT switch	70	$3328\times 4096$
28	Hologic Selenia Dimensions (3)	March 13	Direct	a-Se/TFT switch	70	$3328\times 4096$
29	Hologic Selenia Dimensions (4)	November 13	Direct	a-Se/TFT switch	70	$3328\times 4096$
30	Hologic Selenia Dimensions (5)	February 13	Direct	a-Se/TFT switch	70	$3328\times 4096$
31	Hologic 3Dimensions (1)	December 18	Direct	a-Se/TFT switch	70	$3328\times 4096$
32	Hologic 3Dimensions (2)	May 18	Direct	a-Se/TFT switch	70	$3328\times 4096$
33	Hologic 3Dimensions (3)	Jan-18	Direct	a-Se/TFT switch	70	$3328\times 4096$
34	IMS Giotto Image 3D[L] (1)	July 16	Direct	a-Se/TFT switch	85	$2016\times 2816$
35	IMS Giotto Image 3D[L] (2)	August 08	Direct	a-Se/TFT switch	85	$2816\times 3584$
36	IMS Giotto Image 3D[L] (3)	June 10	Direct	a-Se/TFT switch	85	$2016\times 2816$
37	IMS Giotto Class (1)	December 17	Direct	a-Se/TFT switch	85	$2812\times 3580$
38	IMS Giotto Class (2)	September 17	Direct	a-Se/TFT switch	85	$2812\times 3580$
39	IMS Giotto Class (3)	August 17	Direct	a-Se/TFT switch	85	$2812\times 3580$
40	IMS Giotto Class (4)	February 18	Direct	a-Se/TFT switch	85	$2812\times 3580$
41	IMS Giotto Class (5)	August 17	Direct	a-Se/TFT switch	85	$2812\times 3580$
42	Planmed Clarity 3D	May 18	Direct	a-Se/TFT switch	83	$2796\times 3584$
43	Siemens Fusion (1)	July 19	Scintillator	CsI/a-Si TFT switch	83	$2790\times 3580$
44	Siemens Fusion (2)	August 16	Scintillator	CsI/a-Si TFT switch	83	$2790\times 3580$
45	Siemens Fusion (3)	April 17	Scintillator	CsI/a-Si TFT switch	83	$2790\times 3580$
46	Siemens Mammomat Inspiration (1)	October 10	Direct	a-Se/TFT switch	85	$2800\times 3518$
47	Siemens Mammomat Inspiration (2)	December 10	Direct	a-Se/TFT switch	85	$2800\times 3518$
48	Siemens Mammomat Inspiration (3)	January 12	Direct	a-Se/TFT switch	85	$2800\times 3518$
49	Siemens Mammomat Inspiration (4)	April 11	Direct	a-Se/TFT switch	85	$2800\times 3518$
50	Siemens Mammomat Inspiration (5)	January 09	Direct	a-Se/TFT switch	85	$2800\times 3518$
51	Siemens Revelation (1)	June 19	Direct	a-Se/TFT switch	85	$2800\times 3518$
52	Siemens Revelation (2)	March 19	Direct	a-Se/TFT switch	85	$2800\times 3518$
53	Siemens Revelation (3)	November 18	Direct	a-Se/TFT switch	85	$2800\times 3518$
54	Siemens Revelation (4)	April 19	Direct	a-Se/TFT switch	85	$2800\times 3518$
55	Siemens Revelation (5)	August 18	Direct	a-Se/TFT switch	85	$2800\times 3518$

Table 2.

Image acquisition settings for the MTF measurements.

No	Mammography system	No. of tests	No. of detector(s)	Age of detector at test (months)	Acquisition factors of the presampling MTF
					A/F	kVp	mAs	DAK ($\mu \mathrm{Gy}$)
1	Fuji Amulet (1)	18	1	$>108$	W/Rh ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$61.1\pm 5.6$	$240\pm 28$
2	Fuji Amulet (2)	18	2	$\mathrm{Det.}1:\le 84$; $\mathrm{Det.}2:>24$	W/Rh ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$61.3\pm 5.5$	$238\pm 27$
3	Fuji Amulet (3)	13	2	$\mathrm{Det.}1:\le 48$; $\mathrm{Det.}2:>30$	W/Rh ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$63.7\pm 2.3$	$259\pm 15$
4	Fuji Amulet f	6	1	$>36$	W/Rh ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$63.3\pm 0.8$	$267\pm 38$
5	Fuji Amulet s (1)	9	2	$\mathrm{Det.}1:\le 30$; $\mathrm{Det.}2:>24$	W/Rh ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$55.6\pm 10.9$	$277\pm 55$
6	Fuji Amulet s (2)	10	2	$\mathrm{Det.}1:\le 54$; $\mathrm{Det.}2:>6$	W/Rh ($50\mu \mathrm{m}$)	$28.2\pm 0.4$	$56.0\pm 6.7$	$279\pm 12$
7	Fuji Amulet s (3)	8	1	$>48$	W/Rh ($50\mu \mathrm{m}$)	$28.2\pm 0.4$	$54.4\pm 9.4$	$256\pm 35$
8	Fuji Amulet Innovality (1)	12	1	$>72$	W/Rh ($50\mu \mathrm{m}$)	$28.2\pm 0.4$	$58.1\pm 6.3$	$270\pm 26$
9	Fuji Amulet Innovality (2)	10	1	$>60$	W/Rh ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$59.3\pm 6.4$	$269\pm 21$
10	Fuji Amulet Innovality (3)	8	1	$>48$	W/Rh ($50\mu \mathrm{m}$)	$28.4\pm 0.5$	$60.4\pm 8.3$	$276\pm 45$
11	Fuji Amulet Innovality (4)	8	1	$>48$	W/Rh ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$60.4\pm 8.3$	$274\pm 46$
12	Fuji Amulet Innovality (5)	11	2	$\mathrm{Det.}1:\le 12$; $\mathrm{Det.}2:>54$	W/Rh ($50\mu \mathrm{m}$)	$28.9\pm 0.3$	$62.1\pm 5.3$	$272\pm 30$
13	GEHC Senographe DS (1)	14	2	$\mathrm{Det.}1:\le 60$; $\mathrm{Det.}2:>24$	Rh/Rh ($25\mu \mathrm{m}$)	$28.7\pm 0.5$	$46.0\pm 12.5$	$266\pm 44$
14	GEHC Senographe DS (2)	18	1	$>108$	Rh/Rh ($25\mu \mathrm{m}$)	$27.5\pm 6.7$	$40.8\pm 10.1$	$287\pm 28$
15	GEHC Senographe Essential (1)	17	1	$>102$	Rh/Rh ($25\mu \mathrm{m}$)	$29.0\pm 0.0$	$42.9\pm 12.3$	$295\pm 21$
16	GEHC Senographe Essential (2)	15	3	$\mathrm{Det.}1:\le 66$; $\mathrm{Det.}2:\le 12$; $\mathrm{Det.}3:>12$	Rh/Rh ($25\mu \mathrm{m}$)	$29.0\pm 0.0$	$44.9\pm 16.0$	$277\pm 41$
17	GEHC Senographe Essential (3)	19	1	$>114$	Rh/Rh ($25\mu \mathrm{m}$)	$29.0\pm 0.0$	$41.0\pm 15.0$	$268\pm 51$
18	GEHC Senographe Essential (4)	16	2	$\mathrm{Det.}1:\le 72$; $\mathrm{Det.}2:>24$	Rh/Rh ($25\mu \mathrm{m}$)	$29.0\pm 0.0$	$44.3\pm 15.6$	$265\pm 37$
19	GEHC Senographe Essential (5)	16	1	$>96$	Rh/Rh ($25\mu \mathrm{m}$)	$29.0\pm 0.0$	$45.9\pm 16.4$	$269\pm 38$
20	GEHC Pristina (1)	6	1	$>36$	Rh/Ag ($30\mu \mathrm{m}$)	$34.0\pm 0.0$	$63.3\pm 0.8$	$280\pm 42$
21	GEHC Pristina (2)	2	1	$>12$	Rh/Ag ($30\mu \mathrm{m}$)	$34.0\pm 0.0$	$64.0\pm 1.4$	$281\pm 19$
22	GEHC Pristina (3)	5	1	$>30$	Rh/Ag ($30\mu \mathrm{m}$)	$34.0\pm 0.0$	$63.4\pm 0.9$	$274\pm 29$
23	Hologic Lorad Selenia (1)	18	3	$\mathrm{Det.}1:\le 66$; $\mathrm{Det.}2:\le 6$; $\mathrm{Det.}3:>36$	W/Rh ($50\mu \mathrm{m}$)	$29.1\pm 1.7$	$52.9\pm 7.3$	$268\pm 28$
24	Hologic Lorad Selenia (2)	18	4	$\mathrm{Det.}1:\le 18$; $\mathrm{Det.}2:\le 60$; $\mathrm{Det.}3:\le 6;\mathrm{Det.}4:>24$	W/Rh ($50\mu \mathrm{m}$)	$29.1\pm 0.5$	$53.1\pm 5.5$	$269\pm 17$
25	Hologic Lorad Selenia (3)	18	1	$>108$	W/Rh ($50\mu \mathrm{m}$)	$28.3\pm 0.5$	$55.6\pm 6.2$	$274\pm 26$
26	Hologic Selenia Dimensions (1)	17	4	$\mathrm{Det.}1:\le 30$; $\mathrm{Det.}2:\le 6$; $\mathrm{Det.}3:\le 54$; $\mathrm{Det.}4:>12$	W/Rh ($50\mu \mathrm{m}$)	$28.6\pm 0.9$	$61.1\pm 4.2$	$248\pm 22$
27	Hologic Selenia Dimensions (2)	15	2	$\mathrm{Det.}1:\le 60$; $\mathrm{Det.}2:>30$	W/Rh ($50\mu \mathrm{m}$)	$28.4\pm 0.5$	$64.3\pm 8.8$	$270\pm 43$
28	Hologic Selenia Dimensions (3)	16	2	$\mathrm{Det.}1:\le 66$; $\mathrm{Det.}2:>30$	W/Rh ($50\mu \mathrm{m}$)	$28.4\pm 0.5$	$59.8\pm 10.8$	$263\pm 44$
29	Hologic Selenia Dimensions (4)	13	1	$>78$	W/Rh ($50\mu \mathrm{m}$)	$28.2\pm 0.4$	$59.5\pm 7.3$	$268\pm 51$
30	Hologic Selenia Dimensions (5)	14	1	$>84$	W/Rh ($50\mu \mathrm{m}$)	$28.4\pm 0.7$	$60.4\pm 5.8$	$248\pm 32$
31	Hologic 3Dimensions (1)	3	1	$>18$	W/Rh ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$63.7\pm 1.2$	$231\pm 2$
32	Hologic 3Dimensions (2)	4	1	$>24$	W/Rh ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$63.5\pm 1.0$	$225\pm 46$
33	Hologic 3Dimensions (3)	4	1	$>24$	W/Rh ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$63.5\pm 1.0$	$213\pm 17$
34	IMS Giotto Image 3D[L] (1)	4	1	$>24$	W/Rh ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$63.5\pm 1.0$	$289\pm 23$
35	IMS Giotto Image 3D[L] (2)	14	2	$\mathrm{Det.}1:\le 12$; $\mathrm{Det.}2:>72$	W/Rh ($50\mu \mathrm{m}$)	$29.1\pm 1.1$	$55.7\pm 8.8$	$269\pm 42$
36	IMS Giotto Image 3D[L] (3)	14	1	$>84$	W/Ag ($55\mu \mathrm{m}$)	$27.8\pm 0.8$	$39.8\pm 8.8$	$258\pm 44$
37	IMS Giotto Class (1)	4	1	$>24$	W/Ag ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$63.4\pm 0.9$	$274\pm 17$
38	IMS Giotto Class (2)	5	1	$>30$	W/Ag ($50\mu \mathrm{m}$)	$27.4\pm 0.5$	$63.4\pm 0.9$	$255\pm 25$
39	IMS Giotto Class (3)	5	1	$>30$	W/Ag ($50\mu \mathrm{m}$)	$27.0\pm 0.0$	$63.4\pm 0.9$	$240\pm 25$
40	IMS Giotto Class (4)	4	1	$>24$	W/Ag ($50\mu \mathrm{m}$)	$27.0\pm 0.0$	$63.5\pm 1.0$	$245\pm 40$
41	IMS Giotto Class (5)	5	1	$>30$	W/Ag ($50\mu \mathrm{m}$)	$27.0\pm 0.0$	$63.4\pm 0.9$	$271\pm 43$
42	Planmed Clarity 3D	4	1	$>24$	W/Ag ($75\mu \mathrm{m}$)	$30.0\pm 0.0$	$63.5\pm 1.0$	$214\pm 12$
43	Siemens Fusion (1)	1	1	$>6$	W/Rh ($50\mu \mathrm{m}$)	28.0	65.0	212
44	Siemens Fusion (2)	7	1	$>42$	W/Rh ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$63.3\pm 0.8$	$255\pm 52$
45	Siemens Fusion (3)	6	1	$>36$	W/Rh ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$63.3\pm 0.8$	$253\pm 35$
46	Siemens Mammomat Inspiration (1)	18	2	$\mathrm{Det.}1:\le 96$; $\mathrm{Det.}2:>6$	W/Rh ($50\mu \mathrm{m}$)	$28.4\pm 0.5$	$61.9\pm 4.3$	$246\pm 24$
47	Siemens Mammomat Inspiration (2)	19	3	$\mathrm{Det.}1:\le 96$; $\mathrm{Det.}2:\le 12$; $\mathrm{Det.}3:>6$	W/Rh ($50\mu \mathrm{m}$)	$28.4\pm 0.5$	$63.2\pm 2.5$	$253\pm 19$
48	Siemens Mammomat Inspiration (3)	16	2	$\mathrm{Det.}1:\le 60;\mathrm{Det.}2:>36$	W/Rh ($50\mu \mathrm{m}$)	$29.1\pm 0.3$	$62.6\pm 6.7$	$277\pm 41$
49	Siemens Mammomat Inspiration (4)	18	2	$\mathrm{Det.}1:\le 66$; $\mathrm{Det.}2:>42$	W/Rh ($50\mu \mathrm{m}$)	$28.4\pm 0.5$	$68.4\pm 9.4$	$286\pm 26$
50	Siemens Mammomat Inspiration (5)	18	3	$\mathrm{Det.}1:\le 24$; $\mathrm{Det.}2:\le 66$; $\mathrm{Det.}3:>18$	W/Rh ($50\mu \mathrm{m}$)	$28.4\pm 0.5$	$66.7\pm 5.9$	$277\pm 21$
51	Siemens Revelation (1)	2	1	$>12$	W/Rh ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$64.0\pm 1.4$	$244\pm 24$
52	Siemens Revelation (2)	2	1	$>12$	W/Rh ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$64.0\pm 1.4$	$205\pm 0$
53	Siemens Revelation (3)	3	1	$>18$	W/Rh ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$63.7\pm 1.2$	$220\pm 15$
54	Siemens Revelation (4)	2	1	$>12$	W/Rh ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$64.0\pm 1.4$	$213\pm 17$
55	Siemens Revelation (5)	3	1	$>18$	W/Rh ($50\mu \mathrm{m}$)	$28.0\pm 0.0$	$63.7\pm 1.2$	$237\pm 63$

2.2.2. QC protocol MTF measurement method and summary data

The MTF is routinely measured using the method described in the European Protocol,³ i.e., with a simpler version of the method used for the CR systems in study 1. For all systems except the GEHC devices, the steel edge is positioned on the x-ray bucky and the antiscatter grid is removed. For the GEHC systems, the x-ray bucky/antiscatter grid is removed, and the edge is positioned on the detector cover. Tube voltage and anode/filter are those selected clinically by the automatic exposure control system (AEC) for a 40-mm thick PMMA block, and the mAs was set to give a higher DAK than used clinically to reduce the influence of noise on the measurements. Typical values ranged between 200 and $300\mu \mathrm{Gy}$ (Table 1). The edge was positioned slightly slanted to the detector matrix ($\sim 3\mathrm{deg}$) such that one edge was centered left-right with the other at 60 mm from the chest wall. Prior to acquiring the edge image, the detector response function was measured,^3,19 using the same tube voltage and A/F setting as used for the edge image acquisition. Two MTF curves were calculated from a single image acquisition, one for the left-right ($x$ direction) and one for the front-back ($y$ direction) orientations; only the MTF averaged for the two directions is presented in this work to reduce the number of tables and graphs (individual data are available upon request). Analysis showed that the conclusions remained unchanged, whether the average MTF or the individual MTF data were analyzed.

This study focused on the $f_{\mathrm{MTF}0.5}$ because this metric is stipulated in the EUREF protocol. Additional analysis was made using ${\mathrm{MTF}}_{f5}$ because this may have more relevance for the visibility of small microcalcifications. Both the $f_{\mathrm{MTF}0.5}$ and ${\mathrm{MTF}}_{f5}$ metrics were obtained via linear interpolation from the MTF curve. The baseline values were defined as the $f_{\mathrm{MTF}0.5}$ and ${\mathrm{MTF}}_{f5}$ results at the start of the QC period. Following a change in detector, the baseline values were changed to the acceptance data for this detector.

Then for each device, the mean values of $f_{\mathrm{MTF}0.5}$ and ${\mathrm{MTF}}_{f5}$ were calculated, taking all tests of the device into account, and for each type of device, the averages of these means were calculated along with the confidence intervals and the COV. These results are proposed as “typical values.” Also included was $\mathrm{\Delta }{\mathrm{MTF}}_{f5}$, as this parameter is sometimes reported in the literature; this parameter is not specified in the European protocol, and there is currently no accepted limit for the variation in MTF value at $5{\mathrm{mm}}^{-1}$.

2.2.3. Analysis of longitudinal MTF data

Two calculations were made using the MTF data. First, the change from baseline was calculated as the relative change in the MTF parameter (for example $f_{\mathrm{MTF}0.5}$) between the last and first test for a given detector. If a detector has been changed on a given system, the relative change between the most recent data result for this new detector was compared with the first test for the new detector:

$${\mathrm{\Delta }}_{\text{baseline}}=\frac{x_{\text{last test}}-x_{\text{baseline}}}{x_{\text{baseline}}}.$$ (1)

Second, the reproducibility of the MTF measurements was assessed by calculating the absolute percentage change between successive half-yearly tests (denoted $i$ and $i-1$):

$${\mathrm{\Delta }}_{i-\text{previous}}=\frac{|x_i-x_{i-1}|}{x_{i-1}}.$$ (2)

The absolute value was used for the successive data for the following reason. For a stable detector, there is random error on the MTF that could be positive or negative. Without using the absolute value, the fluctuation in, e.g., $f_{\mathrm{MTF}0.5}$ would probably average to ∼zero over the visits (unless there is a bias in the MTF method) when in fact there will be some fluctuation between measurements. Using the absolute relative value gives some estimate of this fluctuation. Once calculated for all 55 systems, the parameters in Eqs. (1) and (2) were averaged over the number of systems of a given model to give a “typical” value for that model, e.g., Fuji Amulet Innovality. The $f_{\mathrm{MTF}0.5}$ and ${\mathrm{MTF}}_{f5}$ were also plotted as a function of time to visually check for trends and to indicate outliers.

3. Results

3.1. Study 1: Relating Changes in MTF to Technical Image Quality Measures

3.1.1. Response function, MTF, NNPS, and DQE for the CR detectors

The detector response curves for CR1 and CR2 are shown in Fig. 4(a). The two CR plates have almost identical response curves with a power relationship and correlation coefficients ($R^2$) of 1, indicating that their sensitivity is similar and the chosen curve model is in good agreement with the measurements. For the MTF measurements, the mean angle of the edge was $1.8\mathrm{deg}\pm 0.2\mathrm{deg}$. The MTF curves are shown in Fig. 4(b). The ${\mathrm{MTF}}_{f5}$ values are 0.23 and 0.18 for CR1 and CR2, representing a significant 24% reduction. The $f_{\mathrm{MTF}0.5}$ is $2.58\text{cycles}{\mathrm{mm}}^{-1}$ for CR1 and $2.06\text{cycles}{\mathrm{mm}}^{-1}$ for CR2, i.e., a 20% reduction for CR2. The MTF method reproducibility was assessed by acquiring five edge images consecutively, from which the MTF was calculated. The COV of the MTF values averaged up to $5\text{cycles}{\mathrm{mm}}^{-1}$ was 1.22% for CR1 and 0.63% for CR2, making the measured differences in MTF between the two plates significant.

Fig. 4.

(a) Detector response curve; (b) MTF; (c) NNPS; and (d) DQE at $K_{\mathrm{ref}}$, $0.5\times K_{\mathrm{ref}}$, and $2\times K_{\mathrm{ref}}$ for 2 CR detectors (CR1 and CR2).

The NNPS curves measured at $K_{\mathrm{ref}}$ ($\sim 117\mu \mathrm{Gy}$), $0.5\times K_{\mathrm{ref}}$, and $2\times K_{\mathrm{ref}}$ are shown in Fig. 4(c). The changes in MTF, NNPS, and DQE between the two plates are given in Table 3. At $5\text{cycles}{\mathrm{mm}}^{-1}$, the NNPS of the CR2 plate is lower by a factor of 1.7 as DAK doubles ($\sim 210\mu \mathrm{Gy}$) and increases by a factor 1.6 as DAK is halved ($\sim 73\mu \mathrm{Gy}$). At $5\text{cycles}{\mathrm{mm}}^{-1}$, the differences in NNPS between the two CR plates were 15%, 12%, and 10% from low to high DAK, respectively. The NNPS was compared between the two plates at $0.5\times K_{\mathrm{ref}}$, $K_{\mathrm{ref}}$, and at $2\times K_{\mathrm{ref}}$. This was done at spatial frequencies of 0.5 and $2{\mathrm{mm}}^{-1}$. At $0.5{\mathrm{mm}}^{-1}$, differences at low, medium, and high DAK were 5%, 4%, and 4%, respectively, whereas at $2{\mathrm{mm}}^{-1}$, the differences were 18%, 20%, and 17%. The DQE curves calculated at $K_{\mathrm{ref}}$, $0.5\times K_{\mathrm{ref}}$, and $2\times K_{\mathrm{ref}}$ are shown in Fig. 4(d). At $K_{\mathrm{ref}}$, DQE averaged between 0.5 and $1{\mathrm{mm}}^{-1}$ was 51% and 48% for CR1 and CR2, respectively, whereas at $5\text{cycles}{\mathrm{mm}}^{-1}$, DQE differences between the two CR plates were 32%, 34%, and 36% from low to high DAK, respectively. These data show that the reduction in MTF reduces NPS between 1 and $6{\mathrm{mm}}^{-1}$ for detector CR2, due to an increase in blurring in the x-ray detector. However, when DQE is evaluated, the $\mathrm{MT}{\mathrm{F}}^2$ factor in the numerator of the equation for DQE dominates, and the blurring leads to a loss of signal to noise ratio (SNR) transfer above $1{\mathrm{mm}}^{-1}$.

Table 3.

The change in MTF, NNPS, and DQE evaluated at spatial frequencies of $0.5{\mathrm{mm}}^{-1}$, 2, and $5{\mathrm{mm}}^{-1}$.

Frequency (cycles/mm)	$\mathrm{\Delta }\mathrm{MTF}$ (%)	ΔNNPS			$\mathrm{\Delta }\mathrm{DQE}$
		$0.5{\mathrm{K}}_{\mathrm{ref}}$ (%)	${\mathrm{K}}_{\mathrm{ref}}$ (%)	$2{\mathrm{K}}_{\mathrm{ref}}$ (%)	$0.5{\mathrm{K}}_{\mathrm{ref}}$ (%)	${\mathrm{K}}_{\mathrm{ref}}$ (%)	$2{\mathrm{K}}_{\mathrm{ref}}$ (%)
0.5	$-3$	$-5$	$-4$	$-4$	0	0	0
2.0	$-15$	$-18$a	$-20$a	$-17$a	$-12$	$-10$	$-13$
5.0	$-24$	$-15$	$-12$	$-10$	$-32$	$-34$	$-36$

^aThis value exceeds the limiting value of the EUREF Guidelines.

3.1.2. Comparison with technical image quality tests

The graphs of threshold gold thickness (T) of the CDMAM as a function of gold diameter for the two CR plates are shown in Fig. 5(a). At diameters of 0.1 and 0.25 mm, the gold thickness threshold values together with their CI95 are $1.16[1.07-1.24]\mu \mathrm{m}$ and $0.28[0.26-0.29]\mu \mathrm{m}$ for CR1 and $1.22[1.13-1.30]\mu \mathrm{m}$ and $0.29[0.27-0.31]\mu \mathrm{m}$ for CR2, respectively. The Mann–Whitney tests resulted in $p=0.1366$ for the 0.1-mm disk and $p=0.4507$ for the 0.25-mm disk. Threshold gold thickness was lower for the 0.1-mm disk for CR2 compared with CR1, but this is not significant.

Fig. 5.

(a) Gold thickness threshold (T) of the CDMAM and (b) the psychometric curves of PC as a function of object diameter of microcalcifications.

The psychometric curves of PC as a function of target diameter of the L1 phantom are presented in Fig. 5(b). The diameter threshold values for microcalcifications with their CI95 for CR1 and CR2 are 0.118 [0.108–0.128] mm and 0.125 [0.117–0.132] mm, respectively. A higher PC was seen for the smallest calcifications for CR1 versus CR2, but the confidence intervals were overlapping. The Mann–Whitney tests resulted in $p=0.5959$ for the 0.12-mm microcalcifications and $p=0.6072$ for the 0.15-mm diameter microcalcifications, clearly showing that the differences in $d_{\mathrm{tr}}$ between CR1 and CR2 are not significant.

3.2. Study 2: Reproducibility of the MTF in Digital Mammography QC

3.2.1. Average MTF curves for DM models from different vendors

Figure 6 shows typical presampling MTF curves for 16 different DM models, from six vendors. The curves were averaged from between 3 and 19 consecutive tests, depending on the data available for the DM model. The error bars show the standard deviations on the MTF values. Table 4 shows $f_{\mathrm{MTF}0.5}$ and ${\mathrm{MTF}}_{f5}$ data along with confidence intervals calculated from averaging over all values of all devices of the same model. The COV shows the variation between devices of the same model. These average values may prove useful to physicists in centers with access to a limited number of devices from a particular vendor, i.e., for bench marking. The data of the individual systems are shown in Appendix 1.

Fig. 6.

Average presampling MTF curves for 16 different DM models. The MTF curves were averaged from between 3 and 19 consecutive QC test results, depending on the data available for a given model. Error bars (shaded areas) show $\pm 1$ standard deviation for a given curve.

Table 4.

Average values of $f_{\mathrm{MTF}0.5}$ (cycles/mm) and ${\mathrm{MTF}}_{f5}$ (U) together with 95% confidence intervals and COV.

No	Device	$f_{\mathrm{MTF}0.5}$ [cycles/mm]		${\mathrm{MTF}}_{f5}$ [U]
		Mean [CI95]	COV	Mean [CI95]	COV
1	Fuji Amulet	3.52 [3.30–3.75]	0.06	0.38 [0.37–0.40]	0.03
2	Fuji Amulet f	6.44		0.63
3	Fuji Amulet s	7.06 [5.90–8.23]	0.15	0.66 [0.59–0.72]	0.09
4	Fuji Amulet Innovality	6.09 [6.02–6.16]	0.01	0.64 [0.63–0.66]	0.02
5	GEHC Senographe DS	3.53 [3.42–3.64]	0.02	0.32 [0.31–0.32]	0.02
6	GEHC Senographe Essential	2.51 [2.32–2.69]	0.09	0.17 [0.14–0.20]	0.18
7	GEHC Pristina	2.87 [2.86–2.89]	0.00	0.22 [0.22–0.23]	0.02
8	Hologic Lorad Selenia	5.28 [5.00–5.56]	0.05	0.52 [0.50–0.54]	0.03
9	Hologic Selenia Dimensions	5.63 [5.20–6.05]	0.09	0.55 [0.52–0.58]	0.07
10	Hologic 3Dimensions	6.00 [5.90–6.09]	0.01	0.58 [0.57–0.59]	0.02
11	IMS Giotto Image 3D[L]	4.09 [3.98–4.20]	0.02	0.41 [0.39–0.43]	0.03
12	IMS Giotto Class	5.24 [5.14–5.33]	0.02	0.52 [0.51–0.53]	0.02
13	Planmed Clarity 3D	4.17		0.41
14	Siemens Fusion	4.10 [3.89–4.31]	0.05	0.40 [0.37–0.43]	0.06
15	Siemens Mammomat Inspiration	5.20 [5.11–5.30]	0.02	0.52 [0.51–0.53]	0.02
16	Siemens Revelation	5.04 [4.85–5.22]	0.04	0.50 [0.49–0.52]	0.03

There are considerable differences between the systems, even between the types of systems of one vendor, probably reflecting developments in detector technology. Fuji shows the largest evolution in their $f_{\mathrm{MTF}0.5}$ values, from $3.51\text{cycles}{\mathrm{mm}}^{-1}$ for the first generation Amulet up to $7.02\text{cycles}{\mathrm{mm}}^{-1}$ for Amulet S and $6.09\text{cycles}{\mathrm{mm}}^{-1}$ for the Innovality system. The variability for the Fuji Amulet data is larger than for the Fuji Innovality systems. There has also been some evolution in the CsI-based detectors used by GEHC. MTF was highest for older generation Senographe DS with an $f_{\mathrm{MTF}0.5}$ of $3.51\text{cycles}{\mathrm{mm}}^{-1}$; this was reduced to $2.52\text{cycles}{\mathrm{mm}}^{-1}$ for the Senographe Essential and is now at $2.87\text{cycles}{\mathrm{mm}}^{-1}$ for the Pristina. The COV of all of the devices is similar to that found for the short-term reproducibility test. This is an important indication of the robustness of MTF measurements in routine QC practice and confirms earlier work describing excellent reproducibility of MTF measurements, when applied in a QC program.⁹ The underlying detector technology plays a large role in determining the absolute sharpness of DM detectors,^1,20 and this is reflected in the data. The $f_{\mathrm{MTF}0.5}$ and ${\mathrm{MTF}}_{f5}$ values for the a-Se based detectors are significantly higher than those for the CsI-based units. Similar values are seen for the IMS Giotto Class and Siemens Inspiration and Revelation systems, and all of these units use a detector supplied by the same original equipment manufacturer.

3.2.2. Longitudinal MTF data and reproducibility

Table 5 summarizes the change in $f_{\mathrm{MTF}0.5}$ and ${\mathrm{MTF}}_{f5}$ for the 55 systems, averaged for a given DM model. For example, the data for Hologic Selenia Dimensions systems are an average of data for these five systems. The largest change in $f_{\mathrm{MTF}0.5}$ compared with baseline is $-15.0\%\pm 5.2\%$ for the GEHC Senographe Essential, while values of $-10\%$ are also seen for the Fuji Amulet F and the GEHC Senogaphe DS. Many systems have a relative change of $\sim 0\%$, while a value of $+7.7\%\pm 4.9\%$ is seen for the Giotto Image 3D. These changes are echoed in the ${\mathrm{MTF}}_{f5}$ data, but the changes seen are larger. The absolute relative changes in $f_{\mathrm{MTF}0.5}$ and ${\mathrm{MTF}}_{f5}$ are rather consistent over all of the systems. For $f_{\mathrm{MTF}0.5}$, the values range from $1.2\%\pm 1.2\%$ for the Fuji Amulet Innovality to $5.1\%\pm 2.2\%$ for the Fuji Amulet F. This suggests good reproducbilty between six monthly tests for the method. Averaging over all systems gives a value of 3% for absolute relative change in ${\mathrm{f}}_{\mathrm{MTF}0.5}$.

Table 5.

Averaged values ($\text{mean}\pm \mathrm{SD}$) of the relative change from baseline (${\mathrm{\Delta }}_{\text{baseline}}[\%]$) between last and first tests for ${\mathrm{f}}_{\mathrm{MTF}0.5}$ and ${\mathrm{MTF}}_{f5}$, averaged for a given system model. Averaged values ($\text{mean}\pm \mathrm{SD}$) of the absolute relative change in ${\mathrm{f}}_{\mathrm{MTF}0.5}$ and ${\mathrm{MTF}}_{f5}$ between successive tests (${\mathrm{\Delta }}_{\text{successive}}[\%]$), averaged for a given system model.

No	Device	$f_{\mathrm{MTF}0.5}$		${\mathrm{MTF}}_{f5}$
		${\mathrm{\Delta }}_{\text{baseline}}$ (%)	${\mathrm{\Delta }}_{\text{succesive}}$ (%)	${\mathrm{\Delta }}_{\text{baseline}}$ (%)	${\mathrm{\Delta }}_{\text{succesive}}$ (%)
1	Fuji Amulet	$-3.6\pm 4.5$	$2.7\pm 2.1$	$-1.0\pm 2.3$	$2.5\pm 2.1$
2	Fuji Amulet f	$-10.0$	$5.1\pm 2.2$	$-4.9$	$2.2\pm 1.7$
3	Fuji Amulet s	$-2.1\pm 5.7$	$3.3\pm 2.5$	$-1.5\pm 3.0$	$2.1\pm 1.8$
4	Fuji Amulet Innovality	$-0.7\pm 1.3$	$1.2\pm 1.2$	$-0.7\pm 2.3$	$1.9\pm 1.9$
5	GEHC Senographe DS	$-10.4\pm 9.6$	$3.1\pm 3.4$	$-14.7\pm 15.1$	$5.4\pm 4.9$
6	GEHC Senographe Essential	$-15.0\pm 5.2$	$2.6\pm 2.6$	$-25.9\pm 9.2$	$6.7\pm 7.1$
7	GEHC Pristina	$-8.7\pm 4.2$	$2.7\pm 1.7$	$-12.9\pm 4.7$	$5.6\pm 3.2$
8	Hologic Lorad Selenia	$0.7\pm 3.2$	$2.1\pm 1.7$	$0.1\pm 1.3$	$1.6\pm 1.2$
9	Hologic Selenia Dimensions	$0.3\pm 4.8$	$3.4\pm 3.0$	$-0.2\pm 3.9$	$2.9\pm 2.7$
10	Hologic 3Dimensions	$-3.2\pm 6.1$	$2.3\pm 1.9$	$-3.3\pm 2.6$	$2.0\pm 1.5$
11	IMS Giotto Image 3D[L]	$7.7\pm 4.9$	$3.7\pm 3.2$	$7.9\pm 5.4$	$4.3\pm 3.0$
12	IMS Giotto Class	$0.0\pm 6.6$	$4.7\pm 4.1$	$-0.2\pm 5.5$	$4.5\pm 3.9$
13	Planmed Clarity 3D	$-4.0$	$2.9\pm 1.7$	$-6.0$	$2.1\pm 1.8$
14	Siemens Fusion	$1.7\pm 0.3$	$1.5\pm 1.2$	$1.2\pm 1.4$	$2.0\pm 1.3$
15	Siemens Mammomat Inspiration	$-0.1\pm 7.4$	$3.3\pm 2.8$	$1.4\pm 3.6$	$3.3\pm 2.7$
16	Siemens Revelation	$-1.1\pm 1.3$	$3.7\pm 3.1$	$-0.3\pm 1.2$	$2.7\pm 2.0$

The measured $f_{\mathrm{MTF}0.5}$ and ${\mathrm{MTF}}_{f5}$ values for all systems are shown in Fig. 7 as a function of time (i.e., QC test made every six months). The trends described in the baseline change analysis of ${\mathrm{f}}_{\mathrm{MTF}0.5}$ and ${\mathrm{MTF}}_{f5}$ are visible. For the GEHC Senographe Essential systems, there is a small, gradual drop in both $f_{\mathrm{MTF}0.5}$ and ${\mathrm{MTF}}_{f5}$ over time. Whether this trend exists for the GEHC Pristina is not known as there are currently insufficient data points. This gradual decrease was not observed in other systems.

Fig. 7.

(a) $f_{\mathrm{MTF}0.5}$ and (b) ${\mathrm{MTF}}_{f5}$ as a function of time from acceptance test (55 DM systems). Points with a circle indicate data for a replacement detector (same system, new detector installed).

4. Discussion

The aim of this study was to investigate the relevance of MTF measurements in medical physics QC testing. First, a systematic change in MTF for CR detectors was studied to see whether a significant change in the MTF leads to a measurable change in a technical image quality parameter. The effect of an MTF change that was outside the limiting value of 10% set for the MTF between successive tests³ was examined using CR detectors. While integrated digital detectors have largely superseded CR technology, the two CR plates with different MTF curves but similar sensitivity were ideally suited for this study. To perform the same study using an integrated detector would mean waiting for a fault condition to arise that influenced the MTF. Instead, the CR plates could be used with the same x-ray tube, geometry and anti-scatter grid, allowing the influence of the x-ray detector MTF to be isolated.

Compared with CR1, there was a 20% reduction in ${\mathrm{f}}_{\mathrm{MTF}0.5}$ for CR2. While there was a consistent trend to lower detectability for CR2, none of the changes were significant. The practical consequence is that tracking the ${\mathrm{MTF}}_{f0.5}$ value of the MTF provides a valuable means of detecting a loss in sharpness when image quality metrics with the CDMAM and the L1 phantom did not yet show a significant change/reduction. This gives the MTF measurement the status of a warning test: tracking detector MTF over time can point to changes prior to any significant change in a technical quality parameter.

The curves for PC in Fig. 5(b) cross, showing higher PC for CR1 for the smaller details (e.g., 0.12-mm diameter) and higher PC for the $\sim 0.15\text{-}\mathrm{mm}$ diameter microcalcification for CR2. One might conclude that the improved MTF and SNR transfer for spatial frequencies above $\sim 2{\mathrm{mm}}^{-1}$ [see the DQE graph in Fig. 4(d)] leads to higher PC at small detail diameter for CR1. The lower NNPS values at higher spatial frequencies for CR2 might mean reduced image noise at these frequencies, and for larger details this could lead to improved detection (PC). We cannot be definitive about this given the large error bars on the L1 data.

To make technical image quality test objects more sensitive to changes in sharpness, one could include more finely detailed features, although one could argue that the $100\mu \mathrm{m}$ details in CDMAM are sufficiently fine while remaining clinically relevant. One could indeed try to improve the phantoms or the evaluation methods by improving sensitivity or reducing the variance on the measurements, however this generally involves the acquisition of more images. Furthermore, after a certain point, acquiring additional images only leads to a small reduction in the variance of the measured parameter, and it becomes impractical for routine QA work.²¹ MTF measurements require only 12 images: 10 images for a response curve—which is used for a number of other tests – plus 2 edge images from which the MTF is calculated. The MTF acquisitions did reveal the sharpness problem. To expressly link changes in sharpness with clinical imaging performance, (virtual) clinical trials may be required.^10,22 This was out of the scope of this work and is certainly not possible during routine QA. In the future, procedures that assess the impact of sharpness or other detector characteristics²³ may be part of the tests at the introduction of a new detector or new tube technology.

Turning to the reproducibility data, these results have implications for QC protocols. Measurement of MTF is often specified as optional in QC protocols; the reasons cited for this include the difficulty in accessing the required image types (for processing or equivalent), the lack of experience in terms of calculation or interpretation of the results, and the lack of evidence that MTF measurements are a useful addition to a QC protocol. The latter was the direct trigger of the current study. The second study shows that MTF measurements—and by implication, the x-ray detectors themselves—have excellent long-term reproducibility, with a typical COV of 4% (see Table 4). This supports early work on the reproducibility of quantitative measurements (including the MTF) and their implementation in a QC program, made for just seven DM systems.⁹ Similar work has also reported excellent reproducibility for MTF measurements as part of QC for general radiography detectors.^24,25

Systematic differences in MTF are expected between systems that employ different technologies. The detector presampling MTF is a product of the MTF of the different stages, and for DM detectors this is the MTF of the x-ray sensitive (conversion) layer and the MTF of the signal collecting aperture (typically the pixel).^1,21 Photoconductor based detectors (e.g., with a-Se) have very little presampling blurring, and therefore higher MTF values are seen for these detector types compared with the CsI detectors (Table 2), results that are consistent with previous work.^21,26 The lower MTF values seen for indirect conversion detectors are due to blurring that results from light spread in the CsI converter.²⁷ The QC MTF measurements are clearly sufficiently sensitive and stable to capture these differences between technologies/models. It is worth noting that while MTF values are typically lower for CsI-based detectors, MTF (or sharpness) by itself, while an important determinant in task performance, is not a sufficient means of ultimately specifying system imaging performance. Technical image quality, as quantified by contrast detail measurements, easily meets the levels specified in the European Protocol for breast screening for the CsI-based systems.²³

We expect some variations in the MTF curves of different systems (i.e., device by device variation) of the same brand, possibly due to manufacturing variations/tolerances and local operating conditions. The data in this study show that typical values can be produced and that some models have MTF curves that vary more than for other models. As an example, large differences were seen between different Fuji Amulet models, while much lower variability was seen for the Fuji Amulet Innovality MTF data. At the introduction of any new system, a careful assessment of the detector characteristics by the QC physicist, including MTF, provides valuable information on image sharpness. These data can be compared against the results in Table 4 for comparative benchmarking.

The 10% limiting value in the EUREF protocol applies to changes in ${\mathrm{f}}_{\mathrm{MTF}0.5}$ between successive QC tests. This can be compared with the averaged absolute change in ${\mathrm{f}}_{\mathrm{MTF}0.5}$ between tests in Table 5, which ranges between 1.2% and 5.1%, with an average for all systems of 3%. This implies that changes in MTF will generally not exceed the 10% limit. The limiting value in the EUREF protocol is designed to find sudden changes in MTF that occur between visits. The value of using a baseline value to catch gradual changes in a parameter is clearly seen in Fig. 7. Limiting values should ensure appropriate quality when met and/or alert the QC physicist of changes that are potentially detrimental to clinical image quality, if limits are exceeded. The good reproducibility demonstrated by these MTF data suggests that finding a change is straightforward; the difficulty comes in deciding at which point the change starts to potentially influence clinical performance. We also note that action levels involving a change from baseline to find changes in some parameters are in line with typical routine QC testing protocols,²⁸ while the philosophy of the EUREF protocol leans more toward tests that aim to guarantee a minimum level of performance.

The longitudinal analysis of the $f_{\mathrm{MTF}50}$ and ${\mathrm{MTF}}_{f5}$ as a function of time revealed that the MTF was changing gradually for a number of the GEHC Essential systems. This was further explored in the light of the first study for the CR detectors as follows: the threshold gold thickness ($T$) for the 0.1-mm diameter disk, normalized for MGD, for all DM systems was examined. To normalize for the influence of dose, $T$ values were multiplied by the square root of the mean glandular dose calculated for the 50-mm PMMA block.^13,14 Threshold gold thickness normalized to 1 mGy is shown in Fig. 8 and the data is summarized in Table 6 for the five Essential systems in this study. Regarding the other systems, no trend or systematic change in normalized threshold gold thickness was found for systems in which there was no change in MTF over time (data available on request).

Fig. 8.

Threshold thicknesses T normalized to a mean glandular dose of 1 mGy and for the 0.1-mm gold disk as a function of time for GEHC Senographe Essential systems.

Table 6.

Threshold gold thickness T normalized to 1 mGy, with confidence interval, and difference from baseline and between successive tests for the 0.1-mm gold disk. Threshold thicknesses are normalized to a mean glandular dose of 1 mGy. For each system, these data are for a single detector, i.e., from the first test with a given detector to the last test with that detector.

No	Devices	Age of detector at test (months)	Detector ID	${\mathrm{T}}_{\text{norm}}$ baseline	${\mathrm{T}}_{\text{norm}}$ latest test	$\mathrm{\Delta }{\mathrm{T}}_{\text{norm}}$ baseline
				Mean [CI95]	Mean [CI95]	Mean (%)
1	GEHC Senographe Essential 1	$>102$	NPL0088_06	1.33 [1.22–1.42]	1.30 [1.17–1.39]	$-1.85$
2	GEHC Senographe Essential 2	$\le 66$	NPL0226_02	1.25 [1.16–1.33]	1.35 [1.25–1.45]	8.83
3	GEHC Senographe Essential 3	$>114$	NPL0017_05	1.33 [1.25–1.40]	1.51 [1.44–1.57]	13.47
4	GEHC Senographe Essential 4	$\le 72$	NPL0187_08	1.15 [1.06–1.20]	1.30 [1.20–1.33]	13.09
5	GEHC Senographe Essential 5	$>96$	NVL0035_03	1.25 [1.15–1.32]	1.48 [1.35–1.55]	18.27

For four of the five systems, there is a change in $f_{\mathrm{MTF}0.5}$ from baseline that is $>10\%$, but this is not consistently associated with a significant change in normalized threshold gold thickness. There are two systems with a significant change (non-overlapping confidence intervals) in threshold gold thickness: No 3 and No 5. The change in MTF may be related to the fact that CsI is hygroscopic,²⁹ a phenomenon in which water is absorbed within the CsI layer, leading to a change in the physical properties of the phosphor. The x-ray CsI layer is therefore sealed to resist the infiltration of water, but there is some ingress over time. It appears that this leads to a change in MTF over time for some detectors. The water ingress and subsequent blurring in the CsI layer is probably different from the process that changes the sharpness in the CsBr needle detectors, and this in turn leads to different c-d performance. We are currently investigating these differences with the manufacturer. This behavior, in which 2 of the five CsI-based systems have a significant change in $T$, is somewhat different from that seen for the needle CR detectors, in which a 20% reduction in ${\mathrm{f}}_{\mathrm{MTF}0.5}$ was associated with an increase in threshold gold thickness, but the change was not significant. We note that the detectors in the Essential units are robust and have been in service for a long time; one consequence of this is that baseline changes in $f_{\mathrm{MTF}0.5}$ can develop and be detected by MTF measurements. One could envisage normalizing the reduction in $f_{\mathrm{MTF}0.5}$ to some unit of time; this would lessen the influence of the age of the systems in cohort on the maximum baseline change. It must be stressed that, when evaluated at the clinical working point (i.e., at the clinical dose used for a 50-mm thick PMMA block), the unnormalized threshold gold thickness for the Essential units remained well within the image quality levels specified in the European Protocol.^3,13

The data in Table 4 and Table 5 show that the MTF measurement method, as currently implemented in the Medical Physics QC service, yields results that are reproducible to a degree that a remedial action level of 10% between successive tests could be implemented – the question is whether this should be done. The action level should be chosen to avoid false alarms while still catching a meaningful reduction in system image quality. One must also bear in mind what kind of remedial action is available; a change of detector would clearly not be justified if there was a reduction in ${\mathrm{f}}_{\mathrm{MTF}0.5}$ of for example 13%. Judged solely on the CR data, then the action level should be somewhat $>20\%$, as even a 20% in $f_{\mathrm{MTF}0.5}$ did not cause a change in the technical (task-based) IQ metric. However, the GEHC Essential data are somewhat more mixed. Here, a significant increase in T was found for changes in $f_{\mathrm{MTF}0.5}$ of 6% and 20%. For the three other Essential systems, in which ${\mathrm{f}}_{\mathrm{MTF}0.5}$ reduced by $\sim 15\%$, there was an increase in T, but this was not significant. It must be noted that threshold gold thickness is a multifactorial quantity, and thus factors other the MTF may be at play in these data, in contrast to the well-controlled CR data presented in the first study. Overall, the measurement of MTF offers a reproducible, additional check on the technical image quality results and highlights explicitly changes occurring within the x-ray detector.

The 10% level in the European Protocol is not a suspension level and should instead trigger an investigation of clinical image quality and a check that the system still meets the image quality criterion as evaluated by a contrast-detail test object. Additional checks could also be made on the NNPS, as changes in detector MTF directly influence image noise level.³⁰ Given these results, one might suggest a remedial level of 25% for a change in the $f_{\mathrm{MTF}0.5}$ to trigger further investigation into system image quality. The system should be suspended from use only if the system cannot meet the Acceptable level for technical image quality within the Acceptable dose. Unless there is catastrophic blurring occurring within the detector, an MTF measurement alone cannot be used to make this decision.

5. Conclusion

In the first part of this study, the impact of a 20% change in the value of $f_{\mathrm{MTF}0.5}$ of the MTF curve for a needle-based CR system was evaluated on three different technical image quality test objects, including the threshold gold thickness of a 0.1-mm diameter detail. Although there was a trend to somewhat poorer performance for the CR plate with the lower MTF, neither of the image quality metrics showed a significant change. This study then examined the long-term reproducibility of MTF measurements and produced averaged MTF curves for 16 different models in use today from six vendors. The results showed that the MTF measurement method employed yielded reproducible and sufficiently stable MTF results, with an absolute relative difference from between successive tests of 3%, averaged over all 55 systems. The analysis also revealed a maximum change baseline of $-10\%$ for the Fuji Amulet F and GEHC Senographe systems, while the change from baseline was $-15\%$ for the GEHC Essential systems. An important role of MTF measurements in routine QC therefore lies in finding changes in sharpness, regardless of detector type, before there is an impact on the detectability of lesions, as evaluated with test objects. If a change in $f_{\mathrm{MTF}0.5}$ exceeds the QC action level in place, then evaluation by a task-based image quality method such as contrast-detail must be performed; only task-based image quality metrics should be used when deciding on the clinical acceptability of a system. These results provide evidence that MTF measurements are reproducible, sensitive, and easily applied as a means of tracking x-ray detector sharpness and should be included in QC protocols.

6. Appendix

Typical values of $f_{\mathrm{MTF}0.5}$ (cycles/mm) and ${\mathrm{MTF}}_{f5}$ (U) together with 95% confidence intervals and COV taken from a series of half-yearly QC tests on 55 DM devices are given in Table 7.

Table 7.

A1. All of the MTF data ($f_{\mathrm{MTF}0.5}$ and ${\mathrm{MTF}}_{f5}$) of all systems.

No	Device	${\mathrm{f}}_{\mathrm{MTF}0.5}$ [cycles/mm]		${\mathrm{MTF}}_{\mathrm{f}5}$ [U]
		Mean [CI95]	COV	Mean [CI95]	COV
1	Fuji Amulet (1)	3.30 [3.24-3.35]	0.04	0.37 [0.36–0.37]	0.03
2	Fuji Amulet (2)	3.62 [3.56–3.68]	0.03	0.39 [0.39–0.40]	0.03
3	Fuji Amulet (3)	3.65 [3.62–3.69]	0.01	0.39 [0.38–0.39]	0.02
4	Fuji Amulet f	6.44 [6.17–6.71]	0.05	0.63 [0.61–0.64]	0.03
5	Fuji Amulet s (1)	6.26 [6.06–6.47]	0.04	0.61 [0.60–0.62]	0.02
6	Fuji Amulet s (2)	8.22 [8.11–8.34]	0.02	0.72 [0.71–0.73]	0.02
7	Fuji Amulet s (3)	6.70 [6.52–6.89]	0.04	0.63 [0.62–0.65]	0.03
8	Fuji Amulet Innovality (1)	6.14 [6.12–6.17]	0.01	0.65 [0.65–0.66]	0.01
9	Fuji Amulet Innovality (2)	6.13 [6.09–6.17]	0.01	0.65 [0.64–0.66]	0.02
10	Fuji Amulet Innovality (3)	6.12 [6.06–6.17]	0.01	0.65 [0.64–0.66]	0.02
11	Fuji Amulet Innovality (4)	5.95 [5.91–6.00]	0.01	0.62 [0.61–0.63]	0.02
12	Fuji Amulet Innovality (5)	6.09 [6.04–6.15]	0.01	0.64 [0.64–0.65]	0.02
13	GEHC Senographe DS (1)	3.59 [3.52–3.65]	0.03	0.32 [0.31–0.33]	0.03
14	GEHC Senographe DS (2)	3.48 [3.40–3.55]	0.05	0.31 [0.30–0.32]	0.07
15	GEHC Senographe Essential (1)	2.65 [2.59–2.71]	0.05	0.19 [0.18–0.20]	0.09
16	GEHC Senographe Essential (2)	2.20 [2.12–2.28]	0.06	0.13 [0.12–0.13]	0.12
17	GEHC Senographe Essential (3)	2.74 [2.71–2.77]	0.03	0.20 [0.20–0.21]	0.05
18	GEHC Senographe Essential (4)	2.55 [2.46–2.65]	0.07	0.17 [0.16–0.19]	0.13
19	GEHC Senographe Essential (5)	2.39 [2.29–2.48]	0.08	0.15 [0.14–0.16]	0.18
20	GEHC Pristina (1)	2.88 [2.77–3.00]	0.05	0.22 [0.21–0.24]	0.08
21	GEHC Pristina (2)	2.88 [2.75–3.01]	0.03	0.23 [0.21–0.24]	0.06
22	GEHC Pristina (3)	2.86 [2.77–2.96]	0.04	0.22 [0.21–0.23]	0.06
23	Hologic Lorad Selenia (1)	5.29 [5.22–5.37]	0.02	0.52 [0.52–0.53]	0.02
24	Hologic Lorad Selenia (2)	5.52 [5.47–5.57]	0.02	0.54 [0.53–0.54]	0.01
25	Hologic Lorad Selenia (3)	5.03 [4.98–5.08]	0.02	0.50 [0.50–0.51]	0.02
26	Hologic Selenia Dimensions (1)	5.88 [5.78–5.97]	0.02	0.57 [0.57–0.58]	0.02
27	Hologic Selenia Dimensions (2)	5.07 [4.94–5.19]	0.04	0.50 [0.50–0.51]	0.03
28	Hologic Selenia Dimensions (3)	5.12 [5.01–5.24]	0.04	0.51 [0.50–0.52]	0.04
29	Hologic Selenia Dimensions (4)	6.03 [5.88–6.17]	0.04	0.58 [0.57–0.59]	0.04
30	Hologic Selenia Dimensions (5)	6.03 [5.96–6.10]	0.02	0.58 [0.57–0.59]	0.02
31	Hologic 3Dimensions (1)	6.09 [5.88–6.29]	0.03	0.59 [0.58–0.60]	0.02
32	Hologic 3Dimensions (2)	5.93 [5.84–6.02]	0.02	0.57 [0.57–0.58]	0.01
33	Hologic 3Dimensions (3)	5.97 [5.75–6.19]	0.04	0.57 [0.55–0.59]	0.03
34	IMS Giotto Image 3D[L] (1)	4.04 [3.95–4.12]	0.02	0.41 [0.40–0.42]	0.03
35	IMS Giotto Image 3D[L] (2)	4.20 [4.10–4.30]	0.04	0.42 [0.41–0.43]	0.04
36	IMS Giotto Image 3D[L] (3)	4.02 [3.95–4.10]	0.03	0.40 [0.39–0.41]	0.04
37	IMS Giotto Class (1)	5.27 [4.97–5.57]	0.06	0.53 [0.50–0.56]	0.06
38	IMS Giotto Class (2)	5.10 [4.98–5.22]	0.03	0.51 [0.50–0.52]	0.03
39	IMS Giotto Class (3)	5.38 [5.05–5.71]	0.07	0.53 [0.51–0.56]	0.05
40	IMS Giotto Class (4)	5.14 [5.02–5.26]	0.02	0.52 [0.51–0.52]	0.02
41	IMS Giotto Class (5)	5.28 [5.08–5.49]	0.04	0.53 [0.51–0.55]	0.05
42	Planmed Clarity 3D	4.17 [4.06–4.29]	0.03	0.41 [0.39–0.42]	0.03
43	Siemens Fusion (1)	3.94		0.38
44	Siemens Fusion (2)	4.05 [4.02–4.08]	0.01	0.39 [0.39–0.40]	0.01
45	Siemens Fusion (3)	4.30 [4.25–4.36]	0.02	0.43 [0.42–0.43]	0.02
46	Siemens Mammomat Inspiration (1)	5.34 [5.27–5.41]	0.03	0.54 [0.53–0.54]	0.03
47	Siemens Mammomat Inspiration (2)	5.29 [5.18–5.39]	0.03	0.53 [0.52–0.53]	0.03
48	Siemens Mammomat Inspiration (3)	5.17 [5.05–5.29]	0.04	0.52 [0.51–0.53]	0.03
49	Siemens Mammomat Inspiration (4)	5.09 [4.95–5.22]	0.04	0.51 [0.50–0.52]	0.04
50	Siemens Mammomat Inspiration (5)	5.12 [5.01–5.23]	0.03	0.51 [0.50–0.53]	0.04
51	Siemens Revelation (1)	5.11 [5.08–5.14]	0.00	0.51 [0.50–0.52]	0.01
52	Siemens Revelation (2)	4.70 [4.54–4.86]	0.02	0.48 [0.48–0.48]	0.00
53	Siemens Revelation (3)	5.22 [4.96–5.48]	0.04	0.51 [0.50–0.53]	0.03
54	Siemens Revelation (4)	5.16 [5.16–5.16]	0.00	0.51 [0.51–0.52]	0.00
55	Siemens Revelation (5)	4.99 [4.87–5.10]	0.02	0.50 [0.49–0.51]	0.02

Biographies

Kristina Tri Wigati is an academic staff at Department of Physics, Faculty of Mathematics and Natural Sciences, Universitas Indonesia, and currently working as a PhD researcher at Department of Imaging and Pathology, Faculty of Medicine, KU Leuven. Her research interest is medical physics, particularly diagnostic radiology.

Biographies of the authors are not available.

Disclosures

H. Bosmans is the PI of research projects with Siemens-Healthineers, General Electric Healthcare (GEHC), and Agfa HealthCare. D. Vandenbroucke is employed by Agfa Healthcare. No conflict of interest is declared by the other authors.