Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Aug 9.
Published in final edited form as: Invest Radiol. 2022 Mar 4;57(8):517–526. doi: 10.1097/RLI.0000000000000866

New-Generation Low-Field Magnetic Resonance Imaging of Hip Arthroplasty Implants Using Slice Encoding for Metal Artifact Correction

First In Vitro Experience at 0.55 T and Comparison With 1.5 T

Iman Khodarahmi *, Inge M Brinkmann , Dana J Lin *, Mary Bruno *, Patricia M Johnson *, Florian Knoll *, Mahesh B Keerthivasan , Hersh Chandarana *, Jan Fritz *
PMCID: PMC9363001  NIHMSID: NIHMS1827366  PMID: 35239614

Abstract

Objectives:

Despite significant progress, artifact-free visualization of the bone and soft tissues around hip arthroplasty implants remains an unmet clinical need. New-generation low-field magnetic resonance imaging (MRI) systems now include slice encoding for metal artifact correction (SEMAC), which may result in smaller metallic artifacts and better image quality than standard-of-care 1.5 T MRI. This study aims to assess the feasibility of SEMAC on a new-generation 0.55 T system, optimize the pulse protocol parameters, and compare the results with those of a standard-of-care 1.5 T MRI.

Materials and Methods:

Titanium (Ti) and cobalt-chromium total hip arthroplasty implants embedded in a tissue-mimicking American Society for Testing and Materials gel phantom were evaluated using turbo spin echo, view angle tilting (VAT), and combined VAT and SEMAC (VAT + SEMAC) pulse sequences. To refine an MRI protocol at 0.55 T, the type of metal artifact reduction techniques and the effect of various pulse sequence parameters on metal artifacts were assessed through qualitative ranking of the images by 3 expert readers while taking measured spatial resolution, signal-to-noise ratios, and acquisition times into consideration. Signal-to-noise ratio efficiency and artifact size of the optimized 0.55 T protocols were compared with the 1.5 T standard and compressed-sensing SEMAC sequences.

Results:

Overall, the VAT + SEMAC sequence with at least 6 SEMAC encoding steps for Ti and 9 for cobalt-chromium implants was ranked higher than other sequences for metal reduction (P < 0.05). Additional SEMAC encoding partitions did not result in further metal artifact reductions. Permitting minimal residual artifacts, low magnetic susceptibility Ti constructs may be sufficiently imaged with optimized turbo spin echo sequences obviating the need for SEMAC. In cross-platform comparison, 0.55 T acquisitions using the optimized protocols are associated with 45% to 64% smaller artifacts than 1.5 T VAT + SEMAC and VAT + compressed-sensing/SEMAC protocols at the expense of a 17% to 28% reduction in signal-to-noise ratio efficiency. B1-related artifacts are invariably smaller at 0.55 T than 1.5 T; however, artifacts related to B0 distortion, although frequently smaller, may appear as signal pileups at 0.55 T.

Conclusions:

Our results suggest that new-generation low-field SEMAC MRI reduces metal artifacts around hip arthroplasty implants to better advantage than current 1.5 T MRI standard of care. While the appearance of B0-related artifacts changes, reduction in B1-related artifacts plays a major role in the overall benefit of 0.55 T.

Keywords: low-field, magnetic resonance imaging, hip arthroplasty, metal artifact, SEMAC


Although the higher field strength, gradient power, and receive channel elements of 1.5 and 3 T magnetic resonance imaging (MRI) systems allow for fast acquisition of high-quality images,1 the total cost of ownership can be a value-limiting factor of global availability.2 As such, there has been a recent interest in the MRI community in revisiting lower-field MRI systems between 0.1 and 1.0 T.35 New-generation low-field MRI systems utilize many performance-enhancing advancements developed with high-field systems while benefiting from substantially lower material and production costs, as well as reduced installation and maintenance complexity.

Compared with their ancestors, the new generation of low-field MRI systems allows the implementation of many new imaging techniques such as parallel imaging, simultaneous multislice excitation, and multispectral metal artifact reduction imaging.6 Although signal-to-noise ratios (SNRs) are inherently lower at low field strength, resulting in long acquisition times, diagnostic quality similar to 1.5 T can be obtained, representing a viable alternative globally.7 The US Food and Drug Administration has recently cleared the first 80-cm wide-bore 0.55 T system for human imaging, which additionally promises improving access for obese and claustrophobic patient populations.8

Low-field MRI improves visualization of the soft tissues near the metallic hardware, owing to the physics principle that susceptibility artifacts are proportional to the field strength. Although this concept has been shown for smaller metallic implants,911 the current gap of knowledge on larger metallic components, such as hip arthroplasty, stems from poor access to whole-body low-field systems, unavailability of advanced metal artifact reduction techniques on such systems, and often poor image quality of the traditional low-field scanners. Initial studies suggest that new-generation low-field MRI systems offer advantages for MRI-guided catheterizations with metal-containing devices and MRI in high-susceptibility regions.12,13 We hypothesized that the application of slice encoding for metal artifact correction (SEMAC) with a new-generation low-field MRI system results in smaller metallic artifacts and better image quality than standard-of-care 1.5 T MRI.

We aimed to investigate the feasibility of SEMAC MRI of hip arthroplasty implants on a new-generation 0.55 T system, optimize MRI protocols intended for clinical use, and compare MRI characteristics with a clinical 1.5 T system.

MATERIALS AND METHODS

Phantom Setup

Two total hip arthroplasty systems (DePuy Synthes, Warsaw, IN, USA) were tested in this prospective in vitro study to span a wide range of magnetic susceptibility values. The first “lower susceptibility” system was a ceramic-on-polyethylene construct with titanium (Ti)-based acetabular cup and femoral stem, cross-linked polyethylene liner, and a ceramic head. This combination is widely used in our institutional orthopedic practice. The second “higher susceptibility” system was a metal-on-metal construct with cobalt-chromium (CoCr)-based acetabular cup, bearing, and femoral components. The implants were embedded in a tissue-mimicking gel medium according to the American Society for Testing and Materials F2182–11A standard.14

To investigate blur effects of sequence parameters on magnetic resonance images, a resolution phantom made of 8 rows of 6 parallel plastic strips was designed similar to previous descriptions.15 Strip thicknesses and the distance between the strips from the top to the bottom rows were 2.0, 1.6, 1.0, 0.8, 0.6, 0.5, 0.4, and 0.25 mm (Supporting Information Fig. S1, http://links.lww.com/RLI/A686). The resolution phantom was placed in a coconut oil-filled container and imaged separately with each pulse sequence.

MRI System and Pulse Sequence Protocols

MRI was performed with a modified commercial MRI system (1.5 T MAGNETOM Aera; Siemens Healthcare GmbH, Erlangen, Germany) that operated as a prototype system at 0.55 T field strength with a maximum gradient strength of 25 mT/m and a maximum slew rate of 40 T/m/s. The vendor-provided 6-channel body and 18-channel spine array coils tuned to operate at 0.55 T were used for signal reception.

Intermediate-weighted turbo spin echo (TSE), view angle tilting (VAT),16 and combined VAT and SEMAC (VAT + SEMAC)1719 protocols in coronal and axial planes were used to image the hip arthroplasty constructs and the resolution phantom. Investigated parameters included the range of SEMAC encoding steps of 6 to 15, receiver bandwidths of 200 to 450 Hz/px, turbo factors of 8 to 23, and parallel imaging acceleration (generalized autocalibrating partial parallel acquisition; GRAPPA) factors of 1 to 3. Other parameters, such as matrix and voxel sizes, were matched to the clinical 1.5 T protocols and are summarized in Table 5.

TABLE 5.

Optimized Pulse Sequence Protocol for MRI of Hip Arthroplasty Implants at 0.55 T Field Strength

Plane Axial Coronal/Sagittal



Pulse protocol VAT + SEMAC (Replaceable by TSE for Ti implant) VAT + SEMAC (Replaceable by TSE for Ti implant)

Number of SEMAC steps* 6 (Ti)–9 (CoCr) 6 (Ti)–9 (CoCr)
Repetition time, ms 3300–3900 3700
Echo time, ms 27 27
Radiofrequency pulse mode Fast Fast
Echo train length 7 13
Receiver bandwidth, Hz/px 450 450
Phase oversampling 0% 50%
Refocusing flip angle, degree 180 170
Field-of-view, mm2 230 × 230 300 × 300
Matrix 256 × 183 320 × 240
Slice thickness/gap, mm 5/0 4/0
Number of slices 47–55 30
Number of excitations/concatenations 1/1 1/1
Number of averages 1 1
3 (TSE for Ti) 3 (TSE for Ti)
Acceleration technique GRAPPA 2 GRAPPA 3
Acquisition time, min:s 02:16 (TSE for Ti) 01:47 (TSE for Ti)
04:49–08:29 03:58–05:59
*

For cases of unknown implant material type, the more conservative values of CoCr implant may be selected.

MRI indicates magnetic resonance imaging; VAT, view angle tilting; SEMAC, slice encoding for metal artifact correction; TSE, turbo spin echo; Ti, titanium; CoCr, cobalt-chromium; GRAPPA, generalized autocalibrating partial parallel acquisition.

For comparison with 1.5 T field strength, the same experimental setup was used with a clinical 1.5 T MRI system (MAGNETOM Sola; Siemens Healthcare GmbH) using the standard VAT + SEMAC sequence (encoding steps of 11) and a 1.5 T MRI system (MAGNETOM Area; Siemens Healthcare GmbH) equipped with the compressed-sensing (CS) VAT SEMAC sequence (encoding steps of 19). Detailed sequence parameters are provided in Supporting Information Table S1, http://links.lww.com/RLI/A687. The vendor-provided 18-channel body and 32-channel spine arrays were used for signal reception at both 1.5 T systems.

Protocol Optimization

Four outcomes variables, including the metal artifacts, measured spatial resolution, SNR, and acquisition time, were compared across different protocols. The degree of metal artifacts, irrespective of the image blurring and SNR, was assessed qualitatively by 3 board-certified fellowship-trained full-time musculoskeletal radiologists with 10, 10, and 15 years of experience in MRI interpretations, who ranked the pseudonymized and randomized MRI datasets of different protocols independently. Tied ranking was permitted. Spatial resolution measured independently using the resolution phantom was defined as the blade thickness at which adjacent blades could be clearly distinguished. Signal-to-noise ratio across the entire imaged volume was calculated using the difference method through repeated measurements with subtraction image creations.20 The optimal MRI parameter was determined based on statistically significant differences in ranks (Fig. 1). In the absence of a significant rank difference, the optimal parameter was determined by the highest measured spatial resolution, highest SNR, shortest acquisition time, or a tradeoff between these outcome variables per reader preferences. As illustrated in Figure 1, the optimal parameters obtained at each step were used to assess the subsequent parameter.

FIGURE 1.

FIGURE 1.

Workflow used for protocol optimization.

Comparison of 0.55 and 1.5 T Field Strengths

Signal-to-noise ratio efficiency, defined as SNR divided by the square root of the acquisition time, was calculated and compared for all 0.55 and 1.5 T pulse sequence acquisitions. To calculate the SNR, matching areas in the background gel devoid of metal artifacts were selected, and the SNR was calculated using the difference method through repeated measurements and subtraction.20

To quantify the artifact size across the 0.55 and 1.5 T systems, a reader with 10 years of experience blind to the field strength manually segmented the artifact-degraded regions of the axial images in matching planes along the prosthesis using a commercially available DICOM Viewer (RadiAnt, Mexdixant, Poznan, Poland). All areas of signal void, including the implant itself, signal pile up (bright regions with intensity higher than that of the gel medium), and SEMAC-specific ripple artifacts, were considered degradations and included in measurements.

Statistical Analysis

Inter-reader agreement of rankings was quantified with the use of chance-corrected Gwet agreement coefficient 2 (κAC2) using linear weights.21 A benchmark scale was used for agreement coefficients where 0.2 or less indicated poor, 0.21 to 0.40 indicated fair, 0.41 to 0.6 indicated moderate, 0.61 to 0.8 indicated good, and 0.81 to 1.00 indicated very good agreement.22 Cumulative membership probabilities with a cutoff point of 0.95 were applied to each agreement coefficient to confirm the applicability of the benchmark scale.21 The nonparametric Friedman test was applied for multigroup comparisons among different sequence parameters, followed by post hoc analysis using a Bonferroni-corrected Wilcoxon signed-rank test. For brevity, only the P values of the pertinent parameters have been marked in the tables.

To compare the size of artifacts between 0.55 and 1.5 T, the normality of the data was assessed with a Shapiro-Wilk test. Because the residuals failed to follow a normal distribution, the nonparametric Wilcoxon signed-rank test was applied to evaluate artifact differences between the 2 field strengths. A P value of less than 0.05 was considered statistically significant.

RESULTS

Protocol Optimization

Table 1 summarizes the artifact reduction ability, measured spatial resolution, SNR, and acquisition time of different pulse sequences at 0.55 T, including TSE, VAT, and combined VAT + SEMAC with SEMAC encoding steps of 6, 9, 12, and 15. Corresponding images of the implants and the resolution phantom are shown in Figure 2. There was good agreement among the 3 readers in image ranks.

TABLE 1.

Comparison of Different Pulse Sequences at 0.55 T in Reference to Their Metal Artifact Reduction Ability, Measured Spatial Resolution, SNR, and Acquisition Time

Pulse Sequence Type
Implant Type Imaging Plane Outcome Variable TSE VAT VAT + SEMAC 6 VAT + SEMAC 9 VAT + SEMAC 12 VAT + SEMAC 15

Ti Coronal Rank mode (%) [min–max] κAC2(95 % CI) = 0.75 (0.70–0.81) 2 (61%) [1–3]* 1 (67%) [1–2] 1 (76%) [1–3] 1 (74%) [1–4] 1 (76%) [1–4] 1 (70%) [1–4]
Measured spatial resolution, mm 1.0 1.6 1.6 1.6 1.6 1.6
SNR 11.5 11.5 24.5 30.1 34.6 38.4
Acquisition time, min:s 00:37 00:37 04:00 06:02 08:04 10:06
Axial Rank mode (%) [min–max] κAC2(95 % CI) = 0.79 (0.77–0.81) 1 (50%) [1–3] 2 (50%) [1–3] 1 (84%) [1–3] 1 (88%) [1–4] 1 (84%) [1–4] 1 (84%) [1–4]
Measured spatial resolution, mm 1.0 >2.0 >2.0 >2.0 >2.0 >2.0
SNR 09.2 08.9 18.7 22.9 26.4 29.5
Acquisition time, min:s 00:47 00:47 04:57 07:27 09:57 12:28
CoCr Coronal Rank mode (%) [min–max] κAC2(95 % CI) = 0.78 (0.73–0.83) 3 (55%) [1–6] 2 (66%) [1–5] 1 (58%) [1–4] 1 (91%) [1–4] 1 (93%) [1–3] 1 (95%) [1–2]
Measured spatial resolution, mm 1.0 1.6 1.6 1.6 1.6 1.6
SNR 09.5 09.5 20.5 25.0 28.8 32.0
Acquisition time, min:s 00:37 00:37 04:00 06:02 08:04 10:06
Axial Rank mode (%) [min–max] κAC2(95 % CI) = 0.77 (0.75–0.80) 1 (36%) [1–5] 1 (48%) [1–5] 1 (65%) [1–4] 1 (79%) [1–3] 1 (79%) [1–3] 1 (80%) [1–3]
Measured spatial resolution, mm 1.0 >2.0 >2.0 >2.0 >2.0 >2.0
SNR 7.8 7.7 16.4 20.2 23.2 25.9
Acquisition time, min:s 00:51 00:53 05:48 08:43 11:39 14:35
*

P < 0.05 when compared with any of the other sequences.

P < 0.05 when compared with any of the VAT + SEMAC sequences.

P < 0.05 when compared with VAT + SEMAC 9, 12, or 15.

SNR indicates signal-to-noise ratio; TSE, turbo spin echo; VAT, view angle tilting; SEMAC, slice encoding for metal artifact correction; Ti, titanium; κAC2, chance-corrected agreement coefficient; CI, confidence interval; CoCr, cobalt-chromium.

FIGURE 2.

FIGURE 2.

The effect of pulse sequence without and with VAT and SEMAC on metal artifacts, image blurring, SNR, and acquisition time. Smaller panels show the resolution phantom in each case. A, MRI of the Ti implant using different pulse sequences shows minimal distortion of the acetabular cup with the TSE sequence (white arrows), whereas there is near-complete artifact reduction on VAT and SEMAC pulse sequences. Note the introduction of image blurring with VAT and VAT + SEMAC. B, MRI of the CoCr implant demonstrates the need for SEMAC with at least 9 encoding steps for adequate artifact reduction of the acetabular component.

For the Ti implant, the TSE pulse sequence was ranked lower than other pulse sequences in both coronal (P < 0.05 for TSE vs all other sequences) and axial (P < 0.05 for TSE vs VAT + SEMAC sequences) planes. The VAT pulse sequence was ranked lower than VAT + SEMAC sequences on the coronal (not statistically significant) and axial (P < 0.05 for VAT vs VAT + SEMAC sequences) images. All SEMAC pulse sequences were ranked equally high for artifact reduction. Therefore, the shortest VAT + SEMAC 6 pulse sequence was selected as optimal. Despite the lower ranking of the TSE sequence, the subjective difference in the size of the artifact between TSE and SEMAC sequences was minimal for this type of implant (Fig. 2A). Therefore, TSE with matching SNR to SEMAC through higher signal averages can be considered a viable option for MRI of low magnetic susceptibility implants owing to its short acquisition time and lower degrees of blurring.

For the CoCr implant, TSE, VAT, and VAT + SEMAC 6 were ranked lower than VAT + SEMAC 9 to 15 (all pairwise P < 0.05) (Table 1, Fig. 2B). Slice encoding for metal artifact correction with at least 9 encoding steps was needed to reach adequate image quality, beyond which the metal artifacts remained unchanged with no statistically significant differences (VAT + SEMAC 9 vs VAT + SEMAC 12: P > 0.99 for coronal and axial images, VAT + SEMAC 9 vs VAT + SEMAC 15: P > 0.99 for coronal and axial images), despite more SEMAC encoding steps and longer acquisition times.

With regard to the effect of receiver bandwidth on image quality, there was good to very good agreement between readers in image ranks (Table 2). The higher bandwidth of about 450 Hz/px resulted in the lowest degrees of metal artifact and lower blurring in SEMAC + VAT sequences (Fig. 3).

TABLE 2.

Effect of Receiver Bandwidth at 0.55 T on Metal Artifact Reduction, Measured Spatial Resolution, SNR, and Acquisition Time

Implant (Pulse Sequence) Type Imaging Plane Outcome Variable Pixel Bandwidth (Hz/Px)
200 250 300 350 400 450

Ti (VAT + SEMAC 6) Coronal Rank mode (%) [min–max] κAC2(95 % CI) = 0.83 (0.79–0.87) 1 (61%) [1–5] 1 (65%) [1–4] 1 (70%) [1–3] 1 (78%) [1–3] 1 (85%) [1–2] 1 (96%) [1–2]*
Measured spatial resolution, mm 1.6 1.6 1.6 1.6 1.6 1.6
SNR 31.9 28.1 24.5 23.7 21.4 20.5
Acquisition time, min:s 04:40 04:16 04:00 03:58 03:58 03:58
Axial Rank mode (%) [min–max] κAC2(95 % CI) = 0.94 (0.93–0.95) 1 (83%) [1–4] 1 (76%) [1–3] 1 (91%) [1–3] 1 (91%) [1–3] 1 (93%) [1–3] 1 (98%) [1–2]
Measured spatial resolution, mm >2.0 >2.0 >2.0 >2.0 >2.0 2.0
SNR 23.6 20.6 18.7 17.2 16.4 15.5
Acquisition time, min:s 05:44 05:15 04:57 04:49 04:49 04:49
Ti (TSE) Coronal Rank mode (%) [min–max] κAC2(95 % CI) = 0.68 (0.61–0.76) 3 (26%) [1–6] 3 (33%) [1–6] 2 (41%) [1–5] 2 (46%) [1–4] 1 (67%) [1–5] 1 (76%) [1–3]
Measured spatial resolution, mm 1.0 1.0 1.0 1.0 1.0 1.0
SNR 12.9 11.2 11.5 9.9 9.1 8.7
Acquisition time, min:s 00:37 00:35 00:37 00:35 00:35 00:35
Axial Rank mode (%) [min–max] κAC2(95 % CI) = 0.97 (0.96–0.98) 1 (78%) [1–4] 1 (90%) [1–3] 1 (91%) [1–1] 1 (93%) [1–2] 1 (94%) [1–2] 1 (97%) [1–3]
Measured spatial resolution, mm 1.0 1.0 1.0 1.0 1.0 1.0
SNR 10.6 9.4 9.2 8.2 7.7 7.3
Acquisition time, min:s 00:46 00:43 00:47 00:43 00:43 00:43
CoCr (VAT + SEMAC 9) Coronal Rank mode (%) [min–max] κAC2(95 % CI) = 0.92 (0.89–0.95) 1 (67%) [1–4] 1 (71%) [1–3] 1 (73%) [1–3] 1 (83%) [1–2] 1 (89%) [1–2] 1 (94%) [1–2]
Measured spatial resolution, mm 1.6 1.6 1.6 1.6 1.6 1.6
SNR 31.7 28.1 25.0 23.7 21.7 20.8
Acquisition time, min:s 07:03 06:27 06:02 05:59 05:59 05:59
Axial Rank mode (%) [min–max] κAC2(95 % CI) = 0.98 (0.97–0.99) 1 (88%) [1–3] 1 (89%) [1–3] 1 (91%) [1–3] 1 (96%) [1–3] 1 (99%) [1–2] 1 (100%) [1–1]*
Measured spatial resolution, mm >2.0 >2.0 >2.0 >2.0 >2.0 2.0
SNR 25.1 22.3 20.2 18.8 17.9 17.0
Acquisition time, min:s 10:07 09:14 08:43 08:29 08:29 08:29
*

P < 0.05 when compared with receiver bandwidths of 200, 250, and 300 Hz/px.

P < 0.05 when compared with receiver bandwidths of 200 and 250 Hz/px.

P < 0.05 when compared with any of the other sequences.

SNR indicates signal-to-noise ratio; Ti, titanium; VAT, view angle tilting; SEMAC, slice encoding for metal artifact correction; κAC2, chance-corrected agreement coefficient; CI, confidence interval; TSE, turbo spin echo; CoCr, cobalt-chromium.

FIGURE 3.

FIGURE 3.

The effect of receiver bandwidth on metal artifacts, image blurring, SNR, and acquisition time. Smaller panels show the resolution phantom in each case. A and B, MRI of the Ti implant. At higher receiver bandwidth of 450 Hz/px the artifacts (white arrows) are smaller with VAT + SEMAC 6 (A) and TSE (B). C, MRI of the CoCr implant. At higher receiver bandwidth of 450 Hz/px the artifacts (white arrows) are smaller with VAT + SEMAC 9. VAT-associated blurring decreases at higher receiver bandwidths.

The turbo factor in the tested range had no perceivable effect on the metal artifact or acquisition duration. However, to prevent any potential blurring that may occur with T2 weighting, turbo factors of 13 and 7 were selected for coronal and axial acquisitions, respectively (Table 3). Similarly, readers chose acceleration factors of 2 and 3 for coronal and axial images without affecting the metal artifacts, respectively (Table 4). A summary of the proposed pulse sequence protocol for MRI of the hip arthroplasty implants at 0.55 T is shown in Table 5.

TABLE 3.

Effect of turbo Factor at 0.55 T on Metal Artifact Reduction, Measured Spatial Resolution, SNR, and Acquisition Time

Implant (Pulse Sequence) Type Image Plane and Outcome Variables Turbo Factor

Ti (VAT + SEMAC 6) Coronal 8 10 13 15 23
Rank mode (%) [min–max] κAC2(95 % CI) = 0.97 (0.95–0.98) 1 (96%) [1–2] 1 (92%) [1–2] 1 (90%) [1–3] 1 (90%) [1–3] 1 (92%) [1–2]
Measured spatial resolution, mm 1.6 1.6 1.6 1.6 1.6
SNR 17.5 19.7 24.5 23.9 27.2
Acquisition time, min:s 04:14 04:01 04:00 03:44 03:42
Axial 7 9 12 18
Rank mode (%) [min–max] κAC2(95 % CI) = 0.99 (0.98–0.99) 1 (99%) [1–3] 1 (98%) [1–2] 1 (95%) [1–2] 1 (95%) [1–2]
Measured spatial resolution, mm >2.0 >2.0 >2.0 >2.0
SNR 18.7 19.6 21.4 23.9
Acquisition time, min:s 04:57 04:56 04:48 04:38
Ti (TSE) Coronal 8 10 13 15 18
Rank mode (%) [min–max] κAC2(95 % CI) = 0.99 (0.99–1.00) 1 (98%) [1–2] 1 (100%) [1–1] 1 (98%) [1–2] 1 (98%) [1–2] 1 (98%) [1–2]
Measured spatial resolution, mm 1.0 1.0 1.0 1.0 1.0
SNR 11.6 11.5 12.0 11.8 13.2
Acquisition time, min:s 02:43 02:10 01:47 01:31 01:34
Axial 4 7 9 12 18
Rank mode (%) [min–max] κAC2(95 % CI) = 0.99 (0.99–1.00) 1 (98%) [1–2] 1 (98%) [1–2] 1 (99%) [1–2] 1 (99%) [1–2] 1 (99%) [1–3]
Measured spatial resolution, mm 1.0 1.0 1.0 1.0 1.0
SNR 10.3 10.6 11.4 12.9 16.0
Acquisition time, min:s 3:47 02:16 02:03 01:56 02:06
CoCr (VAT + SEMAC 9) Coronal 8 10 13 15 23
Rank mode (%) [min–max] κAC2(95 % CI) = 1.00 (1.00–1.00) 1 (100%) [1–1] 1 (100%) [1–1] 1 (100%) [1–1] 1 (100%) [1–1] 1 (100%) [1–1]
Measured spatial resolution, mm 1.0/1.6 1.0/1.6 1.0/1.6 1.0/1.6 1.0/1.6
SNR 17.8 19.9 25.0 23.9 27.1
Acquisition time, min:s 06:23 06:03 06:02 05:38 05:37
Axial 7 10 12 18
Rank mode (%) [min–max] κAC2(95 % CI) = 1.00 (0.99–1.00) 1 (99%) [1–2] 1 (99%) [1–2] 1 (99%) [1–2] 1 (99%) [1–2]
Measured spatial resolution, mm >2.0 >2.0 >2.0 >2.0
SNR 20.2 21.3 22.7 24.7
Acquisition time, min:s 08:43 10:29 08:29 08:14

SNR indicates signal-to-noise ratio; Ti, titanium; VAT, view angle tilting; SEMAC, slice encoding for metal artifact correction; κAC2, chance-corrected agreement coefficient; CI, confidence interval; TSE, turbo spin echo; CoCr, cobalt-chromium.

TABLE 4.

Effect of Acceleration Technique at 0.55 T on Metal Artifact Reduction, Measured Spatial Resolution, SNR, and Acquisition Time

Acceleration Technique
Implant (Pulse Sequence) Type Imaging Plane Outcome Variables None GRAPPA 2 GRAPPA 3 GRAPPA 3 and 2 Averages

Ti (VAT + SEMAC 6) Coronal Rank mode (%) [min–max] κAC2(95 % CI) = 0.93 (0.89–0.96) 1 (94%) [1–3] 1 (94%) [1–2] 1 (98%) [1–2]
Measured spatial resolution, mm 1.6 1.6 1.6
SNR 39.9 30.0 24.5
Acquisition time, min:s 10:18 05:29 04:00
Axial Rank mode (%) [min–max] κAC2(95 % CI) = 0.99 (0.99–1.00) 1 (98%) [1–2] 1 (98%) [1–2] 1 (97%) [1–3] 1 (98%) [1–2]
Measured spatial resolution, mm >2.0 >2.0 >2.0 >2.0
SNR 23.7 18.7 13.7 17.5
Acquisition time, min:s 08:57 04:57 03:37 07:17
CoCr (VAT + SEMAC 9) Coronal Rank mode (%) [min–max] κAC2(95 % CI) = 0.99 (0.99–1.00) 1 (100%) [1–1] 1 (100%) [1–1] 1 (100%) [1–1]
Measured spatial resolution, mm
SNR 39.7 30.1 25.0
Acquisition time, min:s 15:24 08:15 06:02
Axial Rank mode (%) [min–max] κAC2(95 % CI) = 1.00 (0.99–1.00) 1 (99%) [1–2] 1 (99%) [1–2] 1 (99%) [1–2] 1 (99%) [1–2]
Measured spatial resolution, mm >2.0 >2.0 >2.0 >2.0
SNR 25.2 20.2 14.5 18.4
Acquisition time, min:s 15:46 08:43 06:23 12:50

SNR indicates signal-to-noise ratio; GRAPPA, generalized autocalibrating partial parallel acquisition; Ti, titanium; VAT, view angle tilting; SEMAC, slice encoding for metal artifact correction; κAC2, chance-corrected agreement coefficient; CI, confidence interval; TSE, turbo spin echo; CoCr, cobalt-chromium.

Comparison of 0.55 and 1.5 T Field Strengths

Signal-to-noise ratio efficiency and artifact size of the 0.55 and 1.5 T field strengths for different protocols and implant types are presented in Figure 4. Turbo spin echo images of the CoCr and Ti implants showed 57% (P = 0.001) and 56% (P = 0.008) smaller artifacts, respectively, at 0.55 than 1.5 T (Fig. 4A). Similar trends were observed for VAT acquisitions, with 48% (P = 0.003) and 55% (P = 0.008) smaller artifacts for the CoCr and Ti implants, respectively (Fig. 4B). Signal-to-noise ratio efficiencies of the 0.55 T acquisitions were 14% to 28% of the corresponding 1.5 T images for these pulse sequences.

FIGURE 4.

FIGURE 4.

Artifact size of the matching axial planes at 0.55 and 1.5 T for TSE (A), VAT (B), and proposed VAT + SEMAC (C) pulse sequences. SNR efficiency, defined as SNR divided by the square root of the acquisition time, is shown for each case.

Figures 4C and 5 compare the performance of our proposed 0.55 T protocols with those of the 1.5 T VAT + SEMAC and VAT + CS/SEMAC pulse sequences. For the Ti prosthesis, the optimized 0.55 T protocols resulted in overall 64% smaller artifacts (P = 0.007), 47% shorter acquisition time, and 23% lower SNR efficiency than 1.5 T VAT + SEMAC and 64% smaller artifacts (P = 0.007), 4% longer acquisition time, and 28% lower SNR efficiency than 1.5 T VAT + CS/SEMAC. Similarly, the CoCr implant demonstrated 53% smaller artifacts (P = 0.001), 20% shorter acquisition time, and 17% lower SNR efficiency than 1.5 T VAT + SEMAC and 45% smaller artifacts (P > 0.05), 57% longer acquisition time, and 25% lower SNR efficiency when compared with 1.5 T VAT + CS/SEMAC.

FIGURE 5.

FIGURE 5.

Comparison of 0.55 T optimized pulse protocols with our clinical 1.5 T VAT + SEMAC and VAT + CS/SEMAC pulse sequences for Ti (A) and CoCr (B) implants. Smaller panels show the resolution phantom in each case. Metal artifacts are significantly reduced primarily surrounding the femoral stem for both implant types. Areas of signal loss around the CoCr acetabular cup at 1.5 T (white arrows) are replaced by smaller areas of signal pileup at 0.55 T (hollow arrows).

The artifact reduction advantage of 0.55 T MRI was more pronounced at the longitudinal femoral stem than in the regions of the spherical acetabular cup and femoral head (Fig. 6). For both Ti and CoCr stems, the proposed protocols at 0.55 T resulted in significantly smaller artifacts surrounding the femoral stem (Ti: 120% smaller artifacts than 1.5 T VAT + SEMAC, P = 0.043, and 117% smaller than 1.5 T VAT + CS/SEMAC, P = 0.042; CoCr: 91% smaller artifacts than 1.5 T VAT + SEMAC, P = 0.012, and 90% smaller than 1.5 T VAT + CS/SEMAC, P = 0.012) (Fig. 4C).

FIGURE 6.

FIGURE 6.

Comparison of the 0.55 T and 1.5 T images using matched protocols at the implant head and femoral stem levels to visualize field strength effects on B0- and B1-related artifacts. B0-related areas of signal loss at the regions of the acetabular cup and femoral head (white line with arrows near the CoCr acetabular cup) are smaller at 0.55 T, while there are new areas of signal pileup at 0.55 T (hollow arrows). For both Ti and CoCr implants, the B1-related artifacts are substantially smaller surrounding the femoral stem.

In most cases, 0.55 T protocols also reduced metal artifacts in the region of the acetabular cup and femoral head (Ti: 8% smaller artifacts compared with 1.5 T VAT + SEMAC, P = 0.080, and 11% smaller artifacts compared with 1.5 T VAT + CS/SEMAC, P = 0.080; CoCr: 9% smaller artifacts compared with 1.5 T VAT + SEMAC, P = 0.018), owing to the reduction in the size of susceptibility related signal void near the femoral head (white arrows in Figs. 5 and 6). The exception to this observation was the high-susceptibility CoCr head, which had slightly larger, although statistically insignificant, artifacts at 0.55 T compared with the VAT + CS/SEMAC 1.5 T protocol (7% larger artifacts, P = 0.310). This increased artifact was due to a change in the appearance of the susceptibility artifacts and the introduction of new areas of signal pileup near the prosthesis head at 0.55 T (hollow arrows in Figs. 5 and 6).

DISCUSSION

Despite significant progress in the past 2 decades, artifact-free visualization of the bone and soft tissues contacting metallic orthopedic implants remains an unmet clinical need.2328 New-generation low-field MRI systems now include SEMAC, promising smaller metallic artifacts and better image quality than standard-of-care 1.5 T MRI. In this study, we used ceramic-on-polyethylene Ti and metal-on-metal CoCr total hip arthroplasty implant constructs to compare the degree of metal artifacts of magnetic resonance images obtained with modified 0.55 T prototype and clinical 1.5 T MRI systems, using the current spectrum of basic to advanced metal artifact reduction techniques. Our results indicate that the 0.55 T SEMAC MRI with 6 to 9 encoding steps substantially reduces metal artifacts within clinically viable sequence acquisition times of 6 minutes or less to 8 minutes. Higher spatial partitions, beyond 6 to 9 SEMAC encoding steps, failed to reduce metal artifacts further, which follows computational models.29 For lower-susceptibility hip arthroplasty implants, such as Ti, SEMAC may not be required if minimal artifacts are tolerated. In addition to shorter acquisition times, this approach eliminates VAT-associated blurring,30 which may obscure small abnormalities.

Our optimization derived slightly different pulse sequence protocols for Ti and CoCr implants. Although CoCr arthroplasty implants are now less commonly used, accurate MRI surveillance of patients with well-functioning CoCr hip arthroplasty constructs remains crucial. The CoCr protocol may also be used for MRI of unknown hip arthroplasty implants to err on the side of higher metal artifact reduction capacity. Applying high receiver bandwidths was associated with smaller artifacts and sharper images, which mitigates the blurring of the VAT + SEMAC sequences. Other parameters, such as the turbo factor and acceleration technique,31 did not affect the size of metal artifacts but can be adjusted to optimize the tradeoff between the acquisition time and SNR.

Metal-related artifacts were substantially smaller at 0.55 T than 1.5 T, with the added benefit of shorter scan times of 20% to 47% in the absence of CS acceleration. Our proposed 0.55 T protocols had 17% to 28% lower SNR efficiency than the clinical 1.5 T protocols. In addition to the inherently lower SNR of the 0.55 T field strength, the lower number of receiver elements in our experiments potentially contributed to the observed reduced SNR efficiency. Although this level of SNR may still be adequate for diagnostic purposes and is subject to future investigation, the lower SNR of the low-field systems could be improved by developing more efficient receiver coils and future applications of deep learning reconstruction algorithms based on neural networks or iterative denoising.

Lower B0 and B1-related artifacts play a role in the smaller metal artifacts at 0.55 T and manifest as smaller artifacts adjacent to the spherical acetabular and longitudinal femoral components, respectively (Fig. 6). Of interest is the change in the appearance of the B0-related artifacts at areas with high B0 distortion, such as the femoral head of the CoCr implant. A simplified model of metal-induced changes in the precession frequency (Δfm) as a function of distance from the metal surface (z) is shown in Figure 7. For similar radiofrequency (RF) pulse bandwidths (Δf(RF)), spins precessing at frequencies outside the RF pulse bandwidth (z < z1 for 0.55 T and z < z2 for 1.5 T) will not be excited, explaining the larger areas of signal loss due to off-resonance at 1.5 T. Through plane signal misregistration for each excited point with a frequency offset of Δfm equals Δz=ΔfmγGss,, with γ and Gss representing the gyromagnetic ratio and slice-selection gradient, respectively. Whereas the signal displacement of points A1 at 0.55 T (A1A1) and B1 at 1.5 T (B1B1) are equal, the signal from points z1 < z < z2 at 0.55 T (not excited at 1.5 T) will be distorted and displaced into the adjacent slices, resulting in signal pileups. This is of high clinical significance, as the improved visualization of the periprosthetic tissues may be associated with signal pileups that can be misinterpreted as a structural or tissue-based pathological signal abnormality. Because of the nonlinearity of the signal distortion, SEMAC correction of through-plane signal displacements, even at high encoding steps, cannot resolve the signal pileups, suggesting that more sophisticated artifact reduction techniques, such as hybrid multiacquisition with variable resonance image combination-SEMAC,32 may be needed.

FIGURE 7.

FIGURE 7.

Simplified model of metal-induced changes in the precession frequency (Δfm) as a function of distance from the metal surface (z) for 1.5 T (red) and 0.55 T (blue) field strengths. For similar RF pulse bandwidths (Δf(RF)), those spins outside the RF pulse bandwidth (z < z1 for 0.55 T and z < z2 for 1. 5 T) will not be excited, which explains the larger areas of signal loss due to off-resonance at 1.5 T. Through plane signal displacement for each excited point with a frequency offset of Δfm equals Δz=ΔfmγGss, with γ representing the gyromagnetic ratio and Gss representing the slice-selection gradient. Whereas the signal displacement for points A1 at 0.55 T (A1A1) and B1 at 1.5 T (B1B1) are equal, the signal from points z1 < z < z2 at 0.55 T (not excited at 1.5 T) will be distorted (eg, compressed) into the adjacent slices and appear as signal pileups.

The B1-related artifacts manifest as flame-shaped areas of shading and brightness along the femoral stem33 and are also smaller at 0.55 T (Fig. 6). In our experiments, the RF pulse profile was similar across the 0.55 and 1.5 T platforms, and the larger B1-related artifacts at 1.5 T can be attributed to the closer relationship between the implant length and RF pulse half wavelength at 1.5 T (~25 cm) than 0.55 T (~70 cm), which results in resonance coupling.3436

Our study has limitations. Our results may serve as a basis for MRI of hip arthroplasty implants in humans; however, the optimized protocols may need further refinement for clinical use. Second, we did not incorporate short tau inversion recovery pulse sequences in our study because of the water-based nature of our experimental setup. Although the proposed parameters can be used with short tau inversion recovery sequences, the image quality may be hampered by the inherently lower SNR. Lastly, we conducted our experiments in a gel phantom that emulated the electromagnetic properties of the human muscle. In vivo, the implant is embedded in cement or bone with different electromagnetic characteristics, which may affect the artifact profiles.

In conclusion, our in vitro study suggests that new-generation low-field MRI using SEMAC improves metal artifact reduction MRI of hip arthroplasty implants compared with current 1.5 T MRI standard of care. While the appearance of B0-related artifact changes, the reduction in B1-related artifacts also plays a major role in the overall benefit of 0.55 T.

Supplementary Material

Supporting Table 1
Supporting Figure 1

ACKNOWLEDGMENTS

The authors acknowledge the assistance of Siemens Healthcare in the modification of the MRI system for operation at 0.55 T under an existing research agreement between New York University and Siemens Healthcare GmbH.

Footnotes

Conflicts of interest and sources of funding: I.M.B. and M.B.K. are paid employees of Siemens Healthcare USA Inc. H.C. receive research support in form of software from Siemens under master research agreement. Others have no related disclosures.

Supplemental digital contents are available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.investigativeradiology.com).

REFERENCES

  • 1.Khodarahmi I, Fritz J. The value of 3 tesla field strength for musculoskeletal magnetic resonance imaging. Invest Radiol. 2021;56:749–763. [DOI] [PubMed] [Google Scholar]
  • 2.Del Grande F, Guggenberger R, Fritz J. Rapid musculoskeletal MRI in 2021: value and optimized use of widely accessible techniques. AJR Am J Roentgenol. 2021;216:704–717. [DOI] [PubMed] [Google Scholar]
  • 3.Marques JP, Simonis FFJ, Webb AG. Low-field MRI: an MR physics perspective. J Magn Reson Imaging. 2019;49:1528–1542. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Runge VM, Heverhagen JT. The clinical utility of magnetic resonance imaging according to field strength, specifically addressing the breadth of current state-of-the-art systems, which include 0.55 T, 1.5 T, 3 T, and 7 T. Invest Radiol. 2022;57:1–12. [DOI] [PubMed] [Google Scholar]
  • 5.Hori M, Hagiwara A, Goto M, et al. Low-field magnetic resonance imaging: its history and renaissance. Invest Radiol. 2021;56:669–679. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Fritz J, Guggenberger R, Del Grande F. Rapid musculoskeletal MRI in 2021: clinical application of advanced accelerated techniques. AJR Am J Roentgenol. 2021; 216:718–733. [DOI] [PubMed] [Google Scholar]
  • 7.Chandarana H, Bagga B, Huang C, et al. Diagnostic abdominal MR imaging on a prototype low-field 0.55 T scanner operating at two different gradient strengths. Abdom Radiol (NY). 2021;46:5772–5780. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.US Food and Drug Administration. 510(k) premarket notification. https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=K210611. Accessed August 25, 2021.
  • 9.Van Speybroeck CDE, O’Reilly T, Teeuwisse W, et al. Characterization of displacement forces and image artifacts in the presence of passive medical implants in low-field (<100 mT) permanent magnet-based MRI systems, and comparisons with clinical MRI systems. Phys Med. 2021;84:116–124. [DOI] [PubMed] [Google Scholar]
  • 10.Stecco A, Arioli R, Buemi F, et al. Overcoming metallic artefacts from orthopaedic wrist volar plating on a low-field MRI scanner. Radiol Med. 2019;124: 392–399. [DOI] [PubMed] [Google Scholar]
  • 11.Schröder FF, Verdonschot NJJ, ten Haken B, et al. Low-field magnetic resonance imaging offers potential for measuring tibial component migration. J Exp Orthop. 2018;5:4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Campbell-Washburn AE, Ramasawmy R, Restivo MC, et al. Opportunities in interventional and diagnostic imaging by using high-performance low-field-strength MRI. Radiology. 2019;293:384–393. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Heiss R, Nagel AM, Laun FB, et al. Low-field MRI—a new generation of break-through technology in clinical imaging. Invest Radiol. 2021. [DOI] [PubMed] [Google Scholar]
  • 14.American Society for Testing and Materials International. Standard Test Method for Measurement of Radio Frequency Induced Heating on or Near Passive Implants During Magnetic Resonance Imaging: ASTM Standard F218211a, 2002 (2011). West Conshohocken, PA: ASTM International; 2011. [Google Scholar]
  • 15.Fellner C, Müller W, Georgi J, et al. A high-resolution phantom for MRI. Magn Reson Imaging. 2001;19:899–904. [DOI] [PubMed] [Google Scholar]
  • 16.Cho ZH, Kim DJ, Kim YK. Total inhomogeneity correction including chemical shifts and susceptibility by view angle tilting. Med Phys. 1988;15:7–11. [DOI] [PubMed] [Google Scholar]
  • 17.Lu W, Pauly KB, Gold GE, et al. SEMAC: slice encoding for metal artifact correction in MRI. Magn Reson Med. 2009;62:66–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Sutter R, Ulbrich EJ, Jellus V, et al. Reduction of metal artifacts in patients with total hip arthroplasty with slice-encoding metal artifact correction and view-angle tilting MR imaging. Radiology. 2012;265:204–214. [DOI] [PubMed] [Google Scholar]
  • 19.Fritz J, Ahlawat S, Demehri S, et al. Compressed sensing SEMAC: 8-fold accelerated high resolution metal artifact reduction MRI of cobalt-chromium knee arthroplasty implants. Invest Radiol. 2016;51:666–676. [DOI] [PubMed] [Google Scholar]
  • 20.Dietrich O, Raya JG, Reeder SB, et al. Measurement of signal-to-noise ratios in MR images: influence of multichannel coils, parallel imaging, and reconstruction filters. J Magn Reson Imaging. 2007;26:375–385. [DOI] [PubMed] [Google Scholar]
  • 21.Gwet KL. Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement Among Raters. Gaithersburg, MD, USA: Advanced Analytics, LLC; 2014. [Google Scholar]
  • 22.Altman DG. Practical Statistics for Medical Research. London: Chapman & Hall; 2020. [Google Scholar]
  • 23.Jungmann PM, Agten CA, Pfirrmann CW, et al. Advances in MRI around metal. J Magn Reson Imaging. 2017;46:972–991. [DOI] [PubMed] [Google Scholar]
  • 24.Khodarahmi I, Fishman EK, Fritz J. Dedicated CT and MRI techniques for the evaluation of the postoperative knee. Semin Musculoskelet Radiol. 2018;22: 444–456. [DOI] [PubMed] [Google Scholar]
  • 25.Fritz J, Lurie B, Miller TT, et al. MR imaging of hip arthroplasty implants. Radiographics. 2014;34:E106–E132. [DOI] [PubMed] [Google Scholar]
  • 26.Khodarahmi I, Nittka M, Fritz J. Leaps in technology: advanced MR imaging after total hip arthroplasty. Semin Musculoskelet Radiol. 2017;21:604–615. [DOI] [PubMed] [Google Scholar]
  • 27.Koff MF, Burge AJ, Koch KM, et al. Imaging near orthopedic hardware. J Magn Reson Imaging. 2017;46:24–39. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Khodarahmi I, Fritz J. Advanced MR imaging after total hip arthroplasty: the clinical impact Semin Musculoskelet Radiol. 2017;21:616–629. [DOI] [PubMed] [Google Scholar]
  • 29.Keskin K, Hargreaves B, Nayak K. Realistic simulation of MRI metal artifact and field strength dependence. In: Proceedings of the 30th Annual Meeting of ISMRM. 2021. Abstract 3539. [Google Scholar]
  • 30.Butts K, Pauly JM, Gold GE. Reduction of blurring in view angle tilting MRI. Magn Reson Med. 2005;53:418–424. [DOI] [PubMed] [Google Scholar]
  • 31.Kumar NM, de Cesar Netto C, Schon LC, et al. Metal artifact reduction magnetic resonance imaging around arthroplasty implants: the negative effect of long echo trains on the implant-related artifact. Invest Radiol. 2017;52: 310–316. [DOI] [PubMed] [Google Scholar]
  • 32.Koch KM, Brau AC, Chen W, et al. Imaging near metal with a MAVRIC-SEMAC hybrid. Magn Reson Med. 2011;65:71–82. [DOI] [PubMed] [Google Scholar]
  • 33.Khodarahmi I, Kirsch J, Chang G, et al. Metal artifacts of hip arthroplasty implants at 1.5-T and 3.0-T: a closer look into the B1 effects. Skeletal Radiol. 2021;50:1007–1015. [DOI] [PubMed] [Google Scholar]
  • 34.Graf H, Lauer UA, Berger A, et al. RF artifacts caused by metallic implants or instruments which get more prominent at 3 T: an in vitro study. Magn Reson Imaging. 2005;23:493–499. [DOI] [PubMed] [Google Scholar]
  • 35.Khodarahmi I, Bonham LW, Weiss CR, et al. Needle heating during interventional magnetic resonance imaging at 1.5- and 3.0-T field strengths. Invest Radiol. 2020; 55:396–404. [DOI] [PubMed] [Google Scholar]
  • 36.Khodarahmi I, Rajan S, Sterling R, et al. Heating of hip arthroplasty implants during metal artifact reduction MRI at 1.5- and 3.0-T field strengths. Invest Radiol. 2021;56:232–243. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supporting Table 1
Supporting Figure 1

RESOURCES